Tactile, Visual, and Bimodal P300s: Could Bimodal P300s Boost BCI Performance?

Brouwer, Anne-Marie; van Erp, Jan B. F.; Aloise, Fabio; Cincotti, Febo

doi:https://doi.org/10.3814/2010/967027

SRX Neuroscience

On this page

Abstract References Copyright Related Articles

Research Article | Open Access

Volume 2010 | Article ID 967027 | https://doi.org/10.3814/2010/967027

Tactile, Visual, and Bimodal P300s: Could Bimodal P300s Boost BCI Performance?

Anne-Marie Brouwer,¹Jan B. F. van Erp,¹Fabio Aloise,^2,3and Febo Cincotti²

Received23 Feb 2010

Revised06 May 2010

Accepted19 May 2010

Published01 Jul 2010

Abstract

The P300 is a positive peak in EEG occurring after presentation of a target stimulus. For brain-computer interfaces (BCIs), eliciting P300s by tactile stimuli would have specific advantages; the display can be hidden under clothes and keeps the user's gaze free. In addition, robust classification is especially important for BCIs. This motivated us to investigate P300s in response to tactile and visual stimuli unimodally and bimodally. Tactile stimuli were delivered by tactors around the participant's waist. Visual stimuli were flashed circles on a monitor, schematically representing the tactors. Participants attended to the vibrations and/or flashes of a ‘‘target’’ presented in a stream of standards. The P300 amplitude for the different modalities was comparable in size and depended on electrode location. Classification accuracy was highest in the bimodal condition. We conclude that bimodal stimuli could enhance classification results within a BCI context compared to unimodal presentations.

References

N. Birbaumer, “Breaking the silence: brain-computer interfaces (BCI) for communication and motor control,” Psychophysiology, vol. 43, no. 6, pp. 517–532, 2006.
View at: Publisher Site | Google Scholar
T. M. Vaughan, D. J. McFarland, G. Schalk, W. A. Sarnacki, D. J. Krusienski, E. W. Sellers, and J. R. Wolpaw, “The wadsworth BCI research and development program: at home with BCI,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 14, no. 2, pp. 229–233, 2006.
View at: Publisher Site | Google Scholar
J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, no. 6, pp. 767–791, 2002.
View at: Publisher Site | Google Scholar
C. Guan, M. Thulasidas, and J. Wu, “High performance P300 speller for brain-computer interface,” in Proceedings of the IEEE International Workshop on Biomedical Circuits and Systems, December 2004, S3.5.INV-13-16.
View at: Google Scholar
L. A. Farwell and E. Donchin, “Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials,” Electroencephalography and Clinical Neurophysiology, vol. 70, no. 6, pp. 510–523, 1988.
View at: Google Scholar
M. Kaper, P. Meinicke, U. Grossekathoefer, T. Lingner, and H. Ritter, “BCI competition 2003—data set IIb: support vector machines for the P300 speller paradigm,” IEEE Transactions on Biomedical Engineering, vol. 51, no. 6, pp. 1073–1076, 2004.
View at: Publisher Site | Google Scholar
E. W. Sellers, D. J. Krusienski, D. J. McFarland, T. M. Vaughan, and J. R. Wolpaw, “A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance,” Biological Psychology, vol. 73, no. 3, pp. 242–252, 2006.
View at: Publisher Site | Google Scholar
H. Serby, E. Yom-Tov, and G. F. Inbar, “An improved P300-based brain-computer interface,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 13, no. 1, pp. 89–98, 2005.
View at: Publisher Site | Google Scholar
B. Rebsamen, E. Burdet, and E. Burdet, “A brain-controlled wheelchair based on P300 and path guidance,” in Proceedings of the 1st IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 1101–1106, February 2006.
View at: Publisher Site | Google Scholar
J. D. Bayliss, “Use of the evoked potential P3 component for control in a virtual apartment,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, no. 2, pp. 113–116, 2003.
View at: Publisher Site | Google Scholar
J. D. Bayliss and D. H. Ballard, “Single trial P3 epoch recognition in a virtual environment,” Neurocomputing, vol. 32-33, pp. 637–642, 2000.
View at: Publisher Site | Google Scholar
M. D. Comerchero and J. Polich, “P3a and P3b from typical auditory and visual stimuli,” Clinical Neurophysiology, vol. 110, no. 1, pp. 24–30, 1999.
View at: Publisher Site | Google Scholar
B. H. Jansen, A. Allam, P. Kota, K. Lachance, A. Osho, and K. Sundaresan, “An exploratory study of factors affecting single trial P300 detection,” IEEE Transactions on Biomedical Engineering, vol. 51, no. 6, pp. 975–978, 2004.
View at: Publisher Site | Google Scholar
J. Polich and M. R. D. Heine, “P300 topography and modality effects from a single-stimulus paradigm,” Psychophysiology, vol. 33, no. 6, pp. 747–752, 1996.
View at: Publisher Site | Google Scholar
R. J. Croft, C. J. Gonsalvez, C. Gabriel, and R. J. Barry, “Target-to-target interval versus probability effects on P300 in one- and two-tone tasks,” Psychophysiology, vol. 40, no. 3, pp. 322–328, 2003.
View at: Publisher Site | Google Scholar
J. Polich, T. Brock, and M. W. Geisler, “P300 from auditory and somatosensory stimuli: probability and inter-stimulus interval,” International Journal of Psychophysiology, vol. 11, no. 2, pp. 219–223, 1991.
View at: Publisher Site | Google Scholar
S. Lang and B. Kotchoubey, “Brain responses to number sequences with and without active task requirement,” Clinical Neurophysiology, vol. 113, no. 11, pp. 1734–1741, 2002.
View at: Publisher Site | Google Scholar
J. P. Rosenfeld, J. R. Biroschak, and J. J. Furedy, “P300-based detection of concealed autobiographical versus incidentally acquired information in target and non-target paradigms,” International Journal of Psychophysiology, vol. 60, no. 3, pp. 251–259, 2006.
View at: Publisher Site | Google Scholar
J. B. F. Van Erp and P. Padmos, “Image parameters for driving with indirect viewing systems,” Ergonomics, vol. 46, no. 15, pp. 1471–1499, 2003.
View at: Publisher Site | Google Scholar
J. B. F. Van Erp, H. A. H. C. van Veen, C. Jansen, and T. Dobbins, “Waypoint navigation with a vibrotactile waist belt,” Transactions on Applied Perception, vol. 2, no. 2, pp. 106–117, 2005.
View at: Google Scholar
J. B. F. Van Erp, L. Eriksson, B. Levin, O. Carlander, J. A. Veltman, and W. K. Vos, “Tactile cueing effects on performance in simulated aerial combat with high acceleration,” Aviation Space and Environmental Medicine, vol. 78, no. 12, pp. 1128–1134, 2007.
View at: Publisher Site | Google Scholar
J. B. F. van Erp and H. A. H. C. van Veen, “Touch down: the effect of artificial touch cues on orientation in microgravity,” Neuroscience Letters, vol. 404, no. 1-2, pp. 78–82, 2006.
View at: Publisher Site | Google Scholar
J. B. F. Van Erp, “Presenting directions with a vibrotactile torso display,” Ergonomics, vol. 48, no. 3, pp. 302–313, 2005.
View at: Publisher Site | Google Scholar
A.-M. Brouwer and J. B. F. van Erp, “A tactile P300 brain-computer interface,” Frontiers in Neuroscience, vol. 4, no. 19, pp. 1–12, 2010.
View at: Google Scholar
R. Romero and J. Polich, “P3(00) habituation from auditory and visual stimuli,” Physiology and Behavior, vol. 59, no. 3, pp. 517–522, 1996.
View at: Publisher Site | Google Scholar
J. Y. Bennington and J. Polich, “Comparison of P300 from passive and active tasks for auditory and visual stimuli,” International Journal of Psychophysiology, vol. 34, no. 2, pp. 171–177, 1999.
View at: Publisher Site | Google Scholar
E. Schröger and A. Widmann, “Speeded responses to audiovisual signal changes result from bimodal integration,” Psychophysiology, vol. 35, no. 6, pp. 755–759, 1998.
View at: Publisher Site | Google Scholar
R. S. Nickerson, “Intersensory facilitation of reaction time: energy summation or preparation enhancement?” Psychological Review, vol. 80, no. 6, pp. 489–509, 1973.
View at: Publisher Site | Google Scholar
B. E. Stein and M. A. Meredith, The Merging of the Senses, The MIT Press, Cambridge, Mass, USA, 1993.
T. R. Stanford and B. E. Stein, “Superadditivity in multisensory integration: putting the computation in context,” NeuroReport, vol. 18, no. 8, pp. 787–792, 2007.
View at: Publisher Site | Google Scholar
H. Jasper, “Report of the committee on methods of clinical examination in electroencephalography,” Electroencephalography and Clinical Neurophysiology, vol. 10, no. 2, pp. 370–375, 1958.
View at: Google Scholar
R. P. W. Duin, P. Juszczak, P. Paclik et al., PRTools4.1: A Matlab Toolbox for Pattern Recognition, Delft University of Technology, 2007.
D. Ravden and J. Polich, “On P300 measurement stability: habituation, intra-trial block variation, and ultradian rhythms,” Biological Psychology, vol. 51, no. 1, pp. 59–76, 1999.
View at: Publisher Site | Google Scholar
N. Srinivasan, “Cognitive neuroscience of creativity: EEG based approaches,” Methods, vol. 42, no. 1, pp. 109–116, 2007.
View at: Publisher Site | Google Scholar
C. Bledowski, D. Prvulovic, K. Hoechastetter, M. Scherg, M. Wibral, R. Goebel, and D. E. J. Linden, “Localizing P300 generators in visual target and distractor processing: a combined event-related potential and functional magnetic resonance imaging study,” Journal of Neuroscience, vol. 24, no. 42, pp. 9353–9360, 2004.
View at: Publisher Site | Google Scholar
A. D. Gerson, L. C. Parra, and P. Sajda, “Cortically coupled computer vision for rapid image search,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 14, no. 2, pp. 174–179, 2006.
View at: Publisher Site | Google Scholar
F. Aloise, I. Lasorsa, F. Schettini et al., “Multimodal stimulation for a P300-based BCI,” International Journal of Bioelectromagnetism, vol. 9, pp. 128–130, 2007.
View at: Google Scholar
J. Polich, “Updating P300: an integrative theory of P3a and P3b,” Clinical Neurophysiology, vol. 118, no. 10, pp. 2128–2148, 2007.
View at: Publisher Site | Google Scholar
M. S. Treder and B. Blankertz, “(C)overt attention and visual speller design in an ERP-based brain-computer interface,” Behavioral and Brain Functions, vol. 6, article 28, 2010.
View at: Publisher Site | Google Scholar
J. J. Foxe and C. E. Schroeder, “The case for feedforward multisensory convergence during early cortical processing,” NeuroReport, vol. 16, no. 5, pp. 419–423, 2005.
View at: Publisher Site | Google Scholar
J. J. Foxe, I. A. Morocz, M. M. Murray, B. A. Higgins, D. C. Javitt, and C. E. Schroeder, “Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping,” Cognitive Brain Research, vol. 10, no. 1-2, pp. 77–83, 2000.
View at: Publisher Site | Google Scholar
S. Molholm, W. Ritter, M. M. Murray, D. C. Javitt, C. E. Schroeder, and J. J. Foxe, “Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study,” Cognitive Brain Research, vol. 14, no. 1, pp. 115–128, 2002.
View at: Publisher Site | Google Scholar
D. Talsma, T. J. Doty, and M. G. Woldorff, “Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration?” Cerebral Cortex, vol. 17, no. 3, pp. 679–690, 2007.
View at: Publisher Site | Google Scholar
W. A. Teder-Sälejärvi, T. F. Münte, F.-J. Sperlich, and S. A. Hillyard, “Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study,” Cognitive Brain Research, vol. 8, no. 3, pp. 327–343, 1999.
View at: Publisher Site | Google Scholar
Y. Nakajima and N. Imamura, “Relationships between attention effects and intensity effects on the cognitive N140 and P300 components of somatosensory ERPs,” Clinical Neurophysiology, vol. 111, no. 10, pp. 1711–1718, 2000.
View at: Publisher Site | Google Scholar
T. G. Philippi, J. B. F. van Erp, and P. J. Werkhoven, “Multisensory temporal numerosity judgment,” Brain Research, vol. 1242, pp. 116–125, 2008.
View at: Publisher Site | Google Scholar
B. Kotchoubey and S. Lang, “Event-related potentials in an auditory semantic oddball task in humans,” Neuroscience Letters, vol. 310, no. 2-3, pp. 93–96, 2001.
View at: Publisher Site | Google Scholar
F. Piccione, F. Giorgi, and F. Giorgi, “P300-based brain computer interface: reliability and performance in healthy and paralysed participants,” Clinical Neurophysiology, vol. 117, no. 3, pp. 531–537, 2006.
View at: Publisher Site | Google Scholar
E. W. Sellers and E. Donchin, “A P300-based brain-computer interface: initial tests by ALS patients,” Clinical Neurophysiology, vol. 117, no. 3, pp. 538–548, 2006.
View at: Publisher Site | Google Scholar
K. M. Spencer and J. Polich, “Poststimulus EEG spectral analysis and P300: attention, task, and probability,” Psychophysiology, vol. 36, no. 2, pp. 220–232, 1999.
View at: Publisher Site | Google Scholar
J. Yordanova, V. Kolev, and J. Polich, “P300 and alpha event-related desynchronization (ERD),” Psychophysiology, vol. 38, no. 1, pp. 143–152, 2001.
View at: Publisher Site | Google Scholar
T.-P. Jung, S. Makeig, M. Westerfield, J. Townsend, E. Courchesne, and T. J. Sejnowski, “Analysis and visualization of single-trial event-related potentials,” Human Brain Mapping, vol. 14, no. 3, pp. 166–185, 2001.
View at: Publisher Site | Google Scholar
B. Z. Allison and J. A. Pineda, “Effects of SOA and flash pattern manipulations on ERPs, performance, and preference: implications for a BCI system,” International Journal of Psychophysiology, vol. 59, no. 2, pp. 127–140, 2006.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2010 Anne-Marie Brouwer et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

268

Downloads

0

Citations