Abstract
A method for decoding the subjective contents of perceptual systems in the human brain would have broad practical utility for communication and as a brain-machine interface. Previous approaches to this problem in vision have used linear classifiers to solve specific problems, but these approaches were not general enough to solve complex problems such as reconstructing subjective perceptual states. We have developed a new approach to these problems based on quantitative encoding models that explicitly describe how visual stimuli are (nonlinearly) transformed into brain activity. We then invert these encoding models in order to decode activity evoked by novel images or movies, providing reconstructions with unprecedented fidelity. Here we briefly review these results and the potential uses of perceptual decoding devices.
Chapter PDF
References
Carlson, T.A., Schrater, P., He, S.: Patterns of activity in the categorical representations of objects. J. Cog. Neurosci. 15, 704–717 (2003)
Cox, D.D., Savoy, R.L.: Functional magnetic resonance imaging (fMRI) ’brain reading’: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270 (2003)
Haynes, J.D., Rees, G.: Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 8, 686–691 (2005)
Haynes, J.D., Rees, G.: Decoding mental states from brain activity in humans. Nat. Neurosci. Rev. 7, 523–534 (2006)
Hanson, S.J., Matsuka, T., Haxby, J.V.: Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area? Neuroimage 23, 156–166 (2001)
Kamitani, Y., Tong, F.: Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8, 679–685 (2005)
Kay, K.N., Naselaris, T., Prenger, R.J., Gallant, J.L.: Identifying natural images from human brain activity. Nature 452, 352–355 (2008)
Thirion, B., Duchesnay, E., Hubbard, E., Dubois, J., Poline, J.B., Lebihan, D., Dehaene, S.: Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage 33, 1104–1116 (2006)
Van Essen, D.C., Gallant, J.L.: Neural mechanisms of form and motion processing in the primate visual system. Neuron 13, 1–10 (1994)
Wu, M.C.-K., David, S.V., Gallant, J.L.: Complete functional characterization of sensory neurons by system identification. Ann. Rev. Neurosci. 29, 477–505 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gallant, J. et al. (2009). Bayesian Reconstruction of Perceptual Experiences from Human Brain Activity. In: Schmorrow, D.D., Estabrooke, I.V., Grootjen, M. (eds) Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience. FAC 2009. Lecture Notes in Computer Science(), vol 5638. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02812-0_46
Download citation
DOI: https://doi.org/10.1007/978-3-642-02812-0_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02811-3
Online ISBN: 978-3-642-02812-0
eBook Packages: Computer ScienceComputer Science (R0)