“Acoustical Vision” in Patients with Visual Deficit: A Neural Network Study

Abstract

Perception of external events relies on integration of different sensory information (multisensory integration). Multisensory integration is maximally beneficial when information from one sensory modality is impaired. Bimodal audiovisual stimulation has been shown to improve perception of visual events in patients with visual deficit due to a lesion in the primary visual cortex (hemianopia) or in higher-level fronto-parietal visual areas (neglect). Improvement was higher when auditory and visual stimuli were spatially coincident or at moderate spatial disparity, while no significant improvement was found at larger disparity. A neural network model, that includes the interaction between cortical areas and subcortical structures (the Superior Colliculus), is presented to investigate these phenomena in both kinds of patients. The model untangles the specific contribution of the circuits - spared by the lesion - that drive the emergence of visual perception under audiovisual stimulation, and interprets the dependence of the effect on audiovisual spatial disparity. The model extends our knowledge on the neural mechanisms subserving brain multisensory capabilities and how to take advantage of them for sensory loss compensation.