You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution

Article type: Research Article

Authors: Abboud, Sami | Hanassy, Shlomi | Levy-Tzedek, Shelly; | Maidenbaum, Shachar | Amedi, Amir; ;

Affiliations: Department of Medical Neurobiology, The Institute for Medical Research Israel-Canada, Faculty of Medicine, The Hebrew University of Jerusalem, Jerusalem, Israel | The Edmond and Lily Safra Center for Brain Sciences (ELSC), The Hebrew University of Jerusalem, Jerusalem, Israel | The Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, Israel

Note: [] Equal contribution.

Note: [] Equal contribution.

Note: [] Corresponding author: Amir Amedi, Ph.D., Department of Medical Neurobiology, The Institute for Medical Research Israel-Canada, Faculty of Medicine, The Hebrew University of Jerusalem, Jerusalem 91220, Israel. Tel.: +972 2 675 7259; Fax: +972 2 675 8602; Lab website: http://brain.huji.ac.il/; Email: [email protected]

Keywords: Sensory substitution devices, SSD, blindness, rehabilitation, color, pleasant sounds

DOI: 10.3233/RNN-130338

Journal: Restorative Neurology and Neuroscience, vol. 32, no. 2, pp. 247-257, 2014

Published: 2014
Get PDF open access
Supplementary Materials:

Supplementary Material

Abstract

Purpose: Sensory-substitution devices (SSDs) provide auditory or tactile representations of visual information. These devices often generate unpleasant sensations and mostly lack color information. We present here a novel SSD aimed at addressing these issues. Methods: We developed the EyeMusic, a novel visual-to-auditory SSD for the blind, providing both shape and color information. Our design uses musical notes on a pentatonic scale generated by natural instruments to convey the visual information in a pleasant manner. A short behavioral protocol was utilized to train the blind to extract shape and color information, and test their acquired abilities. Finally, we conducted a survey and a comparison task to assess the pleasantness of the generated auditory stimuli. Results: We show that basic shape and color information can be decoded from the generated auditory stimuli. High performance levels were achieved by all participants following as little as 2–3 hours of training. Furthermore, we show that users indeed found the stimuli pleasant and potentially tolerable for prolonged use. Conclusions: The novel EyeMusic algorithm provides an intuitive and relatively pleasant way for the blind to extract shape and color information. We suggest that this might help facilitating visual rehabilitation because of the added functionality and enhanced pleasantness.

Show more