Abstract
The goal of the See ColOr project is to achieve a non-invasive mobility aid for blind users that will use the auditory pathway to represent in real-time frontal image scenes. More particularly, we have developed a prototype which transforms HSL coloured pixels into spatialized classical instrument sounds lasting for 300 ms. Hue is sonified by the timbre of a musical instrument, saturation is one of four possible notes, and luminosity is represented by bass when luminosity is rather dark and singing voice when it is relatively bright. Our first experiments are devoted to static images on the computer screen. Six participants with their eyes covered by a dark tissue were trained to associate colours with musical instruments and then asked to determine on several pictures, objects with specific shapes and colours. In order to simplify the protocol of experiments, we used a tactile tablet, which took the place of the camera. Overall, experiment participants found that colour was helpful for the interpretation of image scenes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Algazi, V.R., Duda, R.O., Thompson, D.P.: Avendano. The CIPIC HRTF Database. In: IEEE Proc. Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA’01), Mohonk Mountain House, New Paltz, NY (2001)
Bologna, G., Vinckenbosch, M.: Eye Tracking in Coloured Image Scenes Represented by Ambisonic Fields of Musical Instrument Sounds. In: Mira, J., Álvarez, J.R. (eds.) IWINAC 2005. LNCS, vol. 3561, pp. 327–337. Springer, Heidelberg (2005)
Bamford, J.S.: An Analysis of Ambisonic Sound Systems of First and Second Order. Master Thesis, Waterloo, Ontario, Canada (1995)
Capelle, C., Trullemans, C., Arno, P., Veraart, C.: A Real Time Experimental Prototype for Enhancement of Vision Rehabilitation Using Auditory Substitution. IEEE T. Bio-Med Eng. 45, 1279–1293 (1998)
Cronly-Dillon, J., Persaud, K., Gregory, R.P.F.: The Perception of Visual Images Encoded in Musical Form: a Study in Cross-Modality Information. Proc. Biological Sciences 266, 2427–2433 (1999)
Daniel, J.: Acoustic Field Representation, Application to the Transmission and the Reproduction of Complex Sound Environments in a Multimedia Context. PhD thesis, University of Paris 6 (2000)
Gerzon, M.A.: Design of Ambisonic Decoders for Multispeaker Surround Sound. Journal of the Audio Engineering Society 25, 1064 (1977)
Gonzalez-Mora, J.L., Rodriguez-Hernandez, A., Rodriguez-Ramos, L.F., Dfaz-Saco, L., Sosa, N.: Development of a New Space Perception System for Blind People, Based on the Creation of a Virtual Acoustic Space. In: Mira, J. (ed.) IWANN 1999. LNCS, vol. 1607, pp. 321–330. Springer, Heidelberg (1999)
Kay, L.: A Sonar Aid to Enhance Spatial Perception of the Blind: Engineering Design and Evaluation. The Radio and Electronic Engineer 44, 605–627 (1974)
Lakatos, S.: Recognition of Complex Auditory-Spatial Patterns. Perception 22, 363–374 (1993)
Malham, D.G., Myatt, A.: 3-D Sound Spatialisation using Ambisonic Techniques. Computer Music Journal 19(4), 58–70 (1995)
Meijer, P.B.L.: An Experimental System for Auditory Image Representations. IEEE Transactions on Biomedical Engineering 39(2), 112–121 (1992)
Ruff, R.M., Perret, E.: Auditory Spatial Pattern Perception Aided by Visual Choices. Psychological Research 38, 369–377 (1976)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Bologna, G., Deville, B., Pun, T., Vinckenbosch, M. (2007). Identifying Major Components of Pictures by Audio Encoding of Colours. In: Mira, J., Álvarez, J.R. (eds) Nature Inspired Problem-Solving Methods in Knowledge Engineering. IWINAC 2007. Lecture Notes in Computer Science, vol 4528. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73055-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-73055-2_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73054-5
Online ISBN: 978-3-540-73055-2
eBook Packages: Computer ScienceComputer Science (R0)