Skip to main content

Advertisement

SpringerLink
Log in

Optimal nonlinear cue integration for sound localization

  • Published:
Journal of Computational Neuroscience Aims and scope Submit manuscript

Abstract

Integration of multiple sensory cues can improve performance in detection and estimation tasks. There is an open theoretical question of the conditions under which linear or nonlinear cue combination is Bayes-optimal. We demonstrate that a neural population decoded by a population vector requires nonlinear cue combination to approximate Bayesian inference. Specifically, if cues are conditionally independent, multiplicative cue combination is optimal for the population vector. The model was tested on neural and behavioral responses in the barn owl’s sound localization system where space-specific neurons owe their selectivity to multiplicative tuning to sound localization cues interaural phase (IPD) and level (ILD) differences. We found that IPD and ILD cues are approximately conditionally independent. As a result, the multiplicative combination selectivity to IPD and ILD of midbrain space-specific neurons permits a population vector to perform Bayesian cue combination. We further show that this model describes the owl’s localization behavior in azimuth and elevation. This work provides theoretical justification and experimental evidence supporting the optimality of nonlinear cue combination.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257–262.

    Article  CAS  PubMed  Google Scholar 

  • Albeck, Y., & Konishi, M. (1995). Responses of neurons in the auditory pathway of the barn owl to partially correlated binaural signals. Journal of Neurophysiology, 74, 1689–1700.

    CAS  PubMed  Google Scholar 

  • Angelaki, D. E., Gu, Y., & DeAngelis, G. C. (2009). Multisensory integration: psychophysics, neurophysiology, and computation. Current Opinion in Neurobiology, 19, 452–458.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bala, A. D. S., Spitzer, M. W., & Takahashi, T. T. (2007). Auditory spatial acuity approximates the resolving power of space-specific neurons. PLoS ONE, 2, e675.

    Article  PubMed  PubMed Central  Google Scholar 

  • Battaglia, P. W., Jacobs, R. A., & Aslin, R. N. (2003). Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A, 20, 1391–1397.

    Article  Google Scholar 

  • Beck, J., Ma, W. J., Latham, P. E., & Pouget, A. (2007). Probabilistic population codes and the exponential family of distributions. Progress in Brain Research, 165, 509–519.

    Article  CAS  PubMed  Google Scholar 

  • Beck, J. M., Latham, P. E., & Pouget, A. (2011). Marginalization in neural circuits with divisive normalization. Journal of Neuroscience, 31, 15310–15319.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Brainard, M. S., & Knudsen, E. I. (1993). Experience-dependent plasticity in the inferior colliculus: a site for visual calibration of the neural representation of auditory space in the barn owl. Journal of Neuroscience, 13, 4589–4608.

    CAS  PubMed  Google Scholar 

  • Brainard, M. S., Knudsen, E. I., & Esterly, S. D. (1992). Neural derivation of sound source location: resolution of spatial ambiguities in binaural cues. Journal of the Acoustical Society of America, 91, 1015–1027.

    Article  CAS  PubMed  Google Scholar 

  • Bregman AS (1994) Auditory scene analysis: The perceptual organization of sound. Cambridge, MA.: MIT press.

  • Carr, C. E., & Konishi, M. (1990). A circuit for detection of interaural time differences in the brain stem of the barn owl. Journal of Neuroscience, 10, 3227–3246.

    CAS  PubMed  Google Scholar 

  • Cazettes, F., Fischer, B. J., & Pena, J. L. (2014). Spatial cue reliability drives frequency tuning in the barn Owl’s midbrain. eLife, 3, e04854.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cazettes, F., Fischer, B. J., & Peña, J. L. (2016). Cue reliability represented in the shape of tuning curves in the owl’s sound localization system. Journal of Neuroscience, 36, 2101–2110.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Christianson, G. B., & Peña, J. L. (2006). Noise reduction of coincidence detector output by the inferior colliculus of the barn owl. Journal of Neuroscience, 26, 5948–5954.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Cox, W., & Fischer, B. J. (2015). Optimal prediction of moving sound source direction in the owl. PLoS Computational Biology, 11, e1004360.

    Article  PubMed  PubMed Central  Google Scholar 

  • Edut, S., & Eilam, D. (2004). Protean behavior under barn-owl attack: voles alternate between freezing and fleeing and spiny mice flee in alternating patterns. Behavioural Brain Research, 155, 207–216.

    Article  PubMed  Google Scholar 

  • Egnor, R. S. (2001). Effects of binaural decorrelation on neural and behavioral processing of interaural level differences in the barn owl (Tyto alba). Journal of Comparative Physiology A, 187, 589–595.

    Article  CAS  Google Scholar 

  • Eliasmith C., & Anderson C.H. (2004). Neural engineering: Computation, representation, and dynamics in neurobiological systems. Cambridge, MA: MIT Press.

  • Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429–433.

    Article  CAS  PubMed  Google Scholar 

  • Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2011). Neural correlates of reliability-based cue weighting during multisensory integration. Nature Neuroscience, 15, 146–154.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fischer, B. J., & Konishi, M. (2008). Variability reduction in interaural time difference tuning in the barn owl. Journal of Neurophysiology, 100, 708–715.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fischer, B. J., & Peña, J. L. (2011). Owl’s behavior and neural representation predicted by Bayesian inference. Nature Neuroscience, 14, 1061–1066.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Fischer, B. J., Peña, J. L., & Konishi, M. (2007). Emergence of multiplicative auditory responses in the midbrain of the barn owl. Journal of Neurophysiology, 98, 1181–1193.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fischer, B. J., Anderson, C. H., & Peña, J. L. (2009). Multiplicative auditory spatial receptive fields created by a hierarchy of population codes. PLoS One, 4, e8015.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fux, M., & Eilam, D. (2009a). How barn owls (Tyto alba) visually follow moving voles (Microtus socialis) before attacking them. Physiology and Behavior, 98, 359–366.

    Article  CAS  PubMed  Google Scholar 

  • Fux, M., & Eilam, D. (2009b). The trigger for barn owl (Tyto alba) attack is the onset of stopping or progressing of the prey. Behavioural Processes, 81, 140–143.

    Article  PubMed  Google Scholar 

  • Girshick, A. R., Landy, M. S., & Simoncelli, E. P. (2011). Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nature Neuroscience, 14, 926–932.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Gold, J. I., & Shadlen, M. N. (2001). Neural computations that underlie decisions about sensory stimuli. Trends in Cognitive Science, 5, 10–16.

    Article  Google Scholar 

  • Gu, Y., Angelaki, D. E., & DeAngelis, G. C. (2008). Neural correlates of multisensory cue integration in macaque MSTd. Nature Neuroscience, 11, 1201–1210.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Hillis, J. M., Watt, S. J., Landy, M. S., & Banks, M. S. (2004). Slant from texture and disparity cues: optimal cue combination. Journal of Vision, 4, 1.

    Article  Google Scholar 

  • Hollensteiner, K. J., Pieper, F., Engler, G., König, P., & Engel, A. K. (2015). Crossmodal integration improves sensory detection thresholds in the ferret. PLoS One, 10, e0124952.

    Article  PubMed  PubMed Central  Google Scholar 

  • Jacobs, R. A. (1999). Optimal integration of texture and motion cues to depth. Vision Research, 39, 3621–3629.

    Article  CAS  PubMed  Google Scholar 

  • Jazayeri, M., & Movshon, J. A. (2006). Optimal representation of sensory information by neural populations. Nature Neuroscience, 9, 690–696.

    Article  CAS  PubMed  Google Scholar 

  • Jeffress, L. A., Blodgett, H. C., & Deatherage, B. H. (1962). Effect of interaural correlation on the precision of centering a noise. Journal of the Acoustical Society of America, 34, 1122–1123.

    Article  Google Scholar 

  • Johnson, D. H. (1980). The relationship between spike rate and synchrony in responses of auditory-nerve fibers to single tones. Journal of the Acoustical Society of America, 68, 1115–1122.

    Article  CAS  PubMed  Google Scholar 

  • Keller, C. H., Hartung, K., & Takahashi, T. T. (1998). Head-related transfer functions of the barn owl: measurement and neural responses. Hearing Research, 118, 13–34.

    Article  CAS  PubMed  Google Scholar 

  • Kita, H. & Armstrong, W. (1991). A biotin-containing compound N-(2-aminoethyl)biotinamide for intracellular labeling and neuronal tracing studies: comparison with biocytin. Journal of Neuroscience Methods, 37, 141–150.

  • Knill, D. C., & Pouget, A. (2004). The Bayesian brain: the role of uncertainty in neural coding and computation. Trends in Neurosciences, 27, 712–719.

    Article  CAS  PubMed  Google Scholar 

  • Knudsen, E. I. (1982). Auditory and visual maps of space in the optic tectum of the owl. Journal of Neuroscience, 2, 1177–1194.

    CAS  PubMed  Google Scholar 

  • Knudsen, E. I. (1983). Subdivisions of the inferior colliculus in the barn owl (Tyto alba). Journal of Comparative Neurology, 218, 174–186.

    Article  CAS  PubMed  Google Scholar 

  • Knudsen, E. I., & Konishi, M. (1978). A neural map of auditory space in the owl. Science, 200, 795–797.

    Article  CAS  PubMed  Google Scholar 

  • Knudsen, E. I., & Konishi, M. (1979). Mechanisms of sound localization in the barn owl (Tyto alba). Journal of Comparative Physiology, 133, 13–21.

    Article  Google Scholar 

  • Knudsen, E. I., Konishi, M., & Pettigrew, J. D. (1977). Receptive fields of auditory neurons in the owl. Science, 198, 1278–1280.

    Article  CAS  PubMed  Google Scholar 

  • Knudsen, E. I., Blasdel, G. G., & Konishi, M. (1979). Sound localization by the barn owl (Tyto alba) measured with the search coil technique. Journal of Comparative Physiology, 133, 1–11.

    Article  Google Scholar 

  • Koch, C. (2004). Biophysics of computation: information processing in single neurons. USA: Oxford University Press.

    Google Scholar 

  • Konishi, M. (1993). Neuroethology of sound localization in the owl. Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology, 173, 3–7.

    Article  Google Scholar 

  • Köppl, C. (1997a). Phase locking to high frequencies in the auditory nerve and cochlear nucleus magnocellularis of the barn owl, Tyto alba. Journal of Neuroscience, 17, 3312–3321.

    PubMed  Google Scholar 

  • Köppl, C. (1997b). Frequency tuning and spontaneous activity in the auditory nerve and cochlear nucleus magnocellularis of the barn owl Tyto alba. Journal of Neurophysiology, 77, 364–377.

  • Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22, 79–86.

    Article  Google Scholar 

  • Landy, M. S., & Kojima, H. (2001). Ideal cue combination for localizing texture-defined edges. Journal of the Optical Society of America A, 18, 2307.

    Article  CAS  Google Scholar 

  • Landy, M. S., Maloney, L. T., Johnston, E. B., & Young, M. (1995). Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Research, 35, 389–412.

    Article  CAS  PubMed  Google Scholar 

  • Landy MS, Banks MS, Knill DC (2011) Ideal-observer models of cue integration. Sensory Cue Integration: 5–29.

  • Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006). Bayesian inference with probabilistic population codes. Nature Neuroscience, 9, 1432–1438.

    Article  CAS  PubMed  Google Scholar 

  • Manley, G. A., Koppl, C., & Konishi, M. (1988). A neural map of interaural intensity differences in the brain stem of the barn owl. Journal of Neuroscience, 8, 2665–2676.

    CAS  PubMed  Google Scholar 

  • Middlebrooks, J. C., & Green, D. M. (1991). Sound localization by human listeners. Annual Review of Psychology, 42, 135–159.

    Article  CAS  PubMed  Google Scholar 

  • Mogdans, J., & Knudsen, E. I. (1994). Representation of interaural level difference in the VLVp, the first site of binaural comparison in the barn owl’s auditory system. Hearing Research, 74, 148–164.

    Article  CAS  PubMed  Google Scholar 

  • Moiseff, A. (1989). Bi-coordinate sound localization by the barn owl. Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology, 164, 637–644.

    Article  CAS  Google Scholar 

  • Moiseff, A., & Konishi, M. (1983). Binaural characteristics of units in the owl’s brainstem auditory pathway: precursors of restricted spatial receptive fields. Journal of Neuroscience, 3, 2553–2562.

    CAS  PubMed  Google Scholar 

  • Mysore, S. P., & Knudsen, E. I. (2012). Reciprocal inhibition of inhibition: a circuit motif for flexible categorization in stimulus selection. Neuron, 73, 193–205.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Mysore, S. P., & Knudsen, E. I. (2013). A shared inhibitory circuit for both exogenous and endogenous control of stimulus selection. Nature Neuroscience, 16, 473–478.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Niven, J. E., & Laughlin, S. B. (2008). Energy limitation as a selective pressure on the evolution of sensory systems. Journal of Experimental Biology, 211, 1792–1804.

    Article  CAS  PubMed  Google Scholar 

  • Nix, J., & Hohmann, V. (2006). Sound source localization in real sound fields based on empirical statistics of interaural parametersa). Journal of the Acoustical Society of America, 119, 463–479.

    Article  PubMed  Google Scholar 

  • Ohayon, S., van der Willigen, R. F., Wagner, H., Katsman, I., & Rivlin, E. (2006). On the barn owl’s visual pre-attack behavior: I. Structure of head movements and motion patterns. Journal of Comparative Physiology A, 192, 927–940.

    Article  Google Scholar 

  • Ohshiro, T., Angelaki, D. E., & DeAngelis, G. C. (2011). A normalization model of multisensory integration. Nature Neuroscience, 14, 775–782.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Oruç, I., Maloney, L. T., & Landy, M. S. (2003). Weighted linear cue combination with possibly correlated error. Vision Research, 43, 2451–2468.

    Article  PubMed  Google Scholar 

  • Palmer, A. R., & Russell, I. J. (1986). Phase-locking in the cochlear nerve of the guinea-pig and its relation to the receptor potential of inner hair-cells. Hearing Research, 24, 1–15.

    Article  CAS  PubMed  Google Scholar 

  • Pecka, M., Siveke, I., Grothe, B., & Lesica, N. A. (2010). Enhancement of ITD coding within the initial stages of the auditory pathway. Journal of Neurophysiology, 103, 38–46.

    Article  PubMed  Google Scholar 

  • Peña, J. L., & Konishi, M. (2000). Cellular mechanisms for resolving phase ambiguity in the owl’s inferior colliculus. Proceedings of the National Academy of Science, 97, 11787–11792.

    Article  Google Scholar 

  • Peña, J. L., & Konishi, M. (2001). Auditory spatial receptive fields created by multiplication. Science, 292, 249–252.

    Article  PubMed  Google Scholar 

  • Peña, J. L., & Konishi, M. (2002). From postsynaptic potentials to spikes in the genesis of auditory spatial receptive fields. Journal of Neuroscience, 22, 5652–5658.

    PubMed  Google Scholar 

  • Peña, J. L., & Konishi, M. (2004). Robustness of multiplicative processes in auditory spatial tuning. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience, 24, 8907–8910.

    Article  Google Scholar 

  • Roman, N., Wang, D., & Brown, G. J. (2003). Speech segregation based on sound localization. Journal of the Acoustical Society of America, 114, 2236–2252.

    Article  PubMed  Google Scholar 

  • Saberi, K., Takahashi, Y., Konishi, M., Albeck, Y., Arthur, B. J., & Farahbod, H. (1998). Effects of interaural decorrelation on neural and behavioral detection of spatial cues. Neuron, 21, 789–798.

    Article  CAS  PubMed  Google Scholar 

  • Shi, L., & Griffiths, T.L. (2009). Neural implementation of hierarchical Bayesian inference by importance sampling. In Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, & A. Culotta (Eds.), Advances in neural information processing systems 22 (pp. 1669–1677). Cambridge, MA: MIT Press. 

  • Simoncelli, E.P. (2009). Optimal estimation in sensory systems.  In M. Gazzaniga (Ed.), The cognitive neurosciences (vol. 4, pp. 525–535). Cambridge, MA: MIT Press.

  • Spitzer, M. W., Bala, A. D., & Takahashi, T. T. (2003). Auditory spatial discrimination by barn owls in simulated echoic conditions. Journal of the Acoustical Society of America, 113, 1631–1645.

    Article  PubMed  Google Scholar 

  • Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Review Neuroscience, 9, 255–266.

    Article  CAS  Google Scholar 

  • Takahashi, T. T. (2010). How the owl tracks its prey–II. Journal of Experimental Biology, 213, 3399–3408.

    Article  PubMed  PubMed Central  Google Scholar 

  • Takahashi, T., Moiseff, A., & Konishi, M. (1984). Time and intensity cues are processed independently in the auditory system of the owl. Journal of Neuroscience, 4, 1781–1786.

    CAS  PubMed  Google Scholar 

  • van Beers, R. J., Sittig, A. C., & Gon, J. J. (1999). Integration of proprioceptive and visual position-information: an experimentally supported model. Journal of Neurophysiology, 81, 1355–1364.

    PubMed  Google Scholar 

  • Van Trees HL (2004) Detection, estimation, and modulation theory. New York, NY: Wiley.

  • Wagner, H., Takahashi, T., & Konishi, M. (1987). Representation of interaural time difference in the central nucleus of the barn owl’s inferior colliculus. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience, 7, 3105–3116.

    CAS  Google Scholar 

  • Whitchurch, E. A., & Takahashi, T. T. (2006). Combined auditory and visual stimuli facilitate head saccades in the barn owl (Tyto alba). Journal of Neurophysiology, 96, 730–745.

    Article  PubMed  Google Scholar 

  • Wightman, F. L. & Kistler, D. J. (1993). Sound localization. In W. A. Yost, A. N. Popper, & R. R. Fay (Eds.), Human psychophysics (pp. 155–192). New York: Springer-Verlag.

  • Xu, J., Yu, L., Rowland, B. A., Stanford, T. R., & Stein, B. E. (2012). Incorporating cross-modal statistics in the development and maintenance of multisensory integration. Journal of Neuroscience, 32, 2287–2298.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We thank F. Cazettes, K. Keller and T. Takahashi for helpful discussions and comments on the manuscript. We thank K. Keller and T. Takahashi for providing the HRTFs. This work was funded by the National Institutes of Health (Grant DC012949).

Author information

Authors and Affiliations

  1. Department of Mathematics, Seattle University, 901 12th Ave, Seattle, WA, 98122, USA

    Brian J. Fischer

  2. Department of Neuroscience, Albert Einstein College of Medicine, 1410 Pelham Parkway South, Bronx, NY, 10461, USA

    Jose Luis Peña

Authors
  1. Brian J. Fischer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose Luis Peña
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian J. Fischer.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Action Editor: Susanne Schreiber

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fischer, B.J., Peña, J.L. Optimal nonlinear cue integration for sound localization. J Comput Neurosci 42, 37–52 (2017). https://doi.org/10.1007/s10827-016-0626-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10827-016-0626-4

Keywords

Search

Navigation