Hostname: page-component-76fb5796d-zzh7m Total loading time: 0 Render date: 2024-04-28T06:44:49.392Z Has data issue: false hasContentIssue false
  • English
  • Français

Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses

Published online by Cambridge University Press:  06 December 2023

Tal Golan Open the ORCID record for Tal Golan [Opens in a new window] ,
JohnMark Taylor Open the ORCID record for JohnMark Taylor [Opens in a new window] ,
Heiko Schütt ,
Benjamin Peters ,
Rowan P. Sommers ,
Katja Seeliger ,
Adrien Doerig Open the ORCID record for Adrien Doerig [Opens in a new window] ,
Paul Linton ,
Talia Konkle  and
Marcel van Gerven Open the ORCID record for Marcel van Gerven [Opens in a new window]
...Show all authors
Tal Golan
Affiliation:
Department of Cognitive and Brain Sciences, Ben-Gurion University of the Negev, Be'er Sheva, Israel golan.neuro@bgu.ac.il brainsandmachines.org
JohnMark Taylor
Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/
Heiko Schütt
Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/ Center for Neural Science, New York University, New York, NY, USA
Benjamin Peters
Affiliation:
School of Psychology & Neuroscience, University of Glasgow, Glasgow, UK benjamin.peters@posteo.de
Rowan P. Sommers
Affiliation:
Department of Neurobiology of Language, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands rowan.sommers@mpi.nl
Katja Seeliger
Affiliation:
Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany katjaseeliger@posteo.de
Adrien Doerig
Affiliation:
Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany adoerig@uni-osnabrueck.de tim.kietzmann@uni-osnabrueck.de kietzmannlab.org kietzmannlab.org
Paul Linton
Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/ Presidential Scholars in Society and Neuroscience, Center for Science and Society, Columbia University, New York, NY, USA Italian Academy for Advanced Studies in America, Columbia University, New York, NY, USA
Talia Konkle
Affiliation:
Department of Psychology and Center for Brain Sciences, Harvard University, Cambridge, MA, USA talia_konkle@harvard.edu https://konklab.fas.harvard.edu/
Marcel van Gerven
Affiliation:
Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands artcogsys.com
Konrad Kording
Affiliation:
Departments of Bioengineering and Neuroscience, University of Pennsylvania, Philadelphia, PA, USA koerding@gmail.com kordinglab.com Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada blake.richards@mila.quebec linclab.org
Blake Richards
Affiliation:
Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada blake.richards@mila.quebec linclab.org Mila, Montreal, QC, Canada School of Computer Science, McGill University, Montreal, QC, Canada Department of Neurology & Neurosurgery, McGill University, Montreal, QC, Canada Montreal Neurological Institute, Montreal, QC, Canada
Tim C. Kietzmann
Affiliation:
Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany adoerig@uni-osnabrueck.de tim.kietzmann@uni-osnabrueck.de kietzmannlab.org kietzmannlab.org
Grace W. Lindsay
Affiliation:
Department of Psychology and Center for Data Science, New York University, New York, NY, USA grace.lindsay@nyu.edu lindsay-lab.github.io
Nikolaus Kriegeskorte
Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/ Departments of Psychology, Neuroscience, and Electrical Engineering, Columbia University, New York, NY, USA
Get access Rights & Permissions [Opens in a new window]

Abstract

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 46 , 2023 , e392
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bashivan, P., Kar, K., & DiCarlo, J. J. (2019). Neural population control via deep image synthesis. Science (New York, N.Y.), 364(6439), eaav9436. https://doi.org/10.1126/science.aav9436CrossRefGoogle ScholarPubMed
Cichy, R. M., Roig, G., & Oliva, A. (2019). The Algonauts project. Nature Machine Intelligence, 1(12), 613–613. https://doi.org/10.1038/s42256-019-0127-zCrossRefGoogle Scholar
Cohen, J., Rosenfeld, E., & Kolter, Z. (2019). Certified adversarial robustness via randomized smoothing. In Chaudhuri, K. & Salakhutdinov, R. (Eds.), Proceedings of the 36th international conference on machine learning. Proceedings of Machine Learning Research, Long Beach, CA, USA (Vol. 97, pp. 1310–1320). https://proceedings.mlr.press/v97/cohen19c.htmlGoogle Scholar
Doerig, A., Sommers, R., Seeliger, K., Richards, B., Ismael, J., Lindsay, G., … Kietzmann, T. C. (2022). The neuroconnectionist research programme. Nature Reviews Neuroscience, 24, 431–450. https://doi.org/10.1038/s41583-023-00705-wCrossRefGoogle Scholar
Dujmović, M., Malhotra, G., & Bowers, J. S. (2020). What do adversarial images tell us about human vision?. eLife, 9, e55978. https://doi.org/10.7554/eLife.55978CrossRefGoogle ScholarPubMed
Dwivedi, K., Bonner, M. F., Cichy, R. M., & Roig, G. (2021). Unveiling functions of the visual cortex using task-specific deep neural networks. PLoS Computational Biology, 17(8), e1009267. https://doi.org/10.1371/journal.pcbi.1009267CrossRefGoogle ScholarPubMed
Feather, J., Durango, A., Gonzalez, R., & McDermott, J. (2019). Metamers of neural networks reveal divergence from human perceptual systems. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., & Garnett, R. (Eds.), Advances in Neural Information Processing Systems, Vancouver, BC, Canada (Vol. 32, pp. 10078–10089). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2019/file/ac27b77292582bc293a51055bfc994ee-Paper.pdfGoogle Scholar
Funahashi, K. I., & Nakamura, Y. (1993). Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks, 6(6), 801–806. https://doi.org/10.1016/S0893-6080(05)80125-XCrossRefGoogle Scholar
Geirhos, R., Narayanappa, K., Mitzkus, B., Thieringer, T., Bethge, M., Wichmann, F. A., & Brendel, W. (2021). Partial success in closing the gap between human and machine vision. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. S., & Wortman Vaughan, J. (Eds.), Advances in Neural Information Processing Systems (Vol. 34, pp. 23885–23899). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2021/file/c8877cff22082a16395a57e97232bb6f-Paper.pdfGoogle Scholar
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2019). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International conference on learning representations, New Orleans, LA, USA. https://openreview.net/forum?id=Bygh9j09KXGoogle Scholar
Golan, T., Raju, P. C., & Kriegeskorte, N. (2020). Controversial stimuli: Pitting neural networks against each other as models of human cognition. Proceedings of the National Academy of Sciences of the United States of America, 117(47), 29330–29337. https://doi.org/10.1073/pnas.1912334117CrossRefGoogle ScholarPubMed
Güçlü, U., & van Gerven, M. A. J. (2015). Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27), 10005–10014. https://doi.org/10.1523/JNEUROSCI.5023-14.2015CrossRefGoogle ScholarPubMed
Guo, C., Lee, M., Leclerc, G., Dapello, J., Rao, Y., Madry, A., & Dicarlo, J. (2022). Adversarially trained neural representations are already as robust as biological neural representations. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., & Sabato, S. (Eds.), Proceedings of the 39th international conference on machine learning. Proceedings of Machine Learning Research, Baltimore, MD, USA (Vol. 162, pp. 8072–8081). PMLR. https://proceedings.mlr.press/v162/guo22d.htmlGoogle Scholar
Hermann, K., Chen, T., & Kornblith, S. (2020). The origins and prevalence of texture bias in convolutional neural networks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., & Lin, H. (Eds.), Advances in Neural Information Processing Systems, Vancouver, BC, Canada (Vol. 33, pp. 19000–19015). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2020/file/db5f9f42a7157abe65bb145000b5871a-Paper.pdfGoogle Scholar
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., … Bandettini, P. A. (2008). Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron, 60(6), 1126–1141. https://doi.org/10.1016/j.neuron.2008.10.043CrossRefGoogle ScholarPubMed
Lakatos, I. (1978). Science and pseudoscience. Philosophical Papers, 1, 1–7.Google Scholar
Lindsay, G. W. (2021). Convolutional neural networks as a model of the visual system: Past, present, and future. Journal of Cognitive Neuroscience, 33(10), 2017–2031. https://doi.org/10.1162/jocn_a_01544CrossRefGoogle Scholar
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2019). Towards deep -learning models resistant to adversarial attacks. In International conference on learning representations, Vancouver, BC, Canada. https://openreview.net/forum?id=rJzIBfZAbGoogle Scholar
Newell, A. (1973). You can't play 20 questions with nature and win: Projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing: Proceedings of the 8th annual Carnegie symposium on cognition, held at the Carnegie-Mellon University, Pittsburgh, Pennsylvania, May 19, 1972 (pp. 283–305). Academic Press.Google Scholar
Nonaka, S., Majima, K., Aoki, S. C., & Kamitani, Y. (2021). Brain hierarchy score: Which deep neural networks are hierarchically brain-like?. iScience, 24(9), 103013. https://doi.org/10.1016/j.isci.2021.103013CrossRefGoogle ScholarPubMed
Nuriel, O., Benaim, S., & Wolf, L. (2021). Permuted AdaIN: Reducing the bias towards global statistics in image classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9482–9491). Online. https://openaccess.thecvf.com/content/CVPR2021/html/Nuriel_Permuted_AdaIN_Reducing_the_Bias_Towards_Global_Statistics_in_Image_CVPR_2021_paper.htmlCrossRefGoogle Scholar
Olshausen, B. A., & Field, D. J. (2005). How close are we to understanding V1?. Neural Computation, 17(8), 1665–1699. https://doi.org/10.1162/0899766054026639CrossRefGoogle ScholarPubMed
Peters, B., & Kriegeskorte, N. (2021). Capturing the objects of vision with neural networks. Nature Human Behaviour, 5(9), 1127–1144. https://doi.org/10.1038/s41562-021-01194-6CrossRefGoogle ScholarPubMed
Rajalingham, R., Issa, E. B., Bashivan, P., Kar, K., Schmidt, K., & DiCarlo, J. J. (2018). Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks. Journal of Neuroscience, 38(33), 7255–7269. https://doi.org/10.1523/JNEUROSCI.0388-18.2018CrossRefGoogle ScholarPubMed
Rust, N. C., & Movshon, J. A. (2005). In praise of artifice. Nature Neuroscience, 8(12), 1647–1650. https://doi.org/10.1038/nn1606CrossRefGoogle ScholarPubMed
Schäfer, A. M., & Zimmermann, H. G. (2007). Recurrent neural networks are universal approximators. International Journal of Neural Systems, 17(4), 253–263. https://doi.org/10.1142/S0129065707001111CrossRefGoogle ScholarPubMed
Schrimpf, M., Kubilius, J., Hong, H., Majaj, N. J., Rajalingham, R., Issa, E. B., … DiCarlo, J. J. (2018). Brain-Score: Which artificial neural network for object recognition is most brain-like?. BioRxiv, 407007. https://doi.org/10.1101/407007Google Scholar
Serre, T. (2019). Deep learning: The good, the bad, and the ugly. Annual Review of Vision Science, 5, 399–426. https://doi.org/10.1146/annurev-vision-091718-014951CrossRefGoogle ScholarPubMed
Spoerer, C. J., McClure, P., & Kriegeskorte, N. (2017). Recurrent convolutional neural networks: A better model of biological object recognition. Frontiers in Psychology, 8, 1551. https://doi.org/10.3389/fpsyg.2017.01551CrossRefGoogle ScholarPubMed
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv. arXiv:1312.6199. https://doi.org/10.48550/arXiv.1312.6199Google Scholar
Walker, E. Y., Sinz, F. H., Cobos, E., Muhammad, T., Froudarakis, E., Fahey, P. G., … Tolias, A. S. (2019). Inception loops discover what excites neurons most using deep predictive models. Nature Neuroscience, 22(12), 2060–2065. https://doi.org/10.1038/s41593-019-0517-xCrossRefGoogle ScholarPubMed