Skip to main content
SpringerLink
Log in

The Role of the Number of Examples in Convolutional Neural Networks with Hebbian Learning

  • Conference paper
  • First Online:
Advances in Computational Intelligence (MICAI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13612))

Included in the following conference series:

  • 759 Accesses

Abstract

Both synaptic plasticity rules (the so-called Hebbian rules) and Convolutional Neural Networks are based on or inspired by well-established models of Computational Neuroscience about mammal vision. There are some theoretical advantages associated with these frameworks, including online learning in Hebbian Learning. In the case of Convolutional Neural Networks, such advantages have been translated into remarkable results in image classification in the last decade. Nevertheless, such success is not shared in Hebbian Learning. In this paper, we explore the hypothesis of the necessity of a wider dataset for the classification of mono-instantiated objects, this is, objects that can be represented in a single cluster in the feature space. By using 15 mono-instantiated classes, the Adam optimizer reaches the maximum accuracy with fewer examples but using more epochs. In comparison, Hebbian rule BCM demands more examples but keeps using real-time learning. This result is a positive answer to the principal hypothesis and enlights how Hebbian learning can find a niche in the mainstream of Deep Learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aguilar Canto, F., Brito-Loeza, C.: El potencial del aprendizaje hebbiano en la clasificación supervisada. Boletín de la Sociedad Mexicana de Computación Científica y sus Aplicaciones (2021)

    Google Scholar 

  • Aguilar-Canto, F., Calvo, H.: A hebbian approach to non-spatial prelinguistic reasoning. Brain Sci. 12(2), 281 (2022)

    Article  Google Scholar 

  • Aguilar Canto, F.J.: Convolutional neural networks with hebbian-based rules in online transfer learning. In: Martínez-Villaseñor, L., Herrera-Alcántara, O., Ponce, H., Castro-Espinoza, F.A. (eds.) MICAI 2020. LNCS (LNAI), vol. 12468, pp. 35–49. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60884-2_3

    Chapter  Google Scholar 

  • Amato, G., Carrara, F., Falchi, F., Gennaro, C., Lagani, G.: Hebbian learning meets deep convolutional neural networks. In: Ricci, E., Rota Bulò, S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11751, pp. 324–334. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30642-7_29

    Chapter  Google Scholar 

  • Bahroun, Y., Hunsicker, E., Soltoggio, A.: Building efficient deep hebbian networks for image classification tasks. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 364–372. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_42

    Chapter  Google Scholar 

  • Bahroun, Y., Soltoggio, A.: Online representation learning with single and multi-layer hebbian networks for image classification. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 354–363. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_41

    Chapter  Google Scholar 

  • Bienenstock, E.L., Cooper, L.N., Munro, P.W.: Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2(1), 32–48 (1982)

    Article  Google Scholar 

  • Bliss, T.V., Lømo, T.: Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J. Physiol. 232(2), 331–356 (1973)

    Article  Google Scholar 

  • Burbank, K.S.: Mirrored STDP implements autoencoder learning in a network of spiking neurons. PLoS Comput. Biol. 11(12), e1004566 (2015)

    Article  Google Scholar 

  • Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vision 113(1), 54–66 (2015)

    Article  MathSciNet  Google Scholar 

  • Chollet, F.: Xception: deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

    Google Scholar 

  • Dayan, P., Abbott, L.F.: Theoretical neuroscience: computational and mathematical modeling of neural systems (2001)

    Google Scholar 

  • Desimone, R., Albright, T.D., Gross, C.G., Bruce, C.: Stimulus-selective properties of inferior temporal neurons in the macaque. J. Neurosci. 4(8), 2051–2062 (1984)

    Article  Google Scholar 

  • Fukushima, K.: Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw. 1(2), 119–130 (1988)

    Article  Google Scholar 

  • Han, K., et al.: A survey on vision transformer. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2022)

    Google Scholar 

  • He, K., Zhang, X., Ren, S., and Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imageNet classification. In: Proceedings of the IEEE international conference on computer vision, pp. 1026–1034 (2015)

    Google Scholar 

  • Herzog, M.H., Clarke, A.M.: Why vision is not both hierarchical and feedforward. Front. Comput. Neurosci. 8, 135 (2014)

    Article  Google Scholar 

  • Holca-Lamarre, R., Lücke, J., Obermayer, K.: Models of acetylcholine and dopamine signals differentially improve neural representations. Front. Comput. Neurosci. 11, 54 (2017)

    Article  Google Scholar 

  • Huang, Y., Liu, J., Harkin, J., McDaid, L., Luo, Y.: An memristor-based synapse implementation using BCM learning rule. Neurocomputing 423, 336–342 (2021)

    Article  Google Scholar 

  • Keck, C., Savin, C., Lücke, J.: Feedforward inhibition and synaptic scaling-two sides of the same coin? PLoS Comput. Biol. 8(3), e1002432 (2012)

    Article  MathSciNet  Google Scholar 

  • Kheradpisheh, S.R., Ganjtabesh, M., Masquelier, T.: Bio-inspired unsupervised learning of visual features leads to robust invariant object recognition. Neurocomputing 205, 382–392 (2016)

    Article  Google Scholar 

  • Kheradpisheh, S.R., Ganjtabesh, M., Thorpe, S.J., Masquelier, T.: STDP-based spiking deep convolutional neural networks for object recognition. Neural Netw. 99, 56–67 (2018)

    Article  Google Scholar 

  • Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In 3rd International Conference for Learning Representations, San Diego (2015)

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 (2012)

    Google Scholar 

  • Lagani, G., Falchi, F., Gennaro, C., Amato, G.: Evaluating hebbian learning in a semi-supervised setting. In: LOD 2021. LNCS, vol. 13164, pp. 365–379. Springer, cham (2021). https://doi.org/10.1007/978-3-030-95470-3_28

  • Lagani, G., Falchi, F., Gennaro, C., Amato, G.: Hebbian semi-supervised learning in a sample efficiency setting. Neural Netw. 143, 719–731 (2021)

    Article  Google Scholar 

  • Lagani, G., Falchi, F., Gennaro, C., Amato, G.: Training convolutional neural networks with competitive hebbian learning approaches. In: LOD 2021. LNCS, vol. 13163, pp. 25–40. Springer, cham (2021). https://doi.org/10.1007/978-3-030-95467-3_2

  • Lagani, G., Falchi, F., Gennaro, C., Amato, G.: Comparing the performance of hebbian against backpropagation learning using convolutional neural networks. In: Neural Computing and Applications, pp. 1–17 (2022)

    Google Scholar 

  • LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Google Scholar 

  • Lillicrap, T.P., Santoro, A., Marris, L., Akerman, C.J., Hinton, G.: Backpropagation and the brain. Nat. Rev. Neurosci. 21(6), 335–346 (2020)

    Article  Google Scholar 

  • Liu, D., Yue, S.: Visual pattern recognition using unsupervised spike timing dependent plasticity learning. In 2016 International Joint Conference on Neural Networks (IJCNN), pp. 285–292. IEEE (2016)

    Google Scholar 

  • Lømo, T.: Frequency potentiation of excitatory synaptic activity in dentate area of hippocampal formation. In: Acta Physiologica Scandinavica, p. 128. Blackwell Science Ltd. Po. Box 88, Osney Mead, Oxford Ox2 0ne, Oxon, England (1966)

    Google Scholar 

  • Magotra, A., Kim, J.: Transfer learning for image classification using hebbian plasticity principles. In: Proceedings of the 2019 3rd International Conference on Computer Science and Artificial Intelligence, pp. 233–238 (2019)

    Google Scholar 

  • Magotra, A., Kim, J.: Improvement of heterogeneous transfer learning efficiency by using hebbian learning principle. Appl. Sci. 10(16), 5631 (2020)

    Article  Google Scholar 

  • Magotra, A., Kim, J.: Neuromodulated dopamine plastic networks for heterogeneous transfer learning with hebbian principle. Symmetry 13(8), 1344 (2021)

    Article  Google Scholar 

  • Markram, H., Lübke, J., Frotscher, M., Sakmann, B.: Regulation of synaptic efficacy by coincidence of postsynaptic aps and EPSPs. Science 275(5297), 213–215 (1997)

    Article  Google Scholar 

  • Masquelier, T., Thorpe, S.J.: Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Comput. Biol. 3(2), e31 (2007)

    Article  Google Scholar 

  • McMahan, H.B., et al.: Ad click prediction: a view from the trenches. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1222–1230 (2013)

    Google Scholar 

  • Miconi, T.: Hebbian learning with gradientes: hebbian convolutional neural networks with modern deep learning frameworks. arXiv preprint arXiv:2107.01729

  • Miconi, T., Rawal, A., Clune, J., Stanley, K.O.: Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity. arXiv preprint arXiv:2002.10585

  • Miconi, T., Stanley, K., Clune, J.: Differentiable plasticity: training plastic neural networks with backpropagation. In: International Conference on Machine Learning, pp. 3559–3568. PMLR (2018)

    Google Scholar 

  • Oja, E.: A simplified neuron model as a principal component analyzer. J. Math. Biol. 15(3), 267–273 (1982)

    Article  MathSciNet  Google Scholar 

  • Panda, P., Roy, K.: Unsupervised regenerative learning of hierarchical features in spiking deep networks for object recognition. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 299–306. IEEE (2016)

    Google Scholar 

  • Pogodin, R., Mehta, Y., Lillicrap, T., Latham, P.: Towards biologically plausible convolutional networks. In: Advances in Neural Information Processing Systems 34 (2021)

    Google Scholar 

  • Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)

    Article  Google Scholar 

  • Rolls, E.: Neurons in the cortex of the temporal lobe and in the amygdala of the monkey with responses selective for faces. Hum. Neurobiol. 3(4), 209–222 (1984)

    Google Scholar 

  • Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386 (1958)

    Article  Google Scholar 

  • Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., Masquelier, T., Maida, A.: Deep learning in spiking neural networks. Neural Netw. 111, 47–63 (2019)

    Article  Google Scholar 

  • Tavanaei, A., Masquelier, T., Maida, A.: Representation learning using event-based STDP. Neural Netw. 105, 294–303 (2018)

    Article  Google Scholar 

  • Tavanaei, A., Masquelier, T., Maida, A.S.: Acquisition of visual features through probabilistic spike-timing-dependent plasticity. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 307–314. IEEE (2016)

    Google Scholar 

  • Van Essen, D.C., Maunsell, J.H.: Hierarchical organization and functional streams in the visual cortex. Trends Neurosci. 6, 370–375 (1983)

    Article  Google Scholar 

  • Wadhwa, A., Madhow, U.: Bottom-up deep learning using the hebbian principle (2016)

    Google Scholar 

  • Wallis, G.: Using spatio-temporal correlations to learn invariant object recognition. Neural Netw. 9(9), 1513–1519 (1996)

    Article  Google Scholar 

  • Yeo, W.-H., Heo, Y.-J., Choi, Y.-J., Kim, B.-G.: Place classification algorithm based on semantic segmented objects. Appl. Sci. 10(24), 9069 (2020)

    Article  Google Scholar 

  • Zhao, B., Ding, R., Chen, S., Linares-Barranco, B., Tang, H.: Feedforward categorization on AER motion events using cortex-like features in a spiking neural network. IEEE Trans. Neural Netw. Learn. Syst. 26(9), 1963–1978 (2014)

    Article  MathSciNet  Google Scholar 

  • Hubel, D.H., Wiesel, T.N.: Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148(3), 574 (1959). Wiley-Blackwell

    Google Scholar 

  • Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106 (1962). Wiley-Blackwell

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratorio de Ciencias Cognitivas Computacionales, Centro de Investigación en Computación, Instituto PolitÉcnico Nacional, Mexico, USA

    Fernando Aguilar-Canto & Hiram Calvo

Authors
  1. Fernando Aguilar-Canto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hiram Calvo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Aguilar-Canto .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, Mexico, Mexico

    Obdulia Pichardo Lagunas

  2. Centro de Investigación Científica y de Educación Superior de Ensenada, Ensenada, Baja California, Mexico

    Juan Martínez-Miranda

  3. Instituto Politécnico Nacional, Mexico, Mexico

    Bella Martínez Seis

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Aguilar-Canto, F., Calvo, H. (2022). The Role of the Number of Examples in Convolutional Neural Networks with Hebbian Learning. In: Pichardo Lagunas, O., Martínez-Miranda, J., Martínez Seis, B. (eds) Advances in Computational Intelligence. MICAI 2022. Lecture Notes in Computer Science(), vol 13612. Springer, Cham. https://doi.org/10.1007/978-3-031-19493-1_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19493-1_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19492-4

  • Online ISBN: 978-3-031-19493-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation