Skip to main content
SpringerLink
Log in

Writing to the Hopfield Memory via Training a Recurrent Network

  • Conference paper
  • First Online:
PRICAI 2019: Trends in Artificial Intelligence (PRICAI 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11671))

Included in the following conference series:

  • 2620 Accesses

Abstract

We consider the problem of writing on a Hopfield network. We cast the problem as a supervised learning problem by observing a simple link between the update equations of Hopfield network and recurrent neural networks. We compare the new writing protocol to existing ones and experimentally verify its effectiveness. Our method not only has a better ability of noise recovery, but also has a bigger capacity compared to the other existing writing protocols.

This work is supported partly by China 973 program (No. 2015CB358700), by the National Natural Science Foundation of China (No. 61772059, 61421003), and by the Beijing Advanced Innovation Center for Big Data and Brain Computing (BDBC) and State Key Laboratory of Software Development Environment (No. SKLSDE-2018ZX-17).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    More generally, function \(f_W\) may take the form \(f_W(\mathbf{x})=\mathrm {sign}(W\mathbf{x}+\mathbf{b})\), where \(\mathbf{b}\) is an off-set or threshold vector. In the context of this paper, dropping this offset term \(\mathbf{b}\) is without loss of generality.

References

  1. Anderson, J.R., Bower, G.H.: Human Associative Memory. Psychology Press, London (2014)

    Book  Google Scholar 

  2. Bottou, L.: Largescale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-7908-2604-3_16

    Chapter  Google Scholar 

  3. Cheng, K.S., Lin, J.S., Mao, C.W.: The application of competitive hopfield neural network to medical image segmentation. IEEE Trans. Med. Imaging 15(4), 560–567 (1996)

    Article  Google Scholar 

  4. Ding, J., Sun, Y.Z., Tan, P., Ning, Y.: Detecting communities in networks using competitive hopfield neural network. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2018)

    Google Scholar 

  5. Duan, S., Dong, Z., Hu, X., Wang, L., Li, H.: Small-world hopfield neural networks with weight salience priority and memristor synapses for digit recognition. Neural Comput. Appl. 27(4), 837–844 (2016)

    Article  Google Scholar 

  6. Elman, J.L.: Finding structure in time. Cogn. Sci. 14, 179–211 (1990)

    Article  Google Scholar 

  7. Folli, V., Leonetti, M., Ruocco, G.: On the maximum storage capacity of the hopfield model. Front. Comput. Neurosci. 10, 144 (2017)

    Article  Google Scholar 

  8. Hillar, C., SohlDickstein, J., Koepsell, K.: Efficient and optimal binary hopfield associative memory storage using minimum probability flow. arXiv preprint arXiv:1204.2916 (2012)

  9. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)

    Article  MathSciNet  Google Scholar 

  10. Kanter, I., Sompolinsky, H.: Associative recall of memory without errors. Phys. Rev. A 35(1), 380 (1987)

    Article  Google Scholar 

  11. Kobayashi, M.: Multistate vector product hopfield neural networks. Neurocomputing 272, 425–431 (2018)

    Article  Google Scholar 

  12. Krotov, D., Hopfield, J.J.: Dense associative memory for pattern recognition. In: Advances in Neural Information Processing Systems, pp. 1172–1180 (2016)

    Google Scholar 

  13. Lee, K.Y., Sode-Yome, A., Park, J.H.: Adaptive hopfield neural networks for economic load dispatch. IEEE Trans. Power Syst. 13(2), 519–526 (1998)

    Article  Google Scholar 

  14. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)

    Article  MathSciNet  Google Scholar 

  15. McEliece, R., Posner, E., Rodemich, E., Venkatesh, S.: The capacity of the hopfield associative memory. IEEE Trans. Inform. Theory 33(4), 461–482 (1987)

    Article  MathSciNet  Google Scholar 

  16. Rebentrost, P., Bromley, T.R., Weedbrook, C., Lloyd, S.: Quantum hopfield neural network. Phys. Rev. A 98(4), 042308 (2018)

    Article  Google Scholar 

  17. Storkey, A.: Increasing the capacity of a hopfield network without sacrificing functionality. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 451–456. Springer, Heidelberg (1997). https://doi.org/10.1007/BFb0020196

    Chapter  Google Scholar 

  18. Wang, S.: Classification with incomplete survey data: a hopfield neural network approach. Comput. Oper. Res. 32(10), 2583–2594 (2005)

    Article  Google Scholar 

  19. Zhen, H., Wang, S.N., Zhou, H.J.: Unsupervised prototype learning in an associative-memory network. arXiv preprint arXiv:1704.02848 (2017)

Download references

Author information

Authors and Affiliations

  1. SKLSDE, School of Computer Science and Engineering, Beihang University, Beijing, China

    Han Bao, Richong Zhang & Jinpeng Huai

  2. Beijing Advanced Institution on Big Data and Brain Computing, Beihang University, Beijing, China

    Han Bao, Richong Zhang & Jinpeng Huai

  3. School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

    Yongyi Mao

Authors
  1. Han Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongyi Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinpeng Huai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richong Zhang .

Editor information

Editors and Affiliations

  1. Department of Computing, Macquarie University, Sydney, NSW, Australia

    Abhaya C. Nayak

  2. RIKEN Center for Integrative Medical Sciences, Yokohama, Japan

    Alok Sharma

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bao, H., Zhang, R., Mao, Y., Huai, J. (2019). Writing to the Hopfield Memory via Training a Recurrent Network. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29911-8_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29910-1

  • Online ISBN: 978-3-030-29911-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation