Skip to main content
SpringerLink
Log in

Pattern Recognition in Nonlinear Neural Networks

  • Conference paper
The Physics of Structure Formation

Part of the book series: Springer Series in Synergetics ((SSSYN,volume 37))

Abstract

In this paper some new techniques to analyze nonlinear neural networks are reviewed. A neural network is called nonlinear if the introduction of new data into the synaptic efficacies has to be performed through a non-linear operation. The original Hopfield model is linear whereas, for instance, clipped synapses constitute a nonlinear model. We examine the statistical mechanics of a nonlinear neural network with finitely many patterns and arbitrary synaptic kernel, study the information retrieval, and show how the abundantly present spurious states which are a consequence of the nonlinearity can be eliminated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. J. Hopfield, Proc. Natl. Acad. Sci. U.S.A. 79 (1982) 2554

    Article  MathSciNet  ADS  Google Scholar 

  2. See D. J. Amit, these proceedings, for details

    Google Scholar 

  3. W. C. McCulloch and W. Pitts, Bull. Math. Biophys. 5 (1943) 115

    Article  MathSciNet  MATH  Google Scholar 

  4. P. Peretto, Biol. Cybernet. 50 (1984) 51

    Article  MATH  Google Scholar 

  5. R. G. Palmer, Adv. Phys. 31 (1982)669

    Article  ADS  Google Scholar 

  6. A. C. D. van Enter and J. L. van Hemmen, Phys. Rev. A 29 (1984) 355

    ADS  Google Scholar 

  7. D. O. Hebb, The Organization of Behavior (Wiley, New York, 1949)

    Google Scholar 

  8. J. L. van Hemmen and R. Kühn, Phys. Rev. Lett. 57 (1986) 913

    Article  ADS  Google Scholar 

  9. H. Sompolinsky, Phys. Rev. A 34 (1986) 2571

    ADS  Google Scholar 

  10. J. L. van Hemmen, D. Grensing, A. Huber and R. Kühn, Nonlinear neural networks: I. General theory, II. Information processing, to be published.

    Google Scholar 

  11. D. Grensing, R. Kühn and J. L. van Hemmen, J. Phys. A.: Math. Gen. 20 (1987)

    Google Scholar 

  12. J. L. van Hemmen, Phys. Rev. Lett. 49 (1982) 409

    Article  MathSciNet  ADS  Google Scholar 

  13. J. L. van Hemmen, D. Grensing, A. Huber and R. Kühn, Z. Phys. B 65 (1986) 53

    Article  ADS  Google Scholar 

  14. J. Lamperti, Probability (Benjamin, New York, 1966) Sec. 7. More generally, one needs ergodicity.

    MATH  Google Scholar 

  15. D. Grensing and R. Kühn, J. Phys. A: Math. Gen 19 (1986) L 1153

    Article  ADS  Google Scholar 

  16. J. L. van Hemmen, in: Heidelberg Colloquium on Spin Glasses, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics 192 (Springer, Berlin, 1983) pp. 203–233, in particular the Appendix.

    Google Scholar 

  17. The intuitive justification of the Laplace argument is clear. For a proof, see: N. G. de Bruyn, Asymptotic Methods in Analysis, 2nd Ed. (North-Holland, Amsterdam, 1961) Sec. 4.2

    Google Scholar 

  18. E. T. Copson, Asymptotic Expansions (Cambridge University Press, Cambridge, 1965) Chap. 5

    Book  MATH  Google Scholar 

  19. J. L. van Hemmen, Phys. Rev. A 34 (1986) 3435

    ADS  Google Scholar 

  20. This convention differs from the one in Ref. 8. There λ denotes an eigenvalue of QP where P = diag (pγ).

    Google Scholar 

  21. G. Iooss and D. D. Joseph, Elementary Stability and Bifurcation Theory (Springer, Berlin, 1980)

    MATH  Google Scholar 

  22. J. J. Hopfield, in: Modelling and Analysis in Biomedicine, edited by C. Nicolini (World Scientific, Singapore, 1984) pp. 369–389, in particular p. 381

    Google Scholar 

  23. G. Parisi, J. Phys. A: Math. Gen. 19 (1986) L 617

    ADS  Google Scholar 

  24. J. P. Nadal, G. Toulouse, J.-P. Changeux, and S. Dehaene, Europhys. Lett. 1 (1986) 535

    Article  ADS  Google Scholar 

  25. D. J. Amit, H. Gutfreund, and H. Sompolinsky, Phys. Rev. A 32 (1985) 1007

    MathSciNet  ADS  Google Scholar 

  26. K. L. Chung, Elementary Probability Theory and Stochastic Processes, 2nd Ed. (Springer, Berlin 1975) Eqs. (7.3.5) - (7.3.7) on pp. 211–212.

    Google Scholar 

  27. Since sgn (x) is odd, λρ vanishes for | ρ | even; cf. (3.6). About half of the remaining eigenvalues is positive.

    Google Scholar 

  28. See Ref. 10. An upper bound is provided by q-1.

    Google Scholar 

  29. J.-P. Changeux, S. Dehaene and G. Toulouse, Proc. Natl. Acad. Sci. U.S.A. 83 (1986) 1695

    MathSciNet  Google Scholar 

  30. J. L. van Hemmen and K. Rzążewski, to be published

    Google Scholar 

  31. J. L. van Hemmen and A. C. D. van Enter, Phys. Rev. A 34 (1986) 2509

    Article  Google Scholar 

  32. J. L. van Hemmen, A. C. D. van Enter, and J. Canisius, Z. Phys. B 50 (1983) 311

    Article  MathSciNet  Google Scholar 

  33. Interestingly, states which are products of two patterns bifurcate first. This allows logical operations such as EQUIVALENCE.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sonderforschungsbereich 123, Universität Heidelberg, D-6900, Heidelberg, Fed. Rep. of Germany

    J. L. van Hemmen

Authors
  1. J. L. van Hemmen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut für Informationsverarbeitung, Universität Tübingen, D-7400, Tübingen, Fed. Rep. of Germany

    Werner Güttinger  & Gerhard Dangelmayr  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1987 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

van Hemmen, J.L. (1987). Pattern Recognition in Nonlinear Neural Networks. In: Güttinger, W., Dangelmayr, G. (eds) The Physics of Structure Formation. Springer Series in Synergetics, vol 37. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-73001-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-73001-6_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-73003-0

  • Online ISBN: 978-3-642-73001-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation