Abstract
This paper presents a method to expand the basins of stable patterns in associative memory. It examines fully-connected associative memory geometrically and translate the learning process into an algebraic optimization procedure. It finds that locating all the patterns at certain stable corners of the neurons’ hypercube as far from the decision hyperplanes as possible can produce excellent error tolerance. It then devises a method based on this finding to develop the hyperplanes. This paper further shows that this method leads to the hairy model, or the deterministic analogue of the Gibb’s free energy model. Through simulations, it shows that this method gives better error tolerance than does the Hopfield model and the error-correction rule in both synchronous and asynchronous modes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- AM:
-
associative memory
- EAM:
-
expanded associative memory
- ECR:
-
error-correction rule
- LM:
-
Little model
- RK:
-
Runge-Kutta method
References
DH Ackley GE Hinton TJ. Sejnowski (1985) ArticleTitleA learning algorithm for Boltzmann machine Cognitive Science. 9 147–169 Occurrence Handle10.1016/S0364-0213(85)80012-4
SI. Amari (1972) ArticleTitleLearning patterns and pattern sequences by self-organising nets IEEE Transactions on Computers. 21 1197–1206 Occurrence Handle0245.94024 Occurrence Handle51 #2373
Boser B, Guyon I, Vapnik VN. (1992). A training algorithm for optimal margin classifiers. In: Fifth Annual Workshop on Computational Learning Theory, Morgan Kaufmann, pp. 144–152
J. Bruck (1990) ArticleTitleOn the convergence properties of the Hopfield model Proceeding of IEEE. 78 1579–1585
TM. Cover (1965) ArticleTitleGeometrical and statistical properties of systems of linear inequalities with applications in pattern recognition IEEE Transactions on Electronic Computers. 14 326–334 Occurrence Handle0152.18206
E. Gardner (1987) ArticleTitleMaximum storage capacity in neural networks Electrophysics Letters. 4 IssueID4 481–485
E. Gardner (1989) ArticleTitleOptimal basins of attraction in randomly sparse neural network models Journal of Physics. A 22 IssueID12 1969–1974 Occurrence Handle90f:82028
DO. Hebb (1949) The Organization of Behavior: A Neuropsychological Theory Wiley New York
JJ. Hopfield (1982) ArticleTitleNeural networks and physical systems with emergent collective computational ability Proceeding of the National Academy of Science. 79 2554–2558 Occurrence Handle83g:92024
Ince DC. (1992). Intelligent machinery. In: Ince DC. (eds). Collected Works of A. M. Turing: Mechanical Intelligence. Elsevier Science Publishers
I Kanter H. Sompolinsky (1987) ArticleTitleAssociative recall of memory without errors Physics Review. A 35 IssueID1 380–392
Kauffman SA (1991) Antichaos and adaptation. Scientific American: 64–70
J Li AN Michel W. Porod (1989) ArticleTitleAnalysis and synthesis of a class of neural networks: linear systems operating on a closed hypercube IEEE Transactions on Circuits and Systems. 36 IssueID11 1405–1422 Occurrence Handle10.1109/31.41297 Occurrence Handle90k:92015
Liou CY and Lin SL (1989) The other variant Boltzmann machine. In: Proceedings of the IJCNN, Washington DC, pp. 449–454
Liou CY and Sou UC. (2003) Loading temporal associative memory using the neuronic equation. In: Kaynak O, Alpaydin E, Oja E and Xu L (eds), LCS, vol. 2714, Springer, pp. 52–59
CY Liou JM. Wu (1996) ArticleTitleSelf-organization using Potts models Neural Networks. 9 IssueID4 671–684 Occurrence Handle10.1016/0893-6080(95)00111-5
Liou CY and Yu WJ (1995) Ambiguous binary representation in multilayer neural network. In: Proceedings of the ICNN, Perth, Australia, vol. 1, pp. 379–384
CY Liou SK. Yuan (1999) ArticleTitleError tolerant associative memory Biological Cybernetics. 81 331–342 Occurrence Handle10.1007/s004220050566
WA. Little (1974) ArticleTitleThe existence of persistent states in the brain Mathematical Biosciences. 19 101–120 Occurrence Handle10.1016/0025-5564(74)90031-5 Occurrence Handle0272.92011
RJ Mceliece EC Posner ER Rodemich SS. Venkatesh (1987) ArticleTitleThe capacity of the Hopfield associative memory IEEE Transaction on information Theory. 33 461–482 Occurrence Handle88h:92019
Szu H. (1989). Reconfigurable neural nets by energy convergence learning principle based on extended McCulloch–Pitts neurons and synapses. In: Proc IJCNN, Washington, DC, vol. 1, pp. 485–496
H. Szu (1999) ArticleTitleThermodynamics energy for both supervised and unsupervised learning neural nets at a constant temperature International Journal of Neural System. 9 175–186
Q Tao T Fang H. Qiao (2001) ArticleTitleA novel continuous-time neural network for realizing associative memory IEEE Transactions on Neural Networks. 12 IssueID2 418–423
Widrow B and Hoff ME Jr. (1960) Adaptive switching circuits. In: IRE WESCON Convention Record, pp. 96–104
PD. Wilde (1997) ArticleTitleThe magnitude of the diagonal elements in neural networks Neural Networks. 10 IssueID3 499–504 Occurrence Handle10.1016/S0893-6080(96)00094-9
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liou, CY., Lin, SL. Finite Memory Loading in Hairy Neurons. Nat Comput 5, 15–42 (2006). https://doi.org/10.1007/s11047-004-5490-x
Issue Date:
DOI: https://doi.org/10.1007/s11047-004-5490-x