Rapid Learning of Pattern Sequences: A Novel Network Model

Abstract

This paper describes a neural architecture -- the KATAMIC memory, designed specifically to store sequences of binary patterns. The model integrates continuous learning with concurrent recall based on a step by step sequence predictions which allows memorized sequences to be recalled in response to cues -- short sub-sequences. A novel neuron-like computing element -- the PREDICTRON is introduced which learns to generate at each time step a prediction of the next input pattern based on the interaction of the memories of previously learned sequences (long term memory) and the recent states of the network (short term memory). A complete mathematical description of the KATAMIC algorithm is presented. The memory was implemented on the CM-2 Connection Machine and in *Lisp and results of some basic simulation studies are discussed.