Deriving Transformers from Knowledge Organized as a Society of Agents

Abstract

A learning program should be capable of organizing the knowledge that it acquires. This organization is necessary not only for the efficient retrieval of relevant knowledge but also for application of the knowledge to situations that are similar but not identical to the situation in which the knowledge was acquired. The “Society of Mind”[Minsky, 1988] provides a framework for organization of knowledge embodied as agents. In this paper, we describe a program that constructs transformers from closely related agents. A transformer abstracts the difference between two related agents. Since the transformer represents a property of the domain, it can be used under different circumstances to obtain a new agent from an existing one. These two agents are related in the same way as the two agents of the original pair. We also discuss how the same methods can be used to abstract differences between transformers to form transformer-transformers and so on. Finally, we examine the links that transformers have to other AI paradigms such as analogical reasoning, level-band theories of memory and generalization.