Reflections on dynamics, adaptation and control: A cognitive architecture for mental models

In this paper, an overview of the wide variety of occurrences of mental models in the literature is discussed. They are classified according to two dimensions obtaining four categories of them: static-dynamic and world-mental, where static refers to mental models for static world states or for static mental states and dynamic refers to mental models for world processes or for mental processes. In addition, distinctions are made for what can be done by mental models: they can, for example, be (1) used for internal simulation, they can be (2) adapted, and these processes can be (3) controlled. This leads to a global three-level cognitive architecture covering these three ways of handling mental models. It is discussed that in this cognitive architecture reflection principles play an important role to define the interactions between the different levels.


Introduction
Mental models are a kind of blueprints or pictures in the mind that can occur in various forms; e.g., (Craik, 1943;Evans, 2006;Furlough & Gillan, 2018;Norman, 1983;Halford, 1993;Johnson-Laird, 1983). One relatively simple example is that you perceive the world state in front of you and after closing your eyes you still see a picture of this world state in your mind. Another, more dynamic example is that you perceive an impressive course of events in front of you and after closing your eyes you see a kind of movie replay in your mind that replays this course of events. Although the notions of 'picture' or 'movie' provide an intuitive way to imagine what a mental model can be, for the general case such notions should not be taken literally but more in a metaphorical sense. For example, in a wider sense you can imagine a situation that you have never seen. Humans often use some form of mental model, as a blueprint or manual to handle situations. Well-known examples are operating a device or machine or software system, but also how to handle somebody else who needs to be handled based on some special personal 'user manual'. Still other examples are standard patterns learnt to solve certain types of problems in the context of certain disciplines, as so often are learnt at school.
All these examples show the wide variety of possibilities for mental models, usually described as structures consisting of collections or networks of certain relations that can be of various types. In this chapter this variety will be discussed, analysed and structured in some more detail in such a way that a basis is obtained for a cognitive architecture to handle mental models.

Mental models and what they model
In this section, part of the extensive literature on mental models is discussed and a structured overview is made based on distinguishing whether they consider an external world or an internal mental world and whether they model a static situation or a dynamic process.

Mental models as small-scale models within the head
For the history of the mental models area, often Kenneth Craik is mentioned as a central person. In his book (Craik, 1943) he describes a mental model as a small-scale model that is carried by an organism within its head as follows: 'If the organism carries a "small-scale model" of external reality and of its own possible actions within its head, it is able to try out various alternatives, conclude which is the best of them, react to future situations before they arise, utilise the knowledge of past events in dealing with the present and future, and in every way to react in a much fuller, safer, and more competent manner to the emergencies which face it.' (Craik, 1943, p. 61) 'the "modelling", by the brain, of the sequence of events whose consequence is sought, so that this model may predict the answer earlier than it occurs in the course of external nature. In other words, by the ability to model… the brain is able to outrun the physical processes which are too rapid for it and so can, on the average, forestall and anticipate the course of nature.' (Craik, 1966, p. 27).
Other authors also have formulated what mental models are. For example, with an emphasis on causal relations, Shih and Alessi (1993, p. 157) explain that 'By a mental model we mean a person's understanding of the environment. It can represent different states of the problem and the causal relationships among states.' De Kleer and Brown (1983) describe it in the following way: • the envisioning of the system, including a topological representation of the system components, the possible states of each of the components, and the structural relations between these components • the running or execution of the causal model based on basic operational rules and on general scientific principles.
Moreover, after an extensive analysis, Doyle and Ford (1998) formulate the following definition, where the focus is on a dynamic system: 'A mental model of a dynamic system is a relatively enduring and accessible but limited internal conceptual representation of an external system whose structure maintains the perceived structure of that system.' All these descriptions strongly focus on how the world functions based on certain dynamic or temporal causal relations (sometimes called the dynamic system view) and that in a mental model similar relations are used to simulate a similar process. This idea of running a simulation inside the head is also called internal simulation; e.g., (Damasio, 1994;Goldman, 2006;Hesslow, 2002;Hesslow, 2012). For example, in (Bhalwankar & Treur, 2020), the functioning of a car in interaction with its driver is internally simulated, and in  addressing PTSD, as a flashback experience a movie of a course of traumatic events is replayed in the brain.

Mental models for individual processes
In principle there are two ways in which a mental model can be considered to describe reasoning: to describe a reasoning state or to describe a reasoning process as a whole.

Describing reasoning states by mental models
For example, (Norman, 1983;Johnson-Laird, 1983;Halford, 1993) put an emphasis on the use of mental models to reasoning states. Here reasoning states are the snapshots of a reasoning process at specific time points (sometimes also called information states or knowledge states). Each of these time-dependent reasoning states is conceptualized by a mental model or by a set of mental models. Such mental models used to describe reasoning states have a slightly different appearance, compared to the mental models according to the perspective discussed above: • In Section 2.1 dynamics is described within the mental model: the mental model represents the dynamics by relations defining a dynamical system • In the current section the dynamics is not represented within a mental model: in contrast, a reasoning step is described as a transition step for mental models by which every step a new mental model is created.
So, based on this perspective where reasoning states are mental models, reasoning steps are considered transitions (maybe by standard generic inference rules, maybe based on other things) of one reasoning state to another one. For example, one reasoning state is described by a mental model including the two items a and a → b both describing a static world situation and by a reasoning step (based on modus ponens in this case), this mental model is transformed into a new mental model including the three items a, b and a → b all describing the same static world situation. Thus reasoning steps are conceptualised as adaptations of the mental models representing these reasoning states, which can also have the form that two or more mental models are used (as antecedents) in combination. In a reasoning process as a whole, these transitions are executed in succession, resulting in sequences of reasoning states (also called reasoning traces). These are then conceptualised by sequences of mental models over time. See (Johnson-Laird, 2004) for more history of this perspective. This view on dynamics of reasoning also has been addressed within AI for different types of reasoning from a formal logical and computational perspective; for example, see (Brazier, Treur, Wijngaards, Willems, 1999;Gavrila & Treur, 1994;Jonker & Treur, 2002;Jonker and Treur, 2003;Meyer and Treur, 2001;Treur, 1994a,b).

Describing a reasoning process by one mental model
A different perspective on conceptualising reasoning processes can be obtained from the dynamical system view, more closely related to the view discussed in Section 2.1. From this view not a reasoning state like above, but instead a reasoning process is described as one mental model by temporal (or causal) relations relating one reasoning state to another one. In other words, in this case a mental model describes the reasoning process by temporal relations defining a dynamical system, similar to how world processes can be modeled by a dynamical system mental model as discussed in Section 2.1. So, this time reasoning steps from one reasoning state to another one are not transitions between different mental models but are described by temporal relations within one fixed overall mental model. For example, such a mental model for a reasoning process can be described by relations of the form a → b (where this time → is interpreted as a temporal-causal relation) expressing that within a reasoning process knowing a will causally affect knowing b, like how, for example, in philosophy of mind, in general mental states are assumed to causally affect each other over time; e.g., (Kim, 1996). This view on reasoning provides a description of reasoning like any other mental process as a dynamical system as described in Section 2.1. This perspective on reasoning is considered within literature in AI such as (Engelfriet & Treur, 1994;Engelfriet & Treur, 1995;Meyer and Treur, 2001), where a 'temporal theory of reasoning' plays the role of a mental model according to a dynamical system view. So, in summary, in principle there are two different ways to model reasoning by mental models: (1) mental models model the reasoning states (as snapshots in a reasoning process), and reasoning steps are adaptations or transitions of these mental models over time, and (2) mental models model the dynamics of the reasoning process by modeling reasoning steps as temporal relations within one overall mental model for the entire reasoning process.

Mental models used to model cognitive metaphors
Cognitive metaphors can also be considered a type of mental model (Cardillo, Watson, Schmidt, Kranjec, & Chatterjee, 2012;Carroll & Thomas, 1982;Kuang, 2003;Leary, 1994;Ponterotto, 2000;Romero & Soria, 2005). Cognitive metaphors are a way to explain our conceptualization and mapping of new concepts on existing knowledge, and how we communicate this to others (Lakoff & Johnson, 2003) Or, in other words, one mental domain is understood in terms of a mental model of another phenomenon. Imagine encountering a novel animal: we will immediately compare its behaviors and looks to the bank of animals that exists in our brain, and try to map an understanding of this new animal, based on the knowledge we already have in our brain.
Another way to explain cognitive metaphors is as analogy making a mapping between a source and a target inside the brain (Gentner, 1983;Gentner & Stevens, 2002;Vosniadou and Ortony, 1989), based on features the source and domain have in common. For example, we often hear the catchphrase 'Love is a Journey'. Of course, literally, love is not a journey, but rather an abstract concept. However, because of the complexity of love as a concept, we use more concrete concepts, like a journey, to understand and communicate our understanding of love. By using the journey metaphor, we unconsciously map concepts like roadblocks and the fact that a journey is something to embark on onto our concept of love, and thus come closer to a (mutual) understanding. Lakoff (1993) stresses that metaphors are an essential mechanism that is systematically mapped in our brain for humans to understand the world, and be able to think and reason, without us even noticing. Furthermore, bodily changes can unconsciously affect our metaphorical thoughts, see (Barsalou, 2008;Landau, Meier, Keefer, 2010;Williams, Huang, Bargh, 2009). Even more so, our mental models can be influenced by the metaphors we use, as constant repetition of particular metaphors will lead to our unconscious acceptance of that particular metaphor as a normal way of seeing that situation El Refaie (2003). Thereby, a metaphor subconsciously constructs how we perceive situations; see (Barsalou, 2008;Landau et al., 2010;Williams et al., 2009). Several studies have shown that our actions are subconsciously influenced by the automated activation of motives (Bargh, Gollwitzer, Lee-Chai, Barndollar, Trötschel, 2001;Bargh & Morsella, 2008). Therefore, through this route cognitive metaphors also affect the way humans make decisions. As an example, in (Van Ments & Treur, 2021b) metaphors for cooperative and competitive joint decision making are modeled as a second-order adaptive mental model.

Mental models in social processes
In this section the focus is on mental models used in a social context. These can concern mental models for bonding and attachment in dyadic relationships or mental models for groups, teams or organisations. A wellknown social type of mental model occurs when one has some 'image' of the mental state of another person, or of oneself. If the dynamics of mental processes are also considered, one can, for example, have a mental model of how your partner will get angry or disappointed after you undertake some specific action. Also more in general in social life, humans often use some mental model to understand each other and interact in an adequate manner based on that, for example, to get something done. And the same even applies to having a user manual for handling oneself. In this section, in particular mental models for attachment in dyadic relationships are briefly discussed, and mental God-models, mental models for bonding by homophily, and team mental models.

Mental models for attachment
The way an individual forms relationships with others can be explained by the Attachment Theory, constructed by Ainsworth and Bowlby. This theory is based on its predecessor, the 'Security Theory', developed by William Blatz and Mary Salter Ainsworth; e.g., (Blatz, 1966;Salter, 1940;Salter Ainsworth, 2010;Salter Ainsworth & Bowlby, 1965). The attachment theory explains how a child develops a set of emotions, memories, thoughts, expectations, behaviours and beliefs about itself and others, based on its early experiences with its primary caregiver. This set is called the 'internal working model of social relationships', which continues to change with age and experience (Mercer, 2006). More specifically, this 'model of self' and 'model of other' that the child initially develops, is based on experiences with the primary caregiver and their behaviour (Bretherton, 1992). Using their internal 'model of other' a child can predict the primary caregiver's behaviour, and using their internal 'model of self', they can plan their own behaviour accordingly (Bretherton, 1992), and the same happens later in life in interaction with significant others. In (Hermans, Muhammad, Treur, 2021) a second-order adaptive network model is presented for development of mental models of self and others according to attachment theory.

Mental God-models
Another interesting place we can find mental models is a person's relationship with God. When a person prays, the same brain regions that are used for interactions with other people become activated, enabling a person to generate an internal representation of 'the other', in this case the image they have of God. This allows people to form a real, meaningful relationship with God, and to construct a mental model of an image of God. (Schjoedt, Stodkilde-Jorgensen, Geerts, & Roepstorff, 2009). This mental God-model that an individual has of God, and how this image has impact on the individual, can involve many aspects. For example, the attachment style discussed in the previous section can be studied in combination with a person's God-model, and how these two influence each other. (Granqvist & Kirkpatrick, 2008). The relationship and mental image of God can also be explained from a mental model or mentalizing perspective, as introduced by (Schaap- Jonker & Corveleyn, 2014). Mentalizing is the capacity of thinking about thinking and feeling. It provides awareness that one's own and others' behaviour is driven by mental states, and gives the ability to selectively activate internal states that fit the individual's particular. Mentalizing also involves a process of internal simulation, where an individual internally simulates mind states to predict effects in the external world or other persons. In other words, a mental model is an interesting way to describe an individual's relationship with God. In (Van Ments, Treur, Roelofsma, 2021) an adaptive network model for developing and using a mental God-model is described.

Mental models for bonding based on homophily
Social networks often are adaptive, for example based on a bonding by a homophily principle for the adaptation of the weights of the network connections between persons over time. A bonding by homophily adaptation principle expresses how 'being alike' strengthens the connection between two persons, also explained as 'birds of a feather flock together'; e.g., (McPherson, Smith-Lovin, & Cook, 2001). Usually, in literature such adaptation processes are considered without taking into account subjective elements for the persons involved. For example, do the persons themselves actually know in how far they are alike? Or are they just willless victims of objective social laws independent of what they know or what they want? Such subjective aspects are often lacking in (computational) research on bonding by homophily, as usually these processes are addressed exclusively from the perspective of an assumed objective social world. However, a more realistic bonding by homophily principle can be obtained if the bonding is not assumed to be based on an objective form of homophily but on the mental models both persons have of each other. If two persons both have a mental models of themselves and the other that show that they are alike, then that will clearly affect their bonding, even if these mental models are not correct and, for example, based on fake information. This subjective mental model based perspective on bonding by homophily is addressed in .

Team mental models
A team mental model is based on the assumption that high performing teams need to have team members that are on the same page in order to perform complex tasks well; e.g., (Burtscher & Manser, 2012;Langan-Fox, Code, Langfield-Smith, 2000;Mohammed, Ferzandi, & Hamilton, 2010). This requires team members to have a shared understanding of the relevant elements to perform a specific task. A team mental model is an emergent team level concept which is generated by each team member's cognition up to the level that it becomes a shared mental model: so, the origin and basis of a team mental model is formed by the individual team members. More specifically, the team mental model itself is an emerging collective phenomenon which is created bottom-up from each team member's cognition in a dynamic manner (DeChurch and Mesmer-Magnus 2010a,b). The main functions of team mental models are improved planning, coordination and alignment (Nini, 2019). Two types of team mental models are distinguished (Mohammed et al., 2010): • task-related team mental models • team-related team mental models The first type provides a team's cognitive representation of taskrelated elements such as goals and subtasks, subtask dependencies, subtask durations, milestones, and resources required for task coordination. The second type covers the team's mental model for the knowledge, skills, competencies and relationships of team members. In (Van Ments, Treur, Klein, Roelofsma, 2021) an example of an adaptive network model for handling a shared mental model in a medical context is presented.

A mental models overview according to mental vs world and static vs dynamic
In the above Sections 2.1-2.3, mental models have been described as consisting of a collection or network of relations. In some cases these relations describe static relationships for a world situation or state (such as 'Joe is taller than Kamala') or for a mental state (such as 'Joe does not believe in complot theories'). In other cases, these relations describe temporal or causal relationships according to a dynamic system view of a world process (e.g., 'human action causes climate change') or a reasoning process (e.g., 'because I believe I have no time left, I now decide to do this action'). In all such examples, for what is represented by a mental model, two dimensions of variation can be recognized. The first dimension is the dimension static-dynamic, where static refers to representing static situations, and dynamic to representing a process. The second dimension is the dimension world-mental where world refers to the external world and mental to mental states or processes. Distinctions according to these dimensions have been used in Table 1 to get a structured overview of the options.
Note that this table is not the end of the story, as several important aspects that occur in relation to mental models are not covered yet. As mental models are usually described as networks of certain types of relations, one characteristic of mental models that also varies is which types of relation are used exactly. Causal relations are often used, especially from a dynamic system view, but also other types of relations often occur in mental models. In addition, also relations of higher-order, as used among others, in analogical reasoning have not been distinguished yet. Moreover, the adaptation of mental models as takes place in learning or development still has to be addressed, and the same holds for the control over such adaptation. These topics will be addressed in next sections.

Table 1
The variety of mental models structured for what is modeled according to state vs process and world vs mental; this provides a summary of the concepts discussed in more detail in the text of Section 2.

World vs mental
Example mental models

Process description
World process ▪ a mental model of a dynamical system for world dynamics ▪ a mental model of a how the water level changes with tide ▪ a mental model of how the climate of the earth changes due to human action ▪ a step-by-step description of a route to follow to get from A to B in a city; e.g., 'when you reach the cinema on your right hand, turn left and get into that street to the supermarket' Mental process ▪ a mental model of how your partner will get angry or disappointed after you undertake a specific action ▪ a mental model of how you yourself will get angry or disappointed after your partner undertakes a specific action ▪ a step-by-step algorithm to calculate the area of a rectangle or a long division State description World state ▪ a mental model of a city in the form of a map; e.g., 'the supermarket is in the street opposite the cinema' ▪ a mental model of the current climate in different regions ▪ a mental model of a rectangle Mental state ▪ a mental model of beliefs someone else has on the world ▪ a mental model of desires or goals someone else has ▪ a mental model of the emotions someone else has ▪ a mental model of the knowledge and skills someone else has ▪ a mental model of any of the above for yourself instead of 'someone else'

Learning and development as adaptation of mental models
Within educational science, sometimes the term model-based learning is used for learning described as constructing coherent mental models; for example, Buckley (2000) formulates this as: 'Model-based learning is a dynamic, recursive process of learning by building mental models. It incorporates the formation, testing, and subsequent reinforcement, revision, or rejection of mental models of some phenomenon.' However, note that in most cases that mental models are considered for learning, the term model-based learning is not explicitly used. This view on learning was also described by Piaget. Although he did not use the term mental model, the ideas he put forward do apply to mental models. Within the literature also the term schema or schemata is often used; this concept has no sharp boundary with the concept mental model and both concepts have much in common. Following the ideas of Piaget (1936Piaget ( , 1954, formation and adaptation of mental models during learning or development can occur in two forms: by assimilation (extension or refinement of a mental model) or by accommodation (revision of a mental model). As an example, suppose that a mental model includes the relation need something → go to shopping area By assimilation, this can be refined into a mental model including the following relations: need something → go to shopping area need book → need something need book & in shopping area → look for book shop This is a refinement and not a revision, as the previous relation still applies. In contrast, accommodation takes place, for example, when due to a lockdown the shops are closed for a long time. Then the mental model including need something → go to shopping area can be revised into a mental model including need something → go to webshop This is indeed a revision and not a refinement as the previous relation does not apply anymore. Such types of examples illustrate how mental models can change over time due to learning or development, as also described by the quote above from (Buckley, 2000). Next, some elements of learning processes are addressed in more detail and the importance of control over the learning is discussed.

Learning of mental models by observation and by instruction
Observational learning indicates when observation is important for the learning or development of a mental model. This can be observation of others but also observation of oneself while 'learning by doing' or 'learning by discovery'. Learners may see someone perform a t type of behavior and then start to imitate it; e.g., (Benbassat, 2014, Yi & Davis, 2003. This is often used to make others learn a specific motor task. A mechanism based on mirror neurons underly the ability to learn by observing and imitating others; e.g., (Hurley, 2008;Rizzolatti & Craighero, 2004;Van Gog et al., 2009). An example of an adaptive network model for learning by observation a mental model of how a car works and how to drive it can be found in (Bhalwankar & Treur, 2020). Another example showing how a mental model is learned by counterfactual thinking and observation can be found in (Bhalwankar and Treur, 2021b).
Instructional learning describes how information provided by an expert instructor can be an important source for the learning. Only learning based on observation often may lead to processes of trial and error; e.g., (Seel, 2006). Instructions from an expert are a useful addition to develop mental models in an effective manner. A format of scaffolded model-based learning in which many supporting actions such as prompts, questions, hints, stories, conceptual models, visualizations are performed, facilitates a learner's progress; e.g., (Hogan & Pressley, 1997). An example of an adaptive network model for learning by instruction a mental model of how a car works and how to drive it can be found in (Bhalwankar & Treur, 2020).

Control for learning of mental models based on metacognition
To handle mental models and in particular the learning of them, control is important; e.g., Gibbons and Gray (2002) claim that instructions are most effective for learning processes when the learner controls them. The already mentioned scaffolded model-based learning format in the previous section supports this (Hogan & Pressley, 1997). As another example, Kozma (1991) claims that persons actively pick external sources for mental model learning. So, the learner's initiatives for instruction and information acquisition are important for mental model learning. The learner has (to be able) to be proactive and in control of the learning. As yet another example, Meela and Yuenyong (2019) have shown that Model-Based Inquiry (MBI) can support a student's mental model formation in scientific learning; see also (Neilson, Campbell, Allred, 2010).
An example of an adaptive network model for controlled learning of a mental model of how a car works and how to drive it can be found in (Bhalwankar and Treur, 2021a).
Metacognition is described in (Darling-Hammond, Austin, Cheung, and Martin, 2008;Mahdavi, 2014;Flavell, 1979;Koriat, 2007;Pintrich, 2000) as cognition about cognition. More specifically, (Koriat, 2007) presents it as what people know about their own cognitive processes and how they put that knowledge to use in regulating their cognitive processing and behavior. Sometimes the term self-regulation and selfregulated learning are used. In (Pintrich, 2000), this is formulated as an active, constructive process whereby learners set goals for their learning and then monitor, regulate, and control their cognition, motivation, and behavior, guided by these goals. Also in learning complex tasks using mental models, control is a crucial element; see  for an example network model for this.
In learning, often different mental models play a role; e.g., (Norman, 1983;Greca & Moreira, 2000;Skemp, 1971;Seel, 2006). An example can be the learning of subtracting numbers. The learner can use a more visual model, drawing out the numbers on a line, or a more abstract model, using formulas to represent the subtraction e.g., (Bruner, 1966;Du Plooy, 2016). Here, metacognition plays an important role for the decisions about when to switch from one mental model to another one. In (Treur, 2021a) more can be found on this case, particularly for learning arithmetic or algebraic skills in primary or secondary schools supported by visualisation.

A cognitive architecture for mental models
In this section several aspects of mental models are discussed that are important to obtain a cognitive architecture to handle mental models. In particular, the following aspects are addressed: • higher-order relations in mental models • adaptation of mental models • control of adaptation of mental models Finally, it will be pointed out how an overall cognitive architecture can be designed covering these aspects.

Higher-order relations
Higher-order relations are relations between relations. In Fig. 2 an example is depicted of a first-order relation R and a second-order relation T. In this example, this second-order relation T expresses that the first-order relation R is transitive. Below the dashed purple line, a firstorder mental model is depicted based on relation R. Above this dashed purple line a second-order mental model is depicted based on transitivity relation T.
In the first-order self-model, the relation R is a relation for objects X , Y and Z, where, for example, R denotes the relation 'is taller than'. Linguistically or logically, such a first-order relation can also be expressed as X:Y or X: R Y or X R Y or R(X, Y). In the second-order mental model, the relation T is also between certain objects, but this time the objects are indicated by terms r(X,Y), r(Y,Z), and r(X,Z) which are names for the relation instances X → R Y, Y → R Z, and X → R Z, respectively, one level lower. These objects can be considered reifications of the relation instances represented in the first-order mental model: they are now represented by objects like r(X,Y) that refer to relational expression R(X, Y); e.g., see (Galton, 2006). This is similar to, for example, how Gödel used a representation of logical statements by natural numbers to obtain his famous incompleteness theorems for mathematical logic; e.g., see (Hofstadter, 1979;Nagel & Newman, 1965;Smorynski, 1977). The upward and downward relations between the two levels can be described by so-called reflection principles; see also Section 4.3 below and (Treur, 1991Treur, 1994aWeyhrauch, 1980). Another example of a second-order relation in a slightly different notation is the relation A:B::C:D where the symbol : denotes the firstorder relation and the symbol :: denotes a second-order relation between the two first-order relational expressions A:B and C:D. This is often used in experiments concerning analogical inference as also discussed in Section 4.2; e.g., (Alfred, Connolly, Cetron, et al., 2020;Holyoak and Monti, 2020;Whitaker, Vendetti, Wendelken, Bunge, 2018). For such a second-order relation A:B::C:D, a picture similar to Fig. 2 can be drawn. In principle, also third-and higher-order relations may be possible; the use of third-order relations for control is discussed in Section 4.3.
The above shows that in addition to the distinctions made in Table 1, also a distinction between mental models according to the orders of the relations they use can be made, where in one mental process multiple mental models of different orders may be used in an integrative manner.

What exactly do mental models do?
Next, distinctions are made for what mental models actually do. In different sections, different types of processes were encountered that in one way or the other relate to mental models. The following overview of these processes can be made.

• Simulation: Mental Models Simulate
As discussed in Sections 2 and 4, mental models are often used for a form of inferencing or internal or mental simulation to relate known facts to unknown facts about world or mental states or processes. This occurs in many forms, varying from prediction, visualisation in sport, flashback movies in PTSD, dreaming, reasoning and many more cases.

• Adaptation: Mental Models Adapt
Mental models often are adapted; they can be formed or learned and they can be revised, as Piaget (1936Piaget ( , 1954 already pointed out. This has been discussed in some detail in Section 3, thereby addressing observational learning and instructional learning in particular.

• Control: Mental Models Respond to Control
Using mental models and adapting them is in principle done in a coordinated manner by some form of control by a form of metacognition. This also has been discussed in some detail in Section 3 in particular for the timing of observational learning and instructional learning.
This shows that in addition to the distinctions made in Table 1 and in Section 4.1, also distinctions have to be made between what mental models actually do, where in one mental process often multiple mental models of different levels will be used in interaction with each other. In Section 4.3, it is pointed out how a cognitive architecture for this may be obtained.

A cognitive architecture for handling mental models
Based on the different processes in which mental models are used as summarised in Section 4.1 and 4.2, it can be assumed that a cognitive architecture for handling mental models has to cover the following three types of processes in an integrated manner (see also Fig. 3): Level 1: Use (base level) This level covers mental models described by relations that can be used to generate internal simulation.
Level 2: Adaptivity (first-order adaptation level) This level covers adaptation of Level 1 mental models by learning, revision, or other change; this can be described by a mental model using relations for changing the relations of the mental models at Level 1. In principle, this will involve a mental model with relations of one order higher than the relations used at Level 1.

Level 3: Control (second-order adaptation level)
This level covers control of adaptation processes described by a mental model using relations for changing the relations used at Level 2 for change of the Level 1 mental models. In principle, this will involve relations of one order higher than the relations used at Level 2, and two orders higher than the relations used at Level 1.
Here the second and third level are higher-order levels (involving higher-order relations; see Section 4.1) compared to the first level. This architecture was inspired by literature on metalevel architectures and reflection such as (Bowen & Kowalski, 1982;Bowen, 1985;Galton, 2006;Sterling & Beer, 1989;Treur, 1991Treur, 1994aWeyhrauch, 1980). To illustrate the levels in Fig. 3 and their relations by an abstract miniexample, assume at the three levels 1-3 relations R, S and T (denoted by → R , → S , → T , respectively) are used as shown in Table 2 (columns 2-4) and Fig. 4; here V, W, X (column 5) model some contextual or situational factors. These relations may be causal relations, but they can also be of any other type of relation. As also mentioned in Section 4.1, an important notion to describe the interaction between the different levels of such an architecture is the notion of reflection principle (Treur, 1991;Treur, 1994a,b;Weyhrauch, 1980); this type of principle (see also column 6 in Table 2) will also be explained below by the mini-example.
The explanation of this mini-example is as follows. At the base level the mental model includes an instance of relation R from X to Y, represented as This relation R is usually called a first-order relation. By an upward reflection principle from level 1 to level 2, at the second level (for adaptation) this R-relation instance relates to an object denoted by the term r(X, Y) referring to relation X → R Y; so, r(X, Y) is a name to refer to relation instance X → R Y (alternatively, sometimes the notation 'X → R Y' is used for such a name). For this object at level 2, in turn an instance of relation S applies that relates the object r(X, Y) to context factor W: This relation S is usually called a second-order relation. By a downward reflection principle from level 2 to level 1, this makes firstorder relation R adaptive, as via the relation W → S r(X, Y) the object r (X, Y) representing X → R Y depends on circumstances modeled by context factor W and by the downward reflection principle, this affects the relation instance X → R Y at level 1 accordingly. Note that for this cognitive architecture, this is called first-order adaptation, as it concerns adaptation of the first-order relation. But note that the term used in the literature for the relation S involved is second-order relation.
However, also the second-order relation S is adaptive, because similarly by an upward reflection principle from level 2 to level 3 it relates to an object denoted by the term s(W, r(X, Y)) at level 3 referring to relation W → S r(X, Y), and this object also depends on circumstances (modeled by context factor V), as at level 3 a thirdorder relation T is applied: V → T s(W, r(X, Y)) Therefore, s(W, r(X, Y)) depends on context factor V and by a downward reflection principle from level 3 to level 2, this affects Srelation instance W → S r(X, Y) at level 2 accordingly. As second-order relation S models the first-order adaptation of first-order relation R, by this control over the first-order adaptation can be exerted. In summary, second-order relation S models adaptation of first-order relation R using context factor W, whereas third-order relation T models control of this adaptation using context factor V. Note that for this cognitive architecture, this is called second-order adaptation, as it concerns adaptation of the second-order relation. But the term used in the literature for the relation T involved is third-order relation.
This simple example illustrates how the adaptation of a mental model and its control can be modeled, and it points out how reflection principles can connect the levels and enable the transfer between the levels.
This structure of three levels for handling mental models can be used in conjunction with the structure of Table 1 in Section 2 to obtain an overview of the many possible occurrences and uses of mental models. Note that due to the relationships between the different levels explained above where objects at each higher level refer to relations at the next lower level, the higher levels can be interpreted as self-models of part of the architecture itself, namely self-models of the part at the next lower level. In this sense it can be considered a self-modeling architecture.

Discussion
In this chapter, an overview of the wide variety of occurrences of mental models in the literature was discussed. They were classified according to two dimensions obtaining four categories of them: staticdynamic and world-mental, where static refers to mental models for static world states or for static mental states and dynamic refers to mental models for world processes or for mental processes. In addition, distinctions were made for what can be done by mental models: they can, for example, be (1) used for internal simulation, they can be (2) adapted, and these processes can be (3) controlled. This has led to a global three-level cognitive architecture covering these three ways of handling mental models. It has been pointed out that in this cognitive architecture reflection principles play an important role to define the interactions between the different levels. These can be modeled in more detail using the adaptive network-oriented modeling approach described in (Treur, 2016;Treur, 2020).

Fig. 3.
Cognitive architecture for mental model handling with three levels of mental processing for mental models where each next level is modeled by relations one order higher than at the level below it.

Table 2
Overview of the mini-example for the three levels. Some more philosophically focused background for mental models and their modeling can be found in (Treur, 2021d) about neural correlates for mental models and (Treur, 2021e) about the emerging informational content of mental models.