Agent-based modelling – a new method for investigating environmental problems

The agent-based modelling (ABM) represents a newly emerged domain with multiple applications. They started to develop in late 1940’s and they reached maturity in 2000’s – 2010’s. Nowadays, more and more studies are being made using ABM, proving that they are a powerful tool in exploring patterns, interactions and behaviours. As their name suggest, they rely on agents. There is still no formal definition for agents, but they are considered to be autonomous entities, capable of having a certain behaviour and to interact with other agents and/or with the environment they belong to. The environment, the agents and the interactions established form the ABM. Even though this type of models has evolved from mathematics, thanks to computer development, and especially computer graphics, they now offer the possibility to visually explore certain behaviours and to identify a pattern. Also, of great importance and contributing directly to the ABMs application expansion, is the development of a variety of toolkits used for designing agent-based models. Many of the toolkits are open source and continuously implement more simplified programming language. These are the main reasons for which ABMs are used nowadays in a very wide range of domains – from ecology to healthcare and medicine, from archaeology to stock markets, from combats and air traffic control to emergency situations, from social and natural sciences to robotics.


INTRODUCTION
An agent-based model (ABM) is a computational simulation in which different agents act and interact, having a certain impact on the whole system they belong to. The name 'agents' is given to any entityindividual or collectivecapable of acting in a certain way, thus having a certain behaviour. The agents are capable of decisionmaking, their actions and interactions with other agents and/or the environment being limited by a set of rules previously established. As mentioned, 'agent' is a generic term used to define entities that can interact with other entities and/or with the environment. Agents can be people, plants, animals, vehicles, institutions, organizations, different groups of people, departments, companies, etc. There is no formal definition for the agents (Orsi, 2018a), but they do have some specific characteristics like (i) autonomy, in decisions and in acting, (ii) heterogeneity, even though they belong to a certain group, every single agent is unique, (iii) agents can learn and adapt, (iv) they can interact with other agents and/or the environment and (v) they can be mobile.
The intelligence level of the agents, translated in their behaviour, depends on the power of computers and on mathematical models (Bonabeau, 2002).
ABM is capable of 'translating' in a visual way the mathematical equations that govern certain behaviours, for example, people moving in a museum, birds flocking through cities, bears avoiding humans in touristic mountainous areas etc. ABM is used to make predictions about possible attitudes given some rules and to establish patterns in different contexts. ABM can help in identifying the connections between two or more agents or the interactions between agents and/or the environment which they belong to. It can be used in a wide range of fields like social and natural sciences, resource allocation, location of facilities, route planning, public services, military and public policies. When the focus of the analysis is on the interactions and/or dependencies between agents, ABM will provide a suitable insight of the analysed phenomenon. For this reason, it is widely used in ecology, biology and agriculture.

DEVELOPMENT AND TOOLKITS
The early development of the ABM dates back in the period between 1940's and 1960's, when the first idea of the concept emergedthe von Neumann's machine. The machine was designed by John von Neumann and it was named 'the universal constructor' because of its designed capability to replicate itself. Von Neumann designed it to have three main components: (i) a memory tape, where the indications for building the machine were stored, (ii) the constructor, that read the memory tape and was able to interpret it and (iii) a construction arm, that built the machine, according to the instructions (von Neumann, John, Burks, 1966;Orsi, 2018a). The universal constructor was based on cellular automata, an invention of a Polish-American mathematician, Stanislaw Marcin Ulam. Cellular automaton is a collection of cells with a certain property organised in a grid and which evolves through a number of discreet time steps, according to a set of rules, the cells being capable of adaptation. In the von Neumann machine, the cells could take on 29 different states at any moment and they could update their status depending on their neighbour's state (Orsi, 2018a). The work of von Neumann was continued and extended over the years. In 1966, John Burks published a book based on the work and designs of John von Neumann after the latest's death in 1957.
Another milestone in ABMs development was another cellular automaton designed by John Horton Conway, a British mathematician, in 1970, called Game of Life. Game of Life is an infinite, 2D grid, consisting of square cells with only two possible statesdead (unpopulated) or alive (populated). The cells can interact with all eight surrounding neighbours. The rules set were (Orsi, 2018a): (i) any live cell that has less than two neighbours dies (by underpopulation), (ii) any live cell that has two or three neighbours survives (the next generation), (iii) any live cell that has more than three living neighbours dies (by overpopulation) and (iv) any dead cell that has three neighbours gets back to life (by reproduction). The first generation of cells are considered seeds and they will apply the four rules, resulting in new generations that are identical to the previous ones (Gardner, 1970). The breakthrough of Life consisted in the way that patterns can emerge, and was of great interest for the scientists of the time from different fields like physics, mathematics, biochemistry, philosophy, computer science etc. Martin Gardner, in his article from Scientific American from October 1970, said that this game belongs to a growing class called 'simulation games' (Gardner, 1970). The game was developed before the wide use of computers, but for many years it presented a lot of interest and so it was in the public's attention when the first generation of inexpensive computers appeared on the market. This resulted in implementing the game on computers that could run the model for multiple times. By performing the game, some patterns could be established. Thus, the main patterns identified were still (Cook, 2003), oscillators and spaceships (Adamatzki, 2010), the former being observed before the use of computers.
In 1971, Thomas Schelling published a paper called 'Dynamic Models of Segregation' in which appeared one of the earliest concepts for ABM. Schelling was an economist and a professor at the University of Maryland, his main field being public policy (arms control, national security and foreign policy). His background in social-related domains allowed him to draw into attention the mechanisms of segregation. In his paper, mentioned above, he states that 'This paper, then, is about those mechanisms that translate unorganized individual behaviour into collective results' (Schelling, 1971, p. 145), the unorganized individuals are actually agents and the collective results are the system, both elements of the agent-based modelling. The models explored in his work are the spatial proximity model and the bounded-neighbourhood model, using as agents' stars and zeros. In the first one, the entities perceive the 'neighbourhood' based on their location, in the second one, the neighbourhood is defined and has boundaries. First, he assumes a linear distribution of 70 entities, represented graphically by 35 stars and 35 zeros. In a first assumption the entities are happy to live together in a ratio of approximately 50:50. By adding more ruleslike the perception of the extend of the neighbourhood, the preference that a certain number of neighbours to be the same, the percentage of each entity in the total population, the direction and length of movement, the entities move on the line. Even though none of the entities know a priori the next movement of others, following the rules, will result in cluster formation and then in segregation. Introducing a movement restriction, some concerted and anticipatory movement can be identified in the actions of the entities. Another distribution explored in the paper is the areal one. It starts with a finite board in which two entities are randomly distributed. The starting rule is that the entities can move only on vacant cells, and by moving they create a new vacant cell that can be occupied, every entity being surrounded by eight neighbours. By introducing a new ruleif an entity is surrounded by certain percentage of entities belonging to other group, the entity will move to another location. The results showed that even in case of high tolerance, segregation patterns may appear. The work was of great importance because it focused not on the social dimensions of the segregations, but on the interactions of the neighbours and their behaviour.
Next step in ABM's developing was registered with the gaining popularity of the mathematical analysis of human behaviour. This new emerged direction of using mathematics dates back in 1940's when the mathematician von Neumann and the economist Morgenstern wrote the book 'Theory of Games and Economic Behavior', along with other strategy works that were published after the World War II. The book of von Neumann and Morgenstern explored the use of game theory in order to identify optimal strategies for military (Copeland, 1945). The game theory is a branch of Applied Mathematics which explores the strategies adopted or that should be adopted by two or multiple players following a strict set of rules, the main objective being obtaining the highest gain/score (Myerson, 2013). The game theory is in fact a study of human behaviour from a mathematical point of view. Today, the applications of the game theory transcend mathematics, being used in social and computer sciences, focusing on logical and rational decision-making. The scientist who made the leap from the game theory to agent-based modelling was Robert Axelrod. He was intrigued by a game called Prisoner's Dilemma. In this game, two prisoners were given the chance to betray or to remain silent with respect to the other fellow prisoner. If they betrayed each other, both would be convicted to two years of prison, if one betrays, but the other remain silent, the one who betrayed would get free and the one who stood silent would spend five years in prison and if both remained silent, they would each spend four years in prison. Axelrod organised two tournaments in order to identify the best strategy for winning the game, the scope being obtaining the lowest conviction. During the tournament it was observed how the cooperation between two rational entities would work (Rapoport, Chammah and Orwant, 1965). Starting with these tournaments, Axelrod continued to develop agent-based models in political and human-related sciences (Axelrod, 1997(Axelrod, , 2006. In the late 1980's, computer graphics are starting to add to agent-based modelling, which now gains a visual component as well, not only logical and mathematical ones. For this period, the work of a computer expert, Craig Reynolds, is important for the development of ABM. Observing the behaviour of some live biological creatures, he worked on modelling the flocking behaviour of biological agents, being one of the pioneers in biological agent-based models (Reynolds, 1987).
Starting with 1990's ABM expanded, since more and more specific software appeared. The first applications were StarLogo, Swarm and NetLogo, followed by AnyLogic and GAMA. Today there are more than 100 applications/toolkits on the market, the majority of them being open source.
After a decade from the first computer programs specially designed for ABM, it started to appear a wide range of tools, that permitted the users to create their own codes and thus, their own models. This period was the point from which mostly anyone could create a code for modelling, not only mathematicians or computer experts. Depending on the software used, some programming skills are required, but the use of these is not limited to the experts anymore. Today, there are many tools available, based on different programming language, designed for wide-range applications or for specific ones. In the table below, the existing open source toolkits are presented.  Abar et al., 2017). From the table above, it can be observed that almost 60% of the open source toolkits are based on Java programming language and more than 85% of them are designed to be used on desktops and personal computers, which are the most popular platforms. Regarding the field for which they were designed, 34% have a wide-range use and only 11% have GIS capabilities integrated.

DESIGNING AN ABM
The agent-based models have three components (Macal and North, 2010): the environment, the agents and the interactionsbetween agents and between the environment and the agents (Fig. 1). The agents are defined by their attributes and behaviour and the interactions are established through a topology of connectedness. All these are part of the set of rules that the developer has to establish.

Figure 1 The components of an agent-based model (Legend: green circlethe environment; red trianglesagents, black dashed arrowsthe interactions between agents; grey dashed arrowsthe interactions between agents and environment).
There is not a definition for agents, but, based on the practice, they seem to have some essential properties (Macal and North, 2010): (i) are selfcontained and unique, despite the fact that it belongs to the same group and this also leads for them to be heterogeneous; (ii) are autonomous, having behaviours that they enable by themselves; (iii) they have a state that it can change over time; (iv) are social, being able to interact with other agents; (v) they may be adaptive, learning by experiences; (vi) they may be goal-directed. The agents interact mainly with their neighbours and they change their behaviour based on these interactions.
When designing an agent-based model, the conceptualisation phase of the model is of great importance (Castle and Crooks, 2006). In this first step, the developer must establish the objectives of the model and must understand the system and its characteristics in order to define its elementsthe attributes of agents, their behaviour and interaction rules, but also the environment.
According to Castle & Crooks (2006), the next step after the conceptualisation is the formalisation of the model. In this phase, the attributes for the agents must be thoroughly established, as well as their behaviour and interactions (Fig. 2). Also, of equal importance as introducing the agents in formalisation, is the environmental setting. The formalisation corresponds to the effective development of the model. Before taking into considerations the results emerged after running the model, there is a need to verify the inner function of the model, meaning it should be verified if the model functions as expected (Castle and Crooks, 2006). This step should be followed by calibration, which is a way of testing the model's capacity to simulate the real world (Orsi, 2018b).

Figure 2
The steps of designing an agent-based model (after Castle and Crooks, 2006) Before implementing or before taking into consideration the results from the model, it must also be validated. But, in many cases, models are constructed in order to prepare or to foresee the development of a phenomenon that will take place in the future. Thus, validating this kind of model is not always possible. In these cases, a sensitivity analysis can be used instead (Castle and Crooks, 2006;Orsi, 2018b). A sensitivity analysis tests the robustness of a model, meaning it 'measures' to what extend is the model sensitive. The most used method for sensitivity analysis is changing or varying some parameters in the model. Sometimes, even for simple models, with a limited number of parameters (or set of rules), testing the sensitivity means combing the parameters' modifications and this can result in a large amount of combinations. These combinations would require further investigation, meaning more resources allocated and more time spent. For these situations, the model developer relies on his intuition, his expertise or on expert knowledge in order to make the combinations which are more susceptible to be sensitive. If the outcome of the model changes significantly when varying the initial conditions by a small amount, it means that the model should be reconsidered.

APPLICATIONS, BENEFITS AND LIMITATIONS
The applications for agent-based models are extremely wide. Starting with computer-related use and finishing with healthcare, teaching, stock markets, military-combat, they expand and grow in popularity. The applications for ABMs are synthetized in the table below, some references being also given as examples of use. Natural sciences Biology Walker et al., 2004;An et al., 2009Ecology Recknagel, 2003Grimm et al., 2010 Social sciences Sociology Macy and Willer, 2002Psychology Epstein, 2002Politics Cederman, 2001Lustick, 2002Archaeology Axtell, 2000Gumerman et al., 2003 Economics Economy Tesfatsion, 2002;Chan andSteiglitz, 2008 Stock markets Campbell andCochrane, 1997;Chen and Yeh, 2001Business Chen et al., 2008Marketing Delre et al., 2007

Transport and Logistic
Ground transportation and traffic situations Dia, 2002  The main benefits of ABM are that it can explore emergent phenomena, it can be used to describe systems that are composed by behavioural entities and it is flexible (Bonabeau, 2002).
ABM is recommended to be used when human behaviour is involved. The best cases in which to use it are (Bonabeau, 2002): (i) when there are complex interactions between different agents -shopping in a mall, evacuating a room, etc.; (ii) when the agents can move and the space is limitedtraffic, evacuating a building; (iii) when the interactions between agents are complexvillage dwellers and wild animals, civils in conflict areas; (iv) when agents can learn and adapt.
The limitations of ABM (Bonabeau, 2002) refer to the fact that they are individual, a certain model cannot be copied for other similar problems. The model can be reproduced and adapted, but not copied. Other limitation is the scale of the modellinga regional analysis differs from a local one due to the details needed. The behaviour of the agents at a regional scale can be different from the one of a local scale and the set of rules that must be previously introduced can differ. ABM is more suitable to be used at a local scale. Also, it is very difficult to oversee all the possible actions of a human-related agent in a given context, so the rules that must be introduced in the beginning of the modelling can sometimes be inaccurate or incomplete. And this leads to another issue, which is the result. Depending of the complexity of the problem and/or of the agents, the results can be more qualitative, giving only an insight of the problem and in other cases, the results can be quantitative and can be used in decision-making and implementation.
The validation is another issue of these models (Orsi, 2018a) due to the fact that they are built to predict something that will happen, so it is very difficult to validate the model, e.g. the number of casualties resulted in case of a wildfire evacuation. They also present a high sensitivity to parameter variations and a lot of guessing is needed in setting the parameters (Orsi, 2018a).
Even though the limitations seem to be more numerous than the benefits, because the ABMs can deal with the behaviour of the agents, gives them a high advantage comparing to other models. And sometimes, only ABMs can be used in a certain problem, especially when human behaviour is implied.

CONCLUSIONS
Despite the fact that the agent-based modelling has evolved from mathematics, their use in social sciences is now preferred to the mathematical theories. The latest used to be the 'standard' in modelling a social behaviour. ABMs rely on computer simulations, that present several benefits regarding to mathematical theories. These benefits are (Castle and Crooks, 2006): (i) programming languages are less abstract that mathematical equations; (ii) the computers can deal with several simultaneous processes; (iii) when using a computer simulation it can be introduced in the model various types of agents and (iv) computer software can be organised in modules, so when a modification is needed, it can be done without perturbing the whole programme.
Even though the agent-based models are used as tools in order to investigate a certain problem, we can say that they are a 'problem' per se. The process of developing a model requires many knowledges, not only in the specific field of its application, but also in using an ABM toolkit, for choosing a proper toolkit for the purpose of the model, for using a programming language, for inner verification of the model, for calibrating and validating it etc. After making sure the model is suitable for use, it can be run, and the results can be relied on.