Agent-based model calibration using machine learning surrogates

Abstract

Efficiently calibrating agent-based models (ABMs) to real data is an open challenge. This paper explicitly tackles parameter space exploration and calibration of ABMs by combining machine-learning and intelligent iterative sampling. The proposed approach “learns” a fast surrogate meta-model using a limited number of ABM evaluations and approximates the nonlinear relationship between ABM inputs (initial conditions and parameters) and outputs. Performance is evaluated on the Brock and Hommes (1998) asset pricing model and the “Islands” endogenous growth model Fagiolo and Dosi (2003). Results demonstrate that machine learning surrogates obtained using the proposed iterative learning procedure provide a quite accurate proxy of the true model and dramatically reduce the computation time necessary for large scale parameter space exploration and calibration.

Introduction

This work proposes a novel approach to model calibration and parameter space exploration in agent-based models (ABM). It combines supervised machine learning and intelligent sampling in the design of a surrogate meta-model, which constitutes a computationally cheap approximation of the real model.¹ Our surrogate can then be employed to explore the parameter space of the model at almost zero computational costs.

ABMs deal with the study of socio-ecological systems that can be properly conceptualized through a set of micro and macro relationships. One problem with this framework is that the relevant statistical properties are a priori unknown, even to the modeler. Such properties emerge from the repeated interactions among ecologies of heterogeneous, boundedly rational and adaptive agents.² This results in dynamic properties that cannot be studied analytically, causal mechanisms that are not always possible to identify and emergent relationships that cannot be deduced by simple aggregation of micro-level interactions (Anderson, et al., 1972, Gallegati, Kirman, 2012, Grazzini, 2012, Tesfatsion, Judd, 2006). This raises the issue of finding appropriate tools to investigate the emergent behavior of the model with respect to different parameter settings, random seeds, and initial conditions (see also Lee et al., 2015).

The primary challenge in exploring the parameter space and calibrating ABMs is the escalation in the number of parameters resulting from increasingly realistic ABM dynamics. For example, recent macroeconomic models use dozens of parameters to capture the complexity of micro-founded, multi-sector and multi-country phenomena (see Fagiolo and Roventini, 2017, for a recent survey). Existing tools for direct estimation and global sensitivity analysis (often advocated as a natural approach to ABM exploration, cf. ten Broeke, van Voorn, Ligtenberg, 2016, Moss, 2008, Thiele, Kurth, Grimm, 2014) are computationally prohibitive, requiring time and computational resources that are not often available to researchers or practitioners. This increase in the parameter set results in what is referred to as the “curse of dimensionality”, i.e. the convergence of any estimator to the true value of a smooth function defined on a high dimensional parameter space is very slow (De Marchi, 2005, Weeks, 1995). There are potentially an exponential number of local critical points in the parameter space that can be mistaken for global maxima or minima.

Traditionally, three computationally expensive steps are involved in ABM calibration; running the model, measuring calibration quality and locating parameters of interest (more on validation of ABMs in Fagiolo et al., 2017). As remarked in Grazzini et al. (2017), such steps account for more than half of the time required to estimate ABMs, even for extremely simple models. Appropriate tools need then to be designed to quickly search for “meaningful” parameters and initial conditions. One approach is to replace the computationally expensive ABM with a cheaper proxy. This is the aim of meta-models or surrogates, which approximate the relationship between ABMs’ inputs and outputs (see Fagiolo, Guerini, Lamperti, Moneta, Roventini, Sapio, 2017, Lee, Filatova, Ligmann-Zielinska, Hassani-Mahmooei, Stonedahl, Lorscheid, Voinov, Polhill, Sun, Parker, 2015) in order to quickly explore the parameter space. Surrogate models are traditionally employed as fast approximations of complex phenomena that are expensive to evaluate in real life or in simulation (see Booker et al., 1999), and are regularly leveraged to locate promising parameter combinations avoiding costly computations. Accordingly, if the approximation error is small, the surrogate can be interpreted as a reasonably good replacement for the original ABM during parameter space exploration, calibration and sensitivity analysis.³

Recently, kriging (Conti, O’Hagan, 2010, Rasmussen, Williams, 2006) has been introduced as a surrogate modeling approach to facilitate parameter space exploration and sensitivity analyses of ABMs (Bargigli, Riccetti, Russo, Gallegati, 2016, Dosi, Pereira, Roventini, Virgillito, 2016, Dosi, Pereira, Roventini, Virgillito, 2017, Dosi, Pereira, Virgillito, 2017, Salle, Yildizoglu, 2014). However, when the model’s response surface is completely unknown and possibly contains non-smooth regions, as it is typically the case in ABMs, kriging requires a large number of evaluations and extensive exploratory data analysis that increase with the size of the parameter space (more on that in Section 2). Such constraints hold also for state-of-the-art extensions (see Herlands, Wilson, Nickisch, Flaxman, Neill, Van Panhuis, Xing, Wilson, Dann, Nickisch) and it forces modelers of large scale ABM to arbitrarily fix a subset of parameters whenever the parameter space is too large (see e.g. Barde and van der Hoog, 2017).

What is needed is an efficient, “hands-off” approach to explore the complex parameter space of agent-based models that practically accounts for the limited computational resources of the user. Our approach explores the ABM parameter space using a non-parametric machine learning surrogate and iterative sampling algorithm that intelligently searches the response surface with few limiting conditions. In particular, no parametric assumptions or knowledge of the topology governing the spatial distribution of the data is required.

In a nutshell, the procedure begins by first drawing a relatively large “pool” of parameter combinations using any standard sampling routine, where each combination contains a value for each initial condition. This pool acts as a proxy for the full parameter space. Next, a (very small) random subset of combinations are drawn without replacement from the pool to initialize the learning procedure (again using any standard sampling routine). The ABM is then evaluated for each of these initial combinations and its outputs receive a “label”. Those outputs satisfying a user-defined calibration criterion are assigned to a “positive” category (label 1), otherwise to a “negative” one (label 0). A surrogate is then learned over the combinations using the selected surrogate algorithm.⁴ The first surrogate is used to predict the probability that unlabeled combinations in the pool belong to the “positive” category. This concludes the first round. In the second and subsequent rounds, a very small subset of the pool is drawn according to the predicted positive probability. These selections are evaluated in the ABM to learn their true labels and aggregated to the set of all other combinations that have been sampled during the previous rounds. This continues over multiple rounds until the user-defined number of evaluations (the so called “budget”) is reached or a predefined level of performance is achieved.

As illustrative examples, we apply our procedure to two well known ABMs: the asset pricing model proposed in Brock and Hommes (1998) and the endogenous growth model developed in Fagiolo and Dosi (2003). Despite their relative simplicity, the two models might exhibit multiple equilibria, allow different behavioural attitudes and account for a wide range of dynamics, which crucially depends on their parameters. We find that our machine-learning surrogate is able to efficiently filter out combinations of parameters conveying the output of interest, assess the relative importance of models’ parameters and provide an accurate approximation of the underlying ABM in a negligible amount of time. The advantages in terms of computation cost, hands-free parameter selection and ability to deal with non-linear characteristics of the ABM parameter space of our approach paves the way towards an efficient and user-friendly procedure to parameter space exploration and calibration of agent-based models.

The rest of the paper proceeds as follows. Section 2 reviews literature on ABM calibration validation, making the case for surrogate modeling. Section 3 presents our surrogate modeling methodology. Sections 4 and 5 report the results of its application to the asset pricing model proposed in Brock and Hommes (1998) and the growth model developed in Fagiolo and Dosi (2003) respectively. Finally, Section 6 concludes.