A general framework for combining ecosystem models

When making predictions about ecosystems, we often have available a number of different ecosystem models that attempt to represent their dynamics in a detailed mechanistic way. Each of these can be used as a simulator of large‐scale experiments and make projections about the fate of ecosystems under different scenarios to support the development of appropriate management strategies. However, structural differences, systematic discrepancies and uncertainties lead to different models giving different predictions. This is further complicated by the fact that the models may not be run with the same functional groups, spatial structure or time scale. Rather than simply trying to select a “best” model, or taking some weighted average, it is important to exploit the strengths of each of the models, while learning from the differences between them. To achieve this, we construct a flexible statistical model of the relationships between a collection of mechanistic models and their biases, allowing for structural and parameter uncertainty and for different ways of representing reality. Using this statistical meta‐model, we can combine prior beliefs, model estimates and direct observations using Bayesian methods and make coherent predictions of future outcomes under different scenarios with robust measures of uncertainty. In this study, we take a diverse ensemble of existing North Sea ecosystem models and demonstrate the utility of our framework by applying it to answer the question what would have happened to demersal fish if fishing was to stop.

is further comp icated by the fact that the mode s may not be run with the same functiona groups spatia structure or time sca e Rather than simp y trying to se ect a best mode or taking some weighted average it is important to exp oit the strengths of each of the mode s whi e earning from the differences between them To achieve this we construct a f exib e statistica mode of the re ationships between a co ection of mechanistic mode s and their biases a owing for structura and parameter uncertainty and for different ways of representing rea ity Using this statistica meta mode we can combine prior be iefs mode estimates and direct observations using Bayesian methods and make Rougier However some mode s are better at predicting some outputs than others An a ternative approach is to try and find the best mode s Johnson Om and Payne et a These methods imp y that at east one of the mode s is correct in the sense that it can predict the true output Not on y is this a bo d assumption but the addition of another mode may a ow an area of the output space to become probab e when before it was not Thus by increasing the number of mode s there is no guarantee that the uncertainty wi reduce One way of deciding which mode is the best is to weight mode s using Bayes factors a so known as Bayesian mode averaging Banner Higgs Iane i Ho sman Punt Aydin As Chand er exp ains there is genera y no mode better in a respects than the others and so there is no natura way of assigning a sing e weight to each mode Furthermore This idea is forma ized using a hierarchica mode for more information Ge man et a to represent the ensemb e of simu ators However there is no reason to be ieve that the popu ation of simu ators wi either contain or be centred on the truth Chand er so we need to a ow some difference between the popu ation of simu ators and the truth To describe the re ationship between the simu ators and the truth we deve oped an ensemb e mode that describes the popu ation of simu ators its dynamics and its re ation with the true quantity of interest We are interested in n true quantities y (t) = (y (t) 1 , … ,y (t) n ) � for examp e the biomass of n species at a time t for times t = 1, … ,T . We regard m simu ators each giving an output representing the quantities of interest ,m as coming from a popu ation with expected output (t) = ( (t) 1 , … , (t) n ) � the simu ator consensus To define our ensemb e mode we describe separate y the difference between y (t) and (t) the shared discrepancy and the difference between x (t) i and (t) simu ator i s individua discrepancy The outputs from simu ator i an n i dimensiona vector u (t) i may not a ways represent the e ements of x (t) i its best guess direct y For examp e the e ements of x (t) i may represent biomasses of individua fish species and the e ements of u (t) i may represent the biomass of functiona groups for examp e biomass of demersa fish We say that for some simu ator specific function f i ( ⋅ ) For examp e if the e ements of u (t) i are e ements of x (t) i or are sums of those e ements perhaps with some resca ing then the re ationship is inear where M i is an n i × n matrix For other examp es see Tab e In genera the simu ators are run with uncertain inputs and parameter va ues This eads to uncertainty in the outputs and is common y known as parameter uncertainty We say that for t ∈ S i where u i has expectation 0 and is samp ed from a simu ator specific distribution and û (t) i is the expectation of the i th simu ator s output at time t The simu ator specific distribution is found from fitting the simu ator to a finite data set Spence B ackwe B anchard Thorpe Le Quesne Luxford Co ie Jennings or by performing sensitivity ana ysis of the simu ator inputs

FIGURE
A schematic that shows an examp e of the ensemb e mode at time t. In this examp e we have four simu ators that are a ab e to predict the e ements of i is the expected output of the i th simu ator see Section The difference between the i th simu ator s best guess i and the simu ator consensus (t) is known as simu ator i s individua discrepancy and is sp it between its ong term i and short term z (t) i individua discrepancy see Section The difference between the truth y (t) and the simu ator consensus (t) is known as the shared discrepancy and is divided into ong term δ and short term (t) shared discrepancy see Section In addition we do not direct y observe the truth but we do observe a noisy version of it ŵ (t) see Section

| Individua discrepancy
At time t the difference between simu ator i s best guess x (t) i and the simu ator consensus (t) is simu ator i s individua discrepancy This divides the individua discrepancy between the ong term individua discrepancy i and the short term individua discrepancy i . i is an n dimensiona random variab e with expectation 0 and covariance C It seems natura to a ow z (t) i and z (t+1) i to be dependent on each other for examp e if at time t z (t) i was ess than 0 then we might a so expect z (t+1) i to be ess than 0 With this in mind we say that z (t) i fo ows a stationary auto regressive mode of order  A summary of the simu ators their outputs used in the case study the simu ator specific function i and a reference to where the parameter uncertainty Σ i was ca cu ated Thorpe et a for t > 1 where each z,t,i is an independent n dimensiona random variab e centred on 0 with covariance Λ i and R i is an n × n matrix with the constraint such that R i is stab e that is lim k→∞ R k i = 0. R i and Λ i describe the dynamics of simu ator i with R i ∼ g R ( ⋅ ) and Λ i ∼ g Λ ( ⋅ ) for some distributions g R and g Λ At t = 1 z (1) i is samp ed from the stationary distribution of the auto regressive mode described in Equation See Supporting information Appendix A for more detai s This formu ation means that the expectation of the ong run behaviour of the individua discrepancy is the ong term individua discrepancy that is | Shared discrepancy The shared discrepancy the difference between the simu ator consensus (t) and truth y (t) is sp it up into the ong term shared discrepancy δ and the short term shared discrepancy (t) that is The short term shared discrepancy is described by a stationary auto regressive mode of order for t > 1 where R is stab e and ,t is an n dimensiona random variab e centred on 0 with covariance Δ At t = 1 (1) is samp ed from the stationary distribution of the auto regressive mode described in Equation See Supporting information Appendix A for more detai s This formu ation means that the expectation of the ong run behaviour of the shared discrepancy is the ong term shared discrepancy that is

| The truth
In the absence of any simu ators our prior be iefs for the truth at time t y (t) fo ow a random wa k for t > 1 where each Λ,t is centred on 0 with covariance Λ y At t = 1 the truth y (1) fo ows a generic prior distribution p(y (1) ).
At times t ∈ S 0 there are n y noisy and possib y indirect observations ŵ (t) of the truth which come from some distribution p(ŵ (t) |y (t) ) that is prob em specific and is caused by data uncertainty Li Wu The e ements of ŵ (t) may not be the same as that of y (t) for examp e if observations are incomp ete or aggregated We assume that the samp ing distribution of observations depends on the truth through some function f y ( ⋅ ) such that and p(ŵ (t) |y (t) ) = p(ŵ (t) |w (t) ).
For examp e if w (t) is some inear transformation of y (t)   The groups are as fo ows and This is saying that after the end of fishing the variance of the truth of model i reduces and the amount that the ast va ue of z (t) i re ates to the next moves towards by a factor of exp (k i ) each year We take k i ∈ [0,6] as there is not much difference numerica y if k i goes above with The diagona e ements of R i fa between −1 and with and the off diagona e ements are set to The simu ator specific variance parameter Λ i is decomposed into a diagona matrix of variances Π i and a corre ation matrix P i such that The form of the prior distribution for the jth diagona e ement of log 10 ŵ (t) ∕ŵ (2010) ∼ N(y (t) ,Σ y ), The difference between the truth at time t and the corresponding simu ator consensus ( all t Unsurprising y the ensemb e mode predicts common demersal fish increase fo owing the fishery c osure as this group contains many species targeted by fisheries

FIGURE
Estimates of the og biomass of each group of species re ative to The so id ine is the median and the dotted ines are the upper and ower quarti es The first vertica ine is at the year that we first have data and the second ine is in the simu ated cessation of fishing   Simu ators that are predictab y wrong are more informative than those that are unpredictab y wrong even if the atter are ess wrong in the abso ute sense In our framework we distinguish between short term and ong term individua discrepancies which a ows us to distinguish between predictab y wrong simu ators with sma short term individua discrepancies z i and unpredictab y wrong simu ators Furthermore we a ow the short term individua discrepancies to be different for each group thus a owing a simu ator to contribute to the ensemb e mode for groups that it is informative about and be ignored for groups that it is not In the case study mizer does not predict the dynamics of sole very we and so the simu ator consensus on y weak y fo ows the mizer predictions On the other hand mizer does a reasonab e job of predicting the dynamics of common demersal and therefore it contributes more to the simu ator consensus for this group Thus the ensemb e mode exp oits mizer s strengths common demersal and discounts its weaknesses sole.
The ensemb e mode enab es forma quantification of uncertainty The ensemb e mode takes account of uncertainty from each of the simu ators through parameter uncertainty and structura uncertainty data uncertainty through noisy and possib y indirect observations of the truth and uncertainty in the ensemb e mode parameters As the simu ators are describing the same system we might expect the dynamics in the individua discrepancies to be simi ar ter for a simu ator to be good at mode ing one aspect of the ecosystem rather than being average at mode ing many things Anderson

| Conc usion
This work a ows for a synthesis of many mode ing studies that have been and are being conducted in such a way that we can obtain more ho istic know edge over a wide scope of comp ex eco ogica systems It a so a ows for inc uding a forma quantitative understanding of uncertainties and know edge gaps This enab es us to make comprehensive mode projections that take into account a that we have earnt from the simu ators co ective y