Model-based policymaking or policy-based modelling? How energy models and energy policy interact

Although modelling has been used for policy advice for


Introduction
To achieve the commitment under the Paris Agreement and the Energy Union (EU) Strategy, the European energy system must be greatly transformed and made entirely carbon-neutral [1]. Renewable energy, as a major component of the transition, brings new dynamics to the current fossil-based energy systems, including supply fluctuations and geographically more decentralised production. Although the way ahead is full of uncertainties, decisions are urgent: policymakers must now make the decisions that put us on track for renewables-dominated energy systems by mid-century. This has multiple dimensions, from designing policies for deployment of new generation assets [2], dealing with the integration of different sector policies [3], or balancing interests of involved actors [4].
Because real-world experimentation with system transitions is impossible, computer-based models can function as tools to allow policymakers to explore different decarbonisation options and policies in virtual 'laboratories' and generate an understanding of the policy domain [5]. As such, models can support designing policies for an uncertain future. Models can, however, also defend and justify already existing political views by providing "convenient arguments" based on "science". With increasing model complexity, such "policy-based evidence-making" [6] is increasingly hard to detect, as model complexity often comes with reduced transparency.
Despite the growing relevance of models for ambitious climate and energy policymaking [3], we know little about the impact of computer-based models in policymaking and almost nothing about the impact of policymakers on energy models. The interaction between modelling and policymaking in specific policy processes has not been investigated empirically. Our research aim is to generate empirical insights about the interaction between energy modelling and energy policymaking. In particular, we investigate (i) whether, how and when models influence policymaking processes, and (ii) whether, how and when policymakers influence the design, use and results of energy modelling.
Overall, our research makes three main contributions: (i) we expand the knowledge about how computer-based modelling tools and policymaking interact along the policy cycle and modelling process; (ii) we add to the literature on stakeholder-informed modelling by investigating forms of collaboration between modellers and policymakers, and (iii) we draw implications for the continued development of both energy models themselves and scientific policy advice in the energy sector to support ambitious national and EU climate and energy policies.

Background: Energy modelling and policymaking
Models are purposeful, mathematical simplifications of reality -"smaller, less detailed, less complex, or all together" [7], but they are also shaped by, and potentially shaping, the social world in which they are embedded [8]. They can function as 'discursive or negotiation spaces', bringing together different social worldssuch as represented by scientists and policymakers. This way, models can enable scientists and policymakers to explore and create a shared understanding about unknown futures and options, and to improve knowledge and inform policy [8][9][10][11]. In such best case usage, energy models inform governmental decision-making processes and help policymakers navigate an uncertain future [12], although model results are not the "final decision for the policy process to simply implement" [5].
Energy models have been used to advice and support policymaking processes in Europe by exploring potential energy futures, alternative socio-technical pathways and scenarios [11,[13][14][15]. Some governments have their own in-house modelling units [16], but most of them commission model-based studies, both to consultancies [17] and scientific institutions [18,19]. Often, scientific authors strive to create "policy impact" to inform and shape energy policy, while also pursuing their curiosity-driven research. Silvast et al. [20] observed that modellers have a widely shared interest in supporting decision-and policymaking, and the 'appropriate' use of models by decision-makers. A recent survey by Chang et al. [21] found that among 48 investigated energy system modelling tools, almost two-thirds had a direct or indirect policy impact. However, over a third of the modelling tools did not have any identifiable policy effect, often because they were rather new developments, mainly used within academic research, or because their application scope was too limited [21]. While this provides an interesting perspective from the modelling teams, we furthermore explore how and when models actually impacted policymaking.
The application of models in policymaking is characterised by several challenges from the perspective of modellers and (policy) users [22][23][24]. These problems include the inability of models to answer specific questions that users need answered [22], low transparency of models [25], lack of trust in models by policymakers, inability of models to deliver timely support for decision-making, missing capacities in institutions to make use of complex modelling, the diversity of stakeholder involvement in the decision-making or changes, and uncertainties inherent in the policy environment [26].
Engaging policymakers and other stakeholders in the modelling process increases the chance of the model's impact on policy output [26][27][28]. As a result, many formats of stakeholder-informed modelling such as participatory modelling, group model building, or participatory simulation exist. In such processes, policymakers and other stakeholders can participate at different stages of the model development, from data collection, through model construction and validation, to interpretation of model results and model use [29]. While such engagement can increase the chance that models answer the precise questions of involved policymakers, it also increases the possibilities for policymakers to influence the modelling process and move modellers towards producing the results policymakers need to confirm their pre-existing beliefs [14], or to justify already made decisions and proposals.
There is very little knowledge about the influence of policy on modelling. From science-policy relations research, we know that politics can generally shape research, especially in commissioned work [30]. Policymakers commission modelling, which implies that policymakers and modellers interact in some way [5,23], but how and to what extent policymakers influence modelling must be further explored.

Analytical framework
To conceptually structure our analysis, we use the policy cycle model [31,32]. The stages of the policy cycle include: agenda setting, policy formulation and adoption, policy implementation, and policy evaluation. The cycle then starts again, as new circumstances or needs generate new policy demands [31]. Along the policy cycle, different actors provide different means and carriers of informationlike modelsto policymakers with different policy impact [33]. The interactions between public policies and actors, contexts, events, and outcomes are complex, and they encompass different sources of pressures and information [33,34], such as interests groups or advocacy coalitions [35]. Since policymakers have only limited temporal, organisational and economic resources available to evaluate information and to base their decisions on them, they need to prioritise some information over others [36]. This raises the question about the influence of models within this process.
We apply the policy cycle model not to make a deep analysis of energy policymaking as such, but to structure how models support political decision-making processes, and at what stage of the policy cycle: to set their agenda/target (exploring), develop policies (ex-ante assessment), justify implementation of policies (validation), and/or evaluate targets and specific policies (ex-post assessment) (Fig. 1). We acknowledge that the policy cycle is a highly simplified description of policymaking, perhaps overly simplistic [31,32]. As we do not analyse how policies are made or what their impacts are, but how and when models and policies interact, it is sufficient for our purposes: it allows us to identify distinct ways in which models and policy may affect each other.
Further, we analyse how policymakers affect modelling and modellers and with what effect. The modelling process can also be viewed as a cycle, a sequence of steps. Based on Refsgaard et al. [37], we distinguish between five steps: (1) model study plan, (2) design and data, (3) model set-up, (4) calibration and validation, and (5) simulation and evaluation.
Step one involves the definition of the problem, modelling requirements and aims. In the second step, modellers conceptualise how the energy system should be modelled in sufficient detail to meet the Fig. 1. The policy cycle and potential use of models in the different stages. requirements of the model study plan, and prepare the input data. Then, the model is developed or improved, and calibrated or validated. Finally, in step five, simulations are run to meet the objectives and requirements of the model study. The results can be then discussed and evaluated with policymakers, and the results used to base decisions on it. Each step holds different possibilities for policymakers to affect the modelling, with the largest effect possible in the initial steps, especially in problem definition and data/assumptions, as it can affect the technical modelling steps in between [37]. Especially when the model study plan is developed, it could imply that modelling assumptions and data sources are openly discussed, but it could also strongly guide, or determine, the possible modelling outcomes.

Case study selection
We empirically investigate model-policy interactions in five different European cases: the EU, Germany, Greece, Poland, and Sweden. We selected these cases as representatives of different policy traditionsincluding the use and function of energy models in policymakingdifferent types of energy systems, as well as different views on the necessity and urgency of climate protection and the energy transition. For these cases, we focused on specific policy processes in which strong policy changes were proposed and the options or impacts were investigated with energy models ( Table 1). Note that we do not attempt to comprehensively analyse all different cases of model-policy interactions in each country. Table 2 shows the different models that were applied in the specific cases of interest, during the time under study. For other policy processes, other models may have been used: our findings refer to the specific models and the way they were used in the specific policy processes.

Method
To empirically study the interaction between energy modelling and energy policymaking, we apply a multi-method approach [68], examining events leading up to major energy-political decisions in the recent past.
First, we analysed policy documents, such as legislative acts, position papers, assessment reports, and (government-commissioned) modelbased studies, as well as secondary literature describing policy processes. Thus, we tracked and created a first timeline of policymaking steps [69], identified the policy-relevant model-based studies, and whether models informed policy decisions and identified relevant actors involved in these processes.
Second, we interviewed key actors involved in the specific policy and modelling processes. We interviewed four different stakeholder groups, including both ministerial staff and energy modellers (Table 3). Not all stakeholder groups have been interviewed for each case study, but were selected based on their relevance in each individual case study context. The classification of stakeholders represents groups to which an interviewee belonged at the time of conducting the interview, whereas in reality some interview partners have gained rich modelling experience, from changing their working environment between policymaking, industry, and research. To reveal how the energy models influenced the policy process, we asked about the role of modelling in policymaking generally and in specific policy processes, and how and to what extent modelling affected policy decisions. To identify how policymakers influenced modelling, we also asked about the collaboration between policymakers and modellers, and explicitly discussed whether, how and when policymakers influenced modellers and the modelling exercises.
The interviews followed a semi-structured guideline. Interviews were conducted in English or the national language of the case study country, and the presented quotations have been translated by the authors. The interviews were recorded and transcribed. We carried out a content analysis to derive evidence on with what purpose and how energy models have been used, using what form of collaboration, as well as how policymaking and modelling affected each other [68]. In addition, we used the interviews to complete the process-tracing, by adding specific aspects of when different actors commissioned or developed models for each case study [46] (the investigated processes are summarised in Figs. 2-6 in Section 4). To structure both the interviews and the analysis, we guided the work with the two cycles outlined above: For modelpolicy effects, we structured our work along the stages of the policy cycle, whereas for policy-model effects, we followed the modelling process. By using the policy cycle and modelling steps as analytical categories, we ensured a rigorous style in the interview analysis [70].

Results
We find that three main categories are underlying the mutual influence between computer-based modelling tools and energy policymaking: influence sources − who are different actors involved, and what is the relevance of models in relations to other sources; model purpose − what models have been used and why; and modelling process − how and when are models used along the policymaking process. Below, we elaborate on the results of each case in detail.

EU's renewable energy directive 2018: Model-backed revision of the renewable energy target
In 2018, the EU defined its energy and climate targets for 2030. Here, we focus on the process of defining the renewable energy target, which was a long and arduous process [40]. Several modelling studies were commissioned to define and revise targets [66]. These model results supported the decision about the 32% of the EU's 2030 renewable energy target (Fig. 2).
The renewable energy target setting process for 2030 was initiated in October 2014, as the European Council decided a 27% target by 2030. This was a political decision, not supported by any modelling (EU_po-licy2). In November 2016, the European Commission presented the 'Clean Energy for all Europeans' package [71], which held proposals for several energy sector reforms, including a proposal for the new Renewable Energy Directive. To support this, the Directorate-General for Energy carried out an impact assessment, which included results from model analyses. The Energy-Economy-Environment Modelling Laboratory (E 3 MLab) of the National Technical University of Athens (NTUA) carried out analyses with the PRIMES modelling suite [52], a set of models organised around PRIMES Energy System [67], and coupled it  [66]. The PRIMES modelling suite applied in this case study encompassed also other models dealing with various aspects of the energy system, coupled with each other. These models were: Prometheus, CAPRI (agriculture), GLOBIOM/G4M (land use change and forestry), GAINS (non-CO2 emissions, pollutants) and different "elements" of PRIMES: PRIMES-Energy systems, PRIMES-TAPEM (transport activity modelling), PRIMES-TREMOVE, PRIMES-Biomass supply and PRIMES-Gas supply. For more details see [67]. ** Model documentations are not publically available.
Remarks: *Two interviews were conducted with more than one person. Numbers in brackets show a total number of interviewees, which represented the same institution or stakeholder group. with the macroeconomic model GEM-E3 [53]. Furthermore, Cambridge Econometrics conducted an analysis with the E3ME model [54] The impact assessment showed that the 27% target was feasible to be achieved by 2030, and the European Commission did not propose a higher target, because of its political mandate coming from the Council (EU_policy3). Experts and the European Parliament criticised the impact assessment for its conservative and high cost assumptions of both renewable energy and CO 2 prices (EU_NGO2; [41]), especially since the European Parliament had already in 2014 called for at least a 30% target [66]. Once the European Parliament took over the responsibility for the directive's text and the accompanying impact assessment and prepared their own report [72], a group of the Industry, Research and Energy (ITRE) committee's parliamentarians investigated the proposal and the accompanying impact assessment, guided by the idea of re-defining and increasing the renewable energy target (EU_modeller1; EU_policy2; EU_NGO2). For that purpose, they commissioned Fraunhofer ISI, Enerdata and SQ Consult to do an analysis comparing various model-based studies concerning the feasibility of higher renewable energy targets, but conducting no modelling themselves [73]. In the meantime, in December 2017, the European Council reached an agreement on a negotiating position prior to the trilogue, confirming the 27% target. In January 2018, the European Parliament voted for a binding 35% renewable energy target and gave the start for trilogue negotiations [72].
During the trilogue negotiations, Member States such as Italy and Spain advocated for a higher target (EU_modeller1; EU_policy2). A large-scale model study financed by the European Commission, but published by the International Renewable Energy Agency (IRENA) shortly before the trilogue [74], played an important role in the negotiations. This study showed that a higher EU renewables target by 2030 is feasible, and IRENA's institutional credibility only strengthened this argument (EU_modeller1; EU_policy2). Against this background, the Parliament requested the Commission to recalculate its impact assessment and include higher renewables targets (EU_policy1; EU_policy2). In response to this, DG-Energy commissioned the same institutions to carry out additional scenario analysis with PRIMES, E3ME and GEM-E3, confirming the feasibility of a higher renewables target. This new, more ambitious scenario was a pivotal input to achieve a political agreement between parties involved in the trilogue negotiations. In November 2018, the European Parliament adopted the final text of the Renewable Energy Directive recast with a 32% renewable energy target by 2030, and a month later, the European Council did the same.
We find that policymakers had a strong influence on which ambitionlevels were modelled in the EU's 2030 renewable energy target setting, which indicates policy's influence in the stage of the development of the study plan. Models had a strong influence in the reform of the target, not only the PRIMES framework prominently used by the Commission for over two decades (EU_NGO1), but also other model studies commissioned to other organisations. Hence, we see clear evidence that models were used to generate results supporting already existing positions, be it less (Council) or more (Parliament) ambitious renewables target. However, we also see that the models were used to explore and increase knowledge about policy options (EU_policy1). In particular, the long and model-heavy discussion enabled an informed, science-based debate about the renewables target. For this, models were highly influential, but "it doesn't mean, however, that whatever comes from the modelling is automatically endorsed as proof by policymakers, but that's the ground. That creates […] a battlefield and based on that, different opinions can be exchanged. But everything starts with the modelling" (EU_policy2). Nevertheless, although modelling had a meaningful influence on policymaking beforehand, "in the end, of course, it's always a very political decision" (EU_policy2) and it "is not the models that fix the target" (EU_policy3).

The German photovoltaic support reform in 2009: Model assumptions under fire
In 2009, Germany implemented the second reform of the Renewable Energy Sources Act (EEG). In the course of the amendment of the Act, the reduction rate of the photovoltaic (PV) feed-in tariff, the so-called 'degression factor', was a main point of discussion. Over almost two years, a heated scientific and political debate took place between an environmental and an economic coalition. The environmental coalition was supported by energy models (Fig. 3). The final reformed EEG contained an increased degression factor from 5%/year to approximately 10%/year as a compromiseup to 50%/year have been demanded [75]. This 10%/year degression made PV generally less attractive but still the declining costs for PV led to an explosion of PV installations in subsequent years.
The political processes around the EEG are characterised by This intra-governmental disagreement is reflected in two sets of studies, commissioned by the two Ministries, giving contradictory recommendations (GER_modeller1; GER_policy1; GER_policy2; [75]) in the context of the EEG monitoring and impact assessment report 1 .
In 2006, the BMU commissioned two model-based scientific studies for the impact assessment and adjustment recommendations of the EEG [57,76]. The first was conducted by the German Aerospace Center (DLR). DLR used the ARES model [77] for a scenario-based economic assessment of renewable energy deployment in Germany [76], showing that immediate ambitious renewables expansion would increase shortterm costs but pay off for Germany's welfare in the long-run. The modellers were asked by the commissioning Ministry to use reliable and plausible data, but what data was used was left to the modellers to decide (GER_modeller1). Due to the difficulty to obtain up-to-date and empirically grounded input data in the rapidly evolving renewables sector, the modellers worked with scenarios for deployment pace and investment costs based on past trends (GER_modeller1). Still, the scenario assumptions were quickly outpaced by the actual development, limiting the usefulness of the study for long-term projections. The modellers communicated the uncertainties to the Ministry, and thus, as a modeller stated "we always emphasised that the model is poor, because […] we don't have the market dynamics in it […]" (GER_modeller1). In the following debate, the argument about the long-term economic benefits was, however, overshadowed and alienated by the debate about the shortterm costs (GER_modeller1).
The discussion about short-term cost originated from the second BMU-commissioned study, conducted by Fraunhofer ISI. Fraunhofer ISI used the electricity market simulation model PowerACE, to investigate the so-called merit order effect [57]. The scientists concluded that, in 2006, the prioritised feed-in of renewables by the EEG lowered the prices on the electricity exchange more than they caused total additional expenses for society. As a result, renewables made economic sense not only in the medium or long-term, which supported the arguments of the environmental coalition [75].
In response, the BMWi commissioned a study to the Institute for Energy Economics (EWI) at the University of Cologne [78]. Their theorybased study strongly criticised the PowerACE analysis, because of its assumed static power plant portfolio. Thus, the EWI argued, the Fraunhofer study neglected the external market costs of inflexible renewable energies compared to controllable plants, and severely lowered the overall importance of the merit order effect [75,78]. Consequentially, the studies started a dispute about the net costs or benefits of the merit-order-effect "that has not really been resolved to the present day" (GER_policy2).
The opposing perspectives made it into the official assessment report of the EEG [79], on which the German Government's EEG reform draft was based [80]. The governmental draft was published, containing a 'medium' degression factor. Despite this compromise proposal, the conflict between the Ministries continued (GER_policy2), both calling for further data about the economic impacts of the different options [75]. The environmental committee of the Parliament scheduled an expert hearing, including one PowerACE modeller as a scientific expert, to provide further information about the EEG draft and facilitate a solution [75]. Energy models were not part of the hearing; however, the model expert used the model results to build his arguments [81], speaking in favour of the ecological coalition [62,56]. In the subsequent coalition-level negotiations, science, including models, did not play an important role anymore, as there was no direct connection made to scientific models or scientific results (GER_policy1). Experts representing both coalitions emphasised the global strategic importance of PV and expressed a will not to endanger the German solar industry by a too radical degression [75]. Eventually, the Ministries agreed on the final reform after intense negotiations between leaders at the two Ministries [75].
Overall, models played an important role in the EEG's reform process, informing policymakers and Ministries about the effects and costs of different policy options. However, we also find evidence that the Ministries commissioned modelling studies with a clear assignment to support their respective policy positions and, not only to explore options and impacts ─ wherewith they defined the model study scope. We do not see any evidence that the Ministries prescribed "acceptable" results. Still, already the selection of institutions, models and framings strongly  1 This report is ex-post-evaluation report of the EEG Act, the so-called 'experience report' (German: Erfahrungsbericht) [79].
indicate that the Ministries knew what type of results would be produced. In the final negotiations, the science base played no significant role, as the reform was re-politicised.

Phase-out of lignite in Greece: Modelling the 'wind of change' towards the 2030 & 2050 targets
During the preparation of the draft version of the Greek National Energy and Climate Plan (NECP) in 2018, indigenous lignite continued to play a major part in the electricity generation in all scenario analyses and policies formulated until 2019. However, in the second half of 2019, following a government change, the newly-elected Government of "New Democracy" took the political decision of completely phasing out lignite-fired power plants by 2028. This called for extensive modelling work to evaluate and justify the decision (Fig. 4), and to update the NECP accordingly. During both development stages of the Greek NECP, several sets of energy system models were important in the design of the 2030 energy policy to achieve a simultaneous expansion of renewables and the gradual phase-out of lignite.
In 2018, the Ministry of Energy and Environment established a committee for the preparation of the NECP, and commissioned the Centre of Renewable Energy Sources (CRES) to support the development with scenario analysis, using TIMES-GR [58] as well as the WASP model [82] and another in-house power system simulation model. Prior, the Government only applied an ad-hoc use of models, though; in this case; the models used for the scenario analysis were an integral part of the planning process. In these scenarios, Greece continued to rely on lignite power, and the draft NECP submitted to the European Commission in January 2019 consequentially contained lignite generation.
After the change of government in June 2019, the newly-elected government announced the complete phase-out of lignite by 2028. "Shutting down the lignite-fired power plants was a political decision taken before modelling exercises took place. However, it is very likely that the target was set after non-official meetings between the Ministry and sectoral experts" (GR_policy1). Models did not affect the decision for the lignite phaseout. Instead, the decision was aligned with the phase-out decisions in other EU Member States, especially Germany, and reflected an increased ambition, as well as the support of clean energy investments in Greece following the government change (GR_policy1).
Following the political announcement, in September of 2019, the Ministry of Energy and Environment again commissioned CRES to perform scenario analysis using the TIMES-GR energy system model, to explore and evaluate how the lignite phase-out can be implemented, and what should be the alternative options to ensure capacity adequacy, including the estimation of investment requirements. Furthermore, CRES used the Dispa-SET model [59] to study the operation of the power system until 2035, and examine potential operational limitations after the decommissioning of the lignite-fired power plants. In parallel, a more detailed analysis of the power system operation under high renewables penetration was performed by the Greek Independent Power Transmission Operator (IPTO), using the ANTARES model [60]. The models, thus, provided ex-post justifications of the technical and economic viability of the lignite phase-out, and explored options to maintain system stability during and after the phase-out.
In addition, the ANTARES model was highly influential when applied to the transport sector, focusing on the introduction of electric vehicles and possible effects to the power system: "The decisions taken for the renewable energy target in the transport sector was explicitly based on the results of the modelling work done in this study" (GR_industry1). The opensource nature of the models used increased the acceptance of the studies and their results (GR_policy1). In parallel, the Ministry commissioned the E 3 MLab to develop the national Long-Term Strategy to 2050, 'Energy Roadmap 2050', using PRIMES to explore the expansion of the NECP modelling scenarios (with 2030 being the reference year of the model) towards climate-neutrality pathways. Finally, once the final version of the NECP was prepared by the modelling teams, and before it was submitted to the European Commission, it "was included in a public consultation to consider the views of the wider public, lobbies, and others" (GR_modellers1).
Overall, over the past decade, energy modelling has been effectively applied at all stages of national energy and policy planning in Greece. As modelling teams are repeatedly commissioned to support the Ministry, a long-term relationship and trust continues to build between Ministry and modellers (GR_industry1). Especially in 2018-2020, the collaboration between modellers and policymakers was intense, bidirectional and trust-building: "The collaboration involved different stages of communication such as meetings purely of modelling purposes and data verification, as well as the participation in a wider roundtable with the Greek Ministry and the panel on the Greek NECP" (GR_modellers2). Policymakers influenced the design and data stage of the modelling process, as they ensured that data inputs/assumptions aligned with official projections, especially for key technology costs and performance characteristics. Moreover, the collaborative procedure is important to ensure continuity/consistency between the outcomes of the different modelling teams (GR_policy1). Finally, although the decision to phase out lignite was made before the modelling, models were used to explore options for how to phase out lignite, and to support the revision process of a lignitefree NECP. The current trend of modelling will be strengthened further with the introduction of a set of monitoring and verification procedures to foster credibility and trust in modelling activities and outcomes, and support upcoming decision-making (GR_policy1).

Energy policymaking in Poland for 2030, 2040 and 2050: Modelling into or out of the carbon lock-in?
Energy models played an important role for the Polish Government in the context of the EU climate and energy frameworks: to define, substantiate and back up its positions for European-level negotiations in 2008, and to prepare its NECP for 2030 and the Long-Term Strategy for 2050 (Fig. 5). The models were however used in very different ways.
During the European Council negotiations in late 2008, Poland was opposing the 2020 climate and energy targets (PL_NGO3) [49,83]. In contrast to the EU's decarbonisation paradigm, Poland has prioritised (and prioritises until today) national energy security, epitomised mostly by domestic coal resources (PL_modeller1; PL_NGO3; PL_policy1; [47]). It was also a result of the powerful position of the energy system incumbents (utilities and the mining sector) in Polish energy policy (PL_NGO1-2; PL_modeller2; PL_modeller4; [84]). Poland built its argument around the results of a model study by the consultancy EnergSys', based on variants analysis using the CGE-PL [85], PROSK-E and EFOM-PL models. The results of this study, which was not commissioned by the Government but by the Polish Electricity Association, showed that ambitious energy and climate targets by 2020 would be harmful for the Polish economy and energy security [48]. This argument remained a reference point for policymakers for many years [49,83], and cemented a conviction that ambitious decarbonisation policies are 'unachievable' (PL_policy1).
Ten years later, in the context of fulfilling the EU's Governance Regulation's requirements, models played an even more important role (PL_modeller2; PL_policy1). Two different processes took place in parallel: the Ministry of Energy (which in November 2019 split into the Ministry of Climate and the Ministry of State Assets) worked on the Polish Energy Policy (PEP) for 2040 and the Polish NECP for 2030, whereas the Ministry of Economic Development defined the Long-Term Strategy for 2050. Each Ministry commissioned their own modelling studies. For the NECP and the PEP, three consultanciesthe Energy Market Agency (ARE), ATMOTERM and (again) EnergSysmodelled the impacts on the economy, the energy system and health, mainly with the STEAM-PL, MESSAGE [61], CGE-PL [85], CALPUFF [62] and GAINS [63] models. The input generated through formalised public consultations and bilateral meetings with selected industries, contributed to the second draft of the PEP, and the final NECP submitted to the European Commission (PL_NGO3; PL_modeller3; PL_policy1). The results of these model runs co-shaped the final Polish NECP and indicated that development of decarbonisation policies in Poland will depend on additional financial support of the EU [50].
For the 2050 Long-Term Strategy, the Ministry of Economic Development commissioned the WiseEuropa Institute to assess the economic impacts of the energy transition, using PRIMES and DCGE PLANE 2.0. At the time of writing in January 2021, the 2050 Long-Term Strategy was not publicly available. Although the three documents and the accompanying modelling are complementary (PL_modeller3), the attitudes represented by institutions that prepared them differed: the Ministry of Energy was conservative in the overall levels of decarbonisation ambitions, while the Ministry for Development was more ambitious in that matter (PL_NGO2; PL_modeller2). In the process of the documents' preparation, the Centre for Climate and Energy Analyses (CAKE) supported and consulted the Ministries in understanding complex details of modelling on a day-to-day basis; at the same time developing its own modelling tool (PL_NGO1; PL_modeller1; PL_modeller4).
Overall, we find two main effects of modelling in Polish climate and energy policymaking. First, and most significantly, models were used to validate governmental positions, providing arguments supporting climate inaction. This was particularly strong in 2008, but is also clearly visible in the PEP and NECP processes. The Ministry of Energy selected the modelling consultants not only based on their credibility (PL_mod-eller1; PL_policy1), but also, in some cases, on conservative positions on decarbonisation policies (PL_modeller1-2; PL_NGO3). In all cases, policymakers strongly influenced modelling study plan, assumptions and limitations (PL_modeller4). As one of the involved modellers admitted: "for example, the assumptions in terms of targets on the share of energy from coal, were generally dictated by this contracting authority of ours [Ministry of Energy]" (POL_modeller3). Second, models have contributed to opening up the policy space and negotiation options to show that there are strategic and economically attractive options for radical decarbonisation in Poland (PL_modeller2; PL_policy1). Nevertheless, the impact of models on decisions was limited (PL_modeller2-4), both because the strategic direction was determined before the models results were finished, and because "a political decision can be made regardless of what the model shows" (PL_NGO2). Therefore, our analysis shows that the influence of policy on energy models was larger than the other way around.

Sweden's climate policy: Modelling for net-zero emissions by 2045
In 2017, the Swedish Government passed the Climate Act, decided with the approval of seven parties and the energy industry (SWE_po-licy3). This Act holds Sweden's net-zero target for 2045. To assist in target setting as well as exploring and evaluating measures (SWE_po-licy1), the Government and governmental agencies commissioned three energy modelling exercisesone before deciding on the Climate Act, and two after, to explore and evaluate policy measures during the implementation of the Climate Act (Fig. 6).
In December 2014, the Government set up a cross-party committee to propose a new climate policy framework [86] and a climate and clean air strategy [87]. For this, the Luleå University of Technology (LTU) was commissioned to support the process with the energy system optimisation model TIMES-Sweden [64,88] "to identify which kind of climate targets Sweden should have and to analyse the consequences of different targets" (SWE_modeller1). To do so, modelling teams used official projections for input data and assumptions in an iterative process: "We tried to be open with what kind of assumptions we make and presented and discussed it", described one modeller, adding that also the governmental organisations expressed their needs: "and then they [policymakers] have been communicating with us that we should use this or that kind of source or we should use this…" (SWE_modeller1).
Furthermore, the TIMES-Sweden model investigated the consequences of different sectoral targets in the non -trading sector, assessing "scenarios with or without the sectoral goals" (SWE_policy1). In this process, TIMES-Sweden was soft-linked to the general equilibrium model EMEC [65] of the National Institute of Economic Research (NIER), which has repeatedly been used to supported the Government's decision-making process in the past [89]. Using EMEC output data in TIMES-Sweden allowed for more transparent and consistent energy demand assumptions, which created a new picture of the economy and the energy system for 2035 [89]. However, the macroeconomic analysis faced challenges, as the model could not fully consider long-term targets and new technologies. Besides the technical modelling challenges, the EMEC model also caused politically difficult situations as it was partially seen to "create more problems than it solves", and as further one interviewee said: "It is not useful for me and it actually creates the opposite, […] a feeling of problem with the energy transition" (SWE_policy3). The scenario analysis with TIMES-Sweden worked well and the final decision explicitly draws on the modelling results, and the decision documentation contains a description of the model and limitations [86,87]. Thus, the modelling of emission scenarios succeeded in supporting the policy formulation: "the results were showing [what] to reduce when in which sector. […] And what I just recently got feedback on is, that this graph was in the end important to agree on the target", confirmed an involved modeller (SWE_modeller4).
In 2017-2018, LTU used the TIMES-Sweden model again, commissioned by the Environmental Protection Agency (EPA), to explore measures to meet the climate targets, as well as to investigate the role of modelling as a policy support tool and how they "can be useful when steering towards those climate goals" (SWE_modeller1). This process concluded that models must answer three central decision-making questions: What measures are needed; where; and when (SWE_policy1; SWE_modeller1). In 2020, LTU was commissioned by the Swedish  Climate Council to use TIMES-Sweden once more, to evaluate the Swedish Government's first climate policy action plan. The plan, presented every fourth years, describes measures towards the achievement of the climate goals of the Climate Act. The scenario analysis is currently ongoing, together with the industry-initiative Fossil Free Sweden, to also assess the industry's decarbonisation roadmaps.
Overall, we found that models play a significant role in the climate policy process: "Models have generally a large impact on policymaking in Sweden […] It's not the only tool but it is a very important tool" (SWE_-policy1). Swedish agencies, including EPA and the Swedish Energy Agency, have strong in-house modelling capacities and commission much modellingand let the results have a meaningful influence on the agencies' support to the Government (SWE_policy1; SWE_modeller1; SWE_policy4). Nevertheless, the decisions are made at Ministries and in Parliament, so that "much is steered by what politicians think themselves", and as one modeller further states "I think a part disappears due to other considerations, which have then more weight than the results from our models" (SWE_modeller3), thus reducing the actual impact of models in policymaking (SWE_modeller4; SWE_modeller1; SWE_policy2; SWE_policy3).
All model runs in the climate policy process were characterised by a close collaboration between governmental institutions and scientists along the modelling process, from defining research questions and model assumptions to designing scenarios and discussing and interpreting results (SWE_policy1; SWE_policy2; SWE_modeller1). Here, models have facilitated discussions and contributed to a mutual learning process (SWE_modeller4). But as the influence of policymakers on the modelling was limited, their role in interpreting the findings is stronger: "We can't influence them [modellers] but we can influence what we do out of these results" (SWE_policy3).

Discussion and conclusions
Our results, summarised in Table 4, show that energy models are used to support and inform policymakers at all different stages of the policy cycle. They are most often used to assess impacts of different targets (including feasibility and costs), but also to explore policy options. The exploratory role seems to be particularly pronounced in jurisdictions with high climate protection ambitions, as found in the EU and Swedish cases, as well as in the Greek case after 2019. We also show that policymakers sometimes influence the modellers and modelling at different stages of the modelling process cycle, especially by expressing demands for data sources and assumptions, and by constraining or prescribing the exploration space and possible results. The latter seems to be particularly the case in jurisdictions with low climate protection ambition (e.g. Poland), or in highly conflict-laden policy processes (e.g. German case).
We conclude that energy modelling and policymaking influence each other, but the main direction of that influence depends on the context and the particular case: models may support ambitious and wellinformed policy changes and target setting but modelling and modellers can also be instrumentalised to justify already made policy decisions. We discuss these findings and their implications below.

Modelling affects policymaking
We observe in several cases, in line with previous research [5,13,90], that models have helped policymakers to explore unknown futures, set appropriate targets and assess policy options for reaching these targets (see Table 4). In some cases, the use of models go beyond being mere "number generators" [3], towards 'negotiation tools' for policymaking processes. However, we, also observe that in most investigated cases the models support decisions, but the decisions are made using model results as one of several inputs, especially during the later negotiation phases. Therefore, models inform but do not make decisions.
We demonstrate that the influence of models on policymaking depends on the countries' experience in using energy models, as well as on the context of the processes in which the models are applied. Whereas in Brussels, Germany and Sweden, energy modelling for policy advice has been conducted for many years, it is a rather new and still not standard approach in Greece and Polandalthough this has changed in the context of the preparation of the NECPs. Such differences in tradition of using models to support policymaking very likely explain parts of the differences in the model use and impact. In addition, it seems that differences in the use of models are based on general policy preferences. At least among our cases, modelling is more used as an exploratory, supporting tool for target setting and impact assessment in the more climate-ambitious jurisdictions. In Sweden, Germany, Greece, and in the EU, models were used to help determine "appropriate" targets and their impacts. In Sweden and Greece, models were also applied to evaluate different measures for meeting the targets. In contrast, the long-standing attachment to coal in Greece (before 2019) and Poland did not require the use of models investigating options for a low-carbon transition (because the systems were not to change strongly) and if energy modelling was used at all, it was mainly to legitimise the status quo in the existing energy system.
Similarly, at least among our cases, policy processes with a lower internal conflict-level tend to rely more on models as exploratory tools for target setting and instrument evaluation, as shown in the Greek and Swedish cases. Although these cases saw debates about the "right" target, most political and societal actors agreed that strong action was needed. In the more conflict-afflicted processes, we observe that models were rather used to justify existing positions than to explore new options. This is clearly illustrated by the German and Polish cases, where divergence between ministries (Germany) and between the EU and national government (Poland), respectively, let each side commission its own studies in support of its arguments.

Policymaking influences modelling
We also find empirical evidence of policy influencing models, especially regarding "acceptable" questions, scenarios to be investigated, and output to be produced. In all investigated cases, and presumably in general, policymakers retain control over exploitation and political use of the results. Therefore, models do not dictate policies (see Table 4). Our results show that policymakers influence modelling, especially at the early modelling stages, such as the definition of the model study plan, by (co-)defining problems, objectives and assumptions, including input data. In almost all of our case, modelling has been commissioned by governmental entities, and this commissioned work may be generally more at risk of being influenced by policymakers [see also : 14].
The European Commission has analytical units performing modelling, but it also commissions modelling to external subcontractors. Such in-house modelling and internal capacities to understand its details increase the likelihood that results are turned into policy action, but the open-endedness and a pluralistic perspective of such work is questionable. As in the case of the EU's 2030 target-setting process, the controversies about which ambition-levels to model are an indication of the constraining effect of policymakers on the modelling process.
In all national case studies, the modelling was commissioned by the responsible ministries to external contractors from both consultancies and academia. Nevertheless, we find clear evidence of policy influence on modelling in the German and Polish cases. In the German case, opposing ministries sought model results to support their existing, conflicting positions, although they did not interfere in the modelling as such. In the Polish case, in contrast, we observed that modelling carried out by some consultancies linked to the state, raised questions about a privileged position of such entities and data monopoly, impacting the legitimacy and credibility of modelling.
Based on our cases, we find that governments tend to commission known and acknowledged modelling teams and, thus, well-established modelling tools. This may be grounded in the acceptance of models by governmental entities, which takes time, as Ittersum and Sterk [23] found. Nevertheless, this is an important informal bias, as ministries typically know what kind of results each model is able to produce. This does not suggest foul play or question the independence of the involved modellers, but rather shows the impact of the socially constructed nature of mathematically complex models. Because they describe perceived realities differently and answer different questions, their results will naturally differ [6] and be politically useful for different political camps.
Given this, the strongest influence of policymakers on modelling is their power over how model results, especially in commissioned works, are used politically. This is both legitimate and expected: naturally, political actors will use model results to support their position. However, taking the findings from the models and considering them scientific law can be problematic, as in the case of Poland in 2008 where the EnergSys model study underpinned the national energy policy inaction for many years: such a result may be technically correct in the context in which it is produced, but hardly corresponds to the climate and energy reality of the past decade.

Limitations, implications and outlook
Our study shows that energy modelling and policymaking affect each other. Our study is a snapshot of the complex interaction between energy modelling and policymaking in five European case studies. We do not know whether other effects or types of interactions can be observed in other cases. As our findings indicate that the interaction is highly context-specific, we expect that other types of interaction exist, and call for further research on this topic, for further countries and times, so as to improve generalisability of findings. We expect that, with increasing complexity of policymaking, model-based climate and energy policy advice will gain importance over time. In this study, we demonstrate that policy and modelling interact in different ways and at different stages of the policy and modelling cycles. However, because of the case study nature and complexity of policymaking processes, we can neither say to what extend models influenced final policy decisions, nor draw strong generalised conclusions for the conditions under which models are particularly impactful. Because "policy impact" is increasingly called for in modern research, we call for dedicated research for when and under which conditions models affect policybut also for studies to generate a systematic understanding of how (and how to avoid that) modelling is instrumentalised by policymakers.
Despite the case study-related limitations, our findings have implications for modelling practices and legitimacy, also beyond our specific cases, and for the role of science in policymaking and what is seen as "good (open) science".
First, we show that there are multiple ways in which policymakers use modelling, both including the optimal exploration of options (supporting evidence-based policy-making) and the less optimal instrumentalisation (policy-based evidence-making). This implies that modellers must be aware of how their models are used and can be used. Modellers must continuously reflect on their role in the political arena and be fully open and transparent about their study aims, constraints and assumptions. We are aware that modellers cannot be 'neutral', if they strive for policy relevance, and models are never 'objective', but it is nevertheless important that models openly explore different energy futures and not only steer towards pre-defined policies.
Second, scientists, including modellers, are under increasing pressure to generate findings that are immediately useful and have a practical "impact" on policy or society [30]. Researchers have a strong incentive to produce modelling results that stay within the current mainstream, because findings suggesting minor modifications of actions with the prevailing governance paradigm have better chances of achieving practical "impact". In contrast, models generating 'radical' results, or results strongly diverging from the political agenda of the current government, are less likely to be heard and achieve practical "impact". Such incentives are problematic because they may reduce the quality of policy-relevant modelling by limiting the acceptable explorative space of policy-relevant models. However, precisely these 'radical' insights are likely needed to bring the magnitude and speed of the transformation required to achieve the objectives of the Paris Agreement and the EU. Nikas et al. [91] recently added to this point by discussing that modelling needs to expand its comfort zone, such as by exploring extreme scenarios and disruptive innovations, drawing from the COVID-19 pandemic. As a result, ambitious political agendas need to be manifested in the modellers' and policymakers' mental models in order make their way into computer-based modelling tools.
Third, models are and should be only one of several inputs serving to inform policy decisions. While policymaking is complex and involves different actors, the decisions themselves are made by ministries and, eventually, by parliaments. A strong and direct link from models to specific policies is neither to be expected nor would it be desirable. Finding traces of the model results, but not the results verbatim in the final policy output, is a sign that the model had an impact. That model findings are not exactly represented in policy output is a sign of functioning democracywhereas an overly strong link would signify technocracy and a weakening of democracy.
Fourth, the legitimacy of model-based policy advice stands and falls with the model's credibility. With the rise in computational power, the number of sophisticated energy models available has increased strongly, decreasing the usefulness of the "black-box" models of the past [25]. Transparency of models is absolutely imperative for creating trust and is supported by the involvement of different stakeholders in the modelling process. Publishing open code and data can be challenging, for example due to issues of data ownership, privacy and security concerns (see [19,92]). Nevertheless, modellers can take different strategies when opening code and data, including: establishing who owns the intellectual property; choosing a well-known licence; using tools to support the creation of reproducibility, even if you cannot go fully open; distributing code and data; and providing support [25]. Whereas there is a growing open modelling community 2 , and policy increasingly funds only or mainly open modelling frameworks (e.g. the Horizon 2020 projects SENTINEL 3 and openENTRANCE 4 ), openness is still not adequately rewarded within academia. Yet, the benefits of openness go beyond improved model legitimacy, and help to improve work efficiency and quality of models through community efforts. Further-reaching changes, in academia, among research funders and study-commissioning institutions are needed to trigger a change in culture and reward openness in models stronger than today, both for scientific and policy use.
Last, building on the previous point, open-access models and platforms, such as intended within our SENTINEL project, are essential components towards more model transparency, more diversity in model use, and the availability of more comparable and credible results. The simultaneous use of several models, ideally by different teams, can additionally ensure not only diversity, but also disparity among the used models and make these powerful instruments truly useful for decisionmaking. At the same time, open-source models and platforms create opportunities for more transdisciplinary modelling. Co-creative approaches could bring modellers, policymakers and other stakeholders closer together in the modelling process, to best support sound and inclusive European and national policymaking.

Author contributions
D.S. led the work; D.S., A.C., H.G., V.S. and J.L. designed the study; D.S. performed the empirical research and analysed data for Sweden, A. C. for EU and Poland, H.G. for Germany, V.S, A.F and G.G. for Greece; D. S., J.L., A.C., H.G., and V.S. wrote the paper with inputs from all authors.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. on the SENTINEL Deliverable 1.1 by the same authors.