Advancing participatory energy systems modelling

system models are important tools to guide our understanding of current and future carbon dioxide emissions as well as to inform strategies for emissions reduction. These models offer a vital evidence base that increasingly underpins energy and climate policies in many countries. In light of this important role in policy formation, there is growing interest in, and demands for, energy modellers to integrate more diverse perspectives on possible and preferred futures into the modelling process. The main purpose of this is to ensure that the resultant policy decisions are both fairer and better reflect people ’ s concerns and preferences. However, while there has been a focus in the literature on efforts to bring societal dimensions into modelling tools, there remains a limited number of examples of well-structured participatory energy systems modelling processes and no available how-to guidance. This paper addresses this gap by providing good practice guidance for integrating stakeholder and public involvement in energy systems modelling based on the reflections of a diverse range of experts from this emergent field. The framework outlined in this paper offers multiple entry points for modellers to incorporate participatory elements either throughout the process or in individual stages. Recognising the messiness of both fields (energy systems modelling and participatory research), the good practice principles are not comprehensive or set in stone, but rather pose important questions to steer this process. Finally, the reflections on key issues provide a summary of the crucial challenges and important areas for future research in this critical field.


Introduction
The most recent climate mitigation assessment from the IPCC emphasizes the need for a broader societal transformation to achieve the Paris Agreement temperature goals in a fair and just manner [1].Energy system models are important tools for us to understand current and future carbon dioxide (CO 2 ) emissions trends as well as options to reduce them [2][3][4][5].However, as techno-economic tools they tend to omit many social and political contexts.There has been much debate on the prospect of modelling the broader social and political context of the energy transition [6][7][8][9][10][11][12].One strand of this research is participatory approaches to modelling, which seek to bring a more diverse range of views into the energy systems modelling process.This offers a means to build a better understanding of this societal context while also supporting more inclusive and fairer decision-making processes.The need for energy system modellers to pursue participatory approaches is reflective of a growing trend that calls for more societally engaged and action-orientated research throughout sustainability science [13][14][15][16][17][18][19], and there is a history of attempts to embed participatory approaches into integrated assessment [20] and environmental modelling [21][22][23].However, in the energy systems modelling field, participatory methods have received much less attention than efforts to bring societal dimensions into modelling tools.
While there have been efforts to integrate societal factors in energy/ emissions models, Hirt et al. note that these have produced an 'apparent lack of concrete recommendations for climate and energy solutions', and thus call for the exploration of participatory approaches to create more practical and actionable solutions [24, p. 175].Similarly, Geels et al. point out that properly integrating socio-technical theories and computer-based models is not possible, and thus suggest instead that bridging integrated assessment models, socio-technical transition analysis and practice-based action research may be a more useful way of addressing the needs of policymakers at differing levels (international, national and local) [6].Most recently, Göke et al. conclude that "rather than trying to simulate social preferences and convictions within engineering models, scenario development should pursue broad and active participation of all stakeholders, including citizens" [25, p. 9].
A key challenge for energy system modellers is the level of engagement between modelers and users (which are often identified as policy/ decision-makers) [24].It has been shown that commonly used datasets are at odds with stakeholder insights [25], and that model-derived energy scenarios do not reflect the preferences of citizens and experts [26,27].It is thus striking that recent reviews of energy systems modelling trends and proposed research agendas focus on the development of model features but they ignore or neglect the need for this to be steered by stakeholder and public input [28,29].This demonstrates a critical issue whereby model development priorities, as suggested by model builders, are not always aligned with the needs of user groups such as policymakers [30,31].Addressing this misalignment highlights the need for greater stakeholder involvement not just in shaping the model inputs (e.g. through surveys or workshops), but also importantly, in the research design and decision-making around model development priorities [31,32].Participatory methods may strengthen model derived insights during discussion of what the results mean [33], considering which models and outputs are useful [34], or assessing what questions models can and cannot answer [35].
There are many noted benefits to co-creative or transdisciplinary research: identify concrete needs and blind spots [31], build ownership of problems and consensus about best ways forward [36], develop socially relevant and actionable solutions to complex, real-world problems [35], or make models more relevant/useful in answering real-world problems and to achieve greater policy impact [37].However, it is also clear that participation is no panacea [9,38].Simply including a wider group of stakeholders in modelling processes may not generate substantive benefits if the form of engagement closes down scope of the expression and representation of the diverse perspectives.While the growing interest in participatory approaches is clearly a positive development, it is a poorly developed practice within energy systems modelling, which can thus result in 'box-ticking' exercises or modellers merely paying lip service to the idea, as previously warned by Voinov and Bousquet on the topic of stakeholder involvement in environmental modelling more broadly [21].
Despite the noted importance of pursuing co-production approaches to energy systems modelling, recent literature reviews have found a very limited number of existing examples.Upon reviewing examples of participatory methods in energy systems modelling, McGookin et al.
found that roughly one third of studies had involved a single 'extractive' interaction with stakeholders, with only ten out of fifty-three studies involving a collaborative approach whereby stakeholder groups are regularly engaged throughout the process [39].Likewise, Galende-Sánchez & Sorman found that participation remains very focused on top-down approaches, where citizens are increasingly consulted on climate and energy policy issues but cannot directly affect the outcomes [40].
A notable exception is the recent approach developed by Howells et al. which focuses on energy modelling for policy support that includes "engagement and accountability with the communities it involves, and those it will later affect" [43, p. 4].The authors draw on an interesting case study from Costa Rica, where building modelling capacity was combined with stakeholder engagement to develop net-zero deep decarbonization pathways [41].The work of Howells et al. aligns well with the objective of our paper, and indeed both manuscripts can be seen as complementary.Whereas Howells et al. provides a useful framing for distinguishing the different stakeholders, or "affected communities" (see table 1 in Ref. [42]) and their incentives and needs (in the online supplementary material), our paper provides more insights on how-to involve these different stakeholders.In addition, whereas Howells et al. develop their approach along the identified good governance principles, our work presents a framework and recommendations that follow the energy system modelling process (research design, model assumptions, modelling results, outreach and communication and evaluation).Both approaches provide relevant complementary insights due to their different organisation.Another complementary piece of work worth noting is the ongoing effort of IRENA to develop a toolkit for national governments seeking to pursue participatory approaches to long-term energy scenarios [43,44].
This paper sets out good practice for stakeholder and public involvement in energy systems modelling based on the reflections of a diverse range of experts from this emergent field.It is worth noting that this paper focuses on energy modelling in academic contexts, and engagement of broad and diverse groups of stakeholders.There may be other areas such as private or other public sector activities (e.g.national grid planning processes) and government modelling activities that involve consultation and the consideration of other inputs.However, these processes a) often only involve a narrow group of stakeholders (generally not the public at large) and b) are not normally published in the academic literature.Energy systems modelling science can and should lead by example and demonstrate how these processes can be made more open, fair and inclusive.
To advance this critical field, two workshops were held with the group of authors, which was guided by the following three research questions.
1. Should modellers include stakeholders during all stages of the energy systems modelling process?Why or why not? 2. With regards to the different energy systems modelling process stages, what have been the experiences to date?And what good practice should we strive for?3. What are the key challenges associated with involving a more diverse range of perspectives in energy systems modelling?And how might they be addressed?
Two workshops were held with sixteen practitioners, drawn from a variety of modelling and non-modelling backgrounds (see authors), as well as spanning the full range of academic career experience from early PhD through to postdoctoral researchers and lecturers/professors.The workshops also involved two participants from outside academia, who had previously collaborated with other authors on this topic.A summary of workshop process and notes recorded is provided in Appendix A.1.
The paper is laid out as follows.Section 2 outlines our understanding of some of the key concepts used throughout the paper.Section 3 introduces the conceptual framework that was developed to illustrate the different stages of participatory energy systems modelling.The ways in which stakeholders can participate in the process are then outlined in Section 4. The good practice principles to help navigate this process are outlined in Section 5. Finally, Section 6 closes with reflections on the key issues within this emergent field that are important considerations for scholars looking to implement these approaches and areas for future research.

Energy system models
There are a very large range of climate and energy systems modelling tools available for different applications across geographic and technical scales, as demonstrated by the many reviews conducted on the topic [24,28,29,[46][47][48][49].Some commonly used models include: i) energy system optimization models (overview in Fig. 1), which solve for the least cost system under policy constraints, for example limiting CO 2 emissions, ii) energy system simulation models, which explore user defined scenarios, iii) power system dispatch and expansion models that solve for optimal electricity market and system design [48].To inform climate and energy policy, the primary purpose of energy system models is to explore alternative designs of the energy system that will increase the levels of renewable energy and reduce CO 2 emissions.Within this, there are a variety of different applications across scales (global, national, local), sectoral focus (from whole systems coverage to buildings or electricity focused analysis) and temporal ranges (projections over decades or a detailed yearly focus).
Each type of model has its own merits and shortcomings.It has been argued that simulation models are better suited to answering policy questions since the end result is user defined rather than a least cost derived solution as in the case of optimization models [47].However, energy system optimization models are very popular tools for the insights they provide on the techno-economic performance of different systems configurations and ease with which goals such as end year emission objectives (e.g.net zero by 2050) can be examined once the model is built [3].While scenarios modelled over decades are important to understand long-term trajectories, sector specific models such as power systems models are important to explore the operation of the system and/or market across shorter timescales.These generally involve much more detailed temporal resolution with hourly rather than yearly demand profiles.
Regardless of model type, a key weakness is the narrow view of potential futures offered by techno-economic approximations of society.Studies have shown that previous energy scenarios and projections do not match reality [34,50,51], which is no surprise considering the messy, dynamic, and evolving nature of the real-world.There have been great efforts to open energy system models [3,[52][53][54][55], both by documenting how they operate in easy-to-digest language, such as the Integrated Assessment Modelling Consortium (IAMC) wiki 1 or the I2 AM PARIS platform, 2 and by establishing communities of practice around open access models and data such as the openmod initiative. 3However, despite these efforts, the dependence of energy system models on large amounts of data and assumptions means they are very 'opaque' and it is not easy to understand their inner workings [56].In acknowledgement of this, DeCarolis et al. suggest the use of models as a tool to challenge existing assumptions or 'mental models' rather than providing definitive answers [3].The pursuit of participatory methods is an opportunity to enhance modelling insights by increasing the diversity of perspectives feeding into the model, opening up alternative futures, and highlighting what is missing from the model(s).

Inter-and transdisciplinarity
Interdisciplinarity refers to a research mode that combines perspectives, methods, and information from several disciplines towards advancing fundamental understanding as well as addressing contemporary challenges that are too complex and multi-faceted to stay within the boundaries and scope of a single discipline [57].Teams interacting in an inter-or multi-disciplinary mode of collaboration involve a mix of experts within scientific disciplines; technical (e.g.modellers, engineers) and social scientists (e.g.economists, political scientists, sociologists, or psychologists).Transdisciplinarity also includes interdisciplinarity but transcends the boundaries of scientific disciplines.Transdisciplinarity involves non-scientists (e.g.stakeholders, citizens, policy-makers), and it transcends science in relation to the problems involved (e.g.problems derived from societal stakeholders) [58].
In energy systems modelling projects, inter-disciplinary teams are often the ideal.In such interdisciplinary teams the practicesideallytranscend the disciplinary mode of knowledge production and with a high degree of cognitive coupling [59].However, in practice the cooperation in such cross-disciplinary teams often results in a multi-disciplinary cooperation with a low degree of interaction between disciplines, where participants from different disciplines contribute on separate tasks in a common project without losing their individual mono-disciplinary identity and with little cognitive coupling [60].
Multi-disciplinary teams facilitate forums whereby a diverse pool of knowledge can meet to discuss the respective merits of different approaches and investigate opportunities for bridging participatory and modelling methods (as suggested by Ref. [6]).The social sciences would not only challenge who is participating in the debate [38], but also ask important questions on the limitations of quantitative models and what they represent [61].Likewise, the modellers should ask questions of the participatory process and how it contributes to solutions.It is of course acknowledged that there are organisational, institutional, time and funding challenges that make the creation of such diverse teams difficult.
In an ideal situation, the energy systems modelling process might involve three types of experts: technical experts focused solely on model building, social scientists who pose challenges to this group and transdisciplinary researchers sitting between the two groups, who's thematic focus would depend on the question to be answered.Policymakers and other end-users of energy system models are important stakeholders who influence not only the scope of model-based studies and how the results are used, but also the selection of data and assumptions [37].Literature has increasingly focused on the inclusion of the wider publiccitizens [16,39,62,63].Involvement of citizens could also comprise citizens organized in civil society organizations and social movements [64].However, in such cases it might not be 'ordinary' citizens participating in the processes but rather stakeholder representatives or professional lobbyists employed by such organizations [65].There are also instances where citizens are not directly involved, but rather social scientists, with an understanding of public attitudes, being involved in developing model scenarios [66].

The different forms of stakeholder engagement
The concept of stakeholders is often used in the literature without explicit definition [67].In this article, we adopt a contribution to stakeholder theory that understand stakeholders as "any groups or individual who can affect or is affected" ([68], p. 412).The literature has presented several typologies for stakeholders.In this paper, we focus on energy experts, policymakers, and civil society actorsin particular citizens.
There are a wide range of stakeholder engagement activities, and frameworks for categorizing them.Throughout a process stakeholder involvement can perform a variety of functions.Stakeholders can contribute to the definition of the guiding question or to create joint understanding and definition of the problem to be addressed [16,69].They can suggest scenarios or co-design transition storylines [70,69], explore the translation of story-lines into modelling input assumptions, or contribute to prioritization among trends and challenges [65].
There are also several means of classifying engagement activities.Arnstein's 'ladder of participation' is a well-known means of classifying stakeholder involvement in the planning system, which highlights the level of control given to participants [71].The functional-dynamic approach involves different forms of engagement depending on the stage of the study and its goals [62,69].Considering the flow of information during stakeholder involvement in the research process, Trutnevyte et al. [72] summarize activities as.
• Communicatingone-way flow of communication, usually for the purpose of awareness raising or educating, no opportunity for input into a decision-making or model building process, participants cannot influence the outcome of the research.• Consultingtwo-way flow of communication, surveys, interviews or workshops used to elicit stakeholder opinions, participants have the opportunity to shape the modelling and its results but not the research questions or objectives.
• Collaboratingopen and transparent communication throughout the process, participants given the opportunity to shape research questions and direction from the very beginning and throughout the duration of the project.
These three layers were further expanded by McGookin et al. [73] based upon the Wellcome Trust's 'Public Engagement Onion' [74], making distinction between consultation that involves actual discussion/dialogue (e.g.workshops) or when perspectives are gathered through surveys and interviews, as well as collaboration on research design or co-producing outputs.
It is useful to think of this communication-to-consultation-tocollaboration as a spectrum, with activities falling across it.Communication and dissemination of research findings would be currently the most common form of scientific engagement with the public.However, addressing weaknesses in the procedural fairness of critical energy policy decisions requires a move to more deliberative and collaborative forms of engagement.Collaborative research design and coordination, in line with established transdisciplinary research principles [16], requires a flexible and adaptive approach, which responds to stakeholder and public needs and is open to changing the research design as the project evolves.While the move to more collaborative processes is important, this is not to say that it then trumps the others.Each form of activity will be appropriate at different stages in the research cycle and in different contexts.
The practical engagement of stakeholders can be carried out using numerous methods.The literature on public engagement has identified almost 100 methods for stakeholder engagement [75].However, in practice there are four common methods most widely used: workshops, focus groups, interviews and surveys [76].These can appear in different forms; workshops can appear rather simple as steering group meetings [69], or more complex as stakeholder conferences, citizen juries, consensus conferences [62], and many other formats.A review of how these have been used within energy systems modelling studies is available in Ref. [39], here we provide just a brief summary of the workshop participants experiences with commonly used approaches (Table 1) based on the case studies in Appendix A.2.This is of course by no means a comprehensive assessment of participatory methods, but rather some useful considerations with regards energy systems modelling.There is already established literature on participatory approaches and how they have been used in areas such as environmental modelling [22,23], transition governance [77], and public engagement with transitions more broadly [14,78].

Conceptual framework: a participatory approach to energy systems modelling
Energy system models provide a critical evidence base for climate and energy policy.Within this, a commonly utilized tool are energy system optimization models.Despite the prominent use of these models, there are limited standard practices and guidelines available.To address this gap, DeCarolis et al. provide an outline of best practice for energy system optimization modelling [3].They distinguish between six stages from the formulation of the research questions to the communication of modelling insights (Table 2).
With a focus on the modelling process, two omissions in this best practice of DeCarolis et al. are: (1) opportunities for collaboration with researchers from other disciplines and (2) how to involve a broader range of stakeholders in the process [3].
First, there is a growing stream of literature that emphasizes the importance of linking modelling with other disciplines, such as social science [11,70], to be able to better approximate the socio-political developments and dynamics of real-world transitions [34], and account for the preferences of the stakeholder and the public [26,27].Nevertheless, most approaches go no further than considering social factors as qualitative exogenous assumptions [70,79].The opportunities for interdisciplinary learning and broadening the perspective on and understanding of energy transitions remain largely unexploited with a modelling-centric framework.
Second, the awareness of the relevance of engaging stakeholders in the modelling process is growing, since modellers want their work to be transparent [52], trusted [80], and have policy impact [81].Nevertheless, stakeholders and the public are still largely excluded from the presented modelling process, or their engagement is often limited to an exchange at the beginning (stage 1) or/and the end of the modelling process (stage 6).Collaborative energy systems modelling approaches, where stakeholders are involved throughout the process, are an exception [39].
We have developed a new framework which presents the opportunities for inter-and transdisciplinarity in energy systems modelling (Fig. 2).This framework distinguishes between five stages: (i) research design (ii) model assumptions and development, (iii) modelling results, (iv) outreach and communication, and (v) evaluation.It is built on the modelling framework from (DeCarolis et al., 2017) with some revision to the naming of steps and two adjustments: (i) Steps 1 (formulate research question) and 2 (set spatiotemporal boundaries) are combined into 'research design', and (ii) Step 5 'quantify uncertainty' broadened to 'modelling results'.
Each stage of our framework offers opportunities to involve stakeholders and the public in modelling to gain insights on their needs and expectations and receive feedback on the tools and results.Different qualitative and quantitative methods can be used here to facilitate the exchange.In addition, non-modellers, such as social scientists, can be engaged along the modelling process.They can bring in new perspectives from a different field and thus complement the modelling and expand the model boundaries.See [45] for guidance.

Table 2
Key steps associated with the application of energy system optimization models [3].The process is outlined as sequential stages here to make distinction between the different parts.However, in reality the process is not as linear as shown but much more iterative.Several feedback/forward loops and real-life developments (events such as the COVID-19 and energy crises) make it an evolving, dynamic process.
It is also important to note that we make the pre-assumption here that modelling is necessary, and participation is a helpful input.One could equally do this the other way around: participatory deliberative processes are important for insight, and sometimes modelling might help to inform these discussions.The process outlined in Fig. 2 comes from a specific perspective rooted in energy systems modelling -but that other perspectives are also valid, and how these are handled requires careful consideration (Section 6.4).

Unpacking the framework: what does good practice look like?
The following sections provide suggestions for good practice when involving stakeholders in the energy systems modelling process.It is based on the workshop discussions (Appendix A.1), as well as the collection of experiences from the authors (Appendix A.2.).Appendix A.2 contains a summary of nine participatory modelling projects, covering how stakeholders were engaged and the key learnings within each project.The framing of 'good' practice emerged during the discussions as a better reflection of the messiness of such processes as opposed to 'best' practice.The purpose of each of the following subsections is to introduce the different entry points for modellers to bring in participatory elements.These range from areas where there are plenty of existing examples to build on to those that are seen as important but rarely explored.

Research design
The research design process is a key opportunity for involving outside perspectives.At earlier stages, opening up the process of deciding the research questions and methods can enhance the relevance of results and thus lead to greater impact.This allows for the inclusion of different context-specific challenges or worldviews that stakeholders may have.At the same time, this step helps to clarify what can and cannot be modelled.Maintaining engagement throughout the project builds credibility in the process, a shared ownership of the results and supports mutual learning.
Following established best practice from transdisciplinary research, a project committee or partnership should be formed with representative groups to input on the design/decisions throughout the project [16,37,83].This requires a significant time commitment to maintain and coordinate.It largely depends on the way modelling research (funding) works.The research objectives and model will likely already be chosen during the proposal stage, which thus limits the flexibility of the research design.In an ideal setting, research granting authorities should allow representative groups to be involved as partners within the funding application, which would thus ensure the project proposal is aligned with their needs and their time commitment is accounted for from the very beginning.However, the transfer of funds to collaborators is not current practice.Considering the difficulty in compensating collaborators for their time, an alternative option is to identify outputs of value to them.Time should be allowed during the early phase of the project to host a scoping exercise with partners to review the research plan and discuss the key objectives/outputs.It is also important to recognise that the needs of stakeholders are constantly evolving and may differ quite significantly from the research team's needs/questions, thus regular meetings and check-ins are needed throughout the process.
Unless stakeholders are involved at the very early stages, the approach and desired outputs will likely already be chosen.One key consideration is the choice of model (Section 6.2).However, without a partnership, there are still plenty of ways that stakeholders can be involved in co-defining the problem, modelling needs, and research questions.If engagement outside the more formal process is not possible, time can be allocated during workshops or interviews to discuss desired outcomes, the research approach, or impact of the project.In addition, stakeholders may be engaged solely to discuss modelling priorities or research questions.

Model assumptions and development
In studies to date, the most common example of stakeholder engagement is at this stage [39].There are many examples of modelling assumptions, scenario choice, multi-criteria decisions analysis or some form of weighting of technology preferences that use quantitative approaches to engagement [82][83][84][85][86].This can be done through surveys [26,27] or structured workshops with questionnaires [87][88][89].In all these cases, caution is necessary because surveys, interviews and other elicitation methods may provide a snapshot of a current view or technological preferences of the stakeholders, but in models we work with timeframes like 2035, 2050 and even more.'Foresight' or 'vision' processes can encourage stakeholders to think of the future [90,91], for example, to ask participants to imagine 30 years ago before considering 15 years ahead.
These qualitative stakeholder inputs are generally used to provide a framing for scenarios [92], and identify areas that should be prioritised, or storylines that will be modelled [93].However, stakeholders can also participate in discussing the quantitative assumptions, input data or provide suggestions on what should be prioritised or excluded from the energy system model.This requires that the engaged actors have some level of knowledge of and/or expertise in how models work, meaning that it will be easier for some groups of stakeholders (i.e., experts) to meaningfully contribute to and others (e.g., citizens) will find it more challenging.The onus is on modellers to design the process in such a way that the different perspectives can all come together.Stakeholders may be involved in different parts or have different questions posed to them.Modellers need to think about their audience and how they are presenting the model to them.In a well-designed process non-experts will be able to provide input on the underlying model assumptions.
Providing training material and visual interfaces/dashboards are ways in which to facilitate this.
It is important to note that while the gathering of stakeholder inputs can provide important considerations for the model, so long as this remains the only form of engagement then a significant degree of control still lies with the modelling team.Unless stakeholder input is in the form of either quantified parameter choices (in questionnaires etc. as described above), modellers have many degrees of freedom to translate and use these inputs as they wish.Translating inputs to the model is highly subjective.One way to address this area would be to build a conceptual map with stakeholders outlining the translation from stakeholder inputs through the modelling process.However, it may be difficult to get stakeholders to commit time to these discussions, as a certain technical understanding of the model would be required.A key challenge is also accommodating a diverse range of stakeholders who may have conflicting priorities or technology preferences.Another approach, especially with less experienced audiences like the general public, could be to adopt the approach of informed citizen panels [87,89], where the participants go through an extensive learning exercise about the energy system and modelling before their inputs are asked.Another way is to present the models and modelling approaches to the stakeholders, including high-level explanations of the requirements, limitations, and benefits (i.e., rationale, data flows, etc.) associated with model interlinkages, and then ask for feedback on the taken approach.Ideally, the translation of stakeholder inputs into the model should be an iterative process.Evaluating the results and unpacking the assumptions can help to improve the transparency of the process (Section 6.3), as well as highlight key gaps (Section 6.4).
Stakeholders might be also involved in the actual model development, decisions around assumptions or scenario building.This requires a more in-depth and technical understanding of models, energy expertise, or even experience of modelling, which is why certain expert groups like policymakers might be involved.However, it is also important to discuss model development with people who don't understand the model at all.This requires greater effort on the part of modellers to think about how the mental models/beliefs of participants might usefully inform model or scenario choices.Efforts to make models more open and transparent is an important step, but what remains more critical is making them understandable, or comprehensible [33].This requires an exchange between modellers and users to unpack the modelling decisions and explore how the inputs from stakeholders are translated into the model.To enable active participation from a diverse range of stakeholders, modellers must open the 'black boxes' [52], and be able to explain the structure of their models and modelling assumptions.This is crucial to clarify with stakeholders the input models need, information they can represent, what questions they can answer and what models clearly cannot do, or where they would need to be advanced to answer new questions.

Modelling results
Unlike Section 4.2, this is a much less explored area.While results are often shared with stakeholders, this is rarely to inform a reiteration of the model or provide further insight but rather to disseminate the final results.However, stakeholders should also participate in the discussion/ assessment of the modelling results.This includes the necessary clarification of what the model results do and do not mean.Evaluating the model in this way can provide valuable insights into the relevance of modelling results and lead to a redesign of model assumptions or structures.It also offers an essential critique of what has not been included in the model.This needs to be an iterative process, whereby the model is open to changes based on discussion of results.Simply asking participants their opinions or perspectives and not facilitating feedback or evaluation of the analysis is not a meaningful engagement.It is important that participants understand how their input contributed to the research and results [94].Otherwise, they may feel disheartened with the process and subsequently loose trust in research and participation more generally, often referred to as 'research fatigue' [95].

Outreach and communication
Stakeholders can support the design of communication materials, coordination of events to communicate the research outcomes, as well as the publication of written outputs, open-access models, data and results.Transparency about possible outcomes and outputs during the early stages is important.Another key consideration is what range of outputs or events will be tailored to different groups.Publications and reports are very academic outputs.For stakeholders, the relevant outputs or outcomes might be visual displays or interactive tools, presentations, public events, outreach workshops, media articles, etc.Furthermore, code and data openness is not only critical to the research community, as it enables reproducibility and scientific credibility/validation, but also highly relevant for stakeholders too, since any debate on the results and added value of energy models first and foremost requires solid understanding of how modelling is done and what data is used.What is equally important and perhaps even more impactful in terms of maximising the comprehensibility of model-driven prescriptions is publishing model outputs in user-friendly formats: in this context, researchers need games, policy briefs, and/or infographics to make sure the information reaches a broad audience.
In an ideal situation, stakeholders would be involved in co-producing outputs such as policy briefs, reports or even journal papers.This co-

C. McGookin et al.
ownership may greatly improve the impact of the results, but it is very difficult to get stakeholders to commit time to it.What can work is when researchers prepare an initial draft and then share with partners for review or discuss it during a meeting.However, input in this manner is very passive, and there are questions around the extent to which the outputs are easily comprehensible (even when stakeholders are invited to give their feedback).Another issue may be that there is quite a significant lag between when stakeholder input is gathered and the finalising of academic outputs.This may cause some frustration when those involved are keen to share learnings with policy or other practitioners.Thus, reemphasising the importance of clarifying the goals and expected outputs at the beginning of the process.Finally, stakeholder inputs can inform ways how to visualize quantitative modelling results in the most understandable and trustworthy way [96].

Evaluation
It is not just about results or outputs, the outcomes and impacts of the modelling process/research project also deserve greater attention.Those who have been involved in the participatory elements should be given the opportunity to reflect and provide critical feedback on the process.Participants might be asked in a follow-up survey or interview; how well they felt their voices were heard?How might the structure of the engagements be improved?Or are there any noted benefits to being involved in the process?This naturally adds to the time burden for stakeholders, so the research team should carefully assess and plan the way in which this input is elicited, so that they may evaluate the process itself in addition to the outcomes.Recognising the time investment that participants contribute to the research, it is important to evaluate the value they get out of the process.This is not just about finances (which often cannot be transferred, Section 4.1), but rather other noted benefits such as mutual learnings [97], having access to energy expertise or reports and other outputs [35].
Moreover, there is also the important question of what the impact of the research project was?To what extent are the modelling insights translating into real-world actions and policies, does it led to better decisions?Evaluating the contribution of energy systems modelling is a critical way by which to improve the practice (Section 6.6).

Good practice principles
As introduced in Section 4, participatory approaches can be messy and difficult to navigate.Recognising this, and drawing from the collection of case studies in Appendix A2, we outline here a set of good practice principles to assist scholars considering integrating participatory methods into their modelling process (Table 3).

Stakeholder mapping -Be clear which stakeholders need to be involved in the different stages and what the purpose of engaging them is
Treat stakeholders time as a limited resource that must be respected.A good practice in participation is the "functional-dynamic approach" from the field of risk management (as introduced in Section 2.3).Stakeholders should be involved only with a specific purpose/function and this depends on the stage of the process.Researchers must be critical of who is being engaged?And why?What are the different roles (e.g., modeller, stakeholder, non-expert, broad public) and when is it most appropriate to bring in the different perspectives?In addition, it is important to build an understanding of the relevant stakeholders' interests in order to align with them.Without this, researchers may struggle to get support in further phases or other projects.Focus on endusers' or stakeholders needs.Who are the potential and practical endusers of the results provided by the energy systems modelling?What are their needs?And how can we support them?
Be clear about your target audience and tailor your engagement.Involve stakeholders throughout the research process and as often as possible where appropriate.It is important to manage expectations at the start of the process, agree shared goals, be clear about the commitment required, and later, communicate with those involved how their inputs have been used in the research [98].
Flexible/adaptive approachrespond to stakeholder needs, be open to changing the research design as the project evolves.
An essential part of working collaboratively is to allow input into the research design, and as much as possible, adapt the process to the needs of the stakeholders.This flexibility creates tensions with conventional project management approaches but is key to a proper participatory process [98], and ensuring that the research achieves real-world impact [35].
It is essential to be agile in the engagement process.Unforeseen events may require changes in methods and formats or modifications to the project schedule.Moreover, following established best practice in transdisciplinary research, the approach taken should be agreed with the stakeholders involved.This may mean a significant revision to the research plan.However, such revisions should be made after reaching some level of consensus among diverse stakeholders in order to avoid particular stakeholder groups shifting the research agenda to their interests (see also the last principle at the end of this Section).

Acknowledge model limitationsbe clear about what models can say and what they cannot say
It is important to be clear throughout the process that models as technical or techno-economic representations of the problem will be missing broader societal and macro-economic elements.There is a risk that important points raised by stakeholders are not represented in the results as they do not fit neatly into the energy system model.While stakeholder participation is often framed as an important next step to minimise the gaps in energy systems modelling, it may in fact expose new areas of uncertainty and model limitations.Being critical of how well the stakeholder concerns or priorities can be modelled can point to blind spots in the analysis.
Take an interdisciplinary approach to overcome the limitations of individual methods.Models are unlikely to be able to address all research questions and needs of stakeholders.Therefore, greater collaboration with social, economic, and environmental scientists may be useful to minimise the limitations of techno-economic modelling and consider the multiple dimensions of the energy transition.

Be respectful of divergent views
Energy and climate issues are highly contentious wicked problems, and there is thus a need to bring together a diverse range of perspectives in order to form a joint understanding of both the problem and potential interventions [99].However, such a broad range of worldviews coming together for discussion will likely mean that consensus cannot be reached, particularly when the engagements take place over a limited timeframe.As cautioned by [40, p. 43] 'there is a need for caution about how such processes are structured, and what claims are made arising from them'.The pursuit of consensus, particularly over a limited timeframe, risks oversimplifying the complex societal dynamics at play and shutting out some of the voices in the room.Design for inclusion is an important element of good participatory practice [98,100].

Table 3
Good practice principles for participatory energy systems modelling.

1)
Stakeholder mapping -Be clear which stakeholders need to be involved in the different stages and what the purpose of engaging them is 2) Flexible/adaptive approachrespond to stakeholder needs, be open to changing the research design as the project evolves 3) Acknowledge model limitationsbe clear about what models can say and what they cannot say 4) Be respectful of divergent views 5) Be critical of the engagement process and possible biases It is important to recognise that divergent or conflicting views are a valuable contribution to the process as opposed to a difficulty to be overcome.Managing these issues within engagement processes requires proper skilled facilitators, who may be best coming from outside the research team.In terms of the modelling, a constructive way to address these different perspectives is to treat them as a kind of uncertainty, running (when feasible) different modelling exercises for each, and comparing the results.The opening up of the energy systems modelling process to new stakeholders is likely to produce more questions than answers.Issues or concerns raised by stakeholders that do not fit neatly into the energy system model, while a serious methodological challenge, should be given the same attention as model derived insights.
Be critical of the engagement process and possible biases Be critical of the process and claims arising from it.Participation is not a panacea, to avoid 'box-ticking' exercises it is good to challenge your motivations for involving stakeholders.What is the rationale for participation?How does it benefit the stakeholders involved?How does it contribute to the research results and outcomes?
Secondly, in doing participatory research, the researcher becomes much more embedded within the process, so it is essential to be reflective and conscious of bias [45].How strong an influence do the actors running the process have over the outcomes?How can this be minimised?Moreover, there is need for clarity on who is participating in the process (Section 6.1).There may be certain stakeholders such as industry lobbyists that want to be involved to steer the processes toward a particular outcome.Conducting a simple assessment of the relative power held by stakeholders, and their interests in taking part, is likely to be important [100].

Key issues
In this section we reflect on the key issues within this emergent field.This outline of unresolved challenges serves to highlight important areas for future research in order to advance the field of participatory energy system modelling.

Are we engaging a diverse range of stakeholders?
A major challenge lies in ensuring that the 'right people' contribute in stakeholder engagement.Policymakers and experts from industry or academia are some of the most common stakeholder types involved [39,101], but the systematic participation of citizens is a key gap in energy systems modelling.As citizens are directly affected by the impacts of energy infrastructure (e.g., local air quality, global climate change) as well the outcomes of energy transition policies (e.g., changes in energy costs and reliability), every citizen can be assumed to be a stakeholder [102].
Citizen engagement methods are crucial to ensure that a broad range of perspectives is used to form a shared understanding of what constitutes a desirable future [103].While population surveys, citizen panels, and other methods have been extensively used for more than two decades in energy research [104], their results have been rarely used by modellers to inform their scenarios.This can potentially cause the public to distrust modelling studies and, subsequently, transition policies based on modelling outcomes [101,105].The inclusion of voices from marginalised communities is particularly important to ensure a fair and just transition [106].While recent studies have attempted to elicit citizen preferences in scenario formats that can be easily modelled [26,27], further efforts are required to make citizen participation widespread in modelling studies.
Within this issue, there is often a blurred distinction between the inputs of stakeholders and those of the project team [65].Generally, it is hard to detach members of the project from the outputs of engagements if those facilitating the discussion are also part of energy systems modelling.Additionally, the reported backgrounds of participants is often kept vague and the recruitment process is rarely discussed in publications of modelling studies [39], leading to doubts on whether reported stakeholders bring truly external viewpoints or they replicate the ones from within the project team.This critical methodological problem needs to be further examined while future energy systems modelling studies need to be clearer on who is involved in engagement efforts.

The choice of model
A significant amount of time goes into building energy system models, and in transferring their insights into useable policy recommendations.There is thus often a lot of institutional inertia that will limit the scope of model choice.The modelling tool being used is often passed on by supervisors or research groups, particularly for early-career researchers, a key consideration is to select a modelling tool that others in the group know how to use so that help and guidance is available [81].In addition, as noted in Section 4.1, many decisions are already made at the funding proposal drafting stage.This means the model to be used for analysis will likely be determined before reaching out to stakeholder groups.This then risks limiting the input that is sought, and more critically may not align with the needs of the stakeholders.When working at the local community level, simple models are preferred [35].
Modellers may be best placed to understand which models will be most appropriate to answer certain research questions, but a diverse range of perspectives should be involved in identifying those research questions.This may present issues if the questions raised fall outside of those usually considered by the modellers and what the preferred models can accommodate.It is important stakeholders have an opportunity to understand what knowledge models can and cannot provide, and that there is a process for handling key concerns that don't fit in the model (Section 6.4).

Unpacking the assumptions and results
Despite calls for opening up the methods and data of energy modelling [3,[52][53][54][55], the underlying assumptions that determine modelling outputs are often opaque [56], potentially undermining their conclusions.Similarly, there are no guidelines or standard practices for managing the translation of stakeholder inputs into models, identifying when it is not appropriate to do so, and handling inputs that do not fit into the model (following Section 6.4).While efforts to develop open-source models are to be commended, more work is needed to make their assumptions and limitations accessible to non-modelers [33].Transparency within these processes becomes even more pertinent as there have been cases of modelling assumptions that were politically influenced to support desired policy outcomes, giving the impression that "quantitative evidence is for sale" ([107], p. 7).
Unpacking the underlying assumptions within energy system models and what exactly goes into the model is a very subjective process [56], steered by modeller choices and rarely discussed outside the modelling team.Unless working with domain experts or other stakeholders familiar with energy systems modelling, it will likely be difficult to get stakeholders interested in these technical discussions.However, it is critical to find ways to unpack key assumptions in order to understand why results are what they are.
Some opportunities for exploring this area are the use of either interactive computer tools or other serious games approaches that allow stakeholders to choose their own scenarios.For instance, several studies have integrated scenario-building tools into workshops with nonmodellers to help them understand trade-offs in scenario development (e.g.Refs.[89,108,109]).However, when Xexakis & Trutnevyte evaluated a similar scenario-building tool, they have shown that its effectiveness may be potentially constrained by information overload [110].Thus, it is important for future research to find ways to inform non-experts about modelling processes without introducing significant complexity.Additionally, as stakeholders may misinterpret modelling results [111,112], it is imperative to present key messages in ways that reduce ambiguity as much as possible.

What is missing from the models?
A significant challenge is how to handle stakeholder inputs and interests that do not fit in the model?There is a serious risk that perspectives or concerns that cannot be easily included are thus dismissed.The model becomes a tool to shutdown discussion on difficult issues and limit the scope of the participatory process.
There is an imperative for modellers to be brave in facing up to the limitations of their approaches and work with scientists from diverse areas of expertise to explore questions that cannot be answered by the model.An exciting area for research is the development of approaches that complement traditional modelling/outputs and handle elements that cannot be modelled.The focus should not simply be on making better models but rather asking how to make better use of models and the insights they can provide.
This paper has focused on modelling with participation but there is also just participation without modelling, which can be very valuable as well.Deliberations that are not constrained by a modelling process may produce richer outputs [113,114], and the process of bringing stakeholder perspectives together may in fact be a more important outcome than any set of modelling results [115,114].The various ways in which the public can participate in the energy transition is diverse and shaped by a variety of factors [14].This demands we take a broader view of participation beyond the models [35].However, the critical question is then how to bridge modelling insights on transition pathways and more participatory or action research approaches in order to develop effective policy.

Communicating complexity and uncertainty
Energy system models firstly seek to represent a highly complex systems, and then from this attempt to make projections into the future.The process of building and using an energy system model can help users gain a new understanding of the complex dynamics of the systems they are modelling.This spill over of understanding should be available to all stakeholders involved in the model development process, though careful communication of the complexity is required.Modelling a complex system requires trade-offs, e.g. between comprehensive representation and the necessary simplification to make a system tractable, but also due to knowledge limits, i.e. uncertainty.There are several key areas of uncertainty around both model parameters such as cost of technologies, deployment rate, availability, etc. (generally referred to as scenario uncertainty) and also model structure (generally referred to as structural uncertainty).Uncertainty in model structure is particularly problematic given the complex nature of energy systems, in which there is no autonomous control over the whole system, and self-organised emergent behaviour arises that cannot be predicted by understanding each of the component elements separately [116].A recent review by Few et al. found that even studies addressing deep uncertainty often only account for scenario uncertainty [117].When structural uncertainty is taken into account, this is typically considered only in terms of parameter ranges.Developing alternative versions of model structures accounting for diverse understandings of model structures amongst diverse stakeholders could begin to address this uncertainty.
Uncertainty is an issue that is widely acknowledged in the literature but rarely assessed [5].Nemet et al. highlight that experts tend to be overly optimistic regarding the field or technology they are involved in, which leads to an overconfidence bias in future cost reductions and rate of deployment [118].There are analytical approaches for addressing uncertainty in key model parameters such as Monte-Carlo or Global Sensitivity Analysis [5,119,120].In addition, the discussion of uncertainty with stakeholders such as policymakers who make use of modelling results is important [121,122] (see also Section 6.3).It has been shown that stakeholders' perception of energy scenarios plausibility will vary based whether a scenario corresponds with users' own beliefs and expectations [123].
Ensuring the appropriate use of energy systems modelling insights requires discussion on complexity and uncertainty issues.This should include the different types of uncertainty (as outlined here) and also how uncertainty can be used by different stakeholders seeking to influence policy.For example, some might downplay uncertainty to promote radical agendas whereas others might overemphasize uncertainty to prevent a certain course of action [124].However, the process of discussing uncertainty in detail is at odds with the limited time stakeholders have to contribute to modelling processes and the desire for clear and precise answers to inform policy, decisions and/or actions.

What are the outcomes or impact of the process?
As outlined in Section 4.5, an important area that warrants further investigation is to evaluate the participatory energy systems modelling process.Some interesting questions for future research to explore would be.
• What works and what does not when stakeholders are involved?
How can the process be improved?• What are outcomes of the participatory process?Are stakeholders more or less interested in the use of models?Have they benefited from the process?What are their key learnings?• How does the participatory process change the model or scenarios modelled?What can models show us and what are their limitations?• Are modellers more or less convinced of the value of their model?What are their key learnings?• What is the impact of the process?Does it inform decisions, policy or actions?
These reflections would be important to further develop the practice.In addition, besides evaluation of processes that sought to engage stakeholders as part of the modelling process, there is also the question of how energy scenarios are taken and used by different stakeholder groups [125].

Closing remarks
The critical role that energy system models play in energy and climate policy decisions means the process through which they are developed must be transparent and should seek to include a diverse range of opinions on what a desirable future means.However, pursuing meaningful involvement from a diverse range of stakeholders is no easy task for modellers.It takes work to help people understand the model well enough to engage with it meaningfully and critically.Moreover, it takes time for modellers to become good listeners, able to understand the concerns and issues raised by participants and not simply dismiss things that do not align with their (mental) model(s).
Inter-and transdisciplinary approaches offer an exciting prospect to address some of the key weaknesses in energy systems modelling.However, the proper funding/resourcing of stakeholder engagement remains a critical constraint.There is a growing call for greater stakeholder involvement or co-creation within funding programmes but the time required and openness needed to be flexible and responsive to stakeholder needs are underappreciated.This poses a serious risk to the reputation of such approaches as it may result in bad practice or 'box ticking' exercises.In parallel, there are considerable barriers to meaningful collaborations between the energy systems modelling community and other fields.The challenges associated with inter-and transdisciplinary research are well documented.As an emergent area seeking to integrate these practices, participatory energy systems modelling inevitably inherits many of them along with the additional issues inherent to the energy systems modelling process (such as translating qualitative inputs into quantitative parameters, understanding model complexity and uncertainty).
Despite these difficulties, the need for researchers to find creative ways of working together and with a diverse group of stakeholders is nonetheless clear.The framework developed in this paper offers multiple entry points for modellers to bring in participatory elements either throughout the process or at different stages.Recognising the messiness of both fields, the good practice principles offer important questions to steer this process.Finally, the reflections on key issues provide a summary of the crucial challenges and important areas for future research in this critical field.The further development of communities of practice around participatory energy systems modelling approaches is an important way in which research can support the transition to a climate neutral future in a fair and just manner.

Fig. 1 .
Fig. 1.Energy system model overview example detailing inputs and outputs, based on an energy system optimization model, one of the most commonly used.

Table 1
Summary of common stakeholder engagement approaches within energy systems modelling.