Decarbonization of industry: Guidelines towards a harmonized energy efficiency policy program impact evaluation methodology

The decarbonization of EU energy system is under way, but manufacturing industry is still using approximately 25% of the EU total final energy use. To maintain long-term competitiveness while contributing to the EU goal of carbon neutrality by 2050, manufacturing industry needs to improve energy efficiency in a cost-effective way. One important way to achieve this is through energy audits. The Energy Efficiency Directive promotes member states’ development of energy efficiency programs to encourage industry to undergo energy audits. Previous studies have reviewed industrial energy efficiency policy program evaluations and argued that there is no harmonized way to conduct them. This leads to difficulties in: i) comparing energy efficiency and cost saving potentials throughout different programs, and ii) providing necessary information that supports the improvement of the policy program. Therefore, we argue that a harmonized methodology for industrial energy efficiency policy program evaluation is of great importance, and, we have developed a set of five-steps guidelines that lay the foundation for an ex-ante energy efficiency policy program evaluation methodology. The guidelines are to be be conducted during the lifetime of the program, in five steps, as follows: (s1) define key issues, (s2) set the objectives for each key issue, (s3) identify the options for each key issue, (s4) analyze options from an energy and environmental perspective, and (s5) compare options and select the recommended one. Our proposed methodology will support policymakers and evaluators answer questions such as: i) how can the objectives of the policy program be achieved? ii) is there any need to change the policy program? Furthermore, a comparison in terms of relevance, efficiency, effectiveness, and sustainability of all major policy options developed, including the status quo option is proposed in the methodology. This paper can be seen an important step towards the goal of creating a harmonized policy evaluation methodology. © 2021TheAuthors.PublishedbyElsevierLtd.ThisisanopenaccessarticleundertheCCBYlicense (http://creativecommons.org/licenses/by/4.0/).


Introduction
Energy efficiency is seen as the ''the first fuel'' of a country, because it is the one energy resource that all countries own in abundance, and it is the key for cost-effective energy transition to decarbonization (International Energy Agency, 2018).EU 2030 climate and energy framework projects energy efficiency as one of EU's first fuels in each of the 2030 decarbonization scenarios, since the sum of energy savings and renewables will overtake the sum of all imported fossil fuels (Saheb and Ossenbrink, 2015).It is projected also that energy savings will contribute, in the long term, to Europe's energy self-sufficiency.Even more, EC claims that it is necessary to rethink energy efficiency as being an energy source in its own right as it represents the value of saved energy (European Commission, 2015).
The Energy Efficiency Directive from 2018 amended the 2012 Directive to pursue the overall objective of at least 32.5% energy efficiency target by 2030, which is to be achieved collectively across the EU.In the last years, EU targets for energy efficiency have been strengthen from 20% in 2012 to 27% in 2014 until the current target of 32,5% in order to meet the Union's international commitments for decarbonization.To reach the 32,5% target for energy efficiency, Member States are expected to set their national indicative energy efficiency contributions considering that the Union 2030's primary energy use has to be no more than 1 273 Mtoe (Million Tons of Oil Equivalent).Therefore, a regular evaluation of progress towards reaching the 2030 target is deemed necessary by the EU.(European Parliament and the Council of the European Union, 2018).
Furthermore, studies have shown that energy efficiency contributes to climate change mitigation, and although significant progress has been made in recent decades, industry needs to continue improving energy efficiency in order to maintain competitiveness and contribute to achieving EU goal of carbon neutrality by 2050 (Boza-Kiss et al., 2017;Brunke et al., 2014;European Commission, 2012;Sorrell, 2005;Sorrell et al., 2000).In Europe, industry accounts for approximately 25% of the total final energy use (Eurostat, 2019), and achieving long-term competitiveness is essential.One key requirement for achieving long-term competitiveness is to improve energy efficiency in a cost-effective way.
A gap between the cost-effective energy efficiency measures and the actual implemented energy efficiency measures has been identified and is referred to in the literature as the 'energy efficiency gap'.The energy efficiency gap is explained by the existence of various market failures and barriers to the adoption of cost-effective energy efficiency measures (Backlund et al., 2012b;Brown, 2001;Fleiter et al., 2011;Jaffe and Stavins, 1994;Rohdin and Thollander, 2006;Sorrell et al., 2004Sorrell et al., , 2000;;Thollander et al., 2007;Thollander and Ottosson, 2008).Market failures or market imperfection are deviations from a perfect market, justifying a public policy intervention in order to overcome these failures, i.e. see (Jaffe and Stavins, 1994;Sorrell et al., 2004) for a thorough description of this.
Therefore, one important action is to raise awareness within industry of the potential positive impacts of energy efficiency measures, and this can be approached through energy audits, as they quantify energy use and promote further actions to improve energy efficiency.An energy audit is defined as a systematic procedure for the analysis of energy use within a defined energy audit scope, performed in order to identify, quantify and report on the cost-effective energy savings opportunities (European Commission, 2012;International Standard Organization, 2014).
Energy audits have been shown to be a useful tool for accelerating investments in and implementation of energy efficiency measures (Backlund and Thollander, 2015), thus helping companies to become more energy efficient and contribute to CO 2 emissions mitigation.The formal report from the energy audit presents the findings related to energy efficiency potential improvements, estimated energy and cost savings, and estimated costs for the implementation of measures.There are three levels of detail of the energy audits that an organization can choose from depending on their needs.Level 1 audit represents the minimum level of detail for an energy audit conducted for facilities or processes and is suitable for smaller organizations or as a preliminary audit for larger organizations or facilities.A type 2 level for audit is a more detailed energy audit conducted for a single site or process and is a cost-effective solution for organizations with bigger energy budgets.Level 3 of detail for an energy audit is performed for a whole site, process, or system (e.g., compressed air) and is a comprehensive audit with significant input from the organization.In terms of costs, level 3 audit is generally cost effective for organizations with high energy spending's or with targeted capital investment grants.These types of audits are not absolute requirements and depending on the needs of the organizations, the level of detail may be adjusted between type 1 and type 3.The appropriate level of detail required for an energy audit depends on the objective of the audit, the energy uses and the available resources for the audit.(International Standard Organization, 2014) Throughout the years, several energy efficiency policy programs have promoted energy audit programs for industry.The public energy audit programs can be seen as the first generation of energy efficiency policy programs for industry.A second generation or level of industrial policy programs is voluntary agreement programs (VAPs), which include energy management components (Thollander et al., 2020).This category also includes energy efficiency networks, where companies receive support in network form for energy auditing and energy management activities.In an energy efficiency network, companies create a group coordinated by an external specialist with the scope of gaining knowledge about a particular topic and sharing experiences about energy efficiency.The companies can also establish a mutual goal and work cooperatively to achieve it (Paramonova and Thollander, 2016a).By collaborating in an energy efficiency network, companies might reduce transaction costs, minimize risks and increase awareness about energy efficiency (Köwener et al., 2011).
While energy audits track energy usage and efficiency to identify key areas for improvement, the energy management process ensures the implementation and follow-up of these actions for improvement.The process starts with an energy audit to identify opportunities for improving energy efficiency, after which energy management involves taking the energy audit and putting it into action with a number of strategies, monitoring progress against targets with a permanent improvement approach.(European Commission, 2012) Several evaluation studies of different national energy efficiency policy programs have been conducted with the aim of gaining knowledge on the success of an energy policy program.Price and Lu (2011) reviewed 22 energy audit programs around the world from 15 countries.Price (2005) reviewed 23 voluntary agreement programs (VAPs) around the world from 18 countries.Tanaka (2011) reviewed some 300 industrial energy policy programs within IEA countries.Thollander et al. (2015b) reviewed policy programs for SMEs in four countries, and Thollander et al. (2015a) compared energy efficiency policy programs from Japan and Sweden.Further, Johansson et al. (2019) reviewed the scientific publications related to energy efficiency policies for industrial SMEs.
Related to international policy program evaluation, Andersson et al. (2017) performed a study to compare how energy audit policy program evaluations were conducted and how their results are presented.The results of the study show that there are differences in both how the evaluations are performed and how the results are presented.There is a lack of consistency in how the measures are categorized, leading to difficulties in comparing the energy efficiency and cost saving potentials for one category across several programs.And, as Weiss (1998) underlines, a proper comparison of multiple energy efficiency programs is possible to be conducted if there is a certain degree of similarity in terms of measures, goals and activities.
At this point, one may ask what an evaluation of policy or program might look like, and what its intended use is.A definition of evaluation is given by Weiss (1998, p. 4) as follows: ''Evaluation is the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy''.(p.4) Weiss (1998) emphasized five key elements in this definition.The first is systematic assessment.The focus here is on the system, indicating that, whether the research is quantitative or qualitative, it should be conducted with formality and rigor.The second and third elements point to the focus of the evaluation -the operation and/or outcomes of the program.The fourth element is standards for comparison, where the evaluation assesses the program by comparing the evidence with some set of expectations.The fifth element is the very purpose for which the evaluation is performed: to contribute to the improvement of the program and policy.
The use, expectations and contributions of evaluation are key features that need to be discussed.The use and expectations of an evaluation differ with the person's position in the system.For example, top policymakers are interested in the overall effectiveness of the program and ask for information that will help them address broad issues related to the financing, continuation, modification or ending of a program.Legislators are interested if they can initiate new programs and policies and change or terminate old ones.
In practice, evaluation is mostly used to help with decisions about improving the programs, and only rarely with taking 'gono go' decisions.Even when the results of an evaluation show the program to be a failure, the information is used to help with trying again.However, the irony is that evaluation methods are good for assessing the overall impact of the program, which is suited to the rare 'go-no go' decision, but are less developed as tools for understanding how and why programs achieve their impact, or how the programs can be improved.Also, people usually need the evaluation to lead the way towards constructive change by providing the necessary information that will help them to modify the program (Weiss, 1998).
Furthermore, when discussing potential uses and users, one should be aware of and distinguish between formative and summative evaluation.Formative evaluation is performed ex-ante (during the policy program's lifetime), and is useful for program developers, since it is designed to assist them from the early phases of program development until the end, when ex-post evaluation is conducted.Summative evaluation is conducted expost (at the end or after a policy program) and is useful for decision-makers, as it provides the information needed to support decisions about whether to continue or end a program, extend it or cut it back (Weiss, 1998).A more explicit discussion of summative and formative evaluation is found in Section 2.
Regarding the contributions of evaluation, it has been shown historically that, by providing objective information about the implementation and outcomes of the program, policymakers can make rational and wise decisions on program planning and budgeting.Therefore, we argue in this paper that a harmonized methodology for industrial energy efficiency policy program evaluation studies must be seen as being of great importance.
The aim of this paper is to create a harmonized methodology for an ex-ante evaluation of energy efficiency policy programs that will support the industry's transition towards decarbonization.Therefore, we have developed and present a set of guidelines for a harmonized methodology that will support policy makers and evaluators to answer questions such as:

√
How can the objectives of the policy program be achieved?

√
Is there any need to change the policy program?
Even though these guidelines can be applied to any industrial energy efficiency policy program, the major emphasis is on the most common and internationally recognized type of policy, namely industrial energy audit programs.
The method of this paper can be divided into two parts: one part consists of a literature study on the types of energy efficiency policy programs, energy audit standards, and policy process and evaluation.The second part consists of developing the guidelines for creating a methodology for the evaluation of energy efficiency policy programs, based on two existing analytical frameworks: corporate management and energy management.
The structure of the paper is as follows: Section 1 provides an introduction to energy efficiency targets in EU, energy audits, policy evaluation and states the aim of the paper.It continues in Section 2 with providing a background on the types of energy efficiency policy programs, policy process and evaluation.Section 3 presents the guidelines for the methodology of industrial ex-ante energy efficiency policy program evaluation (hereafter referred to as ex-ante EEPPE).The paper ends with a concluding discussion in Section 4.

Types of energy efficiency policy programs
In European industry, more than 40% of the energy end-use emanates from businesses within the energy-intensive sectors.Energy efficiency policies have been deployed in EU member states for both energy-intensive and non-energy-intensive industries.In Sweden, for example, policies and programs were directed towards medium-sized, energy-intensive SMEs and small, non-energy-intensive SMEs.For medium-sized, energy-intensive SMEs, the Swedish Energy Conservation Act, long-term agreements and voluntary agreements were adopted (Thollander et al., 2014).
One of the most commonly deployed policies is voluntary energy audit policy programs for SMEs (Andersson et al., 2017).Another mandatory policy approach is the legislation in EU member states for companies that are not SMEs to conduct energy audits every fourth year in an independent and cost-effective manner (European Commission, 2012).Another approach, albeit less common, is voluntary agreement programs (VAPs), e.g. the Swedish program for improving energy efficiency in energy-intensive industries (PFE), where mandatory energy audits were combined with the implementation of a standardized energy management system (Swedish Energy Agency, 2011).A brief description of the general types of energy efficiency policy programs implemented in EU member states is presented in Table 1.Price and Lu (2011) identified several energy efficiency programs implemented in Scandinavia (see Table 2): It is very important to identify, evaluate and measure performance and potential improvements in relation to energy efficiency, and one common energy performance indicator used for this is specific energy consumption/use (SEC).According to ISO 50006:2017, SEC is an energy performance indicator used in the process of measuring the energy performance in an organization (International Standard Organization, 2017).The results of energy performance can be expressed in SEC as kWh/unit.Also, both in SS-EN 16212:2012 and SS-EN 16231:2012 standards, SEC is used as an indicator of energy use per (physical) unit of output, relating the annual energy use to annual physical production (Swedish Standard Institute, 2012a,b).SEC shows how much energy is used to produce a unit of product.It is also used to evaluate changes in energy efficiency in industry, and for benchmarking at different levels, including process, site, national and international levels (Andersson et al., 2018).Lawrence et al. (2019) conducted an analysis on the meaning, usage and differences regarding SEC in industrial energy efficiency, and discussed the pros and cons of using SEC in this context.One main point discussed in the paper relates to the various factors influencing SEC, such as age of equipment, production rate and environmental conditions, cautioning against the use of SEC as an indicator for monitoring improved industrial energy efficiency (Lawrence et al., 2019).(1995) states that policy analysis has different objectives and relationships to the policy process, and comprises a range of activities on a spectrum of knowledge in the policy process, as follows:

Parsons
• Analysis of policy process: how it is being implemented, how problems are defined, agendas are set, and policies are formulated and evaluated.
• Analysis in and for policy process: includes the use of analytical techniques and research in problem definition, decisionmaking, evaluation and implementation.When studying public policy, a meta-analysis considers using different methods and approaches.Wayne Parsons describes metaanalysis as an ''. . .analysis concerned with the activity of analysis (p. 1)'' (Parsons, 1995).Moreover, Parsons (1995) argues that the analysis of public policy uses open and 'hygienic' models as devices to explore and to form a critical awareness of their assumptions, origins and significance.
A policy analysis involves different disciplines (having taken on a multidisciplinary character recently), theories and models, and analysts have several concerns related to the processes of a policy, that is to the: • received inputs from the environment, in the form of perceptions, demands or need for support, mediated through channels such as parties, media and interest groups, organizations • content of public policies, including regulations, re/distribution, capitalization and ethical ruling • consequences of policy in terms of outputs and outcomes, in addition, analysts focus on several specific stages of the policy process, such as policy formulation, implementation, and evaluation.Therefore, the policy process can be viewed in different ways, one being in terms of received inputs, demands within the political system that leads to policy, and the transformation of inputs into policy outputs and outcomes (see Fig. 1).
In the process of evaluation of a public policy program the aim is to determine whether the program activities have been implemented as intended and resulted in certain outputs.In this way, the policy operator or policy administrator can receive feedback on the various processes involved in the policy program operationalization (Weiss, 1998).
The outputs can be divided into two groups: short-term and long-term impacts.The short-term impacts are useful for decisionmakers, as they provide recommendations regarding opportunities for reform.The long-term impacts help ensure that public officials are socially accountable to their industries, thus giving them the opportunity to pursue continuous improvement of energy efficiency programs towards increased energy efficiency.
This approach helps to create windows of opportunity for changes in current programs, and even future policy program reforms.

Policy evaluation
Policies, plans, and programs form a hierarchy, with policies at the top, followed by plans and programs at the lowest level of the hierarchy.Programs make plans more specific by including details on an array of projects (Ahmed and Sánchez-Triana, 2008).Evaluations can be directed at any level in this hierarchy.

Summative and formative policy program evaluation
An evaluation can be undertaken in a summative manner, meaning that it is summarized in a report or another format, or in a formative manner.The summative evaluation is conducted ex-post (at the end of a policy implementation) with a focus on identifying effectiveness, measuring and documenting quality indicators for decision-making.The results of the evaluation can be used to improve the performance of future policies.The evaluation process focuses on quantification and experimentation, meaning that the policy is tested on a control group and a test group (Parsons, 1995).
Summative assessments provide a means for empowering those who are implementing energy policies by determining the degree to which the expectations (objectives) of those policies are met.Because summative assessments are a central component of measuring the effectiveness of energy policies, the high stakes require that these assessments are valid and reliable.A summative evaluation can provide the following information:

√
Comparative data for making decisions regarding further implementation of the energy policies,

√
Information on the strengths and weaknesses of energy policy performance.
The formative evaluation is performed ex-ante, during the policy program's lifetime, with the aim of evaluating the trends in results, determining whether the goals of the program are likely to be fulfilled, and providing feedback on the strengths and challenges of the program (Janus and Brinkman, 2010).The results from the evaluation are used to further develop and streamline the program and to improve program implementation, i.e. the results form a different design to some extent.
When conducting a formative evaluation, an exploration of whether the program is reaching the targeted population and how resources are being managed can be included.This could be done using different financial management techniques.The usual emphasis of the evaluation is on measuring the performance and the need to control public finances so that higher efficiency and effectiveness are reached (Parsons, 1995).Formative evaluations are designed to guide policy implementation and generally are individualized assessments that are under the control of competent institutions and target specific issues or concerns about policy implementation.
Unlike summative assessments, formative assessments may include any attempt to obtain feedback to improve policy implementation during the implementation process.In brief, formative evaluations can be presented as a way to track program information related to 'when?', 'what?' and 'why?' questions (Parsons, 1995;Salabarría-Peña et al., 2007):

√
When to use it?
• Ex-ante: anytime during the lifetime of the policy or program, • When an existing policy is being modified or is being used in a new setting, • Progress in delivering the results of the policy, • Awareness and knowledge about the strengths and challenges related to specific implementation concepts, • Guidelines for improving strategies for implementing the policy, • Information on the impact of energy policies under implementation.

√
Why it is useful?
• The results of the evaluation are used to further develop, improve and streamline the policy or program, • Maximizes the likelihood that the policy will succeed, • Provides feedback on the effectiveness of current strategies of policy implementation, including on the policy's strengths and challenges, • Monitors the value and impact of implementation practices, • Monitors the progress or increases the efficiency of implementing a policy, • Identifies and adapts to the challenges that occurred during the implementation period, • Allows for modifications to be made to the objectives before full implementation ends.
In order for the formative evaluation to be effective, it needs to be targeted for clear purposes, provide feedback that allows action reviews, and be implemented in a timely manner to allow for revisions in policy implementation.

Energy management as framework for developing the guidelines for ex-ante energy efficiency policy program evaluation methodology
Energy management and corporate management frameworks are used to construct the framework of our methodology.The starting point of our methodology is found in the managerialist framework discussed by Parsons (1995).Parsons' managerialist framework is characterized by the actions designed to improve the efficiency, effectiveness and economy of the public sector through the adoption of several techniques that are considered appropriate only for the private sector.Since the management of the public sector has striven to become more 'business-like', due to pressure to deliver cost-cutting, value for money and a business-oriented culture, the use of private-sector approachessuch as operational management or corporate management -has become the dominant paradigm in the administration of public policy.
Therefore, we continue to develop our guidelines based on Parsons (1995) corporate management approach, which focuses on the analysis of management problems in a strategic and planned way.Nevertheless, the guidelines for the EEPPE methodology includes part of the sub-objectives of energy management, as proposed by Capehart et al. (2016, pp. 1-2), in their guide to energy management, as follows:

√
To improve energy efficiency and reduce energy use, thus leading to reducing costs,

√
To reduce GHG emissions and improve air quality,

√
To cultivate good communications on energy matters,

√
To develop and maintain effective monitoring, reporting, and management strategies for wise energy usage.

Outlining the guidelines for developing ex-ante energy efficiency policy program evaluation methodology
The guidelines for ex-ante EEPPE methodology are inspired from the policy impact assessment technique, which is a logical analytical process, conducted during the early stages of a policy-making exercise in order to identify the best solution for tackling problems related to policy impact.It is made up of a set of steps, conducted in a participatory manner, that provide decision-makers with valuable empirical data and an appropriate framework to assess their options and the possible consequences of their decisions.It also supports the decision-making process when choosing the best policy options.Draskovics (2018) recommends using policy impact assessments on policies or programs that introduce significant changes and address critical areas such as energy.
Therefore, we have used the ex-ante policy impact assessment key steps proposed by Draskovics (2018) to build our guidelines.Fig. 2 presents the set of guidelines consisting of five key steps to be conducted when performing an ex-ante EEPPE.
The proposed steps, together with the subsequent activities, should be performed in two different phases of program implementation, i.e., the planning phase and the operational phase, as presented in Fig. 3.The results from phases 1 and 2 support the policy makers and evaluators in the third phase, i.e. postoperational phase, in deciding whether the objectives of the program can be achieved from an energy and environmental perspective and if there is any need to intervene in the program.Therefore, positioning the activities throughout the program timeline provides the necessary support to answer the 'when?', 'what?' and 'why?' questions discussed above.
Guidelines steps 1 and 2 will help answer the first question regarding how to achieve the objectives of the program, and, steps 3, 4 and 5 will help determine whether there is a need to either intervene in or change the program, as presented in Fig. 4. Therefore, in order to answer these questions, we provide an analytical tool that can be used in different cases, regardless of the type of industry and the complexity of the program, since exante energy efficiency policy program evaluation is an analytical  tool that can be used for clear purposes and specific issues or concerns about policy implementation.
Our recommendations when conducting an ex-ante EEPPE are as follows:

√
The sequence of these five interconnected steps is essential; however, one should consider taking steps backward and forward and revising the findings in both previous and subsequent ex-ante EEPPE steps,

√
Strive to reach the objective of each step by using primary data, both qualitative and quantitative, since this is the key for better policy analysis and decisions.

Guidelines for developing a harmonized methodology for exante industrial energy efficiency policy program evaluation
As described in Section 3.2., the guidelines developed in this paper are structured in five steps with subsequent activities that should be performed in the planning and operational phase of the program.By mixing components of corporate management and energy management, we have identified and developed five key components that the ex-ante EEPPE methodology will focus on addressing: 1. Energy and GHG estimation, designed to identify the primary energy factors for major energy carriers, 2. Target group estimation, designed to estimate the average annual energy use divided into sub-categories, and communication on energy matters, 3. Energy efficiency estimation, designed to identify the increase in energy efficiency, energy related costs, and the reduction of greenhouse gas emissions, 4. Non-energy benefits estimation, designed to identify and estimate other factors that affect the program in a positive way, thus having the potential to be transferred to the implementation of other programs, and 5.The program's costs estimation and deployment rate, designed to identify and estimate the costs related to the implementation of energy efficiency measures, the development of effective monitoring, reporting and management strategies, operationalization, and evaluation.
Step 1: Defining key issues The key issues identified as being necessary to establish when conducting a thorough evaluation of an energy efficiency policy program are presented in Table 3.The key issues should be seen as interconnected phases, taking place during the lifetime of the program.Clarifications and recommendations for use are included in Table 3.
This step should be seen as a tool that will help evaluate the magnitude of the issue, identify affected groups, clarify the causes of the issues, show why this is a key issue, and show whether and why there is a need for public intervention in the program.
Step 2: Setting objectives for each component developed under Step 1 The objectives are solutions to the key issues identified for each component presented in the first step of the ex-ante EEPPE (see Table 3) and should be established for each of them as briefly outlined in Fig. 5. Objectives provide effective criteria that enables to evaluate the success or failure of the proposed policy options.Therefore, establishing clear and measurable objectives is an important tool for evaluating the implementation of the policy.In the process of setting objectives, the evaluators or the policy makers should bear in mind that the objectives should also be used as a tool for benchmarking in the later process of monitoring of the program.This implies that, considering the framework proposed in this paper, the objectives need to be general, SMART (Specific, Measurable, Achievable, Relevant and Time-bound) and operational, as presented in Fig. 6.
Step 3: Identification of options During this step, the solutions for solving the identified problems or weaknesses for each key issue described in step 1 are identified and described.In terms of number of options or solutions, (Draskovics, 2018) recommends having at least three options to analyze, where a 'status quo option' is one possibility and assumes non-involvement in the existing situation.The status quo option is a useful benchmark for comparison against the other options, or baseline comparison.However, the categories of options could be different and involve e.g.costs, availability of resources or magnitude of the problem.The magnitude of the issue could range from minor to moderate, or major.A minor intervention could be translated into a minor modification of the program, a moderate intervention could be the addition of supplementary program components, and a major intervention could be the development or creation of a new program.
Step 4: Analyzing options from an energy and environmental perspective Step 4 is the most demanding and important step, and the purpose is to identify the strengths and weaknesses of all options based on energy and environmental impacts.The strengths and weaknesses can be classified in terms of economic, social and environmental effects, and are important due to the impact that they can have on long-term sustainability.

Table 3
Guidelines for developing a harmonized methodology for ex-ante energy efficiency policy program evaluation -EEPPE.

Key issues to be addressed Clarifications and recommendations to be considered when working on the key issues Component 1: ENERGY and GHG ESTIMATION
Decide all relevant primary energy factors for energy carriers Primary energy factors for major energy carriers should be decided according to, for e.g. the European Energy Efficiency Directives that suggested a 2.5 energy factor for electricity (European Commission, 2012).
Decide on the methodology used when calculating the carbon footprint of the measures: attributional or consequential method Attributional method as presented in the GHG protocol is an static inventory of emissions accounted or attributed to elements belonging to a defined system boundary, while the consequential method is used to measure the total system-wide change in emissions that occurs as result of a decision or action made by the organization based on the elements in the inventory Brander and Ascui (2015).
Decide on the pathway for calculating the GHG emissions of the measures based on the GHG protocol According to the GHG protocol, direct emissions caused by sources owned or controlled by the reporting organization fall under Scope 1.While indirect emission from the generation of purchased electricity, steam, heating, or cooling consumed by the reporting organization fall under Scope 2. The rest of the indirect emissions fall under Scope 3 and include the indirect emissions from the entire value chain (upstream and downstream).Scopes 1 and 2 are mandatory to be included in the GHG inventory, while Scope 3 it is not.(WBCSD and WRI, 2012;WRI and WBCSD, 2011).According to WCPI (2010), usually Scope 3 stands for the most of the GHG emissions of the reporting organization.

Decide all relevant emission factors for energy carriers
Emission factors for Scopes 1-3 vary between countries and regions and should ideally be based on regional level.Credible source should be used, such as the Greenhouse Gas Protocol or Energy Agency.
Decide on the primary energy use equation and where the major energy use exists among the support and production processes Deciding which main energy end-use processes -production and support processes -are important for the participating companies.While a non-energy-intensive mechanical engineering company can have an annual energy use of 80% for support processes, the support process energy used by a chemical pulp mill, for example, is only a few percent.

Component 2: TARGET GROUP ESTIMATION
Decide target group's approximate average annual energy use for the energy carriers Estimating the average target groups' company-specific annual energy use is of key importance.If annual energy use is not known, the impact estimations may suffer from very high inaccuracies.This is particularly true for SMEs, where annual energy use can vary a lot and be significantly below 1 GWh/year for a non-energy-intensive SME, while for a medium-sized pulp mill, an annual energy use of 1 TWh/year or higher can be expected.
Decide the approximate total number of participating companies and program's expected total annual energy use When the average energy use per company is established, the total number of companies expected to be affected by the program should also be estimated so that the total expected annual energy use for the program is achieved.

Component 3: ENERGY EFFICIENCY ESTIMATION
Estimate the energy efficiency potential Make an estimation of the energy efficiency potential for the program using available international, national and regional or local policy documents together with available scientific papers.If no data is available, carry out a pilot study by asking companies about this (Backlund et al., 2012a).It should be noted that a pilot case study may come with large degrees of uncertainties as companies that respond to the questions, do so in a strategic way, stating too high figures if they would like to see a new policy launched, or the opposite, state too low figures, if they see a threat with such a planned policy program.
(continued on next page) Step 5: Comparison of options and selection of the recommended one The last ex-ante EEPPE step proposed in this paper is the comparison of policy options identified, and this will support policymakers when deciding whether there is any need to intervene in or change the program.During this step, the strengths and weaknesses identified in step 4 will be compared to decide which is the most effective in terms of achieving the objective of the program, with fewer disadvantages.This implies that the advantages and disadvantages of each option will be identified.In order to accomplish this, several quantitative and qualitative tools can be used to support the process.Examples of qualitative tools include a multi-criteria analysis, a scenario analysis based on energy prices, and carbon balance scenarios.In terms of quantitative tools, cost-benefit analysis is a common and familiar method for many practitioners.Overall, this step involves comparisons in terms of the relevance, efficiency, effectiveness and sustainability of all policy options developed in the previous ex-ante EEPPE phases.

Concluding discussion
Several studies have previously reviewed policy programs from more than one country, e.g.Price and Lu (2011), Price (2005), and Tanaka (2011).Further, related to international policy program evaluation, Andersson et al. (2017) argued that there was no harmonized way of conducting industrial energy efficiency policy evaluation.This paper provides a conceptual model for evaluating energy efficiency policy programs ex-ante.

Key issues to be addressed
Clarifications and recommendations to be considered when working on the key issues Estimate the expected energy efficiency deployment rate When carrying out an impact estimation it is assumed that the entire energy efficiency potential is not deployed, i.e. not all proposed measures are implemented.E.g., the IAC program showed 50% deployment of the potential (Anderson and Newell, 2004).Results from the Swedish national energy audit program revealed similar findings, i.e. approximately 50% (Paramonova and Thollander, 2016b).Other programs have found figures of 40% (Swedish regional energy audit program) (Thollander et al. 2007) and up to 80% (Australian energy audit program) (Harris et al., 2000).
Quality control your estimation Be sure to quality control your assessment in terms of both the number of estimated energy efficiency measures implemented per company and the expected energy efficiency per measure.Common numbers of measures from an audit can differ considerably from only a few and upwards measures per company.Moreover, while the average energy efficiency per measure from energy audits of industrial SMEs can be in the range of 50 MWh/measure and year or less, measures for production process-intensive companies can be 1 000 MWh/measure or even higher.
Estimate the additionality of the deployment rate A policy program's impact may also be due to other factors than the actual policy.Additionality, i.e., the additional contribution from the policy compared with BaU and other active policy programs and factors affecting the energy efficiency deployment rate, should be estimated.Using a baseline for the expected impact can be a good way of doing this, e.g., including structural and autonomous effects in addition to policy effects a .

Component 4: NON-ENERGY BENEFITS ESTIMATION
Include non-energy benefits or multiple energy benefits of the policy program Estimate other factors that positively affect the program.For an example of non-energy benefits for public policy programs, see (Nehler et al., 2018) and (Johansson and Thollander, 2019).These includes increased greening of companies and reduced maintenance costs for implemented measures.At firm level, non-energy benefits might play an important role for the decision making (Killip et al., 2019).

Component 5: POLICY PROGRAM'S COSTS ESTIMATION and DEPLOYMENT RATE
Scaling up the expected net impacts in terms of energy efficiency and carbon dioxide improvements After estimating the average energy efficiency potential, deployment rate, additionality and non-energy benefits, it is possible to provide an expected net impact of the policy program and related carbon dioxide emissions.

Policy program cost-effectiveness calculation
Estimate the relevant costs associated with the program.Primary public costs are energy audit costs, costs for marketing the program, operationalization costs, and costs for monitoring and evaluation.
a One example can be that a capacity building program for large companies within the EU needs to take into account the fact that all large companies need to conduct an energy audit every fourth year, so the actual net impact of the capacity building program is facing the risk of being very low.The additionality of the capacity building program are only the measures to be implemented when the measures from the energy audit are excluded in the impact assessment.Else, double counting occurs, i.e., leading to a too positive impact assessment.
The set of guidelines consists of five important steps for conducting an ex-ante industrial energy efficiency policy program evaluation are set out in Table 4.The paper's scope is primarily the industrial sector.In that sense, one important area for development is that further research is suggested in developing a harmonized framework for various industrial sectors.Even though the paper results are in a sense generic, it may be argued that for countries with less developed energy efficiency policy programs and schemes, the paper may of great use in speeding up the policy maturity of a country or region.Furthermore, with a harmonized means for evaluation, it may be easier to initiate an evaluation which, in the case of an Australian industrial energy audit policy program meant that the perception of the policy changed from being understood as a failure to a success (Harris et al., 2000).
Overall, the presented guidelines also imply a comparison in terms of the relevance, efficiency, effectiveness, and sustainability of all major policy options developed, including the status quo option.Further, despite diligently following the developed model, the final decision on the policy option to be pursued lies with the political level.The US have a rich history of policy evaluation, while the EU initiated its strategic energy efficiency activities originating with the Energy End-Use and Energy Services Directive in 2006.The present paper, even though applicable beyond the EU, originates mainly from the EU policy work and, if applied to other countries and regions, this must be taken into account.This paper may be seen as one important step in initiating guidelines and later on even developing standards for how to design and evaluate energy efficiency policy programs.With a harmonized methodology for evaluation, policies can be compared and a higher degree of learning from other policies may be reached.Furthermore, the energy efficiency measures deployed can also be monitored compared and evaluated.Present paper can thus be seen as an important step towards the goal of creating a harmonized policy evaluation methodology.Further research is suggested in two ways: one is for validating the framework, and other is towards an ex-ante policy evaluation in the area.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

√
Information on the extent to which energy policy objectives are met,√A basis for comparing the performance of different energy policies,√A way of determining the effectiveness of policy implementation activities,√Objective information to determine the attributions of institutions implementing policies,

Fig. 2 .
Fig. 2. The set of guidelines made of five steps for conducting an ex-ante energy efficiency policy program evaluation.

Fig. 3 .
Fig. 3. Timeline for conducting the key steps proposed in the guidelines.

Fig. 4 .
Fig. 4.Outlined connection between the guidelines for ex-ante EEPPE steps and the questions to be answered through the application of the methodology.

Fig. 5 .
Fig. 5. Outlining the objective setting for each key component of the EEPPE from Step 1.

Fig. 6 .
Fig. 6.Setting the objectives for each component defined in Step 1.

Table 1
Examples of energy efficiency programs implemented in EU member states.
a S: stand-alone energy audit program; I: Integrated energy-audit program.

•
During the development of a new program, • Program expansion.√