1 Introduction

Life Cycle Assessment (LCA) is a comprehensive method considering multiple impact indicators with different metrics, which often leads to contradicting outcomes, that may cause ambiguity in decision support (Itsubo 2015). To facilitate the interpretation of LCA results normalisation and weighting methods are at hand (Laurent and Hauschild 2015), which enable the aggregation of the impact assessment results (e.g. Sala et al. (2018)). For decision-making Huppes et al. (2012) even consider weighting to be an inevitable step. Although the complexity of models increases as a result of adding new parameters, overall uncertainty could even decrease as the environmental relevance is considered (Hauschild and Potting 2005).

In LCA applied weighting methods are amongst others panel weighting, monetarisation or distance-to-target weighting, which have been reviewed in several publications such as Ahlroth (2014), Finkbeiner et al. (2014) or Pizzol et al. (2015, 2017). Distance-to-target methods set the current environmental situation in relation to the target situation and are usually characterised by a transparent documentation and a relatively high simplicity (Pizzol et al. 2017; Sala et al. 2018). According to Castellani et al. (2016) the weight in distance-to-target methods can either be derived from a science-based target as a physical threshold (e.g. in Tuomisto et al. (2012) or Bjørn and Hauschild (2015)) based on the planetary boundaries concept (Rockström et al. 2009) or from a politically legitimised target. Although, the uncertainty of the choice of the target is estimated as high, no uncertainty assessment of distance-to-target methods were identified by Pizzol et al. (2017). A representative of distance-to-target weighting is for instance the developed Ecological Scarcity Method (ESM) for Switzerland (Müller-Wenk 1978), which applies normalisation and weighting factors based on politically legitimised targets (Frischknecht et al. 2021; Frischknecht and Büsser Knöpfel 2013). As those targets depend on nationally and regionally varying legislations, different country- or continent-specific distance-to-target methods (e.g. for China (Miao et al. 2021) or Europe (Castellani et al. 2016)) as well as ESM (e.g. for Europe (Muhl et al. 2019), Germany (Ahbe et al. 2014; Lambrecht et al. 2020), Russia and Germany (Grinberg 2015), Switzerland (Frischknecht et al. 2021) or Thailand (Lecksiwilai and Gheewala 2019)) have been developed.

Although, the ESM prescribes the utilisation of the strictest target value for normalisation and weighting (Frischknecht and Büsser Knöpfel 2013) the target value definition is still subject to individual choices such as: (1) varying temporal and spatial validity, (2) different degrees in legitimacy, (3) the type and (4) prohibitive character of the target values. Grinberg (2015) suggests for instance a harmonisation of the temporal validity of politically legitimised target values as it causes uncertainty and may result in multiple possible weighting sets for a country or continent. For instance, in Miao et al. (2021) weighting sets for 2020 and 2030 are determined on the basis of target values valid for these years for all environmental issues considered. Even further weighting sets can be derived by the utilisation of non-binding targets as in Muhl et al. (2019). An additional source of uncertainty are different types of target values, which are either an environmental quality standard (e.g. a concentration in water or air) or an explicit environmental target load (e.g. annual lead emissions into air) (Ahbe et al. (2014, 2018)). Moreover, Lambrecht et al. (2020) stress that the handling of prohibitive target values varies in different national ESM (e.g. Ahbe et al. (2014) and Muhl et al. (2019)) and thus further increases the parameter uncertainty of the target value. An additional source of uncertainty is described by Muhl et al. (2021), who identify a significant influence of the spatial perspective on the LCA results: As the supply chain is distributed over several countries target values of the consumer countries differ to the target values of the producing regions. Additionally, Muhl et al. (2023) compare policy-based targets with science-based targets and apply them complementary to increase the comprehensiveness of the environmental performance of a product.

To assess the robustness of their results Muhl et al. (2019) and Castellani et al. (2016) conducted a sensitivity analysis of their applied target values and derive further weighting sets. Displacement effects in weights are visualised and compared to the original set. Castellani et al. (2016) apply their developed distance-to-target method to assess a product system and identify a relatively small impact on the results, when utilising alternative weighting sets. Frischknecht et al. (2021) and Lambrecht et al. (2020) classify the legitimacy of the applied target values but do not evaluate the resulting uncertainty. A different type of sensitivity analysis is performed by Lecksiwilai and Gheewala (2019), who developed an ESM based on Thailand’s targets: They compare Thailands’ current environmental impacts to its future impacts in case that all policy targets are met. If the ESM is applied to assess a product system, the impact results are compared to other national ESM (e.g. Lecksiwilai and Gheewala (2020) and Grinberg (2015)) to evaluate the influence of different normalisation and weighting factors.

The ESM has been adapted for several countries (e.g. Ahbe et al. (2014), Lecksiwilai and Gheewala (2019)) and applied for instance in Gebler et al. (2023) and Bach et al. (2022) to provide decision support. The possibility of deriving multiple target values from laws, directives or strategies allows individual decisions, which cause the existing parameter uncertainty in the application of ESM and questions the robustness of the results. To assess the robustness of the ESM results this study conducts an uncertainty analysis to evaluate the parameter uncertainty of the German ESM published by Lambrecht et al. (2020). Thereby, effects on the LCA results are analysed and the ability of the ESM to provide robust decision support evaluated. By deriving possible target values an uncertainty analysis is conducted, including legitimacy, varying temporal and spatial validity, restrictiveness as well as different types of target values. Due to its topicality and the partly controversy debate about the German energy transition the analysis is exemplarily performed for different electricity generation systems and thus the impact on the LCA results is illustrated (Agora Energiewende 2022).

The paper is outlined as follows: First the applied method and the used materials are explained, which includes the mathematical formulation of the ESM, the uncertainty analysis and the derived target values. After presenting the results, these are discussed, limitations elaborated and conclusions drawn.

2 Methods and materials

In a first step the structure of LCA is explained, which is followed by the general methodology of the ESM including the definition of parameters (e.g. target values). The uncertainty analysis is conducted for the parameter uncertainty of the target values defined in the ESM. To determine the uncertainty, multiple possible target values are derived from laws, directives and strategies valid for Germany. Applying these values in the ESM, allows the generation of multiple possible weights, which are used to conduct the LCA. The LCA results in form of multiple single scores are then compared.

2.1 Life cycle assessment: ecological scarcity method

Comparative LCA allows the assessment of environmental impacts of processes or product systems considering multiple impact indicators following the subsequent steps: (1) Goal and Scope Definition, (2) Life Cycle Inventory, (3) Life Cycle Impact Assessment and (4) Interpretation of the results. The ESM is a specific Life Cycle Impact Assessment Method (LCIA), following the step of the Life Cycle Inventory calculation. The Life Cycle Inventory contains all necessary elementary flows taken from or emitting into the ecosphere (ISO 14040:2006 2006; ISO 14044:2006 2006). Life Cycle Impact Assessment Methods characterise the elementary flows of the Life Cycle Inventory and allow the calculation of impact indicator results. Characterisation factors, which describe the relative environmental impact of different substances as compared to a reference substance, are based on different methods depending on the respective midpoint indicator, e.g. IPCC (2013) for global warming.

The basis for our assessment is the ESM for Germany as published in Lambrecht et al. (2020). For normalisation and weighting it relies on target values \((T)\), normalisation values \((NV)\), current values \((A)\) and the constant \((z)\), which are applied to calculate eco-factors. Normalisation values are taken from annual German emission data. Current values correspond to the physical property of the defined target values and sometimes equal the normalisation values. Weights are defined by the squared ratio of the current to the pursued environmental situation \({(\frac{A}{T})}^{2}\). The applied constant \((z)\) ensures practical numerical values and equals \(1E12\) (as both in Frischknecht and Büsser Knöpfel (2013) and Lambrecht et al. (2020)).

Depending on the availability of target values, the ESM applies normalisation and weighting either to impact indicator results (including characterisation, e.g. global warming) or to elementary flows (excluding characterisation, e.g. lead) (Frischknecht and Büsser Knöpfel 2013). Thus, there are two ways of calculating the impact score of an environmental indicator \(({u}_{v})\) either with characterisation or without characterisation requiring the application of Eqs. 1 or 2, respectively.

With characterisation: Following Eq. (1) \({u}_{v}\) is calculated by first aggregating the elementary flows (\({e}_{x}\)) from the Life Cycle Inventory using substance specific characterisation factors \((C{F}_{v,x})\). The summation index refers to the set \(V\), which corresponds to \(v\) (e.g., if \(v=HMIW\), \(V=HMIW=\left\{Ni,Pb, Zn,\dots \right\}\), compare Table 1). The resulting midpoint indicator value is then normalised and weighted using the environmental indicator specific parameters \(N{V}_{v}\), \({A}_{v}\) and \({T}_{v}\).

Table 1 Environmental indicators in the ESM for Germany (Abb. = Abbreviation)
Full size table
$${u}_{v}=\left(\sum_{x \in V}C{F}_{v,x}*{e}_{x}\right)* \frac{1}{N{V}_{v}}*{\left(\frac{{A}_{v}}{{T}_{v}}\right)}^{2}*z$$
(1)

Without characterisation: In Eq. 2 the elementary flows \(({e}_{x})\) are directly normalised and weighted with substance specific parameters \(N{V}_{x}\), \({C}_{x}\) and \({T}_{x}\).

$${u}_{v}= \sum_{x \in V}\left({e}_{x}* \frac{1}{N{V}_{x}}*{\left(\frac{{A}_{x}}{{T}_{x}}\right)}^{2}\right)*z$$
(2)

The total impact score \((u)\) results from the aggregation of the impact score of all environmental indicators \((X)\) following Eq. (3). The application of the ESM results in a dimensionless score (i.e. with \(unit = 1\)). To give it nevertheless a name, we adopt the common practice of ESM and use the unit eco-points for Germany \([E{P}_{G}]\), which should not be misunderstood as physical (SI) unit though.

$${u}=\sum_{v \epsilon X}{u}_{v} \left[E{P}_{G}\right]$$
(3)

Table 1 presents 12 environmental indicators \((v)\) covered by the German ESM published by Lambrecht et al. (2020). For environmental indicators that involve characterisation, the substances \((x)\) involved are not explicitly listed. Instead, the reference is made to the applied characterisation method (e.g. IPCC (2013) for global warming). For environmental indicators directly based on substance specific target values (e.g. “heavy metals into water”), all contributing substances are explicitly listed (e.g. lead).

2.2 Uncertainty analysis: Monte Carlo analysis

According to Rosenbaum et al. (2018) uncertainty analysis aims to enhance the interpretation of results by assessing their robustness. It allows the probabilistic calculation of results once the uncertainty of the input parameters (in our case target values \(T\)) are quantified and thus evaluates the impact of uncertainties of model parameters (input) on model results (output) (Campolongo et al. 2011). Thereby the likelihood of a certain product system to be preferable to another one can be determined. Although uncertainty analysis is a recommended step in LCA according to ISO14040/44, less than 20% of LCA studies include an uncertainty analysis (Bamber et al. 2020). Additionally, Lo Piano and Benini (2022) specify, that most uncertainty analyses are performed on the Life Cycle Inventory, whereas other LCA phases, such as normalisation and weighting, are neglected.

Several methods to quantify uncertainty in LCA are at hand: e.g. the pedigree matrix approach, analytical or numerical uncertainty propagation, and fuzzy sets (Rosenbaum et al. 2018). Amongst others Lloyd and Ries (2007) analyse quantitative uncertainty assessment approaches applied in LCA e.g. the Monte Carlo simulation or the fuzzy set theory. To conduct the uncertainty analysis in this study the numerical uncertainty propagation method Monte Carlo Analysis (MCA) is applied, which is recommended by Michiels and Geeraerd (2020) and has already been applied in combination with LCA (Heijungs 2020; Igos et al. 2019). According to Rosenbaum et al. (2018) the MCA follows the subsequent steps:

  • Step 1: Sample generation: Calculate random values for all input variables

  • Step 2: Output Calculation: Application of input variables in the model

  • Step 3: Results analysis and visualisation

2.2.1 Step 1: sample generation: definition of alternative target values

In Step 1 a sample of input parameters is defined. As the focus of the study is the analysis of the applied target values used in the ESM for Germany, a sample of different target values is generated. Therefore, alternative target values are defined by literature research of laws, directives and strategies. The time dependency of target values is considered by applying different target values valid for different years. Different degrees of legitimacy are taken into account by taking target values from national or international binding laws or treaties, respectively. Furthermore, the prohibition of emissions is considered by using the strictest published target values, which does not equal the value zero.

From the collected target values, a moderate \({T}_{moderate}\) and a strict target value \({T}_{strict}\) are derived, which represent the upper and lower limit of a target value applied in the distribution function. Based on \({T}_{moderate}\) and \({T}_{strict}\) the boundary weighting sets are calculated by \({(\frac{A}{T})}^{2}\). To compare these with for other in literature existing weighting sets, the published eco-factors of e.g. Muhl et al. (2019) or Lambrecht et al. (2020) are multiplied by the current environmental German situation \((NV).\) Subsequently, the Monte Carlo analysis is performed for \(N = \mathrm{10,000}\) trials. As only upper and lower target values are known and no information about the distribution in between is available, the random values are drawn over the uniform distribution in the interval \(({T}_{strict},{T}_{moderate})\) (Gieck and Gieck 2005). For each Monte Carlo run \((n)\) a random target value (\({T}_{n}\)) is determined within the limits \({T}_{moderate}\) and \({T}_{strict}\) and applied to calculate the total impact score.

2.2.2 Step 2: output calculation

After defining a sample of input parameters, these are applied in the model to calculate the output, which corresponds to the LCIA results (total impact score). A sample of already existing systems in ecoinvent 3.7.1 is exemplary analysed. As the goal of this study is the assessment of the robustness of the ESM results, no full contribution analysis of the used electricity generation systems listed in Table 2 is provided. Therefore, systems with substantial differences in their Life Cycle Inventory and Impact Assessment (UNECE 2022) are chosen resulting in a sample of different renewable and conventional power plants and the German Electricity Mix. All characteristics concerning the systems like assumed yield, lifetime or included life cycle stages are documented in Wernet et al. (2016). The LCA of the electricity generation system is conducted with a functional unit of 1 kWh generated electricity and performed with the open source LCA framework Brightway (Mutel 2017) as it provides a flexible integration and parametrisation of LCIA methods. It is written in Python (van Rossum and Drake 2009) and therefore allows connections to other Python libraries e.g. for data analysis and plotting. Furthermore, MCA in the context of LCA is already integrated within the Brightway framework (Mutel et al. 2013).

Table 2 Analysed electricity generation systems
Full size table

2.2.3 Step 3: result analysis and visualisation

To analyse and visualise the results, the Rank Acceptability Index (\(RAI\)) is applied, which is a common applied method to evaluate the robustness of results (Corrente et al. 2014; Greco et al. 2018; Prado et al. 2020; Tervonen et al. 2011) and assists in stochastic results visualisation (Bertola et al. 2019). The \(RAI\) describes the relative frequency of how often an electricity generation system reaches a certain rank over all Monte Carlo runs. As we are examining nine systems, it is possible to have ranks ranging from 1 to 9 in each MCA run. As a minimal total impact score is the overall goal, rank 1 corresponds to the most and rank 9 to the at least environmentally favourable technology (\(rank= \{1,\dots ,9\}\)).

The \(RA{I}_{p,rank}\) for an electricity generation system is calculated following Eq. (4): First, for all analysed energy generation systems (\(p\)) and Monte Carlo runs \((n)\) the total impact score \({u}_{n,p}\) is calculated. In a next step we consider the rank (\({r}_{n,p}\)) instead of the absolute indicator value \({u}_{n,p}\). Finally, the number of times a system takes a certain \(rank\) is counted using the Kronecker delta function and normalized by the total amount of Monte Carlo runs \((N=10000)\).

$$RA{I}_{p,rank}=\frac{1}{N}\sum_{n=1}^{N}{\delta }_{rank, {r}_{n,p} }$$
(4)

Complementary, the average contribution (\(s\)) of each environmental indicator (\(v\)) to the total impact score is calculated as follows: The product system dependent impact score of an environmental indicator per Monte Carlo run (\({u}_{v,n,p}\)) is divided by the total impact score \({u}_{n,p}\) for all Monte Carlo runs (Eq. (5)). To receive the total average share (\({s}_{v,p}\)) the environmental indicators \({s}_{v,n,p}\) of each run are aggregated and divided by the total amount of Monte Carlo runs (\(N=10000\)) following Eq. (6).

$${s}_{v,n,p}=\frac{{u}_{v,n,p}}{{u}_{n,p}}$$
(5)
$${s}_{v,p}= \frac{\sum_{n=1}^{N=10000}{s}_{v,n,p}}{N}$$
(6)

In the supplementary information all information to reproduce the analysis are documented: “SI_2__python_code” contains the full python code, the text file “SI_1__python_packages” lists all necessary python packages, the excel files “SI_3__characterisation_factors_ESM_Germany” and “SI_4__parameter_T_MCA” contain the needed parameter for the assessment. “SI_5__ecological_scarcity_method_2021” contains a compilation of all parameters for the ESM for Germany.

2.3 Model parametrisation and target value definition

To parametrise the model strict (\({T}_{strict}\)) and moderate target (\({T}_{moderate}\)) values are identified (see Table 3). To illustrate the ranges, these are set in relation to the average target value (\({T}_{ average}\)) as depicted in Fig. 1. In case no ranges are depicted, no alternative \(T\) could be identified in the review of laws, directives and strategies. Most of \({T}_{strict}\) and \({T}_{moderate}\) are in the range of 1.5 to 0.5 normalised to \({T}_{average}\). Relative high deviations of \(T\) can be identified for “lead into air” (HMIA-Pb) and for “polycyclic aromatic hydrocarbons (PAH) into water” (WP-PAH) ranging from 0.1 to 2 in relation to \({T}_{average}\).

Table 3 References for the sample of target values (C = Characterised, T = target value, BImSchG = Federal Immission Control Act, OGewV = Surface Water Regulation, Abb. = Abbreviation)
Full size table
Fig. 1
figure 1

Ranges of T; GW = global warming, CSIA = carcinogenic substances into air, HMIA = heavy metals into air, HMIW = heavy metals into water, APP = main air pollutants and PM, OD = ozone layer depletion, WP = water pollutants, MR = mineral resources, LU = land use, ER = energy resources, WTD = waste to deposit; Pb = lead, Cd = cadmium, Hg = mercury, Ni = nickel, As = arsenic, Zn = zinc, Cu = copper, NMVOC = non-methane volatile organic compounds, NOx = nitrogen monoxide, SO2 = sulphur dioxide, NH3 = ammonia, PM2,5 = particulate matter 2,5 mm; N = nitrogen, P = phosphor, CSB = chemical oxygen demand, PAH = polycyclic aromatic hydrocarbons, TOC = total organic carbon

Full size image

To gain a deeper understanding of these variations in the target values, the sources and the specific type of exemplary target values are explained: For instance, in Lambrecht et al. (2020) the environmental target value for “lead into air” (HMIA-Pb) of the United Nations (UNITED NATIONS 2014) is applied (\({T}_{HMIA-Pb}=\mathrm{2,285} \frac{t Lead}{a}\)). Alternatively, \(T\) can also be derived from an environmental quality standard originally defined by a directive of the European Parliament (EP 2008) \(({T}_{HMIA-Pb}=0.5 \frac{\mu g Lead}{{m}^{3} air}\)). Its regulations and target values are translated into the Federal Immission Control Act (German: BImSchG) (German Bundestag 2021) setting environmental quality standards for maximum yearly concentrations of e.g. monitored lead (Federal Environment Agency Germany 2019). The different units of the target values do not pose a problem, as they are reduced when calculating the eco-factor. Compared to the German law, upper limits for maximal environmental loads defined by the United Nations result in a stricter target and consequently in a higher eco-factor.

Target values for substances in the environmental indicator “main air pollutants and particulate matter” (APP) are defined in the Federal Immission Control Act (German Bundestag 2021) and in a European Directive (EP 2016). The maximal environmental loads are defined with different temporal validity. For the assessment the target values for 2020 and for 2030 are utilised to reflect the problem of temporal validity explained in e.g. Castellani et al. (2016). Table 4 in the Appendix depicts the target values for the regulated substances.

For the water pollutants “nitrogen” (N) and “phosphorous” (P), the target values are based on the German water discharge combined with the maximum concentration allowed by the Surface Water Regulation (German: OGewV) (German Bundestag 2016) \(({T}_{WP-N}=2.6 \frac{mg N}{l}, {T}_{WP-P}=0.05 \frac{mg P}{l})\) and compared to the limitations defined by the International North Sea Protection Conference (OSPAR 1998) for N and Ahbe et al. (2014) for P (\({T}_{WP-N}=515 \frac{kt N}{a}, { T}_{WP-P}=9 \frac{kt P}{a})\). Emissions for “heavy metals into water” (HMIW) are also regulated by the Surface Water Regulation but also in regulations defined by the International Commission for the Protection of the Rhine (ICPR) (ICPR 2011). The concentrations are measured, monitored and published for several stations by the Federal Environment Agency (Federal Environment Agency Germany 2019). The alternative environmental quality standard value based on the Surface Water Regulation results in a less strict target value compared to the defined target by the ICPR. Table 5 in the Appendix depicts the target values for the regulated substances.

The impact indicators “global warming” (GW), “land use” (LU) and “energy resources” (ER) are characterised by prohibitive targets in 2045 (GW) or 2050 (LU and ER) with target values corresponding to zero. According to the Eqs. (1) and (2) the term \({\left(\frac{A}{T}\right)}^{2}\) does not allow a division by zero and would result into infinitely high eco-factors for GW, ER and LU for target values which strive to zero. As a result, these environmental indicators would dominate all other indicators and finally result in a binary weighting scheme (Pizzol et al. 2017). To avoid this effect and to still integrate these environmental indicators, target values for \({T}_{moderate}\) are considered reflecting the 2030 target value. For \({T}_{strict}\) a target value, which is not resulting to a binary weighting, is applied: The target value valid for 2040 is applied for GW, LU and ER respectively (see Appendix Table 6). A summary of all defined \({T}_{strict}\) and \({T}_{moderate}\) is given in the Table 3 and contains information about applied characterisation, reason for uncertainty as well as references for the strict and moderate target value.

3 Results

The results section is organised in two parts: First, the boundary weighting sets corresponding to \({T}_{strict}\) and \({T}_{moderate}\) are compared with other weighting sets available from the literature. Additionally, LCA results of the analysed energy generation systems obtained by applying \({T}_{strict}\) and \({T}_{moderate}\) are described without, at this point, taking uncertainty into account. Second, the results of the uncertainty analysis are presented by the rank acceptability index and average environmental indicator contribution.

3.1 Resulting weighting sets and life cycle assessment results for strict and moderate target values

Figure 2 illustrates the contributions of the environmental indicators to the total impact score resulting from the defined targets \({T}_{strict}\) and \({T}_{moderate}\), compared to those published by Lambrecht et al. (2020), Frischknecht and Büsser Knöpfel (2013), and Muhl et al. (2019). To allow the comparison of the resulting weighting sets those are normalized with the current environmental situation in Germany. The ESM for Germany of Lambrecht et al. (2020) is the basis of the developed weighting sets \({T}_{strict}\), \({T}_{moderate}\). However, data updates for normalisation and current values and changes in regulation with new target values explain the difference of \({T}_{strict}\) to \({T}_{Lambrecht}\).

Fig. 2
figure 2

Environmental indicator contribution to the weighted total for \({T}_{strict}\), \({T}_{moderate}\), \({T}_{Lambrecht}\), \({T}_{Frischknecht}\) and \({T}_{Muhl}\) Lambrecht = Lambrecht et al. (2020), Frischknecht = Frischknecht and Büsser Knöpfel (2013) and Muhl = Muhl et al. (2019) applied on the current environmental situation of Germany; GW = global warming, CSIA = carcinogenic substances into air, HMIA = heavy metals into air, HMIW = heavy metals into water, APP = main air pollutants and PM, OD = ozone layer depletion, WP = water pollutants, MR = mineral resources, LU = land use, ER = energy resources, WTD = waste to deposit

Full size image

As depicted in Fig. 2, the variations in target values alter the contributions of environmental indicators. In weighting set \({T}_{moderate}\), “energy resources” (ER) is marked with a relative high contribution of more than 20%, which is substantially lower in the other weighting sets. Nevertheless, with increasing relative weight contributions of “global warming” (GW) and “land use” (LU) the share of ER decreases. A similar effect occurs for GW in \({T}_{Lambrecht}\): Although the same target value for GW is applied in Lambrecht et al. (2020) and the weighing set \({T}_{strict}\), the share of GW reduces from 55% (Lambrecht et al.) to 33% (\({T}_{strict}\)). This effect is driven by the steep rise of the weight of LU, which increases from 4 to 34 and is amplified by “heavy metals into water” (HMIW) and “water pollutants” (WP): Exemplary, the weight of the water pollutant PAH increases by orders of magnitude from 0.005 to 18. Consequently, the relative share of GW is lower in \({T}_{strict}\) compared to \({T}_{Lambrecht}\).

To illustrate the influence of utilising the weighting sets of \({T}_{moderate}\) and \({T}_{strict}\) Fig. 3 depicts the resulting total impact score for both sets. As a smaller target value \(T\) increases the calculated eco-factor (\(EF\)), the application of \({T}_{moderate}\) results in a substantially lower total impact score compared to the results of the \({T}_{strict}\) set. In contrast, the system ranking in terms of the total impact score is less distinctive: only PV-Si open range, Lignite PP, PV-Si rooftop and Biogas CHP exchange its ranks. In line with Fig. 2 the total impact score is stronger influenced by the environmental indicator ER, when applying the weighting set \({T}_{moderate}\). Nevertheless, hard coal and Lignite PP are characterised by a high share of GW and reach comparable results as PV-Si open range (14–17 \(\frac{{EP}_{G}}{kWh}\)), which is dominated by the environmental indicator LU. The other renewable technologies reach scores of 2–7 \(\frac{{EP}_{G}}{kWh}\), with PV technologies being marked with relatively high shares of “heavy metal into water” (HMIW), whereas the environmental indicator “main air pollutants and PM” (APP) contributes substantially to all analysed systems. When applying the weighting set \({T}_{strict}\), Lignite PP, PV-Si open range and Hard Coal PP still reach the highest total impact scores (93–101 \(\frac{{EP}_{G}}{kWh}\)) followed by German Electricity Mix (59 \(\frac{{EP}_{G}}{kWh}\)), Biogas CHP (30 \(\frac{{EP}_{G}}{kWh}\)) and PV-Si rooftop (29 \(\frac{{EP}_{G}}{kWh}\)), Geothermal PP (17 \(\frac{{EP}_{G}}{kWh}\)), PV-Cdte rooftop (16 \(\frac{{EP}_{G}}{kWh}\)) and Wind power reaching the lowest total impact score (6 \(\frac{{EP}_{G}}{kWh}\)). However, the total impact score of Hard Coal, Lignite PP, Biogas CHP and the German Electricity Mix is marked by either high shares of GW or in the case of PV-Si open range by LU. In contrast, the contribution of the environmental indicators to the remaining systems is more diverse. The weighting set \({T}_{strict}\) can be interpreted as the desired ESM because it applies the highest possible eco-factors as demanded in Frischknecht and Büsser Knöpfel (2013).

Fig. 3
figure 3

Total impact scores for \({T}_{strict}\)(top) and \({T}_{moderate}\) (bottom) for the assessed energy generation systems; GW = global warming, CSIA = carcinogenic substances into air, HMIA = heavy metals into air, HMIW = heavy metals into water, APP = main air pollutants and PM, OD = ozone layer depletion, WP = water pollutants, MR = mineral resources, LU = land use, ER = energy resources, WTD = waste to deposit

Full size image

3.2 Results of the Monte Carlo analysis: rank acceptability index and mean average environmental indicator contribution

The application of the MCA allows the presentation of the average total impact score as well as the calculation of error bars showing the standard deviation as depicted in Fig. 4. Besides a rank change of PV-CdTe rooftop and Geothermal PP, the ranking of systems in terms of their total impact score equals the application of the weighting set \({T}_{strict}\) (see Fig. 3). The impacts of lignite, Hard Coal PP (34 and 37 \(\frac{E{P}_{G}}{kWh}\)), the German Electricity Mix (22 \(\frac{E{P}_{G}}{kWh}\)) as well as PV-Si open range (34 \(\frac{E{P}_{G}}{kWh}\)) have relatively high standard deviations, which are mainly caused by GW and LU. The weights of these indicators have a particularly high uncertainty due to the prohibitive character of their target value (\(T\) strives to zero).

Fig. 4
figure 4

Ranking of technologies showing the average total impact score (column sums) for the assessed energy generation systems; the contribution of the environmental indicators (columns) and the error due to the uncertainty of the target values, GW = global warming, CSIA = carcinogenic substances into air, HMIA = heavy metals into air, HMIW = heavy metals into water, APP = main air pollutants and PM, OD = ozone layer depletion, WP = water pollutants, MR = mineral resources, LU = land use, ER = energy resources, WTD = waste to deposit

Full size image

The rank acceptability index is calculated to illustrate the frequency of a technology reaching a specific rank to provide information about the rank stability and robustness of the LCA results (see Fig. 5). Hereby applies: The more the systems are distributed over the ranks the higher is the parameter uncertainty of the target value. Three main results are identified: (1) The uncertainty of the input parameter (the target value) is influencing the total impact score and consequently the ranking of the energy generation systems. (2) Besides PV-Si open range each energy generation system dominates at least a single rank to 50%. (3) Displacement effects in ranks only occur to a very limited extent: The systems switch only to ranks next to each other e.g. Biogas CHP is located on the ranks 3 to 5 or Geothermal PP on the ranks 2 to 4. For the entire range of weights investigated here, Wind power reaches rank 1. Furthermore, there is a discernible tendency for the Lignite PP (Rank 7: 52%, Rank 8: 34%) and the hard coal-fired power plant (Rank 8: 54% and Rank 9: 46%), to dominate the 8th and 9th rank while the PV-Si open range reaches its highest shares for 7th (32%) and 9th (40%) rank.

Fig. 5
figure 5

Rank acceptability index results of the assessed energy generation systems; abbreviations according to Table 2

Full size image

In contrast to Fig. 2, which represents the contributions of environmental indicators normalised to the current environmental situation in Germany, Fig. 6 displays the average contributions of the environmental indicators after conducting the LCA, ESM, and Monte Carlo analysis for the analysed energy generation systems. For this purpose, the environmental indicator contributions to the total impact score for all energy system generation systems are determined and averaged across all Monte Carlo runs. In general the environmental indicator contributions are diverse for all systems and not dominated by a single environmental indicator. The most contributing environmental indicators are “global warming” (GW), “land use” (LU), “energy resources” (ER), “heavy metals into water” (HMIW) and “main air pollutants and PM” (APP). The average environmental indicator contribution of APP is relatively constant for all technologies reaching its peak for the Biogas CHP (26%). “carcinogenic substances into air” (CSIA) reaches contributions of 0.8–4.1%, “water pollutants” (WP) 0.1–2.2% and “heavy metals into air” (HMIA) 0.4–2.2% respectively.

Fig. 6
figure 6

Mean average environmental indicator contribution for the assessed energy generation systems; GW = global warming, CSIA = carcinogenic substances into air, HMIA = heavy metals into air, HMIW = heavy metals into water, APP = main air pollutants and PM, OD = ozone layer depletion, WP = water pollutants, MR = mineral resources, LU = land use, ER = energy resources, WTD = waste to deposit

Full size image

Greenhouse gas intensive technologies, which are the conventional electricity producers as Hard Coal (47%) and Lignite PP (61%) as well as Biogas CHP (39%) show high shares in GW, which is mainly due to carbon dioxide emissions (Hard Coal and Lignite PP) and methane emissions caused by the digestion to produce biogas (Biogas CHP). In contrast, the total impact score of renewable technologies such as PV is less dominated by a single environmental indicator. For instance, the environmental indicator ER gains importance for Geothermal PP and Wind power due to the following reason: The target value of ER is based on primary energy reduction targets for Germany, which results in higher total impact scores for technologies with higher primary energy demand (Geothermal PP = 7.1 \(\frac{{MJ}_{primary\; energy}}{{kWh}_{electricity}}\), Wind power = 3.9 \(\frac{{MJ}_{primary\; energy}}{{kWh}_{electricity}}\) (Wernet et al. 2016)). The environmental indicator LU dominates to 68% the total impact score of PV-Si open range as the full terrain needed for the plant is sealed. In contrast, the environmental indicators “water resources” (WR), “ozone layer depletion” (OD), “waste to deposit” (WTD) and “mineral resources” (MR) take on a subordinate role with impact contributions lower than 1%.

4 Discussion

Muhl et al. (2019) showed that the relative shares of environmental issues of weighting sets can widely differ: In case only binding targets are included, the resulting eco-factors are lower compared to the weighting set including non-binding targets. Moreover, they compare their baseline weighting set to weighting sets from Frischknecht and Büsser Knöpfel (2013) and Ahbe et al. (2014, 2018) and found that the relative shares of the environmental indicators global warming, main air pollutants and PM and heavy metals into water show significant changes. This is in line with our findings: The derived weighting sets based on \({T}_{strict}\) and \({T}_{moderate}\) (see Fig. 2) significantly alter compared to available weighting sets in literature e.g. Frischknecht and Büsser Knöpfel (2013), Muhl et al. (2019) or Lambrecht et al. (2020). Different country- or continent-specific legitimised target values cause varying relative weights for instance of “global warming” (GW) or “heavy metals into water” (HMIW) (Muhl et al. (2019): GW 32%, HMIW 20%, Frischknecht and Büsser Knöpfel (2013): GW 45%, HMIW 3%, Lambrecht et al. (2020): GW 55%, HMIW 6%).

In line with the findings of Castellani et al. (2016), who apply their developed distance-to-target weighting for Europe on a product system, the LCA results show only marginal rank displacement effects when the moderate target values are compared to the stricter ones. When conducting the uncertainty analysis displacement effects in ranks due to the parameter uncertainty of the target value are revealed for the ESM for Germany. Additionally, the utilisation of the stricter target values \({T}_{strict}\) causes shifts in the relative contribution of the environmental indicators to the total impact score for the analysed ESM with dominating shares of GW and LU.

During the uncertainty assessment further sources of uncertainty have been identified influencing the definition of the target values. The federal structure of Germany allows the identification of region-specific target values: For instance, there are different targets for heavy metal emissions defined for Rhine, Main or for Lake Constance or differences for the North or Baltic Sea, which could also be integrated as a source of uncertainty. Furthermore, Muhl et al. (2021) identify a strong regional dependence of the target values: Switching from weighting sets derived from consumer regions to producer regions cause a shift of the main contributor (from acidification to water resources) when assessing the production of aluminium and steel. As the definition of target values for further countries is necessary the parameter uncertainty may be further increased and its influence needs to be analysed. Additionally, an enormous data pool of non-binding target value data is available due to various available reports from NGOs, governmental or EU strategies, which are not reviewed for this research as it is focused on legitimate sources being valid directives, laws or binding treaties.

For the different types of target values, which can be environmental quality standards (EQS) (e.g. maximum concentrations) or environmental targets (ET) (a specific load limit) no conclusion about the degree of strictness and consequently the influence on the eco-factor can be drawn. In practice the definition of an absolute load limit (ET) leaves less room for interpretation and thus subjective decisions. In fact, EQS often leave the choice to the practitioner, how to be handled.

On a methodical level, significant challenges arise from the relatively broad description of the ESM provided by Frischknecht and Büsser Knöpfel (2013), particularly regarding the eco-factor calculation. In contrast to most publications, where a single formula is provided, we found it necessary for the understanding of our work to distinguish at least two cases: eco-factor calculation on elementary flow and indicator level as outlined in Eqs. 1 and 2. Since there are e.g. different approaches to derive target values, which are not captured in detail by our formulation, there is certainly still room for improvement in the mathematical formulation. Further research may help to come to a more rigorous and explicit mathematical formulation of the ESM.

In contrast to Switzerland's ESM, we take multiple contributions of elementary flows to several impact categories into account. This methodical adaptation is based on the following rationale: (a) impact assessment methods for midpoint indicators generally take multiple contributions of elementary flows (e.g. to OD and GW) into account (e.g. ReCiPe (Goedkoop et al. 2013)). (b) Since our objective is to calculate meaningful impact scores on category level (as in Eq. 3) we adopt this approach within the ESM context. “Double-counting” of elementary flows occurs for 44 substances and 185 elementary flows out of a total of 1149. The relative deviation of our ESM results from the original approach (without double counting) is on the order of magnitude 1E-5 and can therefore be neglected. Although not substantial in our case, it could be crucial for other product systems. The issue of double counting multiple impacts should therefore be discussed in the context of further developments of the ESM.

The results of the Monte Carlo analysis are dependent on the parameters used and the probability distribution applied. As the distribution between the moderate and strict target values is unknown a uniform distribution is used, which is a common approach (Gieck and Gieck 2005), but not necessarily represents the real-world data probability distribution. Furthermore, the number of runs in Monte Carlo analysis can affect the results. Generally, a number of 1,000–30,000 runs is recommended to reach stable results (Cassettari et al. 2012; Gregory et al. 2016; Hongxiang and Wei 2013). However, Heijungs (2020) suggests to restrict the number of Monte Carlo runs to a number not greater than the sample sizes used for the input parameters. This corresponds to our observation, that the results do not change significantly with more than 200 runs.

Besides the uncertainty of the target values, LCA methodology includes further uncertainties dependent on the data quality, data depth and data processing steps such as allocation procedures and characterisation factors. We do not evaluate the uncertainties arising from characterisation, although these are influencing the total impact score. As some of the environmental indicators (e.g. “heavy metals into water”) are a compilation of substances their relative environmental impact is disregarded, thus the environmental relevance of a substance is not included (ISO14044). The characterisation factors that were used for the ESM are based on midpoint LCA methods which can include further uncertainties.

In the scope of this study we analyse the influence of uncertainty in impact assessment on the robustness of LCA results in context of decision making. We focus on the weighting, as relatively strong qualitative differences were identified in their derivation in previous studies, e.g. in Lambrecht et al. (2020). However, an extension of the analysis to include normalisation factors would be useful, as the relevant literature shows: For instance, Benini and Sala (2016) express the need of an in-depth uncertainty analysis of normalisation factors as these may have a higher influence on the results than the weighting (Myllyviita et al. (2014)). Furthermore, the consistency between the reference data used for the external normalisation and the LCA modelling are discussed critically e.g. in Hélias and Servien (2021).

The datasets of wind, photovoltaic and biogas power occupying the first ranks are from 2000, 2005–2008 and 2007 and therefore can be partly outdated. Especially for technologies such as photovoltaic with a steep learning curve and an active, continuous state of development, the inventory data can change rapidly on a large scale. This could lead to deviating results and influence the rank stability. Further assessments should examine whether the results of this study are applicable to other systems.

4.1 Comparison with other weighting approaches

The application of external normalisation and weighting to aggregate impact assessment results is a weighted sum based aggregation method and for instance analysed by Prado et al. (2020). Deficits of the weighted sum method as well as guidelines for the definition of weights to reflect preferences of the decision-maker are amongst others described by Marler and Arora (2010). Prado et al. (2020) find relatively stable rank acceptability index results when the weighted sum method is applied to aggregate the midpoint indicator of ReCiPe (Goedkoop et al. 2013) and CML (van Oers 2015). Additionally, a few impact categories dominate the results. In our assessment, systems with comparable total impact scores exchange ranks and a relative high rank stability is identified as well, however without a dominant environmental indicator.

In line with Pizzol et al. (2017), the parameter uncertainty in distance-to-target methods based on politically legitimised targets is present. The application of science-based targets as in Tuomisto et al. (2012) would be a different approach of distance-to-target weighting, which still depend on subjective choices (Muhl et al. 2023). A complementary application could lead to a more comprehensive basis for decision making and a better interpretation of results and is recommended by Muhl et al. (2023).

Next to distance-to-target methods further approaches to determine weights are available in literature such as panel weighting or monetarisation (Pizzol et al. 2017), which are assessed by e.g. Ahlroth (2014) or Finkbeiner et al. (2014). For instance, panel weighting is applied to aggregate the environmental impact categories evaluated in the Environmental Footprint (Sala et al. 2018). LCA experts and internet users are asked to weight the impact categories. However, a possible bias of the panellists cannot be excluded (Pizzol et al. 2017; Sala et al. 2018). The determined weights could vary depending on the surveyed group (see Sala et al. (2018)) and a decision for a weighting set or a further weight aggregation may be necessary (e.g. merging weights of different surveyed groups). In contrast, normative targets are supposed to be derived from a consensus-based process and already reflect the different positions (Pizzol et al. 2017).

In practice, there is no consensus on which weighting scheme is most suitable (Sala et al. 2018). By analysing the uncertainty of the ESM, the transparency of the target value definition and its influence on the LCA results is increased. Therefore, it enables the practitioner to better interpret the assessment results in terms of the underlying assumptions and value choices, which is important in order to apply weighting schemes (Ahlroth 2014).

5 Conclusions and outlook

This paper presents an uncertainty assessment for the target values of the ESM for Germany. The results suggest that the ESM is able to effectively facilitate decision making. The weighting schemes deduced from target values provide a meaningful ranking of alternatives. At the same time, the total impact scores are not excessively sensitive to the uncertainties of the target values, i.e. the inherent uncertainty of the target values does not result in arbitrary results, which is necessary to support robust decisions.

Different sources of uncertainty like different types, the temporal and spatial validity and restrictiveness of target values have been taken into account in order to assess their influence on the LCA results. The rank acceptability index shows that, even though environmental indicators are of course affected by the resulting parameter uncertainty the ranking of the analysed energy generation systems turns out to be quite robust: for eight out of nine technologies it does not vary at all for at least 50% of all Monte Carlo runs. If rank shifting occurs, it is often connected to environmental indicators with high spreads in target values. Especially, restrictive targets in “global warming” (GW) and “land use” (LU) amplify the effect and need to be analysed carefully.

The relatively high uncertainty for technologies with high shares of GW and LU is primarily due to the fact that those targets strive to zero. On a methodical level, target values that strive to zero lead to predominant weights and asymptotically to binary weighting. Therefore, an adequate method for deducing meaningful target values for prohibited substances is necessary. Solutions may be taken from the field of multi criteria decision analysis: e.g. the maximum weight limit or a significance test of the applied weights as suggested by Marler and Arora (2010).

Still, the improvement of existing target values and the definition of politically legitimised targets for as many environmental issues as possible would be favourable to reduce uncertainty when applying the ESM. A consistent approach to consider prohibitive target values would be particularly useful to reduce subjective methodical choices and thus uncertainty. The choice of a specific ESM should adequately reflect the preferences of the decision maker. If possible, different weighting sets should be applied to ensure sound decisions. As uncertainty cannot be completely excluded from the ESM, impact assessment tools should be expanded such as to (a) transparently show the applied target values and their effects on the results to the decision maker and (b) allow for an easy applicable, perhaps even real time sensitivity analysis.