How Climate and Data Quality Impact Photovoltaic Performance Loss Rate Estimations

Different data pipelines and statistical methods are applied to photovoltaic (PV) performance datasets to quantify the performance loss rate (PLR). Since the real values of PLR are unknown, a variety of unvalidated values are reported. As such, the PV industry commonly assumes PLR based on statistically extracted ranges from the literature. However, the accuracy and uncertainty of PLR depend on several parameters including seasonality, local climatic conditions, and the response of a particular PV technology. In addition, the specific data pipeline and statistical method used affect the accuracy and uncertainty. To provide insights, a framework of (≈200 million) synthetic simulations of PV performance datasets using data from different climates is developed. Time series with known PLR and data quality are synthesized, and large parametric studies are conducted to examine the accuracy and uncertainty of different statistical approaches over the contiguous US, with an emphasis on the publicly available and “standardized” library, RdTools. In the results, it is confirmed that PLRs from RdTools are unbiased on average, but the accuracy and uncertainty of individual PLR estimates vary with climate zone, data quality, PV technology, and choice of analysis workflow. Best practices and improvement recommendations based on the findings of this study are provided.


Introduction
The performance loss rate (PLR) represents both reversible (e.g., soiling) and irreversible (e.g., material degradation) losses [1,2] that can occur in a photovoltaic (PV) power plant and is an important parameter for performance modeling, monitoring, and operation and maintenance (O&M).In PV performance modeling, PLR is applied to account for the power reduction over time whereas in monitoring and O&M, it can provide a high-level indication of a power plant's health state.Although PLR has been interchangeably used with the degradation rate (Rd) in the literature, and despite the fact that their calculation approaches are similar, these two metrics are different: Rd represents only the irreversible material degradation of a component (e.g., a PV module) and is only one part of the PLR breakdown. [1]nless all losses are individually monitored (e.g., through a soiling station, periodical module testing under artificial light to estimate degradation, etc.), the true value of PLR (and its breakdown) is unknown when processing only field data, but it is commonly approximated by applying statistical methods on PV performance time series.However, such time series exhibit a seasonal behavior depending mainly on the climate, PV module/cell type, and local operating conditions.Location and climate affect PV seasonality with respect to irradiance and its spectral composition and angle of incidence, ambient temperature, wind speed/direction, etc. PV modules respond differently to these conditions depending on their material.For example, crystalline silicon (c-Si) technologies are characterized by a higher temperature coefficient compared to thin-film technologies.Furthermore, the external quantum efficiency varies between different PV module/cell types, and therefore, the response to spectral variations differs; e.g., amorphous silicon (a-Si) technology is known to be more sensitive to changes in spectrum due to its "narrow" spectral response. [3]Other local conditions, such as the atmospheric composition, rainfall periods, or grid constraints, can also contribute to PV seasonality.
To reduce the signal seasonality and noise for PLR estimations, the PV performance time series are usually processed in different ways with respect to data normalization, applied corrections, aggregation, etc.These steps, however, are not perfect, and the normalized signals still contain fluctuations that affect the accuracy and corresponding statistical uncertainty of PLR estimation.Furthermore, decomposition models can be applied to remove seasonality and noise and extract the trend of PV performance time series.To this end, several methods exist in literature, with varying degree of accuracy, which also depends on the seasonality and noise of PV performance time series.[6] The number of years differs or is often unquantified in literature; this is simply because it depends on the climate, location, PV module type, data processing pipeline, and applied methodology.Furthermore, other effects such as nonlinear performance loss or degradation behavior may also be important [7][8][9][10][11][12] and can increase the statistical uncertainty if the time series are fitted with linear models.[15] Therefore, accurately deriving the real value of PLR from measured data is challenging and fraught with uncertainty.
18] Most of the statistical methods have already been developed and included in open-source libraries allowing for a standardized and reproducible approach. [19]The most widely accepted and used library is called "RdTools" and it was co-developed by the National Renewable Energy Laboratory (NREL). [20]It is a Python package for performance loss, degradation, and soiling rate estimations and it includes three statistical approaches for PLR and Rd: the year-on-year (YoY; default method), ordinary least squares (OLS) and classical seasonal decomposition (CSD).The second library, named "PVplr", was developed in R and it was built as part of the International Energy Agency (IEA) PV Power Systems Programme Task 13 study on the determination and uncertainty of PLR calculations. [21]However, due to the aforementioned challenges of different pipelines for data processing, normalization, corrections, aggregation, statistical methods, and undefined "adequate length of timeseries", there is no proven methodology that would enable a universally standardized procedure for estimating the PLR.For example, the state-of-the-art library RdTools, which serves as the "standardized" procedure for PLR estimation, has only been experimentally verified (i.e., compared to discrete indoor measurements) against a handful of systems, with measurements mainly in Colorado.This study hypothesizes that the PLR estimation and corresponding uncertainty of a system A installed in a "stable" climate will be more accurate and precise compared to an identical system A installed in a more dynamic climate (see, e.g., Figure 1).Therefore, climate-dependent guidelines might be necessary for some locations and systems to ensure accurate PLR estimates with low uncertainty.
To standardize a universal procedure for PLR estimation, multiple validations in different climates must be performed.Because fielding real systems across so many climates would be cost prohibitive, this study reverse engineers the problem of validating PLR/Rd estimation methods by generating %200 million synthetic datasets of known behavior across the contiguous US (and some parts of northern Mexico and southern Canada).These datasets include different types of PV modules, irradiance sensor drift, noise, erroneous data, and, most importantly, known/specified PLR values.This enables systematic analysis of how these variables affect the performance of different analysis workflows.These results can then be summarized to suggest best practices for the PLR estimation and identify sources of uncertainty in the RdTools pipelines.The RdTools library was selected as the most comprehensive, verified, and used opensource tool currently available and the objective of this work was to assess its sensitivity under different conditions and suggest best practices for minimizing uncertainty and improving reproducibility.In this study, phenomena such as soiling, snow, shading, inverter clipping, etc., are intentionally not considered.Incorporating all phenomena into these synthesized signals would recreate the same problem the PLR estimations are already facing in field measurements: the challenge of noisy and messy datasets without ground truth.This is a first step in multi-climate assessment of "standardized" PLR pipelines found in RdTools.If these pipelines are unable to quantify PLR under these "ideal" conditions, then additional concerns are warranted.
Investigations using synthetic datasets and Rd confidence interval (CI) calculations have been performed by Jordan et al. [19] using regression and YoY approaches with different levels of outliers, soiling, number of years, and seasonality whereas Moser et al. [22] performed an inter-analyst comparison applying individual data pipelines and methods on given synthetic datasets.Furthermore, the influence of data filtering and different performance metrics on Rd was studied using indoor measurements by Jordan and Kurtz. [23]To provide insights on how different analytical decisions can influence the estimation and confidence of PLR, this study parameterizes various conditions that can occur in real life and performs the largest ever multi-climate assessment of PLR estimation.

Methodology
The methodology for the synthetic dataset generation and PLR evaluation is shown in Figure 2. Hourly solar resource and meteorological data were retrieved from the National Solar Radiation Database version 3.1.1 [24]from 1998 to 2020 over a 0.5°Â 0.5°s patial grid encompassing the contiguous US with some locations from southern Canada and northern Mexico (total of 3891 locations).Four different module types (LG passivated emitter rear totally diffused [n-PERT], Panasonic silicon heterojunction [SHJ], Canadian Solar monocrystalline aluminum back surface field [Al-BSF], and First Solar cadmium telluride [CdTe]) were simulated using the Sandia PV Array Performance Model (SAPM) [25] in pvlib-python version 0.9.5. [26,27]All input data to the SAPM model were sourced from IEC 61853-1 testing (data and reports are available at https://pvpmc.sandia.gov/pv-research/pvlifetime-project/pv-lifetime-modules/)and the modules were all normalized to 300 W to avoid any possible bias due to varying nameplate values.The Canadian Solar module is the default module in all sections except when the PLR convergence is examined for different PV technologies.
PLR profiles were then applied to the performance data to account for a total performance loss equivalent to À0.75% year À1 , which is the median system-level PLR value in the US according to the findings of the PVFleets program. [28]The same PLR patterns were emulated independently of the PV module technology.
Outlying data are essentially treated similar to missing data (i.e., set as Not A Number [NaN] or Not Available [NA] before being imputed or filtered out) once they are detected in a given dataset. [29]Therefore, the term "missing" will be used for both outlying and missing data.A random missing data rate of 5% was added to all datasets to account for random and outlying data, in addition to continuous missing data periods at 0, 5, 15, and 30% rates.The continuous missing data were distributed in 7and 30-day "chunks" to account for short-and long-term system and/or sensor outages or errors.Irradiance sensor drift of 0, À0.1, and À0.5% year À1 and random Gaussian noise (based on generic sensor accuracies) in power (standard deviation [sd] = AE1 W), irradiance (sd = AE6 W m À2 ), and temperature (sd = AE0.4°C) were also added.
A large parametric analysis including different performance metrics (performance ratio, PR, and temperature corrected PR, PR TC ), aggregation steps (daily, weekly, monthly), dataset length (2-22 years), and trend-fitting methods (YoY, OLS, CSD, seasonal-trend decomposition with Loess [STL]) was performed according to the process in RdTools version 2.1.6. [20,30]his includes filtering (for effective plane-of-array irradiance and cell temperature; inverter clipping was not considered in this study), and the YoY, OLS, CSD methods; STL was added due to the reliable results it demonstrated elsewhere, [15] despite not being included in RdTools.For a given dataset length, each PLR estimation is repeated as a moving window (e.g., if the dataset length is 4 years, then a PLR is estimated for every 2 years: 0-4, 2-6, 4-8,…, 18-22 years; a single PLR value is only extracted when the dataset length is 22 years).For each moving window, the PLR is calculated, and the statistical uncertainty is extracted using the default RdTools method and a confidence level of 95%.This analysis reports uncertainty using the CI width, which is the difference between the lower and upper CI bounds.CIs provide a way to express the precision of an estimate by giving a range of plausible PLR values.Therefore, a 95% confidence level means that if the sampling process was repeated multiple times, about 95% of the intervals would contain the true PLR value.This study uses the CI width instead of upper/lower bounds because the RdTools bootstrapping approach for CI estimation usually results in nonsymmetric lower and upper bounds.
The performance of the PLR estimates is quantified in three ways: 1) the accuracy (residual) of the PLR estimate itself, 2) the PLR uncertainty as reported by the estimation method, 3) whether the stated uncertainty is correct, i.e., whether the reported CI contains the true (known) PLR.
All simulations were made possible using Sandia's highperformance computing (HPC) resources.To generate the synthetic datasets, the HPC utilized %2200 cores and completed the simulations in %12 h: a total of %26 000 core-hours in total.This is equivalent to %9 months of simulations on a 4-core machine.
This study focuses on characterizing the performance of PLR estimation methods when applied to the most ideal conditions that could occur in real scenarios.Confirming the accuracy of these methods (or identifying sources of significant uncertainty in their results) in this "base case" is a necessary first step toward comprehensive multi-climate assessment of PLR estimation methods.Therefore, as mentioned in the introduction, soiling, snow, shading, inverter clipping, and other such real-world performance complications are omitted from the simulated datasets considered here.The results of this work will provide a basis for future study of PLR estimation in the face of these more complex performance effects.

Statistical Methods: YoY Versus OLS, CSD, and STL
The performance of the YoY method, which is the preferred method in RdTools, was compared against OLS, CSD, and STL. Figure 3 evaluates the accuracy and uncertainty of these four statistical methods by comparing a) the estimated PLR, b) CIs (i.e., CI widths), and c) whether the true PLR falls within the CI bounds or not (represented as a fraction).These datasets are temperature corrected, aggregated daily, and they do not include any continuous missing data whereas their true PLR is linear at À0.75% year À1 (dotted horizontal line in Figure 3a).Although the median-estimated PLR values across all 3891 locations and time windows are very close to the true PLR, the lower and upper whiskers and outliers of the decomposition models are up to AE0.05% year À1 wider than the YoY and OLS.When comparing the CI widths in Figure 3b, the decomposition models exhibit tighter whiskers as compared to the other methods with CSD and STL within %0.05 and %0.1% year À1 , respectively.This is because the CIs are estimated from the CSD and STL decomposed signals, which are much "flatter" than when considering a signal with its trend, seasonality, and residual components.However, while commonly the CI estimation of decomposition models incorporates the residual into the trend component, the RdTools CI estimation only considers the moving average, resulting in a very low CI width.Are the true PLR values within the bounds of these CIs? Figure 3c shows the correct fraction of CIs where CSD seems to fail for more than 90% of the time whereas the STL CIs are incorrect by up to %30% when dataset lengths exceed 4 years.The OLS and YoY CIs are correct for more than 85% of the time.This is an indication that RdTools should also consider the residuals when estimating the CSD's uncertainty.In this "perfect" case of no missing data, the OLS residuals are comparable to YoY and, although the OLS CIs are tighter for dataset lengths over 2 years, they are also less reliable.Considering the known OLS sensitivity to missing data and incomplete datasets, this short comparison validates previous findings [19,30] that YoY outperforms other statistical methods.Therefore, the YoY method will be investigated in the remainder of this manuscript according to the steps provided by RdTools.

Climate and Data Quality Sensitivity of the RdTools YoY Approach
Having ruled out the OLS, CSD, and STL methods, the remainder of the analysis will focus on the sensitivity of the RdTools' YoY approach on climate, data processing, and data quality.

Data Processing
Data processing in a PLR pipeline involves more steps than the ones examined in this study.The focus here is on aggregation and temperature correction while different types of normalization are not examined due to the restrictive nature of synthetic datasets (e.g., performance index is not possible without measured data, hence the focus is on the PR).

Aggregation
The default aggregation level in RdTools is set to daily to allow an adequate number of YoY deltas for the PLR extraction.Figure 4 shows the impact of different aggregation levels on the a) estimated PLR, b) CIs (i.e., CI widths), and c) CI correct fraction for different dataset lengths.These datasets include a linear PLR of À0.75% year À1 and 0% continuous missing data due to outages.Although the median-estimated PLRs for all aggregation levels are very close to the true PLR, the lower and upper whiskers of the daily aggregation are slightly wider (<0.03% year À1 ) than the weekly and monthly aggregation, especially considering those locations that appear as outliers in the boxplots.These trends appear to shrink over time with outliers ranging %AE0.25% year À1 in a 2-year dataset, down to %AE0.05% year À1 after 10 years of data.The CI widths follow a similar trend but, in this case, the coarser YoY deltas due to the monthly aggregation have an impact on the statistical uncertainty of the YoY method exhibiting increased widths as compared to the finer aggregation levels.The monthly aggregation slightly outperforms the other aggregation levels, although this difference is lower than 1%; all aggregation levels achieve >98% of correct CI bounds after 4 years of data.Although the weekly aggregation seems to provide a good balance between accuracy, uncertainty, and number of YoY deltas, the remainder of this analysis will continue with the RdTools default daily aggregation.

Temperature Correction
It is expected that temperature correction will reduce a signal's fluctuations but, module temperature measurements might not be available, especially in larger power plants.Since the YoY method is considered to be insensitive to seasonality, [19] the uncertainty due to unavailability of temperature measurements or the incorrect application of temperature coefficients (γ) should be quantified across different locations.Figure 5 shows the a) estimated PLR, b) CI widths, and c) "CI correct fraction" of the YoY method at different dataset lengths for 1) PR TC , 2) PR TC corrected with an incorrect power temperature coefficient (À0.30% °CÀ1 instead of À0.41% °CÀ1 ), and 3) noncorrected PR.Temperature correction improves the PLR estimation accuracy and its corresponding uncertainty by an order of magnitude.While the correct PR TC PLR is within %AE0.25% year À1 , the PR PLR exceeds %AE1% year À1 with a dataset length of 2 years.No temperature correction also causes a high statistical uncertainty with widths up to %2% year À1 whereas the PLR values are outside the CI bounds up to 35% of the time, when dataset lengths are lower than 4 years.An inaccurate temperature correction will also impact the accuracy and uncertainty of the PLR estimation.In this scenario, where γ is underestimated by 0.11% °CÀ1 , the PLR whiskers increase from %AE0.25 to %AE0.50% year À1 .The highest penalty is found on the estimated uncertainty where an absolute drop of up to %20% is found in the correct CIs when using incorrect temperature coefficients; this is reduced with longer time series (>5 years).Therefore, the YoY method is not insensitive to the seasonal and daily variability caused by temperature, and temperature corrections should be mandatory in PLR assessments.Although sometimes the available temperature coefficients are not 100% accurate, it should not be assumed that they do not impact the PLR estimations and uncertainty, especially in shorter time series (<5 years).Finally, when temperature data are unavailable, it is  recommended to use nearby weather stations rather than ignoring temperature.Temperature correction will be applied for the remainder of this analysis.

Data Quality
The quality of field measurements is crucial for the performance and reliability analysis of PV systems.Significant bias may be introduced due to erroneous data caused by outages and component failures or uncalibrated sensors and this may depend on dataset length, outage durations, and/or climate.

Missing Data
The missing data were introduced in a way to represent both random outliers and NA/NaN values (fixed at 5%) and continuous missing gaps due to outages, communication issues, etc.The continuous missing data were added in 0%, 5%, 15%, and 30% rates with "chunks" (durations) of 7 and 30 days to account for short-and long-term issues.Figure 6 shows the impact of missing data on the a) estimated PLR and b) its corresponding CIs, at different rates, outage durations, and dataset lengths.The correct fractions of CIs at increasing dataset lengths are shown in Figure 6c.Overall, the higher the missing data rates, and the longer the outage period, the lower the PLR accuracy and higher the uncertainty is.While outage-free datasets will achieve CI widths within 0.2% year À1 after 5 years, 30% of missing data rates will require 10 years to achieve the same CI with a still lower fraction of correct CIs, as compared to the outage-free dataset.
As such, the dataset length required for an accurate and precise PLR calculation depends on the amount of missing or erroneous data, which raises the question of whether the relationship between dataset length and erroneous data is linear or not.Would a 10-year dataset with 30% missing data be the equivalent of a complete 7-year dataset for a PLR evaluation?To quantify the number of "effective" years based on data quality, a new metric is proposed in this study: the "Effective" dataset length based on the reported CI width.Figure 7 illustrates the relationship between the effective dataset length versus the actual dataset length (i.e., the number of years between the first and last measurements of a particular system; Figure 7a) and the ratio between effective dataset length and actual dataset length (effective fraction; Figure 7b).The dotted lines assume linear relationship for 5%, 15%, and 30% missing data.The solid lines were estimated in two steps.The first was determining the median CI width as a function of dataset length, considering only the simulations with no continuous outages.Then, for any given CI width, this curve could be linearly interpolated "in reverse"  and used to estimate the number of outage-free years necessary to produce that width.It can be observed that the effective dataset lengths (i.e., the dataset length to achieve the same CI as when erroneous data are not present) are shorter than the normal dataset lengths.For example, for a 10-year dataset with 30% missing data, the effective length becomes %5.5 years, which is a %45% "effective dataset length loss" and not 30%.Figure 7b shows that the gap between effective dataset length and the real length is amplified with increasing missing data rates while a slow relatively steady state can also be observed after 10 years of data.

Irradiance Sensor Drift
Irradiance sensor drifts will introduce a bias when assessing a power plant's PLR.Assuming that this phenomenon is linear, À0.1 and À0.5% year À1 rates of sensor drifting were added to the irradiance measurements.RdTools recommends using the clear-sky normalization when ground-based irradiance sensors are not reliable (e.g., due to drifting, unavailability, misalignment, etc.).The default RdTools clear-sky approach assumes steady-state conditions, which is not realistic, considering that weather and atmospheric conditions may vary significantly from 1 year to another. [31]In Figure 8, the left column shows the impact of irradiance sensor drift, whereas the right column applies the RdTools clear-sky method to examine whether it can reduce the uncertainty due to drifting or not. Figure 8a shows that sensor drifting has a compounding effect on the estimated PLR meaning that it causes positive bias in an almost linearly dependent manner (see the À0.65 and À0.25% year À1 dotted lines to guide the eye) whereas the impact on the CI width is negligible (Figure 8b).However, this negligible effect does not mean that the CIs are also correct: depending on the amount of time that the sensor has been drifting, the CI estimates will be incorrect for most of the time (see Figure 8c).In contrast, the whiskers of the RdTools clear-sky normalization method in Figure 8d exhibit a significantly large spread ranging from %À3 to %þ2% year À1 (when the dataset length is 2 years) to %À1.5 to %0% year À1 (when the dataset length is 10 years).Therefore, although the median clear-sky-based PLRs are relatively close to the true PLR value, this method seems to fail most of the time.Although the uncertainty's correctness in Figure 8c is >80% after 3 years of data, the CI widths in Figure 8e are extremely wide making any PLR assessment not practically useful.
To examine whether the clear-sky normalization method is climate dependent or not, multiple maps were generated as shown in Figure 9.These maps show the mean absolute PLR residual (left column: a,d), mean CI width (middle column: b,e),  As expected, the highest errors and uncertainties occur at locations characterized by cloudy conditions whereas relatively clear sky locations seem to exhibit lower errors.This highlights that the RdTools clear-sky normalization method is dependent on climate and caution should be given when reporting clear-sky-based PLR estimates.Replacing the current relatively simple clear-sky irradiance model in RdTools with a more accurate model might make this sensorless approach more useful and less climate dependent.
Periodic calibrations and maintenance are recommended for ground-based sensors.When ground-based measurements are unreliable, satellite data in combination with clear-sky filtering could be leveraged to avoid any bias due to irradiance sensor drifting [32] (although the uncertainty of the satellite data must be considered as well); this is being incorporated in RdTools.Nighttime "irradiance" measurements can provide useful information on the calibration of a particular sensor.Therefore, in cases where the irradiance sensor drift can be known, the PLR estimation could be corrected using this formula where "PLR" is the estimated value using RdTools, DR is the sensor drift rate, and L is the dataset length in years.

Climate and Technology
Most of the impacts that are discussed in this study have (at least partially) been studied elsewhere, but either using a single location [33] or pseudo data. [19]The main hypothesis in this work, and as mentioned earlier in the manuscript, is that climate and how a PV technology responds to it will play a significant role in the convergence ability, accuracy, and uncertainty of RdTools (or any other PLR pipeline).

Climate
In Figure 10, the spatial variability of YoY uncertainty is demonstrated across all locations.These data do not include any continuous missing data, the time series are temperature corrected and aggregated daily.The top (Figure 10a-d As expected, P90 level of confidence will also require longer datasets.Therefore, attention is required when evaluating power plants in different climates, because the same assumptions (e.g., dataset length and associated uncertainty) do not apply everywhere.
To quantify these uncertainties as a function of climate, the locations were divided according to their Koppen-Geiger (KG) climate zone [34] using the kgcPy package. [35]Only climate zones with over 50 locations were considered.As such, 104 out of 3891 locations are not displayed in the climate-dependent plots.Therefore, most "B" (dry), "C" (temperate), and "D" (continental) KG zones are represented in these analyses.Figure 11 demonstrates a strong climatic dependence of PLR estimations (Figure 11a) and CI widths (Figure 11b) in different dataset lengths.The dry climates (with 1st letter "B") exhibit a much faster convergence with lower statistical uncertainty as compared to the "C" and "D" climates; Csa is an exception due to its hot and dry summers, which follow a similar pattern to the "B" climates.While the "B" and Csa climates achieve PLR values within AE0.1% year À1 and CI widths <0.2% year À1 after 4 years, the remaining "C" and "D" climates require at least %5-6 years.Overall, the correctness of the CIs remains relatively constant above 98% after 4 years of data (see Figure 11c).
The results here verify that the study's hypothesis of climatic dependence in PLR estimations (see Introduction) is true: i.e., PLR estimations for systems installed in locations with "stable" conditions will be more accurate and less uncertain when compared to identical systems installed in more "dynamic" climates.This is because "stable" conditions exhibit lower irradiance, temperature, or other atmospheric oscillations, and therefore, they favor the statistical extraction of PLR.When the conditions are more "dynamic" (i.e., moving clouds, higher temperature fluctuations, etc.), the statistical approaches struggle resulting in lower accuracy and higher uncertainty.

PV Technology
Is the RdTools YoY approach a technology-independent pipeline?Would it converge to the correct PLR with low uncertainty equally among technologies?Figure 12 demonstrates that PLR accuracy and uncertainty may vary based on the seasonality that a particular technology will exhibit.For example, due to light or thermal behaviors, a CdTe PV module will respond differently when exposed outdoors, as compared to a crystalline silicon module.Even if corrections are applied when analyzing the data, the signal oscillations are not fully eliminated.This also has an impact on the accuracy and uncertainty of PLR estimations (Figure 12a), CI widths (Figure 12b), and the correctness of these CI widths (Figure 12c).Although the PLR and uncertainty estimates of the Canadian Solar, LG, and First Solar modules are comparable, Panasonic's SHJ exhibits relatively larger deviations from the true PLR value, but significantly larger CI widths (up to 0.3% year À1 as compared to 0.2% year À1 after 5 years).Performance time series analysis showed that this was mainly due to SHJ's behavior in low light conditions (details can be found elsewhere [36] ).For this reason, Panasonic's SHJ PLR was reestimated on datapoints with irradiance over 500 W m À2 (purple boxplots and line).As can be seen, this filtering approach improves the overall PLR estimation accuracy and uncertainty by up to %AE0.1% year À1 .This shows that different data processing Figure 10.Maps demonstrating the spatial variability of YoY uncertainty across all locations considered in this study.From left to right, the columns correspond to CI widths lower than a,e) 0.075% year À1 , b,f ) 0.1% year À1 , c,g) 0.15% year À1 , and d,h) 0.2% year À1 .The top and bottom rows show the P50 and P90 estimates, respectively.These data do not include any continuous missing data, the time series are temperature corrected and aggregated daily and their true PLR is linear at -0.75% year À1 .The maps were created using Cartopy, v0.21.1, 12-Sep-2022, Met Office, UK and data from Natural Earth.may be required for PV technologies exhibiting atypical behavior to reduce their sensitivity to different conditions.

Nonlinearities
Although PLR or Rd might not be linear over a system's lifetime, the RdTools YoY pipeline assumes linearity.This is done by reporting the median value of a YoY deltas distribution as the estimated PLR or Rd.To investigate how nonlinear performance loss profiles may impact the YoY PLR estimations, these were compared against the three theoretical profiles from Figure 13: stabilization, wear out, and both (stabilization þ wear out).These profiles assume abrupt changes in performance loss whereas in reality, this change might be more gradual.While no significant climate dependence was found on the nonlinear PLR estimation, Figure 14 shows the YoY deltas distributions at a single location for different dataset lengths and nonlinear patterns.The red vertical lines indicate the mean true PLR, whereas the orange and green lines show the median and mean PLR values from RdTools.The orange horizontal line indicates the median PLR's uncertainty.While the median-estimated PLR is consistently close to the true mean PLR within 0.06% year À1 before any nonlinearity occurs, it can either under-or overestimate the true PLR by up to %0.5% year À1 once a smaller distribution of deltas appears.This is because the median YoY PLR "ignores" any skewing caused by higher or lower observations or outliers.However, these smaller distributions represent a different PLR behavior.In this case, the mean-estimated PLR value (green line) is more accurate in calculating a representative performance loss (in %) at a given time, since it follows the true mean PLR closely by an up to 0.14% year À1 difference.Therefore, if the time series are not heavily skewed by outliers and a nonlinear PLR profile is suspected, the mean-estimated PLR should be used instead of the median.Alternatively, PLR can be calculated in a 2-year moving window and report multiple values representing the nonlinear PLR changes (see ref. [2]).

RdTools Potential Improvement Recommendations
This study identified a few potential improvements for RdTools, which are recommended in this section.
A minor flaw in the PLRs reported by the RdTools implementation of the YoY method was found (as of RdTools version 2.1.6).Because the RdTools implementation normalizes performance relative to the entire first year of production, the normalized performance begins somewhat above 100% on the first day and only reaches 100% somewhere in the middle of the first year.This causes the estimated PLR to be inflated by a small amount compared to what would be reported if the normalized performance began exactly at 100%.To address this flaw, the following equation is proposed to correct this minor bias Figure 13.PLR profiles that were added to the time series.All profiles correspond to the same performance loss after 22 years, which is equivalent to a linear PLR of À0.75% year À1 (blue line).The nonlinear profiles were selected to represent 1) stable initial PLR increasing during the wear-out phase (green line), 2) a higher initial PLR that then stabilizes during midlife and wear out (orange line), and 3) increased initial and wear-out PLR (red line).
where PLR is the estimated value by the RdTools' YoY method.For example, a reported PLR of À0.7500% year À1 would be corrected to À0.7472% year À1 .Although this is recommended for accuracy improvements, it was not applied in this manuscript to ensure consistency with the RdTools v. 2.1.6and avoid any potential bias.The RdTools CSD method suffers from incorrect estimations of the statistical uncertainty.Section 3 shows that although its statistical uncertainty appears to be much lower than OLS and YoY, it fails more than 90% of the time, even when the datasets do not suffer from erroneous data.Therefore, either incorporating the residual component to the CI estimation or developing a different uncertainty calculation is recommended.
The relatively simple clear-sky method could be improved by either replacing it with a method (e.g., using satellite data with clear-sky filtering [32] ), or by applying some siteadaptation techniques to remove bias.The existing clear-sky method appears to be dependent on climate and fails completely at locations characterized by more dynamic conditions.
The option for reporting mean (in addition to median) PLR/ Rd could be useful in time series exhibiting nonlinearities.Mean PLR/Rd is more representative of the total performance loss at a given time, given that the time series are not heavily skewed by outliers.Alternatively, a 2-year moving window could be used to estimate multiple PLR values to account for nonlinearity.Caution should be taken when the beginning date of the moving window is in an already degraded state; in this case, the PLR estimation should be corrected to account for this offset.
Although all CIs in this work were calculated with the confidence level set to 95%, as many as 99% of them actually contained the true value (see Figure 11), suggesting that the RdTools YoY uncertainty calculation is overly conservative and reports unnecessarily wide CIs.In fact, as shown in Figure 15, the 95% CIs of Figure 11 could have been tightened by 20-30% without causing the "failure rate" (fraction of CIs not containing the true PLR) to exceed 5%.However, that CI tightening factor varies based on dataset length and climate.This highlights the need for the development of an alternative uncertainty calculation that more accurately represents the desired confidence level.

Challenges and Recommended Best Practices in PLR Calculation
Although this study has not considered any soiling, snow, inverter clipping, etc., several recommendations can be made based on the observations.Use the RdTools YoY method for PLR calculations.This study verified its robustness across large geographic regions and compared it to OLS, CSD, and STL.The decomposition methods suffer from incorrect statistical uncertainties whereas the OLS is sensitive to seasonality and provides less reliable CIs.
Correct RdTools YoY offset using the proposed equation (Equation ( 2)), assuming that RdTools will not be incorporating this in a future version.
Apply weekly instead of daily aggregation in RdTools.This analysis showed that the spread in residuals and CI is wider with daily aggregation, whereas monthly aggregation may suffer from the lower amount of datapoints.
Perform temperature correction but recognize that the accuracy of temperature coefficients will impact the PLR estimations and associated uncertainty, especially in shorter time series (<5 years).When temperature data are unavailable, it is recommended to use data from nearby weather stations, recognizing that this will affect accuracy.
Use the "Effective dataset length" to account for the influence of erroneous (missing, outlying, etc.) data.The relationship between dataset length and the amount of missing data is not linear.For example, 30% missing data in a 10-year dataset is not equivalent to a 7-year dataset with no missing data for a PLR evaluation.Table 1 and 2 provide the minimum number of years for a given uncertainty, and erroneous data at different climates, for P50 and P90 estimates, respectively.These tables should be used as a general guidance: a noisier signal consisting of snow, soiling, inverter clipping, or other phenomena is expected to increase the required number of years.Perform periodic calibrations and maintenance of groundbased sensors.When such measurements are unreliable, satellite data can be substituted.If the irradiance sensor drift is known, the PLR estimation can be corrected using the proposed equation (Equation ( 1)).
Do not have the same uncertainty expectations for every location, climate, and scenario when using the RdTools YoY pipeline; it is not a climate-or technology-independent process.
Consider that the uncertainty is highly dependent on the time series length and that the minimum of 2 years might not achieve an acceptable uncertainty in some climates.
Use the mean YoY PLR delta instead of the median when a nonlinear PLR is suspected unless the system is dominated by a long tail of outliers.The mean PLR is more representative of the total performance loss (%) of a system at a given time whereas the median might "ignore" the smaller deltas distribution considering them as outliers.

Conclusions
PLR estimation is challenging and lacks climate-, technology-, and data-quality-dependent validations.This analysis showed that when these factors are not considered or are weighted equally, significant error is introduced.
Overall, this analysis verifies previous findings demonstrating that the RdTools YoY method is robust when compared to linear regression and decomposition models.However, although this method performs with high accuracy and low uncertainty in certain locations and PV module types, this is not the case for all climates and technologies.This study showed that the southwestern US and northern Mexico demonstrate a faster convergence with lower uncertainty as compared to the eastern and northern US.Therefore, some climates will require a longer time series to achieve the same accuracy and uncertainty compared with a different climate.This also holds true for different PV technologies.
Temperature correction from a nearby station may be better than no correction at all, and the common assumption that "any temperature coefficient would work" is not valid, since this tends to bias the PLR estimations, especially in shorter dataset lengths.The clear-sky-based PLR method was shown to be unreliable in relatively cloudy locations.Satellite data with clear-sky filtering could be leveraged when a sensor is drifting, although the uncertainty of such data should also be considered.With respect to erroneous or missing data, it was shown that the amount of such data should not be assumed to be linear.Indicative tables were presented to show the minimum amount of data that is required to achieve a given uncertainty level at different erroneous/ missing data rates.This study highlighted that specific climate-, technology-, and data-quality-dependent assumptions should be made when running RdTools YoY in PLR assessments to increase accuracy and lower uncertainty.Future work will quantify the uncertainty reduction when using weather data from a neighboring station.Furthermore, the synthetic time series will be informed on topical phenomena such as soiling, snow, and inverter clipping and quantify possible uncertainty changes.

Figure 1 .
Figure 1.An illustration of performance loss rate (PLR) estimation using the ordinary least square (OLS) method on synthetic and temperature corrected performance ratio (PR TC ) time series in two Koppen-Geiger (KG) climates (Dfb: continental climate without dry season, warm summer, and BWh: hot desert climate) with the same emulated linear PLR of À0.75% year À1 .PLR calculation is highly dependent on climate resulting in two different estimated PLR values with even the incorrect confidence interval (CI) bounds.

Figure 2 .
Figure 2. The synthetic dataset generation on the Sandia's high-performance computing (HPC) was broken into multiple steps.These are colored to show the various simulation stages with different tools.And, 22 years of performance have been simulated for four different module types and four different methods, with ranges of missing data and aggregation.The PLR estimation was performed using the RdTools library (i.e., filtering and usage of year-on-year [YoY], OLS, classical seasonal decomposition [CSD]).The simulations took approximately 26 000 core-hours.

Figure 3 .
Figure 3.Comparison of statistical methods for PLR calculation including YoY, OLS, CSD, and seasonal-trend decomposition with Loess (STL): a) estimated PLR, b) CI widths, and c) the fraction of correct CIs as a function of dataset length in years.These datasets are temperature corrected, aggregated daily and they do not include any continuous missing data whereas their true PLR is linear at À0.75% year À1 (dotted horizontal line in (a)).

Figure 4 .
Figure 4. Impact of different aggregation levels on the a) estimated PLR, b) width of CIs, and c) the fraction of correct CIs as a function of dataset length in years.These datasets are temperature corrected and they do not include any continuous missing data whereas their true PLR is linear at À0.75% year À1 (dotted horizontal line in (a)).

Figure 5 .
Figure 5. Impact of temperature correction on the a) estimated PLR, b) width of CIs (middle),and c) the fraction of correct CIs as a function of dataset length in years.Three scenarios are investigated: 1) no temperature correction (γ = 0% °CÀ1 ), 2) temperature correction with incorrect temperature coefficient (γ = À0.30%°CÀ1 ), 3) temperature correction with correct temperature coefficient (γ = À0.41%°CÀ1 ).These datasets are aggregated daily, they do not include any continuous missing data and their true PLR is linear at À0.75% year À1 (dotted horizontal line in (a)).

Figure 6 .
Figure 6.Impact of missing data on the a) estimated PLR, b) width of CIs, and c) the fraction of correct CIs as a function of dataset length in years.These time series are temperature corrected, aggregated daily, and their true PLR is linear at À0.75% year À1 (dotted horizontal line in (a)).

Figure 7 .
Figure 7. a) "Effective" dataset length and b) "Effective" fraction (i.e., the ratio of "effective length" and dataset length) versus dataset length in years.The dashed lines indicate the missing data fractions assuming linear relationship.

Figure 8 .
Figure 8. Impact of irradiance sensor drift (left column) on the a) estimated PLR, b) width of CIs, and c) the fraction of correct CIs as a function of dataset length in years.The right column shows the d) estimated PLR, e) CI width, and f ) fraction of correct CIs after applying the RdTools clear-sky normalization method.These time series are temperature corrected, aggregated daily, without continuous missing data and their true PLR is linear at À0.75% year À1 (dotted horizontal line in the top (a) and (d) plots).The À0.65 and À0.25% year À1 dotted horizontal lines are to guide the eye.
and CI correct fraction (right column: c,f ) for a 3-year dataset length (top row: a-c) and a 10-year dataset (bottom row: d-f ).
) and bottom (Figure10e-h) rows show the P50 and P90 estimates, respectively.From left to right, the columns indicate statistical uncertainty with CI widths of 0.075, 0.1, 0.15, and 0.2% year À1 whereas the color bar shows the number of years required to achieve the indicated uncertainty.Strong climatic dependence can be observed across all locations.The southwest US and northern Mexico demonstrate a faster convergence within the given CI widths, whereas the rest of the US and southern Canada will require longer duration datasets to achieve the same statistical uncertainty.

Figure 9 .
Figure 9. Maps illustrating a,d) the mean absolute residual (left column), b,e) mean CI width (middle column), and c,f ) CI correct fraction (right column) when applying the RdTools clear-sky normalization in 3-year dataset length (top row) and 10-year dataset (bottom row), respectively.These time series are temperature corrected, aggregated daily, without continuous missing data, and their true PLR is linear at À0.75% year À1 .The application of clear-sky normalization is dependent on climate.The maps were created using Cartopy, v0.21.1, 12-Sep-2022, Met Office, UK and data from Natural Earth.

Figure 11 .
Figure 11.Climatic dependence of a) PLR estimations, b) width of CIs, and c) the fraction of correct CIs as a function of dataset length in years.These time series are temperature corrected, aggregated daily, and their true PLR is linear at À0.75% year À1 (dotted horizontal line in (a)).The different climates are classified based on the KG climate zones.

Figure 12 .
Figure 12.PV technology dependence of a) PLR estimations, b) width of CIs, and c) the fraction of correct CIs as a function of dataset length in years.These datasets are temperature corrected, aggregated daily, they do not include any continuous missing data and their true PLR is linear at À0.75% year À1 (dotted horizontal line in (a)).Due to Panasonic's stronger low light dependence, its PLR was reestimated on time series with irradiance over 500 W m À2 .

Figure 14 .
Figure 14.YoY deltas distributions at a single location for different dataset lengths (4, 6, 8, 10 and 22 years) using time series with emulated "stabilization" (left column), "wear-out" (middle column), and "stabilization þ wear-out" (right column) nonlinear profiles from Figure13.The red vertical lines indicate the true mean PLR, whereas the orange and green lines show the median and mean PLR values from RdTools, respectively.The black dotted vertical lines are the emulated nonlinear PLRs (i.e., À3 and À0.25% year À1 according to Figure13).The orange horizontal line indicates the median PLR's uncertainty.The solid vertical lines are quantified in the plots.

Figure 15 .
Figure 15.Potential uncertainty reduction for the RdTools YoY in different climate zones and dataset lengths.

Table 1 .
Dataset lengths (in years) required for different combinations of median confidence interval widths and erroneous data rates.

Table 2 .
Dataset lengths (in years) required for different combinations of 90th percentile confidence interval widths and erroneous data rates.