Implications of a regional resource adequacy program for utility integrated resource planning

Resource adequacy (RA) is the ability of an electricity system to reliably satisfy loads using its available resources. Assessing and maintaining RA is becoming more challenging due to increasing coal plant retirements, penetration of wind and solar resources, reliance on bilateral and market transactions, and emerging technologies. RA evaluation and planning have traditionally been conducted by utilities and overseen by their state regulators using integrated resource planning (IRP) processes. However, the Northwest Power Pool (NWPP) is developing a proposal for a regional RA program in the Western U.S. that would set and enforce capacity obligations for member utilities, and achieve RA more cost-effectively by pooling resources and load profiles from across the region. In this paper, we investigate the policy implications of a regional RA program for existing IRP regulations, with the proposed NWPP RA program as our main object of study. We compile the RA assessment practices of Western U.S. utilities, the proposed NWPP RA program design, and lessons from the historical experience of the Southwest Power Pool’s RA program. Our analysis reveals that the IRP components which would be most heavily impacted by the regional program are RA targets, load forecasts, capacity accreditation factors, and transmission upgrades. We conclude by discussing the policy issues that RA program design and state IRP policy would have to address.


Introduction
Resource adequacy (RA) refers to the ability of an electric power system to meet demands for electricity using its supply-side and demand-side resources (NERC, 2011). Monitoring and maintaining RA is becoming increasingly complex and challenging due to plant retirements and higher penetration of variable renewable energy resources that translate to higher uncertainty on the amount of generation that will be available during peak demand periods. This challenge is becoming particularly acute in the Pacific Northwest region (PNW) due to states' environmental policy objectives and evolving resource economics that are prompting impending retirement of coal plants (NWPCC, 2018). A recent study showed that the region could present RA issues as early as 2020 and highlighted the need for substantial reform in RA practices to meet reliability standards in the next decade (E3, 2019).
The task of assessing, monitoring, and planning for RA has typically been performed by balancing area authorities (BAAs). Traditionally, BAAs were electric utilities, most of which developed their RA assessments as part of their integrated resource planning (IRP) processes. Over time, Regional Transmission Organizations (RTOs) and Independent System Operators (ISOs) have adopted BAA roles for the load serving entities (LSEs) in the regions in which they operate. Compared to individual LSEs' RA assessments, regional RA programs can exploit resource, load, and transmission diversity given their expansive footprints and achieve cost savings by pooling capacity resources. While RTOs and ISOs determine RA targets and monitor member compliance to meet these targets, states retain decision-making power over how to meet those targets (i.e., the future resource mix) and how to allocate costs, among other choices.
As a response to challenges in the PNW region, the Northwest Power Pool (NWPP) has begun developing a proposal for a voluntary regional RA program. The NWPP proposal acknowledges the potential overlap with states' IRP processes, focusing on their differences and how they complement each other (NWPP, 2019). In contrast, this paper is focused on how a regional RA program and IRP processes overlap and highlights key resource planning components that may be impacted when a utility joins a regional RA program. For example, a potential technical conflict between IRP and a regional RA program is that both are set up to produce a capacity requirement, or RA target, for IRP-regulated entities. IRP produces an RA target by combining the LSE's peak demand and a designated planning reserve margin. A regional RA program also produces an RA target based on a different set of methods and assumptions. It would be a policy failure for an LSE to find itself having to meet two different capacity requirements.
One critical question for states in the NWPP footprint whose utilities may join a regional RA program is how much control over RA they will have to give up and what the impacts of giving up this control are on other aspects of state energy policy. States exercise control over resource planning through IRP regulations, and hence this paper is focused on the interactions between IRP and a regional RA program. This paper does not (1) advocate for or against a regional RA program for the NWPP or any other regional RA program, (2) make detailed design recommendations for this program, or (3) assess its benefits and costs. This paper addresses three research questions: • How would typical IRP processes change if an LSE joined a regional RA program? • With a new regional RA program, which RA elements would remain local (i.e. within IRP) and which would become regional (i.e. within the RA program)? • How much control would LSEs and states retain over their utility resource mixes? How much influence would a regional RA program have over the resource mix?
This paper is most directly aimed at state regulators, public utility commission staff, and resource planners from states in the NWPP footprint that are pondering how their IRP guidelines and regulations may need to adjust to operate jointly with a regional RA program. The content of this paper may also help the NWPP RA program developer as it interacts with potential member states and utilities to understand what aspects of energy policy may be influenced by the program under development. More broadly, this paper should be useful for other RTOs/ ISOs whose utilities are required to conduct and file IRP processes and improve the connection between these processes and regional RA assessments.
The remainder of this paper is organized as follows. Section 2 summarizes fundamental RA principles and the standard elements of an RA assessment. In Section 3, we review a sample of 11 IRPs from the Western and Midwest U.S. and report the methods that utilities employ to assess and plan for RA. Section 4 provides a detailed description of the current design of a regional RA program proposed by the NWPP. In Section 5, we discuss the Southwest Power Pool's (SPP) experience to understand how its regional RA program interacts with the IRP processes of LSEs who are members of SPP. In Section 6, we identify and discuss how four specific IRP components identified in Section 3 would be impacted by a regional RA program. Section 7 concludes the report by answering the three research questions, summarizing the most important findings, and suggesting follow-up research activities.

Traditional RA fundamentals
RA is the ability of supply-side and demand-side resources to meet the aggregate electrical demand including losses (NERC, 2011). RA has become a prominent topic in both academia and industry due to its importance and the challenge of achieving it in a rapidly evolving electric sector. Traditional RA fundamentals include supply, demand, and RA metrics. On the demand side, electricity system planners forecast the approximate annual peak demand during the planning horizon, and determine how much capacity should be available in the system. On the supply side, RA is provided by the facilities that generate electricity and the transmission and distribution network that delivers power to customers. Furthermore, due to the uncertain availability of supply resources and variation in loads, electricity system operators also need to maintain reserves to ensure that demand can be met even when load is higher than expected or resources experience unplanned disruptions. Utilities and regional regulatory bodies use a variety of metrics to determine the level of RA that is sufficient and to track the actual status of RA on a power system. Assessing whether a system would actually achieve a desired reliability target is inherently a probabilistic problem, but RA targets are often expressed in terms of deterministic metrics which are more easily interpreted by utilities and monitored by regulators. The planning reserve margin (PRM) is the predominant deterministic metric that measures the percentage by which generation capacity exceeds the forecasted peak demand. By contrast, probabilistic models consider stochastic scenarios related to uncertainties such as loads, variable resource capacity factors, and unplanned outages to determine how reliable an electricity system is in terms of avoiding power disruptions, measured using probabilistic metrics such as those listed in Table 1.

Current RA assessments in the Western U.S
All LSEs need to conduct RA assessments, and these utility RA assessments may or may not fall under the umbrella of a regional RA program that coordinates RA planning among multiple utilities on a broader spatial scale. For individual regulated utilities subject to IRP requirements, RA assessments are typically part of-often implicitly-their IRP process. A thorough RA assessment is fundamental to IRP because it ensures that the resource portfolios considered by the utility for future investments are able to satisfy the necessary reliability standards. The role of IRP as an RA assessment platform is examined in detail in Section 3.
Compared to owned resources and long-term firm capacity contracts, LSEs generally rely less on market purchases to meet their capacity requirements, because they are rarely able to rigorously analyze future regional market conditions to determine whether sufficient capacity will be available to purchase. Relying on market purchases might seem like a reasonable strategy for a single utility's planning, but not when examining the region as a whole. Therefore, several regions of the U.S. electricity system conduct regional assessments to monitor the RA status across different balancing areas.
Table 2 summarizes several regional RA programs that are currently operating or have been proposed in the Western U.S., with their enforcement mechanisms. An energy-only market encourages investment by allowing electricity prices to become very high during peak demand periods, thereby rewarding generators who can contribute to RA. By contrast, a bilateral capacity sharing system allows LSEs to comply with regional RA program capacity requirements by contracting to procure generation capacity from generation owners. Specifically, NWPP is currently developing a proposal for a regional RA program to address potential capacity deficits, and it has hired E3 to conduct an RA assessment in the PNW (E3, 2019) and analyzed the challenges of ensuring RA. The proposed NWPP regional RA program is presented in Table 1 Common probabilistic metrics used to assess, measure, and monitor RA. Expected total quantity of unserved energy per year due to loss of load events detail in Section 4. In addition, we also include SPP's regional RA program as an interesting case study in Section 5, since many LSEs in its footprint are required to conduct IRP while also complying with SPP RA requirements.

The role that IRP plays in resource adequacy
We conduct a detailed review of a sample of 11 IRPs from LSEs across the Western and Midwest U.S. (Table 3) to investigate how LSEs currently assess RA within the context of IRP. These LSEs are geographically distributed across the Western U.S., vary in size, and differ according to whether they fall within the jurisdiction of a regional RA program or not. Some IRPs separately and explicitly convey their RA assessments (in which case they might use alternate terminology, such as "reliability analysis"). In other IRPs, RA is certainly assessed, but it is incorporated into other parts of the IRP (such as the construction of candidate resource portfolios) in a manner that often obscures the RA assessment. Table 4 lists the RA targets that the 11 IRPs aim to achieve over their planning horizons. The RA targets are most often specified in terms of a PRM, likely due to the simplicity of interpreting, calculating, and monitoring this metric. However, in several instances, the PRM target is itself the outcome of a more sophisticated analysis carried out to determine the PRM needed to achieve a desired maximum LOLP of one day of lost load every 10 years. We also find that KC-BPU and OG&E have lower PRMs, which were assigned to them by the SPP regional RA program. The regional RA program helps these SPP-based LSEs achieve the same or better reliability with lower PRMs by leveraging the diversity of all regional loads and resources in the SPP area.

RA targets
Comparing the PRM targets of LSEs that are not part of the same regional RA program is difficult because the exact meaning of each PRM depends on each LSE's definitions and assumptions. These can differ considerably, and in fact, standardizing assumptions and practices is an important role of a regional program. For example, some utilities may forecast their peak demands conservatively (i.e., project higher peaks), which would lead to relatively lower PRMs. Others may deal with load forecast uncertainty by specifying a higher PRM rather than building conservatism into the load forecast itself.

Net load forecast
Forecasting load is a crucial aspect of a utility's RA assessment. Table 5 summarizes the methodologies and key inputs that utilities use in load forecasting, as well as the annual peak load growths that they project. Typically, the load forecast is segmented by customer class with the residential load forecast often serving as the most important component. The residential load forecasts are usually expressed as the product of the projected number of customers and projected usage per customer, and they are usually forecasted separately by statistical models, regression models, or end-use models. By contrast, the forecasts for commercial and industrial loads are relatively easier to construct. In addition, utilities prefer to apply individual forecasts for large-scale or geographically specific commercial and industrial customers, which are usually established by customers themselves.
Demand-side programs, including energy efficiency programs and demand response programs, help LSEs reduce the amounts of energy and peak load they need to serve. All 11 utilities in our sample have ongoing demand-side programs, and more programs are proposed or in development to contribute to RA and other objectives. However, not every   utility reports its estimated reduction of peak demand, and very few LSEs report the reduction disaggregated by its specific demand-side programs. Some utilities partially report the methods that they use to estimate the peak load reductions achieved through these programs. For example, Avista translates the peak savings attributed to demand response programs into a peak credit that differs depending on their durations. Generally, demand-side programs are projected to reduce peak demand by 5-10 % in 2030, but corresponding impacts on reliability are rarely reported.

Modeling approaches
Once the load forecast is established, utilities can construct several resource portfolios that are deemed adequate according to their preferred RA metrics. All of the utilities apply capacity expansion models to develop and evaluate their preferred resource portfolios, and use deterministic or probabilistic methods to evaluate power system performance. Some utilities use proprietary capacity expansion models, while others use commercially available software (e.g., Aurora).
It is important to note that the reliability target is often determined prior to the construction of resource portfolios. In capacity expansion models restricted by PRM reliability targets, RA is enforced through a set of model constraints. As RA assessment tools, these deterministic constraints are limited in that they cannot capture probabilistic loss-of-load events. Moreover, since the constraints will usually be binding for the preferred least-cost portfolio, utilities will build just enough capacity to hit the reliability target. Therefore, whether resources are truly adequate depends on the accuracy of that target.
Compared to a PRM-based capacity expansion model, a loss-of-load probability model is a more advanced tool for assessing the RA properties of alternative resource portfolios. Instead of deterministic PRM constraints, loss-of-load probability models simulate many scenarios based on multiple (and potentially correlated) uncertainties to ensure that a portfolio achieves acceptable RA performance. A few utilities, such as Avista and PGE, apply probabilistic models to assess their system reliability. These models incorporate probability distributions for loads, resources, and stochastic outages, then validate RA through sequential simulations.

Capacity credit in resource portfolios
Capacity value refers to the ability of a power plant to reliably meet peak demand. It is usually measured by equivalent firm capacity or a fraction of nameplate capacity. A high penetration of renewable generation complicates a utility's RA assessment (Ibanez and Milligan, 2014;Tanabe et al., 2017), since wind and solar PV have lower capacity values than conventional resources due to the variations of available power across time and space, which cannot be perfectly forecasted. Utilities must develop credible methods to estimate the capacity values of renewable resources, especially their capacity values during peak hours (Munoz and Mills, 2015;Zhou et al., 2018).
All 11 utilities in our sample include renewables as major supply resources and report their contributions to energy and peak demand. The methods that they use to estimate capacity values are reported in Table 6. Most utilities apply an effective load carrying capability (ELCC) study to determine capacity values for renewables. ELCC is defined as the amount of incremental load a resource can reliably serve, considering probabilistic parameters of unserved load caused by forced outages, load uncertainty, and other factors (SPP, 2019a). For variable renewables, ELCC is more comprehensive than simple capacity values because the ELCC captures the correlations among the resource itself, other variable resources, and load, which makes it a popular metric for assessing capacity credits. Utilities' ELCC studies suggest that the marginal ELCCs for solar and wind resources decline as their total capacities grow, since the remaining capacity need is less aligned with the generation profile.

Market transactions
Utility-owned plants, which account for more than 80 % of current generation mixes, are the most prevalent resource type (by ownership status) in the existing portfolios of most LSEs. Utility decisions to "buy versus build" are often driven by reliability obligations (Carvallo et al.,  2020). However, several utilities have commissioned studies into the availability and economic benefits of market purchases, which typically include power supply, electricity price forecasts, power transfer ability, and other operating characteristics. There are other utilities, such as SCE and OG&E, that rely on thorough deliverability studies provided by regional RA programs. Their experience indicates that regional RA studies would help utilities ensure that their assumptions about the future availability of market purchases are compatible with each other, with planned capacity additions across the region, and with the transmission capabilities of the power system.

Transmission upgrades
Utilities must develop robust transmission systems to enable the delivery of power from generation assets to loads. All LSEs include regional transmission planning in their IRPs, but these transmission analyses vary considerably in their degree of detail. Some utilities, such as SMUD and PNM, perform detailed analyses of future transmission needs and typically find that existing transmission networks are not binding constraints on reliability in the present and immediate future. There are other utilities that rely on transmission impact analyses provided by their neighbors, or by the regional system operator (e.g., CAISO, SPP). For example, OG&E depends on the SPP Transmission Expansion Plan to assess its transmission capabilities and needs. The evaluation of transmission upgrades is becoming increasingly important because resources are becoming more geographically diverse and shared among utilities (Luburić et al., 2018).

Emerging technologies
With the ongoing evolution of electricity technologies and market structures, a variety of emerging technologies are being incorporated into utility RA assessments. These innovative elements can all provide RA benefits, but they introduce new challenges for utilities planning for RA.
Distributed generation (DG), typically distributed solar, contributes significantly to meeting the peak load as a demand-side resource for some utilities. Some LSEs project the expected capacity contribution explicitly in future years, and others include it in the set of demand-side programs or as a factor built into the load forecast. Normally, DG should yield significant benefits for RA by reducing the net load that the utility must satisfy and relieving stress on congested transmission and distribution systems (Al-Muhaini and Heydt, 2013). However, in designing the utility's resource portfolio to cost-effectively achieve RA, it is important to accurately project DG investments and their alignment with load and other generation profiles.
Electric vehicles (EVs) could play a significant role in future RA as their adoption increases. EVs could significantly increase energy demand, but their charging could be controlled to limit the effect of their energy demand on the peak load and to absorb excess renewable generation. Unfortunately, thoroughly accounting for EVs in an RA assessment is very difficult due to the lack of historical data and myriad uncertainties. Seven of the 11 LSEs mention the impacts of EVs in their IRPs and five of them model EVs as a distinct element. Their forecasts show that assumptions about EV adoption will have a significant influence on projected load obligations.
Energy storage could improve reliability considerably. Storage could provide peaking capacity to satisfy peak loads, mitigate problems associated with ramp rates, and absorb renewable generation to prevent curtailment (Denholm et al., 2020;Stenclik et al., 2018). In addition, well-sited storage can reduce the need for new transmission and distribution assets (Xu and Singh, 2012). Most utilities include energy storage as a future resource and study its economic benefits. Some utilities conduct an ELCC analysis on storage resources with different durations, especially when they are coupled with solar PV. For example, PGE calculates ELCC values for four types of storage and indicates that storage resources with longer durations have higher contributions to peak demand. However, similar to renewables, the ELCC value of storage declines significantly as more storage is deployed because storage does not generate energy. Utilities also state that, compared to electricity market prices and the cost of a CCGT plant, large-scale energy storage is not yet an economically competitive way to meet capacity needs.

Treatment of uncertainty
Reliability is inherently a probabilistic concept, and some utilities explicitly incorporate uncertainty into their RA assessments. While IRPs tend to account for several uncertainties, LSEs tend to focus more on economic uncertainties than RA uncertainties in their IRP analyses. Table 7 reports whether each IRP in our sample incorporates uncertainty into various elements of its RA assessment.
All 11 utilities conduct scenario or sensitivity analysis on the load forecast. Some utilities evaluate the performances of their portfolios under high and low load growth scenarios, while others construct alternative portfolios based on different load forecasts. However, only some of the utilities evaluate uncertainties in demand-side programs' contributions to peak load. Those uncertainties can be incorporated into load forecast scenarios, but they can also be modeled as supply resources.
On the supply side, since conventional resources still occupy large shares of generation mixes across Western U.S. states, utilities tend to consider different scenarios related to uncertainties on plant retirements, additions, or upgrades of thermal plants. In addition, LSEs are challenged by many uncertainties related to renewable energy and storage. However, beyond the economic uncertainties, only a few LSEs investigate the risks of increasing renewable penetration and its relationship with reliability. About one-third of the utilities include sensitivities on the efficiency of storage, and they show that storage has great potential to reduce capacity needs when its cost falls or efficiency improves, especially under high additions of renewable generation (Denholm et al., 2020;Xu and Singh, 2012).
Market risks are also treated as major uncertainties in utilities' IRPs, but only half of the utilities include sensitivity analysis on the relationship between the capacity deficit and the level of market imports. For example, PNM emphasizes the risk due to underlying market assistance and implements a reserve margin sensitivity analysis by capping the market assistance at different levels. Its simulation indicates that PNM's current 13 % PRM target is not sufficient to meet a 0.2 days/year LOLE requirement because expected market imports are not guaranteed to be available.

The northwest power pool RA program
The Northwest Power Pool (NWPP) was originally formed in 1941 to promote coordination among its members. Its footprint includes the Pacific Northwest U.S., the Canadian provinces of Alberta and British Columbia, and other stakeholders that extend beyond the PNW including the Balancing Area of Northern California, NV Energy, Pacif-iCorp, Excel Energy, and Western Area Power Administration. The NWPP Resource Adequacy Program is an emerging proposal to establish a regional RA process with binding commitments for the participating members of the NWPP. The NWPP RA program seeks to maintain reliability, increase transparency of the region's RA position, and efficiently utilize reserves and resources. This section introduces the main design elements of the NWPP RA program and is based on publicly accessible information through August 2020.
In recent years, there have been concerns in the Northwest that the combination of increasing coal retirements, growing use of renewable generation, and greater reliance on market transactions to meet capacity targets are leading to an RA problem. A series of forward-looking studies over the past several years indicate that the Northwest's collective capacity margins are declining, thereby increasing the risk of electricity shortfalls (BPA, 2019;E3, 2019;NWPCC, 2018). These studies show that the Northwest region could begin to face capacity shortages by the early 2020s, followed by severe capacity shortages in the mid-2020s.
The NWPP RA program is moving forward in two predefined phases. Phase 1 work groups reviewed existing regional studies on RA and best practices, developing a report that summarized the findings and proposed a regional approach to addressing RA concerns (NWPP, 2019). Phase 2A began in Fall 2019 with the objective of developing a preliminary program design for the NWPP RA program. The project Steering Committee took the first step by developing a list of design objectives for the NWPP RA program (Table 8).
Phase 2A work groups were formed to specify the key design elements that were consistent with the design objectives. A Stakeholder Advisory Committee was organized to provide periodic feedback on the proposed design elements for the NWPP RA program. Additionally, the program held public webinars to provide information and collect feedback on proposed design elements. By July 2020, the NWPP completed its Phase 2A preliminary program design. The following section summarizes the framework and key elements of the NWPP preliminary RA program design based upon the public documents and presentations released during Phase 2A.

Forward showing program
The foundation of the NWPP RA program is the creation of a forward showing program. The general principle of a forward showing program is that each participating LSE would be responsible to show that it has adequate capacity resources to meet its expected peak load plus a designated capacity margin. The total capacity across participating members will match or exceed the expected regional peak load plus the desired capacity margin as long as all members demonstrate that they have capacity to meet their own expected peak loads. The challenge for the NWPP, or any entity developing a forward showing program, is to clearly define key parameters, establish mechanisms to promote compliance, and ensure that there is transparent and verifiable enforcement.
The NWPP's preliminary design proposal calls for a forward showing program with two binding seasonal periods corresponding to a summer peak load and a winter peak load. The NWPP RA program proposal defines the winter season to start on November 1 and end on March 30. The summer season would begin on June 1 and end on September 30. Member entities would be required to show that they have sufficient resources to meet their binding seasonal targets seven months prior to the start of the winter and summer seasons. The NWPP RA program proposes a two-month cure period if members fail to meet either the binding winter or summer deadlines. Member entities that fall short of their showing capacity target would have the option to add new capacity or obtain the equivalent amount of capacity through bilateral market transactions. Entities that fail to address capacity shortfalls would be subject to binding financial penalties The NWPP RA program proposes a set of load forecast parameters to derive peak load forecasts in the winter and summer for each LSE and collectively for the region. Peak loads are based on the expected peak load in a season with a 50/50 or one-in-two probability 1 .
Accurate accounting of capacity resources is an important foundation for performing an RA assessment. The NWPP RA program proposes to require a registration and certification process for all resources. The program proposes a set of supply-side parameters to measure capacity contributions of different types of resources. For thermal generators, the NWPP RA program proposes to use the UCAP methodology, which is a relatively comprehensive approach that includes resource-specific outage data. Variable energy resources such as wind, solar, and runof-river hydropower would be assigned capacity values based on an ELCC methodology. Identifying the capacity contribution from large storage hydro in the Northwest is a challenging modeling exercise. The NWPP plans to study in more detail the capacity contribution of storage hydropower in addition to other resources including (but not limited to) demand response, batteries, and pumped storage.

Operational program
The NWPP RA program is designed to be a cooperative agreement among utilities in a region to share capacity resources to meet regional RA targets. Other regions that have developed RA programs have generally been built upon formal wholesale markets (e.g., SPP, CAISO). There is no formal wholesale market in the NWPP footprint and the NWPP RA program does not intend to establish a centralized market  Ensure that Balancing Authorities and Load Serving Entities can continue to operate safely, efficiently, and reliably. Ensure that the recommended RA program and its components deliver investment savings through diversity benefits. Ensure RA program respects local autonomy over investment decisions and operations and continues to respect the rights and characteristics of individual utilities, transmission service providers, BAs, and other entities through program design. Make recommendations that are acceptable within the current and evolving regulations and requirements of each applicable federal, state, and local jurisdiction. Ensure that the participation, evaluation, and qualification of resources is technology neutral. Ensure that all products and services transacted to meet the requirements of the RA program are well defined, voluntarily transacted through existing competitive market frameworks, and accurately tracked. Ensure that the proposed RA program can be extended to other regions in the West. Ensure that entities that voluntarily choose to participate in the RA program equitably pay and receive benefits for services provided by the program. Ensure the RA program provides efficient long-term investment signals as well as a process for exit and entry of resources.
operator. Rather, the operational program would impose constraints on transactions between members and non-members for capacity resources. Participants may be able to rely on imports to meet their capacity needs during hours of a binding season. These participants would need to demonstrate a regional load credit by showing an expected and reliable import level. CAISO has performed historical data analysis of RA contracts and transfers that provides a basis for calculating expected future imports. By contrast, expected future exports by NWPP members are assumed to represent surpluses that do not impact load or the PRM within the NWPP footprint.

Implementation
In July 2020, the NWPP RA program completed its Phase 2A work of developing a preliminary program design. NWPP recently announced that it will hire SPP as a project developer to help manage the startup and implementation of the program (NWPP, 2020). Additionally, the NWPP intends to select a Program Administrator that would be responsible for overseeing program operations.
The NWPP RA program is designed to be a voluntary contractual organization. Utilities in the footprint have the discretion to voluntarily join it or not. Once a utility formally joins the program, however, it will be contractually committed to its privileges and commitments. The NWPP plans to gradually implement the requirements of the RA program over three stages. Stage 1 of implementation would begin with holding two non-binding forward showings covering a winter and summer season. Stage 2 of implementation would proceed with two binding forward showings that include a summer and a winter showing. Unlike Stage 1, the Program Administrator would impose penalties on participants that do not comply with requirements. Stage 3 of implementation would include two binding forward showings (winter and summer season) and the use of a full operational program.
The architects of the NWPP RA program are attempting to build a program that complements the unique history and culture of the Pacific Northwest and the Western Interconnection. The Northwest's electric sector institutions such as the Bonneville Power Administration, public utility districts, and municipal entities have historically been very independent and skeptical of organized electric wholesale markets.
The establishment of the NWPP RA program should not significantly alter or modify how its members interact with other institutions and practices in the Western Interconnection. The Northwest Power and Conservation Council (NWPCC) performs regional RA analyses of the PNW as part of its role to develop regional power plans. The work of the NWPCC will be informative and likely complementary to the goals and tasks of the NWPP RA program. The Western Electricity Coordinating Council (WECC) serves as the reliability entity in the Western Interconnection that enforces reliability standards and performs reliability assessments. WECC's regional RA analysis and the NWPP RA program could become mutually complementary by advancing the art of forecasting and analyzing RA in the NWPP footprint and across the whole Western Interconnection.
The Federal Energy Regulatory Commission (FERC) regulates sales of electricity in interstate commerce. Some utilities in the NWPP footprint are subject to FERC jurisdiction, but others are not. One of the NWPP working groups examined whether a fully functioning NWPP RA program would be subject to FERC jurisdiction. This work group observed that FERC jurisdiction can be triggered under the Federal Power Act by an agreement that affects the rates, terms, and conditions of sales of electric energy for resale in interstate commerce or transmission of electric energy in interstate commerce. Since the NWPP RA program would impose binding commitments and financial penalties on participating members, the work group concluded that FERC would likely have jurisdiction over components of the program. FERC jurisdiction would likely apply to the Program Administrator and the future governance structure of the NWPP RA program will have to meet FERC's independence criteria.

The southwest power pool experience
This section describes the active regional RA program in the SPP, which is an interesting case study because its service territory includes several states that have had IRP regulations for more than a decade. It follows that the SPP experience of running an RA program in states with IRP can provide an understanding of how the two processes have interacted over time. This section is based on semi-structured interviews with SPP staff and public utility commission staff conducted in July 2020. The interviews are complemented by publicly available SPP documents as well as IRP reports.
SPP is an RTO that serves all or parts of 14 states in the Southern, Midwest, and Western U.S. (see Fig. 1). The SPP operational territory encompasses 575,000 square miles and includes more than 61,000 miles of high-voltage transmission lines, over 750 generation resources, more than 4800 transmission substations, and serves 18 million people (SPP, 2019b).
As a transmission provider, SPP has the duty to serve its balancing area peak demand. Its RA program ensures that LSEs have enough capacity available for SPP to serve peak demand and enough reserves to maintain a predefined PRM. RA in SPP is regulated by Attachment AA to the Open Access Transmission Tariff, developed in early 2018.

Resource adequacy in SPP
The SPP RA program distinguishes LSEs, market participants, and generator owners. Market participants are responsible for ensuring that the LSEs they represent comply with their RA requirements. An LSE can apply to be a market participant or be represented by one. One advantage of dividing the roles of LSE and market participant is that the SPP rules allow for a single market participant to represent more than one LSE, given that certain firm power transaction conditions hold. In this case, the market participant is responsible for complying with an aggregate RA requirement based on the load obligations of the represented LSEs.
LSE RA requirements in SPP are calculated by augmenting the summer or winter season net peak demand by the target 2 PRM. The target PRM is the result of a loss-of-load expectation study performed by SPP at least every two years that employs criteria and assumptions agreed upon by SPP members. In this particular case, the LOLE is based on the typical criterion of a 1-day-in-10-years expected outage time. The SPP target PRM had historically been 13.6 %, but in 2016, the SPP Board approved a reduction to 12 %. This change was enabled by a significant transmission buildout as well as load and generation diversity (SPP, 2016). Market participants that are responsible for complying with RA requirements must do so by annually reporting the amounts of deliverable and firm capacity available to them to meet their winter and summer net peak demand obligations.
The peak demand obligations are based on load forecasts that are developed by each LSE, which are typically based on their resource planning reports. SPP does not impose any methodological constraints on the forecasts, but does require that they be 50/50 (one-in-two) forecasts. SPP does not develop its own forecast for each LSE, but for LOLE study purposes aggregates the peak demand forecasts assuming certain levels of load diversity to account for their non-coincident nature. SPP conducts a post-season validation where the load forecast is trued up with the actual outcome; the results of this exercise are analyzed by the Supply Adequacy Working Group to identify potential issues associated with over-or under-forecasting.
SPP market participants can meet their RA requirements with deliverable and firm capacity resources. SPP conducts an annual deliverability study for the summer season to determine how much capacity a given resource can deliver within the SPP balancing area. An LSE can meet its RA requirement with owned resources as well as supply-and demand-side contracted resources. The LSE submits a standardized "workbook" with detailed information on the deliverability and firmness of each owned and contracted resource that will be used to meet RA requirements for the next season.
An RA status report released in June documents how each LSE is complying with its RA requirement, as well as an aggregate RA outlook for the SPP balancing area. The report expands on each LSE's deliverable supply-and demand-side resources and compares them to the peak demand minus demand-side resources, including DR and DG, for that LSE. The latest RA report shows that the SPP balancing area has a ~21 % reserve margin for the 2020 summer season, which decreases to 12.5 % by 2025.
Over time, SPP has produced information and created opportunities to help LSEs meet their RA obligations. For example, interviewees indicated that its deliverability study was designed to help LSEs understand the exact capacity credits of their resources in advance. In addition, SPP made the use of bilateral contracts more flexible by not requiring them to be backed up by firm transmission service agreements to be eligible to meet RA requirements.
While SPP is responsible for the economic operation of the integrated system, it does not have authority over the financial investment decisions of market participants. This distinction allows for states to retain the prerogative to implement policies that can be reflected in their regulated entities' investment decisions. Below, we evaluate the interactions between states' planning mandatescodified in their IRP guidelinesand the SPP planning and operation criteria.

Resource planning in SPP
Earlier, we highlighted how IRPs include RA assessments, and how IRP is, in part, an RA compliance mechanism for state regulators with respect to their regulated utilities. Ten out of the 14 states with SPP member utilities had IRP regulations implemented as of 2020, and nine of them have had these regulations in place for more than a decade.
Here, we examine the relationship between the IRP regulations mandated by states within the SPP footprint and the SPP RA program. The objective of this comparison is to understand the overlaps and boundaries between IRP and the SPP RA program, and where and how their assumptions, methods, and outcomes are shared or not. We employ two methods. First, we build on the IRP analysis in Section 3 and examine in more detail the latest IRP reports from selected LSEs in the SPP footprint. Second, we expand this information through unstructured interviews with public utility commission staff and utility resource planners. Table 9 reports the entities interviewed and/or whose resource plans were examined for this section.
In SPP, resource planning guidelines reflect the relationship between the regulated entities and the RTO. In states with thorough regulatory guidelines, the relationship with SPP is defined in the statutes themselves. This is the case in Missouri, for example, where the IRP guidelines allow the utility to include RTO transmission planning outcomes provided they yield economic benefits for Missouri ratepayers (i.e., these are not solely reliability improvement projects) (MO CSR, 2011, p. 11). In states with broader IRP regulations (e.g., Oklahoma), the relationship between IRP and SPP RA guidelines is mediated by an LSE's SPP membership, which includes duties related to RA. In cases where it is not specified how an LSE should implement an IRP guideline, the LSE has the incentive to follow SPP guidelines to avoid penalties. For example, OG&E's 2018 IRP report states explicitly that "the objective of this IRP is to explore options to maintain OG&E's generation capability in accordance with the SPP planning reserve margin requirement of 12 % in a  manner that achieves the lowest reasonable costs to customers, improves reliability and maintains environmental balance" (OGE, 2018). Upon the recognition of SPP as an RTO, FERC determined that states within the SPP footprint would retain rights over cost allocation, financial transmission rights, planning for remote resources, and RA. In its FERC filing to review its RA policy, SPP stated that, "As the Balancing Authority, SPP is the entity responsible to, amongst other things, integrate the resource plans of the Resource Planners within its region" (SPP, 2018b, p. 2). Then, the IRP components reflect the way SPP and the state guidelines relate. The IRP reports from LSEs in the SPP footprint rely extensively on SPP's transmission planning to frame their RA analyses regardless of the thoroughness of the IRP rules. For example, the OG&E 2018 IRP report states, "OG&E provides input to the SPP planning process, and SPP is ultimately responsible for the planning of the OG&E system" (OGE, 2018). In another case, OPPD (Nebraska) explicitly mentions employing the capacity accreditation factors for wind determined according to SPP's RA guidelines.

Conclusions and policy implications
In this concluding section, we discuss the policy implications of a regional RA program for the various components of its member utilities' IRP processes. The insights in this section should help states identify components of their IRP regulations that might conflict with a regional RA program that their regulated utilities could join, and drive deeper analyses of how the two could be better aligned.
Focusing on the proposed NWPP RA program as a current example, Table 10 summarizes our assessment of how much each IRP component would be affected by the establishment of the program. In addition, the last column indicates the division of authority over each IRP component under the proposed RA program. A "Local" classification describes IRP components that will remain under state authority and discretion of state regulators through their IRP regulations. A "Regional" classification indicates that the regional RA program will assume effective control over a responsibility that has traditionally been part of IRP. Finally, the "Shared" classification reflects a compromise whereby the states retain control over certain aspects of the IRP component while the regional RA program has authority over other aspects of it.
Based on our research, two components of IRP would be significantly affected by a regional RA program: RA reliability targets and capacity accreditation assumptions. Two components of IRP may be moderately impacted: load forecasts and transmission upgrades. The remainder of this section focuses on these four IRP components which stand to be affected most heavily by a regional RA program. Our discussion highlights potential conflicts that will need to be addressed through careful design of the regional RA program and/or changes in IRP policy that ensure proper alignment.

Resource capacity credit
Determining the capacity position in IRP depends on two key assumptions: the capacity accreditation for existing and new resources (see Section 3.3.2), and the target reliability metric (see Section 3.1). Resource capacity accreditation will require much more alignment between IRP and the NWPP RA program. A key potential issue is that the capacity contributions of resources depend on their regional penetration levels and the regional peak demand, neither of which can be determined within a single IRP (Mills and Wiser, 2012a). In addition, states have historically assigned different capacity credit factors for similar resources (especially wind, solar, and demand response), which may create friction among the members if some states recognize much lower capacity than others for similar resources. Finally, if IRP and regional RA capacity accreditation for the same resource differ, there is a risk that an LSE would be adequate at the local level but not at the regional level, and would have to justify additional investment outside its IRP recommendations to comply with regional RA requirements.
There are at least four types of resources that will require specific attention in capacity credit calculations: variable renewable resources, demand-side resources, hydropower, and contracts.
Standardizing capacity credits for variable renewables is relevant because the capacity contributions of wind and solar decrease as their penetration increases (Mills and Wiser, 2012b). It follows that a regional capacity accreditation that considers variable resources pooled across the region would most likely produce a lower capacity credit than a local IRP assessment. Therefore, if states grant capacity credits that are different from what the RA program recognizes, an LSE may have a capacity deficit with the RA program, but meet the state's capacity requirement even when subject to the same target PRM.
It is important to note that renewable resource capacity credit methodologies, policies, and outcomes in IRP differ substantially across LSEs and states (Mills and Wiser, 2012a). States may need to surrender some control over the capacity credits of their solar and wind resources to make them consistent across the NWPP footprint, probably relying on a regional calculation that also accounts for the aforementioned joint impacts of aggregate penetration levels. Across SPP, for example, states reach consensus on capacity accreditation for renewable resources and their LSEs' IRPs incorporate these assumptions into their analyses.
Standardizing capacity credits for demand response, energy efficiency, and DG follows a similar logic as with utility-scale, variable renewable resources. The main issue is that there is substantial variation in the treatment of DR and EE capacity credit in IRP across states. Similarly, there is also substantial variation in the treatment of DG across IRPs (Mills et al., 2016). There may need to be a common agreement on how to classify and treat DR, EE, and DG resources for RA assessments to allow LSEs to show these forms of capacity in their IRPs as well as in a regional program. Furthermore, it is likely that the capacity contributions of these demand-side resources would also decline with higher penetration of similar DR, EE, and DG measures across regions, something that is not currently calculated as part of IRP.
The RA contribution of hydropower has its own complexities, as has been recognized by the NWPP. Hydropower is generally regarded as an energy-constrained resource that can supply most of its nominal capacity as firm. However, in extreme drought events, even this capacity may not be available. From an IRP perspective, this resource has similar challenges as variable renewable resources in that there will need to be a common methodological practice across hydropower-owning LSEs to determine the capacity contributions of these resources.
Finally, the capacity contributions of bilateral and market transactions in IRP also vary substantially across states (Carvallo et al., 2020). This feature may make contracts the most challenging resource to homogenize in a regional RA program. It is possible that the treatment of capacity exchanges within pooled resources without an explicit market will require some sort of centralized system to standardize, aggregate, and verify the capacity contributions of contracts. Contract Table 10 IRP components and level of impact from a regional RA program.

IRP Elements
Report Section Impact Overall Low Local deliverability has not been treated systematically in IRP rules, but it will be a critical component of a regional program. Section 6.3 examines the issues arising from the treatment of transmission upgrades.

RA targets
As indicated in Section 2, the target reliability metric serves two purposes: to set a common regional RA target and to track the status of RA in the region. This value should be the same or higher for an individual LSE compared to that of the RA program. If the state PRM is higher, that would just cause the LSE to be super-adequate compared to the regional requirement; regulators can choose this if they want to forgo economic benefits of pooled resources to maintain higher local RA. Our interviews revealed that this practice is relatively common among risk-averse regulators across the SPP footprint. It would be infeasible for an LSE member of a regional RA program to only meet the lower PRM between IRP regulations and the regional program. Hence, regulators may need to update their IRP guidelines to assure that the regulated LSE will use at least the same PRM as in the regional RA program.
The experience of SPP suggests that states agreed on a target PRM based on LOLE studies developed by SPP. These discussions took place at the Regional State Committee over several years, and the PRM was approved and adopted by states and reflected in their IRP rules once consensus on its benefits was reached. The main benefit of a consensus minimum PRM target across members of a regional RA program is that there are no potential conflicts over LSEs cross-subsidizing the RA of other LSEs in the pool. When a minimum margin is met, all members know they are contributing their fair shares to the region's RA, even if some LSEs or states decide to be super-adequate.

Transmission upgrades
A key role of an RTO is to conduct transmission planning studies to inform development in the region. The ways that these studies lead to actual investment vary, but LSEs do rely substantially on these plans to inform their supply-and demand-side investment decisions. As found in Section 5, IRPs from LSEs in the SPP footprint make extensive reference to the RTO's planning studies to ensure that the resource mixes proposed in their IRP reports are technically feasible.
In contrast, IRP processes in Western states have generally been overly focused on generation technology choices, and transmission expansion studies tend to be implicit and not clearly reported in all IRPs, as indicated in Section 3.4. From an IRP perspective, the main issue is how to ensure that the transmission expansion assumptions built into each IRP are consistent with the assumptions made at the regional level. A recommendation would be to develop a process to collect transmission expansion assumptions from each NWPP member, based on what its IRP is assuming. NWPP or regional entities under FERC Order 1000, including CAISO and NTTG, would then collect these transmission expansion assumptions and determine the reliability levels that they bring to the footprint. This is essentially the opposite of what most RTOs do, but this process could help establish a common understanding of transmission expansion across a large footprint. A complementary process would rely on transmission planning studies already developed by the WECC and PNWCC, but it is unclear how these outcomes could be incorporated into IRP.
An open question, which extends beyond IRP, is how much control over transmission expansion states would need to give up to make a pooled capacity resource mechanism work efficiently across the NWPP footprint. The need for common assumptions for both local and regional transmission expansion may conflict with a state's desire to retain full control over the timing and choice of transmission projects across the NWPP footprint.

Load forecast
The SPP experience shows that load forecasting can be left to the member entities in the regional program provided that they develop and share forecasts with standardized statistical characteristics. The main issue of using forecasts with different statistical properties is that they reflect different assumptions about risk. For example, if a regional program is aiming to manage a certain load forecast uncertainty level, then an LSE whose forecast reflects less risk aversion would be benefiting from the efforts of other LSEs that are aiming to hedge more against load forecast uncertainty. It follows that specifying the statistical properties of the forecast should be consistent with regional agreements on risk assessment and management. According to Section 2, the statistical properties of existing IRP forecasts by Western U.S. LSEs differ. LSEs may not need to petition to change their IRP rules if they are willing to produce a specific and separate forecast for the NWPP RA assessment. However, it would be more efficient if at least one of the existing forecasts created for the IRP were directly applicable to the regional RA assessment.
SPP does not attempt to calculate the coincident peak demand among its member LSEs for RA purposes, but aggregates the individual non-coincident forecasts to produce a regional estimate 3 . This approach may produce a slightly higher regional peak demand, but it ensures that each member LSE maintains local RA by requiring it to meet its own peak demand. The obvious drawback of a simple aggregation of peak demands is that it does not account for the temporal diversity in load profiles. A regional coincident peak demand would probably be lower than the sum of all individual peak demands. It follows that savings accruing from using a coincident peak demand approach could be substantial if there is enough temporal diversity, which could be the case for the PNW. The data for coincident peak demand calculations would most likely not come from IRPs, but would rather be requested from LSEs via a separate process managed by the NWPP RA Program Administrator.
Ultimately, interviewees from public utility commission staff from SPP states indicated that LSEs have an incentive to develop IRP assumptions that are consistent with SPP's in order to fulfill their membership duties. IRP guidelines in these states are generally much broader and more flexible than the IRP rules in Western U.S. states. This flexibility makes it easier for LSEs to adapt their IRP analyses to align with SPP requirements. LSEs should be able to develop NWPP-aligned forecasts as part of their IRP processes and benefit from the public stakeholder engagement as long as IRP regulations in the NWPP states are based on a broad and flexible set of principles.

Funding sources
The work described in this study was funded by the Energy Resilience Division of the U.S. Department of Energy Office of Electricity under Lawrence Berkeley National Laboratory Contract No. DE-AC02-05CH11231.

Declaration of Competing Interest
The authors report no declarations of interest.