Money for Nothing? Why FERC Order 745 Should have Died

Customer baseline load (CBL) measurement is designed to represent participants’ expected usage in a number of electricity demand response (DR) programs. Our empirical results, however, show that CBLs can be systematically higher than DR participants’ estimated load, especially for those experienced in DR activities, likely due to manipulation behaviors. Thus, the integrity of CBL may degrade over time. With an inflated CBL, the impact of DR programs may therefore be highly exaggerated, and consumers can be paid money when they are not actually reducing their demand. In particular, we design a manipulation-indicating variable “seemingly unattractive free-money opportunity” (SUFO) and discover systemwide manipulative behaviors that increase with time and are widely adopted by experienced DR participants. We suggest that policy makers in FERC, RTOs, and states regulatory agencies consider the threat of manipulation when modifying DR market rules following the Supreme Court’s recent upholding of FERC Order 745.


INTRODUCTION
Increasing the responsiveness of consumers to price to create a more efficient and reliable system is an important issue in electricity energy supply markets.By exposing consumers to realtime prices, Demand Response (DR) can reduce peak demand and enhance system reliability.FERC Order 745 (FERC 2011b), which required RTOs to compensate DR with locational marginal prices (LMPs), was vacated by U.S. Court of Appeals for the District of Columbia (USCA Case #11-1486, 2014) on the grounds of both that FERC exceeding its jurisdiction and that the DR pricing formula was "arbitrary and capricious."The court order was widely regarded as the end of traditional DR in the wholesale market.After FERC's appeal, the Supreme Court in January 2016 overturned the lower court opinion and ruled that FERC has the authority to regulate DR.FERC, regional transmission organizations (RTOs) and state governments now have the opportunity to implement and to modify DR programs.In DR programs, demand reduction is measured by comparing a customer's actual load with an administratively determined customer baseline load (CBL).The CBL based DR system requires constant administrative interactions from FERC and RTOs.For example, a recent FERC Order directs PJM to increase the granularity of capacity DR performance monitoring (FERC 2014).Though with all the efforts from FERC and RTOs, DR participants may be able to inflate their CBLs and thus profit by creating artificial load reductions.Obtaining a

BACKGROUND
FERC (2011a) defines DR as "changes in electric use by demand-side resources from their normal consumption patterns in response to changes in the price of electricity, or to incentive payments designed to induce lower electricity use at times of high wholesale market prices or when system reliability is jeopardized."We focus here on DR in energy, as opposed to capacity, markets, the subject of FERC Order 745.Several recent articles discuss the peak load reduction effect of DR (for example, Faruqui and George (2005), and Faruqui, Hledik et al. (2007)) and the DR compensation method (Bushnell, Hobbs et al. 2009, Walawalkar, Fernands et al. 2010).Few papers, however, examine DR manipulation theories and CBL-inflating strategies (Chao 2011, Chao andDePillis 2013), while several documents describe DR manipulation cases (FERC 2012a, FERC 2012b).No previous research has examined whether inflated CBLs have occurred widely in RTOs.
FERC Order 745 (FERC 2011b) requires all RTOs to compensate demand response resources with locational marginal price (referred to as the "full LMP payment"), regardless of CBL measurement methods or participants' retail contracts.Over the last several years, the appropriate payment for DR resources has been a topic of much controversy (Hogan 2010, Kahn 2010, Walawalkar, Fernands et al. 2010, Chao 2011, Chao and DePillis 2013).
In PJM's energy market, DR resources and generators submit supply offers (or bids, i.e., willingness to supply a certain amount of energy with a certain level of compensation), and PJM dispatches generators and DR resources in economic order (lowest cost first) to meet system demands.Before FERC Order 745's implementation in PJM in April 2012, DR resources in PJM were compensated by locational marginal price (LMP) minus the generation (G) and transmission(T) parts of the retail tariff (referred to as LMP-G-T payment) in energy-market economic dispatches (PJM 2011).After April 2012, PJM paid LMP, i.e., an increase of generation and transmission fee from the original LMP-G-T payment, for demand response resources in energy market.Following the FERC directive, PJM calculated a firm's CBL based on its historical usage.CBL for a weekday is determined as the average of the four highest usages of the five most-recent non-event (or non-dispatch) 1 weekdays (in the same hour interval) in the previous 45 calendar days (PJM 2011).Other RTOs also have similar historically determined CBLs.
The historically-based CBL determination method may incentivize potential manipulation strategies, which would lead to a "free-money" problem.Chao (Chao 2011) described moral hazards (over-consumption to increase CBL), adverse selection (consumers anticipating long term declining electricity demand being more likely to enroll in DR program) and behind the meter switching (switching usage between two energy sources to generate fake reduction measured from one source) as three potential free-money problems.Chao discusses DR payments and CBL construction, while reaching the topic of eliminating CBL manipulation through proper market rules.The article does not, however, seek to provide empirical evidence for existence of manipulation and little such evidence is provided.Here we attempt to fill this gap.
In addition to the manipulation strategies discussed above, we suggest an "idiosyncraticdemand bidding strategy" may also result in free money to DR providers.In idiosyncratic-demand bidding, a DR participant's bidding behavior depends on its normal usage schedule instead of the price signal, i.e., the participant uses high consumer-specific usage days as CBL determination days and supplies DR resources on low usage days.Idiosyncratic-demand bidding is thus a CBL-inflating strategy and a market manipulation behavior, since it does not match FERC and RTOs' definition of DR: "reduction from normal usage in respond to price signals." For example, assume a ship factory that produces steel every Monday and Tuesday, consuming 100 MWh per hour.The factory assembles a ship every Wednesday to Friday, consuming 60 MWh per hour.With idiosyncratic-demand bidding, the factory may submit bids for 40MW of DR resources at a low price every Wednesday to Friday, and leave Monday and Tuesday as CBLdetermination days.The factory is thus dispatched by the RTO from Wednesday to Friday and has a CBL 40 MW higher than its expected usage.Thus, without reducing usage, the DR participant has a consumption level below the CBL and, as a result, gains DR revenue.The participant is thus paid for an artificial reduction-one that does not actually take place.
In the above idiosyncratic-demand bidding example, the factory clearly violates PJM rules and FERC Orders by claiming a regular consumption pattern as a DR activity.However, if the consumption pattern in DR days changes in a smaller scale from the regular one (for example, a several percent of usage change due to the weather,) it may be difficult to determine whether the DR participant intends to manipulate the market by idiosyncratic-demand bidding.This "free money" that is taken by DR providers who are able to inflate their CBLs is paid by Load Serving Entities and eventually by other rate payers in the RTO.
The New England ISO (ISO NE) has uncovered evidence of idiosyncratic-demand bidding (ISO-NE 2008) in response to its rules on calculating CBL.ISO NE calculated CBL as the average usage of the previous ten non-event days and did not have a limited historical window for CBLdetermination days (for example, 45 calendar days as in PJM) in 2007.DR participants in ISO NE could submit bids with a low price on most days and leave several high-usage days in the summer as CBL-determination days.Participants thus created a high CBL that was the average usage of several high-usage summer days and remained almost constant across the year.Further, some DR participants, who had operated on-site generators on a regular basis before participating in DR programs, were found reducing output from their generators during CBL-determination days to achieve a high CBL.FERC has announced an investigation of the above CBL-manipulation events (see, for example, FERC (2012a) and FERC (2012b)), and issued penalties for the fraudulent, or manipulative behaviors (see for example, FERC (2013a) and FERC (2013b)).
In EPSA v. FERC 753 F. 3d 216, 225 (2014), the Appeals Court for the District of Columbia struck down Order 745 for two reasons.First, the appeals court concluded that FERC did not have jurisdiction to impose the order.FERC jurisdiction is limited to wholesale markets, and the court viewed DR implement under the order as affecting retail markets.Second, the Court, following the criticism noted above, viewed the LMP payment requirement as "arbitrary and capricious".On appeal, the Supreme Court overturned the Appeals Court in a 6-2 decision (FERC v. EPSA, slip. op. 14-840, January 25, 2016).The Supreme Court decision leaves the door open to further rules by FERC, RTOs, and states.We seek to contribute to the debate on these new rules.

THEORETICAL APPROACH
We consider two manipulative or CBL-inflating strategies: over-consumption and idiosyncratic-demand bidding.Figure 1 presents the decision facing a DR participant with a fixed retail rate.The marginal revenue curve shows the revenue of consuming an additional unit of energy.The marginal cost (MC) curve shows the marginal cost of consuming energy, including the firm's retail rate and DR payment.We consider the following scenarios: 1) When there is no DR, a firm will consume energy at point O, the intersection of the firm's energy demand curve (Marginal Revenue curve) and market energy supply curve (the Retail Rate, or RR.)The firm gains profit equal to area 4. 2) If the firm is dispatched by the RTO to provide DR resources and its CBL correctly predicts future usage, it faces a marginal cost curve as the route CBOA.The marginal cost for consuming more than CBL is still RR.However, the MC for consuming below the CBL becomes the DR payment plus RR.Point E, the intersection of MC and Marginal Revenue, becomes the new equilibrium.With DR payment, the firm thus receives the additional profit represented by area 1.Note that because the firm benefits from the use of electricity, the marginal revenue of consuming power is greater than zero.3) If a dispatched firm has a CBL higher than the non-DR usage, i.e., the participant has lower demand than expected, the MC curve faced by the firm is route CB 1 O 1 A. The firm remains at point E and gains more profit (area 2) than in condition 2. 4) When the wholesale market has a higher LMP, route E'B 2 OA represents the firm's MC, and point E' will be the new equilibrium.Electricity consumption declines and the participant gains more profit (area 3) than it would with a lower LMP.
Given this, a day with a high LMP and an inflated CBL we deem an "attractive free-money opportunity;" while a day with low LMP and an inflated CBL we call a "seemingly unattractive free-money opportunity" (SUFO).A SUFO can be created by a participant's idiosyncratic-demand bidding when the system load of the RTO drops due to, for example, a large change in temperature.
Figure 2 shows an example of a SUFO, with the participant's usage over five days (days 1-5) shown, along with system LMP and the number of cooling degree days.In Figure 2, the number of cooling degree days declines on day 5, so that both the expected usage of DR participants and system load of the RTO decrease.With a lower system load, the RTO generally will have a lower LMP.DR participants thus face a low LMP and low expected use on day 5.If a participant can generate a CBL higher than his expected load on day 5, its apparent curtailment effort on that day may be overstated, and its payments therefore inflated.
Price-responding DR providers make bidding decisions based on LMP.Compared to submitting bids on low-price day 5, participants without manipulation intent may prefer bidding on high-price days 1-4, in response to the high LMP.Days 1-4 thus become DR event days and are excluded in future CBL calculation.However, participants may utilize idiosyncratic-demand bidding to obtain a manipulation-related profit.If they do not submit bids during high LMP days 1-4, the average usage for the 4 most recent non-event days (i.e., days 1-4) become the CBL on low LMP day 5, according to PJM rules.Participants then may take advantage of a free-money opportunity to bid on day 5. Bidding on low LMP days but not high LMP days, an activity that seems economically abnormal, can thus be a manipulation scheme.Thus, without real energy curtailment, participants bidding on SUFO days will earn free money from the RTO.
Participants' ability to inflate their CBLs may also depend on their experiences with DR programs.Taking advantage of a SUFO opportunity may require knowledge of CBL procedures and an ability to predict usage.Participants may learn manipulation-related strategies from previous DR experiences.We thus expect an increase in manipulation-related behaviors as participants become more experienced in DR activities.The integrity of a CBL-based DR policy therefore may degrade over time.While in the above example SUFO depends on weather, a common condition shared by a group of participants, not all customers facing the same weather have the same SUFO.A participant's SUFO CBL is calculated by usages on its past non-event days, thus its SUFO is based on its non-event days choices before the SUFO day, i.e., a firm's bidding history established by the RTO's acceptance of its bids, as well as its idiosyncratic demand.Even though it is influenced by the same usage shock, a participant has different SUFO condition with another consumer who has a different bidding history.In the modeling process, SUFO thus can be delineated from aggregate shocks (such as changes in weather) for all participants.

Data Sources and Description
In this section we summarize our data and provide specifications for our statistical model.Our data includes: 1) Hourly locational marginal prices (LMPs) for the PECO zone in the Philadelphia region in PJM, obtained from PJM historical market records, http://www.pjm.com/markets-and-operations/energy/real-time/lmp.aspx.2) Hourly observations of electricity use, CBL, reduction and transmission fees paid during event hours in the economic DR program for each DR participant in PECO territory, obtained from the PECO Energy Company.While market settlement data is available, other data is not.For example, we know when a participant successfully bid in the market, and that the bidding price is lower than the market clearing price (since the participant is dispatched), but we do not observe its bidding price.Participants in the observation were either charged a fixed rate, or had peak-time pricing contracts.We observe participants' behavior between January 2010 and August 2011 during event hours (hours in which participants' bids are accepted by the RTO in the merit order dispatch process).We do not have data on participants' usage during nonevent hours.The observation period is before FERC Order 745, and participants were paid LMP-G-T.Thus, incentives for manipulation were less in the period studied than under FERC Order 745.Our DR data suffers from two types of censoring.First, some DR participants survived in the market in the observation period.Others, however, exited the market during the observation period, so no further observations were available for them.Second, we observe each participant's behaviors on its event hours, but do not have information about its behaviors on non-event hours.3) Hourly data on temperature and cloudy sky conditions for Philadelphia International Airport, obtained from the National Oceanic and Atmospheric Administration (NOAA).Most DR participants in the PECO zone are located within 20 miles of the airport.

Figure 3: The Relationship between Peak Usage Hours and DR Dispatch in PECO Territory
2. We define a "peak-usage hour" as an hour in which the PECO system usage is higher than the system usage in 90% of all hours.The "number of peak hours" is the total number of such peak-usage hours in a month.

Data Summary
One "participant-dispatch-hour" (PDH) is defined here as a particular participant being dispatched for one hour.Dispatches lasting M hours with N participants dispatched at the same time are considered as M ‫ן‬ N PDHs.About 73 percent of PDHs occur between 9:00 and 20:00.Seventy one participants in the PECO area were dispatched in the observation period, for a total of 25,679 PDHs in the 593 days.
Figure 3 shows the relationship between peak usage and dispatch activities.The solid line shows the total of participant-dispatched-hours (PDH) in the given month.The dashed line shows the number of peak usage hours in PECO territory.The monthly total PDHs has a strong correlation (0.645) with the monthly number of peak-usage hours. 2 LMP and PECO loads have a correlation of 0.972.
We use three dependent variables in our regressions.In our first set of regressions, we model CBL (in KWh).We also model Bid Willingness (the possibility that a participant submits a DR bid in the market) and Reduction Ratio (DR percentage reduction once dispatched) as dependent variables.Reduction Ratio is the amount of reduction from the CBL divided by the CBL.Reduction Ratio is available for individual participants only during dispatched hours.
The independent variables we use are as follows: 3. We have fitted PECO load-temperature pairs into a cubic curve; the results imply that the lowest PECO usage occurs at a temperature of 55.5 ЊF.
4. HUI (CUI) in the past for Saturday (Sunday) is calculated as the average of 2 weekend usages in the most recent 3 non-dispatch Saturdays (Sundays), following PJM's CBL-calculation method.
5. SUFO, i.e., the situation that everyone in the RTO has a lower load, may occur due to drop of HUI (CUI) or weekends and holidays.In PJM, the CBL for weekends and holidays are calculated by the average of past weekends and holidays, which theoretically corrects the potential SUFO problems generated by holidays.However, there was no mechanism to correct the HUI (CUI) SUFO in the observation period.DR participants could require the RTO to conduct a temperature adjustment of CBL, however, they seldom made such a request.
6.The logarithmic form of the Learning Variable is used here to account for a declining marginal value of learning through market participation.
1) Learning, which indicates the number of hours that a participant has been dispatched before the current hour.2) LMP and transmission fees.These determine the DR payments.Hourly LMP has the same value across participants for each hour.Transmission prices vary across participants.3) An indicator variable "Weekday," which shows whether a dispatch hour is on a weekday.4) HUI and CUI are the heating and cooling usage indices, respectively in a particular hour.HUI is defined as max [55.5-temperature (in degrees Fahrenheit), 0] 3 and CUI is defined as max [temperature-55.5, 0]. 5) A participant's Past HUI (CUI) is the average of the highest four HUI (CUI) in the particular hour in its most recent 5 comparable non-event days 4 in the last 45 calendar days.This variable varies across participants.6) HUI (CUI) Seemingly Unattractive Free-Money Opportunity (SUFO) is the difference between past HUI (CUI) and current HUI (CUI), i.e., past HUI (CUI) minus current HUI (CUI).As shown in Figure 2, a decline in HUI (CUI) from past HUI (CUI) may create an opportunity for a SUFO. 5 7) The Variable HUI (CUI) SUFO ‫ן‬ Learning Experience is the product of the above HUI (CUI) SUFO variables and the ln(Learning Hours). 68) Work-Hour Indicator, which is an indicator variable with value 1 for weekday hours between 8:00 to 18:00, 0 elsewise.9) Daytime Sky Clear in Heating (Cooling) is a variable with a value of 0 when the temperature is higher (lower) than 55.5ЊF, or when hours are outside work hours (8:00 to 18:00).For other hours, values are: 0 if more than 7/8 of the sky is covered; 1 if 1/2 to 7/8 covered; 2 if 1/8 to 1/2 covered; and 3 if less than 1/8 covered.Three significant effects may accompany a clear sky condition: a: participants may turn off some of their lights when the sky is clear; b: sunshine may heat the buildings so that there is less need for heat in the winter and more need for air conditioning in the summer; and c: a solar onsite generator to handle demand responses can operate more effectively during the daytime if the sky is clear.Since the sunlight-heating effect reduces usage in winter and increases usage in summer, separate variables are created for heating and cooling conditions.10) We include a list of variables indicating the participants' business or industry.There are four categories: College, Commercial, Hospital, and Other.The category "Other" acts as the null, and an indicator variable is constructed for each of the other categories.and January 15 th for college DR providers, 0 otherwise.12) Peak Time Pricing is an indicator variable with value 1 for participants engaged in a peak-time-pricing rate structure, and value 0 for those in flat-fixed retail rate plan.13) Average CBL represents the average of an individual participant's CBL on dispatch hours in the 20-month observation period.Unlike the time-varying hourly CBL, a participant's Average CBL is a constant across time.14) Percentage SD of CBL represents the percentage standard deviation of CBL for a participant in the 20 months of observations.15) The variable "Total Dispatched Hours" represents the total number of hours that a participant was dispatched by PJM to provide DR resources across the observation period.A participant's Total Dispatched Hours is a constant across time.
Table 1 shows the descriptive statistics for variables.Table 2 presents the distribution of participants in various categories and rate structures.PECO load does not have a strong correlation with HUI (-0.064), perhaps because natural gas and other non-electric heating sources are widely used in winter in PECO. 7However, PECO load is highly correlated with CUI (0.55).

Hypotheses
We model the impact of variables on three aspects: a participant's CBL, i.e., whether a factor increase or decrease CBL; DR participation, i.e., whether a factor results in more or less bids that are accepted by PJM; and reduction in DR event hours.For example, the following hypothesis regarding LMP involves the variable LMP's impact on CBL, bid, and reduction.The three aspects of the impact will be tested in three different set of models.The major hypotheses that reveal market manipulations are: 1) H1: Learning experience increases manipulations.With more learning hours, participants may gain a greater understanding of CBL inflation methods and potential freemoney opportunities.DR experience may therefore increase manipulative behaviors.
Participants may also be more experienced in usage reduction.We expect experience to increase CBL, bidding frequency, and observed reductions.2) H2: A participant's CBL is impacted by the weather conditions on its previous nonevent days.It is clear in theory that CBLs are determined by historically energy use, rather than expected energy use, thus are subjected to manipulations.The paper will test empirically that a high HUI (CUI) in the past may imply a larger CBL. 3) H3: A SUFO decreases bidding willingness for participants without manipulation experience, while increases the observed reduction via an inflated CBL.A high SUFO by definition implies a current HUI (CUI) lower than that in past non-event days HUI (CUI), and further may imply current system usage and LMP lower than those in past non-event CBL-determination days.Since a high SUFO is "seemingly unattractive" due to low system LMP, we expect for SUFOs to decrease participation willingness in modelling of bidding behaviors.In modelling of observed usage reduction, we expect SUFO to have a positive impact, due to CBL inflation.4) H4: Experienced participants bid on SUFO days to exercise manipulative strategies.
As indicated in section 3, the existence of a SUFO and the bidding behaviors that take advantage of the inflated CBL on a SUFO day (low LMP day) may imply idiosyncratic-demand bidding.Participants need experience to exercise SUFO biddings since a SUFO is "seemingly unattractive."The learning variable may indicate participants' experience in understanding the market.In modeling participation willingness and bidding behaviors, if we obtain a negative coefficient for SUFO in testing the third hypothesis, and a positive coefficient for SUFO * Learning in the fourth hypothesis, the coefficients may imply that participants accumulate an understanding of idiosyncratic-demand bidding from their experiences.
There are other hypotheses of interest that may enhance market understanding for demand response behaviors, but are not directly related with market manipulation.They are: 1) Since PJM compensated LMP minus generation and transmission price for DR reduction in the observation period, we expect a greater willingness for participating in DR at higher LMP hours.The electricity grid may have higher load during high LMP hours, and participants are also expected to have loads higher than normal.Since a CBL is likely to under-represents normal usage in high LMP peak hours, the impact on observed reduction level is ambiguous.
8. In contrast to early exit, no significant amount of late entry is observed in the dataset.The amount of Demand Response Resources in PJM was therefore declining in the observation period.
2) Since a college may have lower usage during winter holidays, CBL may thus overrepresent normal usage during this period.Colleges thus may have more bidding behaviors in the market to take advantage of the CBL, which shows as positive coefficients in modeling bidding willingness.3) A participant with a larger demand for electricity may use more electricity and thus may gain some advantage in DR bidding, if economies of scale apply.These economies of scale may appear in both the bidding process and the DR reduction implementation.4) Participants may have higher CBL and greater reduction ability on weekdays and during work hours, compared with weekends and off-work hours.5) Compared with those in flat-fixed rate, peak-time-pricing participants may pay more attention to price changes and may have a stronger ability to adjust their consumptions.They may thus provide more DR resources than those who have a flat-fixed rate.

Modeling Consumer Baseline
Modelling CBL tests a part of the first hypothesis (impacts of learning experiences on CBLs) and the second hypothesis.To model CBLs, we will run an OLS regression, a fixed effect OLS regression, and a Heckman model with various explanatory variables.The fixed-effect OLS regression allows each participant to have an unobserved quality (fixed-effect term) that impacts the outcome.A fixed-effect model thus may produce more robust estimators.However, this model cannot provide estimators for variables that a participant has constant values for, such as a firm's business sector.The tests of several hypotheses thus rely only on the OLS model.The OLS and fixed-effect OLS regression models are: In equations ( 1) and ( 2), X i,t includes vectors for individual participant time-varying variables (Learning Hours, Past HUI, Past CUI, and College Winter Holiday Indicator); X i contains vectors for individual constant variables (Percentage SD, Peak Time Pricing, and participant type); X t is the group of vectors for time-varying variables (Work-Hour Indicator, and weekend Indicators); and e i,t is the normal distributed error term.The fixed-effect model in equation (2) does not include X i , whose variables have the same value across time, and includes a constant fixed-effect vector β i for each participant.
In the observation period, many DR participants exited the market during the first winter. 8Further, our 20 months observation period covers two summers and only one whole winter.A selection problem may therefore exist because many of our observations come from participants who survive in the market.To account for this possibility we employ a Heckman model.9. PECO's highest load in 2011 was 1.33% higher than the 2010 highest load.The average of the 2 percentile peak load (top 175 hours) in 2011 increased 1.56% from 2010.The average of the 5 percentile peak load, 10 percentile peak load and average load slightly decreased from 2010.
In the Heckman two-step model, the first step consists of a Probit regression for the selection function as shown in equation (3) below; the second step is an OLS regression, as shown in equation ( 4).
(3) ‫ן‬ Total Dispatched Hours + e ,e ϳ N(0,σ) In the selection equation, the dependent variable Quit is an indicator with value 1 for a participant after it exited the market and 0 otherwise.Exit behavior serves as the dependent variable in the selection function.Z i,t represents variables "Learning Experience", "HUI (CUI) in the Past" and indicator "College Winter Holiday"; Z i consists of variables "Percentage SD of CBL" and other fixed characters for DR participants; Z t represents variables "Work-Hour Indicator" and "Weekday Indicator."The Inverse Mill's Ratio is calculated from the results of the first step and acts as an independent variable in the second step.The variable Transmission Fee is included in the first step but not the second.The variable Transmission Fee can be expected to impact the exit decision, since PJM paid DR resources LMP-G-T and the transmission fee thus impacted a participant's profit.However, there is no apparent reason why the transmission fee would impact the CBL, the dependent variable in the second step.Transmission fee thus can be the instrument variable in the Heckman model.Similarly, the variable "Total Dispatched Hours" is used in the first step but not the second, and the Z t variables are used in the second step but not the first.
Since the data includes repeat observations for participants, the error terms may be correlated for observations of the same participant.Thus, clustered errors are used in all regressions.

Results for Factors that Influence the Consumer Baselines
Table 3 shows the OLS, fixed-effect OLS regression and Heckman model results with CBL as the dependent variable.
Three positive and statistically significant coefficients are obtained for the variable Learning Hours.The result supports our hypothesis that with increased experience, participants learn about CBL manipulative and inflating methods.We note that this increase in CBL occurred despite the expectation laid out by Chao (2011) that adverse selection of DR participants would results in declining electricity consumption for the participants in the DR program.The load data shows that the zonal peak load for the PECO territory in PJM increased 1%-2% in the observation period; 9 however, the average CBL increase reached 15%.The abnormal increase in CBL over time is consistent with manipulative and inflation behaviors and is not thus consistent with the minor change in load patterns.
As expected, DR participants have higher CBLs during weekday work hours.Commercial participants have higher CBLs, compared with the default category.Peak time pricing does not show significant impact on CBL.Past CUI obtains a significant positive coefficient in the fixed effects equation.This is consistent with our hypothesis that a high previous high temperature (rep- 10.Small participants (with low CBLs) remained in the observations in the first winter (HUI period) but not the following summer (CUI period), thus creating a positive correlation between "low CBL" and HUI, i.e., negative correlation between CBL and HUI.
resented by variable past CUI) may increase the use of energy in cooling in previous non-event days and thus inflate the CBL.
Contrary to expectations, the past HUI variable yields a negative coefficient in the fixedeffect OLS regression.PECO is a summer peaking area and thus we would expect that HUI does not impact the system as much as CUI; sample selection bias also may occur due to the fact that many small participants exited the market during the first winter of our observation period 10 : both may contribute to the unexpected coefficient.In the Heckman model, with the Inverse Mill's Ratio, the regression finds a non-significant positive coefficient for past HUI.These coefficients imply that heating demand may not be an important factor for DR in PECO.

ECONOMETRIC APPROACH AND EMPIRICAL RESULTS FOR DR REDUCTION
In this section, we present our econometric methods and the results for modelling participation willingness and real-time reduction.Section 6.1 shows the construction of a Tobit regression model, to analyze variables' impacts, the combination of impacts on participation (i.e., bidding choice) and reduction, on the performance of DR.In section 6.2, we further break DR performance into two parts, participation and reduction, and analyze each part separately.Analysis of bidding participation tests the bidding parts of Hypotheses 1, 3, and 4, while analysis of reduction tests the reduction parts of Hypotheses 1 and 3.The section shows the constructions of a Heckman model, a survival model, and a two-part model.Section 6.3 shows the econometric results for all regressions.

Tobit Regression Model
Due to the nature of the censored data, the reduction amount is observable only when a participant is dispatched.We therefore run a Tobit model to account for this censoring.To run a Tobit model we construct a variable "Reduction Index" with the variable "Reduction Ratio" and use it as the dependent variable in the Tobit regression.Reduction Ratio, the percentage of curtailment over CBL, i.e., (CBL -real time usage)/CBL, varies between 0 and 1. Reduction Index is defined following equation ( 5), which is a method to create a variable ranging [0, ∞) from a variable ranging [0, 1].

Reduction Ratio Reduction Index = (5) 1 -Reduction Ratio
We then create a latent variable, Reduction Index i,t *, which varies between negative and positive infinity, and assume that can be observed as the variable Reduction Index i,t only when it has a value larger than 0. The Tobit model is as follows:

Heckman Model and a Two-part Model
To model DR bidding choices and reduction amounts we again utilize a Heckman model and a two-part model (for more information about the two-part model, see, for example, Duan, Manning et al. (1984)).In these models the first step or part analyzes the choice of whether participants provide DR in the market, and the second step or part analyzes the amount of DR resources provided.The fixed cost associated with bidding (for example, labor cost for submitting bids, communication cost between PJM and DR customers, etc.) may be a significant consideration for DR customers.To distinguish between bidding and reduction is thus important for modelling DR.We use the two models for two reasons: both models capture the two-step DR process, separately analyzing bidding choices and reduction; and both models are capable of processing the specialstructured data we have.The data observes a participant's bidding choices on all hours, but observes a participant's reduction only when its bidding choice is a "Yes."Under the assumption that a participant constantly adjusts its consumption pattern in accordance to market condition no matter whether it submits a bid, the data would be censored, and Heckman model corrects the censoring bias.Under the assumption that a participant reduces its consumption only when it submits a bid and gets dispatched by the RTO, there is no censoring, and the two-step model is appropriate.Either of the above assumptions may be valid, and we present regression results from both models.
In the Heckman model, the first step Probit regression may capture DR providers' choices and the results can be used to obtain an Inverse Mill's Ratio (IMR).The second step then analyzes the demand reduction once participants decide to provide DR, with IMR as an explanatory variable to adjust for censoring.In the two-part model, the two parts are separated.We employ either Logit regression or survival analysis in the first part, and either OLS or fixed-effect OLS in the second part.No IMR is used in the second part.The two-part model does not adjust for censoring.
The first step of the Heckman model is a Probit regression, as shown in equation ( 7).
In Probit model equation ( 7), all variables X i , X t , and X i,t are included.An Inverse Mill's Ratio (IMR) is generated in the first step via the Probit regression.The IMR is then used as an explanatory variable in the second step.
In the second step of Heckman model, we attempt to determine reduction ability after a DR participation decision is made.Since many explanatory variables range between negative and positive infinity, we seek to have a dependent variable that matches the distribution of independent variables, so that the model may produce more accurate results.We use ln(Reduction Index) as our measure of DR reduction ability (or reduction willingness).The two concepts, reduction ability and reduction willingness, both contribute to energy curtailment behavior, and our data does not enable us to distinguish between the two.The variable "Reduction Ability" here and in the following sections models both factors.
The dependent variable "reduction ability," defined as ln(Reduction Index) and shown in equation ( 8), ranges between negative and positive infinity.The variable "reduction ability" turns out to be sigmoid -1 (Reduction Ratio), where sigmoid -1 is the inversed function of sigmoid function as shown in equation ( 8), and Reduction Ratio is the amount of DR reduction over CBL (See, for example, Barro (1977) for similar construction of a dependent variable.) Equation ( 9) shows the OLS regression equation, and equation ( 10) shows the fixed-effect form with a fixed-effect indicator β i and without characteristic variable X i .Impacts from X i variables that do not vary with time (average CBL, participants' type, etc.) are included in term β i in the fixed effect model.Compared with the first-step Probit regression shown in equation ( 7), the two regressions in the second step do not contain the two variable HUI (CUI) SUFO ‫ן‬ Learning Experience.These two variables impact the Bid Willingness in the first step but not reduction in the second step, according to the theory presented in Section 3.They thus become the instrumental variables for the Heckman model.
In the two-part model, two regressions can be used in the first part-a multiple failure survival analysis by Cox model or a Logit regression.Both regressions may capture participants' choices about whether to offer into the DR market.We employ the Cox hazard function model, as shown in equation ( 11).The Cox survival analysis model has fewer underlying assumptions and produces more accurate results.However, the model does not provide coefficients for X t variables.We use the same set of explanation variables in the Logit, Probit, and hazard model equations.
In equation ( 11), k is the hazard rate; and only individual varying variables X i and X i,t are covered in the proportional hazards Cox model.The time-varying variables in X t have the same value across all individuals (such as weather and temperature), and the impacts of X t variables contribute into the baseline hazard term k t as a combined effect.The model does not generate coefficients for those X t variables.

Empirical Results Modeling Demand Response Reduction
The second column in Table 4 shows the results for the Tobit regression on DR reduction as measured by Reduction Index defined in equation ( 5).Columns three to five show the regression results for the first-stage models (i.e., first step of the Heckman regression and the first part of the two-part model).Table 5 shows the result of the second-stage models (i.e., second step of Heckman model and the second part of the two-part model).Both second-stage models contain either OLS or fixed-effect OLS regression.
In the regression results, the first stage analyzes DR participants' choices whether to bid in the market, and the second stage analyzes the reduction ability or reduction willingness given participants submit bids in the market and are dispatched.The Tobit regression reflects the combination of bidding choice and reduction in consumption.
The learning variable obtains significant positive coefficients in both groups of regressions, consistent with Hypothesis 1.Thus, both willingness to bid in the market and observed usagereduction ability increase with experience.This implies that learning experience may improve participants' skill in utilizing CBL manipulation strategies.Experience may also enhance participants' ability to reduce energy usage, as indicated by the positive coefficients in the second step.
The effect of locational marginal price is complex.Results shown from the first-stage regressions indicate that high LMP increases willingness to bid in the DR market, consistent with our hypothesis.The negative coefficients in the second-stage regressions indicate that participants have lower reduction ratios during high LMP hours.When a high LMP occurs, the system may have a peak load, and simultaneously participants are likely to have high loads, reducing their ability to decrease their consumption below their CBLs.The positive coefficient from the Tobit regression shows that higher LMPs increases DR performance.The positive coefficients in the first stage for "Work-Hour Indicator" show that participants are more likely to bid in the market during work hours.However, negative coefficients in the second stage imply that participants have lower reduction ability during work hours.The positive coefficient from the Tobit regression implies that the overall reduction ratio is higher during work hours.gression results for the variable "College Winter Holiday" indicate a higher bidding willingness in the first stage regressions, as expected.Results show that participants with peak time pricing contracts do not significantly differ with other participants in their Bid Willingness and reduction ability.
The explanatory variable "Average CBL" obtains positive coefficients in the first-stage regressions, and negative coefficients in the second stage regressions.According to the first-step regressions, firms using larger amounts of electricity have a greater probability of bidding in the market.This advantage may stem from economies of scale.The negative coefficients in the second stage indicate that firms using more electricity have lower relative reduction ability once dispatched.

CONCLUSION
Demand response (DR) may potentially play an important role in the electricity systems by reducing peak load and preventing social welfare loss.However, the historical-based customer baseline load (CBL) determination method can induce manipulation strategies, reduce social welfare, increase the burden of rate payers, and at the same time jeopardize system reliability.Vulnerable CBLs that can be manipulated may lead to DR programs that are far from effective.
Regressions based on the PECO data further suggest that participants are utilizing manipulation strategies.The existence of manipulated CBLs is indicated as CBLs dramatically increase with learning experience.In addition, there is substantial evidence that firms engage in DR during Seemingly Unattractive Free-money Opportunities (SUFO) when their CBLs potentially over-represent expected usages.In particular, participants create and use more SUFO days to earn extra profit as their experiences accumulate.
FERC Order 745 envisions that DR participants will provide energy during peak hours, generating a large amount of social welfare and deferring costly infrastructure constructions.However, the incentives for manipulation shown here may well have been undermining DR programs.Indeed, because our data comes from the pre-Order 745 era, the adverse effects of CBL-based DR associated with Order 745's DR payment may be greater than those shown here.(See Lu and Li (2013) for a statistical method to test it.) In paying for perceived demand reductions, rather than allowing consumers simply to consume until their marginal benefit equals the price of electricity, FERC has created a system ripe for manipulation.Keeping the system in place required a regime of constant FERC vigilance -as was shown in the cases of several manipulation investigations (see, for example, FERC (2013c) and a recent FERC Order directing PJM to increase the granularity of capacity DR performance monitoring (FERC 2014)), or else the system would devolve into a large "free-money" machine with increasing burdens on customers unable to participate in such programs.
With the Supreme Court's upholding of FERC Order 745, the future of DR payment levels, as well as the measurement of DR, can be further studied.To achieve a more robust CBL may require the DR customers to submit to RTOs more detailed, or even real-time, meter reading data on both event days and non-event days.With all the costs in obtaining detailed data, RTOs in the CBL verification process may face important weaknesses in their market monitoring stemming from the information disadvantages with respect to DR participants regarding participants' operations.Perhaps regulatory agencies concerned with promoting demand management should shift their attention toward marginal cost pricing, as well as demand response in the ancillary and reserve market, which has recently shown itself to be successful.(See PJM (2014).)

Figure 1 :
Figure 1: Marginal Cost of Consuming Electricity in DR

Figure 2 :
Figure 2: A Condition of SUFO Caused by Temperature Drop

Table 2 : Number of Participants by Contract and Participant Type
7. According to Energy Information Administration, 51.0% of home heating in Pennsylvania were provided by natural gas, 20.7% by electricity, and 19.7% by fuel oil.See http://www.eia.gov/state/data.cfm?sid = PA#Consumption.11) College Winter Holiday is a binary variable with a value of 1 between December 15 th

Table 3 : OLS, Fixed-Effect OLS and Heckman Regression Results (dependent variable: CBL in KWh)
aThe observation numbers differ across regressions because the fixed-effect model omits several variables that contain missing values.