Free Riding, Upsizing, and Energy Efficiency Incentives in Maryland Homes

We use a unique dataset that combines the responses from an original survey of households, information about the structural characteristics of their homes, utility-provided longitudinal electricity usage records, plus utility program participation information, to study the uptake of energy efficiency incentives and their effect on residential electricity consumption. Attention is restricted to homes where heating and cooling are provided exclusively by heat pumps, which are common in our study area — four counties in Maryland — and were covered by federal, state and utility incentives during our study period (2007-2012). We deploy a difference-in-difference study design. We find that replacing an existing heat pump with a new one does reduce electricity usage: the average treatment effect is an 8% reduction. However, the effect differs dramatically across households based upon whether they receive an incentive towards the purchase of a new heat pump. Among those that receive the purchase incentive, the effect is small or nil, and indeed, the larger the incentive, the smaller the reduction in electricity usage. Those that do not receive incentives reduce usage by about 16%. Our results appear to be driven by the numerous free riders in our sample and by persons who — inferred from their responses to survey questions — might be exploiting the subsidy to purchase a larger system and increase usage, with no emissions reductions benefits to society.


Introduction
1 Alberini and Towe are faculty members at AREC, University of Maryand, College Park. Alberini is also an associate researcher with Fondazione Eni Enrico Mattei (FEEM), Venice, Italy, and an affiliate of the Centre for Energy Policy and Economics (CEPE) at ETH-Zürich, Switzerland. Gans is a consultant with NERA, Washington, DC. We are grateful to Alexander Egorenkov, Alejandro Cajo and Stephen Betsock, for skillful research assistance and for conducting the Summer 2012 surveys. We thank Casey Wichman and the participants of seminars, conferences and workshops at FEEM Venice, Cornell University, ETH-CER Zurich, EMEE 2013 in Ottawa for their helpful comments. We gratefully acknowledge financial support for this project from AREC and from professor Marc Nerlove.
Residential energy efficiency policies in the US and several other countries have traditionally relied on standards for equipment and new home construction, on incentives, and, more recently, on the explicit provision of information about the energy efficiency of devices and buildings. 2 These approaches have received much recent attention due to i) the large contribution of buildings to total energy use (30-40%) and the associated carbon dioxide (CO 2 ) emissions, ii) assessments that improving energy efficiency in buildings would reduce carbon emissions at low or even negative cost (Levine et al. 2007;Choi Granade et al., 2009), and iii) the view that homeowners are reluctant to invest in energy efficiency improvements. 3 Incentives usually take the form of tax credits or direct rebates to the consumers who install insulation or energy-saving windows, and/or purchase high-efficiency heating systems, air conditioners, water heaters, and appliances. Between 2005 and 2009, federal expenditure on residential energy efficiency programs was $2.2 billion (2009 $) (Allaire and Brown, 2011), and in fiscal year 2013 federal expenditures on tax preferences targeting energy efficiency improvements in existing and new homes reached almost $4 billion (2013 $) (Dinan, 2013).
Proper assessment of the effectiveness of incentive programs is inherently problematic because of adverse selection (people are replacing equipment at the end of life; Sandler, 2012) and the likelihood that the programs attract people who are systematically (and unobservably) more motivated or productive at reducing usage. These considerations have led observers to conclude that, unless the presence of "free riders"-persons who pocket the incentive, but would have done the energy-efficiency renovation or upgrade anyway-is adequately accounted for, assessments will generally overstate the cost-effectiveness of the programs, i.e., the cost per unit of energy or carbon emissions saved (Joskow and Marron, 1992;Hartman, 1988;Waldman and Ozog, 1996). 4 Other undesirable behavioral responses are possible. For example, using data from Canada, Young (2008) documents that many households do not dispose of old and inefficient refrigerators, once they replace them with new ones, and keep using them as "beer fridges" (to store cold beverages), for a net increase in electricity consumption. This can be avoided with careful incentive program design, which in turn will increase program complexity and the associated administrative and enforcement costs. Similarly, in programs that seek to replace conventionally-generated electricity with electricity generated from renewables, Jacobsen et al. (2009) find that participating households actually increased electricity usage, despite the fact that the price per kWh is higher than that of conventionally generated power.
Past evaluations often relied on engineering estimates of the energy savings from certain technologies or measures, without observing actual behaviors, and as such, depending on study design and implementation specifics, may have either over-or understated the true energy savings (Jacobsen and Kotchen, 2013;Greening et al., 2000;Sorrell et al., 2009;Metcalf and Hassett, 1999). The form of the incentive may matter, above and beyond the incentive amount (Gallagher and Muehlegger, 2011), and the heterogeneity across programs makes it difficult to extrapolate results across studies and contexts. 5 4 Free riding is generally held to be pervasive in energy efficiency incentive programs. Earlier studies (Joskow and Marron, 1992;Malm, 1996) have produced a wide range of estimates of the share of free riders. Grosche and Vance (2009) examine renovations using a cross-section of data from the 2005 German Residential Energy Consumption Survey, and conclude that free-riding occurs in 50% of the cases. Boomhower and Davis (2013) predict that 75% of the households in a large-scale appliance replacement program in Mexico would have still replaced their appliances if the rebate was zero. Energy models often make assumptions about the share of free riders. For example, Allaire and Brown (2011) assume that 50% of the incentive takers are free riders. 5 In a different setting (hybrid passenger vehicles), Gallagher and Muehlegger (2011) find that tax credits are more effective than non-tax incentives (such as allowing single-occupancy hybrid cars to be driven in high-occupancy vehicle lanes) in increasing the sales of hybrids. Consumers are three times more responsive to state sale tax credits It is not, therefore, surprising that the empirical evidence to date about energy efficiency incentives has been mixed. For example, Dubin and Henson (1988) and Walsh (1989) find that they have no effect on energy efficiency renovations (or expenditures), but in Hassett and Metcalf (1995) a 10% increase in the tax credit leads to a 24% increase in the likelihood of performing energy-efficiency home improvements. Boomhower and Davis (2013) deploy a regression discontinuity design to study a large-scale appliance replacement program in Mexico and conclude that the rebate does increase program participation, but not by much. A 54% increase in the rebate offered by the program (from $110 to $170) raises participation by only 21%. The cost-effectiveness of the program in terms of CO 2 emissions is thus poor. Grosche, Schmidt and Vance (2012) predict that as incentives increase, households substitute away from simpler, less expensive energy-efficiency renovations (such as adding insulation or replacing the heating system) to more complex and expensive ones (e.g., windows, doors, or other structural changes). The latter are less cost-effective in terms of energy savings and carbon emissions reductions.
Empirical work on incentive programs and their effects is, however, no easy task. In the handful of US government-conducted surveys about residential energy use and energy-efficiency investments, renovations are not described in sufficient detail and information about energyefficiency incentives is scanty or absent altogether. Some authors use electricity or gas consumption records provided by the utilities to examine responsiveness to shocks (such as price changes or the provision of feedback on consumption, e.g. Allcott, 2011), but these studies usually lack information about the dwelling and energy efficiency upgrades, which are ignored or assumed away. than income tax credits, perhaps as a result of the stronger salience of and ease of calculating an immediate sale tax credit To circumvent these limitations in the literature, we designed and implemented our own survey of households and have carefully attempted to address each data weakness (or omission) mentioned above in the construction of the sample. We conducted our survey in four counties in Maryland. In the five years prior to the survey, Maryland residents had plentiful opportunities to avail themselves of energy-efficiency incentives. In addition to the incentives made available by President Bush's Energy Policy Act (2005)  The survey questionnaire asked owners of single-family homes whether in the last five years they had 1) replaced the heating system, 2) replaced the air conditioning system, and 3) installed wall or attic insulation, new windows, etc. If so, we further asked them whether they received a rebate or tax credit on the purchase, how much that rebate or tax credit was for, and whether they would have still done the replacement(s) or installation(s), had the rebate or tax credit been absent altogether.
We sent letters to 10,000 Maryland households who own the homes they live in. A total of 1153 of them filled out our questionnaire in September-December 2011. We conducted follow-ups of subsamples of participants and non-participants in the summer of 2012.
A unique aspect of our study is that for all of the 10,000 households that were invited to participate in the survey, we have extensive information about the dwelling and its structural characteristics. Additional information about the characteristics of housing and residents in the census tract and census block group where each household lives comes from the Census. We also have these households' monthly electricity usage and billing records (provided by the local utility) from December 2007 to April 2012, and information about participation in utility programs.
These sources of data allow us to create a unique panel dataset that we use to study equipment replacement, uptake of incentives, and their effect on energy use. We ask three key research questions. First, in a setting where energy efficiency standards are present, does replacing the heating/cooling system with a new and (at least on paper) more energy-efficient system truly reduce energy use? Second, is there heterogeneity in the effect of changing the heating and cooling equipment? If so, what are the main drivers of this heterogeneity? Third, do households who apply for and receive incentives reduce their electricity consumption more, either because they are more "productive" at reducing usage (Joskow and Marron, 1992) or because they are required to purchase more energy-efficient equipment?
In this paper attention is focused on homes served solely by heat pumps. A heat pump is a type of heating technology popular in temperate regions due to its ability to function as both a heater in the winter and an air conditioner in the summer. Heat pumps are common in our study area, which is not served by a natural gas line network. We study heat pump replacements. The 2005 Energy Policy Act required that as of 2006 all new heat pumps meet certain energy efficiency standards, which means that households that replaced their heat pumps in the five years prior to the survey must have adopted more energy-efficient equipment.
We use a difference-in-difference approach where the treatment group is comprised of those who changed their heat pumps within the last five years, the control group is comprised of those who haven't, and the treatment is defined as the replacement of a heat pump. We further examine if the treatment effect on electricity usage depends on household and house characteristics, if it is different for households who received an incentive for their purchase, and if it depends on the incentive amount. We investigate heterogeneity and incentive effects with fixed-effects "within" estimation and fixed-effects quantile regressions.
Briefly, we find that replacing an existing heat pump with a new one reduces electricity usage, after we control for household-specific fixed effects, weather and time of the year. The average treatment effect is an 8% reduction. This figure, however, masks a striking difference in usage reductions across "natural replacers" (those that replace units without incentives) and incentive recipients-and the difference is the opposite of what we expected. The former reduce their electricity usage by about 16%; for the latter the reduction is virtually nil. This happens despite the fact that the manufacturer-specific energy efficiency ratings and the expenditure on the new heat pump is virtually identical across the two groups of replacers.
We also find that the larger the rebate, the less the electricity reduction. For all practical purposes, rebates of $1000 or more have no effect on usage. Rebates of $300 and $450 (the typical rebates offered by utility or state programs) result in usage reduction of 6.22% and 5.5% respectively.
Our findings emphasize the importance of relying on actual energy using measurements, which will incorporate behavioral responses, rather than engineering estimates (Jacobsen and Kotchen, 2013), and that manufacturer-specified energy efficiency ratings and expenditure on equipment is only one part of the story about energy-using equipment choice and its utilization.
Much of the debate in the energy-efficiency incentive literature has focused on the argument that incentives are being spent on energy usage reductions that would have happened anyway. Our findings raise the possibility that, in a setting where energy efficient standards are present, rebates or tax credits may actually be counterproductive, in that they result in no reductions in usage (and hence emissions) at all. The survey responses provide suggestive evidence that the "rebaters" were disproportionately replacing "inadequate" units, leading us to conjecture that the rebates are being used to defray the cost of more powerful units, or of units that end up being used more. This is consistent with evidence from the quantile regressions that households with low usage, conditional on weather and dwelling and household characteristics, had smaller usage reductions or even increased their usage. We also find that free riding is pervasive in our sample, and our free riders likewise tend to reduce electricity use less than the other households.
The remainder of the paper is organized as follows. We provide background in section 2.
Section 3 describes the study design. Section 4 presents theoretical considerations and the econometric approach. Section 5 describes the data. Section 5 presents the estimation results, and section 6 concludes.

Background: Energy Efficiency Incentives and Standards for Heat Pumps
In the five years before the main survey of this paper, Maryland residents had ample opportunities to avail themselves of subsidies for replacing their heat pumps with more energyefficient ones. Funding came from the federal government (in the form of tax credits) and Maryland utilities participating in the State's EmPower Program (in the form of rebates).
The 2005 Energy Policy Act established tax credits for 10% of the cost, with a cap of $500. These tax credits were to expire at the end of 2008. The Energy Policy Act also established energy efficiency standards for heat pumps manufactured in and after 2006. For example, they must meet a minimum Seasonally-Adjusted Energy Efficiency Rating (SEER) of 13. 6 6 The SEER is a measure of the efficiency of a cooling unit designed to be representative of how the system performs over the entire cooling season where the outdoor temperature varies. It is calculated as the ratio of cooling output energy over a season over the input of electrical energy over a season. The Heating Seasonal Performance Factor (HSPF) is the heating analog of the SEER: It measures the representative heater efficiency over a season. In practice, both the SEER and HSPF are measured from a test protocol that varies indoor and outdoor temperatures, and considers compressor type (e.g. single stage, variable), as specified under ANSI/AHRI standard 210/240. In both cases, a higher rating connotes a more efficient unit that uses less energy to heat or cool. For heat pumps, HPSF and SEER are very highly correlated.
In February 2009, the American Recovery and Reinvestment Act (the stimulus package) re-instated the tax credits, increasing them to 30% of cost, up to a maximum of $1500. The tax credits were extended to the end of 2010. Many of the tax credits subsequently continued to be renewed until the end of 2013.
In 2008, the State of Maryland established the EmPower Program, with the goal of reducing energy consumption by 15% by 2015. 7 Participating electric and gas utilities established a number of initiatives to help meet this goal, including-starting in January 2010rebates of $200 and $400 on the purchase of air-source heat pumps in tier I and tier II, respectively. This rebate structure remained in place for all of 2010 and 2011. In January 2012, they were revised to $200 and $300 respectively, a rebate of $500 was established for Tier III air-source heat pumps, and a rebate of $300 was offered for ductless mini-split heat pumps (an option that requires much less pipe-and ductwork) that met specified efficiency requirements. 8 The electric utility serving our study area was one of the participants in the EmPower rebate program.

A. Survey Questionnaire
Our survey questionnaire gathered information about energy use at home, including usage in kWh per month, average bill amount, heating and cooling equipment, fuels, and attitudes towards conservation and energy efficiency. We also asked respondents whether in the 7 See http://energy.maryland.gov/home.html (last accessed 24 July 2013). 8 Tier I air source heat pumps have at least 14.5 SEER and 12 EER and 8.2 HSPF. Tier II air-source heat pumps have at least 15 SEER and 12.5 EER and 8.5 HSPF. Tier III air-source heat pumps have at least 16 SEER and 13 EER and 9 HSPF. A ductless mini-split heat pump must meet the same requirements as a Tier III air-source heat pump to qualify for the $300 rebate. See http://energy.maryland.gov/facts/empower.html (last accessed 24 July 2013). last five years they had 1) replaced the heating system, 2) replaced the central air conditioning system or window units (air conditioning is an important driver of electricity usage in the summer in the mid-Atlantic region of the US), and 3) installed wall or attic insulation, new windows, or various other energy-saving purchases.
If so, we further asked them whether they received a rebate or tax credit on this purchase or cost, how much that rebate or tax credit was for, and which entities provided it (e,g, federal government, state, etc.). Finally, we asked respondents whether they would have still done those replacements or installations, had the rebate or tax credit been absent altogether.

B. Universe and Sample
To develop our sample, we combined several sources of data. The first is MDPropertyView (MDPV), a database that documents all properties in Maryland and is compiled by the State of Maryland using data from each county. For each dwelling MDPV lists the premise address, name and address of the owner, and structural characteristics (e.g., size, single-family or attached home, vintage, construction quality, construction materials, heating and cooling equipment, etc.). The records are updated annually with any new sale(s) and modifications in size or structure. We used MDPV to create our universe for the purpose of the survey, namely owners of single-family homes or townhouses built in 1940 -2000 who live in their own homes. We drew a random sample of 6846 addresses from this universe.
Our second source of data is proprietary information from the local electrical utility, which, since January 2010, has been offering rebates on the purchase of high-efficiency HVAC equipment, in part to meet its obligations as per the EmPower Maryland program. We selected program participants (as of April 2011) who lived in pre-2000 single-family homes or townhouses, and added these individuals to the sample drawn directly from MDPV.
Finally, we further added households that had moved into pre-2000 homes in the four counties in January-June 2011 (we purchased this list from a commercial vendor), and pre-2000 homes that had filed for a building permit with the county in 2007-2011, under the assumption that recent movers and recent renovators may be more likely to undertake energy-efficiency upgrades in their homes. In sum, our universe of candidate sampling units were single-family homes and townhomes built before 2000 and occupied by their owners. Our sampling frame was a mix of stratified and choice-based sampling (see table 1), which means that it is necessary to calculate and use appropriate weights when extrapolating sample statistics to the population. 9 The combined sample was comprised of a total of 10,000 households whom we invited to participate in our survey.

C. Survey Administration
We sent letters to the sample of 10,000 households described above, asking prospective respondents to visit a dedicated website and complete the questionnaire. A user name and password were provided to each addressee.
The letters were evenly divided into two waves. The first was mailed in early September 2011, and the second in October 2011. Each mailing was followed by a reminder letter a week later. The survey was closed on January 4, 2012. We received a total of 1153 completed 9 The weights adjust our samples shares to the population shares. We use the ECSML weights described in Manski and Lerner (1977), where the ECSML weight w j for an observation in stratum j is equal to Q j /H j , with Q j the population share in stratum j and H j the sample share in stratum j. Manski and Lerner (1977) show that the maximum likelihood estimates based on the weighted log likelihood function are consistent for the true coefficients. questionnaires, and 44 letters were returned to us by the US Postal Service as undeliverable, for a response rate of 1153/9956=11.58%. 10 We conducted follow-ups on subsamples of participants and non-participants in the summer of 2012. Specifically, we drew a random sample of 500 households that had received the survey invitation letters back in 2011, but had not filled out the questionnaire. We tracked down phone numbers for 429, spoke over the phone to 61 of them, and administered them an abbreviated version of the on-line questionnaire. This group is henceforth termed our summer 2012 "Phase I" sample.
In our summer 2012 "Phase II" survey, we re-contacted all of the 413 households who in the Fall/Winter 2011 survey had reported changing their heating system within the previous 5 years. We spoke to 104 of them over the phone, and gathered additional information about equipment, subsidies, and related decisions.

D. Data Available for Analysis
For all of the 10,000 households who were invited to participate in the 2011 surveywhether or not they actually participated-we have extensive information about the dwelling and its structural characteristics from MDPV, plus housing stock and resident characteristics at the census block group from the Census. We also have these households' monthly electricity meter readings and bills (provided by the local utility) from December 2007 to April 2012, as well as the heating degree days (HDDs) and cooling degree days (CDDs) for each billing period. 11 We thus have a panel dataset, with up to 54 electricity meter readings per household, and ample details about the dwelling, the household, energy efficiency upgrades and renovations, and the weather.
Household energy-using equipment, expenses, recent replacements, rebate uptake and energy-efficiency upgrade decisions are available for 1153 of these households. In this paper, we form samples based on certain subsets of those 1153 respondents. Specifically, we study households who use exclusively heat pumps for both heating and cooling.
Attention is restricted to heat pumps for three reasons. 12 First, they provide heating in the winter and cooling in the summer, and so households with heat pumps (almost) exclusively use only one type of energy-electricity-for which we have monthly consumption levels for the last five years. Second, regulatory standards have resulted in rapid and dramatic improvements in the energy efficiency of these devices over the last few years, implying that replacing an older unit with a new one should reduce electricity usage, even when the old one is only 5-6 years old (Portland General Electric, 2013). 13 Third, heat pumps are a common device in the study area, which does not have access to the natural gas network. Data from MDPV indicate that over 60% of the pre-2000 single-family homes in our study region use heat pumps.

The Model
across respondents and over time comes from the fact that billing periods start and end on different days for different households. 12 By heat pump, we mean a traditional air-source heat pump or a ductless mini-split heat pump. Ground-source (geothermal) heat pumps are excluded from this analysis for four reasons. First, there are only 26 households with geothermal heat pumps in our sample. Second, installing them for the first time in a home may actually increase electricity usage while decreasing gas and heating oil usage. Third, all of the 26 households that installed a geothermal heat pump in the five years prior to the main survey of this paper received incentives. Fourth, the cost of geothermal systems is much higher than that of air-source or ductless mini-split heat pumps, and so are the relevant tax credits and rebates. 13 See http://www.portlandgeneral.com/residential/energy_savings/getting_started/energy_cost_calculator.aspx (last accessed 30 July 2013).

A. Theoretical Considerations
Decisions about energy-using capital and energy usage are usually represented assuming a two-stage utility maximization process. In the first stage, the household chooses the level of consumption of other goods and the desired level of "energy services" (e.g., thermal comfort). In the second stage, the household chooses the combination of capital stock K and energy use E that minimizes expenditure for any given level of energy services. At the optimum, the slope of the isoquant representing the possible combinations of capital and energy for any given technology is equal to the ratio of capital and energy prices.
To illustrate the effects of energy efficiency standards, consider a household who must replace their heat pump, and assume for the sake of simplicity that the household wishes to stay with the same thermal comfort S 0 . In figure 1, the old technology is represented by the upper isoquant and point A represents the old heat pump. New heat pumps are required to meet more stringent efficiency standards, and are represented by isoquants like the one with points B and C: For any level of capital, thermal comfort is the same as before, but energy usage is less.
If prices are unchanged, the new optimum might be point B, which implies less energy usage than before. If, however, more efficient equipment is more expensive, the household may substitute energy for capital, and, depending on prices and substitution possibilities, may even end up using more energy than before. Tax credits that are proportional to the cost of capital tend counter this effect, since they lower the cost per unit of capital, making the isocost line steeper and raising its intercept. 14 In Figure 1, the energy efficiency at point C is higher than that at B, which in turn is higher than A. 15 The outcomes are even more difficult to predict if we allow for the household to revisit the choice of optimal thermal comfort and/or offer a lump-sum incentive, such as a rebate.
Better energy efficiency reduces the price per unit of energy service, and may engender a combination of substitution and income effect known as the rebound effect (Dimitropolous and Sorrell, 2007), with households purchasing more energy services. A rebate is an income transfer that shifts the isocost line outward, and may result in higher consumption of other goods and/or energy services. Depending on the available isoquants and prices of inputs, the households may choose to consume more or less energy than before.
This simple framework assumes that incentives are exogenously assigned to individual.
In real life, however, individuals seek them out and thus self-select into them. It seems plausible that persons who apply for incentives-whether or not they "free ride"-might be more aware about energy efficiency, more productive at combining capital and energy to produce energy services (which means that they would be looking at a different set of isoquants), and face different input prices. If this is the case, they may choose higher efficiency and lower energy usage than non-incentive households.
It is also possible that some individuals may purchase energy-efficient products because their awareness was raised by the incentive itself or the existence of incentive programs ("free drivers," see Gillingham and Palmer, 2013), or that some individuals might derive utility from knowing that they are contributing to the public good through their CO 2 emissions reductions.
In sum, households will select optimal levels of energy services, capital and energy, and the associated energy efficiency level, but unless we have extensive information about prices, technologies and preferences, it is difficult to predict these choices and how they are affected by incentives. The lack of unambiguous predictions requires that we turn to empirical work to see if installing new higher-efficiency equipment and receiving incentives influence the choice of energy efficiency, capital and energy usage. We also see if decisions and behaviors are affected by the type and amount of the incentive, and/or are different for those individuals whom we identify as likely free riders. We use extensive utility-provided usage data and survey-based information about the efficiency choices of the households in our sample to answer these questions.

B. Econometric Model
One major goal of this paper is to examine household energy consumption when existing equipment is replaced with a newer, and more energy efficient, device. We use a difference-indifference approach and the "treatment" is the replacement of a heat pump with a new one during the last five years. Our estimation sample is comprised of observations from this treatment group and observations from a control group (households who have a heat pump, but did not replace it in the last five years). 16 We have monthly electricity usage records spanning December 2007 to April 2013.
The model is where i denotes the household, t the billing period, E is the electricity usage in kWh in billing period t, DDays is the sum of heating degree days and cooling degree days during that billing 16 We exclude from our sample households whose recently installed heat pumps replaced a heater that uses a different type of fuel (electricity bills would rise for these households, since now electricity is used for heating and cooling, and not just for lighting and appliances). We also exclude the 26 households in our sample with geothermal pumps. period, Winter is a dummy denoting the winter months, M is a set of month-by-year dummies, 17  is the vector of associated effects, and HEATTREAT is a dummy that takes on a value of one when the new heating pump is installed.
We are especially interested in , the average treatment effect on the treated (ATT), which measures the percentage change in energy use that occurs when a heat pump is replaced with a new, and presumably more efficient, one. We note that equation (1) is a reduced-form equation that cannot disentangle the extent of the rebound effect (if any). Based on existing literature (Greening et al., 2000;Davis, 2008;Sorrell et al., 2009;Linares and Labandeira, 2010;, we do not expect any rebound effect to be so strong as to completely erode the technological efficiency gains. We therefore expect  to be negative. 18 Equation (1) includes household-specific fixed effects, which address the potential endogeneity of the decision to replace the heat pump as long as any unobservable house or household characteristics that influence both this decision and electricity consumption are approximately constant over time. In the remainder of the paper, we report results based on the "within" estimator, but for good measure also re-ran our models using a first-difference estimation approach, and got virtually identical results. We report t statistics based on standard errors clustered at the household level. 17 All of the household in our study area face the same electricity price, which varied little during the study period. We therefore let M capture any effects on demand due to changes in prices. The utility applies a two-part tariff, with a fixed fee plus a constant price per kWh (in other words, they do not apply block pricing). 18 Greening et al. (2000) notes that many early studies may have overstated the rebound effect because of their reliance on engineering-predicted energy savings, and that others simply did not meet minimum study quality criteria or did not account for changing equipment, etc. They also note that the rebound effect will depend on the level of awareness that consumers during energy service consumption. For example, consumers are aware of the ambient temperature, thermal comfort and heating fuel bill, but pay less attention to the utilization of a refrigerator. Greening et al's estimate the rebound effect to be 10 and 30% for residential heating. Sorrel et al. (2009) suggest best-guess estimates of the long-run direct rebound effect of 10-30% for residential heating and 1-26% for residential cooling. In Davis (2008), the rebound effect for clothes washers is very small. Linares and Labandeira's argument is based on the low elasticity of residential demand for electricity.
The difference-in-difference approach rests on the assumption that treatment and control units have a common trend (if any). We test whether they do-at least before the "treatment" (replacement of the heat pump)-by estimating a slightly simplified version of equation (1), where TG denotes that the household is a "changer" and is thus part of the treatment group, and m is a vector of month dummies, using a sample that contains the controls and the "changers" before they change their heat pumps. We test the null that  is zero. Failure to reject the null implies that there is no evidence of different trends, at least before the treatment.
Equations (1) and (2) assume that the proportional effect of changing the heat pump is the same for all households. We check for heterogeneous effects by adding interactions between the treatment dummy and i) dwelling or household characteristics, ii) an incentive-received dummy, and iii) (log) incentive amount. We also construct alternate samples that includes only the controls and the incentive recipients, or only the controls and the non-incentive changers.
We do not have any prior expectations on the signs and magnitude of the coefficients on ii) and iii) above. If households attracted into rebate programs are systematically more productive at saving electricity than others (Joskow and Marron, 1992), and we do a good job capturing this heterogeneity with household-specific fixed effects, then there is no particular reason why the treatment effect should be different for them. On the other hand, it is possible that program participants are more productive with the new technology, or better aware of new technologies, or that the efficiency requirements that they must meet to qualify for the incentive are sufficiently stringent to make a difference. And finally, it is possible that incentive itself makes incentive recipients more aware about efficiency and energy usage-and they end up using less energy. Greening et al. (2000) and Sorrell et al. (2009) discuss the possibility that the effects of efficient technologies may vary with the technology's share of total energy bills, thermal comfort and sensitivity to it, and opportunities to adjust the rate of utilization of the technology. With space heating, for example, "temperature take-back" (adjustment in internal temperature in the home, which is accounted for in part by the physical characteristics of the dwelling and in part by behaviors) for example, may be greater among low-income households, or in developing countries.
These considerations suggest that we should check if the effects of the incentives vary across light and heavy users. We accomplish this with fixed-effects quantile regressions. We adopt Canay's (2011) representation where  denotes a specific quantile, Y is log usage, x is the set of right-hand side variables in equation (1), as well his assumptions, which imply that the fixed effects do not depend on  and are present in the conditional expectation of Y it : where  denotes the mean.
The simple two-step estimator proposed by Canay transforms the dependent variable into ˆ is the estimated fixed effects from "within" estimation, and then applies conventional quantile regression to the transformed dependent variable. Canay shows that this two-step estimator of the slopes is consistent at the usual rate and asymptotically normal when n (here, the number of households) and T (here, the number of billing periods) go to infinity.

A. The Main Survey Sample
We begin this section with a brief description of the 1153 people that filled out the main survey questionnaire in the fall and winter of 2011. In terms of education and income, 27.32% of the respondents has a college degree and 60% reports that household income is above $120,000 a year. The median home size is 1856 square feet. Based on the utility electricity records, the median electricity usage is about 1400 kWh per billing period (about 16,800 kWh a year, and a monthly bill of about $250). This heavy usage (the statewide average in 2011 was about 12,000 kWh 19 ) is in part explained by the large share of homes that rely on heat pumps for heating and cooling (hence the lack of another fuel to serve the energy load). Table 2 shows that out of 1153 respondents, 413 (35.82%) replaced their heating system over the last five years. Using weights that account for choice-based sampling, the population rate is 30.41%, which corresponds to an annual rate of 6.08%. We computed the replacement rate for the South mid-Atlantic region (which includes our survey area) using the micro data from the 2009 Residential Energy Consumption Survey, and find it comforting that it is 6% per year (over 2005-09). the replacements received incentives. 21 Since many incentives were specifically targeted at heat pump and geothermal systems, it is not surprising that the rate at which incentives were received is higher for these two types of heating technologies than for other types (e.g., propane gas and heating oil).
The median cost (before incentives) of a new heating system in the sample is $5,000 ($6,000 if a rebate or tax credit was received, $3,500 otherwise, $5,500 for heat pumps, and $26,500 for geothermal systems). Almost 56% of those that replaced the heating system availed themselves of a "tax credit or rebate from the federal or state government, the utility or the manufacturer." For almost 40% of them, the rebate or tax credit was in excess of $500. In fact, 27% received a rebate of $1000 or more. We were able to obtain the exact rebate or tax credit amount from 45 respondents whom we re-contacted during Phase II in the summer of 2012. For this group, the rebate or tax credit ranged from $1,000 to $17,000; the mean was $3,137 and the median was $2,000 (see Figure 2). Appendix A displays the results of a probit model evaluating the impact of observables on the likelihood of receiving a rebate. The sample is restricted to those who replaced their heating system, and the dependent variable is whether they received an incentive for it. Most dwelling characteristics have little or no explanatory power. Even more important, the only variables that are significantly associated with receiving an incentive are the size of the home, income and household size, and the presence of a heat pump, which are also determinants of a household's demand for electricity. 22

B. Selection into the Sample
One concern with the above statistics is that they might be affected by sample-selection bias, so that they do not mirror the true rates in the population of pre-2000, owner-occupied single-family homes and townhomes. We checked for possible sample selection bias using three different approaches.
First, we compared dwelling characteristics and Census-based summaries of the local populations across participants (N=1153) and non-participating households (N=9956-1153=8803). The results of the corresponding t tests are displayed in the table in Appendix B. 23 We found that participants have slightly larger homes and tend to live in neighborhoods with higher incomes and higher shares of White-Caucasian residents, but the differences are modest, even when they are statistically significant (first and second panels of the table). Even more important, participation does not depend on the type of heating system (bottom panel of the data). Electricity consumption per billing period is higher among survey participants, but the difference is only 4%. At the same time, electricity intensity (i.e., average usage per billing period divided by the square footage of the home) is slightly lower among participants, due to their larger homes.
Second, we estimated a probit model where the dependent variable is a dummy denoting survey questionnaire completion, and the main independent variable is electricity usage per square foot (Appendix C, specifications (i)-(ii)). We found that this measure of energy intensity is negatively and significantly associated with the likelihood of participation, but in practice the impact of intensity is very small and has little explanatory power. 24 Augmenting this probit with 23 These are unpaired t tests that assume unequal variances. 24 Increasing energy intensity by one standard deviation above the sample mean reduces the probability of participating in the survey by 1.27 percentage points, bringing it from 11.82% to 10.55%. Increasing it by two standard deviations reduces the likelihood of participation by 2.4 percentage points above the sample mean, bringing it to 9.37%. additional regressors (house characteristics, neighborhood characteristics) does little to improve the fit of the model (Appendix C, specification (iii)).
Third, in the summer of 2012 we drew a random sample of 500 households that had received the survey invitation letters back in 2011, but had not filled out the questionnaire. We found phone numbers for 429 and managed to speak to 61 over the phone. The rate at which these households had replaced their heating system since 2007, and resorted to rebates or tax credits, were virtually identical to those reported by the 1153 households in the 2011 survey. 25 Taken together, these findings suggest that if there is selection into the sample, it is probably not very important for the purposes of our analysis.

C. Usage and Energy Efficiency Comparisons
In the remainder of this paper we use the billing and usage data, and other information, from N=70 of the households who changed their heat pumps between 2007 and 2011, and from whom we were able to get detailed information about the time of this change and the features of the new heat pump. 26 We supplement this sample with N=394 households with heat pumps who had not changed them in the previous five years. Because electricity billing has an approximately monthly frequency, and we have usage and billing records from December 2007 to April 2012, the electricity consumption readings form a panel dataset with up to T=53 observations per household.
Our first order of business is to check if the controls are similar to the treatment group (those who have changed their heat pump in the last five years) prior to the treatment (replacing the heat pump). Before changing the heat pump, the average usage of electricity per billing period in our treatment group was 1,776 kWh, whereas that in the control group was 1,650 kWh. 27 The corresponding log usage statistics are 7.3635 and 7.2424. Formal t-test results based on log usage and other measures are reported in table 3. 28 These results show that households who changed their heat pump in the last five years tended to use more electricity per billing period before changing the heat pump than the controls. Log intensity (where intensity is usage divided by square feet) is roughly the same in the two groups. Table 4 displays information about house and household characteristics for the treatment and control groups, showing that treatment households are wealthier and "older" than the control households, and live in larger homes.
Treatment and control groups, however, live in homes of similar vintages, construction materials, and types. 29 About 69% of those who changed their heat pump reported receiving an incentive on this purchase. A federal tax credit was present (alone or in conjunction with other incentives) in 62% of these cases. We check whether-prior to the treatment-those households that received a rebate or a tax credit are similar to those that changed their heat pump but did not receive an incentive (table 5). If anything, incentive recipients use somewhat less electricity in total than non-recipients, but the difference is not statistically significant, and consume significantly less on a per square foot basis. Taken together, the results in tables 3 and 5 confirm that it is appropriate 27 As we describe in more detail below, these statistics are based on accounts that were active at the time of the main survey and Phase II follow-up in the summer of 2012. The sample is restricted to households with heat pumps and is limited to billing periods with length between 28 and 33 days. We reminder the reader that we excluded households who changed heat pump between 2007 and 2012, but in doing so replaced heating equipment that used a different fuel. 28 Because observations within a household may be correlated over time, we first computed averages for each households. The t tests reported in tables 3 and 4 use a single figure per household-the household's average over time. 29 Dwelling "vintage" or cohort effects have been found to be important determinants of residential energy use in other studies (Cost and Kahn, 2011). We don't expect such effects to be at play here, given that the control and treatment groups are similar in terms of construction year. for our econometric models to include household-specific fixed effects, and also suggest that there is no particular reason to expect that the treatment effect should be very different across incentive and non-incentive "replacers." Incentive recipients and non-recipients also appear to be similar in terms of the energy efficiency of the equipment they bought. In Phase II we asked respondents to report the cost and describe certain technical aspects of their new heat pumps. For some of those who did not report the SEER of their heat pumps, we were able to recover information from the rebate paperwork filed with the utility. We thus estimate that the average SEER of the new heat pump is 14.69 for those who did not receive an incentive, and 15.47 among those who did. These figures are very close, indicating only a slightly higher efficiency among incentive takers. As shown in table 6, the median, mean, maximum and indeed the entire distribution of the cost of the new heat pump is virtually identical across these two groups of respondents.

D. Reasons for Replacing a Heat Pump
In Phase II we also asked all "changers" to tell us the reasons for replacing their heat pumps. Table 7 shows clearly that main reason is that the old heat pump broke or "was aging," especially among incentive recipients (89% v. 77% among non-recipients).
Only two out of 22 incentive recipients indicated that "there were offered a rebate or a tax credit" without also selecting the response options "the old one broke" or "the old one was aging." Quite a few incentive recipients-about 42%-also said that the "old one was inadequate." When asked what they would have done in the absence of the rebate or tax credit, most of them (71%) told us they would have bought exactly the same model, 15% said they would have bought a less expensive model, and only about 5% said they would have bought a less efficient system (table 8).
Based on this evidence, it appears that free-riding is pervasive in our sample. If we use a restrictive definition of free rider-a household who replaced the heat pump because it was broken or aging, and would have bought exactly the same model in the absence of the incentive-about 50% of our incentive-takers are free riders. If we use a broader definition (they replaced the heat pump because it was broken or aging, but still took the incentive), then 89% of them are free riders.
Those who did not receive incentives generally weren't sure why they did not apply for the incentives (5 respondents), were not aware of the existence of the incentives (6), claimed (incorrectly) that the incentives were not available at the time they replaced their heat pumps (5), or simply did not want to deal with the related paperwork or found the rebate too small to bother (total 2 persons).
One question at the heart of this paper, which we examine in the next section, is whether electricity usage is reduced just as much among incentive recipients as among persons who changed their heat pump without a rebate or tax credits. In other words, given the predominance of free riding, are the incentive takers at least as productive at reducing electricity usage as the other replacers? This question is all the more important for policy purposes when we consider that many of our incentive takers appear to be free riders.

A. Sample Construction and Preliminary Checks
In our regressions, the sample is comprised of monthly usage readings from households who did and did not change their heat pumps between 2007 and 2011. Attention is restricted to accounts that were active at the time of the fall/winter 2011 survey, and to households that clearly replaced an existing electric heat pump with a new one. We therefore exclude households whose previous heating systems used a different fuel or who fail to report whether there was a fuel switch. We also exclude the 26 households with geothermal systems.
Since we use a difference-in-difference approach, our first order of business is to check that pre-treatment trends are the same for the treatment and control groups. We use two approaches. First, we compute mean log usage by month for control and treatment households before the treatment, which we plot in Figure 3. The graph shows that, as expected, there are obvious seasonal patterns in electricity consumption for both groups of households. There is no obvious evidence of a trend, and the two groups track each other closely, with the (future) changers generally above the non-changers. This could be because the former's equipment is older and less efficient, or because they have larger homes (see table 4), or because they are generally heavy users due to preference and household composition.
Our second approach is to estimate equation (2) using observations from the controls and the heat pump changers before the change. We find no evidence of a trend (ˆ= 0.001185, t statistic 1.24) or of a systematically different trend across changers and non-changers (  = -0.0004913, with a t statistic of -0.15).

B. Results from the Difference-in-Difference Approach
We report the results of regression (1) in table 9. This regression uses our broadest "clean" sample. If a household changed the heat pump in the last five years, and reported the year of the change, but not the exact month, we simply exclude all of the observations from the year of the change ("dropme=0"). The results show that electricity use is well predicted by weather and time of the year, and that a new heat pump reduces electricity usage by almost 8%.
In table 10 we explore the robustness of this result with respect to various criteria for constructing or further cleaning the sample. For example, row (A) refers to the same model as in table 6, but here our control group includes only households who haven't changed their heat pump in the last five years, and haven't installed or replaced insulation, windows, or otherwise improved the thermal integrity of their homes. The average treatment effect is virtually the same. 10 show that the results are robust to, and get somewhat statistically stronger with, dropping two households who were found to have done additions to their homes, further excluding a household with a photovoltaic system, and further dropping observations from the last two months for which we have usage and billing data.

Rows (B)-(D) of table
In table 11 we look for possible evidence of heterogeneity in the ATT. For simplicity, we have entered one interaction at a time in equation (1). The electricity demand reduction associated with replacing a heat pump ranges from 0 to 16%. The strongest effects are observed for homes built before 1990, larger homes, households with persons aged 65 and older, and homes with attic insulation.

C. The Effect of Incentives
In table 12 we investigate whether the ATT depends on energy efficiency incentives. We begin with creating a sample that is comprised of the controls and the changers who did not receive incentives. The results of this run are striking: The ATT is now much stronger, for an estimated reduction in electricity usage of 16%. The associated t statistic is -3.02.
We swap non-rebaters with rebaters in row (B), and the results are starkly different: The reduction in electricity consumption is virtually nil for those who received an incentive. We get similar results if we restrict the sample to households who received a rebate, which means we are comparing their own pre-and post-change usage. This approach produces an estimate of  equal to 0.6% (t statistic 0.15). By contrast, in the no-rebate-only sample the estimated treatment effect is a 14.32% reduction in usage (t statistic -2.90).
In row (C) of table 12 we pool changers and control units, but include in the model an interaction between the treatment dummy and receiving an incentive. "Natural" changers reduce usage by 16.74% (t stat. -3.28) while incentive recipients only by 3.5% (t stat -1.02), which confirms the results in rows (A) and (B). When we distinguish for the source of the incentive (rows (D) and (E)) it would seem that recipients of federal tax credits accomplish more substantial electricity usage reductions that the rest of the incentive takers. This result is consistent with the theoretical considerations in section 4, but these results should be interpreted with caution because most incentive takers appeared to combine incentives from different sources, and because the relevant coefficient in row (D) is insignificant at the conventional levels.
We experimented with different ways of entering the actual rebate amount in the model-linearly, as a share of the cost of the new heat pump along with the cost of the heat pump, summarized into broad categories, etc.-and the results generally agree with the notion that the larger the rebate, the less important the effect on electricity consumption. For example, row (F) in table 12 includes the treatment dummy and its interaction with the log of rebate amount. The coefficients indicate that electricity consumption decreases by 15.86% among nonrebate takers (t stat-3.11), by 6.22% (t stat -.1.97) among those who received a $300 rebate, by 5.5% when the rebate is $450 (t stat -1.66) and by insignificant amounts (3% or less) for rebates of $1,000 or more.
We checked if these results vary with house size or income (by using three-way interactions between HEATTREAT, the rebate dummy or log rebate, and a high income/large house dummy, or by restricting the high/low income or large/small house households), but found that they don't. 30 Normalizing usage by the size of the home does not change these results.
That usage actually increases (if weakly) with the size of the incentive points to two possibilities: That this effect is due to the free riders, which might be insensitive to the incentive amount, or to persons who are upsizing their system, for whom the larger the incentive, the greater the post-change usage. In table 13 we run regressions after excluding from the sample persons who meet our strictest definition of free rider (replaced a broken or aging system, and would have purchased the same model even in the absence of the incentive), persons who replaced a broken and/or aging system, and persons who stated they changed the old heat pump because it was "inadequate." The ATT of changing the heat pump is stronger when we exclude observations from these persons, suggesting that the free riders in our sample do worse, in terms of electricity usage reduction, than the households who changed their heat pump without an incentive. Dropping 30 We also examined a possible technical explanation. Electric heat pumps do not perform particularly well at cold temperatures (below 37 F), and so many households in our study region have a propane gas "back-up" which can be deployed on very cold days. If for some reason the households who received incentives purchased units that perform better at low temperatures and reduce reliance on back-up fuel on particularly cold days, then they might still have reduced overall energy use, even though their electricity consumption has not decreased. We checked the description of the heat pump technology reported by our interviewees, and gas back-ups are just as common among incentive and non-incentive households. We also tested the abovementioned conjecture by adding a two-way interaction between the treatment dummy and the winter season, plus a three-way interaction between the treatment dummy, incentive or log incentive amount, and a winter dummy. The regression results indicate that the effect of changing the heat pump are uniform across seasons for both incentive and non-incentive households. We conclude that technical differences in the equipment are unlikely to be driving our results. these groups from the usable sample, however, does little to the estimated slope of log incentive, which is similar to the base model in table 10.

D. Fixed-Effects Quantile Regressions
We plot in Figure 4 the coefficients on the treatment dummy, HEATTREAT, for selected quantiles  from fixed effects quantile regressions with the same right-hand side variables as equation (1) and table 6. The results indicate that, conditional on the covariates, the effect of changing the heat pump is smallest and statistically insignificant at the lowest quantiles-namely the 10 th and 12.5 th (0.000236, t statistic 0.004, and -0.0079, t statistic -0.17). 31 By about the 40 th percentile, the coefficient is -0.0582 (t statistic -3.296). The coefficient is very stable (about -0.07 to -0.08) for percentiles greater than the 50 th , although it is not statistically significant at the top quantiles.
When we enter the interaction of HEATTREAT with the incentive dummy, the fixed effects quantile regressions (summarized in Figure 5) suggest that households who did not receive incentives reduced electricity usage by no less than 10%, even at the lowest quantiles conditionally on the covariates. The largest reductions are observed at the 50 th percentile (about 14% reduction). At the high percentiles the electricity demand reduction are similar to those at the low percentile. The story is completely different for households that received incentives: Except for the highest percentiles, these households appear to have increased usage. At the low usage percentiles, conditionally on the covariates, electricity usage actually increases by almost 16%. incentive changers shown in Figure 5. At $450, households in the 10 th percentile increase usage by about 11%. The effect becomes smaller as we increase, and by the 90 th percentile usage decreases by about 6%. The pattern is similar for a $1000 incentive, where the bottom 10% and the top 10% of the distribution of usage, conditional on the covariates, experience a 14% increase and 5% increase, respectively. The effect is positive but small at the other quantiles.

Conclusions
We have developed a unique dataset based on an original survey of households, combined with data on home characteristics and neighborhood location, as well as monthly electricity usage records, utility program participation, and other sources, and used it to investigate three key questions about energy-efficiency upgrades and the role of energyefficiency incentives. We have focused on heat pumps and have used a difference-in-difference estimation approach where the dependent variable is the log of energy usage in a billing period.
We account for unobservables (and thus the possible endogeneity of the decision to change the heat pump and/or apply for an incentive on the purchase) using household-specific fixed effects.
We have found that replacing an existing heat pump with a new one does indeed reduce electricity usage. The average treatment effect on the treated is an 8% reduction, the effect being more pronounced for households with larger homes, for homes with insulation, and in households with elderly persons.
From a policy perspective the more important question is whether households who have received energy-efficiency incentives reduce usage to a different extent than the others. Free riding is pervasive in our sample (50-89% of incentive recipients), but we have no a priori reason to believe that free riders are any worse at reducing energy usage. Moreover, incentive takers and non-takers are similar in terms of the manufacturer-specified energy efficiency rating and expenditure on their new heat pumps.
What we find here is striking: The average treatment effect is a 16% electricity usage reduction among non-recipients, and virtually nil among incentive recipients. Further controlling for the rebate amount suggests that the larger the rebate, the smaller (in absolute value) the reduction in electricity usage. At $1000 and more, there are virtually no reductions in usage. This happens despite the fact that rebate takers and the other households who changed their heat pumps are similar in terms of the efficiency and the cost of the new equipment they purchase.
While non-incentive households deliver CO 2 emissions reductions in a cost-effective fashion, 32 only for very low incentive amounts would cost effectiveness be reasonable for incentive takers. For a $450 incentive, for example, each ton of CO2 reduced costs $68.05. The cost increases quickly as the incentive is raised.
Our results appear to be driven by the numerous free riders in our sample (50-89% of those who received an incentive, depending on the definition of free rider) as well as by those who most likely upsized their system, since the old one was "inadequate." Quantile regressions show that rebate takers at lowest end of the distribution of electricity usage, conditionally on the covariates, actually increase usage after they replace their heat pumps. If our findings are correctly interpreted as an instance of the rebound effect, then it is a surprisingly strong rebound effect-and one that is driven by the cost of the durable equipment, rather than the price of electricity or energy efficiency per se, since households who did not take incentive experience a net reduction in electricity usage.