Comparison of two residential Smart Grid pilots in the Netherlands and in the USA, focusing on energy performance and user experiences

(cid:1) Evaluation of energy performance and user experiences in Pecan Street and PowerMatching. (cid:1) City Smart Grid pilots. (cid:1) Energy performance depends mainly on local climate and prevailing behavior. (cid:1) Existing Smart Grid set-ups inﬂuences energy performance. (cid:1) End-users prefer technologies that automatically shift their energy use. and of of of of electricity Households in Pecan Street consumed on average, 8% less electricity with respect to the USA average household domestic electricity consumption of 10.9 GW h; while households in PowerMatching City consumed 19% less electricity compared to the Dutch average household domestic electricity consumption of 3.1 GW h. Households in PowerMatching City appeared to have a higher potential to contribute to electricity demand and supply balancing, because their electricity consumption from the grid was largely reduced with increased self-generation. User experiences revealed that end-users in both pilots preferred technologies that automatically shift their energy use, since this requires minimal effort from them. We conclude that the pattern of households’ electricity generation and consumption in Smart Grid pilot projects, and their contribution to peak load balancing in the electricity network is largely inﬂuenced by existing Smart Grid set-ups, local climate and related needs for heating and cooling, the average capacity of installed energy generating technologies and the prevailing energy behavior.

Comparison of two residential Smart Grid pilots in the Netherlands and in the USA, focusing on energy performance and user experiences 1

. Introduction
What insights can be gained from evaluating current residential Smart Grid pilots from a user perspective, in particular with regards to the energy performance of products and services imple- mented in these projects? Providing answers to this question could help to support the successful deployment of future residential Smart Grids.
Smart Grids are a key feature for future energy scenarios, with the overarching goal of better aligning energy generation and demand [1]. In this new energy field, residential end-users are expected to play a more active role in the management of energy resources as energy co-providers, actively participating in the energy market [2][3][4][5]. According to Goulden et al. [1], an active user who is engaged with energy is very important for energy demand side management.
New energy products and services facilitate an active participation of residential end-users in the management of the electric power system. They also enable end-users to have greater management ability over their energy consumption [3]. From a user perspective, these products and services were classified as micro-generators, storage systems, smart appliances, time variable prices and contracts, and energy monitoring and control systems [3]. According to Darby and McKenna [2], these categories of Smart Grid products and services are considered in terms of how energy-related behavior might be shaped in relation to the four aspects of co-provision; i.e., consuming, planning, producing and trading.
Various Smart Grid pilot projects are currently taking place at the low voltage household and residential areas in Europe and in the USA [6,7]. These pilots aim to enable households to take part in energy management in a Smart Grid.
New Smart Grid products and services have been developed and implemented in these pilots. These products and services are expected to support end-users in their role as co-providers in the energy system [8]. In addition to the implementation of these products and services, end-user interaction with these products and services is considered a requisite for a more active participation and involvement of end-users in Smart Grids [9]. However, many of the energy efficiency measures currently being implemented are very much focused on technology adoption [9,10]. Therefore, interaction between end-users and new energy technologies still remains challenging [10]. Previous studies, such as [11][12][13][14], have concluded that end-user behavior and practices will complement the functioning of Smart Grid products and services, and support an active end-user participation in Smart Grids.
Despite the existence of various residential Smart Grid pilot projects, there is currently little knowledge available regarding the participation of end-users in these pilots, and their experiences and interaction with the novelties introduced in these projects. The exception being the studies conducted by [3,[13][14][15][16]. These previous studies have, however, been limited to individual Smart Grid pilots, or evaluation of a limited number of participating households.
Also, little is still known about the energy performance of households in Smart Grid pilots with strong user involvement, as Smart Grid technologies are only recently available [3]. In addition, a comparison of user experiences and energy performance from two different Smart Grid pilots has currently not been carried out.
This study seeks to fill this gap by: (1) Comparing the design/set-up of two Smart Grid pilots (2) Evaluating the energy performance of these pilots (3) Assessing how existing Smart Grid set-ups influences user behavior: demand patterns and energy-efficiency.
Evaluating and comparing user experiences and households' energy performance in a Smart Grid will help to provide representative insights regarding how current Smart Grid products and services influence energy generation and consumption behaviors in Smart Grid households.
The pilots, which have been evaluated and compared in this study, are: (A) PowerMatching City in Groningen, The Netherlands, and (B) Pecan Street in Austin, Texas, USA. These pilots were chosen because (1) they exist already sufficiently long (namely from 2007 to 2015) to have been evaluated and monitored, for which reason reports and data are available at the moment; (2) the pilots served as short cases of early Smart Grids in residential areas, were some of the co-authors of this paper were previously engaged as researchers; (3) of the strong focus of the pilots on user involvement and participation in energy management in a Smart Grid.
In this paper, the energy performance in the evaluated pilots is measured in relation to the pattern of households' electricity generation and consumption. The energy performance could serve as an indicator of how the smart energy system is functioning, and the extent to which residential end-users can contribute to peak load balancing in the electricity network.

The Smart Grid pilot projects
This section describes the Smart Grid pilot projects evaluated in this study. As the technologies implemented in these pilots are for a large part rather similar, we can compare these with each other, focusing on the energy performance and experience of the endusers in both projects. Below we provide short descriptions of PowerMatching City and Pecan Street.

PowerMatching City in Groningen (the Netherlands)
This pilot started in 2007 and was carried out in the city of Groningen, located in the Northern part of the Netherlands [17]. Technologies implemented include hybrid heat pumps, in-home energy displays, PowerMatcher energy matching software, photovoltaic systems, smart meters and smart appliances, smart thermostats, micro-combined heat and power (CHP) systems and mini gas turbines. At a distance, electric vehicles and a wind turbine were connected as well. Table 1 presents the technologies used.
The project focused on attaining optimum capacity management in a Smart Grid, and matching energy services with the demands and wishes of end-users [17]. Phase 1 of the project started in 2007 with the realization of a local Smart Grid with 22 homes and was concluded in 2011. It focused mainly on the demonstration of technical feasibility of the smart energy system. Phase 2 (2011-2014) explored ways to involve the residential endusers. Additional 18 homes were added in 2011, bringing the total number of participating homes to 40. The households in the PowerMatching City pilot were composed of an average of 3 persons, and were recruited through the network contacts of the project partners, as well as calls for participation in a local newspaper. The participants are mainly early adopters, with higher educational level and income compared to average families in the Netherlands [8].
A detailed set-up of PowerMatching City is described in [17].

Pecan Street Austin USA
The Pecan Street Smart Grid pilot is being carried out in Austin, Texas, USA. The pilot started in 2010, and is still on-going. Technologies implemented in the participating homes include: energy management systems, distributed solar photovoltaic energy, plug-in electric vehicles, smart meters, distributed energy storage, smart appliances, in-home displays, programmable communicating thermostats (see Table 1).
The Pecan Street Smart Grid pilot had over 1000 participating households who shared their home or businesses' electricity con-sumption data with the project via green button protocols, smart meters, and/or a home energy monitoring system [18].
The households in the Pecan Street pilot were involved via communication in newsletters, local media, attendance at neighborhood events, and word of mouth within the targeted geographical area of the project [18]. The participants represent a diverse demographic group with an interest in new products and services. They were volunteers and early adopters, with higher educational level and income compared to average families in Texas [18]. A full description of Pecan Street is given by [19].
With regards to the Photovoltaic (PV) system installed in the PowerMatching City, the term 'virtual production' means that households generate PV solar energy via submetered production of a nearby PV system (virtual coupling) and not their own PV installation (for instance, via sub-metering of a PV system on a different building). The group of houses connected via virtual coupling can therefore be controlled as a Virtual Power Plant (VPP).

Data collection
Available information in 2013 to 2014 was compared, such as (1) electricity generation and consumption data of active house-holds with single-family homes participating in the two residential Smart Grid pilots (see Table 2), and (2) quantitative analysis of user questionnaire surveys.
We used existing reports and data available by data portals, project websites and reports. For the PowerMatching City pilot, existing studies and reports [8,20] with results of interviews and questionnaire surveys of participating users were evaluated. Authorized persons in the PowerMatching City project, that had access to the database, retrieved the electricity meter readings used for the energy performance analysis.
We focused on the years 2013 and 2014 because complete data related to electricity generation and consumption of households in both pilots was available for those years. This was not the case in the preceding years, where missing data related to electricity generation and consumption was reported in many households.
In order to characterize the energy performance in the households that were part of the pilots, the following information was extracted from the meter readings: d Amount of electricity consumption per household d Amount of electricity generated per household d Amount of electricity withdrawn from the grid These extracted information were described in terms of monthly averages.  (1) Database containing monthly meter readings of electricity consumption and generation of 21 single-family households [21] (1) Database containing hourly meter readings of electricity consumption and generation data of 85 single-family households [22] User experiences (1) Thesis report containing quantitative survey results of user experiences between 2009 and 2014 [8] (1) Final technology performance report February 2015 [18] (2) Final report of the working group customer research (2014) with results of user experiences with the implemented smart energy system [20] (2) Data portal of Pecan Street organization containing results of questionnaire surveys of 333 participating households in 2014 [23]

Data analysis
An analysis of the electricity generation and consumption took place in order to gain insight in the balance between electricity generation and consumption of the households. The e-gauge readings for hourly intervals extracted from the data portal of the Pecan Street project [23] were converted to monthly averages for the group of households. The hourly meter readings from PowerMatching City households were converted to monthly generation and consumption data for the individual households.
In order to complement the data analysis, we conducted a desk research to explore factors that could influence the energy consumption of the households.

Electricity generation and consumption
Figs. 1 and 2 show the total average monthly electricity generated, consumed and taken from the grid in the selected households in the PowerMatching City and Pecan Street pilots, respectively. Fig. 1 indicates that in 2013, the total average electricity consumption of the selected households in PowerMatching City was 2656 kW h.

PowerMatching City
The highest monthly average electricity generation was about 136 kW h, and this was recorded in the month of August.
With a value of 284 kW h, the highest total average electricity consumption was reported in the spring month of March.
The highest average electricity taken from the grid was in February, with a value of 191 kW h.
In general, the lowest average electricity generation is observed in the autumn months (October, November) and the winter month of January, while the summer months of July and August accounted for the lowest average electricity consumption in the households and from the grid.
In 2014, the total average electricity consumption of households in PowerMatching City was 2490 kW h. The highest average electricity consumption occurred in March, with a value of 245 kW h.
The highest monthly average electricity generation was 159 kW h, and this was registered in July.
The highest average electricity used from the grid was 168 kW h (January).
Similar to 2013, the average electricity generation decreased from the autumn months to the winter months. The lowest electricity consumption in households and from the grid occurred in the summer months. Fig. 2 shows that in 2013, the total average electricity consumption of households in Pecan Street was 9408 kW h. With a value of 1305 kW h, the highest total average electricity consumption took place in August.

Pecan Street
The highest total average monthly electricity generation was 643 kW h, and this took place in August.
The highest average electricity taken from the grid was 662 kW h, and this was registered in August.
The winter months accounted for the lowest generation and consumption, while the spring months were responsible for the lowest electricity used from the grid.
In 2014, the average electricity consumption of households in Pecan Street was 10,756 kW h. The highest consumption occurred in August, with a value of 1520 kW h.
The highest average electricity generation of 763 kW h was registered in August Similar to 2013, the winter months accounted for the lowest average generation and consumption, while the lowest electricity used from the grid occurred in the spring months.
In general, the electricity consumption of the households increased from the spring months to the summer months, and reduced from the autumn months to the winter months. The electricity generation and the electricity used from the grid also followed the same pattern as the consumption.
Tables 3 and 4 present the total averages related to electricity generation, consumption and usage from the grid of households in PowerMatching City and Pecan Street.
From the values in Tables 3 and 4, we calculated the percentual changes in yearly average electricity generation, consumption, and usage from the grid in PowerMatching City and Pecan Street. The average electricity generation in PowerMatching City was 10 percent higher in 2014. The consumption was 6 percent lower in 2014, while the percentage of electricity used from the grid was 18 percent lower in 2014.
The average electricity generation in Pecan Street households was about 12 percent higher than that of 2014. The consumption

Electricity generation and consumption comparison PowerMatching City and Pecan Street
A comparison of electricity generation and consumption of households in both pilots was made. It could be observed that the electricity generation and consumption in PowerMatching City was far lower compared to Pecan Street. In 2013 and 2014, the average electricity generated by households in Pecan Street was about 5 times higher compared to households in PowerMatching City. The average electricity consumption in the group of households in Pecan Street was also 4-5 times higher compared to households in PowerMatching City in 2013 and 2014. In addition, households in Pecan Street used 2-4 times more energy from the grid compared to households in PowerMatching City in 2013 and 2014.
While the summer months accounted for the peak in electricity consumption in Pecan Street in both years, the winter months were responsible for the peak average electricity consumption in PowerMatching City.
Figs. 3-5 show a comparison of the averages for generation, consumption and grid respectively.
Comparing these values to the average electricity consumption in the Netherlands and the USA, households in both the PowerMatching City and Pecan Street consumed less electricity than the average households in both countries in 2013 and 2014. The average electricity consumption in the Netherlands in 2013 and 2014 was 3100 kW h per year [24], while the average consumption for households in the USA was 10,932 kW h [25]. The average electricity consumption of households in Pecan Street was also lower than the average in Austin, which was around 12,000 kW h per year in 2013 and 2014 [26].
Considering the averages over the total number of households in relation to energy generation, consumption, and usage from the grid in both pilots, a standard deviation calculation was carried out to provide an indication of how far the data used in this study deviates from the mean. The calculation revealed a small standard deviation, with values that are not very far away from the mean. This means that the variations in the measurements are quite minimal, and our dataset is representative.

Household characteristics, involvement, experiences and behaviors in both projects
The aim of this section was to gain insights in the involvement of the participants in the pilots, their experiences with the implemented smart energy technologies, and behavior related to their home energy management.

PowerMatching City
Participants joined the project on a voluntary basis. Two of the participants were employees of the main project consortium (DNV, GL, a Dutch energy consultancy company), and members of the project team. They took part in the design, installation and maintenance of the home energy systems.
The participants were mainly early adopters, with high educational levels (Bachelor and Masters degree) and income. The average monthly income of households in PowerMatching City ranged between € 3000 and € 4000. Households in PowerMatching City have a 19% higher monthly disposable income, compared to average families in the Netherlands, that have a monthly average disposable income of € 2900 [27]. Households in PowerMatching City were made up of an average of 3 persons, with children between the ages of 10 and 14. The households generally have profound interest in sustainability and reducing their energy use [8].
The first part of the end-user research analyzed in this study was carried out between 2009 and 2012. The questionnaire survey   of users by Geelen [8] revealed that more than half of the participants reported an increased awareness of energy consumption as a result of their participation in the pilot. However, minimal behavioral changes to be more active in their energy management were reported. This was attributed to the feedback and control provided.
The PowerMatcher system that regulates energy demand and supply functioned at the background. This was because it was automatically programmed to switch on household appliances at times most favorable for the electricity grid. Therefore, participants did not always understand the moment that the heat pumps, micro-CHP and smart appliances switched on, since the Power-Matcher remotely controlled these. Participants, however, wanted more influence and insight in the functioning of the system. The residents reported that the manually operated appliances gave them a greater sense of satisfaction and control over the system. The majority of the participants of PowerMatching City stated that they preferred the automatic steering of their heat pump or microCHP and the smart function of the washing machine, rather than having to adjust the devices themselves manually. This is because it costs them the least effort.
Analysis of evaluative interviews and questionnaire surveys conducted in the context of PowerMatching City by Geelen [8] and PowerMatching City [20] revealed that while manual thermostats were implemented in phase one of the pilot, 69% of the survey participants had preferences for programmable thermostats. This is because they were not used to the manual thermostats, and did not always routinely adjust the settings. This in their opinion resulted in limited interaction with the home energy system, and their ability to influence their energy consumption pattern. It was concluded in these studies that insights and feedback are important for a more active involvement of end-users in energy management.
These findings were incorporated in the second phase of PowerMatching City, carried out between 2013 and 2014. Two new energy services and an improved 'Energy Monitor' (webportal) were developed and implemented. End-users were involved in developing elements of the interface of the new Energy Monitor. This was to enable them to participate more actively in household and community energy management. The Energy Monitor provided real-time insights and an improved feedback and control. It also displayed all the energy flows in the home and overviews of the historical usage, and could be used to adjust the thermostat. Figs. 6 and 7 show the user interfaces before and after installation of the energy monitor.
A community monitor also provided information on energy generation and consumption of the entire street, thereby supporting the residents to compare their household energy use to other households.
In total, 50% of the surveyed participants expressed satisfaction with the adapted energy monitor, since it provided clear, detailed and reliable information that made them more conscious of their energy use. They also felt more empowered to reduce their energy use. Although participants where positive about the new Energy Monitor, they did not always comprehend the information on the monitor. They still stated that they lacked complete insight and control in the operation of the smart energy system, and were not yet able to reach their energy related goals which were (a) saving energy, (b) using energy at appropriate time and suitable amounts and (c) generating own energy.
A community website was also developed for the Energy Monitor. While half of the surveyed participants were active with the website, the rest of the participants were not, because they did not find the website user-friendly enough. Moreover, they preferred to discuss their energy performance face to face with their neighbors.
The end-user research carried out by PowerMatching City shows that the end-users were satisfied with the degree of living comfort afforded by the smart energy system. However, the expectations of the households were significantly higher for the implemented Energy Services than the experiences. Half of the participants reported that the user interface did not provide adequate control and energy feedback to support an active contribution to balance supply and demand.

Pecan Street
Participants in Pecan Street were recruited through advertisements in newsletters, local media, attendance at neighborhood activities, and word of mouth within the targeted geographical area of the project [18].
Two people were selected via a competitive application process to serve on the Executive Committee [16]. This was to support the incorporation of the participants' perspectives in the project implementation.
Like participants in the PowerMatching City, they were volunteers and early adopters, with higher educational level and income compared to average families in Texas [18]. The households in the Pecan Street pilot were composed of an average of 3 persons, with one-third of the households composed of children between the ages of 5 and 18. They had an average yearly income of between $ 75,000 and $ 300,000. Their disposable income was higher than the average disposable income $ 54,000 per year for the USA [29].
Participants in Pecan Street Smart were interested in reducing their carbon footprints, and saving money on energy bills. Over 200 participants took advantage of Austin Energy and Pecan Street's incentive program and installed rooftop PV systems, acquired energy-efficient appliances, such as air-conditioning compressors, and made retrofits insulation and air-conditioning duct repairs in their homes. In total, 69 households also purchased or leased an electric vehicle through these incentives, and received an electric vehicle charging platform from Pecan Street [18].
The majority of technologies implemented in the project were pre-market or new to market. Pecan Street's electricians installed the thermostats and participants were provided with an inperson training and handbooks describing how to program and operate the thermostats [18].
In total, 86% of the 333 households that completed the Pecan Street user questionnaire survey had smart programmable thermostats installed in their homes. One of the questions in the survey was related to how the participants use the thermostats and other devices in their homes. Overall, 66% of the households that had programmable thermostats reported programming their thermostat settings, while 34% did not. Those who did not program their thermostats found them moderately difficult or very difficult to operate. Two participants mentioned that they could save a lot more energy if they understood the high-tech thermostats. In the words of one participant, ''they have geeks design the program, need to have fifth graders do it for 1, 2, 3 steps that are easy to follow, not complicated". Most of the participants, however, expressed satisfaction with the system implemented, especially the software and application that provided periodic report and online monitoring of electricity generation, usage and costs.
With regards to energy usage behavior, the analysis of the questionnaire surveys from the Pecan Street revealed that: -18% of the participants have their electronic devices, such as computers and security devices constantly switched on. -4% of the households owned more than two computers. -56% of the households had a household member spending a considerable amount of time at home every day of the week. -20% of the residents work from home, and most often have their appliances and electronic devices plugged in. -11% of the households leave interior and exterior lights on when not at home to light their garages, hallways, kitchens, porches, and their entire compounds.
With regards to the use of programmable thermostats, a basic energy portal that provided information about electricity generation and consumption supported the control of the thermostats. Pecan Street Organization [18] has also revealed that 82% of the participants who took part in a biannual survey reported using the provided portal to monitor their energy use on a daily basis, while 12% never consulted the portal. A majority of participants (84%), however, reported that they had become more conscious of their electricity use as a result of information they received through the portal that shows appliance-level electricity use. This awareness improved their energy behavior such as; switching off lights, fans and appliances when not needed; setting airconditioning systems to a higher temperature when not at home; and hang-drying clothing instead of using an electric dryer. The remainder of the participants that had access to the online portal reported no behavioral change. They attributed this to a lack of actionable information that could support behavioral changes.
Most of the respondents in the survey expressed satisfaction with the energy monitoring for their solar panels and electric car, and an increased awareness about their energy use.

Factors influencing household energy performance
Based on the results of this study, we considered factors that could have influenced the electricity consumption and generation patterns of households in both pilots. This was based on desk research of literature related to energy use in households. The influencing factors were thereafter related to the prevailing contexts of the evaluated Smart Grid pilot projects.
From a literature perspective, the following factors influence the energy consumption of households [30][31][32][33]: (1) Environmental characteristics: such as availability of solar irradiance and outdoor temperatures (2) Occupational characteristics: such as how energy is used in households (3) Building characteristics: such as the type and age of buildings, insulation, heating systems, floor surface, and type of energy used (4) System characteristics: such as cooling and ventilation systems (5) Types and usage of appliances With respect to environmental factors, the local climate or environment in which houses are located have a major influence on the energy use [30]. In this regard, the outdoor temperature, the availability of solar irradiance and the wind velocity are important factors that should be taken into account. When the outdoor temperature is close to the desired indoor temperature, little or no energy is needed for heating or cooling [30].
Concerning energy generation, abundance of solar irradiance can be used directly to heat and light internal living space, or indirectly in systems that are capable of storing and/or transforming it, such as thermal solar collectors and photovoltaic panels [34]. Therefore, we explored the potential effect of local climatological conditions, such as solar irradiation and temperatures, on electricity generation and consumption patterns in both pilots. Fig. 8 shows the average monthly global irradiation in the Netherlands and in Austin for 2013 and 2014.
It can be seen from Fig. 8 that, in 2013 and 2014, the total average irradiation in the Netherlands ranged between 25 and 230 Watts per meter squares (W/m 2 ). This is about 31% lower than average irradiation of 80-270 W/m 2 in Texas for the same period. While the average irradiation in the Netherlands was 2.5% higher in 2014, the irradiation in Texas decreased by 7% compared to 2013.
Comparing the irradiation in both locations, the graph revealed that average global solar irradiation in Austin was about 2 times higher than in the Netherlands. Fig. 7. Schematic of the user interface of the home energy system after installation of the Energy Monitor. Source: [28]. Fig. 8 shows that the higher global irradiation in Texas was mainly responsible for the higher electricity generation from solar photovoltaics by households in the Pecan Street.
Another factor that might have supported this higher generation capacity is the higher average installed power of distributed energy technologies such as solar photovoltaics in Pecan Street (8 kW versus 5 kW).
Regarding the influence of local temperatures on energy consumption and generation, Fig. 9 shows the average ambient temperatures in the Netherlands and in Austin for 2013 and 2014. Fig. 9 indicates that the average temperature in the Netherlands ranged between 2 and 20°C in 2013 and 2014, while average temperatures in Austin were in the range of 10°C-31°C. Comparing the temperatures in both locations, the graphs revealed that Austin is about 2 times warmer than the Netherlands.
A large difference in temperatures is also observed in the summer months (30°C in Austin versus 17°C in the Netherlands). With an average temperature of about 4.5°C, the winter months in the Netherlands was 3 times colder than Austin, which recorded an average temperature of 12°C in 2013 and 2014. Fig. 9 reveals that average temperatures in Texas were quite high, while temperatures in the Netherlands could be described as being cold to moderate. Compared to other areas of the United States, the warmer weather in Texas means a higher use of airconditioning units for cooling purposes. The use of air-conditioning systems accounts for a about 18% of electricity use, particularly during the summer months [19,39]. Nearly 90% of new homes in Texas are built with central air conditioning. Airconditioning units are also very common in single-family homes, such as those in this study. The questionnaire survey by Pecan Street Organization [18] revealed that in 2013 and 2014, 70% of the households had split unit air-conditioning systems with installed in their homes. Most households (more than 50 percent) stated that the use of air-conditioning units have a significant impact on their energy use (50-90 percent of their energy usage). In addition to the use of air-conditioning units during the summer, most households have ceiling fans that are left on to maintain aircirculation.
Regarding occupational characteristics, the number and age of residents, and income influences the energy use. Large families, and households with young people are expected to use higher amounts of electricity to power electronic appliances such as computers, mobile telephones, video and computer games, and for laundry purposes [40][41][42][43]. Age of household members also influences the internal climate of homes. For instance, older people prefer warmer houses in contrast to younger people.
Compared to PowerMatching City, households in Pecan Street have a larger number of children under 18 living at home. This implies that the use of electronic appliances and air-conditioning will be more common in Pecan Street households. Single-family  homes also have tendencies to use more energy than those living in social housing [44]. This can be attributed mainly to a higher income level.
With regards to the building characteristics, larger-sized houses would require more energy for heating and cooling purposes and lighting, compared to smaller houses. According to Entrop [30] and Vringer [33], the floor surface has a large influence on heating and cooling.
The houses in PowerMatching City and Pecan Street have similar characteristics, with participants living mainly in relatively new or retrofitted houses. A remarkable difference however is that the houses in Pecan Street have relatively larger square footage than those in PowerMatching City. While the floor area of households in PowerMatching City ranged between 100 m 2 and slightly above 200 m 2 , households in Pecan Street had floor areas ranging between 1000 m 2 and 4200 m 2 .
System characteristics involve the use of ventilation, heating and cooling systems to provide comfortable and healthy living spaces in households. According to Entrop [30], household preferences to maintain a certain minimum indoor temperature also partly influence their energy use. Heating systems are used during the winter, while cooling systems are employed to provide more comfortable conditions during warm summer months. While airconditioning units are mainly employed in Austin, in the moderate Dutch climate, cooling systems are not often applied [30]. In the Netherlands, natural gas is mainly used in the winter for heating purposes and, households with heat pumps are most likely to employ these for heating purposes in the winter, and cooling in the summer, which rarely happens [30].
The type of appliances, and how they are used largely influences the average electricity use in households. The usage behavior in relation to the use of lighting and household appliances could greatly impact energy use [30].
Households with high income have more tendencies to acquire more electrical appliances than households with relatively low incomes [30]. High-income earners are also more likely to pay lesser attention to tiny details of their energy use compared to those with lower incomes [30,31]. In general, larger houses use more electric energy for lighting.
Participants in Pecan Street have more electronic devices such as computers, televisions, and lighting compared to households in PowerMatching City. A higher amount of electricity used for lighting, cooling, refrigeration, and for operating appliances, computers, and electronics is most likely in Pecan Street households. This is due to the prevailing energy usage behavior as reported by Pecan Street Organization [18].

Discussion
In this section we will reflect on the main research questions of this study, which was: what insights can be gained from evaluating current residential Smart Grid pilots from a user perspective, in particular with regards to the energy performance of products and services implemented in these projects? This study sought to fill this gap related to little knowledge available regarding the participation of end-users in residential Smart Grid pilots, and the energy performance of households in Smart Grid pilots with strong user involvement.
Comparing the design and set-up of the PowerMatching City Smart Grid pilot in Groningen (the Netherlands) and Pecan Street Smart Grid pilot in Austin (USA), it is observed that the way participants were involved in the pilots was quite similar. End-users in both pilots also had similar characteristics such as high income and educational level, and motivation to participate in Smart Grid pilots. However, a difference was observed in the involvement of participating end-users in the development of the implemented products and services. While participants in PowerMatching City took part in the development of elements of the Home Energy Management Systems (HEMS), participants in Pecan Street mainly provided feedback to pre-determined HEMS tested in their homes.
With regards to the design of Smart Grids, a previous study by Geelen [8] concluded that the design of Smart Grid pilot projects, and the way end-users are involved could influence the adoption of implemented technologies, and household energy consumption. We therefore assume that the approach employed in the second phase of PowerMatching City, where end-users were more involved in product and service development, appeared to have supported a better interaction with the smart energy system, and a more active participation in their energy management.
In general, participating households in both pilots consumed less energy than the average households in Austin and the Netherlands. The participation of the households in the pilots appeared to have supported an increased awareness in energy utilization.
The energy performance, which is based on households' energy consumption and generation patterns, however revealed a large difference in the electricity consumption and generation patterns of households in the PowerMatching City and Pecan Street. In 2013 and 2014, the electricity generated by households in Pecan Street was about 5 times higher compared to the generation in PowerMatching City. While the summer months accounted for the highest electricity generation in both pilots, the lowest energy generation occurred in the autumn and winter months. The higher solar irradiance and average installed power of distributed generating energy technologies, such as solar photovoltaics was the major influencing factor for the higher electricity generation in Pecan Street.
With regards to electricity consumption, households in Pecan Street consumed 4-5 times more electricity compared to households in PowerMatching City in 2013 and 2014. While peak electricity consumption is observed in Pecan Street in the summer months, the winter months were responsible for the peak consumption in PowerMatching City. Higher average temperatures in Austin, and the usage of air-conditioning systems, appeared to have mainly influenced the electricity consumption patterns in Pecan Street.
Although mean temperatures in Austin and the Netherlands did not vary much between 2013 and 2014, the electricity consumption of households in PowerMatching City decreased. In contrast, the electricity consumption of households in Pecan Street increased. Also, while the amount of electricity households in PowerMatching took from the electricity grid decreased with increased generation from solar photovoltaics, grid consumption in Pecan Street increased with increased self-generation.
In our opinion, additional factors such as types and usage of appliances, and the way energy is used in households also partly influenced electricity consumption of households in both pilots.
The energy performance analysis showed that households in PowerMatching City appeared to have a higher potential to contribute to demand and supply balancing in the electricity network compared to Pecan Street households. In general, they seemed to satisfy their own demand in times of high self-production with minimal reliance on the grid.
The energy performance of households in PowerMatching City also appeared to have improved with the improved products and services that supported a better interaction between the households and the smart energy system. This is evident in the reduced electricity consumption in 2014.
User experiences in both pilots showed that a large percentage of participants in both pilots were not always capable of using the implemented technologies, such as smart programmable ther-mostats. This is mainly due to complexity in comprehension of feedback.
The correct setting of programmable thermostats by end-users could support a better regulation of smart appliances, and heating and cooling appliances. This also supports reduction of peak electricity demand, particularly in areas air-conditioning units are mainly deployed. Optimal use of these thermostats is therefore considered a determinant factor in household electricity use and energy efficiency [45,46]. However, in order to increase the adoption of technologies such as thermostats, end-users should not perceive them as being difficult or cumbersome [15].
Another major insight from user experiences in both pilots is related to the use of manual and automated technologies. Endusers in both pilots had preference for technologies that automatically shift their energy use. This is because these kinds of technologies require minimal effort to operate.
Insights from this study re-affirm findings from [10], that concluded that the interaction between end-users and new energy technologies still remains challenging.
It also highlights the existence of various end-user segments, and the need to better address these various segments in the development of new Smart Grid products and services as suggested by [3,15].
Although this study provides the most recent overview of user experiences and energy performance of two different Smart Grid pilots, some limitations have been identified. First is the limited number of households involved in our evaluation, which limits the generalizability of our findings. Second is the lack of equal data from PowerMatching City related to the usage of individual household appliances. Third is the fluctuating number of persons in the households and the missing data related to these fluctuations in the PowerMatching City database. This is the reason why the evaluation was only based on 21 households, instead of the 40 households participating in the pilot.
Despite these limitations, this research contributes to the literature by adding more quantitative insights to the limited knowledge available on user experiences and energy performance of households in Smart grids.

Conclusions
Two residential Smart Grid pilots, PowerMatching City, Groningen (NL) and Pecan Street, Austin Texas (USA) have been compared regarding their energy performance and the experiences of users in these pilots. The objective of the comparison was to gain new insights that could support the successful deployment of future residential Smart Grids.
Measured data on electricity generation and electricity consumption of households in 2013 and 2014 were evaluated. Existing reports with results of surveys of users were also analyzed.
The energy performance revealed that households in PowerMatching City consumed an average of 2.6 GW h domestic electricity, which is 74% lower compared to the Pecan Street household average domestic electricity consumption of 10.1 GW h. At the same time, households in Pecan Street generated about 6.8 GW h of electricity, which is 83% higher compared to 1.14 GW h generated in PowerMatching City.
Households in Pecan Street consumed on average, 8% less electricity with respect to the USA average household domestic electricity consumption of 10.9 GW h; while households in Pecan Street consumed 19% less with respect to the Dutch average household domestic electricity consumption of 3.1 GW h.
User experiences revealed that end-users in both pilots were not always capable of using the implemented Smart Grid technologies. End-users in both pilots preferred technologies that automat-ically shift their energy use, since this requires minimal effort from them.
In general, households in PowerMatching City appeared to have a higher potential to contribute to demand and supply balancing in the electricity network, because their electricity consumption from the grid was largely reduced with increased self-generation. Also, the energy performance of households in PowerMatching City appeared to have improved with the implementation of the Smart Grid technologies.
We conclude that the pattern of households' electricity generation and consumption in Smart Grid pilot projects, and their contribution to peak load balancing in the electricity network is largely influenced by existing Smart Grid set-ups, especially with regards to products and service development (top-down versus bottom up approaches); local climate and related needs for heating and cooling, the average capacity of installed energy generating technologies and the prevailing energy behavior in the USA and the Netherlands.