How Renewable Energy is Reshaping Europe’s Electricity Market Design

We present a systematic review of the challenges to the regulation of electricity markets that are posed by the integration of variable renewable energy sources. System integration is the key to developing the required flexibility, because flex ibility options exist at all system levels and within the competitive as well as in the regulated (network) domains. The fluctuating nature of variable renewable energy changes the dynamics of investment decisions. We develop a framework for analysing relations between aspects of the regulation of the power sector that need to be coordinated in order to achieve (or at least improve) economic efficiency. We base the framework on the technical functionalities of the electricity infrastructure, which we group along three dimensions: system level (from retail/dis tribution to transmission/wholesale), geographic scope (the connection between electricity systems) and time scales (from real-time operations and balancing markets to the investment time scale). The framework helps identify regulatory chal-lenges—potential inefficiencies due to a lack of coordination—and to place them into context. The picture that emerges from this approach is that the institutional fragmentation of the European electricity sector will become increasingly burdensome as the development variable renewable energy requires ever closer coordination between countries, between the different levels of the electricity system and between markets that serve different time scales. Interactions between elements of market design and regulation such as congestion management, renewable energy policy and system adequacy policy affect each other and are an additional reason for a system integration approach to regulation.

The following article is a preprint of a scientific paper that has completed the peer-review process and been accepted for publication within Economics of Energy & Environmental Policy.
While the International Association for Energy Economics (IAEE) makes every effort to ensure the veracity of the material and the accuracy of the data therein, IAEE is not responsible for the citing of this content until the article is actually printed in a final version of Economics of Energy & Environmental Policy. For example, preprinted articles are often moved from issue to issue affecting page numbers, and actual volume and issue numbers. Care should be given when citing preprint articles.
One of the main challenges for the electricity industry is how to adjust to a large share of variable renewable energy sources (RES). Power system flexibility is widely recognized as the key to dealing with the variability and uncertainty of RES. The International Energy Agency (2011) defines flexibility as "the extent to which a power system can modify electricity production or consumption in response to variability, expected or otherwise". Technically, there are many options for increasing the flexibility of the electricity system: flexible low-carbon a Faculty of Technology, Policy, and Management, Delft University of Technology, Jaffalaan 5, 2628BX Delft, The Netherlands. generation such as biomass and fossil fuel plants with carbon capture and storage; storage of electric energy; increased demand flexibility; more network capacity to spread fluctuations in supply geographically; and stronger integration with other energy infrastructures such as heat and transport. The ideal for a well-designed market is that it provides optimal incentives, for investment as well as for operation, in these disparate options. Current market design is fragmented, however, with separate rules for wholesale and retail markets, a number of single-issue policy instruments, and different implementations per country. Furthermore, the networks, being monopolies, do not necessarily respond to market signals, so special attention needs to be given to aligning their role with market developments. We take the physical characteristics of flexibility options as a starting point for the development of a framework for analyzing the regulation of the sector. Building upon the concept of institutional cohesion, as developed by Finger et al. (2005), Künneke et al. (2007) and Künneke et al. (2010), our analysis framework can be used to identify technical challenges that stem from the integration of renewable energy that need to be dealt with by the regulation of the power sector. We base our analysis on the literature, but we do not claim to provide an exhaustive overview of the literature because of the inherently wide scope of our analysis.
Much attention has been paid to technical aspects of RES integration (cf. Holttinen et al. 2013;Cochran et al. 2014;Lund et al. 2015;. How these technical changes should be reflected in changes to electricity market design and regulation is the subject of a growing body of literature, see e.g. Newbery (2010), Eurelectric (2010), Milligan et al. (2012), Regulatory Assistance Project (2014), Dragoon and Papaefthymiou (2015) and Henriot and Glachant (2015). More recently, in special issue about market reforms in relation to high shares of renewables, a wide range of issues has been discussed by e.g. Neuhoff et al. (2016), Praktiknjo and Erdmann (2016), Green et al. (2016), Pollitt and Anaya (2016) and Brunekreeft et al. (2016). A comprehensive list of policy recommendations is given by Perez-Arriaga et al. (2017). Because the implementation of flexibility affects all system levels, from the household connection to cross-continental energy flows, the regulation of the electricity system likewise needs to be integrated. We use the term 'energy system integration' to refer to the need to coordinate all aspects of the energy system, technically as well as institutionally, in order to make the system work optimally. Arguably, the need for energy system integration is not new, but is increasing as a result of the development of variable renewable energy sources.
A second reason for energy system integration, aside from the introduction of variable renewable energy, is the fact that different policy instruments interact with each other. For example, Herrero et al. (2015) discuss how interactions between short-term market prices and investment incentives are affected by different policy instruments. Beato and Delgado (2015) address interactions between different climate policies and Batlle et al. (2012) analyse how RES support schemes impact electricity markets. Cross-border effects of differences in CO 2 policy are discussed by Richstein et al. (2014), while cross-border effects of capacity markets are discussed by Bhagwat et al. (2017), Meyer and Gore (2015) and Mastropietro et al. (2015).
In this article, we focus on the institutional aspects of electricity system integration (so not all energy carriers): the alignment of regulations, policy instruments and market design at all system levels and between (national) electricity systems. Market design is commonly considered to refer to the organization of the market place, involving decisions such as between a mandatory pool and a voluntary exchange, gate closure times, treatment of start and stop costs, balancing mechanism design and congestion management. Network regulation covers the rules for market parties' access to the networks, including network tariffs, as well as the gov-Copyright © 2018 by the IAEE. All rights reserved. ernance of the network operators, e.g. with respect to investment decisions. The third category of regulations consists of policy instruments such as renewable energy subsidies, carbon policy, capacity mechanisms and nuclear energy policy. Sometimes, some of these are considered as part of the market design. We will sidestep the discussion about definitions and loosely refer to the total set of rules that guide the electricity sector as the regulation of the electricity sector, following Pérez-Arriaga (2013).
We develop and apply our framework to four areas that are particularly affected by the introduction of variable renewable energy sources: 1) system adequacy, 2) coordinating network investment with the market, 3) short-term market design and 4) congestion management. These four areas were chosen because they cover both short and long-term aspects of the market (into which variable renewable energy sources are introduced) and the interactions between the market and the networks. We hope to provide a fresh way of understanding the policy challenges that stem from the introduction of variable renewable energy resources and how they relate to each other.
For a number of reasons, electricity markets rarely are designed according to the textbook, but tend to evolve in a continuous process in which stakeholder interests and knowledge deficiencies (e.g. about future effects of policy interventions) play a large role (Holburn and Spiller 2002;Correljé and De Vries 2008;Henisz and Zelner 2014). We do not intend to explain these dynamics, but provide an analysis framework for identifying improvements. We focus on Europe in order to be able to make context-dependent analyses. To keep the scope manageable, we limit ourselves to the electricity sector, even though it is clear that more flexibility options are available through the integration of electricity with other energy carriers. Furthermore, because our analysis emphasizes the relation between the different issues of power sector regulation, we cannot provide an in-depth and complete literature review of all the single issues themselves, because each issue in itself would merit such a review. In the next section we will present our analytic framework. The following four sections present the four case applications of the framework and in section 7 we conclude.

f 2. A FRAMEWORK FOR ELECTRICITY SYSTEM INTEGRATION g
We will now introduce our framework for the analysis of electricity system integration issues. We derive our framework from the physical characteristics of the electricity system, on the presumption that the institutional design needs to reflect physical (technical) interrelations. This approach follows the principles of institutional cohesion, as developed by Finger et al. (2005) and Künneke et al. (2010). According to this view, all technically necessary functions must be reflected in the institutional design: either they can be performed by market parties (if necessary-usually-with additional regulation) or they are monopoly functions and must therefore be assigned to a specific party, and regulated. Therefore, the list of necessary technical functions provides a checklist for the institutional design, not only for the question whether they are performed, but more importantly whether the institutional organization of these functions takes care of the technical need for system integration. We discern three dimensions along which physical interactions take place in the electricity sector.
The first dimension of our framework is the geographic scope. The topology of electricity networks is a strong determinant of their performance. One of the most cost-effective options for integrating renewable energy is the increase of transmission network capacity within and between European countries in order to smooth out fluctuations. Improving the efficiency Copyright © 2018 by the IAEE. All rights reserved. of network congestion management is a therefore a key issue (cf. Ummels 2009;Brancucci Martínez-Anido et al. 2013;Neuhoff et al. 2013). The increase in the geographic scope by integrating electricity markets has been an objective since the start of liberalization of electricity markets in Europe, initially for the sake of improving economic efficiency through market integration; now, large volumes of intermittent generation and the desire to use the flexibility of hydro plants optimally have added to the call for market integration. System integration challenges stem from the national (cost-benefit based) focus of the regulation of TSOs, the fact that interconnectors have a different juridical status from domestic networks, so congestion and investment are treated differently, and the difficulty of coordinating network development and operation with market decisions.
The second dimension is the system level. The electricity system can be divided-conventionally, but somewhat arbitrarily-into distribution networks (which are beginning to provide more than only the distribution function) and transmission networks, and in some cases also direct-current long-distance networks. Generation is increasingly present at each of these network levels; consumers have always existed at every level. Market design mirrors the network levels, with retail markets at the distribution level, wholesale markets at the transmission level and market coupling/integration at the interconnector level. Hence, at each level, a distinction can be made between networks and market parties, between the regulated and the competitive domains. With the development of flexible resources like responsive demand and storage are developing at all system levels, the need develops to align their economic incentives, whereas in most current systems, small consumers and producers (such as households with PV generation) have limited or no incentives to respond to wholesale market conditions. A second challenge is that the complicated interplay between competitive flexibility options and the networks requires careful coordination of market incentives and network regulation (Brancucci Martínez-Anido et al. 2013).
The third dimension of our framework is the dimension of time scales (not time itself ), which range from less than a second to decades. The operational performance of an electricity system (e.g. with respect to reliability, price and CO 2 emissions) is largely determined by the technical characteristics of the system, which is the result of past investment choices. Vice versa, the success of an investment in a facility like a generator depends on its short-term operational performance. Therefore, there is a strong relation between short-term and long-term decisions. The long lead time for the construction of generation and network assets, coupled with their long life spans, are reasons why an investment equilibrium is not likely to emerge until in the second part of this century, when CO 2 emissions are stabilized. As a result, there is an increasing concern that investment risk needs to be reduced in order to maintain system

FIGURE 1
The dimensions of electricity system integration challenges (arrows) and key elements of these dimensions Source: Own illustration.
Copyright © 2018 by the IAEE. All rights reserved.
adequacy. At the short-term end of the spectrum, there is a need for better integration of the short-term market cascade (day-ahead, intra-day and balancing markets) to facilitate variable renewable energy Neuhoff et al. (2013). Fig. 1 depicts our framework. The figure schematically shows the three dimensions and some of the key elements of these dimensions. f 3. SYSTEM ADEQUACY g

Introduction
The history of the debate about system adequacy is a good example of how an integrated systems perspective is gaining ground. The debate started with the question whether a liberalized electricity market provided sufficient investment incentives to maintain generation adequacy (Read et al. 1999). The debate has broadened to system adequacy: the ability of all elements of the system, including transmission, storage and demand flexibility, to maintain a continuous balance between supply and demand (Batlle et al. 2012). The focus of the debate has also shifted towards the question of how to provide sufficient controllable generation and demand resources in a system with a growing share of variable renewables.

Geographic Scope
In Europe, maintaining system adequacy has primarily been the responsibility of the individual countries. From an infrastructure point of view, this choice of system boundaries is arbitrary, as cross-border effects clearly exist (cf. Mastropietro et al. 2015;Höschle et al. 2016;Bhagwat et al. 2017). A number of recent studies therefore advocate to treat system adequacy in a European context (Cepeda et al. 2009;Newbery and Grubb 2014;Henriot and Glachant 2014;Mastropietro et al. 2015). Neuhoff et al. (2016) make an explicit distinction between the national context and the cross-border European context. As trade of electricity between countries can contribute to system adequacy, an important challenge is how to take this into consideration in the design of capacity mechanisms (cf. Pérez-Arriaga 2013). There are two aspects: trade in electricity and trade in the capacity products that are created by capacity mechanisms (such as capacity credits in a capacity market and payments for reserve capacity in case of a strategic reserve). A key question, both for exporting and importing zones, is how much trade will take place during a contingency. To what extent can an importing country count on these imports during a power shortage? Generators may be expected to sell their power in the zone with the highest price. The price that they receive may be affected by power exchange price caps and by network congestion costs. Their ability to trade may be constrained by longterm contracts or sold capacity credits.

System Level
To be efficient, capacity mechanisms should not suppress incentives for demand response and the development of storage (Aghaie 2017). Because much of their potential is at the distribution level and there is increasingly much electricity generated at this level, the traditional focus on the wholesale market should be extended to include the distribution/retail level. From a point of view of economic efficiency, they should be exposed to real-time electricity prices so as to provide them with an optimal incentive for demand elasticity, but the public acceptance may be an issue. Doorman and De Vries (2017) provide a suggestion for aligning wholesale and retail electricity prices and involving consumers directly in a capacity mechanism.

Time Scales
While system adequacy is an issue of investment, it has important relations with shortterm markets. In Rodilla and Batlle (2013), the issue of security of supply is divided along four closely related time scales: security (very-short-term), firmness (short to medium term), adequacy (long-term) and strategic expansion policy (very long-term). The need for support from a capacity mechanism depends on the expected revenues in the short-term markets, including the balancing market. Therefore, the impact of growing volumes of renewables on short-term market performance and on a capacity mechanism need to be considered together. For example, a (capacity-based) ancillary services market may generate revenues for flexible resources and thereby attract investment, reducing the need for a separate capacity mechanism for flexible generation assets. On the other hand, care should be taken that a long-term instrument such as a capacity mechanism is not used to fix a short-term issue like congestion management.

Policy Challenges
A first challenge is to coordinate capacity mechanisms in adjacent jurisdictions, as they create cross-border effects. Care must be given not to create new cross-border barriers, as market integration is an important instrument for RES integration. This requires clear agreements, e.g. with respect to honoring export contracts. Ideally, if a capacity mechanism is implemented, it is integrated across national borders and its zonal subdivision is a function of network constraints, not of political borders. The EC, ACER and ENTSO-E all call for harmonization of capacity mechanisms, see Mastropietro et al. (2015) and references in this publication.
The introduction of a capacity mechanism should not deter but rather encourage the development of demand response, energy storage, more network capacity and stronger interfaces with other sectors such as transport and heating, as all these factors can stabilize electricity prices and thereby reduce the need for a capacity mechanism. This requires that the design of a capacity mechanism is carefully adjusted to the policies for these other system-stabilizing factors (Regulatory Assistance Project 2014; BMWi 2014). It also means that system adequacy and short-term market design are interrelated.
A final challenge to policy making is to coordinate with other policy instruments that are aimed at influencing generation investment decisions, namely renewable energy support and CO 2 policies, see e.g. Mastropietro et al. (2014) and Fagiani et al. (2014). The support for renewable energy causes the business case for thermal plant to deteriorate and thereby increases the need for a capacity mechanism. CO 2 policy should only change the technology choice, but CO 2 price risk may increase investment risk and thereby also affect the (perceived) need for a capacity mechanism.

Introduction
An important challenge to system integration is the fact that liberalized electricity systems are hybrids of monopolies and markets. The networks have characteristics of natural monopolies and are regulated as such, but as they play a key role in the integration of RES, their development needs to be coordinated with investment in generation and the development of flexibility options by market parties. The regulation of electricity network investment has long been the subject of debate. A key difficulty is that marginal cost pricing does not lead Copyright © 2018 by the IAEE. All rights reserved.
to cost recovery (cf. Rosellón 2003;Brunekreeft et al. 2005;Vogelsang 2006;Wu et al. 2006;Rosellón 2007). In order to recover costs, tariffs need to be higher than marginal cost, but then the incentives for network users are not optimal and additional measures may be needed for congestion management and to stimulate efficient locational choices by generators.
Investment in transmission capacity has strong interdependencies with investments in other flexibility options like storage and curtailment, as shown, for example, by Egerer and Schill (2014) for Germany. The uncertainties that affect network planning are increasing. In the past, demand growth and the locational choices of large power producers were the main concerns of network planners. The introduction of variable renewable energy sources increases the uncertainty regarding future network flows, while they also add significantly to the demand for network capacity. Given the long life span of network components and the strong path dependence of the network topology, it is practically impossible to determine an optimal network expansion strategy. Network regulation therefore is aimed at serving the pragmatic objectives of cost recovery, reasonable tariffs, fair third-party access rules, and investment decisions that balance economic prudence with uncertain future demand for network capacity.

Geographic Scope
There is a large body of literature that shows how increasing interconnection capacity flatten the variable RES production profiles and thus reduce the need for other sources of flexibility (cf. Brancucci Martínez-Anido et al. 2013;Buijs et al. 2011;European Climate Foundation,2010;Heide et al. 2010Heide et al. , 2011MacDonald et al. 2016;Rodríguez et al. 2014;Schaber et al. 2012;Scholz et al. 2014;Steinke et al. 2013;Teusch et al. 2012;Ummels 2009). As forecast errors of variable renewable energy sources are not fully correlated between different locations, the relative forecast error of an ensemble of sites is lower than that of single sites (cf. Gibescu et al. 2009, Lorenz et al. 2009. The differences in national generation portfolios and opportunities for investment in renewable energy further add to the demand for long-distance electricity transmission. On the other hand, the asynchronous development of renewable energy in different countries may add a temporary demand that may decline when national generation portfolios become more similar in the long term. Balancing national renewable energy portfolios by expanding the capacity of the European transmission network requires increasing cooperation between TSOs. This is already occurring, especially in ENTSO-E's Ten-Year Network Development Plan cycle, but more may be necessary (ENTSO 2014). Tohidi and Hesamzadeh (2014), among others, show that proper incentives are necessary for achieving efficient cooperation between TSOs.

System Level
A very different trend is the evolution of distribution networks into a sort of local transmission networks that facilitate local electricity markets (Eurelectric 2013). Distributed generation and demand flexibility require a more active role of distribution network operators and supply companies, perhaps augmented by new roles such as aggregators of distributed generation or smart vehicle charging. The role of the DSO appears to be changing so profoundly that it needs to be reconsidered altogether (Ruester et al. 2014;Meeus and Hadush 2016). Investments are needed to make the networks 'smart' by implementing smart meters and the associated ICT infrastructure. For the TSO, the development of local generation may mean a reduction in the demand for power from the transmission network and, as the volume of Copyright © 2018 by the IAEE. All rights reserved. distributed generation increases, that electricity is fed back from the distribution to the transmission network.

Time Scales
As renewable energy policies change and improvements to Europe's CO 2 policy are being advocated (Richstein 2015;Mulder 2016), as some countries phase nuclear power out while others support it, and with the discussion in some countries whether to phase coal out, TSOs are confronted with fundamental uncertainty regarding the generation portfolio that they need to facilitate. The construction times of power stations is generally shorter than the time it needs to expand network capacity (except perhaps in case of nuclear power stations), posing a challenge to TSOs as how to fulfill their role of facilitating the market. They need to balance their role of facilitating market developments with the unfortunate fact that transmission investments tend to take much longer than generation investments, while fast changes in public policy have made the latter unpredictable. More coordinated development is one possible direction (Spyrou et al. 2017).

Policy Challenges
If network operators are to take a proactive role in providing the infrastructure that we think is necessary for a low-carbon electricity system, they will inevitably make some mistakes in the form of network investments that do not pay off. Alternatively, if a more risk-averse approach to network regulation is taken, network capacity is likely to become even more a bottleneck than it already is. In the foreseeable future, however, the risk of over investment in the transmission networks appears limited. Many studies indicate that the current network capacity is significantly less than would be optimal under the circumstances that apply currently or are expected in the within the next decade (cf. Brancucci Martínez-Anido et al. 2013;Buijs et al. 2011;European Climate Foundation 2010;Heide et al. 2010Heide et al. , 2011MacDonald et al. 2016;Rodríguez et al. 2014;Schaber et al. 2012;Scholz et al. 2014;Steinke et al. 2013;Teusch et al. 2012;Ummels 2009). On the other hand, other studies such as the one presented by Kemfert et al. (2016) show that TSOs may actually be planning too much network capacity, because the network planning process is separated from generation dispatch. As the energy system is becoming more integrated across the continent, TSOs need to coordinate their investments more closely, which requires that regulators do the same. At the same time, as distributed generation becomes more prevalent, the role of DSOs will become more active, which requires closer coordination between them and TSOs.

Introduction
The cascade of day-ahead, intra-day and balancing markets in most liberalized power systems allows market parties to adjust their trading positions as they approach real time and uncertainty decreases. Because variable RES need to be traded in short-term markets, the importance of these markets is growing and therefore the need to design them efficiently is increasing. In addition to fundamental market design choices, practical matters such as the gate closure time have a profound impact on the trading processes.
Copyright © 2018 by the IAEE. All rights reserved.

Geographic Scope
Since the start of liberalization, there has been regulatory pressure to harmonize short-term market design among European countries. The objective of improving the competitiveness of markets is now accompanied by the objective of renewable energy integration. The case for liquid, cross-border markets is clear and so is the role of facilitating them by power exchanges; the challenge is with network congestion. Power exchanges are used for cross-border congestion management, especially in areas with flow-based market coupling, and may also be used in domestic network congestion management, like in NordPool. Congestion management is discussed further in Section 6. Cross-border exchanges of balancing services is another opportunity (cf. Doorman

System Level
There is significant flexibility potential in small-scale facilities such as demand response and storage. At the same time, rooftop solar energy is emerging into an important energy source. As these resources are too small to trade in the wholesale market, a question is how they are coordinated. One approach is that intermediary companies aggregate large blocks of demand and supply and trade them the power exchange. This requires close communication between the aggregator and his customers; the customers need to indicate precisely under which conditions they are willing to adjust their production/consumption patterns and the aggregator needs to be able to influence hourly generation. To reduce transaction costs, this will need to be automated, which requires that the small-scale producers/consumers know how to indicate their preferences. A concern with this model is that suppliers of electricity may gain an effective monopoly over the balance responsibility of the demand side. If they also have their own sources of flexibility, such as conventional generation capacity, they may have an incentive to keep demand-side resources out of the market (Regulatory Assistance Project 2014).
An alternative solution is to provide real-time pricing signals to all consumers. Operators of small storage facilities, electric vehicles and other flexible load and generation can decide for themselves when they produce and/or consume. However, this decision is not simple, as it requires decision making under uncertainty over a rolling time horizon (Verzijlbergh et al. 2014). This solution may raise fewer privacy and security concerns than the aggregator scenario, because the control over the local generation/consumption facilities remains local, but the requirements with respect to (the automation of ) the consumer side are higher. A third alternative is to balance supply and demand locally, to the extent possible, but this is less economically efficient than doing it at a larger scale. If consumer tariffs are not aligned with the wholesale market, there is a risk that self-optimization, e.g. of solar PV and a battery, may not help the system at all (Green and Staffell 2017;Schill et al. 2017).

Time Scales
Because short-term markets offer a platform for dealing with generation forecast errors, the trading possibilities in the three different markets are closely linked. Gate closure times need to be long enough for the system operator to perform all necessary calculations and preparations for secure system operation, but not longer than necessary. The final outcome, after balancing, must be a complete alignment of economic transactions with physical generation and consumption. In order to be economically efficient, the markets need to be cost-re-Copyright © 2018 by the IAEE. All rights reserved.
Some market designs punish renewable energy producers excessively for forecasting errors. On the other hand, RES support schemes that remove the balancing responsibility from RES producers, such as in case of feed-in tariffs, are also undesirable. A number of ways in which RES support schemes negatively impact short-term market outcomes are discussed in Chaves-Ávila (2014). Balancing mechanisms were designed to stimulate controllable generators to match their production schedules. However, the imbalances that are caused by RES are usually the result of weather forecast errors and therefore unintentional. On the other hand, RES producers have some options for reducing imbalances, for instance by under scheduling or by developing better weather forecasts. Therefore, RES operators should have full balancing responsibility, but also be remunerated fully for balancing services that they deliver. Van der Veen and Hakvoort (2016) provide an overview of short-term market design challenges in the context of growing shares of renewable energy.
On a slightly longer time scale, there is the question whether European power exchanges should move towards complex bids or power-based trading (Morales-Espana et al. 2012. Most current European power exchanges accept only 'simple bids': blocks of electric energy for a given price. Some European power exchanges, such as the Dutch APX, allow more complex products like exclusive block bids and linked block bids, in which the acceptance of bids for certain hours is conditional on bids for another hour. Complex bids help generators place an explicit price on ramping and start and stop costs, and may allow the inclusion of ramping constraints, improving the efficiency of the dispatch. Complex bids complicate the market design significantly but may be necessary in a system with more renewable energy. Finally, some argue that there is a need for an additional trading floor for 'flexibility products' (Milligan et al. 2012), but a counter argument is that flexibility is not a product but a characteristic of generation, consumption and storage.

Policy Challenges
The way in which electricity is traded will need to change along all three of the dimensions of our framework: as more variable renewable energy is produced, and more flexibility options are developed at all system levels, trade needs to be better integrated at the same time across borders and between system levels, while trade closer to real time also needs to be facilitated. As a consequence, the role of power exchanges seems to grow. This requires increased coordination between power exchanges, TSOs and DSOs. It also raises questions with respect to how to improve the pricing of energy and networks and ways for consumers to indicate their preferences, for instance whether small prosumers can participate directly, via highly automated trading platforms, or whether aggregator companies are necessary as intermediaries. Finally, the limited liquidity of current intra-day markets is a challenge that may be addressed by improving their design in order to allow variable energy producers to trade closer to real time. These producers also need to be allowed to provide balancing (and perhaps also ramping) services and care needs to be given that demand response incentives are not suppressed by retail companies' other interests.
Copyright © 2018 by the IAEE. All rights reserved. f 6. COORDINATING MARKET CLEARING WITH CONGESTION MANAGEMENT g

Introduction
Congestion management is the instrument by which scarce network capacity is allocated to market parties. Currently, Europe makes use of zonal pricing, with most countries having only one price zone. Within price zones, a 'copper plate' is assumed, meaning that if the network is congested, this does not affect market clearing. Congestion is resolved by redispatching generators within the zone. Redispatching structurally costs the TSOs, and therefore the consumers, money and has been shown to be vulnerable to gaming (Brunekreeft et al. 2005;Baldick et al. 2011;Neuhoff et al. 2013). Between zones, market splitting, market coupling and/or explicit auctions of network capacity are applied (De Vries and Hakvoort, 2002;Meeus et al., 2009;Neuhoff et al., 2011). The fact that network constraints within price zones are not taken into account in the clearing of the markets and the fragmented nature of this approach cause it to be inefficient, with the costs increasing as the share of RES increases (Ehrenmann and Smeers 2005;Neuhoff et al. 2013;Bertsch et al. 2015). Network congestion management relates to several other policy objectives, in addition to the integration of renewables: the development of demand price elasticity and the integration of electric vehicles and system adequacy.

Geographic Scope
The definition of network constraints at the national borders (and only sometimes within countries) often does not correspond to the actual locations of congestion. This may lead to inefficient use of available network capacity. Sometimes, the determination of cross-border capacities appears to be influenced by the existence of internal congestion (because limiting import/export capacity may limit domestic flows and thereby reduce the cost of redispatching for the TSO). The only congestion management method that makes optimal use of available network capacity, at least in theory, is locational marginal pricing (also known as nodal pricing). In this system, market clearing and congestion management are fully integrated. Neuhoff et al. (2013) show that it could save significantly on the cost of generation dispatch and allow less curtailment of renewable energy in Europe, but the institutional barriers to implementation are high. It would probably require a pool-based market design, like in PJM, where it was developed, or otherwise at least very close cooperation between the TSOs and the market operators. A concern could be that TSOs that are owned by commercial energy companies would gain access through this process to commercially sensitive information from competitors.
Integrating congestion management with day-ahead, intraday and balancing markets across Europe is clearly a large and complex task with many potential technical, economic and political obstacles. However, the cost of not doing so is a continuous loss of economic efficiency that increases as the share of variable renewable energy sources increases. A compromise could be to implement smaller price zones like Norway and Sweden have done. This would reduce, but not fully remove the inefficiencies of the current system (Pérez-Arriaga and Olmos 2005;Bjørndal and Jörnsten 2007).
The cost of inefficiencies in congestion management are compounded by the national focus of renewable energy policy that is a consequence of the European Union's national targets. The consequences are particularly apparent in the case of Germany, due to its size, central location and high share of renewable electricity generation. When a surplus of wind energy in northern Germany is sold to consumers in southern Germany, a large share of this flows through neighboring countries. Such parallel flows reduce available cross-border transmission Copyright © 2018 by the IAEE. All rights reserved. capacity in and out of Germany and take up network capacity in neighboring countries. The energy producers and consumers are not confronted with the cost of these flows because there is only one price zone in Germany. In the presence of locational marginal pricing and harmonized or integrated renewable energy support, network congestion would cause more electricity to be sold to consumers close to the generators, whether there was a country border in between or not. In the example of surplus wind generation in north Germany, electricity prices in the north of the Netherlands and Poland would drop (assuming Denmark would also have a surplus of wind energy at the same time) and thermal generators would be ramped down there, instead of in the south of Germany. The claim on network capacity would be smaller. Thus, the renewable energy targets could be achieved more efficiently by a combination of a single European renewable energy target and more efficient, integrated congestion management.

System Level
It is expected that distribution networks will also become more congested, both because local renewable generation and due to flexibility options, such as demand price-elasticity and storage facilities (Biegel et al. 2012;Bach Andersen et al.;Verzijlbergh et al. 2014;Huang et al. 2014). If demand is sufficiently price-elastic, it may be more efficient to manage congestion locally than to expand network capacity (Verzijlbergh et al. 2014;Veldman and Verzijlbergh 2015). One option is to implement locational marginal pricing at the distribution level as well. This would require (near) real-time pricing for all consumers. The option to shift load, through demand response or storage, means, however, that the market needs to facilitate intertemporal arbitrage. Load shifting can reduce the need for network capacity and therefore reduce cost, but may be difficult to implement in a local market (cf. Philipsen et al. 2016).
A second option is the aggregator model, as was discussed in section 5.3. In this case, the DSO would need to allocate the available network capacity to aggregator companies who would manage their consumers within these limits. Whereas the market design might be simpler, this would create a complicated bi-level optimization problem, in which the DSO would need to decide, continuously, how much network capacity to allocate to the aggregators and they would need to manage their customers' consumption within these limits. A balance will need to be found between the feasibility in terms of IT infrastructure, transaction costs and computational complexity on the one hand and economic efficiency on the other hand. Fortunately, a lack of coordination between congestion management in distribution grids and the transmission grid appears to be less harmful than a lack of coordination between European transmission system operators (which is discussed below), because congestion in a distribution network may be handled relatively independently from the transmission network.

Time Scales
Congestion is typically handled day-ahead in Europe. Longer-term network capacity contracts of up to a year are possible in case of explicit auctions. Congestion issues in the intra-day and balancing time frames are handled by incorporating reserve margins in the congestion management mechanism in the day-ahead stage and by redispatching when these margins are exceeded. As more energy is traded closer to real time due to the growing share of renewable energy, congestion management will also need to be applied in the intra-day time frame and it will need to be integrated better with market clearing.
A second reason for improving the integration of electricity market clearing with congestion management is that this would limit opportunities for undesired strategic behavior Copyright © 2018 by the IAEE. All rights reserved. Pillay et al. 2015). A new aspect in the design of both short-term markets and congestion management is that the flexibility of load to shift over time (e.g. in the case of electric vehicle charging) and the development of storage facilities, if they become economically viable, create intertemporal constraints (Verzijlbergh et al. 2014). This means that congestion management will also need to begin to consider intertemporal constraints, probably by handling congestion for multiple time steps at once, over a rolling time horizon.

Policy Challenges
Better market integration, including harmonized congestion management, is a challenging task but could lead to substantial improvements in the usage of the networks and therefore a reduction of the need for transmission investment. The best-known method for congestion management, locational marginal pricing, however, places very high institutional demands, while at the same time it is not clear how it would incorporate intertemporal flexibility such as provided by demand response and storage. Implementing more price zones in Europemeaning multiple zones per country, as in Norway and Sweden, appears to be a pragmatic way forward, if not theoretically optimal.
Trends like distributed generation and electric vehicles may cause congestion at of distribution grids. The development of distribution systems into full-blown local electricity markets means that they should be treated as such, including integration into national electricity markets and congestion management. Fortunately, it appears that distribution grid congestion can be handled independently from transmission grid congestion, but intertemporal flexibility may prove an even larger challenge at this level. Load shifting and even curtailment of local generation may be a better option than more network capacity in some cases, but it is a challenge to design a market that provides efficient incentives. The key challenge that is posed by the introduction of variable renewable energy sources is to develop more flexibility in all parts of the electricity infrastructure. Basing ourselves on the physical dimensions along which energy system integration takes place, we present a framework for identifying regulatory challenges to the regulation the electricity sector. The underlying premise is that technical relations need to be reflected in the regulatory framework and that when current developments create changes along one of the physical dimensions that we identified-geography, system level and time scales-attention needs to be given that the regulation is adjusted accordingly. The framework therefore helps identify regulatory challenges-potential inefficiencies due to a lack of coordination -and to place them into context. The overall picture that emerges from this approach is that the institutional fragmentation of the European electricity sector will become increasingly burdensome as the development variable renewable energy requires ever closer coordination between countries, between the different levels of the electricity system and between markets that serve different time scales.
Along the geographic dimension, the main issues appear to be the improvement (and integration) of cross-border and national congestion management, the cross-border coordination of transmission investment, and the alignment of capacity mechanisms, including cross-border trade in capacity rights. Along the system dimension, the challenges to integrating local markets (with demand response, flexible EV charging and rooftop solar) with the wholesale market can be divided in aligning local energy prices with wholesale prices and managing distribution grid congestion. A novel challenge to market design is the need for intertemporal arbitrage, as in the presence of enough local flexibility, distribution grids no longer need to be dimensioned to accommodate all peak flows. Finally, electricity needs to be traded closer to real time, in order to reduce the impact of forecast errors of variable renewable energy sources. This not only means a better integration of markets with different time horizons, but also better integration of congestion management with market clearing.
The implication for research is that a shift is required from single issue, single country studies to studies with a larger scope with respect to the system levels, the geographic scope and/or the time scales that are included. This will pose a significant challenge to quantitative (modeling) studies, but not taking up this challenge would result in increasingly irrelevant research results.
In this paper we limited ourselves to the electricity sector in order to be able to make sufficiently detailed analyses. However, significantly more benefits can be gained integrating electricity more closely with other energy carriers such as space heating, natural gas and transport, because the flexibility options in the various carriers may be complementary to each other. It is our intuition that a similar approach will help structure the options in the wider context of energy system integration.
The authors thank Prof. Paulien Herder and three reviewers for their helpful comments.