Evaluation of Photovoltaic Storage Systems on Energy Markets under Uncertainty using Stochastic Dynamic Programming

The rising share of intermittent renewable energy production in energy systems increasingly pose a threat to system stability and the price level in energy markets. However, the eﬀects of renewable energy production onto electricity markets also give rise to new business opportunities. The expected increase in price diﬀerences increases the market potential for storage applications and combinations with renewable energy production. The value of storage depends critically on the operation of the storage system. In this study, we evaluate large-scale photovoltaic (PV) storage systems under uncertainty, as renewable energy production and electricity prices are fun-damentally uncertain. In comparison to households who largely consume the stored energy themselves, the major business case for large-scale PV and storage systems is arbitrage trading on the electricity markets. The operation problem is formulated as a Markov decision process (MDP). Uncertainties of renewable energy production are integrated into an electricity price model using ARIMA-type approaches and regime switching. Due to non-stationarity and heteroskedasticity of the underlying processes, an appropriate stochastic modeling procedure is developed. The MDP is solved using stochastic dynamic programming (SDP) and recombining trees (RT) to reduce complexity taking into account the dif-ferent time scales in which decisions have to be taken. We evaluate the solution of the SDP problem against Monte Carlo simulations with perfect foresight and against a storage dispatch heuristic. The program is applied to the German electricity and reserve power market to show the potential increase in storage value with higher price spreads, and evaluate a possible imposition of the feed-in levy onto energy directly stored from the common grid.


Introduction
Electricity systems and markets are increasingly challenged by uncertain production due to new technologies exploiting intermittent resources, such as wind or solar energy. Electricity generation by photovoltaic (PV) installations and wind power plants depends on the availability of the related resources. A lag of several hours or even days may occur between supply from these sources and peak demand. In this case, the demand is covered by gas or (pumpedstorage) hydropower plants, which can react easily to load variations and to variable feed-in of wind or solar electricity into the grid.
Other options to bridge the gaps of volatile supply are electrical energy storage technologies that can be combined with PV or wind power plants at a single site or virtually, so that the combined PV/wind storage system can deliver energy more smoothly. Wind storage systems are evaluated in several studies (Athertona et al., 2017;Keles, 2013). As PV storage systems can effectively contribute to the successful integration of renewable energy sources (RES) electricity production, balancing also day-night fluctuations, it is important to assess their market value and to develop financing schemes in a future electricity market design for this type of energy technology.
The operation of the components of a PV storage system can also be done separately. However, regulations exist in today's energy markets that make a joint operation and optimization reasonable (see Section 6). In future, more regulations are expected to be implemented, to support joint investments in renewables and storage, as they can help to eliminate network congestion by smoothing intermittent generation from RES. It is therefore relevant to develop and apply methods that takes into account positive portfolio effects of a PV and storage plant and that jointly optimizes the operation of PV and storage on energy markets.
Existing studies evaluating PV storage systems under uncertainty focus on their dispatch and profitability on the electricity spot market (see Section 2), but rarely on additional earnings that can be generated on the reserve power market. It can be also noted that if uncertainty is accounted for in stochastic optimization models for the operation of storage systems in energy markets, the appropriate stochastic process of uncertain parameters hardly receives the attention required. However, an appropriate consideration of uncertainties is essential to carry out robust evaluations (Zheng et al., 2015;Davison et al., 2002). Moreover, to the best knowledge of the authors, there is no study that examines the role of network charges for creating portfolio effects for PV storage systems. These charges have to be paid by energy storage plants if they are dispatched at the spot market and if they are not considered as network operation components.
Based on these points, the main contribution of this study can be summarized as follows: • A method is developed to incorporate high-dimensional data for PV power production and electricity prices, generated with advanced stochastic processes, into a stochastic optimization problem of PV storage investment evaluation.
• We describe a joint optimization approach for the dispatch of PV storage on both spot and reserve power markets considering electricity price and PV production uncertainties.
• We discuss, for the first time, the role of network charges for the creation of a portfolio effect that can support the co-location of PV storage plants and their system-friendly operation.
We first provide detailed modeling approaches for the stochastic parameters (electricity prices and renewable energy generation). Thereby, we take crosscorrelations between renewable energy production and electricity price into account. This is very important for the evaluation of the value of renewable power production due to the concurrency of the plant under evaluation and all the renewable plants selling electricity at the same time.
More precisely, we introduce a comprehensive approach starting with the detailed modeling of PV and wind power generation and wholesale electricity prices as well as their cross-correlations. Then we continue with a scenario reduction technique to generate a recombining tree for a stochastic dynamic program (SDP) that solves a Markov decision process (MDP) for the optimal operation and economic evaluation of investments in PV storage systems. The proposed scenario reduction method simultaneously considers multiple uncertain parameters (electricity prices and renewable energy feed-in) as well as the high-dimensionality of these parameters, as the action-space of the decision problem covers the 24 hours of the following day. This significantly increases not only the complexity of the scenario reduction technique, but also of the applied SDP that, in contrast to most applications in the literature, has to cover the high-dimensional action space of the MDP. Furthermore, to keep the problem tractable, we discretize the storage states to a predefined number of storage levels at the beginning of each day and solve a discrete approximation of the optimal SDP.
We model a long period of a whole year for the optimal operation plan by maximizing the annual return, as our objective is not only to derive an optimal operation strategy, but also to evaluate investments in PV storage based on the calculated annual returns. Figure 1 summarizes the applied overall methodology.
For this purpose, the remainder of the paper is structured as follows: After a brief literature review (Section 2), we formulate the optimal operation of a PV storage system as a Markov-decision process (MDP) with the objective to maximize the annual return in Section 3. Thereby, the optimal operation of an energy storage considers the real option to delay the dispatch and to use the stored energy for electricity production in times of scarcity and high prices. Furthermore, the problem formulation focuses on decisions that are to be made regarding operating the PV storage on the spot or reserve power market. The MDP is then approximated applying a SDP with recombining trees.
Before the SDP solution is presented, the modeling approach of the main uncertainties, the volatile PV and wind power generation as well as electricity spot prices, is described in detail (Section 4). The combined modeling approach consists of extended time-series models (ARMAX) that are developed to generate electricity price series considering the impact of PV and wind power generation on prices. The price model is a regime switching model based on a seasonal ARMA process. The procedure takes the stochastic properties of the time series such as non-stationarity and heteroskedasticity into account. Thus, a valid stochastic model is provided. The data used to calibrate the time-series model is derived from EPEX Spot and the European Energy Exchange (EEX).
The time-series models are applied to generate a large number of price and PV output series which then are reduced to a recombining scenario tree as a basis for the SDP model that optimizes the dispatch of PV storage systems (Section 5) in the spot and reserve power market.  Figure 1: Graphical summary of this study.
In Section 6, a case study is carried out to evaluate a PV storage investment based on maximum annual returns which result from the optimal operation of the system under price and PV power generation uncertainty. The storage dispatch is optimized and annual returns are maximized under imperfect foresight on prices and PV feed-in during the year. Finally, the economic value of the PV storage system is analyzed for a future year with possible higher fluctuations in prices. Main conclusions are drawn and possible directions of future research are indicated in Section 7.

Literature Review
In the recent literature, residential PV storage systems are often evaluated based on maximizing self-consumption of generated PV electricity (Quoilin et al., 2016;Luthander et al., 2016;Vieira et al., 2017), investigating optimal investment and operation plans for residential energy systems (Lauinger et al., 2016), and comparing grid-connected with off-grid solutions (Sandwell et al., 2016).
Besides the evaluations based on household use of storage, other studies investigate PV and large-scale storage or aggregated small-scale storage systems and the marketing of their capacity as well as their energy on wholesale markets (Zucker and Hinchliffe, 2014;Sioshansi et al., 2009;Aguado et al., 2009;Muche, 2014). Most of these studies examine the value of storage based on deterministic parameters. They follow an approach with pre-known prices. The uncertainty in future electricity prices and renewable energy generation is not considered in all these studies.
However, studies in the storage literature take uncertain prices or RES electricity production into account and develop different stochastic optimisation models to address these uncertainties in operational planning of storage plants. First studies that consider uncertainties for storage operation and evaluation were already developed for hydropower storages in the 1990ies and earlier. Pereira and Pinto (1991) develop a multistage stochastic model for the planning of several hydro reservoirs and apply stochastic dual dynamic programming (SDDP) with Bender's decomposition in each stage. A piecewise linearisation is used to avoid discretization in the optimisation problem. There are also other studies that focus on the solution algorithm for a two-stage (Fleten and Kristoffersen, 2007) or multi-stage (Flach et al., 2010;Séguin et al., 2017) problem of optimal short-term hydropower planning. However, the appropriate modeling of uncertain parameters, such as prices and inflow to reservoirs, are not addressed in detail. One of the few studies that contains a more detailed mathematical description of uncertain parameters is Löhndorf et al. (2013). They apply an econometric approach to model electricity prices using system load, wind and solar generation as regressors. Furthermore, they provide a model for the short and mid-term operational planning, decomposing the problem to inter-stage and daily intra-stage subproblems, similar to Pritchard et al. (2005).
Besides the broad literature on hydro storage, there are also first studies focusing on battery storage. Bakke et al. (2016) evaluate the profitability of battery storages under uncertainty using Monte Carlo simulations in a real options approach considering spot and ancillary services markets. Kim and Powell (2011) formulate the storage problem as a Markov decision process and use a simple autoregressive process to model the power generation of a wind farm. Sioshansi et al. (2014) calculate the capacity value of storage using a dynamic program and focus on power system outages with loss of load probabilities. The stochastic state of the system is determined rather by the outage probability than by the renewable generation of the system. Zhou et al. (2016) analytically characterize the optimal policies of the storage problem and numerically solve a discrete-state version of the model by standard backwards dynamic programming. Gönsch and Hassler (2016) develop an approximate dynamic programming approach with an analytical derivation of the optimal policy approximating the value function and combining it with classical backward induction. They describe the wholesale prices, renewable energy generation, and the "penalty price" for purchased balancing power as stochastic parameters by means of an autoregressive process AR(1).
A comprehensive review of the literature on electrical energy storage is provided by Weitzel and Glock (2017). A very useful thematic classification of the studies is provided: The perspective of study may be classified by the system scope (storage-only, see Densing (2013) and Steffen and Weber (2016), or combined plants, see Kou et al. (2015) and Motevasel et al. (2013) etc.), or the time horizon: day-ahead (Sioshansi et al., 2014) or intraday trading (Wu et al., 2014)). A methodological classification based on the problem formulation and the applied solution technique is undertaken as well: Fuzzy control, least-squares Monte-Carlo, meta heuristics, or stochastic dynamic programming (SDP), etc.
Generally, it can be noted that the studies incorporating uncertainties mainly focus on the solution approach of the stochastic optimization problem, but, except a few (e.g. Löhndorf et al. (2013)) less on the description of the stochastic processes for uncertainties and their incorporation into the optimization problem of uncertain parameters. Although the broad literature on hydropower provides insights on how to capture the stochasticity of electricity prices and how to apply this to multi-stage optimization problems of storage operation, there is still a need for more detailed mathematical description of solar and wind power generation time-series if the focus is changed from the hydrostorage to wind or PV storage systems. And as the stochastics of PV and wind becomes apparent already in the short and mid term -in contrast to hydro inflows, where the long-term uncertainty counts -the problem formulation has to be adjusted accordingly. Although there are accurate day-ahead forecasting models for PV and wind power, where the mean relative error is below 5% for machine learning based approaches (Atsushi Yona et al., 2008), for the subsequent days and weeks (i.e. in the mid-term) the weather forecast and thus the wind power and PV forecasts become quite uncertain (Foley et al., 2012). For this reason, the problem formulation for the dispatch of PV storage systems needs to take this mid-term uncertainty into account.
Furthermore, the stochastic state of the system and the properties of the time series such as autocorrelation, non-stationarity or heteroscedasticity must be taken into account in the evaluation of PV storages under uncertainty, since they may introduce large bias to model results (Granger and Newbold, 1974). Hence, a comprehensive approach including an appropriate description of the non-stationary processes is developed to evaluate profitability PV-storage systems under uncertainty applying SDP. Furthermore, we compare the SDP solution to a heuristic approach which is based on the same stochastic tree as the SDP approach. We demonstrate hereby the performance of the SDP approach compared to heuristic approaches applied to the evaluation of PV storage systems.

Storage Problem Formulation as a Markov Decision Process
To evaluate an investment in a PV storage system, we start from the perspective of a price-taking power producer operating a renewable-energy plant (for example, producing solar power) and a battery storage system maximizing its profit. We assume the producer has incentives for the joint operation of PV storage systems, as the combination of both technologies bears advantages in firming up otherwise intermittent power production (IRENA, 2019). The power producer is licensed to sell energy on the day-ahead electricity market and on the reserve power market 1 . In both markets, the commitment is decided day ahead. The investment perspective entails the necessity to analyze the longterm profitability of the PV storage system. Additionally, day-ahead electricity prices and renewable production can be forecasted well (Ziel et al., 2015). We thus assume that the power producer neglects the intra-day uncertainties of a possible deviation of the price and renewable forecasts from its realization. Instead, the producer investigates the day-by-day uncertainties arising from the difficulties of mid and long-term forecasts of renewable energy production and prices. While this simplification implies a deviation from the real world, it keeps calculations of the problem with long time horizons tractable.
On every day d ≤ D, the power producer decides upon the commitment X spot d = (X spot d,h ) 1≤h≤24 of the storage facility on the day-ahead market, and on the reserve power market X res d . At the same time, it decides on the energy bought from the day-ahead market X stor d within the next 24 hours and the energy from renewable production (i.e. solar production) that is sold directly R spot d and/or stored R stor d . These operations decide upon the storage level L d . For convenience, all the variables in the following are denoted as 24 × 1 vectors reflecting the structure of the decision problem, and define the decision matrix The producer is thus tapping three different income streams: first, the solar production is marketed directly on the day-ahead market or stored to being sold later; second, the storage capacity can be sold as reserve power; third, the storage is operated in the day-ahead market for arbitrage trading. The last two options may be particularly interesting for combining battery storage with solar power production, as long periods of the day are characterized by the absence of power production and the storage can be used otherwise. The producer makes its decision based on the exogenous power price p d and the power production of the renewable power plant r d 2 that lie out of the control of the producer. We assume that the plant operator has decided on a strategy for reserve power capacity bids p res d , based, for example, on historical prices.
For the sake of simplicity, we do not model reserve energy activation in detail. This is mainly for two reasons. First, data availability constraints for reserve activation forbids the empirical fitting of dedicated stochastic processes that are needed for our approach. Second, this would require modelling intra-day uncertainties, that increase model complexity (e.g., Löhndorf et al., 2013).
Instead, we compare the opportunity costs of selling the electricity on the spot market against the price that can be obtained for blocking capacity for the reserve. The resulting expected value is a conservative estimate of the storage value with regard to reserve energy. On the other hand side, this way we can include the capacity prices that can be collected from reserve markets.
We describe the operation problem as a Markov decision process (MDP) in accordance with the literature (Gönsch and Hassler, 2016;Löhndorf et al., 2013). We define the state of the system first, describe then how the storage level is updated, and finally describe the restrictions that might be imposed by the operation in a reserve power market on the example of the German tertiary reserve. Subsequently, the contribution function is introduced and the objective function of the problem is formulated.
To avoid a potential source of confusion, we briefly introduce the different indices that describe the different layers of the time structure. This is critical to fully understand the problem setting and the time series methodology proposed in this paper. The time structure is imposed by the structure of bids on the considered electricity markets and the different natural behavior of the time series of solar and wind production. The smallest unit referred to in this paper is one hour. The total length of a time series is referred to as T and indexed as t ≤ T . Time series comprise of a number of days d ≤ D. As the regulation of the German tertiary reserve power market demands the availability of the offered capacity for a certain amount of hours (e.g. 4 hours), we divide each day into time slices (ts ∈ T S, e.g. 6 time slices per day) comprising of successive hours (compare Figure 2). Lastly, each day d is subdivided into 24 hours h. The chosen time structure is adapted to the different markets under consideration, but can easily be applied to other use cases such as the intra-day market or other reserve power constraints.

State of the system
The state of the system is described by the matrix comprising of the external power prices and renewable energy production and the storage level on the previous day. The state forms the base for decisions taken on day d ( Figure 2). The state transits from d to d + 1 with is the storage level on day d described below. The exogenous processes update according to the models described in Section 4, where wind production is denoted as w d and solar production as v d which take the place of r d in the application. The transition probabilities to the next state S d+1 can be calculated according to the distributions of the exogenous processes.

Storage level update
The storage levels on day d are given by the following relationship: where µ st ≤ 1 describes the efficiency of the battery. Due to the time structure of the commitment problem, the storage level has two recursive elements. The inter-day relationship is maintained by considering the last storage level of the preceding day (case h = 1, compare Figure 2). The intra-day relationship between hours on the same day is considered by including the storage level L d,h−1 of the preceding hour (case h > 1). The division into inter-day and intra-day problems is established in the literature as well (Löhndorf et al., 2013;Pritchard et al., 2005). The storage level cannot exceed a maximum level, and the energy produced by the renewable energy plant must be split into energy directly sold on the day-ahead market and the energy stored in the battery. Thus, the following restrictions must hold for upper and lower limits L min/max ≥ 0 on the storage level:

Reserve power market restrictions
The compliance with the reserve power market requirements imposes restrictions on the operation of the system. We assume that the operator withholds a certain amount of capacity to be used on the reserve power market. A fraction of the capacity blocked is requested by grid operators and lies beyond the control of the plant operator, as it is dispatched by grid system operators. This amount is blocked by the first constraint, if opportunity costs of reserve capacity are higher than day-ahead market operation. Some regulations demand that reserve power is available for a certain amount of hours i (for example, the German tertiary reserve must be guaranteed over a period of 4 hours). This is reflected in the first two restrictions, which guarantee the availability of a sufficient amount of energy to serve a possible 4 hours request or empty storage in the case of a negative reserve power (Regelleistung.net, 2017). As on the reserve power market, positive and negative reserve power are traded, which are to balance deviations in both directions; we distinguish between X res,pos d,ts and X res,neg d,ts describing offers for both trading options. Furthermore, we introduce a second time structure T S of 24/i time slices ts comprising of several consecutive hours (compare figure 2). The second set of restrictions ensures that the charging/discharging capacity X stor max and X discharge max of the battery is not exceeded by simultaneous operation in both markets.

Contribution function / revenue
The contribution function (4) includes profits originating from storage dispatch (p d − c stor ) · X spot d (where the dot indicates the standard scalar product) and from PV power sold on the spot market including a market premium 3 based on the RES support. c disch and c stor are constant variable costs of the storage operation. Terms representing the daily returns of the renewable plant are added to the contribution function. The term (p d + m d ) · R spot d includes the renewable power sold directly on the spot market on day d, priced with the current spot price p d and the corresponding market premium m d on day d, interpreted as a constant 24 × 1 vector. If some part of the renewable power generation is stored, the amount R stor d of stored energy is valued with the market premium m d , while the earnings from the market for this amount of energy occur when the energy is again released from the storage and sold on the spot market, or the storage volume is used for offering reserve power.
On every day d, the renewable power plant and the battery storage con-tribute to the revenue of the producer: . (4)

Objective function and Bellman equation
The objective of the power plant operator is to maximize the profit over the entire time horizon. Thus, the expectation of the sum of every day's contributions is maximized similar to Gönsch and Hassler (2016) where the maximum is taken over all possible admissible policies π ∈ Π that are constrained as explained above.
In order to solve the problem as a stochastic dynamic program, we formulate the Bellman equations, or, to stay in the terms of MDP, the value functions recursively: The starting storage levels on the first day can be chosen freely. For being sure that the earnings of the storage result from the inner-year operation, we fix L 0 = L D .

Modeling Volatile Electricity Prices and PV Power Generation
As the storage evaluation problem has to be solved taking into account the main uncertainties -in this case electricity prices as well as wind and PV power generation -the characteristics and processes of these parameters have to be modeled in detail. A large number of solar feed-in and corresponding price series as input data are required to capture the stochastic distribution within the solution approach appropriately, i.e. in the SDP model. As a major uncertainty of electricity prices also stems from wind power production, a dedicated model is developed to produce stochastic series of wind power generation. Additional wind power production depresses the power price, so for future scenarios with a high installed capacity of wind turbines the consideration of wind power in the price models enables a more realistic evaluation of a storage plant (compare Section 6). Wind and solar influences are included in the model of power prices. Contrary to the increasing stream of literature on stochastic modeling of wind and solar power, the approach used in this paper is based on power production directly without the detour of modeling wind speeds or solar irradiation (Lei et al., 2009;Jung and Broadwater, 2014). This circumvents the issue of deriving the overall power production from geographically disaggregated wind speed or solar irradiation. We use an extended autoregressive approach to both renewable energy sources, as the autocorrelation is of high relevance to the evaluation of storage technologies (Feijóo and Villanueva, 2016). In the case of power prices, we include a Markov regime switching model for higher and lower price regimes (Möst and Keles, 2010). We specifically take great care to account for the nonstationarity and heteroskedasticity of the time series, since, as we have argued above, they may introduce great bias into price models and, ultimately, into the final evaluation of PV storage systems. These considerations lead to several transformation steps, the elimination of deterministic trends (such as seasons), and the division of the data input into different subsets. The approach to model power production and prices is related to the models in Keles et al. (2012a); Keles (2013); Wagner (2012).
The seasonal and statistical properties of the time series make it necessary to divide the series into different subsets or to restructure the historical data according to the hour of the day. This is reflected in a change of indexing for clarity in the modeling process.

Stochastic modeling of renewable energy feed-in
To model solar power feed-in (SPF) time series, we use autoregressive time series models (AR(p)). In order to achieve a AR(p) process from historical SPF data, several steps are necessary. First, we assume that the stochasticity of solar energy production is largely contained in the time series of the daily maximum production (Wagner (2012) argues along the same lines). After normalization and transformation a seasonal correction is applied as sunshine intensity varies greatly throughout a year. After those steps, we can fit a AR model to the transformed data which can be used for simulation purposes. Re-transformation of the simulated time series yields stochastic processes following the autoregressive, seasonal, and distributional patterns of historical SPF. To match the characteristic daily pattern of SPF, historic monthly averages of every hour are used. In the following, we describe the time series analysis process more formally. We use lower case letters for the stochastic processes to stay consistent with the notation of the parameters of the problem setting above.
In the first step, the SPF time series (v t ) t≤T with hourly resolution is normalized with the yearly installed capacity C v y to make the time series independent of the installed capacities. Let d ≤ D denote the day of production and the total number of days in the time series, respectively. The daily maximum of SPF is denoted as The seasonal yearly cycle is removed by subtracting the trigonometric function where α 1 , . . . , α 7 are fitted to the time series v max d . Thus, the deseasonalized maximum time series is defined bŷ To normalize the time series we transform the deseasonalized time series using the logit transformation similar to Wagner (2012). An autoregressive model is estimated to model the maximum process: where β v i are estimated with least squares and ε d denotes the residuals. Visual inspection of the residual's normal probability plot shows that they are approximately normally distributed. The stochastic process of maximum production is obtained by simulation of the autoregressive model with normally distributed residuals. The seasonal component is added again and the time series is retransformed and again denoted as v max d for d ≤ D.
To model the daily patterns depending on the season, monthly m ≤ 12 averages of SPF are calculated for every hour of the day h ≤ 24: The daily pattern is then multiplied with the re-transformed maximum process to produce stochastic SPF time series where denotes the ceiling operator. The retransformed series for the solar feed-in consider the historical values with a lag up to 24 hours. More precisely, the historical information of the daily cycle and the AR(1) process for the daily maximums carry the historical information of the last 24 hours.
Beside the SPF series, we need to model and generate wind power feed-in (WPF) series, as wind power is an important price driver (see Bublitz et al. (2017)) that has a significant impact on electricity prices. During hours with high wind energy production, prices usually drop. As a consequence, wind energy production becomes vital for the revenue of electricity storage, even though the storage facility may not be directly connected to a wind power plant. The stochastic model of electricity prices (see Section 4.2) needs a large number of WPF series as input variable; we explain the WPF model in the following.
For modeling WPF series, we proceed similarly to the SPF approach described above. Again, we use autoregressive processes to account for autocorrelation. The yearly historical series of the wind power production (w t ) t≤T with hourly resolution (T = 8760) is normalized with the available yearly capacity C w y . After deseasonalization deseas w () accounting for the 10% and 90% quantiles (Keles, 2013), the procesŝ is obtained. The first-order differences are observed to follow a Laplace distribution. Furthermore, the distribution of the differences varies with the height of production in the previous hours. For these reasons, the first-order differences are modeled as a random variable with a Laplace distribution, where diversity and mean are determined dynamically depending on the moving average of the previous hours and historical data (see Keles (2013)). The simulated process is then obtained asŵ Finally, the removed seasonal components are added to the stochastic component, and the resulting series of wind power utilization rates is then multiplied with the overall available wind capacity C w y of the simulated year to obtain final simulated WPF series w sim t for the analyzed year.

Stochastic modeling of electricity prices with merit-order effect
Different methods can be applied for the stochastic modeling of electricity prices. Besides mean-reversion processes (or Ornstein-Uhlenbeck processes, see Uhlenbeck and Ornstein (1930)), autoregressive-moving average (ARMA) models can be applied. We have opted for an ARMA model, combined with regime-switching elements (Weron, 2009) since the autoregressive structure of consecutive hours is important to the evaluation of short-term electricity storage technologies such as batteries.
In this study, the hourly wind and solar time series are used for determining the merit order effect of the wind and solar energy with an additional model for electricity prices. The electricity price model is an extension of the model described in Keles et al. (2012a) and Keles et al. (2012b). Historical electricity price series, wind, and solar power feed-in are used to fit the model parameters and to render the simulated time series as realistic as possible. The model is summarized in Figure 3. A major improvement in comparison to other electricity price models for stochastic programming (Gönsch and Hassler, 2016), the effect of renewable energies onto the electricity price is included. As the electricity production from renewable energies is expected to rise in the coming years, the proposed method brings along another benefit: Investments in the years to come can be evaluated, taking into account the higher share of renewable electricity and its price impact. The price impact stems from the fact that renewable electricity can be produced with vanishing marginal prices, thus decreasing electricity prices on the day-ahead market (the so-called merit-order effect, e.g. (Sensfuß et al., 2008)). Thus, a higher share of renewable electricity production will have an effect on the profitability of storage technologies. Initially, the merit order effect of wind and solar energy production on historical prices is eliminated by fitting linear models. To ensure stationarity, we keep the season and the hour of the day fixed while fitting models over all days. However, for clarity in notation, we do not introduce separate indices for season or day. Keep in mind that this procedure results in 24 × 4 distinct models. Subsequently, the residuals of this first step are free of the deterministic effect of wind and solar power production and are considered a stochastic process itself. The electricity price p t is modeled in dependence on the wind power feed-in w t and solar power feed-in v t fitting least squares regression with coefficients β i and residualsp t free of the influence of wind and solar: Parameter and residual estimates are obtained and used later to include stochastic renewable production. Additionally, the long-term trend and further deterministic components, such as daily or weekly cycles are removed from the residualsp t . The methods for the consideration of deterministic components are based on the observation of the typical characteristics of electricity prices, i.e. that they have a trend, as well as daily, weekly and annual cycles. For example, the weekly cycle is approximated with trigonometric functions such as deseas S (see section 4.1). The daily cycle is described by the hourly mean values of the electricity price during one day. This cycle is determined separately for each season and for two day types (weekday and weekend day). After describing all deterministic components, they are removed from the residualsp t as developed in Keles et al. (2012a). A deseasonalized process with the influence of wind and solar feed-in removed is obtained. Additionally, electricity prices show jumps and high spikes in very short time horizons (Weron, 2009). Due to the electricity price mechanism and the structure of the power production portfolio (merit order), price jumps do not depend linearly on the electricity demand or past realizations of the price process. Therefore, price spikes cannot be modeled using a simple ARMA process. Nonetheless, they are very important to the economic evaluation of energy storage technologies: The capability to store energy over time and to dispatch energy at any point in time enables particularly storage to gain profits in the electricity market. In order to include price spikes adequately in the simulation, a regime switching approach (compare Weron (2009)) is chosen to enhance the ability of the stochastic price model to simulate prices realistically. Price spikes in empirical data are defined as prices above or below a certain threshold. For the underlying study, µ ± 2σ were chosen as the upper and lower limits of the base regime and the different regimes were defined as follows: From the empirical data, regime switching (RS) probabilities of the switch between three regimes ρ i were elicited based on the relative frequency of price spikes in the transformed historical time seriesp t . For the determination of the regime switching probabilities, only two seasons were separated: One period comprising of the months from October to March, the other comprising of the spring and summer months. Additionally, it is assumed that upward jumps only occur between 8am and 8pm, while downward jumps only occur in the remaining night hours. For the simulation, the additional magnitude of the price spikes is drawn as a normal distributed random variable with mean zero and standard deviation according to the historical realizations of the regimes (denoted by σ 2± ): We define a random variable ϕ(p t ) ∈ {−1, 0, 1} that switches states according to the regime switching probabilities pr RS i,j defined on the set of all (preprocessed) observations and to the regimep t belongs to.
The base regime is simulated with an ARMA(p,q) process fitted to the deseasonalized process (p t ) t≤T , where values below and above the thresholds are replaced by the long-term mean to obtain a series without price jumps: where ε t are the residuals, γ i and δ j are estimated with least squares. To simulate a price process, it is assumed that the residuals ε t are independent and normally distributed. Drawing the residuals accordingly, simulated time series for the base regime are obtained. The lower and upper regime are drawn with their respective distributions and added according to the transition probabilities Using the same functions as before, the simulated time series (p sim t ) t≤T are reseasonalized (daily, weekly and yearly cycles) and result in simulated residual processes. The final price simulation (p sim t ) t≤T is obtained by adding the effect of wind and solar feed-in to the simulated residuals from (14): Finally, we choose an AR(5) autoregressive process for electricity prices, which means that the prices of the last 5 hours is used for the price simulation at a specific time t. The chosen order of 5 has revealed very good results for the ARMA model of electricity prices in previous analyses (Keles et al., 2012a). Furthermore, as daily and weekly cycles are determined and added as deterministic components to the price simulation, the price information 24 hours and 168 hours ahead also influences the simulated price at time t.

Solving the Markov Decision Problem Using SDP
The formulation of the storage operation for a whole year as a stochastic optimization problem would result in a multi-stage program with D=365 stages. The complete enumeration of all branches of the related stochastic tree is an np-hard problem (with K 365 solutions with K = number of nodes/number of clusters, see below). Although there are promising scenario reduction techniques, the problem is still extremely complex. Dupacová et al. (2005) found out that reducing electrical load scenario trees, which are similar to those of electricity prices, by 50% achieves still 90% accuracy. However, a 50% tree reduction remains np-hard and the stochastic problem of 365 stages is not solvable as a closed optimization within an acceptable computation time or even generally.
Instead of closed optimization, we make use of dynamic programming, in this case stochastic dynamic programming (SDP). To reduce the complexity of the SDP, the initial large number of price and PV feed-in series has to again be reduced to a representative tree with a small number of price and PV feed-in price clusters (nodes) at each stage. Furthermore, we solve an approximated version of the original problem, where we start fixing the storage levels at the beginning of each day to N discrete states, following the scenario lattice approach in Löhndorf et al. (2013) or known as Markov Chain in Gjelsvik et al. (2010) and Vector Quantization Tree in Bonnans et al. (2012). The reduction to N states is reasonable due to the nature of the decision problem of bidding on the spot or reserve power market with time slices of four hours. For instance, the discretization into five states displays quite well the hourly possible actions and related storage states within four hours in which the storage could be run from state "full", "three quarterly full", "half full", "quarterly full" to "empty". Generally, for realized battery projects, storage volumes are chosen as a small multiple of the charging capacity, which in turn permits the operation of the storage with a few pre-defined states. The small number of states keeps the computation time of the stochastic dynamic program (SDP) acceptable, and we do not require further approximation techniques, such as approximate dynamic programming (ADP).

Scenario tree for strategies under uncertainty
To apply stochastic dynamic programming to the economic evaluation of a renewable storage system, we first generate a stochastic tree. A stochastic tree consists of nodes representing different possible states of the external variables power price and renewable energy production, and transition probabilities between the different states (see Figure 4). Therefore, a large number SC ∈ N of price and RPF series generated with the price and renewable models are reduced to a recombining stochastic tree (Weber, 2005). It is important that in case of a PV storage system the RPF series (r t ) t≤T equals the modeled SPF series (v sim t ) t≤T (see Section 4). Given a RPF series (r t ) t≤T and a price series (p t ) t≤T we call the price-RPF-tuples (p t , r t ) t≤T a scenario. Given a number SC of scenarios we indicate with (p sc t , r sc t ) t≤T , sc ≤ SC that the scenario is indeed part of a collection of scenarios.
The first step of tree generation is that each of the price and RPF scenarios are standardized by their respective meanp sc ,r sc and standard deviations σ p sc , σ r sc .p sc t = p sc t −p sc σ p sc ,r sc t = r sc t −r sc σ r sc .
The standardized series are combined to a new series of price-RPF-tuples z sc t = (p sc t ,r sc t ). Please note that this step is only an auxiliary one. The standardization is needed to successfully apply the scenario reduction algorithm to parameters at the same scale/interval. Originally, the two series (electricity prices and renewable power feed-in RPF) are at different measurement levels and they are standardized to give both series/uncertainties the same weight in the reduction algorithm. However, for the application in the SDP program, the reduced tree and the standardized values of the stochastic parameters are transformed back to original levels.
The scenario tree generation must be adapted to the specific time structure of the day-ahead market. While all the uncertain time series have an hourly resolution, decisions must be taken for a whole day in advance. The nodes thus represent days consisting of 24 hours of the respective time series. Thus, the series (z sc t ) t≤T is divided into D sections z sc d , each representing price and RPF series for a day d.
All the different sections (z sc d ) sc≤SC d≤D are converted to a matrix. Its first dimension stands for the SC scenarios, the second for the D days, the third for the 48 values of hourly prices and RPFs. The k-means algorithm with "City-Block-distance" is found to be an efficient clustering method 4 and is thus applied to the matrix which reduces then the SC series to generate a scenario tree (MacQueen, 1967).
Choosing a fix number of clusters K ∈ N, the resulting tree is described by the clusters (Z d,i ) i≤K for each day d. Each cluster is represented by its centroid 5 .
Besides the clusters of scenarios, transition probabilities between the cluster i on day d and j on day d + 1 are necessary to generate the recombining tree. These transition probabilities are calculated based on the number of transitions between scenario states z sc d clustered in Z d,i on day d and scenario states z sc d+1 4 Initial tests showed that the City-Block distance allows in case of electricity prices more diverse clusters than the Euclidean distance does. This can be explained by the few price jumps distort the price distribution.
5 The centroid is defined as the price/RPF section on day d that has the smallest sum of distance from the other price/RPF sections, which are grouped into the same cluster. clustered in Z d+1,j on day d + 1 (Felix and Weber, 2012). The number of these transitions is divided by the total number of transitions from cluster i to all clusters on day d + 1 to receive the cluster transition probability pr CT d,i,j .
For the first step, we need the probabilities pr CT i of the price clusters on the first day. They are calculated as the ratio between scenarios matched to the cluster Z 1,i and the total number of price scenarios:

Discretization of the general problem
In order to solve the problem (5) we have formulated the Bellmann equations in Section 3 and repeat them here for the reader's convenience: with L 0 = L D and the constraints formulated above. In the following we will describe the procedure to estimate solutions. For MDPs, several efficient estimation procedures are available such as least squares Monte Carlo (Nadarajah et al., 2017) or approximate stochastic dual programming (Löhndorf et al., 2013). However, as the primary goal of this paper is the detailed model formulation and the analysis of results, in the following we limit ourselves to approximating the MDP using classical backwards stochastic dynamic programming. In pursuit of the approximation of the optimization problem, we define the discrete state of the system in the i − th cluster on day d given the fixed storage level sb ∈ {0, 1 . . . , N } in the last hour of the preceding day S i,sb d := (p d,i , p res d , r d,i , sb) ∈ R 24 × R 4 . The storage update L i d is defined accordingly. We approximate the expectation in equation (6) with the expected value over discrete states (Puterman, 1994). This is done using the discrete storage levels sb and a linear interpolation between the discrete levels. The storage levels sb = 0, 1, 2, .., N serve to simplify the model order. While in the general model formulation, the storage level L d,1 is chosen within its constraints in R, we now fix the starting storage level L d,1 with the help of storage level sb to N fixed conditions. For example, if N = 4, sb indicates whether the storage is empty (sb = 0), quarter full (sb = 1), half full (sb = 2), three quarters full (sb = 3), or completely filled (sb = 4) at the beginning of a day. The end storage level L d,24 is interpolated linearly between the fixed storage levels sb, inducing new decision variables λ d to be constrained appropriately later on (see (23)). Fixing the storage levels in the first hour of the day makes the problem tractable.
Given a set of discrete storage levels, and K price clusters on each day, the problem (6) can be approximated by where (X j,sb d+1 , λ j,sb d+1 ) = argmax X,λ B d+1,j (S j,sb d+1 , X, λ), subject to essentially the same constraints we have already defined complemented with an alternation of the storage level update (1) in order to keep the problem solvable in a reasonable time in (23), and the introduction of a new set of decision variables λ d = (λ sb d ) 0≤sb≤N that choose a future storage level. Additionally to the constraints in Section 3, it has to be guaranteed that the end storage level L d,24 of day d and the starting storage level L d+1,1 of the following day d + 1 are equal as described in (1).
As the number of starting states sb is limited to N storage states, we have chosen to interpolate the end storage state in the last hour of each day with λ d = (λ sb d ) 0≤sb≤N for every day d to estimate the expected value of the storage. More formally, this leads to the additional time-coupled constraints Under the consideration of these time-coupled constraints and the storage level update (1), the storage level constraints (2), and the reserve power market constraints (3), the function (22) is solved by backward induction starting on day D ending on the first day, assuming we have decided for K price clusters on every day: For the last step, instead of the transition probabilities, the initial probabilities are used. In the last step, we constrain the storage state at the beginning of the first day to equal the storage state on the last day of the time period. This way we make sure that all value of stored electricity is exploited and the earnings completely come from the inner-year operation: As for the entire SDP, K · D · N subproblems need to be solved, the problem scales linearly with the number of price clusters. In more detail: with increasing N, only the number of optimization steps increases, but not the subproblem size that is solved in each step/node of the SDP. However, if the number of storage states K is increased, not only the number of optimisation steps increases, but also the problem size increases, as the number of the interpolation variables λ d gets larger. That means that the SDP problem is likely to grow exponentially by increasing the number of storage states.

Evaluation of a Large-scale PV Storage System
Based on the optimal annual returns under uncertainty calculated with the help of the SDP model for each year of the lifetime of the PV storage system, the economic profitability can be measured applying a net present value (NPV) approach or the annuity method to a large-scale 6 battery storage system combined with a solar power plant under uncertain spot prices and solar power generation. Thereby, it is important to mention that the case study makes use of historical electricity price and PV generation from the EEX for the German market area to calibrate the stochastic models (EEX, 2017). The reserve power prices are received from the website of the German TSOs tendering the required reserve capacity (Regelleistung.net, 2017).
The German TSOs do not provide detailed data on the activation of particular plants. We therefore cannot properly calibrate a stochastic process describing the activation of contracted reserve power capacity. After weighing the options of including a stochastic activation without empirical basis or not considering the reserve power market at all, we decided for the third option of including only capacity bids for the case study, neglecting the effect of reserve activation onto the storage level. We argue that the actual error is small, as the producer can close its position using the intra-day market. The error is thus the difference between intra-day price and the marginal costs of reserve activation.
We coded the SDP manually in GAMS and ran, for each optimization within the stochastic tree, a MIP that is solved by CPLEX. Depending on the number of applied price clusters and the available CPU performance, the computation time varies between 4 and 24 hours for a model horizon of one year.

Validation of price and renewable energy production models
For evaluation of the proposed model, in-sample and out-of-sample tests are conducted (see Table 1). In order to obtain meaningful error measures, root mean squared errors (RMSE) and mean average percentage errors (MAPE) are calculated on the sorted time series to account for the fact that price components are simulated stochastically ( Figure 5): Obviously, due to the inclusion of simulated solar and wind power feed-in, the correlation of the simulated price series with historical prices is weakened.
The long-term in-sample test reveals that fitting data to an (overly) long time series leads to a distribution close to the Gaussian distribution in the simulated time series (kurtosis of 3.16 as compared to 16.79 in the original time series). Fitting to a shorter time series leads to a better fit even in out-ofsample simulations regarding RMSE and MAPE (Table 1). On that basis, it was decided to choose only one year to fit the parameters for later simulations. It was generally found that, while providing a good approximation of electricity price patterns (Figure 6), the distributions of simulated and historical power prices are not fully identical.

Evaluation and comparison of the SDP approach with other methodologies
As a goal of this study is to provide a comprehensive approach to the analysis of photovoltaic storage systems, we also provide a short comparative evaluation   study of the proposed SDP formulation. We implemented two strategies to benchmark the above model. As upper bound, we run a Monte-Carlo simulation with a perfect foresight variant of storage operation (henceforth, perfect foresight optimization). On the other hand, we show that our SDP approach performs better than algorithms that heuristically operate the storage intra-day (short: heuristic strategy) 7 .
As a test case, all three strategies are applied to a battery storage system with a charging capacity of 5 MW and a storage volume of 5 MWh for a testing period of 30 days (for other parameters, see Table 6 in the Appendix). We simulate 1000 price and PV production paths and reduce them to 50 clusters for the SDP and heuristic strategy solutions as described above, while the perfect foresight is applied for all 1000 paths. The return of the three strategies are calculated for the case in which the storage is dispatched at the spot market.
For the perfect foresight optimization the simulated prices are used as an exogenous input for a deterministic perfect foresight optimization whose target is to maximize the overall return B, optimizing the dispatch of the energy storage not only for the next day, but the entire period. The return covers the cash-flows generated on the spot market: The target function is maximized subject to the same constraints noted in the problem formulation (Eq. (1) and Eq. (2)), where the disambiguation in the storage level update (1) collapses to the hourly case. We report the mean value of the maximized returns and other operation parameters are calculated, as well as standard deviation and confidence levels ( Table 2).
The heuristic strategy is developed to compare the results of the SDP model with a less complex strategy that can be applied under uncertainty. The core idea is to differentiate the hours of the following day into hours with peak prices p p d and off-peak prices p op d , similar to the peak and off-peak marketing products sold at the EEX, and to exploit the price differentials in-between 8 .
The daily return (contribution margin C d,i ) in scenario i is then calculated as the difference between the earnings on the spot market at peak prices and the costs for the stored energy purchased on the spot market at off-peak prices of the same day: We assume that the storage level must be zero (more generally: equal) at the beginning and at the end of each day. Similar to the backwards induction of the SDP solution, cumulative return from d to D at scenario i is formulated as For more details of the heuristic strategy, compare Keles (2013). The comparison of all three strategies show that the SDP strategy achieves 78% of the perfect foresight strategy.
The confidence interval of the Monte-Carlo runs of the perfect foresight monthly return is narrow. In 95% of the cases, the expected monthly return does not deviate from the mean by more than 0.5%. 9 The results of the perfect foresight strategy can be seen as very robust and it can be used as a benchmark for the comparison with the other strategies.
In this case study, the heuristic strategy is only able to reach 63% of the perfect foresight result (see Table 2). However, we remark that both, SDP and heuristic, yield more accurate results, if the number of scenario clusters is increased. Additionally, the SDP results can be more fine-grained by a higher number of storage levels. To explore both effects in detail is beyond the scope of this article, as we now turn to our case study results.

Profitability of different storage capacities
Subsequently, the model is used to determine the best combination of the battery volume and charging capacity for a stand-alone battery system. The price of the battery system depends on the charging capacity (C in kW) in terms of the related power electronics, and the battery volume (V in kWh). To fully characterize different systems, it is sufficient to vary only the ratio V /C as the costs and returns increase proportionally with increasing volume and capacity for a constant ratio.  Table 3 shows the results for different battery system configurations. It is found that the battery is mostly operated on the spot market. Positive minute reserve does not have a large share in the overall operational hours. If at all, the battery is offered for negative reserve. This is despite the fact that the spot market prices are considered as being uncertain while minute reserve prices are handled as being deterministic and perfectly foreseen.
Thereby, it is worth mentioning that for the investigated business case, the historical series of 2015 with the minimum prices of the accepted bids in the minute reserve power market are applied, so that the optimization model could decide to accept these reserve power prices, or to bid on the spot market with expected prices for the next day's hours. Another strategy, such as applying historical mean prices for reserve power, could improve the number of full load hours on the reserve power market. However, the assumption of perfectly known power prices within the optimization process is then hardly reasonable, as recent studies have shown that applying a bidding strategy based on historical mean prices instead of minimum prices reduces the probability that the bid is accepted in the reserve power market (Wagner and Oktoviany, 2012). A deterministic approach with perfectly known prices therefore requires the strategy with minimum price bidding.
As an indicator for the economic comparison of different system configurations we evaluate the return on investment (ROI). It was found that the most preferable battery system configuration is a battery with V /C = 2 except for scenarios with very low costs for the battery size (volume in MWh) (see Figure  7). It is likely that, these economic conditions will not be met in the coming years (Fleer et al., 2016). In these cases, battery systems with a higher volume in comparison to their charging capacity will be favorable. It is to be noted that the low level of ROI leads to a negative net present value (NPV) for large-scale battery systems operated in the day-ahead market and the minute-reserve market if the conditions met today on these markets are applied to the SDP model. The NPV calculated based on 2015 price level and distribution and based on an economic lifetime of 15 years is even negative for very low interest rates (see Figure 8).
However, future developments such as the expansion of volatile RES in the German electricity system (r2b, 2014; ?), and decreasing battery prices, can lead to a higher profitability of the underlying business model. To investigate the future value of battery storages operated on the spot and minute reserve market, the price model in Section 4 is applied to adjusted data. More precisely, we used the expected PV and wind power capacity for the year 2025 from the EU reference scenario to calculate the merit order effect of RES electricity and simulated prices based on this merit order effect. Furthermore, we used the electricity prices and the RES profile of the year 2011 to calibrate the price model. The year 2011 is a representative year for possible developments in the future, as the fuel and carbon prices were quite high compared to 2015 prices. These higher prices for drivers of electricity prices (Bublitz et al., 2017), and the capacity scarcity which occurred in France in the winter of 2017 had a significant impact on German electricity prices. These developments are a realistic scenario that may occur in the next ten years. This is why we ran our simulation and evaluation models for a fictive year in the future that incorporates a similar price development for input fuels and CO 2 emission certificates as in 2011 and the RES expansion numbers for 2025 mentioned above.
The simulated electricity prices for the fictive future year have a mean level of about 26 e/MWh, which is lower than the level in 2015. This is mainly due to the increased merit order effect, that outstrips the price-increasing effect of the fuel and carbon prices. However, the standard deviation of the simulated future prices (about 21 e/MWh) is higher than the one of the historical prices of the last years. This can again be explained by the merit order effect of the variable RES, wind and PV, which themselves are not evenly distributed, but are very volatile. Hence, more volatile electricity prices can be expected in the future, leading to a better profitability of storage technologies.
The NPV of large-scale battery storage systems, operating mainly in the future spot market with the higher prices mentioned above, is again negative for acceptable interest rates (see Figure 8). Only for low interest rates below 4.5%, the NPV becomes positive. Hereby, a strong price decrease for battery storage systems is assumed, reaching even slightly lower battery prices (200 ekWh for storage volume and 80 ekW for its power electronics) than in Fleer et al. (2016). This indicates that large-scale battery storages will remain economically infeasible, if the underlying business model is based on operating in spot and minute reserve power markets.
Finally, it can be concluded that especially due to the current low level of minute power prices, the business model of bidding only in the minute reserve power market (beside the bidding on the spot market) is economically non-profitable. Hence, the business case of operating the battery storage capacity on the other reserve power markets (e.g. providing primary or secondary reserve in the German case or the so-called spinning reserve in other markets) should be investigated by future research.
6.4. Profitability of large-scale PV storage systems, and impact of network and RES charges After analyzing single storage facilities, we investigate the economic value of combining battery storage with large-scale PV plants. Therefore, we calculate the NPV for the combined PV storage based on the annual return of the combined system maximized under uncertainty.
Technically, the PV plant and the storage can be operated separately, optimizing the profits of each component. However, specific market regulations can produce a positive portfolio effect and make a joint optimization more profitable than the stand-alone optimal operations of the PV plant and storage. For instance, if the exemption from network and RES charges for energy storages (EnWG §118, 2009) is removed after 2026 10 , a combined PV storage system is more profitable, as the storage can be charged by the PV electricity produced onsite avoiding these charges. In fact, even a very small network charge for energy storages anywhere in the electricity system would make future PV storage investments more profitable than stand-alone storage. Regulators have high incentives to favor combined installations: The combination of renewable production with storage technologies can defer transmission and distribution grid investments as well as reduce curtailment of renewables (IRENA, 2019). The current German renewable energy legislation already reduces compensation for renewable power that is switched off due to congestion in the electricity network to 95% of the market premium. Furthermore, if negative prices occur on the market for six or more consecutive hours, the market premium is not paid anymore to renewable operators (EEG §51, 2017). In both cases, the availability of storage can shift the energy feed-in to times where these restrictions do not apply and full premiums can be earned, so that a combined operation becomes more profitable than stand-alone operation. The ongoing rapid expansion of renewables will induce both situations much more frequently than today's energy system. Besides regulations in Germany, there also exist regulations in other countries that make a joint optimal operation more profitable, for example cheaper connection charges in UK (DNVGL, 2017).
In the following, we illustrate first the results for different jointly operated PV storage systems under the current legislation of being exempted from network charges for the price level of 2015 and 2025. Afterwards, we compare the current legislation to a case where regulators impose charges, to illustrate the portfolio effect.
Besides the assumptions applied for the battery storage in Section 6.2 (investments for battery: 300 e/kWh and 120 e/kW, 15 years economic lifetime), we assume for the large-scale PV investments expenses in the amount of 550 e/kW 11 . All techno-economic parameters of the PV storage system are sum-marized in Table 6 in the appendix. The results indicate that the NPV is again negative if price simulations based on 2015 prices are applied. However, in the 2025 price scenario, the NPV becomes positive for also high interest rates. This holds for storage systems with a volume/ratio of 1 and 4 (V/C-1 and V/C-4). It is obvious that this improvement of the PV storage combination is due to the additional investment in the large-scale PV plant. The PV plant increases the entire systems profitability. This becomes obvious if one compares the profitability of the PV storage system ( Figure 9) with the stand-alone storage 12 (see Figure 8). The NPV is in the latter case negative for high interest rates, while it is always positive for the PV storage system. This comparison indicates that a stand-alone PV park may even be more profitable than the combined PV storage system. However, the high profitability of PV originates also from the current support scheme with market premiums for renewable electricity. The profitability may change if these premiums are removed by the regulator in the near future according to the current affordability discussion. A PV storage system may then become more profitable than a stand-alone PV park. To survey this, we make two more model runs for the year 2025 without applying market premiums for the produced PV electricity. The results show that in this case the NPV of the PV storage is higher than that of stand-alone PV for an interest rate up to 9% (see Figure 10), but remains lower for higher interest rates. This is mainly due to the fact that the additional returns of the PV storage (on top of the stand-alone For the calculation, a payback period of 10 years and 978 full load hours production per year citepISE2017 are also taken into account. 12 Stand-alone means a single storage system without PV PV) are discounted by high interest rates making the additional investment in the storage in t 0 less profitable. Besides, it has to be considered that the positive net present value is possible only if network and RES electricity charges are not applied to the battery storage during the charging process. The application of RES charges and network charges would decrease the profitability of PV plants and storages as stand-alone facilities drastically. However, PV storage systems can remain profitable if they are combined at a single site avoiding additional payments, such as network and other charges 13 . In a further scenario, we applied the 2017 level of RES charges (6.88 e-ct/kWh) for charging the battery and another 1 e-ct/kWh of network charges assuming that the large-scale battery and PV plant are connected to the 110 kV voltage grid. 14 The application of these charges reduces the annual return of the PV storage system by almost 30% calculated with the 2015 prices (see Table 4). Furthermore, the introduction of RES and network charges hinders almost completely the economic dispatch of the storage on the spot market: Only about 4-5 full load hours can be achieved for the battery operation. This is due to fact that the price spread in 2015 price series is not sufficient to cover the charges that apply in the case of charging and operating the storage in the spot market. However, a reasonable spot market dispatch can be reached for the battery if it is charged by a PV plant that is located on the same site (see Table 4, scenario "with charges, on-site") and if the mentioned charges do not apply for PV electricity that is "self-consumed" by the on-site storage. This change in operation leads to significantly higher annual return compared to the scenario, in which the PV plant and storage are stand-alone facilities. In the scenario "with charges, on-site", a large part of the PV electricity is used to charge the battery (754 full load hours) and to discharge the battery at times with higher prices. Only about 300 full load hours electricity of the PV plant is directly sold in the spot market. Besides, the battery charging with grid electricity diminishes again, as the RES charges apply for grid electricity use in this scenario as well. It can be concluded that it is economically more feasible to build PV plants and storages on a single site to avoid extra costs/charges that may be installed by future energy policy.

Conclusions
In this paper, different approaches to model uncertain parameters on energy markets, such as electricity prices, PV feed-in and wind power generation, are introduced. The electricity price model takes into account the so-called merit order effect of PV and wind power and the inherent non-stationarity of the underlying time series. A second method based on stochastic dynamic programming is then developed to evaluate the economic profitability of PV storage systems operating on the spot and minute reserve power markets, whereby the previously developed electricity and PV models are used to generate a large number of price and PV feed-in scenarios that cover the uncertainty in these parameters.
A large number of price and PV scenarios are necessary to cover the wide range of possible developments. However, a usual stochastic optimization model for the dispatch and evaluation of PV storage systems cannot be executed in an acceptable computation time applying these large numbers (1000 simulations) of scenarios and 365 stages of decision. Hence, an SDP model is found to be more suitable for capturing the uncertainty, and is applied after reducing the initial large number of scenarios to a recombining tree with a small number of price clusters representing the price development on each day. The scenario reduction with 20 price clusters leads to about 6 hours computation time for the SDP model with a time horizon of one year.
As demonstrated in Section 6.2, the SDP model is able to generate more profit for PV storage systems considering the uncertainty of prices and renewable energy generation. Compared to heuristic strategies, it is able to achieve a higher portion (more than 80%) of the upper threshold which can be achieved only under perfect foresight, i.e. opitmization with certainty on price developments within a price path. Hence, the SDP approach is a well-performing method to evaluate systems with energy storage under uncertainty.
Regarding the evaluation results for battery storages, it can be concluded that the business model of operating a large-scale battery on the electricity spot market and minute reserve power market is currently economically unfeasible and will remain so, if no unexpected increase in price level and volatility occurs in the next years. A moderate increase in price volatility increases the economic value of the investigated battery configurations. But the low ROI of 2-4.5% and the negative NPVs for acceptable interest rates (5-10%), even in the case of a significant decrease in investment expenses, leads to the conclusion that battery investments will not become profitable in the next few years if they are operated only on the spot and minute reserve power markets, although they are welcome from the point of view of system security.
However, investments in battery storages can be economically more feasible, if their capacity is operated on the other reserve power markets (primary and secondary reserve), as the prices per MW are currently much higher in these markets (Regelleistung.net, 2017). First use of large-scale installations for primary reserve power operation indicate new areas of storage operation. However, because of non-availability of data for secondary reserve power prices with an appropriate intra-day resolution for the analysed time period, in this study, we focused on the minute reserve power market where data with a 4h-resolution was available. Since, the design of the secondary reserve power market has been changed by the German regulator and this reserve is also now traded day-ahead with 4h block contracts, the developed approach ca n be also applied to an evaluation with secondary reserve, when after some years data is collected for a sufficient period of time. Therefore, future research may focus on the evaluation of PV storage investments if they are operated especially in the secondary reserve market. Additionally, if detailed data for volumes of requested reserve energy from different technologies is available, the numerical study can be updated based on this data. Currently, new market designs for the reserve power market are discussed and tested in the German electricity system. If new design rules are defined, the provided MDP can be adjusted according to these changes and new application results may be derived for the case study.
Combining large-scale battery storage systems with PV plants significantly increases profitability and can make investments economically reasonable, especially if electricity prices become more volatile in the future with larger intra-day spreads. However, it is to be noted that this increase in value of the combined plant originates mainly from the PV plant, which receives market premium payments apart from the spot market returns. The profitability of the combined plant may be reduced if RES and network charges are applied to battery storage. In this case, the annual return would be reduced by almost 30%, and charging the battery on the wholesale market would not be feasible in the economic sense. A solution to avoid these charges (if they are introduced) would be the installation of the PV plant and battery storage on a single site, and to charge the battery with PV electricity. This measure again increases the annual return by more than 12% in our analyses. Hence, it can be concluded that it is economically more feasible to build PV plants and storage on a single site to avoid extra costs/charges that may be imposed by future energy policy.
For future research, it is of high importance to study other business models for battery storage, besides the ones investigated in this study, and to address these in future research. The methodology developed in this study is applicable to other business models or future market design choices. Regime intervals and the switching probabilities between them