Seasonal hydrogen storage decisions under constrained electricity distribution capacity

The transition to renewable energy systems causes increased decentralization of the energy supply. Solar parks are built to increase renewable energy penetration and to supply local communities that become increasingly self-sufficient. These parks are generally installed in rural areas where electricity grid distribution capacity is limited. This causes the produced energy to create grid congestion. Temporary storage can be a solution. In addition to batteries, which are most suitable for intraday storage, hydrogen provides a long-term storage option and can be used to overcome seasonal mismatches in supply and demand. In this paper, we examine the operational decisions related to storing energy using hydrogen, and buying from or selling to the grid considering grid capacity limitations. We model the problem as a Markov decision process taking into account seasonal production and demand patterns, uncertain solar energy generation, and local electricity prices. We show that ignoring seasonal demand and production patterns is suboptimal. In addition, we show that the introduction of a hydrogen storage facility for a solar farm in rural areas may lead to positive profits, whereas this is loss-making without storage facilities. In a sensitivity analysis, we show that only if distribution capacity is too small, hydrogen storage does not lead to profits and reduced congestion at the cable connection. When the distribution capacity is constrained, a higher storage capacity leads to more buying-related actions from the electricity grid to prevent future shortages and to exploit price differences. This leads to more congestion at the connected cable and is an important insight for policy-makers and net-operators.


Introduction
Renewable energy sources have become increasingly popular. For example, renewable energy production in the EU has increased from 9.6% in 2004 to 18.9% in 2018 [11]. However, seasonality mismatches between supply and demand are one of the main challenges that should be dealt with to facilitate growth in renewable energy production. Large solar parks tend to be located in rural areas where land is relatively cheap, even though the electricity grid infrastructure is often limited. This typically causes cable congestion at the location where the solar park is connected to the grid, which in turn causes outages, grid balance problems, and affects operational costs of the electricity grid [25,37]. The high peaks of solar energy production in summer often cause cable congestion which may inhibit the installation of solar parks. Designing electricity grids that can accommodate location-specific energy generation and take into account physical constraints is therefore a challenging task [26].
Hydrogen storage is a useful flexibility option and can spread the feed-in to the electricity grid throughout the year, thereby mitigating cable congestion and bridging seasonality gaps in demand and production [1]. Storage is one of many flexibility-related options that also include market mechanisms for power re-dispatch, curtailment, conversion, and demand-side management [21,23,37].
Storage owners connected to an external electricity grid face variable electricity prices as a result of market mechanisms to balance supply and demand, and to reduce congestion. For example, location-specific (nodal) electricity prices can be an effective solution to reduce grid congestion caused by renewable energy sources [28].
Furthermore, time-of-use tariffs are expected to become more common since these provide advantages to grid operators in alleviating expected congestion [33]. The resulting market mechanisms facilitate the re-dispatching of energy production by stimulating additional production in areas without congestion and reducing production in areas of congestion by using nodal prices [39]. However, this requires storage owners to take into account these market mechanisms to maximize their profits.
Electricity generated by solar parks can be fed to an external electricity grid, used to directly supply a local electricity demand of households at a nearby location or stored in a nearby storage location. The local consumption of the generated electricity avoids the transportation of energy over long distances and enables fulfilling a local electricity demand. Electricity demand of consumers is also characterized by differences in seasonality [2]. The confinement of the produced energy in the area of generation, seasonality differences in supply and local demand, and the presence of variable nodal electricity prices have important consequences for the decision to store energy. Efficient operational strategies that determine when energy is stored, sold, or bought from the electricity grid are key in achieving successful renewable energy penetration. Hydrogen storage is characterized by relatively high investment costs and limited conversion efficiencies. Therefore, strategies related to storage and interaction with the electricity grid become even more important in enhancing the economic viability of hydrogen storage as a flexibility option, that is especially suitable for long-term seasonal storage.
In this study, we examine the operational problem of a hydrogen storage owner who needs to decide how much to store, buy from, and sell to the electricity grid throughout the year. This decision is affected by the presence of seasonality in supply and demand. This decision is dependent on the level of solar energy production, the amount of local electricity demand, the amount of hydrogen in storage, and the current electricity price. Additionally, solar energy production levels, electricity demand, and prices in the future are uncertain. For example, even though solar energy production can be predicted rather accurately for several days in advance, solar energy production levels for specific days are uncertain when predicted for longer periods such as months. As a result of seasonal patterns, the stochastic behavior of solar energy production and electricity demand is time-dependent. These aspects affect storage decisions throughout the year. A Markov decision process formulation is proposed to obtain optimal policies for the above problem.
Our main contributions are as follows. In an extensive numerical study, we show how hydrogen storage can either solve or cause congestion problems at the cable to which the solar park is connected as a result of selling or buying-related decisions. Secondly, we identify the characteristics of optimal storage policies that underlie the above-mentioned issues. These characteristics include price thresholds for each period which depend on the inventory level, price, and net production after demand. We show policies in summer and winter by taking into account seasonality related to supply and demand. This includes the actions which are taken during overages and shortages throughout the year. Thirdly, we indicate how different combinations of storage and distribution capacity affect these decisions. Next, we analyze profits, congestion levels, and electrolyzer utilization. Finally, we show how conversion losses and differences between selling and buying prices affect these results.
Results show that optimal policies are characterized by price thresholds which separate different types of actions. These include buying the maximum possible quantity, selling exact overages or buying exact shortages, storing overages or obtaining shortages from storage, or selling the maximum amount possible. When distribution capacity is constrained, congestion at the cable connection is mostly caused by buying-related actions in winter, which are used to cover potential future shortages. For higher levels of distribution capacity, congestion is mostly caused by selling-related actions of the overages in the summer. While it may be expected that storage enables reducing congestion, increased levels of storage capacity have an adverse affect on congestion. This is because storage enables increasing buying-related actions to prevent future shortages and exploiting price differences. Results also indicate that a lower electrolyzer utilization as a result of a large capacity is associated with higher profits than a low electrolyzer capacity with a higher utilization rate. This indicates that a high utilization level of the electrolyzer is not necessarily an indication of improved feasibility.
The remainder of this paper is organized as follows. A literature review is presented in Section 2. Section 3 describes the problem and Section 4 formulates a model. Section 5 provides an overview of the calibration of the parameter settings and the base case system that we consider. Section 6 provides a sensitivity analysis of key performance indicators based on the parameter settings of the capacities of each of the system elements. Section 7 provides concluding remarks.

Literature review
The existing literature has mostly addressed energy management strategies in which the owner of an energy storage device decides when to buy or sell energy from or to the grid. For a detailed review of energy management decisions for electric storage systems, we refer to [40] and [42]. While seasonality differences between supply and demand are important characteristics of renewable energy systems, most papers focus on intraday and day-ahead buying and selling decisions using battery storage for short planning horizons and small discretization levels. In contrast, our approach focuses on hydrogen storage decisions for longer planning horizons covering seasonality during a year. We take into account limited electricity grid distribution capacity which support the need for local storage. Seasonal patterns in supply and demand, limited grid infrastructure, and electrolyzer and fuel-cell constraints all have consequences for storage decisions.
The literature on energy management decisions optimizes grid interactions using storage by treating supply levels as deterministic [8,29,43] or by optimizing the buying, selling or storage decisions in which the uncertainty of the generated renewable energy is taken into account [14-19, 22, 32]. Literature on energy procurement decisions without storage include Wang and Deng [38] and Woo et al. [41]. For example, Wang and Deng [38] analyze an energy procurement problem for a centralized energy aggregator who can control both procurement and consumption within a 24 hour planning horizon in which wind energy is generated. From the perspective of a grid distribution operator, Woo et al. [41] address the energy procurement decisions of a grid distributor that need to balance procurement risks and expected costs. Regarding prices, Densing [7], Hassler [17], Jiang and Powell [18,19], Zhou et al. [44] specifically take into account stochastic electricity prices, whereas Grillo et al. [15], Keerthisinghe et al. [22], Shin et al. [32], Steffen and Weber [35] treat these as deterministic. These studies are explained in more detail below.
Studies that optimize the use of storage with buying or selling from or to the grid mostly focus on detailed decisions within one day [14-19, 22, 32]. For example, Jiang and Powell [19] address the arbitrage problem with energy storage to place bids in an hour-ahead spot market. Grillo et al. [15] optimally schedule batteries with renewable energy. Gönsch and Hassler [14], Hassler [17] optimize energy arbitrage decisions for short time horizons within one day and time intervals of 15 minutes. Keerthisinghe et al. [22] develop energy-storing policies for a battery in a residential home within single days. In contrast, Shin et al. [32] have addressed the problem for both intraday and yearly planning horizons. Zhou et al. [44] address the energy storage arbitrage problem within a week for 5-minute periods. They address seasonality by using specific parameter settings for each week that is solved. While these papers all address detailed arbitrage and storing decisions for batteries and short time horizons using batteries, none of them consider hydrogen which is seen as a more viable option for long-term storage. Most papers also do not provide an integrated approach to address the issue of seasonality over one year.
Related literature on energy storage and arbitrage which has included transmission or distribution capacity constraints is relatively scarce. Fertig and Apt [12] investigate the economics of pairing a wind farm with compressed air energy storage and limited transmission capacity. They use heuristic control policies to decide on buying and selling to the grid. Most work that incorporates transmission constraints focuses on energy storage planning from a strategic perspective rather than an operational perspective. For example, Babrowski et al. [3] optimize storage planning for the German electricity sector while including transmission constraints. Wang et al. [39] examine to what extent transmission congestion affects the profitability of arbitrage by energy storage including transmission constraints. Jorgenson et al. [20] analyze to what extent transmission or storage can assist in reducing curtailment.
To the best of our knowledge, Zhou et al. [44] and Gönsch and Hassler [14] are the only papers that have included transmission constraints in focusing on the operational decision of when to buy, sell or store. They have addressed wind-based electricity with co-located storage. However, their numerical study encompasses only one week and does not consider hydrogen storage which would become relevant for longer terms. To the best of our knowledge, the seasonality effects related to production, demand, and nodal prices of large-scale renewable energy generation and storage have not yet been considered.

Problem description
We consider a profit-maximizing renewable energy producer using a photovoltaic (PV) system (i.e., solar panels) who is also the storage owner and is responsible for satisfying local electricity demand. For instance, it may form a self-sufficient community together with a small village. The energy producer can also sell or buy electricity from an electricity grid, and a co-located hydrogen storage is used to temporarily store electricity in the form of hydrogen. We consider a time horizon T that resembles a complete year, and each period t ∈ T resembles a single day. Figure 1 provides a graphical overview of the considered system [13]. The left side of Figure 1 shows the solar energy producer and the hydrogen storage facility and the right side shows the local electricity demand and electricity grid connection. Our goal is to decide upon when and how to 1) sell and buy electricity from the grid, 2) store or consume hydrogen from our local storage to satisfy local demand and maximize profits. We assume the owner of the storage and PV facilities, and the households are connected to the grid through an external connection and are the single users of this connection.  In the following, we describe our system in detail. Table 1 provides an overview of all the parameters and variables.
The installed capacity of solar energy production in MWp is assumed constant throughout the year and denoted by w. Solar energy production per day is a random variable Y t , where the dependency on the period follows from seasonal differences in energy production throughout the year. Local demand for electricity is denoted by the random variable D t and is assumed to be normally distributed with mean µ t and a period-independent standard deviation σ.
As it is optimal to always satisfy local demand with local supply of solar energy (this comes at no cost), we assume that the produced electricity Y t is first used to supply the local demand D t . We then convert our production and demand process in a net solar energy production levelȲ t := Y t − D t . The net solar energy production per daȳ Y t can then be modeled via a truncated probability distribution f¯y(t) with a maximum of l + t in period t due to a restricted installed capacity of solar energy.
Hydrogen inventory x t is held inside a hydrogen storage tank with energy capacity m and is filled using an electrolyzer with a maximum rate k + at which energy can be stored per period and a conversion efficiency of α where α ≤ 1. Moreover, the producer can obtain at most k − energy units per period from storage due to a limited fuel cell capacity. The maximum amount of electricity that can be sold to or bought from the grid at the end of a period is denoted by the grid distribution capacity k c . Throughout this paper, capacities (with the exception of installed solar energy capacity) are defined as the maximum amount of energy per period that can be handled by the related component in our system. Electricity prices are stochastic and are modeled as an AR(1) process via Similar to [7,44], we assume that energy producer and storage owner is sufficiently small so that it is a price taker that cannot influence prices. Consequently, solar energy can be sold to the grid at a price of C t . The producer can also buy from the grid at price C t + c + , where c + ≥ 0 is a fixed price markup. As is commonly assumed, it is not possible to simultaneously buy and sell from the grid [see, e.g., 44].
The solar energy producer makes the buying, selling and storing decisions at the end of each period t ∈ T .
Since hydrogen conversion is associated with relatively high conversion losses and electrolyzers perform better at stable loads, we assume that intraday load fluctuations are handled by a battery. Since our focus is on seasonal differences on an annual time horizon, conversion losses of batteries are limited to 82% and 90% [36] and the modeling of intraday battery load fluctuations are considered outside the scope of this paper.
The objective of the storage owner is to maximize the expected future profits related to interacting with the electricity grid during the planning horizon. The decisions made in each period affect the total profit. These decisions have to satisfy several detailed constraints and depend on the state of the system at the end of a period.
We discuss these aspects in the next section.

Markov Decision Process Formulation
We formulate our problem as a Markov decision process (MDP). We first describe our state and action spaces, and the constraints upon them. We also specify our reward function. We then discuss how we discretized our state

Stochastic variables
Y t Solar energy production in period t Y t Net production level after demand in period t C t Electricity prices in the local spot market in period t D t Local electricity demand in period t and action spaces, and formally define our MDP which we solve via backward dynamic programming.

State and action space
At the end of period t, we observe an inventory level x t , price level c t and net production level after demand y t . Let S t (ȳ n , x t , c t , c t−1 ) ∈ S be the state of our system in period t. We write S for the state space. For each state S t ∈ S, we define the action u(S t ) ∈ R as the amount of energy units to buy from or sell to the grid at the end of period t. Negative values represent the amount of energy units to buy from the grid.
The action u is bounded by the characteristics of the state S t . It is most easily described if we consider the range of actions [−u min (S t ), u max (S t )], where u min (S t ) ≥ 0 denotes the maximum amount of energy bought from the grid at the end of a period, and u max (S t ) ≥ 0 denotes the maximum amount of energy that can be sold to the grid at the end of a period. In the following, we describe for each state S t ∈ S how to obtain u min (S t ) and u max (S t ).
The maximum amount of energy that can be bought u min (S t ), for all S t ∈ S is the largest value which must satisfy three constraints such that The first indicates that the action should not exceed the maximum amount bought from the grid per period. The second indicates that the maximum inventory level should be considered. The third indicates that the maximum amount obtained from storage per period should be respected. The maximum amount of energy that can be sold which indicates that the cable distribution capacity should be respected, that we can sell at most the inventory we have plus the net overage, and that the fuel cell capacity should be respected. Note that these actions allow for unmet demand, in caseȳ t < 0. We, therefore, introduce a penalty s per unit of unmet demand that is set sufficiently large to avoid this could happen under any optimal policy. The action space U can then be defined as (1) and (2), t ∈ T .
The reward r(u(S t )) of taking action u(S t ) is the sum of the revenues and costs during period t as a result of interacting with the grid. It is defined as where I (·) equals 1 if (·) evaluates to true, and is 0 otherwise. Here, by slight abuse of notation, we denote with x t+1 | u the hypothetical inventory level at time t + 1 given action u(S t ). If that is negative, demand is unmet and penalty costs s are incurred.
Note that if energy is sent to storage, that is for any state S t , −u +ȳ t ≥ 0, the amount of energy that is stored depends on the conversion efficiency α. To avoid numerical issues associated with round-trip conversion losses and resulting fractional numbers, the total round-trip efficiency is only calculated when sending energy to storage.

Discretization
For the numerical analysis, we need to discretize the state space. We discretize the amount of hydrogen inventory, the net production throughout the year, and the observed prices. We define X as the set of possible net inventory levels, where ∆x t represents the interval size of the inventory levels. The intervals to which an action u belongs are split into equally-sized intervals which correspond to the discretization of the inventory levels ∆x t . We denote the discretized set of actions by U. The stochastic net solar energy productionȲ t is discretized according to ∆ j t , and electricity prices are discretized with ∆c t . Accordingly,

MDP for storing, buying from or selling to the grid
The selling and buying policies result in an inventory process over time in which an immediate reward of r(u(S t ))) is earned after choosing an action at the end of period t. The action is chosen after the solar production and electricity prices have been fully observed at the end of a period. Therefore, the decision-maker knows with certainty to which new inventory level the action will lead in the next period. The future inventory x n−1 in period n − 1 with n periods to go n = T − t can be defined as 1 The transition probability p n−1 (ȳ n−1 , c n−1 , c n ) is defined as the probability of net production realization ofȳ n−1 and a price realization of c n−1 in period n − 1 given price c n in period n. Similar to Zhou et al. [44], we assume that the underlying stochastic processes are independent. We define V 0 (S 0 ) as the total expected profit at the end of the horizon with n = 0 periods to go. We assume For all other time periods in which n > 0, action u(S n ) ∈ U can be executed. Accordingly, we define Via backwards dynamic programming, V n (S n ) can be obtained for all periods-to-go n ∈ T . The associated optimal periodic policy (u * 1 (S 1 ), u * 2 (S 2 ), . . . , u * T (S T )) that minimizes long-term average rewards is then obtained by iteratively applying backwards dynamic programming upon this system, where V 0 (S 0 ) is calculated according to equation (10) with V n−1 (S n−1 ) equal to V T (S T ) of the previous iteration (equaling zero for the first iteration).
Convergence to optimality is proven if all values V n (S n ) change with the same value (i.e., the long-run average reward) between iterations [30].

Numerical Analysis
We start our numerical section by introducing a base-case system for which we provide a detailed numerical analysis (see Section 5.1) and then describe how the price and production processes are fitted (see Section 5.2). We end the section by examining the optimal policy for the base-case system while focusing on the differences between summer and winter. To provide a comparison for the viability of the base-case system, we compare the optimal policy to a system without any hydrogen storage options. A more extensive sensitivity analysis in which all system parameters deviate one-by-one is postponed to Section 6.

Base-case system
The experiments are based on a planned project of a rural village in the Netherlands in which electricity needs are supplied by a solar park. It comprises a hypothetical solar park with a peak capacity of 5 MWp that is connected to a local electricity grid with a maximum capacity of 1.25 MW (which corresponds to 30 MWh per day). Distribution capacities of 2.5 MW are common in practice for lines that operate at the distribution (local) level rather than the (national) transmission level. In our experiments, we set a more constrained distribution capacity of 1.25 MW, to represent the situation in which the distribution capacity is more constrained. We assume the solar park is connected to a 2.1 MW electrolyzer and a 2.1 MW stack of fuel cells. For both of the electrolyzer and fuel cell, this translates to a capacity of 50 MWh per day. Since only relatively small amounts (up to 1 MWh) can be stored inside pressurized vessels, we assume that hydrogen is stored into a co-located large-scale storage location with a capacity of 1000 MWh. Moreover, we assume a round-trip efficiency α of 0.5.

Fitting the price, production, and demand process
Day-ahead hourly wholesale electricity prices in euro/MWh in the Netherlands between 2015 and 2019 are obtained from ENTSOE Transparency Platform [9]. These prices are aggregated to daily prices using the intra-day mean. We assume that the electricity prices which apply to the storage owner exhibit similar behavior to wholesale day-ahead electricity prices.
The average daily day-ahead prices exhibited strong autocorrelation (0.872 for a time lag of 1). Moreover, weekly autocorrelation was observed in which autocorrelation was stronger for weekly time intervals than for intraweek intervals. For example, the autocorrelation decreases to 0.773 for lags up to 6 and jumps to 0.815 for lag 7, suggesting that weekday effects are existent. Seasonal effects are not directly apparent from the data.
Inspired by existing approaches in literature (e.g., [44]), we test three different AR(1) models of the form N(0, σ c ), C t is the predicted value, C t−1 is the observation at t − 1, φ is a constant, and θ is the AR term with a time lag of 1. First, we fit an AR(1) process to the daily electricity prices in which both monthly and weekday effects are removed from the original observations. Secondly, we fit an AR(1) process to prices in which only weekday effects are removed. Thirdly, we fit an AR(1) process to the original observations. We compare the models by evaluating the standard error of the estimate in relation to the actual observations.
The month day and weekday effects are removed similarly as in [44]. To evaluate the fit of the AR(1) models, the standard error was calculated as T t=2 (C t −Ĉ t ) 2 /T , where T is the number of periods andĈ t = φ+θC t−1 + f ′ (t)+ξ t is the predicted price in period t. The seasonality effects of the electricity prices are described by f ′ (t) which is defined similarly as in [44]. Because the standard error of the model which was fitted to the original data is lowest (7.7), as is given in Table 2, we do not include month and weekday effects in our AR(1) process. To model the demand in our base-case system, we assume that the solar park and hydrogen fuel cells connect directly to a set of 1500 houses that are responsible for the local electricity demand. We have obtained data on electricity consumption from the society of the Dutch Energy Data Exchange (NEDU). The data represents average electricity consumption levels per 15 minutes as a fraction of the total yearly consumption level for 3001 measurements during the years 2016, 2017, and 2018. Since the data is highly aggregated, the data can be scaled to 1500 households to represent our base-case system. We assume that one household on average consumes 2990 kWh in electricity per year [27]. The scaled daily consumption levels, as used in our base-case system, have a minimum of 9.9 MWh, a mean of 12.3 MWh, and a maximum of 16.1 MWh per day.
The data exhibits a strong linear relationship with time in which marginal consumption levels are negative between winter and summer and positive between summer and winter. Accordingly, we split the data into two subsets of observations and fit a linear regression to each subset. The splitting procedure was based on minimizing the sum of the standard errors for both models. Accordingly, model 1 was based on day 1 to 199, whereas model 2 was based on day 200 to 365. The fitted models are displayed in Table 3. Since the original data is highly aggregated, we assume the data is normally distributed (i.i.d.) with a daily µ t and a constant standard deviation σ.
Since the standard errors are relatively similar, we chose the highest standard error of both models (σ = 0.62) as the standard deviation of electricity demand in the experiments.  We assume that the daily solar energy production levels are stochastic. For each day, hourly solar energy production levels between 2005 and 2016 were obtained from PVGIS [31] and aggregated to daily amounts. The production levels correspond to an installed capacity of 5 MWp. The original data was aggregated to daily production levels. For each week, the daily observations within the week of 11 years were normalized to a range between 0 and 1 and fitted to a beta distribution to increase the number of observations. Accordingly, daily solar energy production levels are represented by shape parameters α t and β t . Beta distributions are commonly used in modeling daily solar energy production levels [10,24]. Similar to Boland [4] and Soubdhan et al. [34], we assume that Y t is independent and identically distributed for each period.

Optimal Policy Structure
We solved the MDP associated with the base-case system employing backward dynamic programming. As our states are dependent on the day of the year, we considered 50 years. The results presented correspond to the optimal policy of year 0, and can be interpreted as the policy that maximizes long term average rewards (see, for a similar approach, Byon and Ding [5]). Implementation is done in C++17 and the MDP is solved on an Intel Xeon 2.5Ghz processor using 4 threads. In the following, we discuss the structure of the optimal policy of the base-case system.
The performance of this policy is discussed separately in Section 5.4. Figure 2 shows the optimal policies (i.e., the amount of electricity to sell to the grid) for a period in the winter (Period 1) and in the summer (Period 2). For both periods, we present the optimal policies for the first and third quantile of the net production distribution. On the x-axis, we show the observed electricity price and the probability of occurrence. On the y-axis, we portray the inventory level of the hydrogen storage. In this way, the four graphs represent a cloudy winter day (top left), a sunny winter day (top right), a cloudy summer day (bottom left), and a sunny summer day (bottom right).
From Figure 2, one can observe four different types of actions that are taken in the optimal policy, in any of the depicted situations: First, if prices are relatively low the optimal policy prescribes to buy as much electricity from the grid as possible. Second, if prices start to increase, it is best to buy or sell the observed net production. Third, dependent on the actual day of the year, it might be optimal to not buy or sell electricity from the grid, as long as enough inventory is on hand. Fourth, if prices are high enough, it is best to sell as much electricity as possible. We observe that the action to not interact with the grid takes place for opposite inventory levels in the summer and the winter. In the summer, "no interaction" is optimal for relatively high price levels and the net production is then converted to hydrogen, in the winter net shortages are fulfilled by converting hydrogen to electricity, which also avoids interaction with the grid. To better understand how the optimal policy differs throughout the year, Figure 3 presents the optimal action as a function of time and the inventory level for the 25% net production percentile and for the two price levels (c t = 20 and 60). Figure 4 does the same for the 75% net production percentile. The action is represented as the resulting change in inventory level. It is observed that the change in inventory has a period-dependent threshold. In the summer (the middle part of both pictures), the inventory at which optimal actions lead to an inventory increase is lower than in the winter, due to oversupply of electricity in the summer and shortages in winter. The inventory levels at which these thresholds occur are similar for both low and high net production levels. Figure 4 shows that inventory-increasing actions are also prevalent in summer when prices are low (i.e., c t = 20). This indicates the prevalence of seasonal effects in the optimal policies. This behavior can be attributed to the following dynamics. Early in the year, a higher probability exists of encountering future electricity shortages than later in the year. Therefore, energy is stored at times of low prices early in the year to enable accumulating sufficient inventory for moments of shortages later in the year. Furthermore, buying decisions made early in the year facilitate the potential to benefit from price differences later in the year.
These dynamics will be explained in more detail in Section 5.4.

Optimal Policy Performance
We simulated the optimal policy of the base-case system for a total of 1,100,000 years, using the first 100,000 years as a warm-up for the simulation. Key performance indicators are given in Table 4. As a benchmark, we also provide the statistics of our base-case system if no hydrogen storage is available (BM1), and in case the optimal policy for stationary net production (i.e., ignoring seasonal effects) is applied to our base-case system (BM2). Our key performance indicators are mean profits, electrolyzer utilization, and the mean percentage of the time in which congestion occurs at the cable connected to the solar park. Similar to [6], we define congestion as the event in which the amount of electricity sent to or obtained from the distribution grid equals the distribution capacity to which the supplier or consumer, in this case, the solar park with storage, is connected. Accordingly, the mean percentage of time congestion is measured as the mean percentage of time in which selling or buying energy equals the grid distribution capacity. The electrolyzer utilization is given as the percentage use of its full capacity. to the base-case system. The mean electrolyzer utilization denotes the amount of electricity converted to hydrogen given that the electrolyzer is used. It increases 1.6% when seasonality ignored. The mean percentage of time the cable is used to its full capacity, reflecting situations in which congestion occurs at the cable to which the solar park is connected, equals 8.6% and 8.8% for the base case and when ignoring seasonality (BM2), respectively.
We further detail the expected number of times particular actions are taken throughout the year. In Figure 5, we detail these actions for a net overage (a) and a net shortage (b).
Given a net shortage, the red points indicate the fraction of times less than the shortage is bought while the remainder is obtained from storage. The green points indicate that more is bought than the shortage while the remainder is stored. The blue points indicate the fraction of times the exact amount of the shortage is bought. The purple points indicate that more inventory is sold than the shortage.
Given a net overage, the red points indicate the fraction of times exactly the net overage is sold. The green points indicate the fraction of times the overage is sold plus additional inventory. The blue points indicate the part of the overage that is sold while the remainder is stored. The purple points indicate the fraction of times the overage is stored and additional inventory is bought.
In the case of a net overage for the base-case system, the red points in Figure 5 (a) show that policies in which exactly the overages are sold to the grid are most prevalent. These follow the seasonality pattern associated with solar electricity production and occur most frequently at 69% of the time in the summer. Policies in which additional electricity is sold out of storage are associated with exploiting price difference possibilities. These are the least common during overages and are highest in summer occurring 5% of the time. Selling less than the overage is also not a common strategy and occurs maximally at 7.7% of the time. This indicates that storage is not used frequently in cases of overages. This also suggests that excess net production can at best be sold directly to the grid to avoid conversion losses when using storage, even at low prices.
For cases of net shortage in Figure 5 (b), occurrences in which the exact shortage is bought from the grid are most prevalent. The blue points show that the fraction of times the exact amount of the shortage is bought follows an inverse pattern compared to policies in which exactly overages are sold during net overages. These are highest in winter up to 79% and lowest in summer down to 5%. Other policies are very uncommon for the conducted experiments.
Even though these policies are relatively uncommon, the less frequent action types that include not simply selling or buying net production differences, are important for the feasibility of a solar park in rural, possibly congested, areas. For instance, in The Netherlands, it is not allowed to install a solar park with a maximum capacity higher than the distribution capacity, even peaks only occur at clear summer days. The results presented in Figure 5 indicate that using a storage facility will not interact structurally different from a classic solar park without storage, only its distribution capacity is limited. These are exactly the moments when the less-frequent actions depicted in Figure 5 play an important role to keep the base-case system feasible. This includes the ability to exploit price differences and to fulfill shortages at times when prices are high. As can be seen in Table 4, a solar park without a storage facility (connected to local electricity demand) yields negative mean profits when demand needs to be solely fulfilled from the electricity grid.

Sensitivity Analysis
We further investigate the performance of our system, using the key performance indicators already presented in Section 4. We first investigate the impact of changing the distribution capacity (Section 6.1) and afterward discuss the impact of the storage capacity on the performance of the system (Section 6.2). For the distribution and storage capacity, we also provide insights into the interaction with the grid, as this is relevant for future solar park owners and legislators due to its relation to grid congestion issues. We end this section by concisely showing the impact of changing the electrolyzer capacity, conversion efficiency, price mark-up, and production capacity in Sections 6.3-6.6. In each section, we only vary the parameter that is being discussed and set the other parameter values equal to the base-case system.  We vary the distribution capacity between 1 MWh to 80 MWh per day, which corresponds to 0.04 to 3.3 MW.
In Figure 6, we see that the mean profit increases for larger distribution capacities, between 4 MWh and 80 MWh per day, because our system becomes less constrained. Distribution capacities below 4 MWh per day are infeasible for our parameter settings, due to unmet demand.
Low distribution capacities up to 10 MWh per day lead to negative profits, which is due to the limited possibility to exploit price differences as local demand should always be satisfied first. For increasing distribution capacities, the electrolyzer utilization increases up to 36.8%. This due to the exploitation of price differences. If prices are low, electricity is bought from the grid to sell it again when prices are high. Finally, the percentage of the time the distribution capacity is fully utilized with congestion at the connected cable approaches 0%, which is expected since the distribution capacity is then only constraining the system for high net production overages.  Figure 7 shows that relatively low levels of distribution capacity (e.g., k c = 5) cause a combination of buyinginduced and selling-induced congestion. This is the result of preventing future shortages and exploiting price difference opportunities. Both types of congestion follow a seasonal pattern. Whereas buying-induced congestion is highest in the winter months, selling-induced congestion is highest in the summer. This can be attributed to net production shortages that occur in the winter and net production overages that occur in the summer. In line with the results in Figure 5, this indicates that selling net overages and buying net shortages from the grid is a preferred action in general.
When the distribution capacity becomes larger (i.e., k c = 10), more selling-induced congestion occurs since the optimal policy is less impacted by possible shortages in winter. Consequently, less energy is stored and more energy is sold to exploit price differences. Buying-induced congestion simply decreases for higher distribution capacities as the distribution capacity becomes less-often the limiting factor when net-shortages occur. When capacity increases even more (i.e., k c = 20, 60), the occurrence of both types of congestion decreases, as the netproduction realizations can be sold or bought completely from the grid without being restricted by the distribution capacity. Price differences are exploited in higher quantities, while fewer congestion events are observed.   Figure 8 shows negative mean profits which increase at a marginally decreasing rate with storage capacity for each distribution capacity. For distribution capacity k c = 10 (a), the mean profits are negative due to the highly constrained distribution capacity. Note that storage capacities smaller than 300 MWh are not displayed in Figure 8 (a). This is because these capacities result in systems where it is not possible to always satisfy local demand. Here, the percentage of unmet demand ranges between 0.005% and 8% and a penalty for unmet demand is incurred. Electrolyzer utilization levels remain relatively constant (between 6.9% and 7.5%), and congestion levels are not affected by the storage capacity of 300 MWh or higher (the mean percentage time congestion remains between 38.2% and 38.3%). These results indicate that the distribution capacity is too limited across all levels of storage capacities to enable positive profit levels, more effective use of the electrolyzer and to reduce congestion issues.
Results for k c = 40 and k c = 80 in Figure 8 (b) and (c) show that congestion issues are not relevant anymore for our distribution capacity. Furthermore, increasing storage capacity to k c = 40 leads to more congestion at the cable connection, due to increasing interactions with the electricity grid. This is in line with the observed electrolyzer utilization for higher storage capacities.
Concluding, hydrogen storage used to supply electricity does not lead to profits when distribution capacity is too small and the storage owner aims to maximize profits. Moreover, seasonal storage also does not aid in reducing local congestion at the connected cable when the associated distribution capacity is too small. We advise to carefully decide upon the offered distribution capacity and the installed storage capacity, as this impacts the frequency at which distribution capacity is fully utilized, and, therefore, affects local congestion issues.

Electrolyzer capacity
We vary electrolyzer capacities k + between 2 and 50 MWh per day. Figure 9 shows that mean profits increase, at a marginally decreasing rate, for increasing electrolyzer capacity. Utilization levels decrease and range between 50% and 29.9% for k + = 2 to k + = 50. Profits are positive when the electrolyzer capacity is at least 6 MWh per day (250 kW). Without considering capital expenditures, this suggests that over dimensioning the electrolyzer capacity leads to increased profits, even though utilization levels are reduced. These results suggest that electrolyzer utilization is not a good proxy for profitability as large electrolyzer capacities with a relatively low electrolyzer utilization may lead to more profits than smaller electrolyzer capacities with a relatively high electrolyzer utilization. This only applies to operational profits as capital expenditures are not taken into account.
The mean time in which congestion occurs due to the full utilization of the connected cable ranges between 0.7 and 8.7% for these electrolyzer capacities. This is in line with what is expected, as lower electrolyzer capacities lead to a more constrained system.
A change in the type of (optimal) actions taken is observed for different electrolyzer capacities. We portray the two combinations of electrolyzer capacities in Figure 10 (k + = 2, k + = 10). It shows the mean fraction of times Given a net shortage and a low electrolyzer capacity (i.e., k + = 2), the green points in Figure 10 (a) show that buying the exact amount of the shortage is most prevalent in winter and least prevalent in summer. Other actions are almost non-existent. For a high electrolyzer capacity (i.e., k + = 10), buying more electricity than the shortage occurs 8 percent points less frequently on average than for a low electrolyzer capacity (i.e., k + = 2).
Given a net overage and a low electrolyzer capacity (i.e., k + = 2), the blue points in Figure 10 (b) show that actions in which part of the overage is sold and the remainder stored are more prevalent than for high capacity (i.e., k + = 2), indicating that storing part of an overage is mostly needed for low electrolyzer capacities to cover potential future shortages.
These results indicate that the higher electrolyzer utilization at low capacity (i.e., k + = 2) is caused by buying and storing additional electricity from the grid in times of shortages or storing in times of overages. The stored energy can be used to cover potential future shortages during times of high prices. When capacity is higher (i.e., k + = 10), the risk of supplying future potential shortages from the grid at high prices is reduced, and storing energy is not beneficial. Figure 11 shows that mean profits are positively related to conversion efficiency since storage becomes increasingly beneficial in both exploiting price differences and covering shortages that do not need to be bought from the grid. For this reason, electrolyzer utilization is also positively related to conversion efficiency. Moreover, congestion at the connected cable increases for higher conversion efficiencies due to increased interaction with the grid.

Price markup
We vary the price markup related to buying electricity from the grid (i.e., c + ) between 0 and 5. Figure 12 illustrates that mean profits are negatively related to the price markup on the buying price. This is expected because price markups on buying electricity discourage the use of storage to benefit from price differences over time. The electrolyzer utilization is reduced from 29% to 22% between price markups of 0 and 5. Reduced grid interaction as a result of higher price markups reduces congestion levels as well. This indicates that the electrolyzer is used less to buy energy from the grid for the purpose of benefiting from price differentials enabling the use of storage to prevent congestion. This confirms that price markups can be used as an instrument to reduce congestion levels in which markups can be raised until using storage is not profitable anymore.

Production capacity
We vary the production capacities of the solar park (i.e., w) between 1 and 10 MWp. Figure 13 indicates that mean profits appear to increase linearly with increased production capacities. Up to 4 MWp of the conducted experiments, profits are negative, due to a high reliance on the grid to cover shortages and the inability to store energy when prices are low to cover future shortages. Increased reliance on the grid at low production capacities is reflected in the electrolyzer utilization rates, which increase for capacities between 3 and 6 MWp and decrease for higher capacities. At low production capacities (e.g., w = 1), the electrolyzer is deployed to store energy which is bought from the grid to cover future shortages. Congestion at the connected cable increases nearly linearly for production capacities above 6 MWp. This is attributed to increased overages which are sold to the grid during summer.

Conclusion
Increased decentralization of renewable energy sources such as solar parks leads to grid congestion in rural areas where grid distribution capacity is limited. At the same time, supplying local villages in the vicinity of solar parks reduces the need for long-distance energy transportation through the electricity grid. To reduce congestion and supply electricity to a local demand of households, hydrogen storage can be an important flexibility option to bridge the seasonality gap associated with supply and a local electricity demand when external distribution capacity is limited. In this paper, we examine the problem of of the owner of a solar park with local hydrogen storage who needs to decide how much to store, sell to or buy from an external electricity grid throughout the year and can supply energy to local electricity demand by households. Furthermore, the solar energy production and local electricity demand are seasonal and there is uncertainty associated with solar electricity supply, electricity demand, and variable electricity prices in the external electricity market. We propose a Markov decision process formulation to the above problem to optimize the expected profits per year. We detail the optimal policies with regard to the period in which the actions take place (e.g., summer or winter). Moreover, we illustrate which actions are taken during overages and shortages throughout the year. We show how congestion levels and electrolyzer utilization are affected by conversion efficiency and strategic decisions such as the distribution capacity, storage capacity, and production capacity.
It is found that optimal policies are characterized by price thresholds which separate different types of actions.
These include buying the maximum possible quantity, selling exactly overages or buying exact shortages, storing overages or obtaining shortages from storage, or selling the maximum amount possible. When distribution capacity is unconstrained, storage is not used for large periods of time. When distribution capacity is constrained, local congestion at the cable to which the solar park is connected is mostly caused by buying-related actions in winter, which are needed to cover potential future shortages. Under these conditions, increasing the level of storage capacity does not reduce congestion levels, because buying actions in winter remain necessary to cover shortages. For higher levels of distribution capacity, local congestion is mostly caused by selling-related actions of the overages in the summer. Counter-intuitively, local congestion increases for increased levels of storage capacity, because this enables increasing buying-related actions to prevent future shortages and exploiting price differences.
Mean profits are highly sensitive to the level of electrolyzer capacity and appear to increase linearly with capacity. Moreover, a lower electrolyzer utilization as a result of a large capacity is associated with higher profits than a low electrolyzer capacity with a higher utilization rate as a result of interacting with the electricity grid. This indicates that a high utilization rate of the electrolyzer is not necessarily an indication of increased profits. Hydrogen storage used to supply electricity does not lead to profits when distribution capacity is too small. Moreover, storage also does not aid in reducing local congestion at the connected cable when the associated distribution capacity is too small. This is because buying actions to prevent future shortages and benefit from price differences cause buying-related congestion at the cable connection. These actions are not aimed at reducing congestion, but at maximizing profits. Higher production capacities are associated with higher profits, even though this also causes higher congestion levels due to increased selling to the grid at times of abundant supply or high prices.
The opportunities for future research are numerous and can be divided into two areas. The first area is centered around extending the model that we present in this work. For instance, new concepts arise in which local demand not only consists of electricity but also of hydrogen and, for example, demand for heat. Additionally, one could investigate the potential correlation between production and demand, or physical properties of using hydrogen as an energy carrier. Other future approaches may be focused towards minimizing congestion instead of maximizing the storage owner's profits.
The other avenue for further research is the transition towards more strategic models. For instance, one may investigate the impact of multiple renewable energy systems with co-located storage facilities in grid-congestion issues. It would be interesting to research how the location of renewable energy systems in a grid can be optimized, with the aim of minimizing grid congestion.