Hidden Power System Inflexibilities imposed by traditional unit commitment formulations

(cid:1) Quality and accuracy of traditional-energy-and power-based UCs are evaluated. (cid:1) Real-time performance evaluation simulating ‘‘perfect” stochastic UCs. (cid:1) Ideal energy-based stochastic UC formulations impose hidden system inﬂexibilities. (cid:1) A deterministic power-based UC may outperform an ideal energy-based stochastic UC. (cid:1) Power-based UC overcomes ﬂaws of energy-based UC: lower cost and wind curtailment.


Introduction
Many power systems worldwide now face a sustained and significant growth of variable and uncertain generation, such as wind and solar, driven by concerns for the environment, energy security and rising fuel prices.To maintain the supply-demand balance, enough system resources (reserves) must be scheduled in advance to compensate for possible variations in load and renewable output.The day-ahead Unit Commitment (UC) is the short-term planning process that is commonly used to schedule these resources at minimum cost, while operating the system and units within secure technical limits [1][2][3][4].
Increasing levels of variable generation demand higher flexibility from the power system.Flexibility is defined in Lannoye et al. [5] as ''the ability of a system to deploy its resources to respond to changes in net load, which is defined as the remaining system load not served by variable generation".Other definitions of flexibility can be found in Nosair and Bouffard [6].There are different ways to measure flexibility of a power system, and most of them are quantified by running UC models [3,[5][6][7][8][9].In deterministic UC models, the required amount of flexibility from the system is defined by specifying a required amount of reserves, including spinning, replacement, and regulation reserves, and, in some markets, flexible ramping products [10].The key question is then to define the optimal amount of reserves to the net load uncertainty.Assuming instead that we can solve a stochastic UC that fully describes net load uncertainties, the model can balance the benefits and costs of providing flexibility by balancing expense cost of reserving more capacity against the reduction in expected load curtailment [11].
In short, UC formulations are used to guarantee a given level of flexibility for system operations, where the underlying assumption is that the UC generation schedule can always deliver what it apparently promises [12][13][14].However, conventional day-ahead UC formulations make coarse approximations of system ramp capabilities by modeling output and loads as averaged energy levels within a large (usually 1 h) scheduling interval and imposing ramp-constraints on these average hourly levels.Consequently, energy schedules may be infeasible, as widely reported [15][16][17][18][19].In addition, traditional UC models assume that units start and end their energy production at their minimum output (or above).That is, the intrinsic units' startup and shutdown power trajectories are ignored.As a consequence, there may be a high amount of energy that is not allocated by the UC but which is nevertheless present in real time operations, thus affecting the total load balance.Disregarding these trajectories can result in inefficient realtime operations, and even endanger power system security [19,20].Although all these drawbacks suggest that traditional UC formulations may lead to unfeasible real-time operation and incorrectly characterize system flexibility, they are hidden within the UC formulations and hence are difficult to assess.
To overcome these drawbacks the power-based UC formulation proposed in Morales-Espana et al. [21], Morales-España et al. [22], and presented in Section 2.2, includes the units' startup and shutdown power trajectories, and also draws a clear distinction between power and energy, thus guaranteeing feasible energy delivery.Yang et al. [17] proposes a sub-hourly UC model guaranteeing feasible energy schedules; and although finer granularity of UC models also helps to reduce operational costs [23], their solving times increase significantly even for relatively small systems [24].None of these works, however, quantify the impact of the aforementioned drawbacks.
The goal of this paper is to reveal and quantify the impact of the above theoretical drawbacks of traditional UC formulations.This is done by following the day-ahead UC with a simulated real-time dispatch stage to check if UC solutions are actually able to supply demand in every real-time interval.For this purpose, a 5-min optimal dispatch is used to mimic actual real-time system operations.As shown in our numerical results, this 5-min real-time dispatch stage reveals that traditional energy-based UCs perform very different than expected and can even lead to infeasible real-time operation.
We show that as a direct result of using hourly-averages for energy, traditional stochastic UC is unable to manage real-time uncertainty, even if the stochastic UC considers the full range and correct probability distributions of the net load.To evaluate the performance of a ''perfect" stochastic UC, the real-time (5-min) dispatch stage uses the same net load scenarios that are considered when solving the stochastic UC.Our results demonstrate that even such an ''ideal" stochastic UC formulation imposes a hidden system inflexibility by incorrectly representing ramp capabilities and by ignoring the units' startup and shutdown trajectories.This leads to a failing to optimally exploit the actual system flexibility and consequently inefficiencies in dispatch.This demonstration is accomplished by a range of experiments.The experiments include comparisons of the results of traditional energy-based UC with power-based UC, with and without startup and shutdown trajectories.This is done for both deterministic and stochastic network-constrained UC formulations.Additional sets of experiments presented below consider the impact of alternative assumptions about wind and load flexibilities.One set of experiments analyze how negative wind bids (i.e., curtailment penalties) affect system flexibility.Another set analyzes different degrees (standard deviations) of load variation.
The remainder of this paper is organized as follows.Section 2 conceptually describes the drawbacks of energy-based UC approaches that distort the actual flexibility of the system, and summarizes an alternative, more accurate power-based UC model.Section 3 present the different UC formulations that were implemented.Section 4 presents the case studies, also provides and discusses results from the numerical experiments.Finally, conclusions are presented in Section 5.

Inherent system inflexibility imposed by UC formulations
This section summarizes the main drawbacks of energy-based and power-based UCs, and shows how these drawbacks incorrectly characterize system flexibility.

Traditional energy-based UC
Traditional UC formulations fail to deal with ramp capabilities appropriately.Inefficient ramp management arises from applying ramp-constraints to energy levels or (hourly) averaged generation levels, which is the standard practice in traditional UC models [1,25].Representing generation in a stepwise fashion (energy blocks) may lead to misleading estimates of a system's ramp availability.This in turn could leave the system unprepared to face realtime uncertainties.
As we noted above, infeasible energy generation schedules may result from energy-based scheduling [15][16][17][18][19]21].To illustrate this problem, consider the following scheduling example, modified from Morales-España et al. [22], for one generating unit.This example assumes that the minimum and maximum generation outputs of the unit are 100 MW and 300 MW, respectively, with a maximum ramp rate of 200 MW/h.It is also assumed that the unit is committed all the time, i.e., the unit is always producing at least 100 MW.Fig. 1a shows a typical result that can be expected To produce an energy output of 100 MWh for the first hour, the unit must have a constant power output of 100 MW for the complete first hour (there is no other way to produce this energy output due to the maximum and minimum output of the unit, and also because the unit cannot produce below its minimum output when committed); similarly, the unit must have a constant power output of 300 MW during the complete second hour to produce 300 MWh.However, once the unit has been producing 100 MW for the first hour, the unit is just physically capable to reach its maximum output before the end of the second hour due to its limited ramp rate, as shown in Fig. 1b.Consequently, the solution obtained in Fig. 1a by the energy-based UC is not feasible.In fact, the unit requires an infinite ramping capability to be able to reproduce the energy schedule presented in Fig. 1a.More examples of this energy infeasibility problem can be found in Guan et al. [15,16], Morales-Espana et al. [19], Morales-España [26], Philipsen et al. [27], and references therein.
In addition, assuming that the previous drawback could be addressed under the energy-based scheduling approach, this approach cannot guarantee that a given power demand profile can be supplied.To illustrate this problem consider the following example.Fig. 2 shows two power demand profiles that present the same energy profile.Notice that the two power profiles present distinctly different ramp requirements, even though the hourly energy requirements are identical.One energy profile has an infinite number of potential power profiles.Not all of the possible power profiles are necessarily feasible, even if the energy-based UC model shows the energy profile to be feasible.Thus, even though the energy-based UC could in theory provide a given energy profile, this approach cannot guarantee that the final resulting power profile can be supplied [21,26].
Another drawback is that conventional UC formulations assume that units start/end their production at their minimum output or above, see for example Hobbs et al. [1], Morales-Espana et al. [25].That is, traditional UCs ignore the production below the units' minimum output caused by their intrinsic startup and shutdown power trajectories, which are inevitably present in the real-time operation.Consequently, there is energy that is not allocated by day-ahead scheduling approaches that will affect the total load balance during real-time operation.Considering these power trajectories in the scheduling stage can significantly change commitment decisions and also decrease operating costs [20].

Power-based UC
To overcome the drawbacks of conventional energy-based UC formulations, the power-based UC formulation was proposed in Morales-Espana et al. [21], Morales-España et al. [22].The power-based model draws a clear distinction between power and energy, and it also takes into account the often neglected power trajectories that occur during the startup and shutdown processes.Although the power-based UC seeks to adequately represent the operation of generating units to efficiently exploit their flexibility and avoid infeasible energy delivery, it also imposes a level of inflexibility on the system through its approximation of net load ramps.
In particular, the power-based UC [21,22] assumes that power varies linearly from the beginning to the end of each hour.Let us consider a thermal unit that is required to increase its production from 100 MW to its maximum output (see Fig. 3).In practice, the unit would operate at its maximum ramp and therefore reach its maximum output before the end of the hour (Fig. 3b).Nevertheless, if an hourly linear power profile is considered, the model assumes that the maximum generation will be attained only at the end of the hour (Fig. 3a), leading to a deviation from the solution.This can be avoided by dividing each hour into shorter time intervals.
If startup and shutdown trajectories are ignored in a powerbased UC, the same problems described for the energy-based UC can arise.However, the power-based UC proposed in Morales-Espana et al. [21], Morales-España et al. [22] models startup and shutdown power generation in a piecewise-linear fashion, avoiding any power discontinuity during the unit's startup and shutdown processes, thus lessening the distortion.
Maximum ramp-up and ramp-down capabilities can be underestimated for fast-ramping units in the power-based UC model.This is because the maximum ramping up/down capabilities that can be modeled in one period equal the difference between Pmax and Pmin (Fig. 3).As a result, the maximum flexibility of high ramping units will not be fully exploited in power-based UC.Therefore, although power-based UC ensure feasibility over energy-based UC, it may sacrifice some optimality.In the case of energy-based UC, however, the solution cannot even guarantee feasibility, because unit ramping capabilities are generally overestimated, as explained in the previous subsection.

Unit commitment formulations
This section presents the set of constraints for the energy-based and the power-based UC formulations.The energy-based UC is the commonly used UC approach, in which the energy demand is represented using energy levels (hourly-averaged generation) in a stepwise fashion over time.All constraints involving generation levels are applied to these energy levels.
On the other hand, the power-based UC proposed in Morales-Espana et al. [21], Morales-España et al. [22] draws a clear distinction between power and energy.Demand and generation are modeled as hourly piecewise-linear functions representing their instantaneous power trajectories.The schedule of generating unit output is no longer an energy stepwise function, but a smoother piece-wise power function.

Nomenclature
Upper-case letters are used for denoting parameters and sets.Lower-case letters denote variables and indexes.

Definitions
In this paper, we use the terminology introduced in Morales-Espana et al. [20] to refer to the different unit operation states; see Fig. 5.

Online
the unit is synchronized with the system Offline the unit is not synchronized with the system Up the unit is producing above its minimum output.During the up state, the unit output is controllable

E-UC: traditional stochastic energy-based UC
In the traditional energy-based UC, energy is considered to be the direct output of generating units, as shown in Fig. 4. Also, all generating units are considered to produce either zero (when being offline) or above their P g (when being online).What is usually considered in traditional UC formulations is that units have a startup and shutdown capabilities (e.g., [25,[28][29][30]); that is, when the unit starts up (shuts down) its generating output is above its minimum output P g and below its startup (shutdown) capability SU g (SD g ), as shown in Fig. 4.Although these startup and shutdown capabilities could be valid models for quick-start units, they are not valid for slow-start units which startup and shutdown power production is below P g (see Fig. 5).That is, traditional Energy-based UC completely disregard the units' startup and shutdown power trajectories, although they are intrinsically present during actual operation [20].
The UC seeks to minimize all production costs: min 3.2.1.System-wide constraints Energy demand balance for hour t is guaranteed as follows: and power-flow transmission limits are ensured with

Individual unit constraints
The commitment, startup/shutdown logic and the minimum up/down times are guaranteed with Different startup costs are modeled depending on how long the units have been off-line.The startup type is selected with where (7) allows that the startup segment s can be selected As discussed in Morales-Espana et al. [20,25], the variables d gst take binary values even if they are defined as continuous.This is due to the tightness characteristic of the startup-cost formulation.See Morales-Espana et al. [20] for details on how the initial conditions define d gst for the first periods.
Energy production must be within the power capacity limits: where G 1 is defined as the units in G with TU g ¼ 1.
Ramping-capability limits are ensured with ÀRD g 6 e gxt À e gx;tÀ1 6 RU g 8g; x; t The total energy production for thermal and wind units are obtained as follows: b e gxt ¼ P g u gt þ e gxt 8g; x; t It is important to highlight that the set of constraints ( 4)-( 6) together with ( 9)-( 11) and ( 13) is the tightest possible representation (convex hull) for a unit operation under the energy-based scheduling approach, as proven in Gentile et al. [30].

Es-UC: stochastic energy-based UC, including startup and shutdown trajectories
The slow-start and quick-start units are now distinguished in the Es-UC formulation.The quick-start units are defined as those that can ramp up from 0 to any value between P g and SU g within one period (typically 1 h) as shown in Fig. 4 and modeled in the previous Section 3.2.They can also ramp down from any value between SD g and P g to 0 within one period.On the other hand, the slow-start units are defined as those units that require more than one period to ramp up (down) from 0 (P g ) to P g (0), see Fig. 5.For slow-start units, the power output follows a predefined power trajectory when the unit is starting up or shutting down [20].
The objective function and all the constraints for Es-UC are modeled as in the E-UC formulation in Section 3.2, except for the total energy production for slow-start thermal units (13), which is now replaced by b e gxt ¼ X Sg s¼1

ffl{zfflfflfflfflfflfflfflfflffl ffl}
Output when being up where G S is the set of units in G which are slow-start units.
Notice that (15) also models different startup power trajectories for Es depending on how long the unit has been off-line.
The minimum down time TD g is a function of the minimum offline time, i.e., TD g is equal to the (hottest) startup and shutdown duration processes (SU D g1 þ SD D g ) plus the minimum offline time of the unit.Thus avoiding the possible overlapping between the startup and shutdown trajectories.That is, constraint (6) ensures that the unit is down (u gt ¼ 0) for enough time to fit the unit's startup and shutdown power trajectories.
Similarly to quick-start units, constraints (4)-( 12) describe the operation of slow-start units during the up state (where usually SU g ; SU g ¼ P g ).For a better understanding of the modeling of slow-start units, the reader is referred to Morales-Espana et al. [20], Morales-España et al. [22].
Note that the no-load cost C NL g in ( 1) ignores the startup and shutdown periods.This is because the C NL g only multiplies the commitment during the up state u gt , see Fig. 5.In order to take into account the no-load cost during the startup and shutdown periods, gs and C SD0 g are used in (1) instead of C SU gs and C SD g , respectively, and defined as This only applies for slow-start units (since the startup and shutdown duration for quick-start units is zero in the energybased approach).

Traditional deterministic energy-based UC
The previous Sections 3.2 and 3.3 presented stochastic energybased UC formulations.For the deterministic formulation, just one scenario is modeled for wind: the nominal wind energy output e w E bt which is now defined as where f W E bt is the average wind energy available obtained from the available wind scenarios: For the deterministic case, demand is satisfied for the nominal wind scenario, replacing (2) by and to deal with uncertainty, the deterministic formulation guarantees a level of reserves requirements: where sup Á ð Þ and inf Á ð Þ are the supremum and infimum functions, respectively.Now, the energy output and reserves must be within the unit's technical limits, replacing ( 9)-( 11) by where G 1 is defined as the units in G with TU g ¼ 1.Finally, ramping constraints must also guarantee that reserves can be deployed, replacing ( 12) by 3.5.P-UC: stochastic power-based UC The power-based UC draws a clear distinction between power and energy, where power is the direct output of generating units and the energy is then obtained from the power profile, as shown in Fig. 6.
The UC seeks to minimize all production costs: min Note that the no-load cost C NL g in ( 28) ignores the startup and shutdown periods, see Fig. 6.In order to take into account the noload cost during the startup and shutdown periods, C SU0 gs and C SD0 g are introduced in (28) and defined as

System-wide constraints
Power demand balance at the end of hour t is guaranteed as follows: where (29) is a power balance at the end of hour t.Be aware that the energy balance for the whole hour is automatically achieved by satisfying the power demand at the beginning and end of each hour, and by considering a piecewise-linear power profile for demand and generation [21].
The power-flow transmission limits are ensured with:

. Individual unit constraints
The commitment, startup/shutdown logic, the minimum up/down times, and the variable startup-type logic constraints are guaranteed with ( 4)-( 8).Power production must be within the power capacity limits: and ramping-capability limits are ensured with ÀRD g 6 p gxt À p gx;tÀ1 6 RU g 8g; x; t The total power and energy production for thermal and wind units are obtained as follows: It is interesting to note that even though SU g ; SD g P P g (by definition), the resulting energy from (34) for the power-based UC may take values below P g during the startup and shutdown processes, see Fig. 6, unlike the traditional energy-based UC.
It is important to highlight that the set of constraints ( 4)-( 6) together with ( 31) and ( 33) and ( 34) is the tightest possible representation (convex hull) for a unit operation under the power-based scheduling approach, as proven in Morales-España et al. [22].

Ps-UC: stochastic power-based UC, including startup and shutdown trajectories
All constraints presented in the previous (3.5),apply for both quick-and slow-start units, except for the total power output for slow-start thermal units (33) where G S is the set of units in G which are slow-start units.Also, for slow-start units, SU g and SD g are defined to be equal to P g , for a seamless transition between the down and up states, as shown in Fig. 7.For a better understanding of the modeling of slow-start units under the power-based scheduling approach, the reader is referred to Morales-España et al. [22].
The set of constraints ( 4)-( 6) together with ( 31), (34) and (37) is the tightest possible representation (convex hull) for a slow-start unit operation (for one startup type) under the power-based scheduling approach, as proven in Morales-España et al. [22].
Similarly to the power-based quick-start units, the no-load cost C NL g in ( 28) ignores the startup and shutdown periods.In order to take this no-load cost into account, C SU0 and C SD0 are defined as and they apply for both slow-and quick-start units.For quick-start units, SU D gs ¼ SD D g ¼ 1 by definition, see Fig. 6.

Deterministic power-based UC
The previous Sections 3.5 and 3.6 presented stochastic powerbased UC formulations.For the deterministic formulation just one nominal scenario is modeled for wind.This nominal wind power output e w P bt is now defined as 0 6 e w P bt 6 f where f W P bt is the average wind power available obtained from all wind scenarios: For the deterministic case, demand is satisfied for the nominal wind scenario, replacing (29) by and to deal with uncertainty, the deterministic formulation guarantees a level of reserves requirements: Now, the power output and reserves must be within the unit's technical limits, replacing (31) by Finally, ramping constraints must also guarantee that reserves can be deployed, replacing (32) by

Numerical results and discussion
This section presents the case studies.After describing the different case studies, a nominal case is discussed.Next, additional sets of experiments are presented, considering the impact of alternative assumptions about wind and load flexibilities.One set of experiments analyze how negative wind bids (i.e., curtailment penalties) affect system flexibility.Another set analyzes different degrees (standard deviations) of load variation.Finally, we compare the performance of stochastic with deterministic UCs.

Case studies
To evaluate the performance of different UC scheduling approaches, we use the modified IEEE 118-bus test system described in Morales-España [26] for a time span of 24 h.The system has 118 buses, 186 transmission lines, 91 loads, 54 slow-start thermal units, and three buses with wind production.To this system we added 10 quick-start units that can be committed during real-time operation, unlike slow-start units.The quick-start units can produce from 0 to above the minimum output in 5 min.In short, the system has 64 thermal units in total.Load is assumed to be known, so that forecast uncertainty is entirely in wind.The total (5-min) load averages 3578.6 MW, and has a peak of 5117.5 MW and a minimum of 1435.4MW.Fig. 8 shows the 20 scenarios of wind production for one of the three buses.All system data, including the 5-min and hourly demand and wind profiles, are available online at www.iit.upcomillas.es/aramos/IEEE118-bus_10fastGen.xls.
Two different UC approaches are implemented, along several variations: the traditional energy-based UC and the power-based UC.As presented in (3), the variations include with and without startup and shutdown power trajectories, as well as deterministic and stochastic versions.Thus, there are eight UC formulations in total, one stochastic and one deterministic for each of the following cases: E-UC: traditional energy-based UC, in which startup and shutdown trajectories are not included (see Section 3.2) in the day-ahead scheduling stage, although they actually do occur in real time.Es-UC: traditional energy-based UC, including startup and shutdown trajectories (see Section 3.3) in both day-ahead scheduling and real-time operations.P-UC: power-based UC, excluding startup and shutdown trajectories (see Section 3.5) from the day-ahead, but they occur in real-time.Ps-UC: power-based UC, including startup and shutdown trajectories (see Section 3.6) in both day-ahead and real-time models.This experimental design allows us to test for the relative cost of errors stemming from three sets of approximations: energyvs.power-based UC; exclusion vs. inclusion of startup and shutdown power trajectories; and deterministic vs. stochastic UC.In addition, sensitivity analyses explore the effect of different levels of load variability and wind curtailment penalties.This section starts comparing the stochastic versions of the above four cases (E-UC, Es-UC, P-UC, Ps-UC).Subsequent subsections compare the performance of the stochastic and deterministic UC formulations, as well as the sensitivity analyses.
All optimizations were carried out using CPLEX 12.6.1 on an Intel-Xeon (64-bit) 3.7-GHz personal computer with 16 GB of RAM memory.The problems are solved until they hit a time limit of 2 h or until they reach an optimality tolerance of 0.05% (none of the UC problems exceeded the time limit).
To observe that hidden inflexibilities imposed by the different UC formulations, we carry out an in-sample evaluation of the UC policies.That is, in the stochastic UC case, the ideal stochastic UC formulations are mimicked by evaluating their performance in real-time (5 min) dispatch using the same scenarios used in the scheduling stage.By doing this, we can show the problems that are related with the formulations rather than with the representation of the uncertainty itself, because in the in-sample evaluation, the stochastic UC formulations have perfect information about the uncertainty distributions. 1o assess the performance of the different network-constrained UC approaches, we differentiate between the scheduling stage and the real-time dispatch stage.In the scheduling stage, the 20 wind scenarios in Fig. 8 are used to solve the stochastic UC problems and to obtain a single commitment schedule for the 54 slowstart units for a time span of 24 h.On average, the 20 scenarios represent 24.4% of the energy demand, with the individual scenarios providing 21-29% of the demand.In the real-time dispatch stage, the single slow-start unit commitment result from the scheduling stage is fixed (i.e., taken as a constraint), and the real-time (5 min) dispatch as well as commitment decisions for 10 quick-start units are optimized for each of the 20 individual wind scenarios using a network-constrained economic dispatch/quick-start commitment problem.The dispatch stage mimics the actual real-time system operation in which generating units are dispatched to supply the demand every 5 min, while commitment decisions are allowed to be taken for the quick-start units every 15 min.This is an approximation of the California ISO market design, in which the fifteen minute market includes both dispatch and quick-start commitment decisions, and a 5 min market is dispatch-only.To represent the high costs due to corrective actions in real-time operations, we introduce penalty costs of 10000 $/MWh and 5000 $/MWh for violations of supply-demand balances and transmission-limits, respectively [32].

Nominal case
The nominal case consists of the stochastic UC formulation using the 20 wind scenarios presented in Fig. 8 under the nominal assumptions about demand variability and wind bidding.Equal probabilities are assumed for all the scenarios.We assume that the wind units submit negative bids of À50 $/MWh.In this paper, negative bids are used to solve the UC scheduling and dispatch problems, but the resulting negative costs are excluded from the total costs (see TC in Table 1) to separate the effect of the quantity of curtailment from the total costs, thus providing an insight of the actual fuel costs that were incurred during the operation.Certain subsidies, such as feed-in-tariffs, motivate renewable energy generators to submit negative price offers.This practice may increase system operation costs and even emissions due to lessened flexibility.In particular, larger negative bids for wind increases that resource's dispatch priority, which may in turn produce more startups of conventional plants, higher costs and higher emissions than in the case of having a more flexible system [33].The reason for excluding negative bids from reported total cost is that, at least in the short-run, renewable subsidies represent income transfers from taxpayers or ratepayers to renewable producers, and are not an actual social cost.
Table 1 shows the performance of the different stochastic UC formulations in terms of eight metrics, three related to the dayahead scheduling stage, three to real-time dispatch, and two comparing the day-ahead schedules with the actual dispatch.The three scheduling stage metrics: (1) Total (scenario-averaged) production costs (TC); excluding negative wind bids, obtained from the optimal UC solution; (2) the fixed commitment costs (FxdCost) for the 54 long-start units, including non-load, startup and shutdown costs; and (3) percentage of potential wind production curtailed (Curt).In the real-time dispatch/quick-start commitment stage, the metrics include: (4) the average of the total production costs including wind curtailment penalties (negative bids) (TCW); ( 5) average total costs excluding curtailment penalties (TC); and (6) percentage of wind production curtailed (Curt).The final two metrics compare the outcomes predicted by the scheduling model vs. what was actually realized in the real-time dispatch stage (Sch vs Rtd): (7) the ratio of the TC obtained from the dispatch stage to that predicted in the scheduling stage, excluding wind penalties (TC rtd/sch); and (8) the ratio of the dispatch stage (actual) curtailment to that predicted by the scheduling stage (Curt rtd/sch).
In terms of overall economic efficiency, we should not be surprised that Ps-UC (the scheduling model with the fewest approximations) has the lowest actually realized value of the objective function ($766,100 for TCW).Considering startup and shutdown trajectories improved the day-ahead UC schedule more than considering power rather than energy (For instance, Ps-UC reduces TCW by $26.95k compared to P-UC, and by $8.31k compared to Es-UC).When wind penalties are excluded, the Ps-UC solutions remain the best, although the margins are generally shrunk.
We now take a more detailed look at the solutions.From the scheduling stage in Table 1, it can be observed that the Es-UC and Ps-UC (UC formulations including startup and shutdown trajectories) calculate higher fixed costs day-ahead than the E-UC and P-UC (UC formulations without startup and shutdown trajectories).This is because Es-UC and Ps-UC commit more resources to accommodate the (inflexible) startup and shutdown trajectories of the committed units.Conversely, the total costs (TC) that are anticipated day-ahead are lower for the case of Es-UC and Ps-UC because these UCs optimally allocate the startup and shutdown trajectories to also meet part of the demand.
From the scheduling stage, one can also observe that UCs without startup and shutdown trajectories are expected to accommodate more (curtail less) wind than UCs with startup and shutdown trajectories.The reasoning behind this is that units are not flexible when starting up and shutting down and this adds a level of inflexibility to the problem.Additionally, when startup and shutdown trajectories are not included, units are assumed to produce from 0 to an output equal or above their minimum capacity within one hour, which seemingly (but not actually) makes the system more flexible to accommodate large changes in wind or load.
In the results of the real-time dispatch stage in Table 1, it can be observed that the TC difference between Es-UC and E-UC is less significant than what was expected from the scheduling stage.The same happens with the difference between Ps-UC and P-UC.Although E-UC and P-UC were expected to present less curtailment than Es and Ps, thanks to their presumed higher level of flexibility, the opposite was encountered in the real-time dispatch.Notice that the level of curtailment of E-UC in the real-time dispatch stage (8.06%) is more than 6 times higher than in the scheduling stage (1.33%).Less curtailment was found in Es-UC, where the level of curtailment of Es-UC in the real-time dispatch stage (5.11%) was twice that in the scheduling stage (2.53%).Both in the energybased and in the power-based formulations, this paradoxical result derives from the fact that when the startup and shutdown trajectories are ignored in the scheduling stage, reserves that were supposed to be used to manage wind are used instead to accommodate the startup and shutdown trajectories.Furthermore, TC increased in the case of Es-UC and Ps-UC for the real-time dispatch when compared to the scheduling stage (ratios greater than 1), whereas the TC for E-UC and P-UC decreased because the latter did not recognize that startup and shutdown energy would displace other sources.However, curtailment levels increased by larger proportions for E-UC and P-UC than in the case of Es-UC and Ps-UC.Bear in mind that the TC values in Table 1 do not include the negative bid of the wind production (but the negative bid is included to solve the UC problem), otherwise E-UC and P-UC would show higher values than in the scheduling stage.
It should be noted that curtailment was much greater in the real-time dispatch stage than in the UC stage, especially for UCs without startup and shutdown trajectories.As shown in the last column of Table 1, all the curtailment ratios between the results of the real-time dispatch and the scheduling stage were higher than 1.The degree of unexpected curtailment in the dispatch stage indicates how well the UCs scheduled the extra resources (reserves) to provide flexibility to the system.A value of one means that the hourly scheduling day-head perfectly predicted the 5 min dispatch curtailment.A value greater than one means that the resources/reserves that were scheduled to deal with wind are in the end used to deal with other inflexibilities of the system that were not considered in the UC scheduling stage (hidden inflexibility).A value lower than one means that the system had actually over-scheduled resources to deal with the planned level of uncertainty.
Based on this comparison of results from the scheduling and real-time dispatch stages for the four different UC formulations of the nominal case, we can conclude the following: (1) Stochastic UCs: Bear in mind that the real-time dispatch stage uses the same scenarios that were used in the scheduling stage.Therefore, the stochastic formulations might be expected to present an optimal performance since they are being evaluated using the same in-sample scenarios.That is, the curtailment and total cost during in the real-time dispatch stage is expected to be the same as in the scheduling stage.However, these stochastic UCs are not able to face the perfectly known conditions (excluding within-hour power variability), leading to unplanned and inefficient use of resources to manage deterministic events that were ignored in the scheduling stage.(2) Startup and shutdown power trajectories: UCs ignoring these trajectories presume that there is a level of flexibility that the units actually do not have.This inevitable leads to higher curtailment in real-time operation than anticipated in the scheduling stage.(3) Energy-based UC: once startup and shutdown trajectories are included in the energy-based UC (Es-UC), a curtailment ratio above 1 still occurs, indicating that there is some inflexibility (not related to the startup and shutdown trajectories) that is hidden in the energy-based scheduling formulation.As discussed in Section 2, the traditional energybased UC over-estimates the units' ramping capabilities and cannot guarantee that the commitment decisions can actually provide the resulting energy schedule, hence extra resources are needed in real-time operation to compensate this ramping over-estimation.This was not reflected in load curtailment or transmission violation penalties, as all realtime dispatches were feasible; instead, additional quickstart units were started up or wind was curtailed, increasing costs.(4) Power-based UC: Ps-UC results in the most accurate scheduling-stage estimate of wind curtailment in the scheduling compared to real-time dispatch (rtd/sch = 1.08).
It is important to acknowledge that the curtailment ratio increases due to any intra-hour variation that could not be taken into account into the hourly UC model.However, this ratio could also decrease because the hourly Ps-UC underestimates ramp capabilities for faster ramping units (Section 2), which are fully exploited during real-time system operation.

Different bids by wind
We now consider the sensitivity cases (wind bids, demand variation, and deterministic UC models).
This section shows the wind curtailment and operational costs that are obtained from the UC formulations under alternative wind bids.The results are shown by means of boxplots.A boxplot should be interpreted as follows.The central mark inside each box represents the median, while the black dot is the average.The edges of the box provide the 25th and 75th percentiles.The whiskers indicate the most extreme data points (that are not considered to be outliers).Finally, outliers are plotted individually.
Fig. 9 shows the results for each UC formulation.Note that each column in the graph (white or gray) represents a different negative wind bid.As mentioned previously, negative bidding diminishes system flexibility as a result of conferring wind a higher dispatch priority, which makes thermal generation vary more in compensation [33].
From Fig. 9 it can be observed that for all UC formulations, the operating costs generally increase as the negative wind bids increase in magnitude.Moreover, the differences in curtailment percentages between the formulations that include startup and shutdown trajectories and those that exclude them becomes more significant as the wind bid decreases.Ps-UC tends to present the lowest operating costs with exception of the case when the wind bid is equal to À500 $/MWh, in which E-UC and P-UC present lower total costs.However, as shown in Fig. 10, when combining the operation costs and the curtailment penalization in the total costs (TCW), Ps-UC performs the best followed by Es-UC, while E-UC performs the worst, followed by P-UC.
In short, the more inflexible the system becomes (lower wind bids), the better the performance of Ps-UC and Es-UC become with respect to P-UC and E-UC, respectively.Furthermore, Ps-UC also gains an additional advantage over Es-UC when more flexibility is demanded.This can also be observed in Table 2, where the TC and Curt ratios between real-time dispatch and scheduling stages are shown.Notice that Ps-UC shows the most accuracy overall (i.e., ratios closest to 1).Ps even shows its best Curt ratio performance for the lowest wind bid (À500) whereas the performance of the others deteriorated under larger bids.This means that Ps better exploits the flexibility of the system to face wind uncertainty under different levels of the wind bid.

Different standard deviations of demand
This subsection shows the performance of the UC formulations under different variations of load over the day.For the nominal case, the difference between the peak, and offpeak of the demand is 3682.1 MW; this range is varied by factors between 0.5 and 1.5.The results can be observed in Fig. 11.As expected, Ps-UC and Es-UC present lower curtailment, consistent with results discussed above.Ps-UC and Es-UC also tend to show lower TC.This is even clearer when adding the curtailment penalization to the total costs (TCW), as shown in Fig. 12.
Demand-balance violations are experienced for the first time in the E-UC solutions, which are infeasible for cases in which the demand is more variable (ratios of 1.25 and 1.50), while P-UC is infeasible for the highest ratio case (1.50).This is due to the steep demand profile that results from having a higher difference between the peak and off-peak of the demand.During the offpeak period most generators were offline, but should have rapidly come on-line to supply the steep demand ramp.In the real-time operation, the startup and shutdown trajectories of the 54 slowstart units add up in a way that the startup, shutdown and the minimum output of the online units are greater than the demand, causing negative demand-balance violations.This is a consequence of ignoring the startup and shutdown trajectories in the scheduling stage.Meanwhile, Es-UC and Ps-UC optimally schedule the units and their startup (and shutdown) trajectories to supply the steep demand.A similar phenomenon might be expected when the demand is at peak values and rapidly decreases; however, during the peak demands the deficit of generation can in fact be supplied by quick-start units.
Es-UC was also infeasible for the steepest demand case, in which the peak/off-peak demand difference equaled 1.5 times the nominal difference.In this case, the infeasibility is not caused by the startup and shutdown trajectories, because Es-UC takes them into account.Instead, the infeasibility results from overestimating the ramp-up capability of the units, which is one of the drawbacks of the traditional energy-based scheduling approaches, as discussed in Section 2.1.In other words, the energy approach schedules a steeply ramping energy profile that cannot be delivered in real-time operation.
The results can also be observed in Table 3. Ps-UC shows the best results overall.Not only there are no infeasibilities in this case, but the cost ratio remains close to 1 for the different variations of demands.Moreover, the curtailment ratio is the best for most of all cases.As already mentioned, the curtailment ratio gives an indication of how optimally the flexibility of the system is scheduled.A value of one means that everything went as optimally as planned    for the day-ahead UC, and this is the case for Ps-UC, all values are close to 1.

Stochastic vs. deterministic UC
This section compares the real-time dispatch performance of deterministic UCs with stochastic UCs.The deterministic paradigm uses the mean wind production (averaged over the 20 scenarios in Fig. 8) in the scheduling model, but still uses the actual 20 realizations in the dispatch stage to evaluate the performance of the dayahead commitment.Unlike the stochastic model, the deterministic energy-based (Section 3.4) and power-based (Section 3.7) UC include two hourly reserves constraints, upwards and downwards, which are defined as, respectively, the mean wind production minus the minimum wind envelope, and the maximum wind envelope minus the nominal wind production, respectively (as modeled in (20) and ( 21) for E-UC and Es-UC, and in (42) and (43) for P-UC and Ps-UC).It is assumed that reserved capacity can be used to generate energy in the real-time market based on their costs, similar to the California ISO and MISO flexible ramping products [10].
In the nominal case, Fig. 13 shows that for a given UC, the stochastic formulation performs better during real-time operation than the deterministic formulation in terms of both costs.This is as expected, since the stochastic model optimizes the reserves considering all 20 possible wind scenarios.However, he following two extreme cases, in demand variability and wind bid, did not perform as expected and hence we discuss them in more detail as follows.

Maximum wind inflexibility
Fig. 14 compares the deterministic and stochastic UC models with the lowest negative bid (WB = À500 $/MWh) which demands more flexibility from the system.One can observe that the deter-ministic Ps-UC formulation provides a better result than the stochastic Es-UC formulation.This means that even though a stochastic approach outperforms a deterministic approach in general, the fact that the stochastic Es-UC formulation doesn't account for the hidden inflexibility discussed in Section 2, makes the Ps-UC deterministic approach to outperform the Es-UC stochastic approach.Not only the costs are lower for the Ps-UC deterministic approach, but the curtailment is also lower, which again is a result of a UC formulation that better represents the flexibility of the power system.

Maximum demand variation
Fig. 15 portrays the deterministic and stochastic UC results in the case of the highest demand variation (1.5 times the maximum-minimum load difference over the day).Noteworthy is that the deterministic Es-UC formulation provided better results than the stochastic Es-UC.The reason for this is that the Es-UC stochastic formulation had supply-demand balance infeasibilities that were not present in the Es-UC deterministic formulation.In the deterministic formulation a higher amount of reserves was imposed, whereas in the stochastic approach the level of reserves was lower because they are optimized in the UC stage.Because of the infeasibilities that were actually encountered in real time due to a shortage of reserves, the overall costs were higher for the Es-UC stochastic formulation.The deterministic formulation is usually less optimal in this paper, but in this particular case, thanks to having more reserves that provide flexibility, the deterministic formulation was able to cope with the higher demand variability, and in this way outperform the stochastic formulation.Similar phenomena were encountered for E and P, as shown in Fig. 16.
In short, if the UC formulation is not properly defined, it is even better to choose a deterministic approach that considers enough reserves to deal with the formulation's hidden inflexibilities, than  to choose a stochastic approach that could present infeasibilities due to flaws in the formulation.

Conclusions
We have shown how the performance of traditional energybased UC formulations could be more affected by inaccurate system representations than by wind uncertainty itself.This was demonstrated by comparing alternative formulations of the dayahead hourly commitment problem (energy-vs.power-based; excluding or including startup and shutdown trajectories; and deterministic vs. stochastic), and evaluating the quality and accuracy of their schedules through simulation of real-time dispatch and quick-start unit commitment in response to wind and load variations with a 5 min granularity.In the real-time (5-min) dispatch stage, the commitment decisions of stochastic day-ahead UC models were evaluated using the same net load scenarios that were considered when solving the stochastic UC, thus simulating the case of ''perfect" stochastic UCs, which accurately represent the probabilities of net load.
The results demonstrate that even such an ''ideal" energy-based stochastic UC formulation imposes a hidden system inflexibility, because the traditional energy-based UC formulation incorrectly represents ramp capabilities and also disregards the intrinsic units' startup and shutdown trajectories.In fact, if the formulation is not properly defined, it can be better to choose a deterministic approach that requires enough reserves than to use a stochastic approach that could present infeasibilities due to flaws in the formulation.Results also showed that a power-based deterministic formulation, including startup and shutdown trajectories, can outperform an energy-based stochastic formulation, if sufficient reserves are procured in the deterministic run.In general, a power-based UC formulation with startup and shutdown trajectories resulted in the schedules with the least expected actual cost and wind curtailments than energy-based UC.This is especially the case when more flexibility is demanded by the system, for instance due to high demand variability and larger negative wind bids.
There are many interesting directions for future research.For example, it would be interesting to carry out analyses, similar to those presented in this paper, on real electric systems, thus quantifying the impact of these hidden inflexibilities on current electric systems.It would be interesting to model a robust power-based UC and compare it against traditional robust energy-based UCs.Although the natural conservatives of the robust approach could help to protect the schedule against the imposed energy-based infeasibilities, it would be expected that the robust power-based UC will avoid these hidden inflexibilities since the beginning.It would also be interesting to compare the power-based and the energy-based approaches under sub-hourly schedules, what period duration would make them similar?what period duration would make the energy-based match the performance of the powerbased approach?Another interesting research direction would be to create a market based on power products instead of energy products, thus avoiding the hidden system inflexibility imposed by the energy-based scheduling approach.That is, the powerbased approach has been proposed to produce optimal schedules for (power) quantities, but it remains unknown how to price these obtained quantities.

Fig. 5 .
Fig. 5. Unit's operation under the energy-based scheduling approach, including startup and shutdown trajectories.

Fig. 7 .
Fig.7.Unit's operation under the power-based scheduling approach, including startup and shutdown trajectories.

EFig. 9 .
Fig. 9. Performance of different UCs under different negative wind bids: Total cost and wind curtailment.

EFig. 10 .
Fig. 10.Performance of different UCs under different negative wind bids.Total cost including wind curtailment penalization.

Fig. 12 .
Fig. 12. Performance of different UCs under different deviations of demand.

Fig. 15 .
Fig. 15.Stochastic vs. deterministic UC: variation of 1.5 the standard deviation of the nominal demand.
, which is now replaced by

Table 1
Performance of different stochastic UC policies.
a 'Sch' denotes the scheduling stage, 'Rdt' denotes the real-time dispatch stage.

Table 2
Ratios of total cost and wind curtailment under different wind bids (WB).Performance of UCs under different deviations of demand.

Table 3
Ratios of total cost and wind curtailment under different demand variabilities (R, ratio of max-min load relative to nominal case).