Strategic Bidding of Retailers in Wholesale Markets: Continuous Intraday Markets and Hybrid Forecast Methods

The deregulation process of the electricity sector has led to competition in wholesale and retail markets. In particular, retailers submit bids to wholesale markets to satisfy the energy needs associated with portfolios of end-use customers. This paper describes a strategic process for retailers bidding in a wholesale market composed of a day-ahead market, an intraday market, and a balancing market. It considers a market design that involves a hybrid model for the intraday market, based on daily auctions and a continuous procedure. The paper also presents a computational study to illustrate and test both the market design and the strategic bidding process of retailers. The results confirm the advantages of considering a continuous intraday market, show that bidding in short-term markets is more beneficial than bidding in medium-term markets, and indicate important aspects to consider when selecting customers to add to the portfolios of retailers.


Introduction
The liberalization process has led to competition in wholesale and retail markets [1]. Market players now have the option to trade electricity in three different (sub-)markets: spot, bilateral, and non-organized markets (private bilateral contracts). In spot markets, players can submit bids to electricity pools based on day-ahead and intraday auctions. In bilateral markets, players can sign standard financial and physical contracts to hedge against spot price volatility. In non-standard markets, players can negotiate privately and set the terms and conditions of bilateral contracts (see, e.g., [2,3]).
Wholesale markets were designed when the vast majority of generation units were controllable. Thus, market players that experience difficulties in complying with a dispatch procedure may incur real-time deviations from schedules. Such players-called balance responsible parties (BRPs)-may need to pay/receive the down/up balancing costs [4]. For this reason, and also other important reasons, there is a growing need to adapt market rules to the new market realities (see [5][6][7]). In 2019, the European Commission published new rules for the internal market for electricity [7][8][9]. For wholesale markets, the new rules focus on several aspects, including short term markets to improve competition and liquidity. The lead time-that is, the time between the end of a trading session and the start of a delivery period-is a key issue [10]. Ideally, it should be lower than or equal to 1 h (e.g., 5, 15, 30 or 60 min). Another key issue is the structure and organization of the intraday market. Traditionally, this market operates by considering several daily sessions based on auctions. However, in the past few years, the EU published new operational rules for the intraday market [11]. Among other aspects, the rules define a continuous trading procedure throughout the day, which can be complemented with regional intraday auctions. This raises important questions (e.g., 'Should EU countries consider a continuous intraday market only?', 'Should they complement the continuous market with intraday auctions?', and 'How to integrate the continuous market with the auctions?').
In retail markets, retailers can sign bilateral contracts with consumers [12]. They usually follow a business-as-usual strategy, proposing high tariffs that are equal in each consumer segment. Additionally, they often consider a high-risk premium, meaning that the energy element of tariffs is substantially higher than the wholesale price of electricity. Such a risk premium depends on the risk attitude (also known as risk preference or risk appetite), which can be characterized as risk-averse, risk-neutral and risk-seeking. To avoid unexpected losses, a key aspect that retailers need to consider when proposing tariffs to consumers is the forecast of market prices [13]. Another important aspect is the forecast of the energy needs associated with their portfolios. Consumption variability is dependent on meteorological conditions, consumption days (weekdays, holidays or weekends), and type of consumers (residential, commercial or industrial). Minimizing this variability is a good measure to avoid errors associated with bids submitted to the wholesale market, which can result in the payment of penalties. Thus, retailers should consider an appropriate process to mitigate such errors [14].
This paper focuses on both market design and the strategic bidding process of retailers. It extends the work presented in [15] by considering the following: (i) a continuous intraday market, where players can negotiate 1h-ahead of real-time, and (ii) a slightly updated forecast method for submitting bids to the intraday market. It considers a wholesale market composed of a day-ahead (sub-)market, an intraday (sub-)market, and a balancing (sub-)market. The intraday market is assumed to operate continuously, seven days a week, all year around. This market is also assumed to involve six daily sessions based on auctions. The rationale for this assumption is simply to consider the traditional operation of the Iberian intraday market. Thus, for simplicity, we consider a market design based on the traditional Iberian market design, extended with a continuous intraday market. We note, however, that future work will investigate new market designs based on different intraday auctions and their close interaction with the continuous intraday market. We also note that future work will examine market design from different points of view, i.e., other than that of retailers (as this work does). This includes, naturally, the perspective of renewable energy producers. The purpose of the paper is threefold: To analyse a hybrid model for the intraday market, based on daily auctions and a continuous procedure.

2.
To analyse a strategic process for retailers submitting bids to the wholesale market.

3.
To present a computational study that illustrates and tests both the aforementioned market design and the strategic bidding process of retailers; the study involves six retailer agents with different risk attitudes.
The work presented here refines and extends our previous work on electricity markets [7,10], portfolio optimization [12,13], risk management [3,14,16], and strategic bidding [15]. The experimental work was carried out with the help of the MATREM system [17]. The remainder of the paper is structured as follows. Section 2 presents a literature review of the strategic bidding of retailers in wholesale markets. Section 3 presents an overview of bilateral contracting, risk management and portfolio optimization. Section 4 describes a model for the strategic bidding of retailers. Section 5 presents the computational study. Finally, concluding remarks are presented in Section 6.

Literature Review
Hatami et al. [18] developed a decision support framework for retailers that need to buy electricity to satisfy their customers. The main goal of retailers was to determine the best tariffs by considering different options, notably spot markets, forward contracts, call options and self-generating facilities. The optimization process aimed at increasing profit and reducing the risk inherent to portfolios. The authors considered a fixed number of clients, using a fixed demand and the associated elasticity. They did not consider real consumption data. Hatami et al. [19] extended this work by considering interrupted contracts and an elasticity matrix that considers different load elasticities for each time-ofuse block.
Kettenun et al. [20] described retailers with different risk attitudes to obtain an optimal portfolio of forward contracts. The retailers faced risks associated with spot price volatility and consumption variability. The authors considered a strong correlation between spot prices and the demand of retailers, which can be true for large retailers, but not necessarily for all retailers. Their study considered the demand from the whole Nord Pool, a pan-European power exchange, meaning that the effectiveness of the model was not tested with typical retailers. They concluded that risk-neutral retailers are more concerned with price-related uncertainty, while risk-averse retailers with forward contracts hedge against spot price volatility. Nojavan et al. [21] presented an optimization model to minimize the electricity cost of retailers in wholesale markets. The authors considered a day-ahead market, forward contracts, and demand-response programs. They concluded that riskaverse retailers tend to trade large quantities of energy by signing forward contracts as a risk mitigation measure to hedge against the volatility of the day-ahead price.
Ayón et al. [22] indicated that large and non-uniform portfolios of end-use consumers may reduce demand forecast errors. The authors also stated that aggregations of flexible demand are typically beneficial to reduce demand forecast errors, increasing the return of aggregators. Wei et al. [23] presented a complete review of 128 forecast models for energy demand. They considered a mean absolute percentage error (MAPE) of 10% as a threshold for highly accurate forecasts. They concluded that small-scale forecasts (e.g., a consumer or building) have larger errors than large-scale forecasts (e.g., national level consumption). They also concluded that the forecast accuracy increases with the time horizon-that is, long-term (yearly) and medium-term (monthly or quarterly) forecasts have smaller errors than short-term forecasts (daily or sub-hourly). Koponen et al. [24] presented a review of 12 models to forecast the short-term electricity demand. The authors used these models in six different scenarios. They concluded that the normalized root mean square error (NRMSE) of the forecast decreases with the number of aggregated consumers. They also stated that hybrid methods should be used to compute demand forecasts.
Shah et al. [25] compared different demand and price forecast approaches. The authors considered the following two components in their forecasting methodology: a deterministic component based on parametric and non-parametric approaches, and a stochastic component based on univariate and multivariate vector autoregressive models. The autoregressive models were estimated using the following four methods: least squares, Lasso, Ridge, and Elastic-net. They tested these approaches in the day-ahead market of the North-European power exchange. They concluded that a multivariate vector autoregressive model using an Elastic-net estimator leads to the lowest demand forecast errors with a MAPE of 2.017%.

Bilateral Contracting, Risk Management and Portfolio Optimization
Sellers and buyers of electricity negotiate bilateral contracts to hedge against spot prices volatility. Standard bilateral contracts can be physical or financial and include the following [3,5]: forwards, futures, options, and swaps (also known as contracts for differences). For non-standard contracts, several terms and conditions can be negotiated, such as energy price, energy quantity and duration (see, e.g., [2]). Although these contracts are used for risk hedging, they also involve some risk.
Sellers and buyers can follow a risk management process in order to select the best options to trade electricity, which involves the following three phases [26,27]: • Risk assessment phase: players recognize the risk factors and identify the deterministic and stochastic variables. • Risk characterization phase: risk is measured using different methods, such as correlation, regression, value-at-risk (VaR), or conditional VaR (CVaR); VaR measures the potential losses of investors to a certain degree of confidence in a given time interval; CVaR has a higher dimension than VaR, because it considers the case when the worst scenario is surpassed; VaR only computes the expected loss of the worst scenario.
• Risk mitigation phase: players select the best set of market products to reduce the risk of their transactions.
Retailers aim to optimize the risk-return output of their portfolios. To this end, they sign bilateral contracts with key consumers and select the best set of market options to trade energy. In [12], we introduce an optimization model to select key consumers for the portfolios of retailers. The model constitutes a basis for the present (conceptual) work and we describe some important aspects of it in the following section. The model considers the three aforementioned phases of the risk management process and the Markowitz theory to define an efficient frontier. In the first phase, retailers face the following risk factors: market price volatility and consumption variability. In the second phase, retailers use the VaR to analyse how these risk factors can affect their return. In the third phase, retailers optimize the share of consumers in their portfolios by considering their attitude towards risk and the electricity tariff proposed to consumers. The efficient frontier is defined by considering the risk factors defined in the first phase, as well as the risk analysis carried out in the second phase and the efficient points obtained in the third phase. Retailers obtain the efficient frontier from the efficient points (a particular point is considered efficient if no other point can surpass its result in terms of risk or return). The electricity tariff involves two parts, namely a fixed price for contracted power, and a variable price per unit of consumed electricity (which, in turn, is divided into several parts). Notice that the most common tariffs for household consumers are single tariffs (equal during all day-periods), two-rate tariffs (involving peak and off-peak), three-rate tariffs (involving peak, intermediate and off-peak), and four-rate tariffs (involving peak, intermediate, off-peak and super off-peak). Commercial and industrial consumers connected to high or very-high voltage grids can also choose other tariffs.
Additionally, in [12], we present several strategies that can be adopted by retailers to negotiate with customers. In particular, the "Equal Return OPtimization" strategy (EROP) defines the minimum tariff that retailers may offer to each consumer to receive an equal target return from each one. The "Equal Tariff Optimization at a Minimum Return" strategy (ETOMinR) is not personalized, meaning that it considers the same tariff for all consumers. Retailers compute a tariff for each consumer and select the highest one because it guarantees a minimum target return. The "Equal Tariff Optimization at a Maximum Return" strategy (ETOMaxR) is also not personalized. To guarantee a maximum target return from all consumers, retailers compute a tariff for each consumer, and the minimum tariff between all the computed tariffs is selected. This pricing strategy may be used when regulators define a maximum return that retailers cannot surpass. The "Equal Return Tariff based on Market-Costs" strategy (ERTMC) reflects the expected costs of retailers with each consumer, considering their consumption patterns. These strategies can be selected by considering the type of tariff (i.e., two-rate, three-rate, four-rate, and so on), the equality of tariff (i.e., personalized or not), and the equality of return (i.e., if each consumer presents a similar return for retailers).
The forecast method presented in [12], and adopted in this work, consists of a Multivariate Time series (MTS) that uses the wholesale market prices of electricity, the electricity consumption, and the share of renewable energy associated with electricity production to predict the arithmetic cost of electricity for each consumer. Additionally, future predictions of electricity consumption are computed by adopting an MTS forecast method from [13], involving electricity consumption, retail electricity prices, and gross domestic products. The so-called Calinski-Harabasz (CH) criterion computes the Euclidean distance between different clusters and compares it with the internal sum of squared errors for each cluster. The optimal number of profiles (clusters) is obtained by considering the consumption profile of each consumer and using the CH criterion. Each cluster represents the consumption segment of each consumer, with a typical load profile. The division of consumers by profile is computed by using K-means clustering, a robust algorithm that minimizes the distance between each point to the centre of its respective cluster.

Strategic Bidding in Wholesale Markets
Retailers submit bids to the spot day-ahead (DAM) and intraday markets (IDM), and to the continuous intraday market. The day-ahead market is used to buy/sell potential needs/excesses of energy. Each session of the IDM can be used to compensate for the expected imbalances between the energy traded and consumed. The continuous intraday market can be used to make adjustments close to real-time operation. Additionally, retailers can sign bilateral contracts to acquire energy to satisfy the needs associated with their portfolios. This section outlines the strategic bidding process of retailers (see also Figure 1). For convenience, the process is described as a sequence of steps (step 1 to step 7). 1. Retailers determine the consumption forecast for guiding the submission of bids to the DAM. For each consumer, j, they select the past day,D, with the minimum Euclidean distance, d, between the historical weather data of a particular past day, W D−i , and the weather forecast of the target day,Ŵ D , by considering one or more weather variables (e.g., temperature and humidity): They compute the expected consumption,q j,D , by determining the consumption of the past day, q j,D , and comparing the consumption forecast of the current year,q t , with the consumption of the past year q t−1 :q They compute the hourly consumption forecast of the portfolio,q D,h , by adding the forecasts of each consumer for each time period, h, of day D: whereq j,D,h is the forecast of consumer j.
2. Retailers determine the bids submitted to the DAM by taking into account the energy of the whole portfolio, q 0,h . For each time period, they can sign several contracts, Ct, involving an energy quantity, q c,h , so that: where q c ct ,h is the energy associated with contract ct. For a particular time period, in case the quantity of energy guaranteed through bilateral contracts, q c,h , is higher than the energy forecasted,q D,h , retailers try to sell the excess in a specific session of the the IDM. The associated bids are defined by following a simple strategy. If retailers are buying energy they offer the price-cap (maximum price) of the market to guarantee that they buy the energy required to satisfy the needs associated with their portfolios. Otherwise, they offer the price of the bilateral contracts to avoid economic losses. 3. Retailers compute the consumption forecast,q s,j,h , for bidding at each session, s, of the IDM, by considering a procedure that considers the meteorological conditions, the bids submitted to the DAM, x j,h , the growth-rate method for a short-run period of 1 to 7 h, y j,h , and the average consumption behaviour of each consumer between adjacent time periods, z j,h :q where j denotes a specific consumer and J the total number of consumers. For time period, h, this formula considers the real consumption of the previous time period, if it exists, or its forecast,q s,j,h−1 , and also the weight, w, of a particular method from a set of three methods. One method considers the bids submitted to the DAM,q j,D,h−1 and q j,D,h , at time periods h − 1 and h, respectively: x j,h =q j,D,ĥ q j,D,h−1 Another method considers the traditional behaviour of each consumer (obtained by analyzing the historical consumption data): The last method considers the traditional behaviour and the last consumption tendency (to increase/decrease) of each consumer: 4. Retailers compute the bids to submit to each intraday session, q s,h , at time period, h, by considering the short-run forecasts and all energy traded in the DAM, q 0,h , in the previous intraday sessions, q i,h , and by bilateral contracts, q c,h : where s denotes a specific intraday session.

5.
Retailers compute the bids to submit to the continuous intraday market. The energy price, P cont,h , is assumed to be the day-ahead price, P 0,h . The energy quantity, q cont,h , is computed as follows: 6. Retailers compute the imbalances, q dev,h , for time period h by considering the real-time consumption of each customer of the portfolio, q j,h : 7. Retailers compute their balance responsibility, C dev,h , for time period h by considering their deviations, q dev,h , and the prices of the excess, P up,h , or lack of, P down,h , energy, in cases of up or down deviations, respectively: Now, we note that a bilateral contract ct corresponds to an energy price, P c ct ,h , and the associated investment, I h , of retailers is computed as follows: The profit of retailers per time period is: The return of the investment (ROI) is a performance indicator to measure the profitability of investments, and is computed as follows: The following three indicators are used to evaluate the performance of the forecast methods: where q max is the maximum consumption (per time period) associated with the portfolio. MAPE is one of the most used indicators. NRMSE is used to evaluate energy forecasts, but overvalues high errors concerning low errors. MAE indicates the absolute error of forecast methods and also the average demand to be balanced. However, it cannot be used to compare results between different data sets.

Computational Study
The study considers six competing retailers (software agents). The retailers are assumed to start operating at the Iberian Electricity Market (MIBEL) in 2013. They have a target of 312 real Portuguese consumers for their portfolios, corresponding to around 5% of the total consumption of the country [28]. Table 1 presents the main characteristics of each retailer. Consumers are assumed to be connected to the middle voltage grid, meaning that some of them are part of aggregations of residential and small commercial consumers. They are divided into the following five consumption segments: industrial, large commercial, aggregation of small commercials, aggregation of residential areas, and other aggregations. The time period of the study ranges from 1 January 2012 to 31 December 2013. Retailers consider the optimization model described in [12] to obtain their optimal portfolios. For 2013, they use the real market prices of 2012, updated with forecasts for 2013, and the real consumption data from 2012, also updated with forecast for 2013, to compute their portfolios. In general, the optimized portfolios only consider a small number of end-use customers.
Retailers offer tariffs to end-use customers with the aim of signing bilateral contracts with them. The tariffs have two terms, either a fixed term, which charges for the power purchase guarantee, and a variable term, which charges for the energy, grid access, global use of the system, and the commercialization part of the tariff. The energy part of the tariff is the only part that can be negotiated with customers. Retailers can offer personalized tariffs to different consumers or the same tariff to consumers inside the same consumption segment. Contract duration is assumed to be twelve months.
Retailers trade energy in the wholesale market only (i.e., we do not consider bilateral contracting to obtain energy sold to customers). We consider a day-ahead market and a hybrid intraday market based on both daily auctions and a continuous procedure, where retailers can adjust their short-term positions 1 h ahead of real-time operation.
To compute the bids to submit to the DAM, forecasts based on the expected weather are considered. Deviations from the bids submitted to the DAM are adjusted in the different sessions of the intraday market. The bids to submit to the IDM are computed by considering the following weights: 0.85 for w x , 0.05 for w y and 0.10 for w z (recall last section). Tables 2 and 3 show the results of the study. Retailers with portfolios containing a small number of consumers experienced higher errors in their forecasts, indicating that versatility and complementarity between different consumers are important factors to take into account. For instance, retailer Ret 1 has the smallest number of consumers and the highest forecast errors. We note that a consumption reduction occurred in 2013, meaning that retailers sold energy in the continuous IDM during most hours (see Table 2).  Table 3 shows the expected and real returns of retailers. From 2012 to 2013, we note that the wholesale market prices decreased by around 9.2%, a greater reduction than expected, leading to better outcomes for retailers. Accordingly, the real return was higher than the expected return. By comparing the ROI associated with the participation of retailers in the continuous IDM and the optimal ROI, we conclude that retailers with large portfolios and small forecast errors have smaller differences in these indicators.
The intrinsic variation (IR) computes the difference between the real ROI and the expectedROI as follows: By analysing the IR, we conclude that risk-seeking retailers have a lower increase in expected return, which means that even with higher forecast errors, the portfolios of risk-averse retailers can be considered more stable, leading to better outputs. Additionally, by analysing the MAE, we conclude that the average deviations of retailers are low.
Overall, retailers met more than 90% of the energy needs associated with their portfolios in the DAM. However, they also traded some energy in the intraday market. We note that the extended forecast methods used in this work reduced the errors of retailers and increased their return (when compared with the results presented in [15]). The forecast accuracy of the bids submitted to each intraday session has been significantly improved. Furthermore, all retailers benefited from participating in the continuous intraday market by increasing their return.

Conclusions
This article described a strategic bidding process for retailers to submit bids to a wholesale market composed by a day-ahead market, an intraday market, and a balancing market. It considered a market design that involves a hybrid model for the intraday market, based on daily auctions and a continuous procedure. The continuous intraday market was used to make adjustments 1 h ahead of real-time operation. The strategic bidding process considered four different hybrid forecast methods: multivariate time series for long-term prediction of electricity prices and consumption, historical meteorological comparison of consumption, short-run trend, and typical consumption behaviour of consumers. The last method was considered for forecasts associated with the continuous intraday market (IDM).
Additionally, the article presented a computational study to illustrate and test both the hybrid market design and the strategic bidding process of retailers, by using real data from the Iberian electricity market for the period 2012-2013. The results confirmed the advantages of considering a continuous intraday market. The continuous IDM significantly improved the forecasts of retailers, providing a good solution to adjust their short-run deviations, without involving energy storage and demand-response programs (although these important aspects can be considered in future work).
The results also indicated important aspects to consider when selecting customers to add to the portfolios of retailers. Risk-seeking retailers showed a behaviour similar to traditional retailers. They considered large portfolios with substantial value at risk and tariffs involving a significant risk premium. Such tariffs seemed to be more competitive that the tariffs proposed by risk-averse retailers. However, risk-averse retailers obtained a better output (ROI) from the market, even with higher demand forecast errors, since their portfolios were more stable (according to both price and consumptions risk).
Future works intend to study how the strategic bidding process can be extended to consider prosumers as part of the portfolio of retailers. Additionally, we aim to study the smooth integration of the continuous intraday market with one or more auctions (or sessions), i.e., we will examine how many auctions should be considered and when they should start. Data Availability Statement: Real consumption data for the consumers considered in study can be found at: https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014#. Iberian market data are available online at: https://www.mercado.ren.pt/EN/Electr/MarketInfo/MarketResults/ Pages/default.aspx. Tariffs proposed to consumers and optimal portfolio data can be found at: https://doi.org/10.7910/DVN/WFQ5V0. Observed and forecast meteorological data can be found at http://www.meteomanz.com/index?l=1. Data were accessed on 16 January 2023.

Conflicts of Interest:
The authors declare no conflict of interest.