Load Forecasting Using Time Series Models

Load forecasting is a process of predicting the future load demands. It is important for power system planners and demand controllers in ensuring that there would be enough generation to cope with the increasing demand. Accurate model for load forecasting can lead to a better budget planning, maintenance scheduling and fuel management. This paper presents an attempt to forecast the maximum demand of electricity by finding an appropriate time series model. The methods considered in this study include the Naïve method, Exponential smoothing, Seasonal Holt-Winters, ARMA, ARAR algorithm, and Regression with ARMA Errors. The performance of these different methods was evaluated by using the forecasting accuracy criteria namely, the Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Absolute Relative Percentage Error (MARPE). Based on these three criteria the pure autoregressive model with an order 2, or AR (2) under ARMA family emerged as the best model for forecasting electricity demand.


INTRODUCTION
Malaysia's National electricity utility company (TNB) is the largest in the industry, serving over six million customers throughout Malaysia.TNB's core activities are in the generation, transmission and distribution of electricity.The Transmission Division is responsible for the whole spectrum of transmission activities ranging from system planning, evaluating, implementing and maintaining the transmission assets.One of the requirements of the system planning is load forecasting.
Load forecasting is a process of predicting the future load demands.It is important for electricity power system planners and demand controllers in ensuring that there would be enough supply of electricity to cope with an increasing demand.Load forecasting can also determine which generators need to be dispatched, or kept as a backup or on spinning reserve status (Izham Zainal Abidin 2005).Thus, accurate load forecasting can lead to an overall reduction of cost, better budget planning, maintenance scheduling and fuel management.
Load forecasts can be divided into three categories: short-term (STLF), medium-term (MTLF), and long-term forecasts (LTLF).STLF, which is usually from one hour to one week, is concerned with forecast of hourly and daily peak system load, and daily or weekly system energy.It is needed for control and scheduling of power system, and also as inputs to load flow study or contingency analysis.Some of the techniques used for STLF are multiple linear regression, stochastic time series and artificial intelligence based approach.MTLF relates to a time frame from a week to a year and LTLF relates to more than a year.MTLF and LTLF are required for maintenance scheduling, fuel and hydro planning, and generation and transmission expansion planning.The common techniques used for MTLF and LTLF are time trend extrapolation and econometric multiple regression (Feinberg & Genethlion 2005;Lee & Park 1992;Weerakorn Ongsakul 2006).
However, time series modeling is one of the popular methods used by many researchers for load forecasting.Cho et al. (1995) proposed ARIMA model and transfer function model for customer load forecasting during one week by considering weather-load relationship.Results showed that ARIMA Transfer Function Models could achieve better accuracy of load forecast than the traditional ARIMA model.Nirma Amjady (2001) proposed a modified ARIMA, which combined the operators' estimation as the initial forecasting with the temperature and load data in a multi-variable regression process.The forecasting accuracy of the modified ARIMA was found to be better than ARIMA.Carter & Zellner (2003) found out that the non-linear least squares estimation of the ARAR estimation of the parameters required less iteration than ARMA estimates.Gould et al. (2005) discussed the weakness of Holt-Winters (HW) exponential smoothing approach in forecasting the hourly electricity demand.They claimed that HW failed to pick up the similarities from day-to-day at a particular time and proposed a new approach for forecasting time series with Multiple Seasonal Pattern (MS).The MS model, which employed single source of error models, provided more accurate forecasts than the HW models because of its flexibilities.The MS model allowed for each day to have its own hourly pattern or to have some days with the same pattern.
In this paper, an attempt was made to forecast the maximum demand of electricity by finding an appropriate time series model.The time series models considered in this study include Naïve, Seasonal Holt-Winters, ARMA, ARAR algorithm and Regression with ARMA Errors.The performance of these different models was evaluated using the forecasting accuracy criteria namely, the Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Absolute Relative Percentage Error (MARPE).

Time series modeling
A time series is a set of observations x i , each one being recorded at a specific time t and denoted by {X t }.It can be represented as a realization of the process based on the general model called Classical Decomposition Model, and specified as follows: , where m t is a trend component, s t is a seasonal component and Y t is a random noise component which is stationary (Brockwell & Davis 2002).
The goal for a time series modeling is to predict data series that are typically not deterministic but contain a random component.The deterministic components, m t and s t need to be estimated and eliminated as to make the residue or noise component Y t to be stationary time series.The time series {X t } is said to be stationary if the mean and the auto-covariance function of {X t } are independent of time.A non-stationary time series needs to be transformed to a stationary time series.Then only a satisfactory probabilistic model can be determined for the process Y t to analyze its properties and to use it for prediction purposes.

ARIMA processes
ARIMA (auto-regressive integrated moving average) processes are a major part of time series modeling and used for a wide range of non-stationary series.Each ARIMA process has three parts; the autoregressive part (or AR), the integrated (or I) part, and the moving average (or MA) part.The models are denoted by ARIMA (p, d, q).ARMA (auto-regressive moving average) models denoted by ARMA (p, q) come from an important parametric family of linear time series models, which provide a general framework for studying stationary processes.Method of differencing is introduced to transform the nonstationary ARIMA into stationary series ARMA and parameter d stands for the degree of first differencing involved.In other words, when d = 0, the model represents a stationary process (Box et al., 1994& Makridakis et al., 1998).
A stationary ARMA (p, q) model is defined as a sequence of random variables {X t }, given by where {Z t } is a sequence of uncorrelated random variables with zero mean and constant variance denoted as {Z t }∼WN(0,σ 2 ) and the polynomials (1-φ 1 z -...-φ p z p ) and (1+φ 1 z +...+ φ q z q ) have no common factors.
The process {X t } is said to be an ARMA (p, q) process with mean µ if {X t -μ} is an ARMA (p, q) process and conveniently written in the more concise form of where φ(.) and θ(.) are the p th and q th degree polynomials, B is the backward shift operator (B j X t = X t-j , B j Z t = Z t-j , j=0, ± 1,...).
The time series {X t } is said to be an autoregressive process of order p (or AR (p)) if φ(z) = 1 and a moving average process of order q (or MA (q)) if θ(z) = 1 (Brockwell & Davis 2002).

METHODOLOGY
This section describes the procedures of establishing an appropriate time series model for load forecasting.The procedures include data plotting, data transformation, model selection, parameter estimation, validation tests, and forecasting.Analysis is done using Interactive Time Series Modeling (ITSM).ITSM is a totally windows-based computer package for univariate and multivariate time series modeling and forecasting.

The data set
The load data used in this research was a Power Load Profile for a utility company.The data represented the monthly mean maximum demand measured in Megawatts (MW) in 52 months from September 2000 to December 2004.The time series plot of the monthly mean maximum demand is given in Figure 1.It appears from the graph that the maximum demand has an upward linear trend.The variance of the series is stable and thus no logarithmic or any other transformation is needed.There is a seasonal pattern with a few troughs occurring between November to February each year.This may be due to various holidays such as school holidays, Hari Raya and Chinese New Year.These patterns reveal that the series is not stationary and hence need to be transformed before attempting to fit a stationary model.

ARMA model
Transformations are applied to produce data that can be successfully modeled as stationary time series.The series clearly shows a seasonality of period 12 as it is derived from a monthly data with an annual seasonal pattern.The data was differenced at lag 12 and 1 to obtain an approximate stationary series.Figure 2 shows the differenced series derived from the monthly mean maximum demand has no apparent deviations from stationarity.
These differenced series were 'mean-corrected' by subtraction of the sample mean, so that it is appropriate to fit a zero-mean ARMA model to the adjusted data.The selection of the appropriate parameters of ARMA (p, q) model depends on a variety of tools, which include the sample ACF (autocorrelation function), the sample PACF (partial autocorrelation function) and the AICC statistic (Brockwell & Davis 2002).-200. -400. -600. -800. -1000.
-1200.The graphs of the sample ACF and PACF shown in Figure 3 suggest an appropriate ARMA model for the data.The ACF will represent a pure MA (q) model and the PACF will represent a pure AR (p) model.Since the ACF vanishes for lags greater than 1 and the PACF vanishes for lags greater than 2, MA (1) and AR (2) are possible models.However, other models such as AR ( 1) and a combined model of ARMA (2, 1) might also be considered as the potential models.
Even if the sample ACF or PACF does suggest an appropriate ARMA model for the data, it is still advisable to explore other models.The AICC criterion provides a rational criterion for choosing between competing models and it is an asymptotically biased estimate of the fitted model relative to the true model.AICC statistic is given by where φ ∧ = a class of AR parameters, ∧ θ = a class of MA parameters, ∧ σ = estimated variance of white noise, n = number of observations, p = order of AR component, q = order of MA component.Makridakis et al.1998).Smallness of the AICC value is indicative of a good model and this can be achieved using the maximum likelihood estimation, which estimates the parameters iteratively.
Once a model is obtained, it is important to check for the appropriateness of the model.If the data were truly generated by the fitted ARMA (p, q) model with white noise sequence { Z t }, then for large samples the properties of the residuals should reflect those of { Z t } .Various validation tests are performed on the suggested models.These tests are the McLeod-Li Portmanteau Test, the Turning Point Test, the Difference Sign Test and the Rank Test.The residuals of the suggested models have to pass all the tests before it can be considered as the best model for forecasting (Brockwell & Davis 2002).
If there are instances where many models pass the validation tests, the most adequate model can still be assessed by looking into the forecasting accuracy criteria.The criteria chosen to measure the accuracy of the forecast in this study are Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Absolute Relative Percentage Error (MARPE) which are given respectively by the following equations, where x i and x ) i are the actual observed values and the predicted values, respectively while n is the number of predicted values.

Co m p a r i s o n w i t h o t h e r f o r e c a s t i n g techniques
Comparisons are made between the ARMA models with the other time series models such as Naïve, Holt-Winter's Trend and Seasonal, ARAR 2 n n forecast and Regression with ARMA errors.These methods are briefly described as follows:

Naïve
Naïve forecasting neglects all past data except for the time period that occurred last.It may be adequate for dealing with many of the minimal consequence decisions of daily life and more effective at short-term applications.The next forecasted period F t -1 , is based on the most recent observation Y t , the relation between them is given by the following equation: If recent observations are given more weight in forecasting than the older observations, the method will be called as Simple Exponential Smoothing.However this method works best for data, which have no trend, no seasonality, or other underlying pattern (Makridakis et al. 1998)

Holt-Winter's Trend and Seasonality Method (HW)
The HW method is an extension of Holt's Linear Method that considers series with trend and seasonality.The method is based on three smoothing equations -one for the level, one for trend, and one for seasonality, which can be either additive or multiplicative seasonality.Multiplicative seasonality is considered in this paper since it is more commonly used.The basic equations are: Trend: where s is the length of seasonality , L t is the level of the series, m t is the trend , S t is the seasonal component, and F t+q is the forecast for q periods ahead (Makridakis et al.1998).

ARAR forecast
ARAR model is suitable for forecasting the series {Y t } whereby a memory-shortening transformation sequence has been applied.The memory-shortened series is where 1, ψ 11 , ... ψ k are the coefficients of the chosen filter and t = 1, …, T. Let S denotes the sample mean of S 1 , ... , S T .Thus, the fitted model is given by X where X t = S t -S , {Z t } ∼ WN (0, σ 2 ), and for given lags l 1 , l 2 , and t , the coefficients φ j and σ 2 are from Yule-Walker estimation (Brockwell & Davis 2002).

Regression model with ARMA errors
Regression model with ARMA errors is a combination of a multiple regression model with an ARMA model.The general model takes the form using matrix notation, where Y = (Y 1 , Y 2 t , …Y n )' is the response vector observed at time t = 1, 2, …, n, X is the design matrix consisting of the n explanatory vatriables with columns being 1, t, t 2 ,…,t k and β = (β 1 t , β 2 ,…, β n )' is the vector of regression coefficients and W = (W 1 , W 2 , …, W n )' are observations from a causal zero mean ARMA (p, q) process (Brockwell & Davis 2002).First ordinary least estimates are computed for β and then the estimated residuals ARMA (p, q) model is fitted by the maximum likelihood method.
Finally, for the fitted ARMA (p, q) model, generalized least squares are computed for the regression coefficients and the process is repeated until the estimates have stabilized.

ARMA Model
The estimated ARMA models for forecasting the maximum demand of electricity with their corresponding AICC values are given in Table 1.Clearly AR (2) has the minimum AICC value and can be considered as the most appropriate model if compared among the other models under ARMA.The equation for the model is given by X t = -0.9381Xt-1 -0.4508X t-2 + Z t (14) where Z t ∼ WN( 0, 61556.9 ).Validation tests were performed on the AR (2) model that had the minimum AICC value and the result of the tests are shown in Table 2. AR (2) model passes all the tests with p-values greater than 5% indicating that there is insufficient evidence to reject the null hypothesis that the residuals are white noise.
The graphs of the ACF and PACF ( see Figure 4) of the residuals also has no more spikes beyond the 95% confidence limits indicating further that AR (2) is indeed an appropriate model.
Based on Equation ( 14), the forecasted values from January 2005 (Month 53) to May 2005 (Month 57) and the 95% prediction bounds are computed and presented in Table 3.The percentage difference of each forecast value compared to the actual value is less than 1%. Figure 5 shows a plot of the forecasts for 5 months as given by Table 3.

Regression model with ARMA errors
The regression results with ARMA errors are as follows.A linear regression fit, is given by ( 15) where, Y t represents the maximum demand of electricity and W t are the residuals.The autocorrelation function (ACF) and partial autocorrelation function (PACF) of the residuals obtained after the regression fit are shown in Figure 6 from which it is clear that the residuals are correlated to a large extent.
Hence to the residuals {W t } , a stationary ARMA process was fitted based on the AICC criterion and the parameters were estimated by the maximum likelihood method.The fitted ARMA process was ARMA (4, 1) given by the following equation, W t = 0.8733 W t-1 + 0 .01334W t -2 -0.1234 t -3 -0.2889 W t -4 + Z t -0.7961Z t -1 (16) where {Z t }, ∼ WN(0, 60451.1).The values of AICC and AICC (corrected for regression) were 735.9 and 741.4 respectively.With these ARMA (4, 1) errors in Equation ( 16), a generalized least squares fit was obtained and it is given by Y t = 8432.5476+ 49.581074t + W t . (17) Based on the model given by Equation ( 17

Figure 1 .
Figure 1.The Maximum Demand from September 2000 to December 2004

Figure 2 .Figure 3 .
Figure 2. The Time Series of the Residuals after Differencing at lag 1 and 12

Figure 4 .
Figure 4.The ACF and PACF of sample residuals

Table 1 .
Estimated Models Based on the Maximum Likelihood

Table 3 .
Forecasting maximum demand in MW for 5 months

Table 4 .
), a plot of the data and five forecasted values from January 2005 (Month 53) to May 2005 (Month 57) are shown in Figure 7.Both models of AR (2) and Regression with ARMA errors are compared with other time series models.Post Sample Accuracy Criteria of each time series model are summarized in Table 4. From Table 4, AR (2) records the lowest MARPE and thus is a better model for forecasting the maximum demand of electricity in a utility company.Post Sample Accuracy Criteria