Long-term wind speed prediction using artificial neural network - based approaches

: In the current scenario, worldwide renewable energy systems receive renewed interest because of the global reduction of greenhouse gas emissions. This paper proposes a long-term wind speed prediction model based on various artificial neural network approaches such as Improved Back-Propagation Network (IBPN), Multilayer Perceptron Network (MLPN), Recursive Radial Basis Function Network (RRBFN), and Elman Network with five inputs such as wind direction, temperature, relative humidity, precipitation of water content and wind speed. The proposed ANN-based wind speed forecasting models help plan, integrate, and control power systems and wind farms. The simulation result confirms that the proposed Recursive Radial Basis Function Network (RRBFN) model improves the wind speed prediction accuracy and minimizes the error to a minimum compared to other proposed IBPN, MLPN, and Elman Network-based wind speed prediction models.


Introduction
Artificial Neural Networks (ANN) developed from the inspiration of the biological nervous system. ANNs are nonlinear information processing systems designed from interconnected elementary processing devices called neurons [1,2]. ANN has good self-learning ability, adaptability, real-time operation, fault tolerance ability, easy implementation, and inexpensive because of these features. ANN is widely used for various applications. We can classify ANN as feedforward and feedback (recurrent) networks. A network arranged into layers with no feedback path is called a feedforward network. Through one (or) more hidden layers, it is structured as input to the output layer: MLP, BPN, RBFN, etc. A network with a feedback path between the layers (or) within the layer is called a feedback network, for example, Elman, Hopfield, Boltzmann machine, etc. [3,4].
Because of the environmental degradation and depletion of conventional energy, much attention has been focused on wind energy [5]. Since wind is one of the most flexible and tractable of all energy sources, the mechanical energy derived directly from wind can be readily and efficiently converted into another form of energy. Improving energy security and reducing greenhouse gas emissions to maintain the Kyoto Protocol demanding wind energy for electricity generation [6]. The long-term prediction horizon ranges from 1 day to 1 week or more ahead, which is essential for wind farms and energy systems to plan operation management and maintenance effectively.
The research gap concerned with the wind speed prediction application is simple, and the highly accurate long-term prediction model is needed to be addressed. Wind possesses uncertainty due to the influence of various atmospheric variables. Thus, a reliable prediction model is obligatory. This paper addressed the above-said research gaps by the proposed long-term wind speed prediction models and considered the significant influence variables as inputs to the proposed models. Due to time-varying and randomness, wind speed prediction is a crucial hot topic in research.
The major contributions of this paper are as follows: (1) Proposed four artificial neural network-based prediction models; (2) Applicability substantiated for wind speed prediction with respect to long-term horizon; (3) Evaluated the proposed model on a real-time measured wind dataset; (4) Performance and stability-based analyses were carried out concerning hidden neurons; (5) Carried out various input-based performance analyses; (6) Proposed models outperform with better prediction accuracy and reliability.

Importance of wind speed prediction
A slight fraction deviation of wind speed will lead to a significant error output of the wind driving system. The importance of wind speed prediction is given below: To promote high-quality and reliable power system operation; To integrate wind energy into the electrical power grid effectively; To reduce wind energy generation operating costs; To achieve low spinning reserve; To help plan and control power systems and wind farms [7,8].
Many research works suggested an ANN-based wind speed prediction model by several researchers [4]. I have reviewed related work on wind speed prediction using ANN and discussed it as follows.
A wind speed prediction model based on BP (BackPropagation) algorithm was presented in Perez-Llera et al. 1998 [9]. Bilgili M et al. 2007 [10] performed long-term wind speed using ANN with RP (Resilient Propagation) algorithm. Autoregressive moving average (ARMA)-based wind speed prediction for short-term horizon pointed out by Torres JL et al. 2005 [11]. Junfang Li et al. 2010 [12] suggested Elman neural network-based one step ahead of wind speed prediction. H. Selcuk Nogay et al. 2012 [13] analyzed different ANN models for short-term wind speed forecasting. Karakuş O et al. 2017 [14] suggested a polynomial AR (autoregressive) model-based wind speed prediction concerning 12 hours ahead horizon. Wind speed forecasting in the long-term horizon was carried out using ANN [10,15], and a spiking neural network [16]. Neshat et al. 2021 [17] pointed out the evolutionary algorithm associated hybrid model for wind turbine power prediction. Chen et al. 2021 [18] presented the kernel mean squared error loss function incurred wind speed prediction model using a deep learning approach. Neshat et al. 2021 [19] performed short-time wind speed prediction using an evolutionary algorithm that adopted a deep learning model. Liang et al. 2021 [20] presented a multi-objective hybrid optimization algorithm-based optimized Bi-LSTM deep learning with transfer learning associated with the wind speed prediction model. Lahouar et al. 2017 [21] random forest-based hour-ahead wind power prediction carried out and exempted irrelevant input. Casella 2019 [22] suggested a multivariate probabilistic model that predicts wind speed.
This paper proposes four ANN-based methodologies such as Improved Back Propagation network (IBPN) [2], Multilayer Perceptron network (MLPN) [3], Recursive Radial Basis Function Networks (RRBFN) [7,8], and Elman network [6] applied for wind speed prediction. The primary aim is to improve the prediction accuracy with reduced minimal error compared to other existing approaches.

Proposed artificial neural network-based approaches for wind speed prediction
This paper proposed the five inputs based on different artificial neural network approaches, such as improved backpropagation network, multilayer perceptron network, Elman neural network, and novel recursive radial basis function neural network for wind speed prediction. The detailed discussion about major topics as a problem description, proposed model conceptual overview, data collection, data normalization, design, training, testing, denormalization, and performance metrics are described as follows.

Problem description
Wind speed prediction models based on the Improved Backpropagation network, Multilayer perceptron network, Recursive Radial Basis Function neural networks, and Elman network are presented in this paper. An accurate prediction of wind speed is one of the essential issues in renewable energy systems because of the dilute and fluctuating nature of wind. The wind has the uncertain irregularity characteristic. The appropriate modeling of the wind speed prediction model regarding inputs, hidden neurons, and hyperparameters aid in achieving better generalization capabilities [23]. In the current scenario, with a lot of predictions, research fields have to be heuristic in nature. The proposed wind speed prediction models confirm that even though large hidden neuron numbers in the proposed networks get stable performance in training. The ultimate aim of the proposed ANN-based wind speed prediction models is to improve the prediction accuracy with minimum statistical for wind speed prediction in renewable energy systems.
In neural network design, different heuristics have existed. Still, there is a need for a fast and accurate model for wind speed prediction. The author's expertise and literature knowledge propose four artificial neural network-based models for applying wind speed prediction. The reason for selecting these four models is that artificial neural networks (ANN) are generally classified into feedforward and feedback. In this paper, the author considered two models (IBPN and MLPN) that are well famous feedforward neural networks; and Elman network is a feedback neural network, and the author proposed a novel feedforward neural network (RRBFN). The ultimate aim is to analyze the ANN effectiveness in wind speed prediction application. The main reasons to select the proposed four forecasting models are not complex and costly. The proposed wind speed forecasting model is developed with five inputs. We can remove the optional input, which does not impact the wind speed [21], but this paper considers the inputs to have a high impact to cause the variance in the wind speed, so it is necessary to consider all inputs. The proposed models' input and output target vector pairs are described as follows.
Where, TD w : Temperature; WD w : Wind Direction; RH w : Relative Humidity; PW w : Precipitation of Water Content; N w : Wind speed and N pw : Predicted wind speed.

Proposed wind speed prediction models conceptual overview
An Improved Backpropagation network (IBPN) is a multilayer feedforward network that incorporates the backpropagation (error) learning algorithm to balance the network's memorization and generalization. The backpropagation network comprises an input layer, a hidden layer, and an output layer. The processes involved in backpropagation training are the feedforward process, error computation process, and weight updating process. The feedforward network comprising processing elements that perform independent computation based on a set of input data and weights with a continuously differentiable activation function, and the computed results are transferred to the next layer. Lastly, the network output (predicted wind speed) is calculated, then the error is computed based on the difference between the actual target and the predicted result. The computed error is propagated back to the hidden unit and then passed to the input layer. For a training input and target pair, the weights get changed and updated in the backpropagation network to achieve the accurately predicted wind speed with reduced error. The presented backpropagation convergence speeds up by incorporating the momentum factor (η).
Multilayer perceptron network (MLPN) is a feedforward neural network with supervised learning rules to search for the weight of binary and linear activation functions to solve complex problems. The architecture of the multilayer perceptron network is designed based on the single-layer perceptron network. The multilayer perceptron network comprises an input layer, a hidden layer, and an output layer. The multilayer perceptron is superior to the single-layer perceptron because it overcomes the drawbacks of the single-layer perceptron and has significant computational efficiency. The multilayer perceptron networks learn linear and nonlinear relationships between the input and output vectors because the hidden layer neuron has the nonlinear transfer function. The most commonly used nonlinear transfer function is the hyperbolic tangent sigmoid activation function which is applied over the net input of the hidden layer. The backpropagation learning rule is used to train multilayer perceptron networks. The multilayer perceptron networks are fully connected networks. The hyperbolic tangent sigmoid activation function is adapted for the hidden layer, and the purelin activation function is used for the output layer.
The Recursive Radial basis function network (RRBFN) comprises the input, hidden, and output layers. I apply the radial basis function for hidden nodes in order to compute the input, and the Gaussian activation function calculates the output. Radial basis function emphasis is on the data existing over a region of the input space. The radial basis function network improves the convergence faster and better function approximation. Besides that, recursively update the error values to improve accuracy.
Elman neural network is used for different applications, such as time series prediction, modeling, control, and speech recognition. From the hidden layer, get the output. The feedback information is stored in the recurrent link layer and keeps the memory. The hyperbolic tangent sigmoid activation function is adapted for the hidden layer, and the purelin activation function is used for the output layer.

Data collection and data normalization
The real-time data was collected from the National Oceanic and Atmospheric Administration, United States, for a period from January 1998 to December 2018 from the location of Theni, INDIA, latitude: 10º E and longitude: 77.5º N. The temperature (℃), Wind direction (Degree), Relative Humidity (%), Precipitation of Water Content (%), and Wind speed (m/s) are inputs to the proposed four artificial neural networks based on wind speed prediction models, and the network output is the predicted wind speed (m/s). The proposed IBPN, MLPN, RRBFN, and Elman network models are developed by 80,000 number of samples to predict the wind speed in the long-term horizon (1 day to 1 week (or) more ahead).
Normalization is very important for dealing with real-time data; the real-time data has a different range and different units. Hence, the normalization is used to scale the real-time data within the range of 0 to 1. The normalization process helps to achieve better accurate numeric computation and also improves output accuracy. The proposed approach utilizes the min-max normalization technique, which is important to effectively minimize the systematic noise present in the dataset. The following transformation equation is used for the normalization of the real-time data.
where, X i is actual input data, X min is minimum input data, X max is maximum input data, X' min is the minimum target value, X' min is the maximum target value.

Artificial neural networks based wind speed prediction models design
The proposed artificial neural networks-based wind speed prediction model's designed parameters include the dimensions and epochs shown in Table 1. The dimensions such as input neuron number, hidden neuron number, and output neuron number are defined in the network design. The presented network design has five input neurons such as Wind Direction (WD w ), Temperature (TD w ), Relative Humidity (RH w ), Precipitation of Water Content (PW w ) and Wind speed (N w ), one hidden layer, one output neuron, i.e., Predicted wind speed (N pw ).
Considered Improved Back-Propagation network (IBPN) inputs are transferred to the hidden layer that multiplies the weight W with a hyperbolic tangent sigmoid activation function. The hidden layer outputs are transferred to the output layer by multiplying with weight V using the tangent sigmoid activation function. The IBPN network convergence speeds up by incorporating the momentum factor (η). Performed Multilayer Perceptron network (MLPN) inputs are transferred to a hidden layer that multiplies the weight W with a hyperbolic tangent sigmoid activation function. The hidden layer output is transferred to the output layer that multiplying with weight V using the purelin activation function. Carried out Recursive Radial Basis Function Network (RRBFN) input layer and hidden layer are interconnected through the hypothetical connection. The hidden layer has a Gaussian function. Weighted connection interconnects the hidden layer and output. The output layer has a linear function. Performed Elman network input is transferred to the hidden layer that multiplies weight W 1 using the hyperbolic tangent sigmoid activation function, and the hidden layer output is transferred to the output layer by multiplying with weight W 2 using the purelin activation function. Because of the training, previous information is reflected in the Elman network.

Training and testing of the proposed models
The wind speed forecasting model is developed based on the training dataset, and the proposed models are evaluated by using the testing dataset. The collected 80,000 real-time dataset is divided into training and testing. The collected 70% data (56,000) is used for the training phase, and 30% collected data (24,000) is utilized for the testing phase of the network. The real-time data are divided into training and testing datasets. Neural network learning uses a training dataset, and the testing set is used to calculate the error. The recursive radial basis function network (RRBFN) with 31 hidden neurons gets the lowest minimal statistical errors; the optimal hidden neuron is selected by trial and error between 1 to 31 hidden neurons.

Denormalization and performance metrics
The denormalization process is the post-processing of the data. The output is carried out after denormalization. The proposed approach performance is analyzed based on statistical errors like MAPE (Mean Absolute Percentage Error), MSE (Mean Square Error), RMSE (Root Mean Square Error), MAE (Mean Absolute Error), MRE (Mean Relative Error) [24]. MSE measures the prediction accuracy; close prediction with respect to the target is quantified by MAE. RMSE, MAPE, and MRE are useful for gauging the prediction model's effectiveness. The statistical errors are used to evaluate the quality of the predicted wind speed obtained by four ANN-based wind speed prediction models.

Results and discussion
Based on the proposed recursive radial basis function network (RRBFN) based wind speed prediction model, a comparison between the actual and predicted wind speed is shown in Figure 1a. For the clarity of the graph, 240 samples of actual and predicted data are depicted in the result are noticed from Figure 1a. Apparently, the results are exactly matched with the measured values. The prediction error for RRBFN based model is depicted in Figure 1b. The suggested four ANN-based wind speed prediction models were simulated using MATLAB and achieved minimal statistical errors compared to other existing models in the literature. The recursive radial basis function neural network model for wind speed prediction used 31 hidden neurons to improve prediction accuracy and reduce statistical errors. The simulation results reveal that the predicted wind speed is in best agreement with the experimentally measured values. From the result, it has been confirmed that the recursive radial basis function network-based wind speed prediction model gets minimum errors. The results prove that the proposed approach gets better results than the other existing approaches. The presented four ANN-based wind speed prediction designs were run on an Acer laptop computer with a Pentium (R) Dual-Core processor running at 2.30 GHZ with 2 GB of RAM. Statistical errors evaluate the network performance. Table 2. Depict that performance measures of ANN-based four wind speed prediction models with various hidden neurons. The suggested RRBFN model outperforms and achieves minimal statistical errors for long-term prediction horizon two years ahead of wind speed prediction.  Furthermore, the RRBFN with identified optimal hidden neuron (31 hidden neurons) framework-based wind speed prediction model performance is analyzed with various inputs. The obtained results are tabulated in Table 3. Concern about wind speed prediction, past knowledge of input, wind direction is highly influenced other than these input temperature, relative humidity, and precipitation of water content is very important because it influences air density. Each input is important and has a significant correlation to wind speed, so it is impossible to say which input gives better accuracy but increases the relevant inputs parameter to help the prediction model learn the correlation very well, which makes it achieve good prediction. The advantages of the proposed model are simple to implement, has less computational complexity, has a highly accurate prediction, and is generic. A disadvantage of the proposed model is the needed advanced computational power to handle the big data.

Conclusions
This paper presents various artificial neural network approaches, such as IBPN, MLPN, RRBFN, and Elman network, with five input parameters based on long-term wind speed prediction models. The suggested wind speed prediction models were adapted and evaluated with the collected real-time wind dataset. Figure 2 Illustrate the graphical summary of the proposed approach for the better understanding.  The primary aim of the proposed models is to assist in the planning, integration, and control of power systems and wind farms. The presented four neural networks based on wind speed predictions achieve a better agreement with the actual target. The result proves that compared with other proposed wind speed prediction models, the Recursive Radial Basis Function Network (RRBFN) based wind speed prediction model with 31 hidden neurons achieves better accuracy with minimal statistical errors such as MSE of 1.2781e-11, RMSE of 3.5751e-06, MAE of 2.4435e-06, MRE of 7.1195e-07 and MAPE of 7.1195e-05. The proposed RRBFN effectiveness was evaluated with various inputs-based analyses. Based on the performance analysis concerned with long-term horizon-based wind speed prediction, the proposed RRBFN model enhanced the accuracy of wind speed prediction than other models like IBPN, MLPN, and Elman networks.