Performance comparison of solar radiation forecasting between WRF and LSTM in Gifu, Japan

Three months comparison of hourly solar radiation forecasting from 1st January to 31st March 2017 between Weather Research and Forecasting (WRF) mesoscale model and Long short-term memory (LSTM) algorithm is presented in this study. One-way grid nesting technique of the WRF model is applied for the simulation with a six-hourly input dataset downloaded from the National Oceanic and Atmospheric Administration - National Operational Model Archive and Distribution System (NOMADS) website. Three years’data of solar radiation from 1st January 2014 to 31st December 2016 are used as input data for Long Short Term Memory (LSTM) algorithm to simulate solar radiation. The results show the root mean square error of the LSTM algorithm is 310 W m−2 higher compared to 210 W m−2 from the WRF model. The MBE and the nMBE of the WRF model are obtained positive value 96 W m−2 and 9% compared to −101 W m−2 and −9% of LSTM for 2160 h prediction. Meanwhile, the performance error percentage of WRF is 19% lower compared to 28% of LSTM for the nRMSE error metric. Although this study found that the WRF model performed better and lower error compared to the LSTM algorithm, however, it also recommends the LSTM algorithm configuration can be used for long-term prediction.


Introduction
Weather predictions are mainly applied to predict the meteorological framework obtained by directly processing generation data meteorological measurements from a nearby station to create the necessary forecasts. Weather predictions are generated using Numerical Weather Prediction (NWP) models associated with the atmospheric processes which are solved by using the physical equations which can predict meteorological parameters for several days. However, some complex problems such as calculations, a lack of local considerations and computational cost may affect the results [1]. Varying the parameterizations, boundary and initial conditions of the NWP models can calculate the different possible evolutions of the atmosphere. Meanwhile, the diversity of the predicted variables in a given time horizon informs about the forecasted values at that point making a difficult comparison among studies of the various methodologies. The utilization of weather forecasts must be conducted to provide complete information about the meteorological predictions and the information must include descriptions of predicted variables, sources of variables, forecasting horizons, processing techniques, and analysis.
The technology of solar radiation forecasting has been given attention to understand solar energy. The photovoltaic (PV) power generation is dependent on solar radiation. The forecasting of solar radiation is mostly implemented using numerical weather prediction (NWP) method which is suitable for the long term period. Meanwhile, solar forecasting using Deep Learning as part of the Artificial Neural Network (ANN) can be used to simulate solar radiation forecasting between input and output datasets. The artificial neural network (ANN) model commonly used to estimate solar radiation can be determined also as multi-variable parameters such as coordinates (latitude and longitude), altitude, temperature, relative humidity, rainfall, and duration of the hour Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. and month of the year. The performance of solar radiation forecasting to get good accuracy always uses meteorological forecast datasets. Aerosol and cloud cover are elements which may cause the unstable solar radiation forecasting [2,3].
Many studies have been implemented to perform the optimizing of the ANN models by approaching the most relevant input parameters for the ANN or by selecting the best configuration of the ANN structure. Most of the results show the accuracy of the comparison for the ANN model compared to the empirical models to predict solar radiation. However, most of these comparisons have been performed with models that use meteorological data as inputs, but few studies compare the ANN model with models that do not require meteorological input parameters. Indeed the meteorological data may not be available for all the desired regions which makes a model very appropriate to be implemented in order to perform solar radiation prediction in an unknown site. By given the importance of the knowledge of the available solar radiation and the measuring stations in a region, few studies evaluate and compare the use and the accuracy of an artificial neural network and a model in order to spatially interpolate the global solar radiation. ANNs are typically organized layers which are made up of interconnected nodes of input, output, and hidden layers and contain the activation function. ANN models have been widely used in controlling, image processing, pattern recognition, and time series forecasting. The structure of ANN consists of an input layer (i.e. meteorological and climatological data), a hidden layer and an output layer [4]. Despite clear and simple configurations given by ANN for estimation of solar radiation, the prediction accuracy depends on the accuracy of a meteorological parameter as the input data.
Over many years, many case studies have been presented in a short-, medium-and long-term period in order to optimize near or far future predictions for a better analysis. These three-time horizons are commonly used to analyze future predictions such as variables of weather, energy, stock prices, and so on. The conditions of timevarying with time-series forecasting will simply identify the better model for the estimation. Moreover, they can have a great influence on the time period based on the number of steps being forecasted. Based on definitions of meteorological forecasting ranges from World Meteorological Organization (WMO) [5], short-range weather forecasting beyond 12 h and up to 72 h, medium-range is from 72 h to 240 h and long-range forecasting counts from 30 days up to two years. Short-term forecasts with high resolution will provide critical information to safely and efficiently deploy resources for a wide range of users. It will achieve accurate and reliable convective weather forecasts due to the uncertainty of initial conditions, model physics, computational capability and limitations of understanding how it works. Generally, short-term (day-ahead) forecasting is based on statistical approaches, especially ANN, because of the time-consuming operation of the NWP. Meanwhile, medium-range forecasts that provide information up to 10 days in advance aim to capture large parts of day-to-day variations of data.
Many researchers have conducted many methods of study by using solar radiation parameters such as Global Horizontal Irradiance (GHI), Direct Normal Irradiance (DNI) and Diffuse Irradiance (DIF) to analyze the PV power generation. Xiangyun Qing et al (2018) [6] using Long Short-Term Memory (LSTM) compared to backpropagation neural networks (BPNN), linear regression (LR), and persistence algorithm for one day of the rain season on August 20, 2013, in the island of Santiago, Cape Verde. This study found that the LSTM algorithm error is smallest compared to the other three algorithms. Richard et al (2010) [7] conducted a study of forecasting from consecutive geostationary satellite images for seven climatically distinct locations in the United States. This study extends up to 6-h for short term and 6-days for medium-term for hourly Global Horizontal Irradiance (GHI) using numerical weather prediction (NWP) based upon cloud motion. V. Lara-Fanego et al (2012) [8] presented a study of a three-day evaluation in Andalusia (Southern Spain). This study presented Global Horizontal irradiance (GHI) and Direct Normal Irradiance (DNI) of different seasons of the year and three different sky conditions: clear, cloudy and overcast using WRF mesoscale atmospheric model.
Wang et al (2016) [9] estimated the prediction of solar radiation that compares three types of ANN methods such as Generalization Regression Neural Network (GRNN), Multilayer Perceptron (MLP), and Radial Basis Function Neural Networks (RBFNN). These three models are applied to estimate daily global solar radiation. These models are developed based on the air temperature, relative humidity, air pressure, water vapor pressure and sunshine duration hours measured at 12 stations in different climate zones as input variables. This study found that MLP and RBFNN models provide better accuracy than GRNN.
Olatomiwa et al (2015) [10] conducted a study to estimate solar radiation in Nigeria using an ANFIS model. This model is developed to measure the duration of sunshine and monthly mean minimum and maximum temperatures based on a series of measured meteorological data. Respectively, the root mean square error (RMSE) and determination coefficient (R2) for training and testing phases of the model are obtained as 1.0854 and 0.8544, and 1.7585 and 0.6567. Alsina et al (2016) developed a model for the prediction of monthly average daily global solar radiation over Italy. This study used 13 input parameters to construct the model for consideration and showed the results of the mean absolute percentage error (MAPE) of the model, ranges between 1.67% and 4.25%.
Khosravi et al (2018) [11] estimated hourly solar radiation in Abu Musa island, Iran using machine learning algorithms. They compared the results of predicted hourly solar radiation from five different algorithms such as multilayer feedforward neural network (MLFFNN), radial basis function neural network (RBFNN), support vector regression (SVR), fuzzy inference system (FIS) and adaptive neuro-fuzzy inference system (ANFIS). They proposed developing models based on input variables such as temperature, wind speed, relative humidity, pressure, and local time.
The objective of this work is to compare three months of data of solar radiation between the WRF model and the LSTM method for location in Gifu, Japan. Also, the results of WRF simulation and Long Short-Term Memory (LSTM) estimation are used to validate against a single point of ground truth station for local climatology predictions over an extended area. The model configuration and parameter setting of 3×3 km spatial resolution of WRF nesting model simulation compared with LSTM algorithm estimation for three months from 1st January to 31st March 2017 can validate with locally observed data. The performance of simulation and estimation in this work will use the physics parameterization of the WRF model and configuration of the LSTM algorithm to investigate the results from both models, WRF and LSTM, respectively. Finally, this study proposes the physics parameterization of the WRF model and the configuration of the LSTM algorithm for medium-term solar radiation can be used for long term future prediction.
The structure of this work is organized as follows: section 2 describes material and methods which are composed of the study area and data collection, the implementations of the WRF model and LSTM algorithm simulation, and four error evaluation metrics and post-processing method. The results of the performance of the two models, error analysis and comparison data are presented in section 3. Finally, section 4 gives a summary of the conclusions.

Sites and data collection
Gifu [12] is a city located in south-central Gifu prefecture, Japan. Gifu is a cropland area mixed with forests where the temperatures are lowest on average in January and the highest in August. Gifu recorded a low of minus 14°C in January and a high of 38°C in August. The temperature gradient between the cool and the hot inland can lead to the formation of strong cold fronts in summer, which in turn are responsible for various cloud types and heavy rains. This area is chosen as it has a mega solar PV system installed with free available downloaded data. Hourly forecasts data from WRF model simulation and LSTM algorithm estimation are tested against solar radiation data from the weather station of the Gifu network. This weather station covers several weather variables such as temperature, wind speed, wind direction, and output power generation including solar radiation ranging from arid in the summertime (August) to humid in the wintertime (January). Weather station installation with instruments and tools for measuring the weather conditions provides information for prediction and study of the weather variations. However, Gifu city has a significant provision of sufficient motivation to study, and because solar PV is the prime concern, only the solar radiation component is used in the present study. Figure 1 shows the land cover of Gifu as the study area.
In this present study, six-hourly interval 0.5°×0.5°of Climate Forecast System version 2 (CFSv2) spatial resolution of analysis data obtained from the National Oceanic and Atmospheric Administration -National Operational Model Archive and Distribution System (NOMADS) website (https://nomads.ncdc.noaa.gov/ modeldata) are used as the Global Forecast System (GFS) for initial and lateral boundary conditions data for the meteorological fields with 03 as 00 UTC, 09 as 06 UTC, 15 as 12 UTC, and 21 as 18 UTC where the Japan Standard Time (JST) is UTC+9. Two types of the dataset from NOAA-NOMADS such as 'flxf' and 'pgbh' are used in this simulation. The 'flxf' stands for Full-Resolution Surface and Radiative Fluxes and the dataset is freely downloaded from the website 'https://nomads.ncdc.noaa.gov/modeldata/cfsv2_analysis_flxf/'. These datasets of 'flxf' are 6-hourly and contain 104 variables with latitude from 89.844 to −89.844 and longitude from 0 to 359.795 and the grid resolution is 0.205×0.205-degree Latitude/Longitude. The 'pgbh' stands for High-Resolution 3D Forecast Pressure Level Data and the dataset is freely downloaded from the website 'https:// nomads.ncdc.noaa.gov/modeldata/cfsv2_analysis_pgbh/'. These datasets of 'pgbh' are 6-hourly, and contain 611 variables with latitude from 90 to −90 and longitude from 0 to 359.5 and the grid resolution is 0.5×0.5degree Latitude/Longitude with 37 Pressure (hPa) levels. These data are default containing land cover datasets based on the U.S. Geological Survey (USGS) 24 category land-use at 26 mandatory levels from 1000 millibars to 10 millibars. It contains parameters of weather such as temperature, u-and v-winds, surface pressure, sea level pressure, geopotential height, sea surface temperature, ice cover, relative humidity, vertical motion, vorticity, solar radiation, water vapor, and ozone. The simulation is carried out for three months prediction starting from

WRF model
The WRF model [13] is a numerical weather prediction developed by the National Center for Atmospheric Research (NCAR), the National Oceanic and Atmospheric Administration (NOAA) represented by the National Centers for Environmental Prediction (NCEP) and the Forecast System Laboratory (FSL), the Air Force Weather Agency (AFWA), the Naval Research Laboratory, the University of Oklahoma, and the Federal Aviation Administration (FAA). This model can be used to simulate atmospheric data and it is divided into two dynamical cores: Advances Research WRF (ARW) and Nonhydrostatic Mesoscale Model (NMM) with advances in dynamics, physics and data assimilation. To stabilize the performance of the WRF model, various configurations such as nesting level, resolution of the spatial grid and resolution of time have been tested in the centers of a domain. The WRF allows nesting resolution horizontally to focus over a particular region by introducing finer grids in the simulation. This model supports two techniques such as one-way and two-way grid nesting to interact with a fine domain and a coarse. These two techniques including initial and lateral boundary conditions are provided by the parent domain with input from higher resolution.
In this study, the WRF-ARW version 3.9.1 model with one-way nested domain are used in this simulation from west to east and south to north for horizontal grid resolution 9000 m and 3000 m (figure 2(a)) with grid size 90×64 as parent domain (d01) and 94×88 as the nested domain (d02)( figure 2(b)). The model is first to run for the outer domain to create an output which is interpolated in space and time to supply the nest as a boundary file, which is run after the coarser domain has finished. This study applied Thompson and Eidhammer (2014) scheme for microphysics. This scheme considers water-and ice-friendly aerosols variables to specify initial and boundary conditions with the user_aero_icbc set to false to assume all model in horizontal grid points have the same ice nucleating aerosols and have the same vertical profiles of water nucleating aerosols [14]. Rapid Radiative Transfer Model is used for longwave (LW) radiation [15]. This scheme has been developed for application and implementation to general studies of atmospheric radiative transfer and dynamical models ranging from single-columns to general circulation model and it has been developed to reproduce line-by-line results. Dudhia scheme is used for shortwave (SW) radiation [16]. Monin-Obukhov MM5 similarity theory is used for the surface layer. This theory uses the framework to compute the surface turbulent fluxes and also provides surface layer within the information of the profiles that are used to diagnose meteorological variables such as temperature and moisture at 2 m and wind at 10 m [17]. Noah Land Surface Model is used for the land surface. Noah LSM is a major result collaborative among with the input data of NCEP, NCAR, and AFWA to evaluate the impact and the performance of land surface coupled with WRF model simulation. This model improves time-varying snow density and snow length, snow cover, frozen-ground physics, and some fields like maximum snow albedo, etc Yonsei University scheme is used for the planetary boundary layer [18]. This PBL scheme is first-order use to express the turbulent fluxes and the vertical gradient of the mean variable. This study applied the Betts-Miller-Janjic Scheme [19] for cumulus parameterization to compute eddy diffusion coefficients from an additional (1.5-order) prognostic equation for the turbulent kinetic energy (TKE). This simulation used a time step of the 20 s for the configurations of two domains. The results of this simulation are used as a benchmark to demonstrate the performance efficiency of WRF model. Plotting for its grid points and its variable uses NCL (NCAR Command Language) programming language version 6.5.0 [20]. Table 1 shows the details of the configuration of WRF parameters. In this study, the data from domain d02 are used as results for analysis for prediction purposes.
In this study, solar radiation from the WRF model is calculated using three parameters such as solar direct, solar diffuse, and cos zenith. The formula is defined as below; The input gate, forget gate, and output gate is the structure of the LSTM [21]. The input gate controls the activations of input flowing into the memory cell, the output gate controls the output cell activations flowing into the network, and the forget gate will scale the internal of the cell before adding it as input to the cell through the connection of the cell. The LSTM algorithm is mostly implemented using the Keras package for training and testing datasets [22]. The capability of LSTM to remember information for a long time with good accuracy make it becomes powerful for forecasting. This study applies a moving-forward window technique to predict the next time step [23][24][25]. The selection of the number of hidden layers, number of neurons, number of epochs, and batch size play an important rule in the implementation of Long-Short Term Memory. So, in this study, these parameters are selected based on trial and error with a range of 1-275 neurons, 1-250 batch size and a number of the epoch with a range of 1-285 are evaluated until it converged into close results with the observed data. In this case, the window of size 50 is used for moving forward to make a one-step prediction, which means that, the first 50 data points are used as out input X to predict y1-51st data point, next is data point of window between 1 to 51 used as X to predict y2 and so on. The input data are normalized to (−1,1) using the min-max scaler technique before implementing the algorithm. Table 2 shows more configuration about LSTM network using two-layered LSTM architecture of 266 hidden neurons coupled with a dense output layer with linear as the model activation to predict with time steps 50 and number of features is 1. Maximum epoch is set to 285 with batch size 250 and the validation split is 0.09. Figure 3(a) shows the architecture of LSTM network, where it can be seen that each LSTM block receives the following signals: input signal (x), input gate signal (i), forget gate signal (f), recurrent signal (h), and produces output gate signal (o). From the diagram in figure 3(b), shows that each LSTM block receives several signals: input gate signal (i), recurrent signal (h), forget gate signal (f); and produces output gate signal (o).
The flow of the process in each LSTM memory block can be represented by the following diagram.

Error metrics
To evaluate the accuracy between observed data from mega solar Gifu weather station with simulated data from WRF model and LSTM algorithm, the two of three equations from David et al [26] such as root mean square error (RMSE) and mean bias error (MBE) are used to measure forecast accuracy and expressed in W/m 2 . Meanwhile, the normalization is done concerning the mean ground measurement of solar radiation in the considered period. In this study, the normalized RMSE (nRMSE) and the normalized MBE (nMBE) for error percentage given by equations (10) and (11), and expressed in % [27]. These four error metrics are defined as below; Where n is the number of the time step, pred represented WRF simulated data from domain d02 and LSTM algorithm and obs represented Gifu weather station observed data, Rmax represented the maximum value solar radiation of simulated and observed data, Rmin represented the minimum value solar radiation of simulated and observed data. All error estimates computed using hourly values for the considered period. Night values with no  indicators are presented for solar radiation forecasting. Due to the effects of cloud effective radius, cloud fraction, cloud water path, cloud cover, water vapor, and aerosol optical depth in the atmosphere would reduce the solar radiation. Meanwhile, the performances of a model in various terrains, water vapor absorption that varies with time and location, and climate zones may cause the reduction of solar radiation when solar rays pass through the atmosphere. In this present study, clear-sky conditions from the WRF model are produced on the days of 1-, 4-, 5-and 6 January 2017 from WRF model simulation, where on these days, there is no cloud presence. Small cloud from 20 kg m −13 to 160 kg m −13 observed from 1 pm to 4 pm on the 2nd of January as the cloudy day. Meanwhile, a small cloud from 10 kg m −13 to 90 kg m −13 observed from 1 pm to 10 pm on the 3rd of January, a cloudy day. A large amount of cloud, about 10 kg m −13 to 530 kg m −13 , is observed on the 8th of January from morning time from 8 am to 11 pm representing the rainy day for solar radiation forecasting. The cloudy days are days when solar radiation changed rapidly due to the presence of the cloud. These clouds presences provoke variations in solar radiation and make prediction difficult when the clouds do not block the direct radiation from the Sun. This result shows that the WRF model and LSTM algorithm solar radiation are quite good because both models and algorithms precisely predicted values that are almost close to the observed data from Gifu, although the weather type at 2160 h prediction is composed of a rainy day, sunny day and cloudy day. However, it is a highlight that the prediction of solar radiation values of these 2160 h can change rapidly based on hourly weather values. Particularly, the WRF model performs considerably better than the LSTM algorithm during the rainy day for the validation with the weather station. On the other hand, the WRF model performance shows a lower dependence on the time horizon during this part of the year, indicative of a higher ability of the model to properly forecast cloudy conditions.
The error calculation results of the WRF model and LSTM algorithm compared to the one meteorological station's daily measurements for three months from 1st January to 31st March 2017 are also presented in this work. The error metrics of MBE, RMSE, nRMSE, and nMBE of 2160 h predictions between WRF simulated data from domain d02 and LSTM algorithm comparing with the observed data are presented in table 3. As we can see, the two error metrics of the WRF model are lower compared to the LSTM algorithm in three months prediction for RMSE and nRMSE metrics. The RMSE of WRF is obtained 210 W m −2 lower compared to 310 W m −2 from the LSTM network. The more accurate prediction is obtained when the RMSE and nRMSE are getting lower. In the case of LSTM solar radiation, the RMSE varying a small comparing result from WRF model simulation. The MBE of the WRF model is observed about 96 W m −2 , on the other hand, it is obtained minus 101 W m −2 for the LSTM algorithm. The overestimation of the WRF for the MBE can be related to the limited ability of the WRF model. As the cloud increases, MBE value increases. Meanwhile, the LSTM networks showed almost no solar radiation forecasting at the end of February continue to the beginning of March. The nRMSE of WRF is lower compared to the nRMSE from the LSTM algorithm, 19% and 28%, respectively. The nRMSE of the WRF model has a value greater than the LSTM algorithm indicating the WRF model presented better results of solar radiation prediction. On the other hand, the value of nMBE from LSTM network was obtained also minus 9%. The MBE and nMBE values of the LSTM network in this study presented low values (lower than 4%) indicating that the solar radiation prediction was underestimated. Meanwhile, the nMBE value of solar radiation from the WRF model showed the best result with 9% compared to the other cases. This difference might be representative, but looking at these four error metric values, these only varied in a few units, therefore these differences are not considered representative.

Conclusions
In this study, the performance comparison between the LSTM network based on the algorithm and the WRF model for prediction purposes was conducted in Gifu, Japan. The three months data comparison were presented in this study because of there were only three months observed data available to the author for comparison purposes. The performance and configuration of the WRF model and LSTM algorithm used in this study successfully simulated and estimated the solar radiation forecasting and this study shows that these two models can be used for advance forecasting of any kind of variables with reasonable confidence. Despite inherent  19 9 limitations in this study, these two methods configurations were able to simulate and estimate solar radiation reasonably well, with simulated and observed data of daily solar radiation. However, the WRF model performed better and more accurate values in close agreement with the observed data, as it is indicated in the table, the WRF error metrics were lower than LSTM networks. Also, the present results are indicative of the region of study as a whole and were obtained by using the results of one analyzed measurement station. The LSTM algorithm used three-years hourly weather to predict the output of future data of solar radiation. For the WRF model case, the 6-hourly input datasets of weather forecast were used to predict the output on the same days. Although this study shows that the WRF model performed considerably better than the LSTM algorithm, as it shows in the table of MBE, RMSE, and nRMSE error analysis WRF is smaller than the LSTM for 2160 h of solar radiation forecasting, however, this study also proposed to apply the LSTM algorithm for future prediction of any weather forecast. In this study, a solar radiation forecasting method is proposed for medium-term, where solar radiations are predicted under all sky conditions and presented the classifying of the results into the Sunny day, cloudy day and rainy day conditions. Solar radiation predictions from the WRF model and LSTM algorithm under rainy day conditions achieved the best results. The decrease in accuracy is caused by the presence of different cloud layers. Finally, it is concluded that the presented two methodologies (WRF and LSTM) are very useful tools for technologies related to solar energy forecasting, such as in a solar farm, where the solar forecasting data can be included as decision systems for the correct adaptation of solar energy management systems and improve the quality of electricity generation for a long-term analysis period. In the near future, this study will continue to use and conduct solar radiation comparison with three more different methods -GEFCom2014, M4, and ARIMA algorithms, where GEFCom2014 and M4 are much promising for forecasting purposes, and ARIMA algorithms which use the R package for forecasting. Meanwhile, in the near future, the study will also focus on the interval of the cloud as it is the most difficult for cloudy days.