Wenrui Li
Pengcheng Zhang
Hai Dong
Abstract
Short-term rainfall forecasting plays a critical role in meteorology, hydrology, and other related areas. Currently, data-driven approaches have made considerable progress in rainfall forecasting. However, these approaches suffer from the following major drawbacks. First, they do not accommodate a reasonable and effective model to mathematically represent the complete changes of a rainfall block. Second, scale division rainfall forecasting is overlooked in existing literature. Third, generalization is not well validated in these approaches. To address these issues, we propose a novel Regional Scale Division Forecasting model using attention mechanism and long short term memory network (RSDF-AM-LSTM) for short-term and scale-division rainfall forecasting. The forecasting model can take full account of the regional characteristics of the rainfall blocks. It is established by employing a method similar to image evolution. The approach also provides a reasonable and effective model to formulate the complete changes of rainfall blocks, including newborn, splitting, strengthening, weakening, and merging. A deep learning algorithm based on AM-LSTM is proposed to forecast the change of rainfall block. AM-LSTM describes temporal and spatial association among four parallel LSTMs through AM. The rainfall data used in our experiments are obtained from 3,200 meteorological stations in and around China. The experimental results show that RSDF-AM-LSTM outperforms two popular atmospheric models and other traditional machine learning approaches.
1 INTRODUCTION
Rainfall is one of the most common weather phenomenons. It is extensively studied in the fields of climatology, hydrology, and agrometeorology [37]. Short-term rainfall forecasting plays a critical role in these fields [33] and related government decision-making [34]. Short-term rainfall forecasting usually refers to rainfall forecasting in 1 or 2 days ahead [47, 48, 49]. Currently, many studies are focusing on rainfall forecasting [24]. Substantial theories and models have been developed for rainfall forecasting [4, 48]. However, a large-scale and high-precision rainfall forecasting system is still urgently required in some fields, such as meteorology and hydrology. This results from two main challenges: (i) the complex and dynamic changes of inner atmosphere and (ii) the strict requirements on real-time short-term rainfall prediction. Rainfall forecasting results are communicated to governments and the public via the Internet, short messages, phones, and so on. However, governments and the public have no intuitive sense of the specific rainfall values. What they usually provided are rainfall levels (e.g., light, medium, or heavy) or numeric values (e.g., probabilities).
There are two primary ways to predict rainfall: atmospheric models and data-driven methods. Atmospheric models follow the physical formation mechanism and perform numerical calculation [17]. At present, atmospheric models form an important basis for rainfall forecasting. Rainfall is predicted by solving the atmospheric motion equations using atmospheric models. Due to a large amount of data and the complexity of calculations, the realization of atmospheric models usually relies on high-performance computers, which requires extensive computing resources [6]. Consequently, atmospheric models are often implemented by regional government organizations [16].
Data-driven approaches are usually based on statistical or machine learning theories [26, 43]. In comparison to atmospheric models, data-driven approaches have a faster prediction speed at a lower computing cost in case there are a large amount of rainfall data available. Therefore, there is a growing demand on designing more suitable and high-performance statistical approaches or machine learning algorithms in this area [11, 22]. Fitting and classification are common goals and means in machine learning. Rainfall forecasting products of machine learning approaches are usually divided into two categories: numerical results and level results [50]. For certain machine learning approaches, rainfall values can be predicted via fitting. Some other machine learning approaches can directly predict rainfall levels through their classification ability.
Artificial Neural Network (ANN) is one of the most active branches in machine learning models. It is regarded as an effective classification, fitting and labeling tool. The traditional ANN models used for rainfall forecasting include Feed-forward Neural Networks (FNNs), Radial Basis Function Neural Networks (RBFNNs) and BP Neural Networks (BPNNs) [19, 29]. In recent years, Deep Neural Networks (DNNs) have been developed rapidly in the application fields of image recognition, speech recognition, natural language processing, and so on [32]. Compared with traditional ANN models, DNNs have more powerful learning ability that can learn deep links in data or models. DNNs are an effective means of classification and prediction. Common DNN structures include Multi-layer Perceptions (MLP), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM), Convolutional Neural Networks, deep belief network (DBN), and so on.
Currently, researchers have made many attempts to forecasting rainfall using different kinds of DNNs [14, 30, 31, 47, 49]. However, most studies are usually limited to the time series forecasting of rainfalls in a region. The relationship between rainfall and environmental physical quantities (such as temperature, cloud, wind, and pressure) is only studied from the temporal dimension [49]. These studies usually use environmental physical quantities as inputs and the actual precipitation as outputs. The relationship between physical quantities and rainfall is learned by DNNs. These models and algorithms seem reasonable for rainfall forecasting in single geographical points. Nevertheless, a rainfall process is an integrated system. There is interference among different regions. For example, a heavy rain in an area tends to cause light rains in the surrounding areas. Consequently, the single point-based forecasting is less persuasive. In general, from the previous descriptions, we summarize the three major limitations in current DNN-based rainfall forecasting:
Limitation 1: There is no DNN-based approaches considering both the temporal and spatial characteristics of a rainfall to the best of our knowledge. A rainfall is continuous in time and space. Current DNN-based approaches only consider the continuity of a rainfall in time and ignore the continuity in space. Most approaches do not contain an effective model to mathematically represent the spatiotemporal changes of a rainfall block, thus leading to weaker forecasting performances.
Limitation 2: Current DNN-based approaches mostly focus on forecasting a specific rainfall value for a single area. In contrast, scale division refers to the magnitude of a rainfall, which can be divided into light rain, moderate rain, heavy rain and torrential rain, according to the meteorological operational standards. A large rainfall may contain multiple rainfall blocks with different magnitudes. In comparison to single value-based rainfall forecasting, scale division-based rainfall forecasting is able to forecast a rainfall by precisely predicting the magnitudes of its comprised rainfall blocks. This can more intuitively reflect the geographical distribution of rainfall intensity and local weather conditions [51]. In addition, short-term rainfall forecasting can better assist weather observers and government departments to make flexible decisions and quick responses [36]. At present, to the best of our knowledge, there is no study carried out on scale division-based short-term rainfall forecasting.
Limitation 3: The generalization of the existing DNN-based approaches is not well validated. The existing approaches are mostly tested in smaller datasets collected by limited numbers of meteorological observation stations. Whether or not an existing approach is feasible for large-scale data environments has not been fully verified in existing studies.
Rainfalls are influenced by time and space. RNNs and LSTM are popular DNNs to deal with time series [7, 38]. Compared with RNN, LSTM contains a memory structure, which is the mainstream deep network suitable for modeling time series. Regional rainfall is an integrated weather system, which contains massive information about the weather. The observed weather data usually include thousands of meteorological stations throughout the country. The evolution of the rainfall block can be foreseen within the next several hours. The movement and development of a rainfall block result in the change of rainfall areas and intensity. It is, therefore, necessary to forecast the evolution of the complete rainfall block to predict a large-scale rainfall. LSTM is a compelling solution to forecast rainfall change. However, LSTM can only be used to represent temporal correlations other than spatial correlations. Attention Mechanism (AM) includes both temporal attention mechanism and spatial attention mechanism. Effective employment of AM can improve LSTM’s descriptive capability of not only temporal correlation but also spatial correlation. A new deep learning algorithm based on attention mechanism and long short term memory network (AM-LSTM) is proposed to make up the limitation of LSTM in the spatial descriptive capability. In summary, the main contributions of this article are described as follows:
To address Limitation 1, we propose (1) a series of new models simulating the spatiotemporal change of rainfalls and (2) a novel forecasting algorithm (AM-LSTM), by fully considering the characteristics of rainfalls.
To address Limitation 2, we propose a novel Regional Scale Division Forecasting approach using the AM-LSTM algorithm (RSDF-AM-LSTM) based on the proposed rainfall model. The forecasting model can predict short-term rainfall with different magnitudes.
To address Limitation 3, the generalization of the model has been rigorously validated based on a large-scale and comprehensive real-world rainfall dataset. The experimental dataset includes the rainfall data obtained from 3,200 meteorological stations in and around China from 2015 to 2017. The experimental results demonstrates the superior performance of RSDF-AM-LSTM in comparison to two popular atmospheric models and several existing machine learning models.
The rest of this article is organized as follows. Section 2 reviews the related works. Section 3 presents the main concepts related to the research. The rainfall process model is provided in Section 4. Section 5 describes the details of the RSDF-AM-LSTM model. A set of dedicated experiments for RSDF-AM-LSTM is conducted in Section 6. Section 7 concludes the article and provides a list of suggestions for future work.
2 RELATED WORKS
2.1 Traditional Atmospheric Model
Generally speaking, atmospheric models are a set of partial differential equations. These equations contain common variables, one of which is time [20]. The function of atmospheric physical quantities along with time can be achieved by solving the system of partial differential equations. The atmospheric models have powerful prediction ability. They are the main reliance of rainfall forecasting in meteorological operations in many countries at present [2]. Nevertheless, there are two main problems in atmospheric models. First, the exact solutions are difficult to be obtained in the partial differential equations. Instead, only approximate solutions can be deduced, which, however, contain errors. The scale of errors will be magnified with the iterations of the approximate solutions. Second, the computational complexity of atmospheric models is high. They are usually implemented by mainframe computers. The implementation of the atmospheric models is quite difficult considering the required high computing resources. The atmospheric models used for China’s rainfall forecasting mainly include the Japan meteorological and numerical weather prediction model (JAPAN) [35], the European Centre for the Medium Range Forecasts (ECMWF) [25], the Regional Assimilation Prediction Systems [39], and so on. In general, atmospheric models are usually used as benchmarks for comparisons.
2.2 Data-driven Forecasting
Data-driven forecasting approaches mainly include statistics-based methods and machine learning-based methods. Statistics-based methods are a branch of mathematics that studies the characteristics of data distribution. This kind of method has strict mathematical proof processes. Machine learning-based methods use learning-based algorithm theories in computer science. Although the proving of machine learning-based methods is not as strict as statistical methods, the application scope and applicability of the former far exceeds the latter [3, 5, 8].
The commonly used statistical methods include linear regression and time series analysis. In general, time series analysis is a series of typical multivariate statistical analysis methods. It is often used to forecast rainfall according to the time series characteristics of rainfalls. However, it does not consider the impact of climate and other environmental factors on rainfalls. Instead, time series analysis only attempts to grasp the general directions of rainfall changing trends [10].
Machine learning-based methods allow computers to gain knowledge through a large number of data. The knowledge can be represented by intrinsic or functional relationships among feedings. Machine learning-based methods play a critical role in fitting, classification, prediction, recognition, and so on. Rainfalls can be considered as a result of changing atmospheric environments. Therefore, rainfall is affected by environmental physical quantities. They can be viewed as functions of physical quantities. Machine learning-based methods can learn the relationships between meteorological factors and rainfall. Currently, many machine learning-based methods are applied in rainfall forecasting, including Support Vector Regression, Grey Forecasting, Support Vector Machine (SVM), ANN, and so on [9, 13]. Compared with statistics-based methods, these methods can obtain better rainfall forecasting performance [45]. However, the predictive performance of machine learning algorithms is diverse in different problem settings. Consequently, it is of particular importance to select the suitable machine learning algorithm against a specific rainfall forecasting problem.
Neural networks have been broadly used in rainfall forecasting. Traditional neural networks include FNNs, BPNNs, RBFNNs, and so on. Farajzadeh et al. [9] adopt FNN contains a hidden layer to forecast rainfall in the Urmia basin, northern Iran. Abbot et al. [1] employ a traditional neural network to forecast monthly and seasonal rainfall in Queensland, Australia. The network takes meteorological or climatic factors, such as temperature, as inputs and the rainfall as output. Wu et al. [40] utilize 500-hPa height field as well as the sea surface temperature field as the inputs of principal component analysis. RBFNN is then used to predict the average rainfall in central Guangxi, China. Traditional networks usually contain a single hidden layer, which is simple and shallow. They are sometimes not suitable for some complex problems. The parameter adjustment algorithm of neural networks is also an important factor affecting the application and promotion of neural networks. The required computing power is increased dramatically.
Researchers defined networks that are more complex and contain more hierarchies than traditional networks, with the advancement of the machine learning theories. This type of network contains many layers and more complex structures. They are generally termed as DNNs [28, 41, 44]. At present, different types of DNNs are utilized to predict rainfall in various countries [18]. They usually take meteorological or some physical factors of one area as the input of a DNN, and the actual precipitation of the area as the output. The internal relationship between those factors and the precipitation is learned for rainfall prediction. Lin et al. [21] employ MLP to forecast typhoon. By using a self-organizing mapping algorithm, they define some possible physical factors including typhoon positions, moving directions, moving speed, air pressure, and so on. Zhang et al. [49] establish a novel prediction model using the DBN. The proposed model is validated using the rainfall data in southern China. Mekanik et al. [23] analyze the relationship between physical factors and rainfall in Victoria, Australia, and forecast spring rainfall using MLR and MLP. Shi et al. [31] employ an LSTM network for rainfall forecasting in Hongkong, China. The model can predict the structure and direction of clouds by using the radar picture data. It can more accurately predict short-term rainfall than other approaches. Zhang et al. [47] further propose a novel model, namely, dynamic regional combined short-term rainfall forecasting (DRCF). The experiments show that DRCF can outperform existing approaches. However, as we already presented, these approaches do not model and simulate the complete rainfall blocks and also their generalization is not well validated.
To address existing problems, we propose a Regional Scale Division Forecasting model using AM and LSTM, namely RSDF-AM-LSTM.
3 PRELIMINARIES
3.1 Criteria for Rainfall Levels
The rainfall levels are specifically defined in the national standards of the China Meteorological Administration. This national standard document is labeled as “GB T 28592-2012” [15]. The detailed specification of rainfall levels provides a standard not only for prediction services but also for the research and experiment of rainfall prediction methods. The rainfall levels in this national standard are respectively regulated for 12 hours and 24 hours, which are shown in Table 1. The data used in the experiment are collected every 6 hours. Therefore, the prediction frequency of the model is also set to every 6 hours to maintain the synchronisation. The rainfall on a certain day is mainly concentrated in a relatively short period. Comparing the 24-hour and 12-hour criteria, it can be found that the upper limit of the same level is mainly three-fifths of the relationship. Following the preceding rules, the 6-hour rainfall criterion is added in the last column of the table. Similarly, we limit the 6-hour threshold to two-thirds of the 12-hour threshold. There are two reasons for this. First, two-thirds can guarantee that the threshold is an integer. Second, two-thirds are larger than three-fifths to reflect the concentration of rainfall.
3.2 Long Short-Term Memory
LSTM is a time recursive neural network [42]. LSTM is suitable for dealing with and predicting time series-based events with relatively long intervals and delays. LSTM differs from RNN in that it adds a “processor” to the algorithm to judge whether the information is useful or not. The structure of the processor is called a cell. There are three doors in a cell, namely input gate, forgetting gate and output gate. When a piece of information is entered into the LSTM network, whether or not the information is useful can be judged according to the cell. Only the information conforming to the algorithm authentication will be retained, whereas the inconsistent information will be filtered through the forgetting door. The generalization of this technology is very good, resulting in a great variety of application opportunities [12, 46]. Various versions of LSTM have been proposed by researchers, which enables LSTM to better deal with long-time series. Gated Recurrent Unit Network (GRU) is a popular variant of LSTM. It simplifies the structure of LSTM while keeping its effectiveness. The specific structure of GRU is shown in Figure 1. The main structure of the network includes two gate structures: the update gate and reset gate. The parameters of the network mainly include the input and output dimensions. The calculation of the time \( t \) added to \( h_{t-1} \) shows that the information of the previous time can be retained. The feed-forward calculation of GRU is shown in Figure 1. The streamline arrows indicate the flow of information or data. LSTM provides several nodes to filter or retain information. These nodes need to train and adjust with the weight parameters. Therefore, the calculation of the network is mainly carried out according to the nodes. The main calculation formulas are as follows: (1) \( \begin{equation} f_t=\sigma (W_f\cdot [h_{t-1},x_t]+b_f), \end{equation} \) (2) \( \begin{equation} i_t=\sigma (W_i\cdot [h_{t-1},x_t]+b_i), \end{equation} \) (3) \( \begin{equation} \tilde{c}_t=\sigma (W_c\cdot [h_{t-1},x_t]+b_c), \end{equation} \) (4) \( \begin{equation} o_t=\sigma (W_o\cdot [h_{t-1},x_t]+b_o), \end{equation} \) (5) \( \begin{equation} c_{t}=\left(f_{t}\bigotimes c_{t-1}\right)\bigoplus \left(i_{t}\bigotimes \tilde{c}_{t}\right), \end{equation} \) (6) \( \begin{equation} h_t=o_{t}\bigotimes tan{c}_{t}, \end{equation} \) where \( f_t \), \( i_t \), \( \tilde{c}_t \), \( o_t \) are neuron operations. The function of \( \sigma \) can be adjusted. Its commonly used format is shown in the formula. \( t \) stands for the time \( t \). \( f_t \) is an update of the previous information. \( i_t \) is the retention degree of the information. \( \tilde{c}_t \) is the information obtained. \( o_t \) is the reset of information. Two types of multiplication operations are emphasized in the formula. The point multiplication represents the normal matrix and vector multiplication. The signal represents the multiplication of the corresponding elements of two vectors.
Fig. 1. Structure of LSTM.
4 RAINFALL PROCESS MODELLING
RSDF-AM-LSTM is primarily designed for achieving the goal of rainfall forecasting. It is expected that the model can forecast the movement, deformation and intensity change of a whole system of rainfall blocks. At present there is no such attempt in current rainfall forecasting models or even a formal modelling of the complete rainfall processes of a rainfall block. Therefore, it is necessary to effectively model the rainfall processes. The model can provide a solid foundation for the effectiveness of the proposed forecasting model. The rainfall process modelling will be explained in this section before the introduction of the forecasting model. This model can forecast rainfall blocks with different rainfall levels. The time series of the rainfall blocks can be modeled if we can map the corresponding rainfall blocks in a time sequence. This mapping is not limited to one-to-one. It can also be many-to-one or one-to-many. This model then uses AM-LSTM to learn the temporal variation of a rainfall block.
A rigorous modelling of the dynamic changes of a rainfall system is a prerequisite of an effective rainfall forecasting model. A rainfall system is used to describe the weather system in a certain period. It usually consists of one or more rainfall blocks. A rainfall block is used to describe a continuous and compact rainfall area in the national rainfall distribution map at a given time point. The key of rainfall forecasting is to forecast the moving path and shape change of these rainfall blocks. Rainfall blocks might appear, move, change and disappear over time. The evolution sequence of a rainfall block can be represented as a time series. Consequently, the study of a rainfall block is reformulated as the study of a special time series. Different from the traditional time series, the time series of rainfall blocks may generate a variety of changes. If \( S \) is used to denote a rainfall block, then the merging and splitting of the rainfall block may occur anytime. If there are \( m \) rainfall blocks at time \( t \), then there might be \( n \) rainfall blocks at time \( t + 1 \) (Equation (7)). \( m \) and \( n \) can be arbitrary natural numbers, including 0, (7) \( \begin{equation} (S_{1},S_{2},S_{3},\ldots ,S_{n})_{t+1}=f((S_{1},S_{2},S_{3},\ldots ,S_{m})_{t}). \end{equation} \) For the time series of individual rainfall blocks, splitting and merging may occur anytime. The splitting of rainfall blocks means that a rainfall block at time \( t \) being divided into multiple rainfall blocks at time \( t+1 \) (Equation (8)). The merging of rainfall blocks refers to multiple rainfall blocks at time \( t \) being combined to form a rainfall block at time \( t+1 \) (Equation (9)), (8) \( \begin{equation} (S)_{t+1}=f((S_{1},S_{2},S_{3},\ldots ,S_{m})_{t}), \end{equation} \) (9) \( \begin{equation} (S_{1},S_{2},S_{3},\ldots ,S_{n})_{t+1}=f((S)_{t}). \end{equation} \) Rainfall blocks contain multi-dimensional features, such as location, shape, intensity, and so on. The size of a time series feature dimensions determines the accuracy and difficulty of a prediction. The general mathematical expression of a rainfall block is described as (10) \( \begin{equation} rainfall \ block (S) :=[x_1,x_2,x_3,\ldots ,x_r], \end{equation} \) where \( x_i \) is used to describe the parameters of rainfall blocks, which are slices of multivariate time series, and \( r \) is the number of parameters. These parameters may include longitude, dimension, width, length, rainfall intensity, and so on. For the forecasting of rainfall blocks, the evaluation standard is the deviation between the predicted and actual rainfall blocks, which are expressed as: (11) \( \begin{equation} [x_1,x_2,x_3,\ldots ,x_r]=arg\ min\sum _{j}\sum _{i=1}^{r}(x_i-\widehat{x_i})^{2}, \end{equation} \) where \( x_i \) denotes the parameters of the forecasted rainfall block, \( \widehat{x_i} \) denotes the parameters of the actual rainfall block, \( r \) denotes the number of parameters or the dimension of time variables. The goal of rainfall block forecasting is to minimize the deviation.
5 REGIONAL SCALE DIVISION FORECASTING MODEL BASED ON AM AND LSTM
5.1 Overview
According to the aforementioned process, RSDF-AM-LSTM is mainly composed of three steps, as described in the following:
The required data are collected and preprocessed. In this step, the location, the shape, and intensity of rainfall blocks are obtained from site data.
The rainfall blocks of different rainfall levels are identified. The temporal variation of these rainfall blocks is modeled by the time aware rainfall mapping.
The temporal variation of the rainfall blocks is learned by AM-LSTM. Then, the trained AM-LSTM and Softmax classifier are used for rainfall forecasting.
Figure 2 shows the whole execution process of RSDF-AM-LSTM, which is divided into three steps. The first step includes data collection and preprocessing. The main task of preprocessing includes data extraction, data standardization, and rain value added and average. The model divides a map into grids, collects rainfall values of meteorological stations in the same grid, and averages the values as the rainfall values of the grid. The second step is segmentation and mapping of rainfall blocks. During this step, we first recognize the classified rainfall blocks, followed by composing rainfall blocks into rainfall systems, to map between rainfall systems and blocks over time. The third step is rainfall forecasting using AM-LSTM. During this step, the model uses the Softmax classifier to predict the newly generated rainfall blocks. Finally, the forecasting of a rainfall is realized.
Fig. 2. Main process of RSDF-AM-LSTM.
5.2 Data Collection and Preprocessing
In the data preprocessing stage, the distribution of rainfall blocks is obtained from discrete site data. Observation data of ground meteorological stations are commonly used for rainfall forecasting. Real-time rainfall is mainly observed by meteorological stations. The distribution of meteorological stations is discrete other than continuous in geographical space. The site density is diverse in different areas. There are usually errors in the data for township sites. The density of meteorological stations determines the accuracy of the identification and prediction of rainfall blocks. In some parts of China, there may be multiple meteorological observation stations in a small area, such as the eastern coastal areas. In contrast, in some areas, meteorological observation stations may be very sparse due to the influence of topography and other factors, such as the western plateau of China. Rainfall blocks may cover a large area. Some of them are densely distributed in meteorological stations, while others are contrary. Therefore, uniform specifications of rainfall systems are needed. According to the distribution of existing meteorological stations, the longitude and latitude range of 1 degree*1 degree is used as an elementary spatial grid unit in the model. The average rainfall of several meteorological stations in the same unit is regarded as the rainfall of the unit. Two statements are made on the rationality of this division:
The distribution of rainfall level presents continuity in geography from the arithmetic point of view. It is, therefore, reasonable to use the average of rainfall in a spatial grid unit.
The segmentation accuracy will be higher and higher by using meteorological data from more meteorological stations and data acquisition of higher accuracy. It can completely replace the operation at this stage if there is a high-precision rainfall map.
5.3 Segmentation and Mapping of Rainfall Blocks
In this step, we provide effective samples for AM-LSTM by classifying and mapping rainfall blocks. Rainfall forecasting is very complicated. Multiple rainfall systems may run simultaneously. Each rainfall system may contain multiple rainfall blocks. Continuous rainfall detection and accurate tracking of rainfall blocks are difficult. We propose the technique of rainfall level division and rainfall block mapping to obtain effective time series samples of rainfall blocks.
We need to predict the movement and development of a rainfall system. We divide rainfall systems into different levels and identify the rainfall systems according to the levels to predict the intensity change of the systems. Our solution performs rainfall forecasting from the perspectives of the movement of rainfall blocks and the change of rainfall intensity. The data abstraction of rainfall blocks needs to reflect the location and shape of rainfall blocks. Common descriptions of rainfall blocks include circles, rectangles, ellipses, and complex graphics. Rainfall systems can be described by different graphical representations in a two-dimensional plane, the parameters of which are as follows:
If a rainfall block is described by a circle, then there will be only three parameters: the two coordinate of the center of the circle and the radius of the circle.
If a rainfall block is described by a rectangle, then there will be four parameters of two coordinates and length and width.
If a rainfall block is described by an ellipse, then there will be four parameters: ellipse coordinates, short radius, and long radius.
If a rainfall block is described by a complex image, then the parameters will be more than four.
The higher the number of parameters, the more accurate the description of a rainfall block shape and the more complex the shape to be grasped. Rectangles and ellipses are better choices for describing rainfall blocks. The rectangular representation is chosen here because rainfall data are usually in the form of matrices, (12) \( \begin{equation} rainfall\ block:=[x,y,le,wi(,mr)], \end{equation} \) where \( x \) and \( y \) are the longitude and latitude location of a rainfall center, \( le \) is the rainfall block length, \( wi \) is the rainfall block width, and \( mr \) is the maximum rainfall in the center (required only for storm levels). The matrix of a rainfall block is described by a quaternion \( [x, y, le, wi] \). A five-tuple can be used to characterize a large number of central rainfall blocks in the form of \( [x, y, le, wi, mr] \). \( x \) and \( y \) correspond to the lower left corner of the rectangle, \( le \) and \( wi \) are the width and length of the rectangle, and (\( mr \)) represents the maximum precipitation in the center. The movement of a rainfall block can be considered as the movement and deformation of this rectangle. Since the same rainfall block might contain multiple rainfall levels, we need to distinguish different rainfall systems to forecast their level changes. During this step, we first recognize classified rainfall blocks, then compose rainfall blocks into rainfall systems, and, finally, map rainfall systems and blocks over time.
5.3.1 Classified Rainfall Block Recognition.
Rainfalls can usually be categorized into light, moderate, heavy, and rainstorms, according to the levels in meteorological operations. Therefore, the identification and prediction of rainfall are also following these four rainfall levels. The rainfall levels in a rainfall block also fall in the same range. Four types of rainfall blocks are identified, corresponding to four types of rectangles. These four types of rectangles can be nested layer by layer, that is, the small rain block contains the medium rain block, the medium rain block contains the heavy rain block, and the heavy rain block contains the rainstorms block. The types of rectangular blocks in each rainfall block are identified. This process will be repeated until all the rain blocks are identified.
Figure 3 shows the structure of rainfall blocks in reality and illustrates the principle of rainfall block recognition. Generally, in reality, the rainfall of the rainfall block centre is the highest, and the rainfall of the rainfall block edges is the lowest. Rainfall is mainly caused by the convection of air. The rainfall center is usually with the strongest air convection, which has the highest rainfall. The rainfall in the surrounding areas decreases with their distance to the rainfall center, since the air convection in these areas become weaker [47]. Therefore, the rainfall shows some extent of continuity in space. The mechanism of rainfall block recognition is to describe the rainfall blocks by rectangles [15]. As shown in Figure 3, the rectangles with higher rainfall are nested in the rectangles with lower rainfall.
Fig. 3. Four levels of rainfall blocks in the actual rainfall block.
A rectangular representation of a rainfall block includes two parts: a main part and an expansion part. The location(s) of the main part is retrieved from the location(s) of the maximum rainfall in a rainfall block. The expansion part is retrieved outward centered on the main part. We try to retrieve the grid units that appear on the edge but still meet the minimum rainfall level criteria. The main part and the expansion part make up the rainfall block.
Algorithm 1 introduces the rainfall block recognition process. Lines 1–10 depict the recognition process of the main part. Lines 11–19 describes the recognition process of the expansion part. Figure 4 visualizes the process of a single rainfall block recognition. The same process can be directly applied to the four levels of rainfall blocks. The first four images show the identification process of the main part of a rainfall block. The red dot in the figure represents the grid with the highest rainfall in the rainfall block. A box centered around the red dot is expanded until it reaches the boundaries of the rainfall block. This box is then viewed as the main part. Because of irregular shapes of actual rainfall blocks, it is easy to miss the irregular parts by only using the main part. The expansion part is therefore leveraged to improve the recognition accuracy. The expansion part is obtained by expanding the boundaries of the main part in four directions. The expansion will terminate until the rainfall block periphery overlaps less than one third of each side. The last two images in Figure 4 shows the expansion part recognition process.
Fig. 4. The process of rainfall block recognition.
We use the rainfall distribution in China from 8:00 to 14:00 on June 4, 2015, as an example. The rainfall in China from 14 to 20 June 2015 is also used as a second example for comparison. The map ranges from 70\( ^\circ \) east longitude to 135\( ^\circ \) east longitude and 10\( ^\circ \) north latitude to 55\( ^\circ \) north latitude. There are multiple rainfall blocks within this range. These blocks are scattered in many areas. Their shapes, sizes, and intensity are different. Figure 5(a) and 5(b) show the distribution of the rainfall blocks and the component level identification of the rainfall blocks. Because of the complexity of the rainfall distribution, there are some deviations when describing rainfall blocks in a rectangular form. These deviations mainly exist in the description of the range of light rainfall. The rectangular frame of light rain contains a certain rainfall-free area. However, the deviations in the levels of moderate rain, heavy rain, and rainstorms are significantly smaller. Heavier rainfall forecasting is more significant, compared with light rainfall forecasting. The result of light rainfall forecasting is more likely to be represented as a probability.
Fig. 5. Classified rainfall blocks.
5.3.2 Composing Rainfall Blocks into Rainfall Systems.
There might be multiple scattered rainfall blocks in a large-scale rainfall system. At the same time, there might be multiple rainfall systems throughout the country, with multiple rainfall blocks in each system. If there are \( m \) rainfall systems in both time slots and \( n \) average rainfall blocks in each rainfall system, then the complexity of mapping rainfall blocks in the two-time slots is \( (m*n)^2 \). Therefore, the mapping of rainfall blocks becomes very complex. If we can classify the rainfall blocks according to their belonged rainfall systems, then the mapping problem will turn to two sub-problems, namely rainfall system mapping and rainfall block mapping. The complexity becomes \( m^2+m*n^2 \). We, therefore, try to merge the rainfall blocks into their belonged rainfall system.
A fully connected graph is built if we abstract all the rainfall blocks occurring at the same time as points. The weights between the points represent the distance between two rainfall blocks. We want to process each rainfall system separately. This needs us to divide the fully connected graph to subgraphs. The principle of segmentation is that all the rainfall blocks in a rainfall system need to be in the same weather system, that is, the distance between two rainfall blocks should be small enough. The range of a large-scale weather system is usually 1000–3000 km, that is, 10–30 longitude and latitude [27]. We limit a standard rainfall system’s North–South range to 10 latitudes and East–West range to 20 longitudes, by considering the wide East–West and narrow South–North characteristics of the weather system.
5.3.3 Rainfall Block Mapping.
The rainfall block mapping is to match between a pair of rainfall blocks in two chronologically ordered rainfall distribution maps. It can be used to deduce the movement and change of a rainfall block over time. The mapping result will be used for LSTM training. We divide the rainfall block mapping process into two sub-processes: rainfall system mapping and rainfall block mapping. We start with mapping the rainfall systems over time, followed by mapping the rainfall blocks within the mapped rainfall systems. The two mapping processes adopt similar algorithms. The mapping process is illustrated from two perspectives: the possible mapping types and the mapping algorithm (Algorithm 3).
Suppose that there are two chronologically ordered rainfall distribution maps, respectively, in time \( A \) and time \( B \) (\( A\lt B \)). There are five different mapping types.
Newborn: There is no rainfall system (block) in time \( A \) and there is a rainfall system (block) in Time \( B \).
Disappearing: There is a rainfall system (block) in time \( A \) and no rainfall system (block) in time \( B \).
Merging: Several rainfall systems (blocks) in time \( A \) compose to a rainfall system (block) time \( B \).
Splitting: The rainfall system (block) in time \( A \) is divided into several rainfall systems (blocks) time \( B \).
Changing: The rainfall system (block) in time \( A \) becomes another system (block) in time \( B \). It can be further divided into strengthening, and weakening. Strengthening means the later system is stronger than the former. Otherwise, it is considered as weakening.
We use the two rainfall distribution maps in Figure 5(a) and 5(b) to explain the outcome. Figure 5(a) and 5(b) are the results of Classified Rainfall Block Recognition. In the two figures, the darker the lattice color, the greater the rainfall is. From these figures, we can see that for a large rainfall block, there are multiple colored areas, which represent different rainfall levels. The task of Classified rainfall block recognition is to get these areas. In Composing rainfall blocks into rainfall systems, these rainfall blocks need to be grouped into different rainfall systems. The results are shown in Figure 6. Two small rainfall blocks in the previous moment are merged into System 3. In the latter moment, two rainfall blocks are merged into System 2. In Rainfall Block Mapping, the model needs to establish a mapping relationship between the former and latter rainfall blocks. The specific mapping results are shown in Table 2 and Figure 7. In Table 2, Columns 1 and 3 indicate the amounts of the rainfall systems. Columns 2 and 4 are the id of the rainfall blocks, which are shown in Figure 7. Line 1 indicates that the rainfall block numbered 1 at the former time is mapped to the rain block numbered 1 at a later time. Lines 2 and 3 indicate that the rainfall block numbered 2 at the former time is mapped to the rain blocks numbered 2 and 3 at the later time. Lines 4 and 5 indicate that the two rainfall blocks numbered 3 and 4 at the former time have dissipated.
Fig. 6. Classified rainfall blocks.
Fig. 7. Network structure diagram of AM-LSTM.
5.4 Rainfall Forecasting Using AM-LSTM
This article proposes AM-LSTM to fully consider the temporal and spatial continuity of rainfalls. AM-LSTM is a novel network model that integrates a time attention module and a space attention module on the basis of LSTM.
5.4.1 Attention Mechanism.
The rainfall block prediction can be transformed into a quadruple time series prediction problem, if we do not consider the spatial-temporal correlation between different rainfall blocks. This problem can be formalized as follows: (13) \( \begin{equation} [x_{t},y_{t},le_{t},wi_{t}]\rightarrow \cdot \cdot \cdot \rightarrow [x_{t+n},y_{t+n},le_{t+n},wi_{t+n}]. \end{equation} \) This problem can be addressed by training four networks. However, rainfall blocks of the four levels are nested. This type of time-space association (i.e., the area and location of these rainfall blocks change over time) cannot be reflected in the above solution, or it can be interpreted as lack of attention to the important relationship between each other. Therefore, our model introduces the attention mechanism.
Attention mechanism is a machine learning concept broadly applied in deep learning, image recognition and text prediction. According to the way human neurons work, our main consciousness is reflected in attention. In a neural network, the distribution of attention can be represented as different weights. The part with higher weight is the part with relatively concentrated attention. The application of attention mechanism in our problem context can be divided into spatial attention and temporal attention. Using attention mechanism, we can specify the key level of rainfall block driving the movement of a rainfall block.
5.4.2 AM-LSTM.
The core model of this method is a novel network model based on multiple LSTMs and combined with attention mechanism. The specific structure of AM-LSTM is shown in Figure 7, in which the four LSTM modules represent four independent parallel LSTM modules, and the spatiotemporal attention module is the core of AM-LSTM. The spatiotemporal attention module is based on the LSTM output cell matrix in previous time. This matrix size is initially designed to be 4 * 3. The four rows represent the outputs of the four rainfall block levels, indicating their spatial relations. The three columns represent the first three time slots, representing their temporal relations. The connection is established using the 12 cells and the 4 inputs at that time. According to the principle of attention mechanism, the main spatiotemporal information of rainfall blocks is utilized for training.
The main process of spatiotemporal attention module is depicted as follows. A rainfall block usually covers a large area. The physical quantities may vary dramatically within the same rainfall block. It is therefore unreasonable to take the average value of the physical quantity in the rainfall block. In the weather forecast operation, the key physical field characteristics, such as shear line, convergence area, and so on, are usually drawn during the manual analysis. The spatiotemporal attention module can make the model focus on the main physical variables (such as wind shear).
For AM-LSTM, only when the sequence length is greater than 3 can it be used. For the first three LSTM modules of the sequence, the calculation of cell is exactly the same as that of LSTM. For the fourth subsequent sequence, the attention mechanism is used in cell computing.
5.4.3 Calculation of AM-LSTM.
Compared with the classical LSTM, the forward calculation of AM-LSTM only adds one input cell group. The calculation formulae of the three doors are exactly same, but \( C_{t-1} \) in the LSTM needs to be replaced by \( l_{t-1} \), and \( l_{t-1} \) is calculated as follows: (14) \( \begin{equation} l_{t}=W_{cc}\bigotimes C=\sum _{j=1}^{4}\sum _{i=t-3}^{t-1}w_{i}^{j}c_{i}^{j}. \end{equation} \) Among them, \( W_{cc} \) is the cell weight matrix, which represents the spatiotemporal self attention mechanism of the cell arrays. The cell matrix \( C \) is expressed as (15) \( \begin{equation} C=\left(\begin{array}{ccc} c_{t-3}^{1} & c_{t-2}^{1} & c_{t-1}^{1}\\ c_{t-3}^{2}& c_{t-2}^{2} & c_{t-1}^{2}\\ c_{t-3}^{3}& c_{t-2}^{3} & c_{t-1}^{3}\\ c_{t-3}^{4}& c_{t-2}^{4} & c_{t-1}^{4}\\ \end{array} \right), \end{equation} \) where the superscripts represent the four rainfall block levels and the subscripts represent time. Compared with the classical LSTM, the cell value in the original formula is replaced by \( l_{t} \). The corresponding calculation only needs the following formula: (16) \( \begin{equation} c_{t}=\left(f_{t}\bigotimes l_{t}\right)\bigoplus \left(i_{t}\bigotimes \tilde{c}_{t}\right). \end{equation} \) The key principle of error back propagation of AM-LSTM is to calculate the gradient, and the error is adjusted according to the momentum SGD after the gradient being calculated. The gradient calculation includes two parts: time reversal error and space reversal error. Momentum SGD algorithm is still used in the calculation. First, the error of AM-LSTM is defined as (17) \( \begin{equation} J=\sum _{i=1}^{4}J_{i}=\frac{1}{2}\sum _{i=1}^{4}(y_{i}-\hat{y}_{i})^{2}, \end{equation} \) where \( y_{t} \) is the predicted value of the current Time \( t \) and \( \hat{y}_{t} \) is the actual value of the current Time \( t \). The calculation of time back propagation error is basically the same as that of a typical LSTM. Since \( J \) is the sum of the prediction errors in the four rainfall block levels, and the four basic units of the LSTM are parallel and independent of each other, \( J \) has no effect on the weight or the partial derivative of the intermediate time. The formula is calculated as follows: (18) \( \begin{equation} \frac{\partial J}{\partial W}=\frac{\partial J_{i}}{\partial W},\frac{\partial J}{\partial h_{t}}=\frac{\partial J_{i}}{\partial h_{t}}. \end{equation} \) To calculate the gradient of the cell weight matrix \( W_{cc} \), the error propagation along cells is defined based on the propagation of the LSTM along time errors and space errors. For the error back propagation in time and space, the calculation method of AM-LSTM is basically the same as that of a typical LSTM (only paying attention to the difference of cell value). The difference is the weight \( W_{cc} \) of an extra cell array in AM-LSTM. According to the chain rule, the 12 weights will be modified for each time error transfer, and the calculation method of each weight gradient is as follows: (19) \( \begin{equation} \frac{\partial J}{\partial W_{i}^{j}}=\frac{\partial J}{\partial h_{t}}\cdot \frac{\partial h_{t}}{\partial W_{i}^{j}}=\frac{\partial J}{\partial h_{t}}\cdot \frac{\partial h_{t}}{\partial l_{t}}\cdot \frac{\partial l_{t}}{\partial W_{i}^{j}}=\frac{\partial J}{\partial h_{t}}\cdot \frac{\partial h_{t}}{\partial l_{t}}\cdot c_{i}^{j}, \end{equation} \) (20) \( \begin{equation} \frac{\partial h}{\partial l_{t}}=o_{t}\bigotimes \frac{\partial tanC_{t}}{\partial l_{t}}=o_{t}\bigotimes \left(sec^{2}c_{t}\cdot f_{t}\right). \end{equation} \) AM-LSTM needs to use the output cells in first three time slots. Therefore, the cell weight matrix can only be modified if the sequence length is 4 or longer. For sequences shorter than 4, the LSTM calculation can still be used.
5.4.4 Forecast of Rainfall.
In the actual rainfall prediction, the aforementioned algorithm can predict the existing rainfall blocks, but it cannot predict new rainfall blocks. Therefore, a Softmax binary classification process is added before the prediction model to classify whether or not the grids without rainfall on a map are raining. If a rainfall is predicted, then the grid will be viewed as a rainfall block with an area of 0 for subsequent prediction. If the coordinate of the grid is \( [x, y] \), then the current time will be recorded as a quaternion \( [x, y, 0,0] \). In addition, the following heuristics are defined for the rainfall block prediction:
If there is no higher level rainfall block contained in a rainfall block, and the quaternion array of a lower level rainfall block is \( [x,y,le,wi] \), then the higher level rainfall block is recorded as \( [x,y,0,0] \);
If the area of a higher level rainfall block exceeds that of a lower level rainfall block in the same location and time, then the area of the lower level rainfall block will be expanded to contain the higher level rainfall block.
6 EXPERIMENTAL VALIDATIONS
We design a set of experiments to validate RSDF-AM-LSTM using the meteorological data from China Meteorological Administration.1 Our experiments aim to investigate three research questions:
RQ1: Which geometric shape is better for accurately describing a rainfall block? What size of rainfall block can make RSDF-AM-LSTM perform better?
RQ2: What is the forecasting accuracy of RSDF-AM-LSTM?
RQ3: Is RSDF-AM-LSTM better than state-of-art rainfall forecasting models or approaches?
6.1 Experimental Setup
All the experiments are performed on a Windows PC with Intel Core i3 CPU, 8G RAM, and Windows 7 operating system. All the images and data processing are implemented in MATLAB2013a.
Root mean square error (RMSE) is a commonly used index to evaluate the average error. It is also used for our experiments.
Threat score (TS) 2 is generally used to measure the proportions of correctly predicted samples, including accurately predicted precipitation and no rainfall. The detailed formula of \( TS \) score is described in the following: (21) \( \begin{equation} TS=\frac{N_{1}+N_{2}}{N_{1}+N_{2}+N_{3}+N_{4}}, \end{equation} \) where \( N_{1} \) represents the number of samples with correctly predicted precipitation, \( N_{2} \) represents the number of samples with correctly predicted no-precipitation, \( N_{3} \) represents the number of samples with incorrectly predicted no-precipitation, and \( N_{4} \) represents the number of samples incorrectly predicted precipitation.
6.2 Data set
The experimental dataset includes the surface mapping data, the atmospheric forecasting results and the altitude mapping data of China from 2015 to 2017 issued by China Meteorological Administration.3 The data format is text. The data of 2015 and 2016 are used as training data, and the data of 2017 are used as test data. The physical factors required by the experiments are obtained from the surface and altitude mapping data. The rainfall data in the forecasting areas are obtained from the surface mapping data. The results of the atmospheric models are used to compare with this model.
The east longitude is from 70\( ^\circ \) to 135\( ^\circ \), and the north latitude is from 10\( ^\circ \) to 60\( ^\circ \). There are 3,200 meteorological stations in this area, including 1,208 foreign meteorological stations. These weather stations are all above the county level, and they are shared by all countries in the world. Figure 8 shows the distribution of meteorological stations. The purpose of the data is to obtain a rainfall map covering China. The model identifies and predicts the rainfall blocks from these data.
Fig. 8. Distribution of meteorological station.
6.3 Experimental Results
The segmentation and recognition of rainfall blocks directly affect the accuracy of the forecasting. Forecasting accuracy is affected by the following aspects:
Description of the rainfall block shapes. When the shapes used to depict rainfall blocks are changed, the accuracy may be different. The variety of shapes also affects the range of rainfall forecasting. The model has deviation in the identification and prediction of some small rainfall blocks.
Accuracy of rainfall blocks mapping. The inaccurate mapping may result in a large number of error training samples.
Learning ability of AM-LSTM and classification ability of Softmax classifier. The learning ability of AM-LSTM directly affects the accuracy of rainfall forecasting.
There are no optimal solutions for the first and third points above. However, we can find a relatively more reasonable solution for better performance through experiments. The maximum runtime duration of the approach is within 10 minutes and the average runtime duration is 350 seconds. Since rainfall forecast is performed every 6 hours, it is impossible to describe the continuous evolution of a rainfall system. The runtime of the approach has little influence on the prediction results, as its runtime duration is far less than the requested forecast period. The focus of the experiment is to investigate the forecasting accuracy of the approach. For the second point, the 100% accuracy of the rainfall block mapping cannot be guaranteed. At present, cloud images or rainfall echoes are mainly taken by satellites or radars. These images and rainfall echoes do not represent accurate rainfall areas. They, therefore, cannot reflect the actual evolution of rainfall systems. Accurate rainfall areas can only be observed every three hours by ground observation equipment. The model uses actual rainfall data from ground observation equipment. This mapping can be rationalized by merging rainfall blocks into rainfall systems. The amount and distribution of rainfall blocks are dynamic in the sample data. The model screens out a class of samples as the training set to reduce the impact of mapping errors on the performance of the model. There is only one rainfall block at each time in this class of samples. The remaining samples, namely complex samples or samples that probably generate mapping errors, are used as the validation set. The experiment will validate the model from the above three perspectives.
6.3.1 Geometric Shapes and Sample Sizes.
For RQ1, we analyze the normal three types of shapes to depict rainfall blocks: circular, rectangular and elliptical. The model uses four rainfall levels: light, moderate, heavy, and storm rains to denote different rainfall intensity. The experiments are designed to compare the \( TS \) score when using these three shapes. We use a confusion matrix to describe the use of \( TS \) score in the experiment (Table 3). We use the example of heavy rainfall to explain how a confusion matrix is used. Event \( A \) refers to that a point is delineated by heavy rain graph. Event \( B \) refers to that the same point actually has heavy rainfall. The results are described in Table 4.
From the table, we can find that the rectangle shape has the best performance. The accuracy of the rectangle and the ellipse are close. Compared with the rectangle and the ellipse, the accuracy of the circle is obviously lower. These results are from the less number of parameters in the circular model. For small rainfall levels, the accuracy of the ellipse is higher than that of the rectangle. For larger rainfall levels, the accuracy of the rectangle is higher than that of the ellipse. In general, the overall forecasting effect of the rectangle is slightly better than that of the ellipse. Because the data are in the matrix form, rectangular processing is more convenient. Consequently, the rectangle is chosen as the shape to describe rainfall blocks.
The next experiment expects to screen out the samples with appropriate sizes for training. This is because samples with certain sizes would generate larger deviation regardless of the used shape types. If most of the samples in a similar size can be highly covered by their corresponding rectangular shapes, then this size can be viewed as an appropriate size. Hence, coverage accuracy is used to measure this description ability of the shape over sample size. We use heavy rain as an example. For a recognized heavy rain block (including lighter rainfall areas), if its predicted area is \( N \) and its actual area is \( M \), then the coverage accuracy is defined as (22) \( \begin{equation} Accuracy=N/M. \end{equation} \) The size of a rainfall block sample is defined as (23) \( \begin{equation} SampleSize=Lo*La, \end{equation} \) where \( Lo \) is its spanned longitude and \( La \) is its spanned latitude.
Figure 9 is a diagram of sample size versus rainfall block size. Logarithmic coordinates are used for both horizontal and vertical axes. The horizontal axis represents the size of the rainfall block. The size of the experimental map is 40*65 grid units. From the figure, we can see that most of the samples’ sizes are around 10 grid units. Figure 10 is a diagram of coverage accuracy corresponding to rainfall block size. Logarithmic coordinates are used for the horizontal axis. The vertical axis represents the average coverage accuracy of the corresponding rainfall block size. The coverage accuracy of rainfall blocks with the size of 10 grid units is the highest. With the increase of rainfall block size, the coverage accuracy decreases gradually. The greater the rainfall level, the higher the recognition accuracy. Especially for rainstorms, the coverage can reach more than 95%. According to the above statistics of rainfall block sizes and coverage accuracy, the model chooses appropriate samples as training samples. The sample selection criterion is that the area of a rainfall block is less than 20 grid units before and after the time.
Fig. 9. Sample Size Versus Rainfall Block Size.
Fig. 10. Coverage Accuracy versus Rainfall Block Size.
6.3.2 Accuracy.
For RQ2, we first conduct an initial experiment and conclude. Then, a set of large-scale experiments is then performed to validate the conclusion. In the AM-LSTM prediction stage, rainfall prediction is divided into two processes: the change prediction of existing rainfall blocks and the forecasting of new rainfall blocks. The \( Softmax \) function is the key technique for predicting new rainfall blocks. For the \( Softmax \) function, the process of parameter adjustment is very strict. Only the coordinates with obvious environmental physical quantities are predicted as the future rainfall points. In most cases, there are no new rainfall blocks. The new rainfall blocks mainly exist in the transition from sunny days to rainy days. An example of rainfall forecasting is given in Figure 11. This figure shows the actual rainfall evolution charts (above) and the predicted rainfall evolution charts (below) of China from 8:00 May 3 to 2:00 May 4 Beijing time. The time interval between every two neighbouring charts is 6 hours, that is, the first chart in the above column corresponds to 8:00 on May 3, the second chart corresponds to 14:00 on May 3, and so on.
Fig. 11. Comparison of Actual rainfall and Forecast rainfall series on May 3–4, 2017 (the legend can be referenced from Figure 6).
According to the forecasting results, we can see the general situation of the rainfall predicted by the proposed model. The main differences are as follows:
The predicted area of the light rain is larger than that of the actual light rain.
When a rainfall block area is very small, DRCF has no prediction ability. Compared with the actual rainfall map, the smaller rainfall blocks in the predicted rainfall map almost disappear.
The main advantages of the forecasting model are as follows:
The predicted rainfall areas are large enough to cover the actual rainfall areas.
The deviation between the predicted and the actual rainfall areas is small. The predicted locations of the rainfall areas are mostly consistent with the actual areas.
We have preliminarily analyzed the forecasting characteristics of the model through the above example. However, evaluating the forecasting ability of a model needs a large number of experiments and statistics. The large-scale experiment is to carry out rainfall forecasting for 3,200 meteorological stations around and in China. The forecasting period is from March to October 2018. The shortest forecasting time is 6 hours and the longest forecasting time is 48 hours. RMSE of the predicted rainfall is calculated by comparing the predicted and actual rainfall. The average TS score or accuracy can be calculated by averaging the rainfall prediction of all the weather stations. The accuracy varies with the increase of the forecasting time. According to the intensity of rainfall, five accuracy rates are defined: sunny accuracy (\( TS_{ra} \)), light rain accuracy (\( TS_{li} \)), moderate rain accuracy (\( TS_{mo} \)), heavy rain accuracy (\( TS_{he} \)), and rainstorm accuracy (\( TS_{st} \)), respectively. Figure 12 shows that the fluctuation of the five accuracy rates along with the forecasting time. We can see that the five accuracy rates decrease with the increase of the forecasting time. The prediction accuracy of 6 hours is the highest for all five accuracy rates. These first three accuracy rates are significantly lower than the other two. The accuracy of heavy rain and rainstorm is the lowest, and there is little difference between them.
Fig. 12. Fluctuation of Accuracy Rates along with Forecasting Time.
6.3.3 Comparisons.
For RQ3, we compare AM-LSTM with six traditional machine learning methods and two typical atmospheric models. BPNN and RBFNN are commonly used shallow neural networks for rainfall forecasting. BPNN and RBFNN are standard single hidden layer neural networks. There is a hidden layer between the input layer and the output layer. The number of nodes in the hidden layer is changed according to specific applications. Generally speaking, the number of hidden neurons in RBFNN does not exceed the input dimensions, while the number of hidden neurons in BPNN has no definite restriction. For RBFNN, the number of neurons in the hidden layer is traversed from 1 to the dimensions of the input layer. The RBFNN with minimal RMSE is optimal. Similarly, for BPNN, the number of neurons in the hidden layer is traversed from 1 to 40. The BPNN with minimal RMSE is optimal. DBN is composed of multi-layer RBM. RBM is a single hidden layer of neural networks. RBM is similar to encoders. The difference between RBM and encoders is that RBM is based on probability theories. There are two ways to build the last layer of DBN, including the \( Softmax \) functions and continuous functions. \( Softmax \) is used for classification, and continuous functions are usually used for fitting. The DBN in this experiment contains two layers of RBM and \( sigmod \) functions. ARIMA is a common time series analysis model. SVM is a classification or fitting method commonly used in machine learning. ARIMA does not consider physical quantity factors. It directly forecasts rainfall using the historic rainfall data.
First, the accuracy rate among six machine learning approaches is compared. The six machine learning methods include two shallow neural network models, two deep neural network models, and two non-neural network models. The two non-neural network models are commonly used forecasting models, including ARIMA and SVM. The two shallow neural networks are BPNN and RBFNN, and the two deep neural networks are DBN and AM-LSTM. The comparison results of the experiments (Table 5) show that AM-LSTM has better forecasting ability. Second, we compare the accuracy of AM-LSTM and two atmospheric models. The two atmospheric models include ECMWF and JAPAN. The experimental results show that the performance of AM-LSTM is better than two atmospheric models.
Table 5. Accuracy of AM-LSTM, Six Machine Learning, and Two Numerical Methods
LSTM can solve the problem of long-term information loss. Attention mechanism can enhance the learning ability of important information and thus improve the accuracy of the network. Accordingly, AM-LSTM has significant advantages in comparison to the other methods. The six machine learning methods are all built upon the same conditions. The inputs of the six machine learning methods are the same set of mapped rainfall blocks. The comparison between AM-LSTM and the numerical models is based on the prediction results. There is no difference among them. Table 5 compares the accuracy of AM-LSTM, the machine learning methods, and the two atmospheric models. It can be seen that the accuracy of AM-LSTM is better than the other machine learning methods, among which the accuracy of SVM and ARIMA is relatively lower, and the accuracy of DBN is only slightly lower than AM-LSTM. For the atmospheric models, the accuracy of ECMWF is higher than that of JAPAN. The accuracy of AM-LSTM and ECMWF is different for different rainfall levels. ECMWF has higher prediction accuracy for lighter rainfall levels. The accuracy of AM-LSTM is higher for heavier rainfall levels. AM-LSTM has more economic value than ECMWF considering the higher economic significance in heavier rainfall forecasting.
We further compared the performance between AM-LSTM and LSTM in terms of average prediction accuracy and prediction accuracy of the four rainfall block levels in Table 5. After adding attention mechanism, it shows that the average rainfall prediction accuracy of AM-LSTM is 2% higher than it of LSTM, especially the prediction accuracy of LSTM on light rain being improved by 6%. This is because the designed attention mechanism is able to capture the spatio-temporal conversion and correlation among the rainfall block levels.
7 CONCLUSION AND FUTURE WORK
There is still a lack of effective deep learning-based modeling approaches for rainfall forecasting. The main contribution of RSDF-AM-LSTM lies in its higher accuracy for scale-division rainfall forecasting compared with state-of-the-art approaches.
The future works are summarized below. First, further fine-grained parameters are considered. For instance, the description of rainfall blocks is one of the key factors to improve the accuracy of prediction. The process of describing the shape, segmentation and mapping of rainfall blocks can be further refined. Second, more effective DNN models for rainfall forecasting are needed. There may emerge DNNs that are more effective for rainfall forecasting. However, the modeling principle of RSDF can still be adopted.
Footnotes
- [1] . 2014. Input selection and optimisation for monthly rainfall forecasting in Queensland, Australia, using artificial neural networks. Atmos. Res. 138 (2014), 166–178.Google Scholar
Cross Ref
- [2] . 2018. Short-term precipitation forecast based on the PERSIANN system and LSTM recurrent neural networks. J. Geophys. Res.: Atmos. 123, 22 (2018), 12–543.Google Scholar
- [3] . 2014. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv: 1409.0473.Google Scholar
- [4] . 2016. A new rainfall forecasting model using the CAPSO algorithm and an artificial neural network. Neur. Comput. Appl. 27, 8 (2016), 1–15.Google Scholar
- [5] . 2002. Model selection for small sample regression. Mach. Learn. 48, 1 (2002), 9–23.Google ScholarDigital Library
- [6] . 2016. Convection-permitting models: A step-change in rainfall forecasting. Meteorol. Appl. 23, 2 (2016), 165–181.Google Scholar
- [7] . 2015. Complex rotation quantum dynamic neural networks (CRQDNN) using complex quantum neuron (CQN): Applications to time series prediction. Neur. Netw. 71, C (2015), 11–26.Google ScholarDigital Library
- [8] . 2014. Mining big data:current status, and forecast to the future. ACM SIGKDD Explor. Newslett. 16, 1 (2014), 1–5.Google Scholar
- [9] . 2014. Modeling of monthly rainfall and runoff of Urmia lake basin using feed-forward neural network and time series analysis model. Water Resourc. Industr. 7 (2014), 38–48.Google ScholarCross Ref
- [10] . 2012. Ensemble method based on anfis-arima for rainfall prediction. In International Conference on Statistics in Science, Business and Engineering (ICSSBE). IEEE, 1–4.Google Scholar
- [11] . 2015. Error modeling approach to improve time series forecasters. Neurocomputing 153 (2015), 242–254.Google ScholarCross Ref
- [12] . 2000. Learning to forget: Continual prediction with LSTM. Neural Comput. 12, 10 (2000), 2451–2471.Google ScholarDigital Library
- [13] . 2004. a new method for non-linear classify and non-linear regression: Application of support vector machine to weather forecast. Quart. J. Appl. Meteorol. 3 (2004), 011.Google Scholar
- [14] . 2015. Forecasting the weather of Nevada: A deep learning approach. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’15). IEEE, 1–6.Google Scholar
- [15] . 2015. A spatial-Temporal projection model for 10–30 day rainfall forecast in South China. Climate Dynamics 44, 5–6 (2015), 1227–1244.Google Scholar
- [16] . 2016. An improvement on the radiation effects of shallow convective clouds in GRAPES-Meso model. Acta Meteorol. Sin. 74 (2016), 715–732.Google Scholar
- [17] . 2016. Method to Determine microscopic parameters of PFC-2D numerical model. J. Northeast. Univ.: Natur. Sci. 37, 4 (2016), 563–567.Google Scholar
- [18] . 2015. Deep learning. Nature 521, 7553 (2015), 436.Google ScholarCross Ref
- [19] . 2014. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 4 (2014), 541–551.Google ScholarDigital Library
- [20] . 2019. Bulk-type cloud microphysics parameterization in atmospheric models. Atmosphere 29, 2 (2019), 227–239.Google Scholar
- [21] . 2009. A hybrid neural network model for typhoon-rainfall forecasting. J. Hydrol. 375, 3–4 (2009), 450–458.Google Scholar
- [22] . 2017. Nonparametric risk bounds for time-series forecasting. Comput. Sci. 18, 1 (2017), 1044–1083.Google Scholar
- [23] . 2013. Multiple regression and Artificial Neural Network for long-term rainfall forecasting using large scale climate modes. J. Hydrol. 503 (2013), 11–21.Google ScholarCross Ref
- [24] . 2017. Optimizing member selection for Neural Network ensembles using Genetic Algorithms. In Proceedings of the IEEE International Conference on Information & Automation for Sustainability. IEEE.Google Scholar
- [25] . 1997. The ECMWF ensemble prediction system. Meteorol. Appl. 4, 4 (1997), 301–304.Google Scholar
- [26] . 2016. Improving the accuracy of a flood forecasting model by means of machine learning and chaos theory. Neural Comput. Appl. 27, 5 (2016), 1129–1141.Google Scholar
- [27] . 2002. Multi-scale description of a Sahelian synoptic weather system representative of the West African monsoon. Quart. J. Roy. Meteorol. Soc. 128 (2002), 1229–1257.Google Scholar
- [28] . 2015. Deep Convolutional Neural Networks for large-scale speech tasks. Neural Netw. 64 (2015), 39–48.Google ScholarDigital Library
- [29] . 2015. Deep learning in neural networks: An overview. Neural Netw. 61 (2015), 85–117.Google ScholarDigital Library
- [30] . 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in Neural Information Processing Systems. 802–810.Google ScholarDigital Library
- [31] . 2017. Deep learning for precipitation nowcasting: A benchmark and a new model. In Advances in Neural Information Processing Systems. 5622–5632.Google Scholar
- [32] . 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489.Google ScholarCross Ref
- [33] . 2018. Prediction of heavy rainfall over Chennai Metropolitan City, Tamil Nadu, India: Impact of microphysical parameterization schemes. Atmos. Research 202 (2018), 219–234.Google Scholar
- [34] . 2016. Temperature, extreme precipitation, and diurnal rainfall changes in the urbanized Jakarta city during the past 130 years. Int. J. Climatol. 36, 9 (2016), 3207–3225.Google Scholar
- [35] . 2018. Japan meteorological agency/meteorological research institute-coupled prediction system version 2 (JMA/MRI-CPS2): Atmosphere–land–ocean–sea ice coupled prediction system for operational seasonal forecasting. Clim. Dynam. 50, 3–4 (2018), 751–765.Google Scholar
- [36] . 2017. Identifying climate services needs for national planning: Insights from Malawi. Clim. Policy 17, 2 (2017), 189–202.Google Scholar
- [37] . 2018. Estimation of groundwater recharge via percolation outputs from a rainfall/runoff model for the Verlorenvlei estuarine system, west coast, South Africa. J. Hydrol. 558 (2018), 238–254.Google Scholar
- [38] . 2016. Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38, 8 (2016), 1583–1597.Google ScholarDigital Library
- [39] . 2010. Parallelism of the GRAPES global model and its loading balance strategy. Acta Meteorol. Sin. 68, 5 (2010), 591–597.Google Scholar
- [40] . 2009. An empirical seasonal prediction model of the East Asian summer monsoon using ENSO and NAO. J. Geophys. Res.: Atmos. 114, D18 (2009).Google Scholar
- [41] . 2015. A regression approach to speech enhancement based on deep neural networks. IEEE/ACM Trans. Audio Speech Lang. Process. 23, 1 (2015), 7–19.Google ScholarDigital Library
- [42] . 2018. Video captioning by adversarial LSTM. IEEE Trans. Image Process. 27, 11 (2018), 5600–5611.Google Scholar
- [43] . 2017. Maximum margin partial label learning. Mach. Learn. 106, 4 (2017), 573–593.Google ScholarDigital Library
- [44] . 2013. Deep learning: Yesterday, Today, and Tomorrow. J. Comput. Res. Dev. 20, 6 (2013), 1349.Google Scholar
- [45] . 2000. Application of gray and fuzzy methods for rainfall forecasting. J. Hydrol. Eng. 5, 4 (2000), 339–345.Google Scholar
- [46] . 2011. CAPTCHA recognition method based on RNN of LSTM. Pattern Recogn. Artif. Intell. 24, 1 (2011), 40–47.Google Scholar
- [47] . 2018. Short-term rainfall forecasting using multi-layer perceptron. IEEE Trans. Big Data 6, 1 (2018), 93–106.
DOI: Google ScholarCross Ref - [48] . 2018. A deep belief network based precipitation forecast approach using multiple environmental factors. Intell. Data Anal. 22, 4 (2018), 843–866.Google ScholarDigital Library
- [49] . 2017. A deep-learning based precipitation forecasting approach using multiple environmental factors. In Proceedings of the IEEE International Congress on Big Data (BigData Congress’17). IEEE, 193–200.Google Scholar
- [50] . 2017. The statistical extended-range (10–30-day) forecast of summer rainfall anomalies over the entire China. Clim. Dynam. 48, 1–2 (2017), 209–224.Google Scholar
- [51] . 2020. An empirical ensemble rainfall nowcasting model using multi-scaled analogues. Nat. Hazards (2020), 1–24.Google Scholar
Index Terms
- RSDF-AM-LSTM: Regional Scale Division Rainfall Forecasting Using Attention and LSTM
Recommendations
Effective multinational trade forecasting using LSTM recurrent neural network
Highlights- Changes in international trade constitute a crucial topic in economics.
- We ...
AbstractChanges in foreign trade (export and import) constitute a crucial topic in international economics, international business management, and economic development. Numerous academics and industry leaders have sought effective means of ...
Power Load Forecasting Using a Refined LSTM
ICMLC '19: Proceedings of the 2019 11th International Conference on Machine Learning and ComputingThe power load forecasting is based on historical energy consumption data of a region to forecast the power consumption of the region for a period of time in the future. Accurate forecasting can provide effective and reliable guidance for power ...
Residual LSTM based short-term load forecasting
AbstractAs the modern energy systems is becoming more complex and flexible, accurate load forecasting has been the key to scheduling power to meet customers’ needs, load switching, and infrastructure development. In this paper, we propose a ...
Highlights- Propose a novel short-term load forecasting framework that integrates ResNet and LSTM.
Comments