Comparison of rainfall nowcasting derived from the STEPS model and JMA precipitation nowcasts

Quantitative precipitation estimation and precipitation nowcasting are important components of systems that aim at minimizing or managing flash flooding. This study used the Short Term Ensemble Prediction System (STEPS), one of the most advanced Quantitative Precipitation Forecast (QPF) systems currently available. The Japan Meteorological Agency (JMA) radar rainfall data (1-km resolution) from the Kanto region, Japan, covering various periods, were used in STEPS to generate ensemble nowcasts of rainfall. Hourlong 30-member-ensemble rainfall nowcasts were generated for five separate rainfall events using 5-minute time steps. The ensemble nowcasts were verified using radar rainfall data, and the results showed that the STEPS forecasts are in good agreement with the observed data for forecast periods of <1 hour. To check the performance of the STEPS model output in more detail, it was compared with JMA precipitation nowcast data, and both nowcasting datasets were also compared separately with rain gauge data. The skill scores suggest that STEPS generates more accurate nowcasts, especially for higher-intensity rainfall events. Combining all members of the STEPS nowcasting results appears to improve the reliability of short-term rainfall prediction, and the output of such ensemble nowcasts could be used in hydrological models to generate probabilistic forecasts in the future.


INTRODUCTION
Seasonal flooding, flash floods, and landslides pose a risk to life and livelihoods, and intense precipitation is the main driver of such events.Flash floods caused by torrential rainfall are typically highly localized in space and time, and in basins of only a few hundred square kilometers the response times are often less than an hour, meaning that there is little time available in which to forecast the flood and issue a warning (Georgakakos, 2006;Kobiyama and Goerl, 2007).The frequency of heavy rain and floods are projected to increase over many regions of the world in coming years (Hirabayashi et al., 2008).Research shows that the size of the flood-affected population could now be increasing in Japan (Hirabayashi and Kanae, 2009;Maki et al., 2012).Accordingly, improvements to short-term Quantitative Precipitation Forecasts (QPFs) will help to minimize or manage such rainfall-related disasters.To develop a monitoring and prediction system for extreme weather events, the TOMACS (Tokyo Metropolitan Area Convection Study for Extreme Weather Resilient Cities) research project has installed dense meteorological observation networks in collaboration with related government institutions, local governments, private companies, and residents in Japan (Maki et al., 2012;Nakatani et al., 2015).Dense radar networks with a high spatial (0.5-1 km) and temporal (ca. 5 min) resolution, and frequent satellite images, also contribute to the TOMACS observations.
The generation of short-term (i.e., 0-3 hr) future precipitation distributions from a sequence of radar images is known as precipitation nowcasting, but this approach is based solely on a simple extrapolation technique and its accuracy rapidly decreases with increasing lead time (Dixon and Wiener, 1993;Mecklenburg et al., 2000;Fox and Wilson, 2005).This is because storm initiation, growth, and subsequent dissipation of the echo are not included in the process.Several methods have been used in precipitation nowcasting (Tuttle and Foote, 1990;Dixon and Wiener, 1993;Johnson et al., 1998;Germann and Zawadzki, 2002;Seed, 2003).However, all of the models are based on the advection echo of the vector field, and a number of strategies for this extrapolation have been considered in the literature; e.g., Eulerian or Lagrangian persistence (Dixon and Wiener, 1993;Germann and Zawadzki, 2002), area tracking, cell tracking (Johnson et al., 1998;Dixon and Wiener, 1993;Yoshida et al., 2012), and spectral algorithms (Seed, 2003).Each algorithm has its own benefits and limitations.MAPLE (McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation) is a QPF algorithm that has been under constant development since the early 1970s (Bellon and Austin, 1978).In this model, the radar echo velocity field is determined using variation echo-tracking, and the model generates a nowcast using Lagrangian advection of the echo map (Germann and Zawadzki, 2002).A typical example of an area-based algorithm is used in the TREC (Tracking of Radar Echoes by Correlation) approach, where the locations of the maximum cross-correlation coefficients between subgrids of successive radar data fields are determined to estimate motion (Tuttle and Foote, 1990).The TITAN (Thunderstorm Identification Tracking Analysis and Nowcasting; Dixon and Wiener, 1993) model is most widely used to detect storms, and its algorithm is based on a cell tracking system.Another cell tracking algorithm for volumetric radar data is SCIT (Storm Cell Identification and Tracking;Johnson et al., 1998).The major difference between TITAN and SCIT is that TITAN uses a single threshold to identify storms, whereas SCIT uses multiple thresholds to identify storms and the nearest neighbor method to track storms.S-PROG (a modified version of spectral prognosis) is a spectral decomposition model that uses scale-dependent temporal evolution to generate forecasts (Seed, 2003).The usefulness of such a decomposition model is that it allows for a degree of implicit uncertainty in the final rainfall nowcast.However, although several techniques have been used in precipitation nowcasting, most models focus on deterministic nowcasts at various lead times and provide no assessment of their uncertainty (Berenguer et al., 2011).
In general, nowcasting models do not include complex microphysical processes and give little consideration to the highly complex behavior of the atmosphere.Therefore, there is always uncertainty in the model outcomes, even if a sophisticated algorithm is applied.To address the physical uncertainties associated with precipitation nowcasting, a new concept known as ensemble nowcasting has been proposed (Bowler et al., 2006;Berenguer et al., 2011;Seed et al., 2013).This ensemble approach generates a number of forecasts based on the uncertainty of the initial stage of rainfall and so is able to provide some sense of how likely an event is to occur.It is possible to generate a number of realistic future rainfall scenarios using various techniques, and each can be incorporated as an ensemble member within a QPF (Berenguer et al., 2011;Seed et al., 2013).Very few models provide ensemble nowcasting.One of them is the STEPS (Short Term Ensemble Prediction System) model, which generates ensembles of precipitation nowcasts and is discussed in the next subsection.
The objective of this paper is to improve our understanding of the impact of ensemble nowcasting on QPF, as this approach has yet to be used in Japan.In this study, we used the STEPS model to produce ensembles of rainfall nowcasts using data obtained from the Japan Meteorological Agency's (JMA) weather radar network.The performance of the members of the ensemble rainfall nowcasts was tested using observed data and JMA precipitation nowcast data.

STEPS model
STEPS is one of the most advanced QPF systems currently available.A detailed mathematical description of the STEPS model can be found in Bowler et al. (2006) and Seed et al. (2013).The model generates ensembles of precipitation nowcasts using observations from weather radar, and has been revised and extended to account for the effects of radar observation errors; to improve certain aspects of model design and performance, notably noise generation; and to extend the modeling framework to facilitate the combination of precipitation fields from multiple sources (Seed et al., 2013).S-PROG, introduced by Seed (2003), is used in the STEPS model to perform the precipitation nowcasts.The S-PROG model combines three components: estimation of the advection field, spectral decomposition, and the temporal evolution of precipitation.An explanation of these three components is presented in Supplement Text S1.STEPS produces the forecast by combining the extrapolations, and the NWP (Numerical Weather Prediction) and noise cascades are given appropriate weights proportional to the skill of the extrapolation and NWP forecasts.NWP data is mainly used for longer period forecasts (ca.1-6 hr).Bowler et al. (2006) reviewed the STEPS model and found that it attempts to capture all sources of uncertainty in the motion during the development of the precipitation field by generating ensemble nowcasts.

DATA SETUP AND ANALYSIS
This study focuses on the Kanto region of East Japan (Figure 1).In Japan, approximately 80% of the country is covered by hills and mountains.The Kanto region covers a wide range of mountainous areas in the West region and a flat area in the East region where one of the largest urbanized areas (e.g.Tokyo) is located.It is a good idea to include rainfall events over such mixed topography in the selected domain.We used estimated rainfall data collected by the JMA weather radar network, which covers the whole country at a resolution of 1 km every 5 minutes using C-band radar observations (Nagata, 2011).Five separate rainfall events were analyzed based on their type and impact in the region (Table I).
In their original format, the JMA radar data cover the whole country at a spatial resolution of 1 × 1 km, and consequently include a large number of grid cells.The standard default grid size of input rainfall data for the STEPS model is 256 × 256 grid cells, but it is possible to increase the number of cells used in the model.In this study, we used the default grid size and resampled the JMA rainfall data to match the grid size that covers the Kanto region (Figure 1).
In the case of qualitative verification, three contingencytable scores (Wilks, 2006); i.e., the Probability of Detection (POD), the False Alarm Ratio (FAR), and the Critical Success Index (CSI) were obtained, and to test the nowcasting results quantitatively, four statistical tools (i.e., the Mean Absolute Error, MAE; the Root Mean Square Error, RMSE; the Correlation Coefficient, COR; and the Mean Field Bias, BIAS) were obtained by considering each point of the grid of rainfall observations for different lead times.A mathematical explanation of verification tools is presented in Supplement Text S2.

RESULTS
The JMA radar rainfall dataset is abbreviated to R_JMA, and the ensemble rainfall nowcast from the STEPS model is referred to as ERN_JMA.In this study, we generated 30 ensemble members of the nowcast at each 5-minute time interval up to a total forecast period of 1 hour.An overview of 30 members of ERN_JMA and time series of average verification scores for a 30-minute lead time are shown in the Supplement Figures S1 and S2 respectively.A detailed explanation on performance of the STEPS ensemble nowcast is presented in the Supplement Text S3.

Comparison between STEPS ensemble and JMA precipitation nowcasts
JMA has developed a QPF product using both radar and rain gauge precipitation data, and issues a variety of forecasts, ranging in lead time from a few hours to several days, and including the NWP model (Saito et al., 2006;Nagata, 2011).The JMA precipitation nowcast (up to 1 hour) has a spatial resolution of 1 km and has been used for operational purposes, particularly for disaster prevention and mitigation in Japan.This nowcast is derived from a combination of Mesoscale Model (MSM) predictions and the extrapolation of radar/rain gauge precipitation data without considering NWP.The ongoing operational use of the JMA precipitation nowcast (abbreviated to FR_JMA in this study) system suggests it is able to successfully forecast localized heavy rainfall.
The JMA nowcasts are issued every 10 minutes.To match the time frames used by the ERN_JMA and FR_JMA data, a comparison test was carried out for every 10, 20, 30, 40, and 50 minutes for each time period.Five different rain events from the summer season were considered in this study.Each rain event shows different characteristics in terms of type, location and time.The average time series of the skill scores of ERN_JMA and FR_JMA data events were compared separately.We found that there was not much significant difference between the JMA and STEPS nowcast comparisons.However, skill scores of each event showed slightly different values, which may be due to the spatial coverage of rain echo in the domain, types of rainfall system and time periods of the events.Then, the average and standard deviations of the skill scores from all five events at the same lead times were averaged to generate a summary of the verification results (Figure 2).POD, CSI, and FAR from the STEPS ensemble nowcast (solid lines with error bars in Figure 2) and from the JMA precipitation nowcast (dotted lines in Figure 2) follow a similar trend.It should be noted that the skill score profile from the STEPS model represents the mean of 30 ensembles with error bars (standard deviation).It is clear that the range of the standard deviations for each lead time can be used to reduce the uncertainty associated with the nowcasting.Overall, the results from both nowcast systems are qualitatively similar to each other.
Nowcasting of higher rainfall rates is important if we are to predict the impact of these events on hydrology and water resources management.STEPS nowcast may give a good skill with high rain rates greater than 1 mm hr -1 (Bowler et al., 2006).Seed et al. (2013) highlighted that STEPS provides a cost effective means of addressing the generation of skillful, very short range rainfall forecasts, and the more effective quantification of forecast uncertainty.Therefore, we also performed the comparison test using various thresholds  of observed rainfall.In this study, thresholds of 5, 10, 15, and 20 mm hr -1 were considered.It can be possible to set a higher rainfall threshold but, there could be a high chance of sudden decrease of numbers of rain data which may reflect different results.It is believed that selected thresholds reflect some clue about the STEPS performance for higher rainfall rates.A summary of the comparison tests is shown in Figure 3.
The POD profile of the STEPS nowcast for the selected rainfall thresholds was better than the JMA nowcast and showed no distinct difference with increasing threshold; indicating that all occurrences of the event were correctly forecast for lower to higher rainfall rates.The FAR profiles of both nowcasts showed a similar trend for the >5 mm hr -1 rainfall threshold, and a slightly lower profile was found with increasing threshold in the STEPS nowcast; indicating that non-occurrences of the rainfall events were forecast to occur.The CSI mainly depends upon the relationship between POD and FAR.It is clearly seen that the CSI profile of the STEPS nowcast offers better improvement than the JMA nowcast for all selected thresholds (see Figure 3).Hence, based on the skill score profile, the performance of the STEPS model (based on the mean of the ensemble members) was better than that of the JMA nowcast for forecasts of <1 hour.Moreover, the changing size of the standard deviation at each lead time could be used to statistically assess the likely accuracy of the STEPS forecast; i.e., if all ensemble members are in good agreement the standard deviation will be low and the probability of an accurate forecast will be high, and vice versa.This could be of benefit to both nowcasting and other applications of the STEPS model.

Comparison with ground data
ERN_JMA data were compared with the data from 19 rain gauge stations of AMeDAS (Automated Meteorological Data Acquisition System), JMA.Most of these stations were located in the lowland areas of the domain.To demonstrate a time series comparison of nowcast data and gauged data (R_RG), we have selected Otaki station for the rain event of 03 July 2012.Figure 4 compares the time series of R_RG, R_JMA, FR_JMA, members of ERN_JMA, and Mean_ ERN_JMA (Mean of ERN_JMA members) for a 30-minute lead time at the Otaki station (red star in Figure 1).Each ensemble member has its own trend, and they differ in some cases and appear more sensitive at high rain rates.There is a close relationship between RR_JMA and R_RG in the time series data.A comparison of instantaneous rainfall rates derived from radar and ground-based gauges at any point may be subject to many errors caused by the spatial and temporal variability of rainfall (Maki et al., 2005;P.C. and Maki, 2014) and, practically, it is almost impossible to obtain a zero error between them.
The time series of the ERN_JMA members follows the general trend of the R_RG and R_JMA time series well, though with some differences, and this divergence is slightly greater at higher rain rates (Figure 4).The time series profile of FR_JMA (blue line in Figure 4) indicates that it is typically either overestimated or underestimated with respect to the R_RG data.To improve our quantitative understanding of the accuracy of the STEPS ensemble data and JMA forecast data when compared with the rain gauge data, a detailed quantitative statistical analysis of all selected gauge stations for 5 rain events is presented in the next section.

Quantitative verification of STEPS ensemble nowcasts
In the first step, the MAE and RMSE were calculated between the FR_JMA and R_RG data, and the ensemble members of ERN_JMA and R_RG at different lead times.To represent the 30 members of ERN_JMA, the average and standard deviation for all 30 members were calculated at each lead time (Figure 5).There were fewer errors between members of the STEPS ensemble nowcast and R_RG than between the JMA precipitation nowcast and R_RG for all lead times.Moreover, error bars on the profiles of MAE and RMSE can help to reduce the errors and so approach the true values more closely.We also obtained average MAE and RMSE values between R_JMA and R_RG of about 4.3 and 9.0 mm hr -1 , respectively, which indicate the uncertainties associated with the estimation of radar rainfall.This result indicates that the errors obtained from the nowcast data are small, but increase with increasing lead times.
Similarly, we also calculated the COR and BIAS in the same way as for MAE and RMSE above (Figure 5).The value of COR between ERN_JMA and R_RG was greater than that between FR_JMA and R_RG up to a lead time of 40 minutes.It should be noted that the RMSE profile at selected lead times is found lower between members of the STEPS ensemble nowcast and R_RG than between the JMA precipitation nowcast and R_RG, which suggests that there could be less reliability between the JMA precipitation nowcast and R_RG especially for large rainfall rates.In STEPS, uncertainties in the evolution of the precipitation pattern are modelled using the S-PROG model (Seed, 2003) and the rainfall distribution of different cascades is separated into different sizes of rainfall feature, so the large rainfall events are the more predictable (Seed et al., 2013).However, the value of BIAS was similar at all lead times.The average COR and BIAS between R_JMA and R_RG was about 0.63 and 0.85, respectively.It is difficult to calculate the error for each member of the STEPS nowcast; consequently, we instead used the mean with error bars (standard deviation) at different lead times (Figure 4).The range of the error bars was similar at all lead times for the averaged results.It is interesting that BIAS was less than one under all conditions, which suggests a slight underestimation by R_JMA, FR_ JMA, and the mean of ERN_JMA.
Finally, the spatial distribution of the 1-hour accumulated rainfall was obtained from R_JMA, FR_JMA, and the 30 members of ERN_JMA, to enable a comparison of the rain echoes.Figure 6 shows the accumulated rainfall from R_ JMA (Figure 6a) and FR_JMA (Figure 6b) for one hour (12:00-13:00 UTC) over the Kanto region on 03 July 2012.Similarly, the spatial distribution of the 30 members of ERN_JMA for the same period is shown in Figure 7.The selected 1-hour represents the peak rainfall hour over the  6 and 7.The exact location of the core of high-echo rainfall (located within the latitude and longitude of 34.95°N and 138.85°E in Figure 6a) from a single nowcast model may not be sufficient to track the exact size and location of the observed value (Figure 6a), which is one of the most significant errors associated with estimating actual runoff rates, especially for small watersheds.However, the combination of ensemble members from the nowcast model may also reduce such errors.

SUMMARY AND CONCLUSIONS
The focus of this study was short-period QPF and its application to hydrological forecasting.This paper intro-duces an ensemble rainfall nowcasting model and tests its performance against radar rainfall, rain gauge and existing short-term forecasting data in Japan.We generated a 30-member ensemble nowcast from the JMA radar data using the STEPS model.The nowcasting lead time was fixed to a very short period (< 1 hour).We considered five different rainfall events of different durations.The results from each event were compared with the estimated radar data, JMA precipitation nowcast data, and rain gauge data.Comparison between the rainfall nowcasting data from the STEPS model and the JMA precipitation nowcast data showed qualitatively similar trends up to 1 hour.There is a very high chance that the ensemble members from the rainfall nowcast may be able to reduce the uncertainties in predictions, especially for higher rainfall; however, dealing with the ensemble members remains a challenging problem.The skill of a radar advection nowcast decreases rapidly with lead time (Dixon and Wiener, 1993;Mecklenburg et al., 2000;Fox and Wilson, 2005).STEPS is based on the idea that the temporal development in Lagrangian coordinates is scale dependent and this can be modelled using a combination of a multiplicative cascade with autoregressive (AR) updates (Seed et al., 2013).Moreover, it is a stochastic ensemble that is designed to provide a spread in the ensemble that is equal to the forecast error, not a set of physical solutions to the observed meteorological boundary conditions as is the case for a NWP ensemble.The ensemble members of the rainfall nowcast from the STEPS model generate somewhat better results than the JMA nowcast data when quantitatively compared with the rain gauge data.
Although rain rates estimated from radar observations provide high-resolution spatio-temporal data for a given area, there are also some uncertainties associated with radar observations (Berenguer and Zawadzki, 2008;P.C. and Maki, 2014).In such situation, the single nowcasting of rainfall may be insufficient for the application purpose.The use of ensemble rainfall nowcasts may be able to fill the gap generated by these uncertainties.This type of ensemble rainfall nowcast data is desirable in the case of hydrological modeling to improve stream-flow prediction and to predict the likelihood of flash floods over small to large watersheds (Moreno et al., 2013).Consequently, ensemble nowcasting of rainfall data may present a novel approach to the generation of input data for hydrological modeling and flood forecasting, which are essential elements of the work of the hydrological community.However, obtaining a precise single deterministic value from the ensemble members remains a challenge, and will require a highly sophisticated approach to the processing of the ensemble data.

Figure 1 .
Figure 1.Topographic map of East Japan showing the selected domain for the JMA radar rainfall data (black rectangle).Gray rectangle indicates the verification test zone and stars mark the selected rain gauge stations (Red star represents Otaki station)

Figure 2 .
Figure 2. Comparison of average verification scores of the five rainfall events.Solid lines represent verification scores between ERN_JMA (STEPS model) and R_JMA with error bars (one standard deviation), and dashed lines are for FR_ JMA (JMA precipitation nowcast) and R_JMA

Figure 3 .
Figure 3.Comparison of average verification scores of the five rainfall events for different rainfall thresholds.Solid lines represent verification scores between ERN_JMA (STEPS model) and R_JMA with error bars (one standard deviation), and dashed lines are for FR_JMA (JMA precipitation nowcast) and R_JMA

FigureFigure 6 .
Figure Files Click here to download Figure Files: FIG_7.eps

Table I .
Dates, durations, and summary of selected rainfall events.Time periods indicate the start and end of rainfall