International Journal of Applied Earth Observation and Geoinformation

Wildfires frequently occur around the world, which seriously threaten the ecology, environment, economic development, even human safety. In this work, we propose a novel framework for near-real-time and early-stage wildfire detection using Himawari-8 satellite 10-min data. Different from most of the existing methods, the proposed framework jointly combines spatial, temporal and spectral information for wildfire detection. The integrated time-series spatial variance, temporal difference and spectral difference can comprehensively judge the wildfire points and exclude disturbed points. Dispense with cloud detection and setting too many manual thresholds, a spatial–temporal–spectral recurrent neural network (STS-RNN) is developed to adaptively learn the time-series spatial–temporal–spectral curves. Compared with JAXA’s wildfire products, the proposed framework can more accurately detect the small, early-stage, day-time, night-time and forest wildfire points in three experimental scenarios. Especially for the early-stage wildfire detection, the proposed framework may provide the rapid alarm for the local fire department and emergency management agency. This greatly breaks through the limitations of existing wildfire detection methods.


Introduction
Due to the natural or artificial causes, wildfires often occur especially in forest and grassland regions (Maffei et al., 2021).Most wildfires lead to serious economic damage, environmental pollution, public health threats, even personal casualty (Chatzopoulos-Vouzoglanis et al., 2023).If these wildfires grow stronger as time goes on, they will bring about more unmanageable situations for fire extinction (Hu et al., 2021).Therefore, how to reduce the hazard of wildfires through the timely disposing is extremely urgent and significant.However, earlystage wildfire is usually hard for detection, especially in depopulated, mountainous and rugged regions.With the development of remote sensing technology, geostationary satellites could provide a potential solution for near-real-time wildfire detection (Chen et al., 2022;Yu et al., 2022b).The crucial advantages of geostationary satellites are listed below.Firstly, geostationary satellite could effectively observe large-area, which captures the 2-D spatial information of surface land (Llorens et al., 2021;Yu et al., 2022a).Secondly, geostationary satellites are equipped with the multi-spectral sensors, which provide various spectral information including visible, near-infrared, middleinfrared and far-infrared spectrums.Thirdly, geostationary satellites devote themselves to high-frequency observation, which could generate sequential and near-real-time temporal information (Xiao et al., 2022;Zhang et al., 2023b).
Taking the Himawari-8 geostationary satellite as an example, it carries the Advanced Himawari Imagers (AHI) for earth observation.Compared with polar-orbit satellites (temporal resolution: 1∼16 day), Himawari-8 geostationary satellite could acquire all-time and highfrequency data with 10-min temporal resolution.Besides, AHI sensor can offer 16 bands (from 0.455 μm to 13.3 μm wavelength) for applications.Himawari-8 satellite has been widely worked on atmosphere, ocean and land observation (Yu et al., 2021).
In recent years, plenty of literatures have utilized satellite data for wildfire detection.According to the information type, these wildfire detection algorithms could be classified into three categories: spatialbased, temporal-based and spectral-based methods.Detailed descriptions and analyses are listed as follows: (a) Spatial-based methods: Spatial-based methods (also named contextual methods) are the most common strategy for satellite wildfire detection.These methods utilize the brightness temperature difference between fire pixels and corresponding background pixels to detect fires (Liew, 2019;Chen et al., 2017).The fire thresholds are usually https://doi.org/10.1016/j.jag.2023.103506Received 28 March 2023; Received in revised form 20 September 2023; Accepted 22 September 2023 calculated by the spatial statistics (such as mean value, standard deviation, square deviation and so on) in the background window (Wang et al., 2022;Zhang et al., 2022).
Generally, the spatial-based methods are simple, quick and easy to operate for wildfire detection.Nevertheless, due to the spatial heterogeneity, geographical location, weather and other factors, these spatial-based methods are unstable and unreliable especially for small fire and early-stage fire detection.
(b) Temporal-based methods: The temporal-based methods utilize the brightness temperature of adjacent time to detect wildfire (Yan et al., 2020).When the temporal difference between the observed brightness temperature and predicted brightness temperature exceeds the threshold, which indicates that there is a fire point (Hally et al., 2019).
The temporal-based methods take advantage of the high-frequency data offered by geostationary orbital sensors (Viana-Soto et al., 2022;De Marzo et al., 2021).Hence, they can detect the near-real-time wildfire.Moreover, these methods are not affected by spatial heterogeneity.However, when the observed pixels are contaminated by cloud or smoke, the time-series data is usually difficult to exploit for wildfire detection (Xie et al., 2018;Zhang et al., 2021bZhang et al., , 2020)).
(c) Spectral-based methods: The spectral-based methods take advantage of the spectral differences between different bands to detect the wildfire (Zhang et al., 2019).The spectral difference between MIR band and TIR band of the fire pixels is significantly greater than that of non-fire pixels.
Overall, the spectral-based methods could fully exploit the spectral characteristics of wildfire points, which distinguish the fire points from the other ground objects (Xu and Zhong, 2017;Rahmi et al., 2020).However, these methods require more prior knowledge and are difficult to set adaptive thresholds (Zhang et al., 2021a(Zhang et al., , 2023a)).In addition, the spectral-based methods are easily disturbed by clouds, which severely affects the accuracy of wildfire detection (Zhang et al., 2018).In summary, above three methods can effectively capture the characteristics of early wildfire from different angles.Nevertheless, these methods still exist inherent shortages and limitations.Therefore, could we jointly utilize spatial, temporal and spectral information for Himawari-8 near-real-time wildfire detection?In addition, could we implement earlystage wildfire detection, which is extremely significant for the local fire department and emergency management agency?What is more, could we develop a self-adaptive algorithm, instead of setting too many manual parameters for wildfire detection?
From these perspectives, we propose a novel machine learning framework for Himawari-8 near-real-time and early-stage wildfire detection.The main contributions are listed as follows.
• The proposed framework jointly combines spatial, temporal and spectral information for Himawari-8 10-min near-real-time wildfire detection.The integrated time-series spatial variance, temporal difference and spectral difference can comprehensively judge the wildfire points.The reminder content of this work is organized below.Section gives the related data description of Himawari-8 AHI.Section 3 depicts the proposed wildfire detection methodology.Section 4 carries out three different wildfire detection experiments.Finally, Section summarizes a brief conclusion of this work.

Himawari-8 satellite AHI data
In this work, we utilize Himawari-8 satellite AHI data as the processing object, its spatial resolution is 2 km.Himawari-8 satellite was launched in 2014, which was designed by Japan Aerospace Exploration Agency (JAXA).Himawari-8 satellite is the third generation of geostationary satellite for earth observation (Bessho et al., 2016).It mainly covers the areas of east Asia and Australia (60 as shown in Fig. 1.The AHI sensor is onboarded at Himawari-8, which can receive 16 bands (six albedo bands and ten brightness temperature bands).

Wildfire validation
For validating the effectiveness of wildfire results, we compare the proposed method with JAXA's level-2 10-min wildfire products.The spatial resolution of JAXA's level-2 10-min products is 2 km.Hot-spots are detected via a contextual threshold algorithm, based on MIR and TIR bands in Himawari-8 AHI data.It provides the fire radiative energy, reliability and wildfire position in level-2 products.
In addition, we acquire the accurate occurred-time and position from official's wildfire investigation reports.Wildfire detection could be validated through above precise information.

Overview
For wildfire pixel, it has three typical characteristics.As shown in Fig. 2. Time-series spatial variance, temporal difference and spectral difference are generated via the time-series BT 07 and BT 14 information ( 1 ∼  ).Then, if the previous moment  −1 (, ) is not wildfire pixel, the proposed framework carries out the early-stage wildfire detection.Otherwise, the proposed framework executes the continuous wildfire detection.Later, STS-RNN is put forward to exploit the jointly spatial, temporal, and spectral information.STS-RNN model determines the wildfire pixels and generates the near-real-time wildfire products.Subsequently, the proposed framework performs the next moment ( =  + 1) and updates related values for Himawari-8 wildfire detection.

Spatial-temporal-spectral information
Firstly, the spatial variance making the spatial characteristics of wildfires more prominent.Secondly, using spectral difference information can sensitively capture wildfires and reduce interference from factors such as clouds and fog.Finally, temporal difference can enhance the sensitivity of the model to wildfires.To sum up, by integrating spatial, temporal, and spectral information, our method can effectively reduce the impact of spatial heterogeneity, geographical location and other factors, and eliminate the interference of clouds and smoke.At the same time, the proposed method avoids setting too many thresholds.It can effectively improve the accuracy of wildfire detection and the robustness of the algorithm.

Spatial information
For BT 07 band, the spatial value of wildfire pixel is obviously higher than its neighborhood regions.Based on this principle, we calculate the spatial variance ( ) to reflect the spatial information of wildfire pixel: where  stands for the   moment in Himawari-8 AHI data.Where (, ) is the pixel position. represents the distance size.Excluding the central pixel can amplify the deviation and improve the wildfire detection accuracy for the proposed method.Through stacking each spatial variance as a time-series curve, we acquire the time-series spatial variances, which could be fully exploited by the subsequent STS-RNN model.

Temporal information
Relied on the high temporal resolution, Himawari-8 could distinguish tiny and imperceptible changes.In terms of wildfire detection, we logically introduce the temporal information into the proposed framework.For the time-series BT 07 band ( 1 ∼  ), the temporal difference is derived below: If a wildfire occurs at   , its temporal difference obviously increases to a large extent.In other words, the temporal difference reflects the gradient of time-series BT 07 .Therefore, we could employ this characteristic for Himawari-8 wildfire detection.Then the time-series temporal difference curve is imported into the posterior STS-RNN model.

Spectral information
Himawari-8 satellite simultaneously captures the MIR band (BT 07 , 3.9 μm) and TIR band (BT 14 , 11.2 μm).MIR band is highly sensitive to the hot fire, while TIR band is stable to the variation of land surface temperature.Relied on this intrinsic characteristic, Himawari-8 satellite could effectively perceive wildfires.Thus, the spectral information is leaded into the proposed framework: where  denotes the spectral difference between BT 07 and BT 14 .
Through stacking each spectral difference as a time-series vector, we acquire the time-series spectral difference curve for wildfire detection.This time-series curve reveals salient abnormity, which could be adequately utilized by the subsequent STS-RNN model.

Early-stage/continuous wildfire detection
Generally, a wildfire pixel could be divided into two types: earlystage and continuous wildfire detection.For the early-stage wildfire, the time-series spatial variance, temporal-difference and time-series spectral difference curves usually perform smooth before the wildfire occurs.In contrast, these time-series curves will suddenly rise for the early-stage wildfire.Then the proposed framework directly inputs the spatial-temporal-spectral information for the STS-RNN predict model, as described in Fig. 2.
Nevertheless, for the continuous wildfire, the temporal difference may be negative after the wildfire occurs a period of time, due to the continuous high-temperature in BT 07 .Hence, we bring in a filtering solution to ensure the robustness of continuous wildfire detection.For current moment   (, ), if the previous moment  −1 (, ) is a wildfire pixel, the spatial variance, temporal difference, and spectral difference at  −1 are all replaced by the predicted values of STS-RNN model.Through this strategy, we can effectively exclude pre-existing wildfire points for continuous wildfire detection.

STS-RNN predict model
After the judgment and preprocessing of early-stage or continuous wildfire, the time-series spatial variance, temporal difference and spectral difference curves are imported into the STS-RNN predict model, respectively.RNN can effectively exploit the consistency and variability of time-series vector.In the proposed wildfire detection framework, the structure of STS-RNN model is given in Fig. 3(a).For the timeseries vector [ 1 ,  2 …  −2 ,  −1 ,   ] in STS-RNN model,   is the value of current target point at the moment .It can be represented as the spatial variance, temporal difference or spectral difference, respectively.
As depicted in Fig. 3(a), the overall structure of the STS-RNN model is composed of three parts: input layer, hidden layer, and output layer.In the input layer, STS-RNN model utilizes the time-series vector as input elements.In the hidden layer, STS-RNN model estimates current hidden node via both current input value and the last hidden node.In the output layer, STS-RNN model exports the predicting output via the corresponding hidden node.At time step , the hidden node is determined as: where  stands for the activation function.Then the predicting output   is denoted as: where  ,  and  represent the weight parameters for input layer, output layer and hidden layer, respectively.Apparently, current hidden node is simultaneously influenced by the corresponding input value and previous hidden node.So, STS-RNN model could effectively fit the time-series curve.It adaptively exploits the sequential relationship via self-spatial, temporal and spectral information.Notably, we employ the existing time-series curve (moment 1 to  − 1) to train the STS-RNN model.For current moment , STS-RNN model estimates the predicted value in Fig. 3(b).The deviations between actual values and predicted values of spatial variance, temporal difference and spectral difference are calculated as follows: where     ,      and     are the predicted values of STS-RNN model.Then the proposed framework determines the wildfire point through the integrated spatial-temporal-spectral view: where ( −1  =1 (   )) denotes the maximum deviation for previous moment. and  stand for the scale factor and bias threshold.If these three deviations are together larger than the setting thresholds, it usually satisfies the characterize of wildfire point.For the early-stage wildfire detection, the judging criterion is defined below: For the continuous wildfire detection, the judging criterion is decided below: where  refers to the Boolean flag (''True'' is fire and ''False'' is non-fire) for wildfire detection.After finishing current moment , the proposed framework carries out the next moment ( =  + 1).Table 1 The confusion matrix of actual/predicted fire or non-fire.
Predicted fire Predicted non-fire Actual Fire   Actual Non-Fire

Training data
In our experiment, long-term 10-minute time-series data are used for training.STS-RNN can use these time-series curve data for model self-supervised optimization through time series recursion.It can make full use of the high correlation in long-term curve, to continuously update the training data.The training data used by the proposed model includes three kinds: Spatial, temporal and spectral information.Detailed description is provided in chapter 3.2.

Training method
STS-RNN model could effectively fit the time-series curve.It adaptively exploits the sequential relationship via self-spatial, temporal and spectral information.Meanwhile, STS-RNN uses the loss function to optimize the model.The loss function for self-supervised optimization is given in Eq. ( 14) (Zhang et al., 2021c): where   and   represent the predicting and true value at time step , respectively.In our validation experiment, the epoch number is set as 500 for model optimization.The learning rate is denoted as 0.01 for back propagation and gradient descent.

Parameters setting and evaluation indexes
After multiple parameter adjustments, we found that =7 can well reflect the statistical characteristics of spatial information, so  is set as 7.The scale factor  and bias threshold  in Eqs. ( 9) to (11) are fixed as 2 and 1, respectively.These parameters can more sensitively capture the occurrence of early wildfire.The wildfire detection results show that these thresholds can adapt well to different regions.In the future, we will study the early wildfire detection method with adaptive parameters for different scenarios.In terms of network training, the epoch number is set as 500 for STS-RNN model optimization.The learning rate is denoted as 0.01 for back propagation and gradient descent (see Table 1).
Firstly, we define the true fire point.Definition of true fire point is manually determined by judging the brightness temperature of the 7th band, and combining with the visible image.
With respect to the quantitative evaluation for wildfire detection, we utilize five evaluation indexes below: fire accuracy (FA), overall accuracy (OA), early-fire accuracy (EFA), false-alarm rate (FAR) and omission-fire rate (OFR).The confusion matrix of actual and predicted fire or non-fire is listed in Table 1.Relied on this confusion matrix, FA, OA, FAR and OFR indexes are defined as follows: In addition, we specially use the EFA index to testify the detection ability of early-stage wildfire.This may provide the rapid alarm for the local fire department and emergency management agency.The EFA index is defined below: where   represents the number of all the actual early-stage wildfire points.  refers to the number of all the predicted early-stage wildfire points.For FA, OA and EFA indexes, the higher these values are, the better the wildfire detection effects behave.Oppositely, for FAR and OFR indexes, the lower these values are, the better the wildfire detection effects behave.The experimental results are demonstrated below.

(a) Jiamusi forest wildfire (2018.04.25)
In this experimental scenario, we perform on the small wildfire detection.The position and time are available in the satellite-earth synchronous observation experiment by the national satellite meteorological center.As shown in Fig. 4, an artificial straw burning site is carried out in Huachuan county, Jiamusi city, Heilongjiang province, China (46.7 • N, 130.4 • E).The burning time starts from 05:30 (UTC Time) to 07:00, in April 25, 2018.
As portrayed in Fig. 5(a), the time-series BT 07 and BT 14 curves at position (46.72 • N, 130.42 • E) of Himawari-8 AHI 10-min dataset are given from UTC time 00:00 to 09:00 in April 25, 2018.The red line and green line refer to the time-series BT 07 curve and BT 14 curve, respectively.The spectral difference between the two curves is covered with pink color.Obviously, starting from 05:40 to 06:20, the BT 07 values behave abnormal due to the hot sensitivity.
In addition, the time-series spatial variance, temporal difference and spectral difference curves are revealed in Fig. 5(b), respectively.Similarly, these curves suddenly shake to different degree when the wildfire occurs at UTC time 05:40.The evaluation index comparison  between the JAXA WLF L2 products and proposed method is listed in Table 2.The optimal indexes are marked with bold format.
As contrasted in Table 3, JAXA WLF L2 products perform poor on small wildfire detection, whose FA index is just 16.35%.These products omit 83.65% small wildfire points in this wildfire scenario.In comparison, the proposed method generates reliable wildfire detection results.These results validate the availability and robustness of the proposed wildfire detection framework.The 10-min time-series  07 , JAXA WLF L2 products and wildfire detection results of the proposed method are displayed in Fig. 6.For the early wildfire at 05:40, JAXA WLF L2 products cannot detect the wildfire pixels.These products only monitor single hot point until 06:00 in the middle row of Fig. 6.In comparison, the proposed method can accurately detect the early-stage fire in the bottom row of As displayed in Fig. 8(a), the time-series  07 curve and  14 curve at position (27.84 • N, 102.18 • E) are given from UTC time 00:00 to 10:00 in March 30, 2020.At day-time, the brightness temperature of  07 and  14 firstly increase, and then decrease smoothly due to the solar radiation and altitude variation in Fig. 8(a).Nevertheless, starting from UTC time 07:50, the spectral difference between  07 and  14 band is increasingly expansile in Fig. 8(a), which indicates the existence of a wildfire pixel.The time-series spatial variance, temporal difference and spectral difference curves are revealed in Fig. 8(b), respectively.The evaluation index is listed in Table 3.
As comparison in Table 3, JAXA WLF L2 products perform poor on early-stage wildfire detection, whose FA and EFA indexes are only 6.53% and 0.00%.In comparison, the proposed method generates reliable wildfire detection results.These results testify the practicability and capacity of the proposed framework.
For Xichang forest wildfire, the 10-min time-series  07 , JAXA WLF L2 products and wildfire detection results of the proposed method are revealed in Fig. 9.For the early wildfire from 07:50 to 08:10, JAXA WLF L2 products cannot detect any wildfire pixels.As shown in the middle row of Fig. 9.In comparison, the proposed method can effectively detect the early-stage fire in the bottom row of Fig. 9.These results adequately manifest the superiority of the proposed framework for distinguishing the small and early-stage wildfire.As shown in Fig. 11(a), the time-series  07 and  14 curves at position (37.04 • N, 129.28 • E) are given from UTC time 22:00 to 07:00 in March 03 and 04, 2022.The spectral difference between  07 and  14 band is quickly expansile in Fig. 11(a), which indicates the existence of a wildfire pixel.
Furthermore, the time-series spatial variance, temporal difference and spectral difference curves are displayed in Fig. 11(b), respectively.The evaluation index is listed in Table 4.
As comparison in Table 4, both the JAXA WLF L2 products and proposed method generates satisfactory wildfire detection results for Korea forest wildfire.However, JAXA WLF L2 products perform poor on small fire and early-stage fire, which only achieve 41.86% FA index, 52.77% EFA index and 58.14% OFR index.The 10-min time-series  07 , JAXA WLF L2 products and Korea forest wildfire detection results of the proposed method are displayed in Fig. 12.As shown in the two last columns of Fig. 12, the proposed method can more accurately distinguish the fire pixels than JAXA WLF L2 products.

Validation with medium-resolution satellite
To verify the accuracy of the wildfire detection results, we compare them with medium resolution satellite images (Tyukavina et al., 2022).
As shown in Fig. 13, the image of Sentinel-2 shows Korea forest wildfire scence, at UTC time 02: 07 in March 05, 2022.It was captured within a day after the fire occurred.The coordinate of the red marked point is 37.04 • N, 129.27 • E, which is also the wildfire detection point via the proposed method.From this image, it can be validated that there is indeed an obvious wildfire at the location where we detected the wildfire.This also proves the reliability of STS-RNN via medium-resolution satellite.

Discussion of experimental results
In the results section, we conducted research on two fire scenarios.For small fires, due to the use of contextual algorithms in the JAXA WLF L2 products, the sensitivity to small fires is poor, resulting in a high rate of missed detections.The proposed method effectively improves the detection accuracy of small fires by integrating spatial-temporal-spectral information.For large-scale forest fires, the JAXA WLF L2 products have poor detection performance for early wildfires.On the contrary, the proposed method obtains reliable wildfire detection results.In summary, compared to the JAXA WLF L2 products, our method can adapt to different fire scenarios and achieve higher detection accuracy.

Analysis of causes
Firstly, wildfire thresholds are usually calculated through spatial statistical information in the background window (Wang et al., 2022).
Therefore, small fires in some scenarios may be missed due to the inability to reach above thresholds.This is also the main reason for the high omission rate of JAXA WLF L2 products in small fire scenarios.Secondly, the temporal-based methods take advantage of the highfrequency data offered by geostationary orbital sensors (De Marzo et al., 2021).It allows us to detect wildfires in near real-time.This is the main reason why our method can detect fires in the early stages.Finally, the spectral-based methods could fully exploit the spectral characteristics of wildfire points, which distinguish the fire points from the other ground objects (Rahmi et al., 2020).Using this information can improve the fire detection accuracy.
From the experimental results, by integrating spatial-temporalspectral information, our method not only has superior detection ability in small fires, but also significantly improves overall detection accuracy, especially in early wildfire detection.

Discussion for cloudy scenarios
To illustrate how these curves change when a pixel changes from being covered by cloud to being clear.We take the center cloud pixel as (27.56 • N, 103.91 • E) and the time period from 03:50 to 10:00 UTC on March 30, 2020.As shown in Fig. 14, the pixel experienced three situations: from cloud to clear, maintaining cloud coverage, and from clear to cloud.At the moment of cloud coverage or cloud change, there are significant change for temporal difference or spectral difference in Fig. 14.In our experiment, we jointly utilize spatial, temporal and spectral information for Himawari-8 near-real-time wildfire detection, only if the temporal-spatial-spectral information meets all the comprehensive judgments, the object pixel could be judged as an early wildfire point.
Through the spatial variance, temporal difference and spectral difference, STS-RNN reduces the interference caused by cloud coverage or cloud change.Relied on these advantages, STS-RNN could achieve fast, robust and effective early-stage wildfire detection results, without additional cloud detection for Himawari-8 AHI data.

Conclusion
In this paper, we propose a novel machine learning framework for Himawari-8 near-real-time and early-stage wildfire detection.The proposed framework utilizes spatial, temporal and spectral information for wildfire detection.An STS-RNN model is developed to exploit the time-series spatial variance, temporal difference and spectral difference curves and determine the wildfire pixels.Dispense with cloud detection and setting too many manual thresholds, the proposed framework performs well for different wildfire types and scenarios.Compared with JAXA's wildfire products, the proposed framework behaves more excellent on early-stage and small wildfire detection, at the cost of small false positives.This can send out the early warning of the potential wildfire, which may provide the rapid alarm for the local fire department and emergency management agency.
In our future work, we will further improve the spatial, temporal and spectral features for early-stage wildfire detection.Moreover, the wildfire detection platform will be established for practical applications.

Fig. 5 .
Fig. 5. Curves at position (46.72 • N, 130.42 • E), from UTC time 0:00 to 9:00 in April 25, 2018.(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Q.Zhang et al.

Q
.Zhang et al.

Fig. 13 .Fig. 14 .
Fig. 13.Korea forest wildfire satellite image of Sentinel-2 satellite.(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Table 2
Evaluation index comparison between the JAXA WLF L2 products and proposed method.
Q.Zhang et al.

Table 3
Evaluation index comparison between the JAXA WLF L2 products and proposed method.

Table 4
Evaluation index comparison between the JAXA WLF L2 products and proposed method.