Evaluating accuracy of four MODIS-derived burned area products for tropical peatland and non-peatland fires

Satellite-based burned area products are accurate for many regions. However, only limited assessments exist for Indonesia despite extensive burning and globally important carbon emissions. We evaluated the accuracy of four MODIS-derived (moderate resolution imaging spectroradiometer) burned area products (MCD45A1 collection 5.1, MCD64A1 (collection 5.1 and 6), FireCCI51), and their sensitivity to burned-area size and temporal window length used for detection. The products were compared to reference burned areas from SPOT 5 imagery using error matrices and linear regressions. The MCD45A1 product detected <1% of burned areas. The other products detected 38%–48% of burned area with accuracies increasing modestly (45%–57%) when smaller burns (<100 ha) were excluded, with MCD64A1 C6 performing best. Except for the MCD45 product, linear regressions showed generally good agreement in peatlands (R 2 ranging from 0.6 to 0.8) but detections were less accurate in non-peatlands (R 2 ranging from 0.2 to 0.5). Despite having higher spatial resolution, the FireCCI51 product (250 m) showed lower accuracy (OE = 0.55–0.88, CE = 0.33–0.50) than the 500 m MCD64A1 C6 product (OE = 0.43–0.79, CE = 0.36–0.51) but it was comparable to the C5.1 product (OE = 0.52–0.91, CE = 0.37–0.67). Dense clouds and smoke limited the accuracies of all burned area products, even when the temporal window for detection was lengthened. This study shows that emissions calculations based on burned area in peatlands remain highly uncertain. Given the globally significant amount of emissions from burning peatlands, specific attention is required to improve burned area mapping in these regions in order for global emissions models to accurately reflect when, where, and how much emissions are occurring.


Introduction
In September 2019, vast amounts of smoke-related haze from regional peatland fires blanketed Sumatra and Kalimantan in Indonesia with the worst air conditions since 2015. Poor air quality in Sumatra (Riau) forced schools to close for weeks. Air quality index (AQI) and fine particles (PM 2.5 ) values in Kalimantan (Palangkaraya) exceeded 2000 and 1400 µg m −3 , respectively, greatly exceeding bounds of the index's worst anticipated conditions (Hazardous AQI >300-500, PM 2.5 >65 µg m −3 ). This latest catastrophic event, emanating from the region's peatlands, pales in comparison to 2015 when the annual peat fires were exacerbated by El Niño drought and burned 2.6 million hectares, releasing greenhouse gases (GHGs) estimated at the CO 2 -equivalent of 1.75 Gt (GFED 2015), greater than the annual emissions of Japan (Field et al 2016). Tragically, the associated toxic regional haze is also estimated to have caused >100 000 premature human deaths (Johnston et al 2012, Koplitz et al 2016. Regional projections anticipate an annual average of 36 000 excess deaths if land management practices are not improved (Marlier et al 2019). The regional health and economic impacts (Glauber et al 2016) and globally significant GHG emissions make detection, monitoring, and ultimately mitigation of Indonesia's peatland fires crucial.
Heavy smoke from smoldering peat soils dominates emissions from burning peatlands (Turetsky et al 2015, Page andHooijer 2016). Smoldering peat fires produce many more gases and aerosol particulates than flaming surface components (Stockwell et al 2016). Consequently, peatland fires are treated differently in emissions models (e.g. the Global Fire Emissions Database (GFED) (van der Werf et al 2017), Global Fire Assimilation System (GFAS) (Kaiser et al 2012) by incorporating higher emission factors (e.g. C, CO, CH 4 ) or organic matter burned than other land cover types (Andela et al 2013). Uncertainties remain high for these variables, resulting in different estimates of fire-related emissions from various inventories for the Indonesian burning in 2015 (Whitburn et al 2016, Heymann et al 2017, Wooster et al 2018. Burned area (BA) is the primary input for estimating associated emissions but providing it accurately is also the main challenge (van der Werf et al 2010, Kaiser et al 2012. In ecosystems, such as savannas, temperate and boreal forests, burned area may not be considered the main error source (French et al 2002, Sparks et al 2015, with phase of combustion and moisture content (Chen et al 2007), or seasonality (Korontzi et al 2003) being seen as more problematic. Unlike these ecosystems, BA is of substantial uncertainty in tropical regions. BA monitoring over large areas is primarily reliant on satellite-based mapping. Given the central importance of BA for assessing GHG emissions, it is critically important to validate and evaluate the accuracy of BA products in Indonesia's peatlands.
Over the last two decades, moderate resolution imaging spectroradiometer (MODIS) imagery has been used to map global and regional fire activities , Alonso-Canas and Chuvieco 2015, Ramo and Chuvieco 2017, Vetrita and Cochrane 2020 and carbon emissions (van der Werf et al 2010, Kaiser et al 2012, providing an essential variable for climate models (Hollmann et al 2013). MODIS BA products, at 500 m resolution, have been modified over time to improve their accuracy with each newly released MODIS Collection of reprocessed imagery. The first MODIS BA product, MCD45A1, used a 'rapid changes bidirectional reflectance model' and time series of surface reflectance data to flag areas of rapid change as potential BAs (Fornacca et al 2017). It has been shown to have adequate accuracy for monitoring fires in some regions or biomes (Chang and Song 2009, Roy and Boschetti 2009, Ruiz et al 2014, Tsela et al 2014, Fornacca et al 2017, with global omission and commission rates of 46% and 72%, respectively (Padilla et al 2014). The MCD64A1 BA product integrates 1 km MODIS active fire (AF) (MOD14/MYD14) detections with the 500 m reflectance data to reduce false detections. Resulting detections are subjected to additional algorithmic validity tests and masking. Although available from both Collection 5.1 and 6, the most recent BA product (MCD64A1 C6) has algorithm changes designed to improve detection of small fires globally (Giglio et al 2018). Product comparisons conducted in Brazil have found MCD64A1 C6 more reliable than the previous version, with lower omission errors (OEs) and more fires detected (Rodrigues et al 2019). Another MODIS-derived BA product, FireCCI51, has a higher spatial resolution (250 m vs 500 m) and was expected to have superior small fire detection capabilities than MCD64A1 C6 products (Chuvieco et al 2018). However, global comparison of the FireCCI51 and MCD64A1 C6 products showed that detections varied spatially and temporally among regions (Humber et al 2019).
Those assessments were largely conducted outside the Indonesian peatlands, where the fire characteristics are not necessarily equivalent to those on other peatlands. Limited studies exist that evaluate Indonesia's peatland areas , Tansey et al 2008, Miettinen and Liew 2009, Albar et al 2018, where fires are predominantly smoldering ground fires that are often small in area and frequently covered by clouds. Previous assessments have reported difficulty assessing accuracy for Indonesia due to the limited reference data, as compared to other regions (Shi et al 2014). Providing reference data and determining standard accuracy methods are needed to get unbiased results (Boschetti et al 2016). Lohberger et al (2017) assessed one of the MODIS products (MCD64A1 C5.1) against a Sentinel-1 derived BA map. Sentinel-1 satellites employ active remote sensing, using a C-band synthetic-aperture radar that can penetrate the clouds and haze in Indonesia at much higher spatial resolution (10 m) than MODIS. Without any specific validation of the MODIS product, Sentinel-1 detected nearly twice as much area burned as either the MODIS product or official BA maps from the Indonesian government.
We initiated this research to assess which product(s) would be best for conducting a national fire frequency analysis. We concentrate here on the relative BA mapping accuracies in the critical peatland areas and compare these to mapping accuracies in regional non-peatland areas. The frequency of fires is a critical parameter for calculating fire emissions from peat fires (Konecny et al 2016). This parameter has been lacking in Indonesia, resulting in many satellite-based carbon emission models excluding this important component (e.g. GFED (van der Werf et al 2017), GFAS (Kaiser et al 2012), the Fire Inventory from NCAR (Wiedinmyer et al 2011), the Global Biomass Burning Emission Product-Geostationarysatellite (Zhang et al 2012)).
In this study, we conducted a comparative analysis of the detection accuracies of four global BA products (MCD64A1 C5.1 and C6, MCD45A1 C5.1, and FireCCI51) acquired during 2014 in peatlands of Indonesia's Central Kalimantan province. We compared the BA data from each product against a reference dataset derived from higher spatial resolution SPOT 5 imagery to determine errors of omission and commission in deep peatlands, shallow peatlands, and nearby non-peatlands of Indonesia. We also investigated how the exclusion of small area fire patches and lengthening the temporal window of detection affected accuracy. Finally, we addressed discrepancies among products, improvements over its predecessors (collection 5.1 vs 6), impact on firerelated emissions models, needs, and future studies.

Study site
The study area covered 1.6 million hectares (Mha), 10.4% of the total area of Central Kalimantan, Indonesia, that were delineated by available SPOT 5 imagery (figure 1). Peatlands underlie 67% of this site, with 56% deep peat, 11% shallow peat, and 33% nonpeatland. We derived these classes by aggregating the two peat maps currently available from the Indonesian Ministry of Agriculture (Ritung et al 2011), and Wetland International peatland atlases (Wahyunto and Subagjo 2004). The Ministry of Agriculture peat maps do not include peaty soils (shallow peat areas <50 cm depth) since these areas are considered agricultural lands while Wetlands International maps peat regardless of depth. Therefore, if both maps agreed as being either peat or non-peat, we classified them as deep peat and non-peat, respectively, otherwise, as shallow peat. Here, the 'deep' term does not imply a specific depth of the peat. The Indonesian government peatland atlas is periodically updated. Although an updated 2019 version now exists, it was not yet publicly available, so our analyses are based on the available maps.
The 2014 MODIS annual land cover product (MCD12Q1) classified 60% of the study area as forests, 35% shrubs (including woody savannas and grassland), 2% croplands and the remainder as nonvegetated areas. The land cover classes were based on the International Geosphere-Biosphere Programme (IGBP) classification, available through the Earthexplorer (https://earthexplorer.usgs.gov/, last accessed on 20 June 2020). Based on the Ministry of Forestry and Environment land cover map of 2014, the majority land cover type was defined as bush/shrubs/regrowth, swamp, and secondary peat swamp forest (land use/cover map Ministry of Forestry and Environment, 2014) The study region has been reported by Indonesian authorities as the greatest contributor to area burned in the country (MoEF 2020), particularly during the El Niño events of 2015 and 2019. El Niño drought events are associated with severe burning due to increased fire susceptibility (Siegert et al 2001), extensive burning and recurring fire events.
Although the 2014 burning event was less severe than the burning during El Niño years, the region has experienced frequent annual burning since 1997. Fire seasons usually occur from August to October every year.
Fires were reported to be associated with the 'ex-Mega Rice Project (MRP)' , 1 million hectares of drained peat-swamp forest, converted into rice plantations in 1997, but later abandoned (Page et al 2002, Ballhorn et al 2009, Konecny et al 2016, Stockwell et al 2016. A grid-pattern of canals, thousands of kilometers in length were constructed across the MRP, reducing water table levels in the peatlands, draining and drying near-surface peat, and providing open access to the remaining forests of the area, resulting in widespread human-induced fires. The site becomes prone to fire due to careless land use practices (e.g. logging and plantation establishment).

MODIS burned area products: brief algorithms, data sources, and processing
The MCD45A1 maps sudden changes of the earth's surface due to burning using bidirectional reflectance (BRDF) models from 500 m MODIS cloud-free surface reflectance data. The bidirectional effect shows changes that are not associated with the Earth' surface change (Roy et al 2002), or variation in observed reflectance attributed to directional effects instead of surface change itself. The algorithm used a 16 d (with maximum 8 extra-day) time window before and after burning, with at least 7 d of available imagery, to predict the reflectance. The MODIS 500 m infrared bands (858, 1240, 1640, and 2130 nm) were used to discriminate the changes due to fire from other types of change (see Roy et al 2005 for detailed information). The MCD45A1 datasets provided two layers on a monthly basis, i.e. burn date and the pixel confidence level. We clipped the imagery to match our study site extent and selected only the approximate date of burning with the most confidently detected pixels flagged in the quality assurance (QA) layer. The raster files were converted into a vector file by conserving the 500 m pixel size to calculate the intersected areas for our analysis.
The MCD64A1 (collections 5 and 6) algorithms integrate the 1 km MODIS AF product (MOD14A1 and/or MYD14A1), MODIS reflectance data, and land cover product to detect area burned (Giglio et al 2009(Giglio et al , 2018. The main differences between collection 5 (C5) and 6 (C6) products are summarized in (Giglio et al 2018) and include, changes to the input data, handling of cloud interference, temporal window change from 10 to 8 d, and changes in how training sample data is applied, among others. The HDF files were reprojected from sinusoidal to geographic coordinates to calculate the areas burned. The products provided the approximate burn date, burn date uncertainty, QA, first day and last day layers. We selected all ordinal pixel days of burn (1-366), flagged Figure 1. Study region in Central Kalimantan, Indonesia. Study areas were delineated by available SPOT 5 imagery footprints (black rectangles) covered 1.6 million hectares. Detected BAs from SPOT 5, the MODIS BA product collection 6 (MCD64 C6) accurately detected or missed in peatland (deep and shallow) and the non-peat cover is shown. Diamond, circle, and square symbols are associated with field assessments in 2014. Photo of burning in tall shrubs and cropland of the non-peatland area (diamond) was taken at nighttime (around 7 PM local time). Young oil palm that burned on non-peatland areas (circle). Burning in degraded areas on deep peat where BOSF measured depth of burn into the peat. Photo credits: LAPAN (diamond and circle) and BOSF (rectangle).
in the QA layer as being in land grid cells flagged and having valid data.
FireCCI51 is the first updated version of Fir-eCCI BA products that are based on MODIS data from the Terra satellite platform. The product was developed under a Climate Change Initiative (CCI) project of the European Space Agency (ESA). It has the longest time series, most improved algorithm, and the best validation results (Pettinari et al 2020). This product is an improvement of the previous collection (FireCCI41, available from 2005 to 2011) to provide long-term data archives. This product is currently available and updated from 2001 to 2019 (last accessed on 16 July 2020). The main inputs to derive this product are the daily MODIS Surface Reflectance product (MOD09GQ) collection 6 images, MODIS Global Monthly Fire Location Product (MCD14ML collection 6), and the Land Cover Project of ESA CCI. Images were composited before the two phase approaches were used. For each candidate burn pixel, pre and post images were defined based on the nearest AF date with at least four valid post-fire observations within a specified time window. To minimize ambiguity, the standard search window was 10 d after selection of the post-fire date for each candidate pixel. Employing two MODIS bands at 250 m resolution (645 and 858 nm), the product provides higher spatial resolution than the other three BA products assessed here (250 m vs 500 m). Detailed algorithms can be found at (Lizundia-Loiola et al 2020, Pettinari et al 2020).
We downloaded the pixel-version product (250 m spatial resolution, 6.25 ha areas equivalent), freely available since November 2018 at www.esa-firecci.org/FireCCI51 (accessed on March 2020). Mimicking the other BA products, FireCCI51 provides a monthly GeoTIFF dataset with three layers, estimated first day of burn (Julian-date), confidence level, and land cover type of a detected burn pixel. All burn pixels regardless the confidence level were used for our analysis.

Reference map
The Indonesian National Institute of Aeronautics and Space (LAPAN) provided the BA reference map (Zubaidah et al 2017). The reference map was manually classified into burned and unburned classes. The protocol of the Southern African Fire Network (SAFNet) was adopted to create the map. The Committee on Earth Observations (CEOS) Land Product Validation Working Group has also approved the procedure for use by the international community (Boschetti et al 2010). Brief methods and validation efforts were as follows.
Trained LAPAN image interpreters created the BA reference map based on visual interpretation and classification of Orthorectified SPOT-5 imagery. Five relatively cloud-free (~10% coverage) SPOT images were acquired on 3, 24 and 29 September 2014. Several procedures were used to standardize evaluation conditions for each interpreter, such as using a fixed screen-scale (1:10 000), false composite bands (Short wave Infrared-Near Infrared-Red) and overlay of the MODIS AF product (MOD14A1/MYD14A1). Visually, the area burned appeared as dark magenta, often with smoke visible, using visual interpretation. Due to limited availability of SPOT 5 images from before burning, Landsat 7/8 images were also used by the experienced interpreters to help them decide if areas had burned.
Many BAs were located in remote locations restricting available locations for field validations. However, preliminary maps of the BA were initially evaluated with local stakeholders. The research team conducted a week-long intensive (18-23 September 2014) field validation activity to evaluate the BA map. Two validation sites are shown in our manuscript in figure 1. Approximately 40 burn positions were marked with GPS (minimum 5 m accuracy) during the field trip, with all accurately detected in the SPOT 5 burned map. Safety and logistical access reasons precluded measurement of burn perimeters. A second validation analysis was conducted in September 2015 and one area erroneously mapped (dried pond) as being burned was discovered and corrected in the final BA map. Subsequently, map accuracy of more remote regions, accessible via canals, was assessed, with collaboration with the Borneo Orangutan Survival Foundation (BOSF), with data collected during parallel activities of a NASA project (Cochrane PI). BOSF manages a permanent site for hydrology and fire monitoring (1.2 million ha). In figure 1, one photo was attached.
SPOT 5 mapped a total of 81 249 ha of BA by September 2014, across both peatlands and nonpeatlands, with 57%, 14%, and 29% occurring in deep peat, shallow peat, and non-peat, respectively. Most burned patches were small (85% ⩽ 25 ha, or 94% < 100 ha) with only 6% of patches larger in size. However, although numerous, small burn patches only comprised 25% of the total area burned. For the study site, land cover was approximately 25% nonpeat and 75% peatland. Comparable proportions of both land cover types burned, 4% of non peatlands vs 5% of peatlands, respectively, indicating both land cover types are vulnerable to fires. In terms of vegetation, of all the areas burned, 68% were shrubs, woody vegetation, and grassland,~30% was damaged forests and 1% was croplands that was almost entirely on non-peatland areas.

Burned area products accuracy analysis
Rules to select MODIS datasets for comparison to the SPOT 5-derived BA reference map were: (a) all land pixels with a valid-data flag noted as having a 'burn date' within the study region; (b) of those pixels, only those having detection dates from the beginning of fire season to (1) the same date of the corresponding SPOT 5 image (hereafter defined as D0), (2) 8 d after (D8), and (3) 10 d after the SPOT 5 images acquisition (D10). These three different temporal aggregation windows were considered to evaluate sensitivity to changed window lengths of the BA products. Ideally, BAs are detected on the day when the burning occurred (D0). However, some areas may be covered by thick clouds or smoke, preventing detection, at the time of occurrence. MODIS products typically report aggregated findings over a period of days to improve detection likelihood by getting several potential observations of BAs. Standard temporal window lengths vary between products but are all designed to reduce uncertainty (Giglio et al 2018, Pettinari et al 2020), with the MODIS C6 product using 8 d (D8), while FireCCI51 and MODIS C5.1 use 10 d (D10) instead. We evaluated the reliability of BA detection for each product at the three observation periods, D0, D8, and D10. Longer window lengths were avoided because of increasing commission errors (CEs) in detections. This approach may still be conservative, however, since SPOT 5 detected burn areas could have occurred several days before image acquisition. MODIS pixels corresponding to regions obscured by cloud cover in the SPOT imagery were excluded from the accuracy assessment.
Product reliability was quantified using an error matrix to compute CE and OE. We followed (Tsela et al 2014) using BA intersection analysis to find the OE (equation (1)) and CE (equation (2)) for different BA sizes. Overall accuracy (OA) was calculated as 1-OE. Since SPOT-5 has a spatial resolution with fifty times more detail than the MODIS products, we divided the error assessment into three groups based on BA size, i.e. all BAs regardless of BA size (G1), all areas excluding small fires (G2, ⩾25 ha), and all areas considered as large fires (G3, ⩾100 ha).
We used linear regression to compare the proportion of the product's detected area burned to that shown in the reference data (Eva andLambin 1998, Smith et al 2007). A 5 × 5 km grid was created over the SPOT 5 coverage. This is the same grid size as used in previous analyses Boschetti 2009, Giglio et al 2018). We excluded grid cells covered by clouds and any land cover polygon ⩽6.25 km 2 . The fraction of the areas burned within each 5 × 5 km grid cell over each SPOT 5 image footprint was aggregated to effectively compare spatial agreement between the coarser BA stands for BA while OE and CE are omission and commission error, respectively. Figure 2 describes the process used for evaluating MODIS BA product accuracy in this study.

Temporal accuracy of burned area products
In order to determine which product(s) more accurately reflected when BA was accumulating, we used visible infrared imaging radiometer suite (VIIRS) 375 m AF (VNP14IMG) to examine when the burning occurred at this site. The VNP14IMG product detects more fire pixels compared to the 1 km MODIS AF (MOD14/MYD14) used by MODIS BA products (MCD64A1 and FireCCI51) due to its higher spatial resolution. This product is also superior for detecting smaller/cooler nighttime fires that are characteristics of fire on peatlands. The product includes burn pixel coordinates, fire radiative power (FRP), and confidence level. The product is available at https://ladsweb.modaps.eosdis.nasa.gov/.
We downloaded a vector file of points representing the center of the burn pixel. We only selected the burn pixel with the type attribute 'presumed vegetation fires' to limit the possible error due to other anomalies such as detection over water or other static land sources. These values were usually located along the river in our study site. Although burning along the river is possible, the number of pixels we removed was very low (1% of the total burn pixels during September and October 2014). We aggregated daily fire counts and then accumulated them from the beginning of September to the end of October. We also converted the points into a raster, pertaining to the original pixel size of 375 m. We aggregated the monthly FRP pixel (September and October only). When more than one point fell within a raster cell, the features were summed. FRP estimates the radiative energy component released during burning which, relates to combustion rate or fuel consumption (Wooster et al 2005).
This dataset was independent from all of the evaluated product algorithms, but has recently been shown to reliably detect fires in Indonesia's peatlands (Sofan et al 2020). The AF product detects fire activity with low levels of CEs (Schroeder et al 2014). Aggregating the FRP pixels may be conservative, but it reliably represents month to month variations and thus verify which month was the peak of the 2014 burning season.

Temporal accuracy
MCD64A1 C6 had the highest single monthly BA of the four products studied (154 km 2 month −1 vs peaks of 126, 101, and 61 km 2 month −1 for MCD64A1 C5.1, FireCCI51, and MCD45A1, respectively) as well as the most total BA for 2014. This corroborates previous studies that concluded that the C6 product detected more BAs than previous MODIS collections (Humber et al 2019). The main difference between MCD64A1 C6 and the other products was that BA was greatest in October as opposed to September for the other products ( figure 3).
We subsequently investigated possible doublecounting of areas burned where the same pixel was labeled as being burned in consecutive months. For the MCD64A1 C6 product, 7.2% of the total  area shown as burned in October had also previously burned in September. Surprisingly, the precursor product (MCD64A1 C5.1) had no pixels detected as potential double-counting across months. The FireCCI51 product had 3.2% of the total area burned in October marked as previously burning in September. The product's user guide indicates double counting is a known issue, specifically in high latitudes (Pettinari et al 2020), due to re-projection of the sinusoidal output to geographic coordinates, however, this is not applicable for this equatorial study site.
We examined independent fire detection data from the daily accumulated VIIRS 375 AF product to corroborate whether large amounts of fires continued into October. The VIIRS data showed that fires continued after September and peaked in mid-October (figure 4). Total October AF counts were 13% higher than in September (6200 vs 5471 pixels, respectively). Fires predominantly occurred in peatlands (~73%), with 65% of all detections located on deep peat. Higher FRP values (>10 MW) were primarily clustered on deep peat regions (see figure 5) for both months. These findings support the MCD64A1 C6 product's representation of October as the peak of the burning season.
Double-counted areas that burned in both September and October, as detected by MCD64A1 C6 and FireCCI51, persisted for one to two weeks after the first detected day of burning (figure 6). The majority of these long burning fires occurred on peatlands (88%), with most occurring in deep peat areas (63%). Only 12% occurred in non-peatlands.

Areal uncertainty
The BA from the MCD45A1 product detected <1% (735 ha) of BAs in the study site, regardless of burnsize group or land cover types and was therefore excluded from further analysis. The other three product's accuracies were analyzed in terms of burn area, using three classes (⩾100 ha, ⩽25 ha, all BAs), and using three different temporal aggregation windows, as defined in table 1. These products approximated the total BA in the study region more accurately for peatlands than for non-peatlands (table 1).
The temporal window length was assessed by summing the area burned from the beginning of the burning season until the date of the corresponding SPOT 5 imagery (D0), as well as 8 d (D8), and 10 d afterward (D10). Product's (D0 temporal window) BA accuracies were evaluated against the reference map (G1, all BA size), first by comparing total reported BAs and then in terms of spatial agreement of the mapped areas (OA). In peatlands, total BAs from MCD64A1 C6, MCD64A1 C5.1, and FireCCI51corresponded to 74%, 64%, and 56% of BA, respectively. When constrained to areas of spatial agreement with the reference map, estimated BAs only corresponded to 48%, 40%, and 38% of validated BAs, respectively (table 1, figure 7). In non-peatlands, all products had low correspondence for total area burned, all less than 40%, with very poor OA (21%, 9%, and 12% respectively for Table 1. MODIS BA product accuracy assessment in peatlands and non-peatland (OA = overall accuracy, OE = omission error, CE = commission error). Three BA classes and the temporal window lengths were assessed; all BAs regardless of size (G1), all areas excluding small BAs (G2, ⩾25 ha), and all larger BAs (G3, ⩾100 ha). The temporal window length was assessed by summing the area burned from the beginning of the burning season until the date of the corresponding SPOT 5 imagery (D0), as well as 8 d (D8), and 10 d afterward (D10). The OE (BAs not detected) and CE (BAs erroneously detected) were calculated based on spatial comparisons to the reference map for each group and product. Total BA of the SPOT 5 reference maps in non-peat and peat for each BA size, respectively, were: G1 =  Of note, the collection 6 MCD64A1 product had somewhat reduced OE and CE by 12% and 3% in peatlands relative to the previous collection 5.1 product, as well as more substantial reductions of 13% and 18%, respectively, in non-peatland. Lengthening the temporal window of observation affected the products differently. For MCD64A1 C5.1, omission levels were substantially reduced for both peat and non-peatlands. Conversely, the C6 product had the least reduction in omission, but the largest increases of CEs. Overall, the FireCCI51 product had the lowest CEs among the products, regardless of changes in the temporal window, most notably in peatlands. However, this has come at the cost of the largest OEs in peatlands.
The exclusion of all BAs smaller than 25 ha (G2) or 100 ha (G3) increased classification accuracy for all products, except for the MCD64A1 C5.1 in nonpeatland (table 1). OA increased by approximately 8% and~21% for all products when smaller fires in peatlands, <25 ha or <100 ha, were removed, respectively. This was due to large reductions in OEs (~8%, −19%, figure 8(a)) with much smaller increases in total CEs (~2%,~9%, figure 8(b)). Significant differences were found in the fire detection of BA products in nonpeatlands where the MCD64 and FireCCI products showed~57% increases in accuracy when only large fires were included, suggesting both products are missing large portions of small non-peatlands fires.
Linear regressions showed generally good agreement in peatlands (R 2 ranging from 0.6 to 0.8, table 2 and figure 7) between the proportions of area burned in 5 × 5 km 2 cells of the BA products and the reference SPOT-5 BA of each grid cell. These results indicate roughly comparable spatial patterns among products and the validated burned map, despite underrepresentation of the proportions burned (positive slope <1).
The MCD64A1 C6 product had the best agreement for both peatlands (including deep and shallow peat) and non-peatland. The product most closely matched the proportional area burned of the Table 2.
Regressions of the proportion of area burned in each 5 × 5 km 2 grid square of the various BA products and the SPOT-5-derived reference map for each land cover type (Zubaidah et al 2017)  reference map in peatlands, specifically in deep peat (Slope =~0.84) but underestimated area burned, yielding moderate correspondence (R 2 = 0.78 and 0.50 for deep peat and non-peat, respectively). Despite having higher spatial resolution, the FireCCI51 product had accuracy similar to MCD64A1 C5.1 product (both with R 2 =~0.61), however, it was less accurate for non-peatland (R 2 = 0.31 vs 0.14, respectively).

Discussion
All products underestimated validated BAs by roughly half, on average. Fires smaller than 100 ha were only responsible for 2.9 ± 0.9% and 2.5 ± 1.6% of area underestimation in peatlands and nonpeatlands, respectively, for all temporal window lengths. Burns in non-peatlands, primarily occurring in croplands, are frequently small and rapid fires that produce less char or ash, making detection difficult in this land cover type, as it has been the case in other regions (Hall et al 2016, Zhu et al 2017. Our results corroborate the previous simulation (Miettinen and Liew 2009), which showed that moderate to coarse resolution in Indonesian peatlands performed better than in non-peatlands, given the larger burn scars in peatland areas. By excluding small fires (<100 ha) from the analysis resulted in the greatest accuracy increases for MCD64A1 C5.1, indicating that this product has the worst small fires detection capability. FireCCI51 was least affected by small fire removal but it was less accurate overall than MCD64A1 C6. The MODIS instruments on the Terra and Aqua satellites have known detection issues when dense clouds and smoke interfere (Giglio et al 2003), common conditions in the study region. Since the Fir-eCCI51 product is generated solely from Terra satellite MODIS data (Chuvieco et al 2018), known issues of regular orbital space gaps at equatorial locations, such as Sumatra and Kalimantan, may partially explain the larger under-estimation of BA by this product despite its higher spatial resolution.
Inclusion of detected AFs (MOD14/MYD14) into algorithms for more recent BA products improves detection rates (Humber et al 2019). The lack of this feature in the algorithm of the older MCD45A1 product (Roy et al 2008) may explain its apparent inability to detect BAs in this perennially cloudy region. MCD64A1, which had the most accurate products, is more tolerant of cloud and aerosol contamination (Giglio et al 2009) since the algorithm relies primarily on both thermal infrared band and changes in vegetation indices using shortwave and near-infrared bands. These two bands discriminate areas burned and unburned more distinctly than other bands (Huang et al 2016). MCD64A1 is currently used by models for global carbon emissions estimation (e.g. GFED4 used MCD64A1 C5.1). The long time series (late 2000present) and broad coverage of the MCD64A1 data make it ideal for producing global emissions estimates. Although there are uncertainties due to other variable model parameters (emissions factors, combustion completeness, peat burn depth) (Whitburn et al 2016, Heymann et al 2017, Wooster et al 2018, our results indicate that BA alone has contributed 50% to the uncertainty of emissions estimates from fire activity in this region in 2014. Our study was limited to a year with moderate burning extent and intensity. Product's accuracy varies spatially and temporally among regions (Humber et al 2019) and this likely affects our study region as well. Accuracies in severe burning seasons (e.g. 2015), when thick smoke blankets the region for weeks on end may have even larger discrepancies because of the lack visibility that precludes BA observations by all of these MODIS-based systems. We encourage further product accuracy assessment at various locations and seasonal periods of burning in Indonesian peatlands.
The frequency of fire activity at specific locations, which may directly relate to the amount of fire emissions in peatlands, has not been accounted for by most emissions models. However, such estimations can only be made if long-term annual BA maps, with better spatial resolution than MODIS, become easily available. Fire frequency at a site controls the risk of peat burns (high emissions rate) in peatlands. Konecny et al (2016) suggest that the first time a peatland burns the peat is consumed to an average depth of (17 ± 16 cm), while subsequent burns in the same area only burn roughly half as deeply. Lohberger et al (2017) incorporated these results and found lower regional emissions rates than the GFED4 emissions model had reported. In 2015, GFED4 estimated a nearly doubled emissions rate for Indonesian fires (1.75 vs 0.89 Gt CO 2 e) despite the lower amount of BA identified in their study using MODIS BA. As shown here, current BA products substantially underreport the amount of fire affected area. Uncertainties are largest in non-peatlands but the greatest impact on global emissions estimations from models comes from under detection of annual fire in peatlands, where smoldering fires lead to disproportionately large amounts of aerosol and gaseous emissions, and inability to account for recurrent fires in subsequent years. Improved BA estimation, particularly in Indonesian peatlands, requires specific attention in order to improve the accuracy and precision of global carbon emissions estimates.
The challenges inherent in mapping BAs in the cloudy and smoky peatlands of Indonesia make alternative approaches necessary to improve BA map reliability. MCD45A1, for example, suffered from missing observations due to clouds and smoke, hampering systematic change-detection efforts using passive sensors to map this region (Roy et al 2005). To overcome the issues with clouds and smoke obscuring the land surface some combination of more frequent visual observations, integration of BA products, and methods that allow for imaging through clouds and smoke must be employed.
The amount of freely available satellite imagery has increased in recent years, providing cost-effective opportunities for developing integrated methods of BA mapping and validation. With the launch of various free datasets with that have higher spatial, temporal, and spectral resolutions there is the potential to provide more comprehensive BA maps (Miettinen et  Studies are needed to explore the complementary nature of these various datasets for mapping BAs, specifically in Indonesian peatlands, to overcome issues including cloud cover/shadow, small fires, and smoldering fires (low intensity). Recent studies have proven that these datasets improve BA detection in this region and other fire-prone areas (Lohberger et al 2017, Roteta et al 2019, Roy et al 2019, Carreiras et al 2020, Hawbaker et al 2020, Sofan et al 2020. However, none of them can provide the longterm burning time series that MODIS does. Even Landsat, which has historically supplied long-term sequences of imagery, is lacking in this region. Newer systems such as Landsat 8 and Sentinel-2 may help rectify this for future years by providing many more potential observations throughout a burning season. Multi-sensor integration, including passive and active remote sensing sources may improve accuracy but, to the best of our knowledge, no studies have investigated this approach, specifically for Indonesian peatlands that are more vulnerable to fires (Vetrita and Cochrane 2020).

Conclusions
We compared the accuracy of four MODIS-derived BA products to a high resolution validated BA reference map for 2014 in Central Kalimantan, Indonesia, including two decommissioned products (MCD45A1 C5.1 and MCD64A1 C5.1) and two currently available products (MCD64A1 C6 and Fir-eCCI51, the product developed under a CCI project of ESA). Currently available products were more reliable than the older ones, as expected. The standard BA MODIS product, MCD64A1 C6, was the best, suggesting a better performance than its precursor (MCD64 C5.1). Despite the higher spatial resolution of FireCCI51 compared to MCD64A1, the BA product showed lower improvements for detection of smaller BAs (<100 ha).
Our findings bring new insight about the performance various MODIS satellite-based approaches for discriminating burned and unburned areas in tropical peatlands/non-peatlands. The globally significant emissions from frequent burning of Indonesian peatlands makes observation and quantification of these fires critical for effective monitoring and application of global emissions models. However, in this region, cloud cover and heavy smoke from persistent burning substantially degrades the effectiveness of existing MODIS-derived BA mapping efforts. Our site in Central Kalimantan is one of the most severely fire-impacted regions in Indonesia, with recurrent burning prevalent for more than a decade (Vetrita and Cochrane 2020). Our study was limited to the dry season of 2014 due to our available reference map which had less severe burning than what often occurs during El Niño events. Since smoke is even thicker and more persistent then, our results showing the inaccuracies of current global BA products may be conservative.
We still recommend using currently available MODIS BA products (MCD64A1 C6 and FireCCI51) for a national scale monitoring. With nearly two decades of observations, the long time-series data provide unparalleled insight into Indonesia's fire history. However, mapping BAs at higher spatial resolutions remains necessary in order to accurately detect changes and spatially locate peat fires. We urge use of both satellites with MODIS instruments, Terra and Aqua, to get better coverage and more chances to improve detection of BAs in the frequently cloudy and smoke covered peatlands of Indonesia. Since the planned operational lifetime of the Terra and Aqua satellites is coming to an end, the next generation of satellites (e.g. Suomi NPP and NOAA-20) will continue monitoring of Indonesian burning. Having a wider swath, without any orbital gaps at the equator and higher spatial resolution than the MODIS precursors, the continuing BA products, combined with multisensor satellites that are currently available, such as Sentinel-2, Sentinel 1, and Landsat 8 will ensure and improve future analyses of long-term burning history in Indonesian peatlands. These products will be useful for users with various applications, including fire frequency analysis, fire ecology, or fire-related and affected social assessments.
Strengthening monitoring systems by incorporating various additional data sources will help stakeholders to manage the land and improve the ability of emission-modelers to accurately map global emission levels, which remain highly uncertain. Indonesia has a critical need for accurate and timely BA mapping to meet a variety of needs and different purposes, including fire-related emissions monitoring of burning peatlands, law enforcement, rapid assessment, and fire suppression efforts. However, clear guidelines for how to accurately interpret these datasets are essential.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).