Emergency flood bulletins for Cyclones Idai and Kenneth: A critical evaluation of the use of global flood forecasts for international humanitarian preparedness and response International Journal of Disaster Risk Reduction

However, many chal- lenges remain in using forecasts systematically for preparedness and response. The research community in place through SHEAR enabled the UK government ’ s Department for International Development to task a collaborative group of scientists to produce probabilistic real-time flood forecast and risk bulletins, aimed at humanitarian decision-makers, for Cyclones Idai and Kenneth, which impacted Mozambique in 2019. The process of bulletin creation during Idai and Kenneth is reviewed and critically evaluated, including evaluation of the forecast information alongside evidence for how useful the bulletins were. In this context, this work seeks to navigate the “ murky landscape ” of national and international mandates, capacities, and collaborations for forecasting, early warning and anticipatory action, with the ultimate aim of finding out what can be done better in the future. Lessons learnt and future recommendations are discussed to enable better collaboration between producers and users of forecast information.

Humanitarian disasters such as Typhoon Haiyan (SE Asia, 2013) and the Horn of Africa drought (2011)(2012) are examples of natural hazards that were predicted, but where forecasts were not sufficiently acted upon, leading to considerable loss of life. These events, alongside international adoption of the Sendai Framework for Disaster Risk Reduction, have motivated efforts to enable early action from early warnings. Through initiatives such as Forecast-based Financing (FbF) and the Science for Humanitarian Emergencies and Resilience (SHEAR) programme, progress is being made towards the use of science and forecasts to support international humanitarian organisations and governments in taking early action and improving disaster resilience. However, many challenges remain in using forecasts systematically for preparedness and response. The research community in place through SHEAR enabled the UK government's Department for International Development to task a collaborative group of scientists to produce probabilistic real-time flood forecast and risk bulletins, aimed at humanitarian decision-makers, for Cyclones Idai and Kenneth, which impacted Mozambique in 2019.
The process of bulletin creation during Idai and Kenneth is reviewed and critically evaluated, including evaluation of the forecast information alongside evidence for how useful the bulletins were. In this context, this work seeks to navigate the "murky landscape" of national and international mandates, capacities, and collaborations for forecasting, early warning and anticipatory action, with the ultimate aim of finding out what can be done better in the future. Lessons learnt and future recommendations are discussed to enable better collaboration between producers and users of forecast information.

Introduction
In early 2019, two tropical cyclones (TCs) made landfall in Mozambique with devastating impacts. Cyclone Idai made landfall in central Mozambique in March and Cyclone Kenneth in northern Mozambique in April. Both were classified as intense TCs, with Kenneth the strongest cyclone to impact Mozambique in modern history (based on records from 1980 onwards); Idai resulted in more than 600 fatalities and left at least 1.85 million people in need of humanitarian assistance [1] in Mozambique alone, with further fatalities and impacts in Zimbabwe and Malawi, while Kenneth caused 45 fatalities and displaced thousands [2] in Mozambique.
Usually the first thing that comes to mind when we hear about TCs is the destructive winds. However, in many cases the water can be much more dangerous, as waves and storm surges flood the coasts and heavy rainfall causes riverine flooding further inland [3]. The impact of the rainfall has a longer timescale and can obstruct humanitarian aid during the weeks and months after a cyclone. It is therefore essential for humanitarian and civil protection agencies to have the right information on upcoming rainfall and flood risks. Since 1980, 18 tropical systems have impacted Mozambique, affecting between 11,000 (Cyclone Hudah, April 2000) and ~1.85 million (Cyclone Idai, March 2019) people, and resulting in a total of more than 2000 fatalities. The most severe of these were Cyclones Idai and Eline. Cyclone Eline made landfall on 22nd February 2000, shortly after severe flooding in January 2000, and was followed just a few days later by Cyclone Gloria, which made landfall on 8th March. This combination of events affected ~650,000 people and resulted in ~750 fatalities [4]. While TC landfalls do not occur in Mozambique every year, cyclones with the intensity of Eline, Idai and Kenneth are not unprecedented in the region.
While the Sendai Framework for Disaster Risk Reduction (SFDRR [5]) recognizes member states' primary responsibility to prevent and reduce disaster risk in their own countries, it also articulates the need for strengthening of international cooperation and global partnership to allow high-risk countries to implement DRR programmes with the overall goal to build resilience. It is not the case that national authorities simply have either 'capacity' or 'no capacity' for using forecasts, providing warnings and taking action. It is much more of a 'murky landscape' demanding "multi-level governance systems" [6] and a complex series of multisectoral, inclusive and accessible collaborations [5]. In addition to governments, humanitarian and development agencies and other relevant stakeholders need to collaborate to prepare for and respond to these types of events and are increasingly looking towards using scientific forecasts to anticipate the impacts and act early. The basic rationale for using forecasts is to reshape humanitarian assistance through innovation that improves efficiency and prevents human suffering and losses [7].
Through initiatives such as Forecast-based Financing (FbF) and the UK's Science for Humanitarian Emergencies and Resilience (SHEAR) research programme, 1 progress is being made towards the use of science and forecasts in taking early action ahead of a disaster. For example, the Red Cross took action based on forecasts of flooding in Uganda and Peru in 2016 [8]. However, many challenges remain for international organisations to use forecasts systematically to respond ahead of disasters. These barriers involve technical, communication and infrastructural issues [9], but also relate to different institutional practices, expectations, values and mandates, which further influences how success and evidence is perceived and measured [10]. Moreover, who produces knowledge and where it will be implemented touch upon deeper questions that revolve around history, epistemic politics and geographic divides [11][12][13] that need to be taken into account in the long-term goal towards DRR and building resilience. While the mandate for providing warnings lies with the national authorities, and triggers for early humanitarian action must be based on these mandated forecasts, international organisations can provide key supporting information. In the case of Mozambique, a WMO mission following Idai found that significant gaps and weaknesses exist in terms of accuracy of the (flood) warnings, but also in terms of overall emergency preparedness, response and coordination. This includes a limited understanding of risk at institutional and individual levels, which might be due to the low frequency nature of tropical cyclones [14].
On 19th March 2019, 5 days following the landfall of Cyclone Idai, the President of Mozambique declared a state of emergency, requesting international assistance [15]. The research community in place through the SHEAR research programme enabled the UK government's Department for International Development (DFID) 2 to task this team of authors, a collaborative group of scientists and model developers, to produce real-time flood forecast bulletins in order to support humanitarian decision-making during the flooding that followed Idai's landfall.
Less than 6 weeks after Cyclone Idai, when forecasts indicated a second TC would impact Mozambique, the same team were able to provide these emergency flood bulletins ahead of Cyclone Kenneth's landfall, after a request for reactivation from the United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA). The bulletins were also shared by DFID with UN OCHA, INGC and humanitarian organisations in real-time, and the team shared the information with research partners at the Red Cross in Mozambique. The bulletins were not disseminated to the public. We used fluvial flood forecasts from the Copernicus Emergency Management Service's Global Flood Awareness System (GloFAS), based on atmospheric forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF), and then undertook detailed flood inundation estimation and impact risk assessment for population exposure estimates, providing daily bulletins for ~2 weeks at a time for each cyclone. An example from the front page of one of these bulletins is shown in Fig. 1, and a full bulletin from Cyclone Kenneth is provided in the Appendix. Fig. 2 details the daily timeline of the bulletin creation.
Emergency briefings and bulletins are a way of communicating natural hazard forecast information to decision-makers and stakeholders such as civil protection and humanitarian actors. They can be part of an online decision support system (e.g. Ref. [16,17]) or stand-alone documents that can be emailed or downloaded [18] and can also feed into synthesis situation reports such as those produced by UN OCHA. How and when the forecast information is communicated is of critical importance [19,20] and such bulletins must be able to rapidly convey the upcoming danger, as well as the uncertainty in the forecasts, through images and clear textual guidance [21,22].
The series of events that led to the request for these emergency flood bulletins suggests that, on an international scale, there are not yet adequate systems in place to make the best use of scientific forecasts of natural hazards. In addition, the rapidly increasing interest from humanitarian and development partners in using forecast information for real-time decision-making before (the impact of) a natural hazard event occurs, requires not only a critical assessment of whether the forecasts achieve an acceptable level of skill and accuracy for the intended purpose, but also of the 'how' and 'when' of the information provided by emergency bulletins, and what the needs of the users are in this process [23][24][25]. This paper contributes to this discussion by critically evaluating this process of real-time bulletin creation for these two events, and makes an assessment of how the bulletins were used and how they could be improved in the future. In doing so, this work seeks to navigate the  "murky landscape" of national and international mandates, and capacities and collaborations for forecasting, early warning and anticipatory action, with the ultimate aim of discussing what can be done better in the future, particularly to enable increased collaboration between producers and users of forecast information.
The following sections provide a hydro-meteorological overview of the two cyclones and their impacts, an overview of the forecasts and warnings available from the national authorities in Mozambique, a description of the forecasts and models used to produce the bulletins and an evaluation of the forecasts of the two cyclones, followed by a critical discussion on the use of and response to the flood bulletins alongside lessons learnt and recommendations for the provision of such information for future events.

Hydro-meteorological summary of Cyclones Idai and Kenneth
The 2018-2019 south-west Indian Ocean (SWIO) cyclone season saw the largest number of intense TCs recorded in one season (based on records from 1980 onwards) in this ocean basin; of the 18 tropical systems, 11 were classified as intense TCs with wind speeds exceeding 165 km/h. In the SWIO, the cyclone season typically runs from September through to April, with the majority of systems occurring between  December and March. In the 2018-2019 season, the first system to impact Mozambique was tropical storm Desmond, which made landfall ~200 km north of Beira (see Fig. 3) on 19th January 2019. While the storm was short-lived and much weaker than Cyclones Idai and Kenneth, with maximum 10-min sustained wind speeds of 65 km/h [26], it brought significant rainfall and some flooding to the region that would later be impacted by Idai.
The precursor of Cyclone Idai originated in the Mozambique Channel (Fig. 4a) and first affected Mozambique as a tropical depression (with wind speeds ≤ 62 km/h) on 4 th March 2019. The rainfall from the first landfall led to significant flooding across central Mozambique and southern Malawi from 5th March onwards, particularly in the Zambezi River and its tributaries. Upstream within the affected area, the flood peak on the Zambezi occurred on 8th March [27]. Further downstream, the flooding from this first landfall peaked more than four days later, at Mutarara on the 12th, Caia on the 14th and Marromeu on the 16th March (see Fig. 3a). Water levels in some locations, including Tete and Marromeu, reached up to 1.2 m above the flood alert levels [27].
On 9th March, the tropical depression moved back over the Mozambique Channel, where it rapidly intensified. Idai was declared an intense TC on 12th March, with maximum 10-min sustained wind speeds of 195 km/h [26], before moving back towards the Mozambique coastline. Cyclone Idai made landfall near Beira on 15th March, with 10-min sustained wind speeds of 165 km/h and a storm surge of ~4.5 m [28], which, combined with intense rainfall, led to further extensive flooding.
After landfall, Cyclone Idai quickly weakened, but continued to move slowly inland, resulting in continuous rainfall for several days that led to widespread and devastating flooding in central Mozambique, especially on the Pungwe and Buzi rivers. The national hydrological bulletins reported that river levels started to rise in the Pungwe and Buzi rivers on 15th March. However, due to a breakdown of communication systems caused by the cyclone, there are no recorded observations of the flood peak. Some discontinuous observations for the Pungwe river at Mafambisse (45 km upstream of Beira) show two clear characteristics of the event: (i) a fast, extreme increase in river levels between 14th and 19th March, from 4.63 m to 9.3 m, exceeding the flood alert level by more than 3 m, and (ii) a slow flood recession from 20th March to 6th April at a rate of around 10 cm per day.
Beyond the hydro-meteorological hazards, flooding from TCs can lead to outbreaks of disease, and a cholera outbreak was declared in Mozambique on 27th March. This outbreak affected more than 6700 people in the flood-affected Sofala Province [29].
Less than 6 weeks later, another tropical disturbance began to organise to the northeast of Madagascar on 21st April and move westward towards Mozambique (Fig. 4b). This system became a tropical depression and later a tropical storm on 23rd April, at which point it was named Kenneth. Kenneth continued to rapidly intensify and was declared an intense TC on 24th April with maximum 10-min sustained wind speeds of 215 km/h [26], before weakening slightly shortly before making landfall on the evening of 25th April in northern Mozambique, near Pemba (Fig. 3b). In the period from 1950 onwards, just 12 TCs have reached intense TC status in the SWIO during the month of April, Kenneth being the latest and strongest of these.
The rainfall from Cyclone Kenneth led to flooding that began on 26th April in the Megaruma river, with a significant rise in river levels from 28th April in all major rivers in the region, including the Megaruma, Messalo, Montepuez, Lurio, Meluli, Monapo and Ligonha rivers (Fig. 3b). Water levels remained above the flood alert levels until 2nd May [27]. This severe flooding across the Cabo Delgado province of northern Mozambique during the days following Cyclone Kenneth's landfall resulted in an estimated 45 deaths and the destruction of at least 2500 homes [1], alongside the loss of a significant number of crops, fishing boats and fishing equipment [30].

Forecasts, data & bulletin creation
This section provides an overview of the forecast and warning information available from national authorities in Mozambique, followed by a discussion of the forecast models and data used to produce the flood bulletins, alongside additional data and methods used for the forecast evaluation undertaken as part of this study. We primarily made use of ensemble forecast products, which provide a range of possible forecast outcomes taking into account the various uncertainties associated with hydro-meteorological forecasting, and allowing the provision of probabilistic forecast information [31]. Sections 3.2 to 3.4 describe the chain of forecasts used to produce the bulletins in real-time during the two cyclones, from the meteorological forecasts that were discussed in the bulletins, and also as input to the flood forecasting system, through to the population exposure estimates, which themselves make use of the flood forecast data and additional flood inundation modelling. In the bulletins, forecast information was provided through a combination of maps and figures directly from the forecasts and forecast data, alongside expert interpretation of the data to provide a written summary of each aspect of the forecasts. The terminology used within these written summaries made reference to the forecast uncertainty and probabilities based on the ensemble forecasts. For this study, we have further evaluated the forecast accuracy through a retrospective analysis using the raw data from the real-time forecasts that were used to produce the bulletins. The bulletins were recommended for use by decision-makers alongside forecasts from the national authorities, and were not publicly disseminated.

Forecasts and warnings from national authorities
The institutions mandated to issue warnings for meteorological and hydrological hazards are the National Institute of Meteorology (INAM) and the National Directorate of Water Resources Management (DNGRH) in collaboration with regional operational water administrations (ARAs). The INGC (National Institute of Disaster Management) is responsible for coordinating the response to warnings issued by INAM and DNGRH. The disaster management structure in Mozambique is shown in Fig. 5.
INAM issue TC warnings detailing the severity of the storm (ranging from a warning for 'heavy rain, severe thunderstorm and strong wind' through to 'intense tropical cyclone'), the target area (regions likely to be impacted), an alert colour code (indicating the number of hours before a TC makes landfall; blue 24-48 h, yellow < 24 h, red < 6 h), and any available observed data for wind speeds and precipitation. These warnings are updated at least daily during an event.
For TC forecasting and warnings, INAM make use of the TC forecasts provided by the Regional Specialised Meteorological Centre (RSMC). RSMCs have the WMO-mandated responsibility to monitor and name TCs in their region and provide forecasts to national hydromet services. In the SWIO, the RSMC is Météo France La Réunion, who provide daily updates on the meteorological situation and potential for cyclogenesis, and issue technical bulletins and graphical warning products every 6 h during a TC. The technical bulletins contain detailed information on the location, size and intensity of the tropical system, in text format designed for the use of operational forecasters at the national authorities. Graphical warnings products are issued through the Météo France website (www.mete ofrance.re/cyclone/). These provide maps of the predicted track of the centre of the tropical system over the next 5 days, including a cone of uncertainty or 'potential track area' based on forecasts from a range of models, alongside an indication of the expected intensity of the storm. The TC forecasts provided by the RSMC do not currently provide information on rainfall or flooding; INAM's operational forecasters use a variety of rainfall forecast products produced by global forecasting centres, to prepare rainfall forecasts based on their expert analysis.
During the two TCs, DNGRH also issued warnings for flooding, based on observations of river levels, whether the river levels exhibited a rising trend, and qualitative assessment of forecasts and observations of a tropical cyclone and heavy rain. The warnings provided for Cyclone Idai, after landfall, also noted the possibility of water release from a dam in the region which could increase the risk of flooding. This knowledge of the local context, and incorporating upstream observations of river levels into warnings is key information that it would not be possible to provide using a global flood forecasting system such as GloFAS. A WMO mission report ( [14], p27-29) provides further details surrounding the warnings from both INAM and DNGRH, and the forecasting capacity of both institutions.
While INAM, DNGRH and INGC are continually working towards improving the forecasts and warnings they provide, including through various research and operational collaborations (e.g. Ref. [14,[33][34][35]) at the time of Idai and Kenneth, there was limited capacity to provide real-time forecasts of flood hazard and risk information for anticipatory action [14]. As such, the flood bulletins for Cyclones Idai and Kenneth sought to provide complementary information on the hazards and risk associated with the cyclones based on real-time global scale hydro-meteorological forecast models. The warnings issued by the RSMC Kenneth from genesis to dissipation, identified in the ECMWF operational analysis data using the methodology described in section 3.1. Tracks progress from light to dark shading, and cyclone symbols depict the portion of the track when the storms were classified as tropical cyclones. Total observed rainfall (mm) is shown for (e) Cyclone Idai, from 1 to 24 March 2019, and (f) Cyclone Kenneth, from 21st to 28th April 2019, using the IMERG satellite precipitation data (see section 3.1). Also shown is the total forecast rainfall (mm) from the ECMWF HRES forecasts at 1 day lead time, for (g) Cyclone Idai and (h) Cyclone Kenneth. This is the sum of all 24-h rainfall accumulations from forecasts produced 1 day ahead (for example, a forecast produced at 00UTC on 12th March for the 24-h total rainfall accumulation on 13th March) for the duration of each storm. Finally, the mean error of the total rainfall forecast (mm) of the ECMWF HRES forecasts at 1 day lead time is shown for (i) Cyclone Idai and (j) Cyclone Kenneth. Red indicates too little rainfall, and blue indicates too much rainfall in the forecasts. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.) and used by INAM were considered during creation of the flood bulletins, for comparison with the ENS forecasts (see section 3.2) and to ensure consistency of the information provided. The information provided by DNGRH regarding the potential for release of water from a dam was also brought to the team's attention by our Red Cross research partners, and was cited in the flood bulletins.

Flood bulletin creation
For the bulletins, we made use of probabilistic meteorological forecasts from ECMWF's Ensemble Prediction System (ENS). The ENS is part of the ECMWF Integrated Forecasting System (IFS, cycle 45r1) providing twice-daily forecasts out to 15 days ahead, with 51 ensemble members at ~18 km horizontal resolution. The ENS graphical forecast products were used to provide contextual information on the predicted track (path) of the cyclones, alongside the amount and spatial extent of rainfall expected from the cyclones. ENS forecasts are also used as input to the flood forecasts; more information is provided in section 3.2. ECMWF also produce a high (9 km) resolution deterministic forecast (HRES), which was used as supplementary information in the bulletins to provide rainfall maps. A recent study by Titley et al. [36] found that, based on analysis of three ensemble forecasting systems from the UK Met Office, ECMWF and the National Centre for Atmospheric Prediction (NCEP), ECMWF provided the most accurate TC forecasts in the SWIO, although a multi-model ensemble can provide improved skill. Forecast skill was also found to be worse in the SWIO than other ocean basins, for the UK Met Office and ECMWF.
ECMWF's TC track forecast products become publicly available (via www.ecmwf.int) once the system is declared a TC by the Regional Specialised Meteorological Centre (RSMC) responsible for the distribution of warnings in the region.

Forecast analysis
In this study, we identify the TC tracks in the ENS and HRES forecast data using the tracking scheme of Hodges [37][38][39]. This method, described in detail by Hodges and Klingaman [40], locates vorticity maxima matching a set of criteria identifying them as TCs. The predicted TC tracks are then verified against the observed tracks, obtained from the International Best Track Archive for Climate Stewardship (IBTrACS [41]), which combines TC track data from weather centres worldwide, providing a dataset of historical tracks. Operationally, the ECMWF TC track forecasts make use of a different tracking scheme [42,43] than we use here. The tracking scheme used in this study is also currently being used to produce a long-term evaluation of TC forecast skill in the SWIO, in collaboration with the Red Cross, to provide information that can be used towards forecast-based early action for cyclones in south-east Africa. We use it here for consistency, and to allow for further comparison of the forecasts of these storms with a long-term analysis, as it is important not to make an assessment of the overall skill of the forecasting systems based on the forecasts of an individual event.
We further assess the accuracy of the rainfall forecasts for the two cyclones. Following the method of Peatman et al. [44] and Guo et al. [45], we produce composites of the rainfall associated with each TC, whereby rainfall within 5 • of a track point is attributed to the cyclone. This is done for both the HRES and ENS precipitation forecast data using the forecast tracks, and for NASA's Integrated Multi-SatellitE Retrievals for Global Precipitation Measurement (IMERG V05B [46]) gridded satellite precipitation data (0.1 • resolution) using the observed tracks, in order to verify the forecasts. Precipitation products based on satellite data provide valuable and consistent information, particularly in data-sparse regions, but it is important to note that while previous studies have found IMERG to satisfactorily represent the spatiotemporal distribution of TC rainfall, it has also been found to over-represent high-intensity rainfall, and in some cases, under-estimate coastal rainfall over land [47][48][49].

Flood bulletin creation
The flood forecasts used were those of the Global Flood Awareness System (GloFAS, v2.0, www.globalfloods.eu), an early warning component of the European Commission Copernicus Emergency Management Service (emergency.copernicus.eu). The system couples ECMWF's ENS forecasts of surface and sub-surface runoff [50] with a hydrological river routing model (Lisflood [51]), to produce ensemble (probabilistic) forecasts of river flow for the global river network, at 0.1 • (~10 km) resolution with 51 ensemble members. The initial conditions for the GloFAS model are generated by the state-of-the-art GloFAS-ERA5 river flow reanalysis [52,53]. GloFAS provides daily forecasts of flooding in major rivers around the globe, out to 30 days ahead [54], but does not currently provide forecasts for coastal flooding, which can be a significant concern during tropical cyclones. Due to this limitation, when available, we pointed to storm surge forecast information from other sources, such as the European Emergency Response Coordination Centre, and the RSMC, in the bulletins.
While GloFAS v2.0 uses an updated version of Lisflood that has been calibrated using river flow observations at 1287 stations worldwide [55], the model is not yet calibrated in the region affected by Idai and Kenneth, as no observed river flow data were available at the time the model was calibrated.
Each new GloFAS forecast is compared against flood thresholds at every grid point, providing a probability of exceeding three different flood severity thresholds. These thresholds are calculated from the GloFAS-ERA5 reanalysis for various return periods [54]; the medium, high and severe alert thresholds correspond to the 2-year, 5-year and 20-year return periods (50%, 20% and 5% annual exceedance probabilities 3 (AEPs)), respectively. This approach limits the influence of systematic biases, which are expected in regions where the model remains uncalibrated. The GloFAS user guide [53] suggests that decision-makers focus on the hydrological variability, trends, timing and relative magnitude of the flood hydrographs, rather than the exact predicted magnitude of the river flow. This is a key aspect of the GloFAS user interface, and of the interpretation of GloFAS forecasts for use in the emergency bulletins, but it should be noted that this is not simple to carry through to the inundation and exposure estimates, which must make use of GloFAS river flow forecasts and thresholds in order to provide estimates of populations exposed to flooding.

Forecast analysis
To evaluate the GloFAS forecasts for Cyclones Idai and Kenneth, we extract and assess the predicted timing of the flood peak and recession, and the probabilities of exceeding critical flood alert thresholds. These characteristics are the key aspects of the forecast information used for decision-making purposes. We compare these aspects of the flood forecast with observations of flood peaks and timings in the affected region, provided by DNGRH through their hydrological bulletins.

Flood bulletin creation
Population exposure due to flooding was estimated by combining GloFAS forecast probabilities of exceeding the flood alert thresholds, with flood inundation and population information. GloFAS' ensemble river flowforecasts were first downscaled to the ~90 m resolution of the flood inundation information, using inverse-distance-weighting. The exposure is calculated as the population exposed to a particular return period flood inundation, multiplied by the probability of exceeding a return period threshold according to GloFAS. The population is described by the High Resolution Settlement Layer (HRSL [56]) dataset, and the return period flood inundation is a binary yes/no (1/0 where wet = 1 and dry = 0) at each grid point of the global flood inundation model. The GloFAS probability of exceedance is calculated using the percentage of ensemble members that exceed the given return period threshold.
To estimate the flood inundation, a global flood inundation model framework [57] was used to delineate flood inundation zones across the region at ~90 m resolution. Return periods ranging from 5-to 1000-years (20%-0.1% AEP) were calculated in order to provide a range of possible scenarios based on the forecasts. The model estimates riverine flooding for all basins with an upstream area >50km 2 using a sub-grid hydrodynamic model within the LISFLOOD-FP code [58]; there is no coastal flooding component. A regionalised flood frequency analysis conducted at the global scale [59] provides model boundary conditions by linking river discharge and rainfall measurements in gauged catchments to ungauged catchments, based on catchment characteristics and climatological indicators. The modelling framework therefore allows for estimation of riverine flooding at a global scale, including data-sparse regions.
Leyk et al. [60] describe the various available gridded population datasets available and their differences. For the bulletins, we used the HRSL [56] dataset, based on data availability and the work of Smith et al. [61], who demonstrated that the method used by HRSL more accurately placed populations just outside of the most hazardous areas, resulting in a better estimate of exposure, especially in rural areas. To estimate population exposed to flooding during Cyclones Idai and Kenneth, the population data (~30 m) were aggregated to the resolution of the flood inundation data (~90 m). In order to provide the total population exposure per administrative unit, zonal statistics were used. Although GloFAS forecasts do not explicitly provide the probability of exceeding return periods greater than the severe (20-year/5% AEP) alert level, many ensemble members indicated that flooding may substantially exceed the severe alert level on some rivers. As such, we additionally calculated exposure to a range of more extreme flood return periodsflood hazard, in order to report a range of exposure estimates. Exposure information was provided in the bulletins through tables and maps (see Appendix, Table 2 and Figs. 7-9).

Cyclone Idai
Retrospective analysis of the raw ENS probabilistic forecast data indicates that the forecasting system first began to consistently pick up the potential development of a tropical system in the Mozambique Channel, from 26th February onwards. From 1st March, the GloFAS flood forecasts indicated a 10-20% probability (based on the forecast ensemble) of severe flooding (exceeding the 20-year return period/5% AEP) across the region affected by Idai's precursor, in southern Malawi (the Shire River basin) and central Mozambique (the Zambezi River basin, including the Zambezi and Cuacua Rivers). At this point, the flood peaks associated with this first landfall were predicted to occur on 9-10th March across the affected river network, which is consistent with the flood timing later reported by the national hydrological bulletins [27]. From 4th March onwards, probabilities of severe flooding increased, exceeding 80% in rivers across the affected region from 5th March, such as along the Cuacua river (see Fig. 6a). The expected exposure also rapidly increased on 4th March (see Fig. 8), with a peak on 6th March of ~200,000 based on the 20-year return period (5% AEP), and a maximum exposure estimate of ~450,000 people (based on the 3 Annual exceedance probabilities (AEPs) are provided alongside return periods throughout. While return periods currently represent commonly-used terminology in hydrological applications, they can be misleading when communicating potential risk to scientists, decision-makers and non-specialists from a variety of backgrounds. For example, it may unintentionally imply that if a 5-year return period flood occurs, it will not be observed again for 5 years, when in fact there is a 20% chance of a flood of that magnitude occurring in any given year (20% AEP).
From 6th March, 9 days ahead of Cyclone Idai's landfall near Beira on 15th March, the ENS track forecasts indicated a high probability (~70%) of the system 'looping around' over the Channel and making landfall as a TC in central Mozambique, although the precise landfall location remained uncertain. An example of the forecast and the ensemble spread (i.e. forecast uncertainty) is shown in Fig. 7b and d, for the forecast produced on 10th March, and the forecast progression throughout the storm's lifecycle is shown in Animation 1 in the supplementary material. This coincides with GloFAS forecasts beginning to indicate the possibility of a second flood event, in the Pungwe and Buzi River basins, with an expected peak ~18th-20th March in the two river basins, which is consistent with the available observations in the Pungwe river. Fig. 6a shows the evolution of the probability of exceeding the severe flood alert threshold for the two main rivers affected by the flooding from Idai, the Pungwe and Buzi Rivers, and for two of the main rivers affected by the first flooding event from Idai's precursor (Zambezi and Cuacua Rivers). The evolution of the GloFAS forecast probabilities, across the region, is shown in Animation 2 in the supplementary material.
From 10th March, coinciding with the intensification of the storm and its upgrade to TC status by the RSMC, the ENS forecasts for the landfall location became much more confident, alongside forecasts of severe rainfall, extreme winds and flooding in the region around Beira. Fig. 7a shows the track location errors (i.e. the distance between the forecast location of the cyclone's centre and the observed location) in the ECMWF ENS and HRES forecasts. At 3 days ahead, the average track location error was ~200 km, and at 1 day ahead, the errors were ~75 km. This is comparable to the average ECMWF forecast track location errors for TCs in the SWIO, based on the forecasts of 35 recent TCs (2014-2018; not shown).
The location of the storm in the forecasts is key for both the precipitation forecasts and the GloFAS flood forecasts. It is also important to consider that track forecasts indicate the predicted location of the centre of the TC, but the winds and rain associated with the storm can extend for hundreds of kilometres around this point (see Fig. 4a-d). This was a consideration after the storm made landfall, when track forecasts produced on 13th to 15th March indicated that Idai was likely to continue moving further west before dissipating. However, the cyclone stalled over central Mozambique rather than moving further west, resulting in sustained periods of heavy rainfall over the same region; this stalling was picked up in the track forecasts with approximately 1 day's lead time, on 16th March, and this resulted in uncertainty in the flood forecasts. This is shown in Fig. 6a, by a drop in the probability of severe flooding, from ~40% to ~20% during the 13 -15th March period when forecasts were indicating the cyclone was likely to move further to the west. When the stalling was picked up in the track forecasts, the probabilities of severe flooding increased rapidly, and remained consistently high throughout the affected river network (particularly the Pungwe, Buzi and Save Rivers) after Idai made landfall.
Evaluation of the HRES rainfall forecasts using IMERG satellite rainfall data (Fig. 4e-j) indicates that, over land and at short lead times, the ECMWF HRES forecasts for Cyclone Idai typically over-predicted the rainfall totals across much of central Mozambique, and under-predicted the rainfall in northern Mozambique and over the Channel. At 0 days lead time (i.e. a forecast produced at 00UTC for the total rainfall over the following 24 h) errors over land are equivalent to <30 mm per day, or <400 mm over the duration of the storm. Taking all 1-day-ahead forecasts for the duration of the storm (shown in Fig. 4 for the HRES), rainfall was over-predicted by up to 300 mm in central Mozambique, and up to 400 mm in western Mozambique. In contrast, with increasing lead time beyond 1 day ahead, the forecasts show an under-estimation over much of the affected area. At 2 days ahead, we see an underprediction in central Mozambique of up to 300 mm, and an overprediction of up to 300 mm in western Mozambique. Results for the ensemble mean ENS forecast (based on the ensemble mean rainfall associated with the ensemble mean track, not shown) indicate a similar over-prediction in the west, but an under-prediction at all lead times across much of the affected area of central and northern Mozambique. These errors in the rainfall can be tied to the forecasts of the cyclone's track, which predicted the storm to continue moving west rather than the observed stalling over central Mozambique, and the impact of this is seen in the GloFAS flood forecasts as the aforementioned drop in the probability of severe flooding before the stalling was picked up.
The locations and rivers affected by the flooding were correctly predicted by GloFAS with a 10-day lead time. However, for severe flooding the probabilities were relatively low (<30%) until 9th March, with large uncertainties in the expected flood peak timing. The exposure estimates began to highlight the potential severity of the event from 4th March. However, at this point the areas with highest exposure estimates were predicted to be in the Mutatara District, on the border with Malawi. In line with the track and flood forecasts, as time progressed the exposure estimates shifted southwards as the landfall location of cyclone Idai became more certain. As a result, districts such as Nhamatanda and Buzi were forecast to be at risk of flooding at or shortly after landfall.
Comparison of exposure estimates with post-disaster reports are challenging as these principally report the total number of affected people, while the bulletins provided estimates of the number of people affected by flooding, rather than by other/all aspects of the cyclone. According to a UN OCHA situation report [62], 198,300 houses were partially or totally destroyed by the cyclone (while many of these may be due to flooding, it is not possible to say if this was the sole or primary cause), with a further 15,794 households flooded. This suggests that our estimates of the number of people exposed were likely reasonable, as for the 20-year flood hazard (5% AEP) the total estimated exposure was ~200,000 people. An assessment of 14 districts in the Sofala and Manica provinces estimated the total affected population to be ~1 million [2], which is at the upper end of our estimates (see Fig. 9). However, the authors of the report state "it is possible that there was some misunderstanding around the terminology used in Portuguese, and that the floods were understood as a synonym of rain", suggesting a potential overestimation of people flooded, and highlighting the complexities involved in comparing such exposure estimates to the available post-disaster assessments.

Cyclone Kenneth
Ahead of Cyclone Kenneth, the ENS forecasts began to indicate that a tropical system may develop north of Madagascar and impact Tanzania or northern Mozambique, from 18th April onwards. The system was declared a TC by the RSMC on 23rd April. Forecasts of the landfall location in northern Mozambique became much more accurate after the storm's genesis, from 22nd April, and the ensemble spread (i.e. forecast uncertainty) continued to decrease with each new forecast until Kenneth's landfall on 25th April.
Track location errors for Cyclone Kenneth are shown in Fig. 7, and indicate that at 1 day ahead, forecast skill was similar to Cyclone Idai, with an error of ~75 km. However, at 3 days ahead, track location errors were much smaller for Kenneth, at ~100 km (compared to ~200 km for Idai). This is also significantly smaller than typical location errors for ECMWF forecasts in the SWIO, which are ~200 km at 3 days ahead, based on the average error across 35 recent TCs (2014-2018; not shown). The errors increased more rapidly with lead time for Idai than Kenneth, implying that Kenneth's track was much more predictable. Typically, forecast location errors are smaller where TCs tend to move more zonally (such as was the case with Kenneth) compared to those which meander or recurve [63,64]. This is reflected in the GloFAS flood forecasts, which, coinciding with the increasing confidence of the landfall location in the ENS forecasts, consistently indicated an increasing probability of severe flooding in the Messalo, Montepuez and Megaruma Rivers, from 18th to 24th April (Fig. 6b). The expected exposure began to increase on 19th April (6 days before landfall), with the most rapid increase also occurring on 22nd April. Similarly to the forecasts for Idai, a drop in the GloFAS probability of severe flooding is seen on 25th-26th April, due to the ENS track forecasts indicating the storm may continue to move west, rather than stalling over the Cabo Delgado province of northern Mozambique, as was observed. The peak expected exposure occurred 2 days after landfall and ranged from 25,000 people for the 20-year return period (5% AEP) flood inundation to 45,000 for the most extreme 1000-year return period (0.1% AEP) flooding. Fig. 8d shows expected exposure per district for the severe flood (20-year return period/5% AEP) probability and the 100-year (1% AEP) flood inundation. Unlike Cyclone Idai, the ranking of the most exposed district does not significantly alter during the event, due to the more predictable track of Cyclone Kenneth. Comparing these estimates for population exposed per district, based on the bulletin produced on 26th April (see Appendix, Table 2), with a post-disaster assessment [30] from the Global Facility for Disaster Reduction and Recovery (GFDRR), indicates that these estimates correctly predicted which districts were at risk. The districts listed in the bulletin with a probability of flooding (based on the 250-year flood inundation/0.4% AEP) exceeding 10% are the same districts that were indeed affected by the cyclone, and the districts estimated to be at risk with a higher (50%) probability of flooding generally correspond to those with the highest number of people affected [30]. Table 1 provides a district-level comparison between the exposure estimates provided in the bulletin, and the number of people affected per district, in the Cabo Delgado province. While the estimates in the bulletin are somewhat lower than the total number of people affected, this is to be expected as the definition of affected covers many more aspects of the impacts than river flooding, such as extreme winds, food insecurity and disease, and these numbers are also "superimposed on previous heavy rains at the beginning of the year, the effects of Cyclone Idai in some districts, and vulnerable population groups that had been resettled as part of the conflict stabilisation efforts of the previous year" [30]. This poses a significant challenge in evaluating such exposure estimates, as even the best available data on the number of people affected have drawbacks, such as to what degree these data indicate impacts of the storm itself, and, for example, information may be provided in terms of the number of households affected, but it is not clear how many people are assumed per household.

Were the emergency flood bulletins useful?
In this section we use evidence from reports, interviews, conversations, letters, emails and written commentary at post-event meetings, 4 to review the use, usefulness and the potential impact of the bulletins. We critically assess to what extent we can be sure those receiving them found them useful, and were able to take better decisions based on the forecast information, or whether they were just an addition to the overload of information for humanitarian actors and governments involved, distracting from the priorities on the ground.

Making the best use of scientific forecasts of natural hazards
Using science actively in planning and responding to natural hazards is the 'holy grail' of forecast development. The key is to be able to generate, disseminate and communicate the information in meaningful ways to different users who can actively use it early enough for decisions to be taken. In our case, this was a request from DFID following the declaration of a state of emergency in Mozambique and a request for international assistance, and therefore there was a lot of active discussion between the forecast producers and those responsible for passing on the information to h umanitarians on the ground (see Figs. 2 and 5 for an overview of the bulletin production and feedback process with DFID, and the national disaster management structure in Mozambique, respectively).
"This is the first time we have been able to use science so early in both planning for and responding to the devastating impact of cyclones. Your expert analysis, collaborative effort across your organisations and with DFID colleagues, and willingness to tailor and communicate the analysis to the needs of the humanitarian agency end users was well received." [Professor Charlotte Watts, Chief Scientific Advisor for DFID] "The real innovation of these bulletins lies in the fact that this information has been produced in real-time, but of course many challenges remain." [DFID] Feedback received from our international humanitarian partners noted that this was the first time that flood risk information had been provided in real-time to them, and that the type of information was perceived as extremely valuable, innovative and promising for future interventions, particularly due to the move from weather forecasts to more impact-based forecasts. Access to the meteorological forecasts used as input to GloFAS allowed the provision of the meteorological context of the flood hazard and risk, and the inclusion of probabilistic meteorological, hydrological and exposure information in one document was found to be extremely valuable and useful. Nevertheless, despite the novelty of the type of information that was produced, it is clear from the series of events that led to the request for these emergency Table 1 Overview of estimated population exposed to river flooding from Cyclone Kenneth, from the bulletin produced on 26 th April 2019 (see Appendix), for the Cabo Delgado province, alongside the total number of people reported affected in each district [30]. It is important to note that the definition of affected also covers many more aspects of the impacts than river flooding, such as extreme winds, food insecurity, previous heavy rains and other factors. flood bulletins that we do not yet have adequate systems in place to make the best use of scientific forecasts of natural hazards for international humanitarian actions both in terms of their real time nature and the content.

Cascading information to decision makers
The information provided in the bulletins was cascaded to high-level international organisations, the government of Mozambique, and local partners and emergency response coordination centres (but not the public), in a number of ways. The government of Mozambique declared a state of emergency and formally requested international support shortly after Cyclone Idai's landfall. The humanitarian response was led by the Mozambique Disaster Management Agency (INGC), which worked closely with UN OCHA, and the UN clusters. The bulletins were provided as an additional information resource to inform situational awareness, preparedness and response planning, initially through OCHA, which is mandated to coordinate humanitarian assistance with the consent of the national authorities (UN General Assembly Resolution 46/182 [65]). UN OCHA's situational reports (SitReps) drew directly from the bulletins. These SitReps are public documents (available via reports.unocha.org) and shared with the Government. The INGC initially received the bulletins indirectly from OCHA and subsequently directly from DFID who commissioned them and were responsible for their dissemination. Through the provision of information to DFID and onwards to the UN OCHA, who included key points from the bulletins in their daily situation reports, the information was able to reach a wide range of decision-makers at international and local levels, in both government and humanitarian organisations. This led to UN OCHA formally requesting reactivation of the bulletin production when forecasts indicated a second TC would impact Mozambique, and the same team were able to provide these emergency forecast bulletins before Cyclone Kenneth's landfall.
"UN humanitarian response actors stated that the reports produced were "tremendously helpful as we continue to analyse the risks in the days ahead". UN OCHA extracted the key analysis to include into their daily sitreps, which all humanitarian actors and the GoM [Government of Mozambique] use as a key reference point." [Professor Charlotte Watts, Chief Scientific Advisor for DFID] "The information was presented to WHH's Emergency Response director on the ground in Mozambique and to the "Emergency Decision Panel" -senior Management in Bonn, Germany, to facilitate the decision" (to send part of the team to conduct an assessment in/ around Pemba) [Welthungerhilfe (WHH) via DFID] In addition to providing the information to DFID and UN OCHA, we were able to share the bulletins with national humanitarian and government organisations directly, through SHEAR collaborations with incountry partners. This provided the opportunity for decision-makers to ask questions directly to the team involved in producing the bulletins, and to receive the information faster than may have been possible through the information cascade from high-level organisations. Feedback received from decision-makers and operational organisations was also useful for the team producing the bulletins and allowed us to refine the methodology and format with each new bulletin produced. Through this process, we were also made aware of some key aspects of the situation on the ground, which could be further incorporated into the following flood bulletins and passed on to DFID, such as knowledge of a dam in the area that may be at risk. This was important information to highlight in the bulletins, as not all reservoirs are represented in the GloFAS hydrological model, resulting in uncertainty in the flood forecasts around this location.

How were the bulletins used in taking decisions?
A key objective of the bulletins was to facilitate decision-making and increased understanding of the situation and nature of the risk. 5 While we learn from partners that the ground-breaking element of the bulletins was the fact that it was "produced, shared and it informed" in real-time, it is more challenging to find out how this type of information directly informed decision-making. It is not always easy for organisations to articulate how the bulletins were helpful. In emergency situations, decision-makers are required to consider numerous and varying pieces of information in order to take a balanced decision, and as such, a specific contribution to a complex decision will always be difficult to convey. Discussing the use of big data (and the so-called four Vs: Volume, Variety, Velocity and Value), for emergency decision-making in the context of natural disasters, Zhou et al. [66] state "one of the important contents of natural disaster emergency decision lies in the way to describe the data with different sources, data mapping and fusion, feature extraction and classification, quick and accurate access to valuable information and intelligent decision in emergency response". The bulletins were therefore one piece of information amidst an array of other types of information within a wider system and in a complex situation. Some operating organisations incorporated the bulletins into their existing knowledge dissemination products (UN OCHA), yet for others it was the first time they had received real-time information and might simply not yet know what to do with it. Furthermore, it is difficult to evaluate whether the use of the bulletins enabled organisations to take better decisions than if they hadn't had the information.
Feedback from partners, both directly and through DFID, indicates that a key contribution of the bulletins was to assist in creating an overview of the situation; where and when flooding was likely to occur, where there were more people at risk, and when the floods were likely to recede. This was best done using a range of information from both the bulletins and other sources of local data. "The bulletins were very helpful. They gave us an overview of which rivers were at greatest risk of flooding, and this helped inform where we gave the greatest attention to. We used them to help inform our daily briefings to partners, as well as in our public information products. All of this meant that the humanitarian community had far greater information, in real-time, about flood risks, than we have often had access to in the past." [Gemma Connell, Head of Regional UN OCHA in Southern and Eastern Africa] "Whether they specifically 'redirected' measures, I don't know, but I am fairly sure that they assisted in creating the overview [of the situation]." [Hanne Roden, Programme Coordinator, FbF Project Delegate, German Red Cross -Mozambique] Through personal communication with DFID, we were informed that 5 It is important to note that national authorities have the mandate for early warning and civil protection. Triggers for taking early humanitarian action should always be based on forecasts and warnings from mandated national authorities. In practice, information from international organisations and global forecasting systems can be used to support the decision-making process. the bulletins enabled decision-makers to understand the flood risk, move assets and equipment, and release supplies. This enabled an early response to take place into the locations that were likely to be at greater risk, and which also may have become cut off when the flooding reached its peak, meaning that aid would have been further delayed until after flooding had receded. This was particularly the case for Cyclone Kenneth, as for Idai these bulletins were produced in response to the cyclone's landfall, when some of the most-affected areas had already been cut off. Partners have told DFID that they relied on that information and that it helped them to make quick, informed decisions rather than more subjective decisions that would have had to be made without the information provided in the bulletins.
"The information was used to decide whether to send part of the team to conduct an assessment in/around Pemba. […] A decision was made to send an assessment team and the analysis informed the composition of the assessment team as well as the preparation:preparedness activities to be considered such as contingency stock,measures considered: immediate availability of wash kits / hygiene kits, water treatment & filters and tarpaulins" [Welthungerhilfe via DFID]

The need for future bulletins
Further communication with partners has highlighted the need for this type of information to be available for future events. Providing all relevant hydro-meteorological hazard and risk information in one place, including expert interpretation of the data and forecast products, proved to be essential for informing decision-making and providing a more indepth overview of the situation. Both DFID and the Mozambique Red Cross (Cruz Vermelha de Moçambique, CVM) would like to continue to explore the utility of the bulletins and possible ways forward, including whether there is potential to establish a standing capability for similar emergency briefings alongside collaboration and training with local partners, to better co-design future bulletins and build capacity for their use. CVM has requested further collaboration with the team, in order to develop simulation exercises and similar analysis of flood hazard and risk in relation to other river basins and past flood events. This could be relevant for further developing their flood and cyclone Early Action Protocols in Mozambique, and to enhance preparedness in the face of future events. "In the FbF context in Mozambique it could be very interesting and relevant to imagine a case with a certain constellation (wind speed, expected rainfall etc.) making landfall in, for example, Sofala and affecting the Buzi river basin […] you could produce the same type of maps, possible impacted areas, etc.
[…] and together we would be able to make some fairly accurate maps, and prognosis of the impact should such an event occur. And if we had this kind of data, beforehand, then it could for sure inform the measures taken by various actors including RCRC/CVM in preparation of the next cyclone/rainy season." [Hanne Roden, Programme Coordinator, FbF Project Delegate, German Red Cross -Mozambique] "Given the demonstrated utility of such analysis, we intend to learn lessons and examine options to better enable this type of science input in future humanitarian responses." [Professor Charlotte Watts, Chief Scientific Advisor for DFID]

Challenges, lessons learnt & recommendations
In this section, we discuss the key challenges faced and lessons learnt during Cyclones Idai and Kenneth in Mozambique, and provide recommendations for the provision of such information for future events. In doing so, we advocate careful consideration of the differences in humanitarian response, management set-ups and different actors, in extrapolating these experiences to other potential scenarios.
During the Discussion Meeting on Cyclones Idai and Kenneth, held in Maputo on 20 th September 2019, participants were asked to reflect on the usefulness of the bulletins, and following this, were also asked to discuss challenges that were faced, and how the forecast information and response could be improved. Sixteen participants, including representatives from INAM, DNGRH, INGC, regional water agencies (ARAs), the Mozambique Red Cross (CVM) and the Technical University of Mozambique (UDM), provided their perspectives through active discussion, interviews and written commentaries. The responses, alongside observations from the team that produced the bulletins and those facilitating the Discussion Meeting, are incorporated throughout the following subsections.
6.1. Towards co-production and embedding of social science knowledge

Challenges
Considering that the bulletins were produced in an ad hoc fashion during a state of emergency, challenges around the co-production of knowledge in a complex landscape inevitably emerged, which warrants reflection on how knowledge production and use can be more systematically integrated as an ongoing effort for future collaborations and capacity building.
While the real innovation was the production of this kind of information for humanitarians and other interested organisations, as social science research on the use of science and technology for meeting human-development needs has demonstrated, one of the remaining challenges is to better link science and decision making in more socially and institutionally embedded ways from the beginning [10]. This means that more research is needed to understand which institutional factors promote or inhibit the use of uncertain forecasts, and how organisations can be better prepared to make use of real time information during emergencies. One of the reasons for international humanitarian organisations setting up early action protocols, is to pre-assess the skill of various forecasts and decide which are the most appropriate to use for a certain region or event. If new forecasts are then produced during an emergency, how can decision-makers judge the quality of the forecasts? When humanitarian decision-makers begin receiving information from organisations outside the mandated national authorities, how do they know whether this information is trustworthy? During Cyclones Idai and Kenneth, an overwhelming amount of information was received by the Red Cross, such that it was necessary to designate a 'gatekeeper' to filter the information and avoid excessive amounts of information reaching decision-makers on the ground.
As research on the use and societal uptake of forecasts has demonstrated, the question of whether scientific information is useful and useable or not, is not only bound to epistemological concerns (is the science 'good enough', or do users understand the information?), but can largely be explained by nontechnical or institutional factors like mandate, capacity, accountability, how success is defined and measured, and by regulatory frameworks and attitudes to risk [67,68]. In other words, less scientific uncertainty does not automatically lead to less policy uncertainty. Moreover, scientific information is likely to be more effective if it is perceived to be not only scientifically credible but also salient and legitimate, which in turn is closely linked to producing knowledge that is socially robust and can be used within the context in which it is intended for [10].

Lessons learnt and recommendations
To address these critical challenges, four institutional functions are proposed as a means to effective coproduction [10]: (1) face-to-face contact between stakeholders, (2) translation, both literal in terms of language and jargon, and metaphorical, (3) bring together experts and decision-makers from all relevant disciplines (collaboration), and (4) represent different interests to ensure fairness. Additionally, because the representation of scientific uncertainty is largely shaped by social relations among scientists and those they advise [69], more research is needed to understand this kind of 'boundary work' for different stakeholders and how to deal with the question of uncertainty and accountability in the context of emergencies. Since the emergency bulletins are so-called "boundary objects", which are outputs that "are both adaptable to different viewpoints and robust enough to maintain identity across them" [70], we have to take the continuous co-production of knowledge across boundaries of science and policy more seriously, rather than viewing it merely as an ad-hoc endeavour.
The Weather and Climate Services for Africa (WISER) and Future Climate for Africa (FCFA) programmes have produced detailed and clear guidance for co-production of weather and climate services [71], aimed at those involved in co-production "ranging from the academic/practitioner project manager to national meteorological services and government officials wanting to integrate co-production principles into their own work processes". The manual sets out guidelines, recommendations and tips for initiating successful and lasting collaborations that work towards the co-production of weather and climate services that are relevant and useful (https://futureclimateafrica.org/coproduct ion-manual/). Effective collaboration and co-production should begin with identifying all the relevant actors, typically within three categories: producers (for example: national hydromet services, local forecasters, regional or international organisations, research institutes), intermediaries (government ministries, NGOs, media, research institutes), and users (government ministries, humanitarians, citizens, private sector, local leaders), and work towards building common ground, and co-developing and co-delivering solutions [71].

Recommendation 1: Based on the guidance set out in the Manual for Co-production in African Weather and Climate Services [71], identify the full range of relevant actors (producers, intermediaries and users) to initiate, or develop existing, partnerships and build trust-based relationships and collaborations that go beyond an emergency event situation.
Ideally, the format of such bulletins would be agreed, through collaboration between the full range of relevant actors, prior to the occurrence of an event, providing a template outlining the scientific information required in each section, and the terminology that would allow the content to be universally understood. This process would allow those producing the bulletins to be sure that the information being provided is really what is needed by decision-makers, and would also work towards establishing a collaboration and mutual understanding between forecast users and producers.
While it would be best to agree ahead of time on the key information to be included, it was imperative that the team remained flexible to changing user needs in order to adapt the information provided in response to the changing situation. The real-time feedback that was provided by humanitarian partners, which may be characterized as an emergent form of co-production, was immensely helpful to understand user needs and improve the bulletins. But the feedback received after the events by national authorities, during stakeholder engagement meetings, also identifies further research gaps related to institutional barriers for using forecasts; how to reduce access limitations to forecast data and information, and how best to involve local communities in preparedness activities, both of which would help to increase the uptake of forecast information for decision-making and could also be applied to other regions and types of event.
Recommendation 2: Work towards understanding which institutional factors promote or inhibit the use of uncertain forecasts, and support organisations to be better prepared to make use of real-time information during emergencies, for example through training that involves all actors.
Recommendation 3: Through collaboration between all relevant actors (producers, intermediaries and users), agree on the most effective and useful format for future emergency bulletins, including both scientific content and terminology, while allowing room for flexibility during an emergency situation.

Challenges
A key consideration and challenge arising from the production of these emergency bulletins, is the systematic production of this type of information for future events. There have since been several discussions around the challenges faced by decision-makers who received and used this information during Cyclones Idai and Kenneth, but they do not know if they will receive this information ahead of future floods, cyclones or other natural hazards. These bulletins were produced primarily by a team of research scientists in collaboration with model developers, which is unsustainable and cannot realistically be replicated every time an extreme event occurs. It is imperative to find a way to ensure that such information can be relied upon ahead of future emergencies. There are however many barriers and challenges to providing an operational forecast product, which requires effective identification, production and use of the best science-based information, collaboration between authorities and actors at a range of levels, and effective coproduction and dissemination of information in real-time.
Another key challenge was the impact of the cyclones on communication infrastructures, that meant it was not always possible to access the internet in order to download the bulletins, raising the question of whether it may be possible to provide a very brief overview of key information, for example, by text/SMS message. However, this has further potential challenges, not least to ensure that any such messages do not conflict with those sent by national authorities, further highlighting the importance of close collaboration between international organisations and mandated national authorities.
A further question is whether the time that the bulletins were sent out, typically in the evening, was useful for those on the ground, or whether a different timeline would have been more efficient. While new forecasts are available from GloFAS each morning, it takes time to provide this data to a team working at another institution who need to run another model, and to write an expert interpretation and summary of the forecast information including new maps and figures, before incorporating feedback and clarifications. For high-level organisations such as DFID and UN OCHA, post-event feedback was that this timeline worked well, as they were able to look into the information at night, ready to incorporate into daily briefings that would be shared first thing in the morning. A potential issue with this, however, is that the forecast models are updated again shortly after these briefings are circulated in the morning, meaning that information is potentially out-of-date within a short time after those on the ground are receiving it. Of course, this timeline and experience would look different in regions of the world where the time difference is more significant than in this particular case. Finally, one of the key challenges for many involved was that the bulletins were provided in English only, when many of the national actors require information in Portuguese.

Lessons learnt and recommendations
The information produced and provided through these emergency bulletins was shown to be valuable and useful for decision-making, but provision of this information by research scientists is not sustainable nor is it the best way to co-produce and disseminate information. It is imperative to find a way to ensure that such information can be relied upon ahead of future emergencies. One project working towards systematically producing forecast bulletins for a range of natural hazards in Europe is the Aristotle Consortium (http://aristotle.ingv.it/tiki-index.ph p), which produces emergency bulletins for the European Emergency Response Coordination Centre. At the global scale, ideally an organisation such as the World Meteorological Organisation (WMO), which was established to represent an authoritative voice for meteorological and hydrological hazards globally, would take a role in supporting collaborations that work towards the operational production and dissemination of such bulletins by national authorities, in collaboration with international centres.
In section 6.1, it was highlighted that a template for the bulletins should be agreed ahead of time, and institutions should work together to establish the level of detail required and the types of maps, tables and diagrams that are most useful. Additionally, it is important to note that terminology such as "fairly likely" or "likely" and the implied differences between these terms, may not be universally accepted and understood internationally; this should ideally be clarified and terminology agreed such that it will be correctly understood by all actors involved. This could be achieved through training and discussion workshops involving all actors, and standalone FAQ documents or guidance for interpreting the information, which could be available for decision-makers during an emergency situation. In an ideal situation, there would also be a systematised chain of communication in order to cascade the information from high-level organisations to decision-makers and local communities, in a much faster and more organised way, that would also make the process of obtaining feedback and communicating with different actors much clearer and more efficient.
Recommendation 4: Operationalise the co-production of forecast bulletins by the previously identified producers, and the dissemination of forecast information and bulletins to intermediaries and users, to ensure that the information can be relied upon during future events.
In terms of the forecasting information provided, based on feedback, we would revise the way in which some aspects were presented in the bulletins. Throughout the two cyclones, the bulletins were regularly improved based on feedback from DFID and other partners. This included adding labels to figures and maps highlighting key points, including maps of the rainfall forecasts that are used as input to GloFAS, and discussing some of the background information on why the forecast had changed and how the movement of the cyclone was impacting the flood risk. Based on feedback and discussions post-event, we would further recommend including in each updated bulletin a brief summary of changes relative to the previous bulletin. This would provide a way for decision-makers to rapidly update their understanding of the situation, and for the team producing the bulletins to explain why any major changes have occurred. It could also be potentially useful to provide a national overview of the information, such as to provide the probability of severe flooding and the number of people likely to be affected across the country, which may be key information for high-level decisionmakers.
A key knowledge gap identified during this process is in understanding who the users are and what information is required for each, alongside the best way to tailor the information to user needs and how to translate the forecasts into useful impact-focussed information. There is a desire to move towards impact-based forecasting, and indeed, future operationalisation of bulletins such as these should aim to incorporate this, and could consider the potential benefits or drawbacks of providing different bulletins tailored to different groups of users. Additionally, while rapid developments in automisation and artificial intelligence mean that it may in future be possible to generate bulletins such as these automatically and directly from forecasting centres, a key aspect of such bulletins is also the human element -the expert interpretation of the forecasts and the changing situation, and continuous dialogue between different institutions, decision-makers and other actors.
Recommendation 5: Provide forecast information that is tailored to the needs of the users: include impact-based forecasts, provide language translations, and engage in two-way communication between forecast producers and users to incorporate realtime information and respond to queries.

Challenges
From the perspective of decision-makers and actors making use of the bulletins, and indeed those tasked with synthesising and disseminating the information, it was found that some of the scientific method behind producing the bulletins could be complex to convey and understand. In addition, where communications infrastructure was impacted during the disaster, there were technological limitations in how much complex information could be visualised in the field. The team producing the bulletins, in a similar way, found that a key challenge was the need to reduce the complexity of the information without losing the nuance of the forecast limitations. This is a well understood issue in environmental forecasting but remains a frustrating challenge. The bulletins were produced making several assumptions regarding the required level of complexity and without exact knowledge of who the forecast users were, or an understanding of their level of existing forecasting knowledge. Furthermore, to make a binary decision based on probabilistic forecast information is a known challenge, and implementing decision-making while accounting for uncertainty, both known and unknown, can be complex. A recent study by Arnal et al. [72] makes several recommendations for successful transition to probabilistic forecasting and decision-making, based on interviews with flood forecasters at the UK's Environment Agency, but with wider significance and applicability. For example, the need to provide "appropriate and custom designed training to all key players", including clear guidance on how to make decisions using new products, and for "everyone using the forecast products and systems […] to have a say in how the system will look and function, through a mutual design strategy", as any new system which does not reflect the complex decision-making landscape, may be mis-or under-used.
Our partners also highlighted that while the bulletins did provide some impact-based forecast information, the bulletins were not impactfocussed enough, and the population exposure information presented was challenging to interpret and overly precise. The GloFAS flood exceedance probability thresholds were also arbitrarily selected, potentially failing to identify at-risk zones that have a low flood forecast probability. There were some substantial challenges in conveying upcoming flood risk with the bulletins because there was no coverage of coastal or flash flooding, which is a key limitation when providing hazard and risk information for tropical cyclones. A lack of data available for both calibration and validation of forecast models [73,74] was also problematic. Access to more observation data would allow the models to be evaluated more thoroughly, but there are often barriers to accessing data, and across large parts of the world, data collection is scarce and data records contain significant gaps.

Lessons learnt and recommendations
When using a complex chain of environmental models to provide probabilistic information about upcoming flood hazard and risk, understanding who all the end users actually are, their level of knowledge regarding such forecasts, and what information is required on the ground for decision-making, is essential. The partners highlighted that it would be desirable to include a short dedicated 'impact summary' in the bulletins. This could provide information on key infrastructure in the region that may be at risk or may impact the response, such as hospitals and key transport routes. An improved method is also needed to communicate population exposure; one suggestion is that each person could be weighted by their probability of experiencing a flood threshold exceedance, favouring areas with a higher GloFAS probability of exceedance when the exposure is summed across localities. An alternative visualisation using this method is presented in Fig. 9, which gives a ranking of the most exposed localities and the range of exposure for multiple flood hazard return periods. Another suggested improvement could be to display multiple flood inundation return periods, alongside the GloFAS probability of exceedance. Beyond this, in order to better understand how reliable (and trustworthy) the forecast information provided is, information on how the forecast models typically perform in the affected region could be provided, alongside remotely sensed data of flood extent in near-real time, which would provide invaluable information on how the events were unfolding, and allow a real-time preliminary evaluation of how well the forecasts were predicting the event.
Recommendation 6: Share knowledge and understanding surrounding forecast uncertainty, quality and limitations, for all relevant actors, for example through training workshops and guidance documents.
The provision of better information in emergency bulletins relies upon the greater availability of data which is a major barrier to forecasting activities. Observational infrastructure networks need to be supported and their value appreciated. In addition, the need for better impact information provides an impetus to improve routine data collection on key infrastructure, exposure and vulnerability, which requires greater coordination between ministries and other national and local authorities alongside international organisations.
Recommendation 7: Greater coordination between ministries and other national and local authorities alongside international organisations is essential, particularly in order to improve routine data collection that would ensure the best emergency decisions can be taken using accurate and up-to-date information.
Currently, neither GloFAS or the flood inundation model are able to account for coastal or flash flooding, which is a key limitation. While these limitations were communicated in the bulletins, in future we hope to have forecast models that are able to forecast compound flooding from combined rainfall and coastal effects [75][76][77]. In addition, the shift towards whole Earth System modelling and forecasting means that in the next decade we should start to see multi-hazard forecasts available including all types of flooding, wind and other hazards such as landslides.
Recommendation 8: In the long-term, a move towards an Earth System approach to forecasting would allow a holistic inclusion of the relevant flood hazard and risk information from all sources of flooding (riverine, pluvial and storm surge), combined with other hazards arising from tropical cyclones such as wind damage, landslides and thunderstorms.

Conclusions
In order to take effective decisions, humanitarian and civil protection agencies need appropriate information on upcoming flood hazards and risk. In this paper we have critically evaluated the collaborative production of emergency bulletins for Cyclones Idai and Kenneth in Mozambique in support of the international humanitarian community. These were produced using global river flood forecasts from the Copernicus Emergency Management Service's Global Flood Awareness System (GloFAS), together with flood inundation modelling and impact risk assessment for population exposure estimates.
The provision of real time hazard and risk information in this way has provided a technically successful proof of concept with a positive real-world impact: information on different components of flood hazard and risk were integrated, provided in real time and informed decisionmaking. There is evidence that the bulletins supported critical actions such as sending an assessment team to the region most likely to be affected and considering the availability of hygiene kits, water treatment kits and tarpaulins ahead of the response.
The forecast information provided in the bulletins was evaluated and the interaction between the different components of the forecast chain discussed. While it is possible to predict the track of a tropical cyclone with relative certainty, the path of Idai was much more challenging to predict than that of Kenneth. Despite this, feedback from partners indicated that the uncertainty in Kenneth's track and associated flood risk in the earlier forecasts, which showed that the cyclone may make landfall in Tanzania, posed further challenges associated with a potential transboundary response. Evaluating the flood hazard and exposed population was challenging based on the data available, as post-event reports often indicate the total number of people affected by a wide range of impacts, including extreme wind, food insecurity and disease, while the exposure estimates provided in the bulletins were for river flooding only. While the evaluation indicated that the districts at greatest risk of flooding were successfully identified, increased collaboration with in-country partners could facilitate the provision of improved risk information, through access to more detailed data and information on population and infrastructure, that could better support decision-making.
So are we making the best use of the best science for humanitarian emergencies and resilience? There is a great potential value in using global operational forecasting models (such as GloFAS) for supporting the international humanitarian community to take actions for TCs in south-east Africa and elsewhere in the world. There is clear scope for Fig. 9. Top 20 most exposed localities for Cyclone Idai on 16/03/2019. The blue circles indicate the exposure to the 20-year return period (5% AEP) flood inundation and red the exposure to the most extreme 1000-year (0.1% AEP) flood inundation. The percentage of the total locality population exposed to flooding is also shown. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.) improving the provision of bulletins such as these -tailoring the information and making it clearer and more concise. However there is a clear need to not only work more closely with the mandated national authorities responsible for disseminating forecast and warning information, to improve the two-way sharing of information that would have the most impact on the ground, but also to support capacity development for a national, operational "End-to-End Multi-Hazard Early Warning System in the context of disaster risk management" [14]. In the interim, forecast information produced by international organisations can be a useful tool to support anticipatory action and complement current forecasting capacity from national authorities, but must navigate the "murky landscape" of national and international mandates, and capacities and collaborations for forecasting, early warning and anticipatory action. To do this successfully, much closer collaboration between international organisations and national authorities, forecast producers and forecast users, is essential. Section 6 outlined key recommendations for the future production of forecast bulletins, focussing on co-production, operationalisation and communication.
In order to be truly impactful, forecast information must not only inform the decisions taken rather than distracting from key actions, but must be co-produced in socially and institutionally embedded ways from the very beginning. The flood bulletins discussed in this paper were produced by University researchers responding to a request for help; responsively building a collaboration in order to provide the international humanitarian community with real-time information that wasn't already available. So how can this methodology be operationalised so that users can begin to rely on this information, and to trust the scientific method and indeed those scientists and institutions providing the information? Certainly University researchers do not have the required 24/7 operational capabilities to reliably produce such forecasts for every extreme event. This work has shown the technical requirements for producing such information, but the processes for producing these bulletins should be developed in a way that provides international cooperation to complement existing capacity (in line with the Sendai Framework [5]), while working towards the goal of building existing capacity of the mandated national authorities in a sustainable way [10,71,78,79].
Forecast producers and those using early warnings need to spend more time together understanding each other and move beyond the 'loading-dock approach' of science towards genuine co-production that counters the idea of technocratic solutions in which scientists should be isolated from decision makers [10,78]. To better support early humanitarian action with the best science and better integrate forecast and impact information produced both nationally and internationally, there is a clear requirement for embedded collaboration between forecast producers (national, regional and international, mandated and research institutes), intermediaries (governments, media, NGOs and research institutes) and users (governments, humanitarians, citizens, private sector, local leaders) [71]. Having a One Voice Principal for early warnings of natural hazards is important, but as of now the question remains: who will take responsibility for delivering reliable, tailored and comprehensive information that integrates all relevant aspects of forecast, risk and impact information? Who should?

Data availability
Real-time GloFAS forecast products are freely available at www.gl obalfloods.eu, and GloFAS data can be obtained through the dedicated GloFAS data service (https://confluence.ecmwf.int/display/COPSRV/ 04.+GloFAS+services). ECMWF ENS forecast data are available through the TIGGE archive after a short delay (https://www.ecmwf. int/en/research/projects/tigge), and HRES data can be accessed through ECMWF's Meteorological Archival and Retrieval System (MARS), subject to licensing. The IMERG satellite rainfall data can be downloaded from NASA (https://pmm.nasa.gov/data-access/downloa ds/gpm) and the observed tropical cyclone data from IBTrACS (https ://www.ncdc.noaa.gov/ibtracs/). The LISFLOOD-FP code is freely available from http://www.bristol.ac.uk/geography/research/hydrolo gy/models/lisflood/, and the global flood inundation maps used in this study are available upon request from Dr Jeffrey Neal or Dr Laurence Hawker.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.