Accident Damage Analysis Module (ADAM): Novel European Commission tool for consequence assessment—Scientific evaluation of performance

This paper summarises the scientific evaluation of the performance of a novel modelling tool, the Accident Damage Assessment Module (ADAM), developed by the Joint Research Centre (JRC) of the European Commission (EC) to assess the consequences of an industrial accident resulting from an unintended release of a dangerous substance. The ADAM tool is specifically intended to assist Competent Authorities in the European Union (EU) and European Economic Area (EEA), and their supporting research institutions, responsible for the implementation of the Seveso Directive in their countries, as well as governmental and research organisations of EU Accession and Candidate Countries, and European Neighbourhood Policy countries involved in chemical accident prevention and preparedness. In particular, the tool provides decision-making support to various functions associated with industrial risk management, enforcement and oversight, including risk analysis, land-use and emergency planning, inspection and monitoring, and the preparation and review of safety reports. Consequence assessment models are characterised by high level of complexity and of uncertainty. It is therefore of paramount importance to assess their limits. The scientific evaluation was conducted across the entire consequence assessment cycle, including the source terms and the physical effects calculations associated with the concentration toxics after airborne dispersion, the thermal radiation of chemical fires, and the explosion of vapour flammable clouds. The evaluation described in this paper was conducted on a series of relevant scenarios, by benchmarking the outcome of ADAM with the results obtained by similar software tools and with the experimental data obtained on a series of reference field campaigns, as taken from the literature. © 2019 The Authors. Published by Elsevier B.V. on behalf of Institution of Chemical Engineers. This is artic an open access


Introduction
In the European Union (EU), establishments that process, handle or store such chemicals in sufficient volume to have accident potential are covered under the Seveso Directive (2012/18/EU). The European Commission (EC) Joint Research Centre has developed the Accident Damage Analysis Module (ADAM) for consequence assessment, to assist EU Member States, Accession and Candidate Countries in implementation of the Seveso Directive (Wood et al., 2008) as well as a support to Neighbourhood Policy Countries involved in chemical accident prevention and preparedness related activities. ADAM became an official tool of the European Commission via the Commission Decision of 28/09/2017. It has been designed to support safety authorities in applying consequence and risk assessment for chemical accident prevention as well as to make decisions on land-use planning, emergency planning, and to assess site risk management during inspection.
Jointly funded by the EC Joint Research Centre (JRC) and the EC Directorate General on EU Humanitarian Aid and Civil Protection (DG ECHO), the ADAM tool is intended to provide competent authorities easy access to the complex technical aspects required to assess potential severity of fires, explosions and toxic releases on major chemical hazard sites, to improve risk management and protection of workers, communities and the environment from such incidents, consistent with the requirements of the Seveso Directive. The tool is also intended to help accelerate capacity of Enlargement and Neighbourhood countries and other third countries to align their national programmes with the Seveso Directive. The ADAM tool became ready for launching early this year and is being gradually deployed to countries who have expressed interest. More specifically, ADAM makes use of models that simulate all possible developments of an unintended release of a hazardous substance from the loss of containment to the final hazardous events, and estimate the physical effects produced by these events. These models cover the different aspects of consequence assessment, by including the unintended release, dispersion, fire and explosion of hazardous substances. The understanding of the performance of the predictive models is essential for the decision making process and a proper evaluation of these models is necessary to demonstrate their credibility and their limits of applicability. In general, the terms verification and validation (V&V) are currently used for this purpose, where the former aims at demonstrating that the tool accurately describes the model as designed whilst the second refers to the accurate correspondence of model prediction and observations. However, as also addressed by Chang and Hanna (2004) the random nature of the accident consequences leads to a certain irreducible inherent uncertainty, which implies "models can only be confirmed or evaluated by the demonstration of good agreement between several set of observations and predictions". In this paper, we refer to this activity as the "evaluation" (or sometimes the "scientific evaluation" to make a distinction from other types of evaluations that might be applied to a software tool, for example, by users).
Such an evaluation is not very straightforward. The main difficulty is associated with the need to establish whether the measurement data of the experimental trials used to evaluate model performance are accurate enough. In some circumstances, it would even be incorrect to assume that a perfectly accurate model would reproduce measured data. For instance, some fluctuations in the environmental conditions occurring during the field tests might influence significantly the test results and cannot be accounted for in the simulations. For this reason, instead of comparing each single prediction with the corresponding observation, a possible way forward to approach the problem is to group the observations and predictions according to a certain criterion, and then to compare the averaged results. In this respect, a model can be only confirmed or evaluated by showing the good agreement between some set of observation data and predictions.
The present paper summarises the evaluation conducted on the overall consequence assessment cycle of ADAM, from the critical event, which consists of the unintended release of the dangerous substance (ADAM Module 1) to the physical effects associated with such a release, by including the airborne dispersions of toxics, fires, and vapour cloud explosions (ADAM Module 2). This evaluation was conducted on a series of relevant scenarios, by benchmarking the outcome of ADAM with the results obtained by similar software tools and with the experimental data obtained on a series of reference field campaigns, as taken from the literature.
More specific information on the technical aspects of ADAM and on validation of its models are reported elsewhere (Fabbri et al., 2017;Fabbri et al., 2018).

Evaluation methodology
Different evaluation methodologies have been suggested for atmospheric dispersion modelling, which provide a quantitative and objective means of comparing observed and predicted physical parameters. Much less exists in other areas of consequence modelling such as methodologies predicting the effects of fires and explosions as well as phenomena derived from the source term (Coldrick, 2017). Amongst the existing evaluation methodologies, the method described in detail by Chang and Hanna (2004) was used for the evaluation of ADAM. The method is based on a range of comprehensive statistical performance measures, including the fractional bias, the normalised mean square error, the geometric mean, the geometric variance, the correlation coefficient, and the fraction of prediction within a factor of two of observations (see Table 1). Although this methodology was developed to evaluate the performance of dispersion modelling, as explained by its authors, the statistical performance measures that they use are generic enough to be applied to other contexts as long as predictions and observations are paired (Chang and Hanna, 2004). These performance measures have been therefore applied to models implemented in ADAM, i.e., for physical effects (fires and explosions), and source term related phenomena (rainout and pool evaporation).
In order to visualise the overall performance behaviour, the geometric variance VG is often graphed as a function of the geometrical mean bias MG (alternatively NMSE vs FB). As a comparison, a parabolic line that represents the minimum possible value of the geometric variance (VG) for a certain value of the geometrical mean bias (MG) is reported. A model that "perfectly" matches with observations would be placed at the vertex of the parabola (i.e., at the 1,1 point in the case of VG vs MG and at 0,0 in the case of NMSE vs FB).
When necessary to establish the confidence intervals on the different performance measures, in order to assess the significance of model differences, a Bootstrap resampling technique was employed (Efron, 1987). This procedure essentially involves random sampling from the original data set (i.e., observations and prediction pairs) with replacement from the original sample. The purpose is to generate a larger number of new sample sets of the same size as the original data set. This approach is normally suggested since the above parameters are not easily transformed by standard procedures to a normal distribution. In the present case, the Bootstrap method was applied by resampling 10,000 estimates to determine the 95% confidence intervals for each performance level.

Source term (ADAM Module 1)
The first module of ADAM refers to the implementation of all models necessary to characterise the accident source term, consisting of the estimation of the amount of the hazardous substance involved in the release process, including the release dynamic, and all the release parameters including those associated with the postexpansion process. This estimate requires the knowledge of the type of substance involved in the accident, the storage properties (i.e., quantity, temperature, and pressure), the type and mode of rupture, and the release time.
Most of the source terms models are quite well established and are somehow uniform amongst the different tools for consequence assessment. In these cases, the ADAM model implementation was simply verified by benchmarking the results obtained on a series of representative accident scenarios with those obtained using similar software tools (i.e., PHAST of DNV and EFFECTS of TNO). For the more complex evaluation of post-expansion phenomena typical of pressurised-liquefied substances, such as droplets' formation and rainout, a quantitative evaluation was conducted by comparing the ADAM outcome with the experimental observations available in the literature, and using the statistical performance measures as described in the previous section. It should be highlighted that all simulations conducted using other software tools for benchmarking purposes were always performed by the JRC independent of the software developers.
Every loss of containment is characterised by the type and properties of the contained substance, the containment geometry, and the rupture type. Therefore, benchmarking of the different models within ADAM requires an approach that addresses all the different ways that an accident can vary. In particular, by combining the initial state of the substance (i.e., compressed gas, non-boiling liquid, and liquefied-pressurised gas) and the three different types of damage mechanisms (i.e., failure from vessel hole, from pipe connected The Fractional Bias (FB) is a measure of mean bias and indicates systematic errors, which allows assessing whether the model underestimates or overestimates the measured values. FB is based on a linear scale and the systematic bias refers to the arithmetic difference between Cp and Co. Geometrical mean Bias (MG) MG = exp(ln (Co) −ln (Cp)) The Geometrical mean Bias (MG) is also a measure of mean bias and indicates systematic errors, but differently from FB that is based on a linear scale is based on a logarithmic scale. Its use is normally preferred in dispersion related applications because of the wide range of magnitudes involved.
Normalised Mean Square Error (NMSE) The Normalised Mean Square Error (NMSE) is a measure of the overall scatter around the true value and accounts for unpredictable fluctuations. It reflects both systematic and unsystematic (random) errors.
The Geometrical Variance (VG) is, analogous to the NMSE, a measure of the overall scatter around the true value. It is based on a logarithmic scale and its use is normally preferred in dispersion related applications because of the wide range of magnitudes involved.
The correlation coefficient (R) reflects the linear relationship between two variables. It is insensitive to either an additive or a multiplicative factor. A perfect correlation coefficient is only a necessary, but not sufficient condition for accuracy. Fraction of Predictions within a factor-of-two (FAC2) FAC2 = fraction of data that satisfies 0.5 ≤ Cp Co ≤ 2 The fraction of predictions within a factor of two of observations (FAC2) is the most robust measure, because it is not overly influenced by high and low outliers.
Cp values predicted by the models.
Co observations from the experimental trials. overbar stands for the average over the dataset.
C standard deviation over the dataset. to a vessel, and catastrophic rupture of a vessel), there are nine different release categories to evaluate. The present paper shows the results of benchmarking on selected cases, whilst the benchmarking conducted on all the different release categories is reported elsewhere (Fabbri et al., 2018).

Source term
For compressed gases, Fig. 1 reports the flow rate dynamic in the case of a release of ethylene from a 25 mm hole in a 0.5 m 3 'bullet' type tank (i.e. cylindrical with hemiheads), where the compressed gas is stored at an absolute pressure of 250 psi at a temperature of −25 • F. The comparison shows a good agreement between ADAM and PHAST 7.22. The curve calculated by EFFECTS 5.5 is less conservative and matches with the curve calculated by ADAM using the ideal gas assumption. The same release type was also calculated by varying the hole diameter as shown in Fig. 2.
The release of non-boiling liquid can be simulated by assuming that the tank is either 'sealed' or atmospheric. In the first case, a depressurization phenomenon will take place within the vessel during outflow, which will last until the internal pressure added to the hydrostatic weight equals the external pressure. This leads to an oscillating behaviour of the flow rate resulting in a gurgling outflow. Both PHAST 7.22 and ADAM model the depressurization of a non-boiling liquid using both assumptions. On the contrary, EFFECTS uses the atmospheric assumption. In order to make the comparison, ADAM was therefore run twice, by using both assumptions. The result for a release of benzene stored at a temperature of 300 K in a horizontal tank (10 m height, 5 m diameter, 70% filling) from a 100 mm hole located at 1 m from the tank bottom is given in Fig. 3. In both cases, ADAM reproduces quite well the results of the other tools, even if in the 'sealed' tank case, the outflow provided by PHAST, which was obtained by setting the 'vacuum relief valve' with the 'not operating' option, is somehow more erratic.
The case of releases of pressurised-liquefied gases are more complex if compared to the previous ones. The sudden depressurisation after exit produces the jet flash and leads to a two-phase release. This process is characterised by droplets' formation and subsequent rainout of a part thereof, which might alter significantly the cloud dispersion phenomenon. The estimate of the initial flow rate calculated with ADAM was compared with the simulation conducted with PHAST on a couple of reference scenarios, which are described  by Kukkonen (1990). The first consisted of a release from the fullbore rupture of a pipe connected to a vessel at a distance of L =2 m from the pipe inlet (pipe roughness =0.05 mm; pipe diameter D =40 mm; released substances: chlorine, propane, ammonia, sulphur dioxide, hydrogen fluoride). The second reference scenario is the release from the full-bore rupture of a pipe connected to a vessel at variable distances from the pipe inlet (pipe roughness =0.05 mm; pipe diameters D =40 mm and D =20 mm; storage temperature 15 degrees Celsius; storage pressure at saturation; released substance: chlorine). The results of this benchmarking are given in Fig. 4 and Fig. 5, which show a good agreement between ADAM and PHAST in all involved cases. Differently from the case presented by Kukkonen (1990), who proposed a coefficient of contraction of Cd = 0.5 to account of the pipe inlet losses, the calculation herewith presented was conducted by assuming no losses at the pipe inlet, as it was not possible to input this information in the short-pipe model of PHAST.
The droplet size estimation was evaluated by comparing the tool predictions with the observations of a series of scaled experiments on different substances (i.e., water, gasoline, cyclohexane, n-butane, propane) as conducted at Cardiff University . Amongst the different methods implemented in ADAM for the calculus of the initial droplet size, the Phase III JIP SMD correlation was used . The results are shown in Fig. 6, where the predicted values of the initial SMD (i.e., Sauter Comparison between ADAM and PHAST on the release from pipe connected to a vessel containing chlorine. Flow rate vs. pipe length at rupture for two pipe diameters (i.e. 20 mm and 40 mm). Pipe roughness =0.05 mm, no losses inside pipe. Mean Diameter) are given as a function of the observed values. For a comparison, the predicted values calculated by Witlox et al. (2010), which were obtained by using the same correlation, are also reported. The results are very similar. A more quantitative analysis was conducted by applying the statistical performance measures described in Section 2, and are shown in Table 2. The rainout calculus module was evaluated against the experimental tests conducted by the American Institute of Chemical Engineers, Center for Chemical Process Safety (CCPS), which were executed in two phases and on several substances (i.e., water, CFC-11, methylamine, chlorine, and cyclohexane) (Johnson and Woodward, 1999). The rainout ADAM model was applied by using the modified CCPS correlation for evaluating the initial droplet size and the ADAM procedure to calculate the final jet vapour fraction. This is based on the selection of the proper droplets distribution function and calculating the evaporation during the droplets flight (see Annex I and Fabbri et al., 2017). An example of the comparison of the different rainout calculations against the CCPS chlorine data is shown in Fig. 7. The different curves are obtained by using the different droplets' distributions that can be selected in the ADAM option menu. In particular, the solid line was obtained by using the log normal distribution with the value for the geometric spread equal to 1.4, as suggested by Woodward (2014). The other curves were calculated by using the Rosin-Rammler distribution, with fixed parameters as suggested by Elkotb (1982) or calculated by using the Kay et al. (2010) procedure. For comparison, the rainout prediction with the Lautkaski correlation, that does not require the full calculation, but can be selected in the ADAM option menu, was also reported (Lautkaski, 2008).
In order to assess the performance of the different droplets' distributions used in the rainout calculus module, the statistical measures described in Section 2 were applied on the overall set of alternative distributions implemented in ADAM. The result is given   in Table 3, which shows that the log normal distribution performs the best (higher FAC2 lower FB and NMSE, overall), and for this reason is addressed as the recommended distribution in ADAM for the calculus of the rainout.
To depict the different performance behaviours, the graph reporting the geometric variance VG as a function of the geometrical mean bias MG is often reported (Chang and Hanna, 2004). This is shown in Fig. 8 for the current case. The 95% confidence intervals on MG were calculated using the Bootstrap technique that indicates that differences in performance across the different distributions are significant.

Pool spreading and vaporisation
The recommended model for pool spreading and vaporisation models implemented in ADAM is GASP (Gas Accumulation over Spreading) developed at the UKAEA (nowadays ESR Technology) on behalf of the UK Health and Safety Executive (HSE) (Webber, 1990). This model is based on the numerical solution of a series of coupled equations for the integral properties of the pool and is applied to both continuous and catastrophic releases. The spreading model is based on the analytical solution of the shallow water equations generalised to include the effect of friction in laminar and turbulent conditions, which was introduced to overcome the negative inertia limitation typical of similar integral models. The detailed set of coupled equations providing the pool radius are reported by Webber (1990). The expression providing the evaporation/vaporisation rate per unit surface from a plane liquid surface into a turbulent boundary layer is given by Brighton's expression (Brighton, 1987).
For completeness, ADAM implemented also alternative models, which are widely in use for the determination of pool spreading and vaporisation. For pool spreading, a method introduced by several authors and recently described in detail by Raj (2011) was selected. For vaporisation, the alternative model to GASP implemented in ADAM is given by the well-known Mackay and Matsugu formula (MacKay and Matsugu, 1973).
The above models have already been the object of validation by their developers, however, since their implementation is characterised by a certain level of complexity, a separate evaluation was conducted. In particular, some selected test cases emphasising either the spreading or the vaporisation mechanisms were chosen. In addition, separate evaluations were conducted for pool spreading on land and on water, since the models involved are quite different.
For the case of spreading on land, the evaluation was conducted using the tests of Belore and McBean (1986), which consisted of the release of water over a plywood surface. Due to the very low vaporisation rate, vaporisation is negligible and it is possible to decouple the spreading mechanism from vaporisation in the considered time lapse. The input data used for the simulations and the results on the three tests (i.e., 29, 29 and 30) are reported elsewhere (Fabbri et al.,  2018). These results show that the GASP model implemented in ADAM performs quite well for both tests 28 and test 29, whilst it tends to overpredict the pool radius for test 30. By contrast, the Raj model implemented in ADAM overpredicts for all the selected tests. This is in agreement with the deductions of Webber who clearly emphasised that the Raj equation rightly "expresses the resistance effect of displaced water, and has absolutely no justification for pools spreading on land" (Webber, 2012). Fig. 9 shows the result obtained for test n. 29.
For pools spreading on water, the reference tests were those provided by Dodge et al. (1983), which involved both instantaneous and continuous releases of a series of volatile and non-volatile hydrocarbons. Due to the considerable number of trials, it was possible to determine the statistical performance measures described in Section 2 for the whole series of tests, and the results are reported in Table 4. Differently for the case of spreading on land, the Raj model implemented in ADAM performs slightly better than GASP, since the performance measures are significantly different within their 95% confidence interval. As a general result, the ADAM version of GASP is recommended for pool spreading on land whilst the Raj variant is recommended in case of pool spreading on water.
For evaluating the pool vaporisation module on land, the trials conducted by Kawamura and Mackay (1987) were selected. These consisted of releases of seven volatile substances in bund, in which the spreading mechanism has no relevance (i.e., with a complete pool formation occurring in fractions of second). Details of the input data use for the simulation as well the procedure used to calculate evaporation rates per unit surface averaged over the whole experiment duration are reported elsewhere (Fabbri et al., 2018). The result of this simulation is depicted in Fig. 10, which provides a comparison between predicted and experimental data. The results obtained by using ADAM GASP and ADAM MacKay & Matsugu models are also compared to the values obtained by Fernandez (2013) using the PVAP-MC model, which is one of the implemented models in PHAST. Overall, using the Brighton value for the pool roughness length (i.e., 0.00023 m) (Brighton, 1987), the performance of ADAM GASP is very good. (This value is the recommended value in ADAM).
For pools on water, the vaporisation module was evaluated against the data of the U.S. Bureau of Mines (Burgess et al., 1972), that referred to several tests involving instantaneous releases of LNG, liquid nitrogen and liquid methane. The number of tests and the measured averaged evaporation rates (calculated during the first 20 s after the release), together with the input parameters used for the simulation are given in Table 5. This table reports also the average vaporisation rates as calculated with ADAM GASP under the two different assumptions i.e., in presence of a film boiling between the pool and the water substrate, or not. ADAM GASP reproduces quite well the experimental results for the LNG and Methane trials without the film boiling assumption (i.e. with a devi-  (Burgess et al., 1972) ation of ca. 5%), with the exception of the case of liquid nitrogen for which the film boiling assumption seems much more appropriate.

Physical effects (ADAM module 2)
The second calculus module of ADAM estimates the physical effects resulting from the accidental development following the loss of containment. As such, they depend on the hazardous event involved, and in the Seveso context, they can be summarised as follows: • concentration of a toxic after airborne dispersion; • thermal radiation of chemical fires; • overpressure/impulse of the vapour flammable cloud explosion.
This calculation is normally influenced by the atmospheric conditions (i.e., air temperature, air stability, wind speed) and by other parameters, such as the average time for vapour dispersions or the ignition time or ignition location for flash fires and vapour cloud explosions. Since the models implemented in the calculation are strongly dependent on the modelling assumptions, all cases were evaluated by comparing the model output with the data of reference experimental trials.

Atmospheric dispersion
ADAM uses an in-house modified version of the SLAB model for modelling atmospheric dispersion of toxic and flammable clouds. SLAB was developed by the Lawrence Livermore National Labo-ratory. This model is based on the solution of spatially averaged conservation equations of mass, momentum, energy, and species, and produces spatially averaged cloud properties via the calculus of similarity function coefficients as a function of the downwind distance (Ermak, 1990).
Before introducing new methodological aspects, the original SLAB code was replicated in C++, and restructured in a more efficient way. A first verification was conducted by comparing the software outcome with the original code on a very large number of reference scenarios and by varying all input parameters in a wide range, in order to make sure that the replicated code would be equivalent to the original. After this verification, some major methodological improvements were introduced into ADAM, as described in detail in Fabbri et al. (2017) and summarised in Appendix B. These modifications make the ADAM-SLAB software in some way different from the original SLAB. As such, it was necessary to carry out a new evaluation exercise of SLAB as modified and implemented in the ADAM tool. The evaluation was conducted against a recognised series of large-scale field experiments, that includes the following: • Burro& Coyote (LNG releases into a water basin) (Koopman et al., 1982a, b, (Goldwire et al., 1983a. • Desert Tortoise (ammonia horizontal jets) (Goldwire et al., 1983b). • Goldfish (hydrogen fluoride horizontal jets) (Blewitt et al., 1987).
The experimental concentrations measured through sensors positioned at different locations during the trials formed the observations' data set (Co) to be compared with the model predictions (Cp).
For the trials involving finite duration releases, sensor raw data, providing concentration vs time, were processed to obtain average values within the release time duration. For the comparison, time average simulations were conducted with ADAM at the same sensors' locations. It was preferred to consider the sensor mean concentrations rather than sensor maximum concentrations in order to make use of the overall sensor curve instead of relying on a single point, which might be more sensitive to stochastic deviations (e.g., outliers). Due to the inherent local fluctuations of the concentration caused by wind direction deviations and turbulence, the construction of the pairs set based on the single observation/prediction couples for each sensor position does not make very much sense. For this reason, as sensors were displaced into equidistance arcs, data were grouped to form pairs (observed and simulated data) for each separate arc. In such a way, the average concentration of sensors located on the same arc was coupled to the average of model calculations carried out for each single sensor location on the same arc.  The result of this comparison is given in Fig. 11 that shows the scatter plot of observation/prediction pairs together with the ideal behaviour (solid line) and the FAC2 curves (dashed lines).
For the Thorney Island instantaneous releases, the pairs were formed from the centre-line maximum concentrations, and the result of the comparison is given in Fig. 12, where the different trials are distinguished.
In order to quantify all the above results, the statistical performance measures defined in Section 2 were calculated for the different trials' set. Clearly, differently from Fig. 11 and Fig. 12 that provide the log-values of concentrations, the performance measures were calculated by starting from the real values of concentrations, as defined on Table 1. The outcome of this calculus is given on Table 6, which shows the general good performance of ADAM. In general, all performance measures are within the acceptability range, with the exception of the Goldfish trials, for which ADAM under predicted the concentration. For Coyote trials, the fractional bias and the geometrical mean bias were just above the acceptability threshold. The overall performance can also be represented in terms of the geometric variance, VG vs the geometrical mean bias, MG (Fig. 13). Each point representing each field cam-  . 13. Model performance indicators, geometrical mean bias MG and geometric variance VG for concentration prediction and observations. The horizontal lines on MG represent 95% confidence intervals; the solid parabola is the "minimum" VG curve. The vertical dashed lines represent the 'factor-of-two' between mean predictions and observations. paign is given with its MG 95% confidence interval as calculated by using the Bootstrap technique. Overall, the results show that the performance of ADAM is very good and in line with the results of similar consequence assessment tools (Witlox et al., 2018).

Flammable effects
This section deals with the evaluation of flammable effect models implemented in ADAM, which are all based on semi-empirical correlations. For fire-related effects, the flame geometry, the radiative properties and the calculation of the radiant flux at receptor location were not analysed separately, but were studied in terms of the overall final effect. Due to modifications introduced by ADAM, both pool and jet fire calculations differ from the original set of reference models and thus, a full evaluation of these calculations within ADAM was necessary. Specifically, these modifications included the property procedure for the calculus of the view factor, and an alternative expression to address the effects of atmospheric absorption (Fabbri et al., 2017). For fireballs, since the implemented model is very well established and simple to verify by hand calculations, its evaluation was not reported. Vapour cloud explosions in ADAM are based on a set of established scaled curves for peakoverpressure and blast duration (or equivalent positive impulse) that refer to different blast severities. The outcome of ADAM was verified against the original scaled curves and evaluated against a series of experimental trials.

Pool and jet fires
For pool fires, three different semi-empirical models are implemented in ADAM (i.e. TNO modified, Shokry and Beyler, and Mudan). The three models differ amongst each other for the use Fig. 14. Model performance indicators, for the pool fire trials. The alternative pool fire models included in ADAM were tested separately. The horizontal lines on MG represent 95% confidence intervals; the solid parabola is the minimum possible value of the geometric variance; the two vertical dashed lines indicate the 'factor-of-two' limit between predictions and observations. of different correlations for the flame length, the flame tilt, the drag diameter, and the surface emissive power. Details on these correlations are given in the related references and in the ADAM technical guidance (Fabbri et al., 2017). The evaluation was carried out against experimental trials on LNG and LPG reported by Johnson (1992) and by Welker and Cavin (1982), respectively. Separate simulations were conducted for the three different models implemented. Details on the input data and assumptions used for the simulations are reported elsewhere (Fabbri et al., 2018). As for the previous cases, the evaluation results are expressed in terms of the associated performance measures and the diagram of the geometric variance (VG) plotted against the geometrical mean bias (MB) that are reported in Table 7 and Fig. 14, respectively. On the basis of these results, the Modified TNO method performs best amongst the models implemented, and as such, it has been established as the recommended method in ADAM. To further illustrate the general trend, scatter plots of the observation/predictions pairs are also shown in Fig. 15, where ADAM simulations on the Modified TNO and Mudan methods are also compared with the simulations of Johnson (1992). For jet fires, ADAM implements the model proposed by Chamberlain (1987), with Johnson's variant for horizontal jets (Johnson et al., 1994). In this model, the flame shape is represented as a tilted frustum of cone, radiating as a solid body with a uniform surface emissive power. Since the Chamberlain model was developed for gaseous substances, in order to extend its validity to two-phase releases (e.g., LPG), Cook's correction was applied (Cook et al., 1990). The detailed expression of the parameters that fully characterise the jet flame i.e., the flame lift-off distance from the  Johnson (1992). Scatter plot of observation/prediction pairs. orifice, the flame length, the frustum length, the width of the of the frustum bases, the deflection angle of the flame, and the proprietary procedure developed for calculating the view factor are reported in the ADAM Technical Guidance (Fabbri et al., 2017). The evaluation was carried out against experimental trials involving natural gas releases (Chamberlain, 1987;Johnson et al., 1994). Specifically, for vertical jets, one reasonable source of test data is reported by Chamberlain (1987), which contains data from large-scale trials carried out at onshore oil installation in Cumbria (Trial n.4). For horizontal jets, the field trials carried out at the British Gas test site reported in the paper by Johnson et al. (1994) were used. These involved four different tests involving natural gas horizontal jets (B, C, D, and E), originated from different orifice diameters and under different operative and environmental conditions. Again, information on the input data and assumptions used for the simulations are reported in detail elsewhere (Fabbri et al., 2018). Fig. 16 and Fig. 17 show the comparison of thermal radiation data predicted by ADAM vs the observed values for vertical and horizontal jet fire trials, respectively. As a reference, the original  Chamberlain (1987). Scatter plot of observation/prediction pairs for the different tests (i.e. A, B, and C).  Johnson et al. (1994). Scatter plot of observation/prediction pairs for the different Tests (i.e. B, C, D, and E).  Chamberlain (1987) and Johnson et al. (1994) were also included in the graphs. The overall evaluation is reported in Table 8, in which the whole set of statistical performance measures is provided. Similarly to the case of pool fires, ADAM shows good performance.

Vapour cloud explosions
The evaluation of the vapour cloud explosion module of ADAM was conducted on the two alternative models implemented in ADAM i.e., the Multi Energy (ME) and the Baker-Strehlow-Tang (BST) methods (Fabbri et al., 2017). For the ME method, the ten scaled curves of the peak overpressure and of the positive phase duration, that correspond to the 10 blast strength levels, were extracted from the original graph produced by the model developers (Mercx and van den Berg, 2005). Each scaled curve was digitised by producing 1000 points and uploaded on the ADAM code. The same was done for the 9-scaled-curves of the peak overpressure and positive impulse of the BTS method, which were extracted from the original work of the model developers (Tang and Baker, 2000). Since the available 9-scaled curves are associated to flame speeds that do not cover all Mach values of Pierorazio's table (Pierorazio et al., 2005), intermediate curves were created by using the spline interpolation method. All scaled curves uploaded in ADAM were carefully verified against the original graphs by superimposition.
The evaluation was completed against the BFETs and EMERGE experiments and on the Shell Deer Park accident case by replicating the detailed comparison of the ME and BST simulations with the reference data conducted by Fitzgerald (2001). Details on the comparisons of results from individual tests and on the input data used for the simulation are reported in Fabbri et al. (2018).
The outcome of the simulations on BFETS tests performed with ADAM by using the BST and ME methods are shown in Fig. 18 and Fig. 19, respectively. These simulations are also compared to those carried out by Fitzgerald (2001) on the same reference tests. In general, ADAM calculations confirm Fitzgerald's results, that is, the BST tends to underestimate the BFETS data, whilst overpressure estimated by using the ME method was found to be more accurate. A clear improvement of the BST performance was indicated when the method of Xu et al. (2009) was applied. This result provides the ground correction factor implemented in PHAST. For this reason, this correction was also implemented in ADAM as an option that can be selected in the ADAM menu.
Concerning EMERGE experiments, similarly to the conclusions of Fitzgerald's paper, all small and medium scale tests were considered not realistic for industrial scenarios. Fig. 20 shows the overpressure simulation as a function of the distance from the edge of the congestion zone for EMERGE Tests L1, 2 (methane, high congestion. Confinement 3D). As for the previous case, the ME method provides the best estimate.
Finally, Fig. 21 shows the simulation and comparison with the observed overpressure data resulting from a large VCE of an accident case occurred at the Shell Company plant in Deer Park, Texas, for which far field overpressures were estimated, based on different observed damage levels. The simulation shows that both ME and BSI methods performed similarly for this case.  A point to note is that the estimate of overpressures in vapour cloud explosions by using BST and ME methods are generally oversimplified especially in the near field, due to the presence of obstacles and the challenge of replicating site geometries in a test environment. The difficulty arises from the presence of varying types of obstructions (e.g., walls, pipes, vessels and other equipment) that the vapour release might encounter. For each obstruction, its particular dimensions, as well the dimensions of the open space around it (based on the distance(s) from other obstructions in the vicinity or the source of the release) can influence the development of the vapour cloud, including its density and dispersion qualities but are not accounted for by the integral models used for the simulation. This is one of the main reasons of typical discrepancies between the simulations conducted using ME and BST methods and the experimental tests.

Conclusions
The application of statistical performance measures for evaluating and comparing consequence analysis methodologies provided valuable support to evaluation of the development of the ADAM tool. In particular, this approach was important for validating recommendations in the ADAM tool for calculating physical effects and source term related phenomena.
The results of the validation efforts gave substantial support to recommended default options in the ADAM tool to users for calculation of physical effects and source term related phenomena. These recommendations are intended in particularly to aid users that may not feel expert enough to select their own calculation method. The validation exercises showed that ADAM's recommendations for calculating rain-out, pool vaporisation, atmospheric dispersion, flammable effects, pool and jet fires, and vapour cloud explosions perform well against other methodologies in replicating consequences using actual accident data.
It should be noted that these results do not suggest that other methodologies developed to calculate these effects should never be used. Indeed, all the methodologies have strengths and weaknesses for particular scenarios. For this reason, ADAM also includes the choice of other well-known and respected methodologies as options for expert users who are experienced in recognising situations where these methodologies are best applied. assess the substance response within the vessel during depressurisation (van den Bosch and Duijm, 2005). These equations are described in detail in the ADAM Technical Guidance report (Fabbri et al., 2017). The post-expansion parameters, typical of releases resulting from compressed gases or superheated substances (i.e., liquefied-pressurised substances) are obtained by assuming a 1D homogeneous jet (i.e., with no phase-slip) in a thermal equilibrium and resolving two alternatives set of conservation equations i.e. mass/momentum/energy or mass/momentum/entropy (Britter, 1994(Britter, , 1995. Flash velocity is estimated by combining mass and momentum conservation equations, whilst the vapour quality after flash is obtained via the energy conservation equation in the first case, or via the isoentropic condition in the alternative case (Fabbri et al., 2017). Despite of the attempts to establish which of these two approaches is best (Britter, 1994), a definite conclusion has yet not been established. In ADAM, both approaches are used depending on the specific selected correlation for the estimate of droplets size as explained below. For compressed gases, ADAM always makes use of the isentropic assumption, since it seems to provide more reliable results (Witlox and Bowel, 2002).
After post-expansion, the liquid mass fraction of the jet will break into droplets by mechanical and/or flashing brake-up. The first is normally produced on subcooled liquids relative to ambient conditions, that are also under enough pressure as they share forces resulting from the velocity difference between liquid and air, the second is typically produced on superheated liquids during a depressurisation process. The understanding of droplet formation and entrainment in the vapour cloud is particularly important to assess their influence on the cloud dispersion behaviour. In addition, droplet size is directly associated with liquid mass fraction falling onto the ground after release (rainout), which is cause of reduction of airborne concentration and extension of duration of the dispersion phenomenon. The modelling of this process is rather complex, since the atomisation produces many different final droplet sizes that follow a certain distribution. In ADAM we use both the Log-Normal (Woodward, 2014) and the Rosin-Rammler  equations to model the distribution of droplet sizes in a cloud. These distributions are fully described by one or two adjustable parameters together with a characteristic droplet size. In the context of consequence assessment, this characteristic size is defined by the Sauter Mean Diameter (SMD or d p32) : where D is the diameter of droplets generated by the depressurisation. The choice on SMD with respect to other means is because droplet size distributions with the same SMD have the same volume to surface ratio, which is very important in the rainout process.
The different heuristic correlations, used by ADAM to determine the value of SMD for a determined release scenario, make use of the post expansion data, calculated by the source term module. They can be selected in the option menu and are briefly described in Table A1.
Once the SMD value is determined for the release under investigation, ADAM produces the estimate of the rainout, i.e., the total mass fraction that falls on the ground after release. This is conducted by considering the overall droplet distribution selected in the option menu (i.e. Log-Normal or Rosin-Rammler) with initial mean size equal to SMD and calculating the mass fraction of single droplets of the distribution reaching the ground, by combining the droplet motion equation (Holterman, 2003) with the equation describing rate of decrease of droplet diameter due to evaporation in air (Williamson and Threadgill, 1974). The resulting mass  Witlox et al., 2007Witlox et al., , 2010 fraction of the jet that remains airborne is given by the following expression: where p(D) is the density function of droplets' distribution, and x sd rain is the mass fraction of the droplet of diameter D that remains airborne. The overall procedure for this calculation is described in detail elsewhere (Fabbri et al., 2017). For completeness, ADAM incorporates also other approaches based on empirical correlations of de Vaull and King (1992) and Lautkaski (2008), which can be selected in the option menu in alternative to the full calculation.

Appendix B. Dispersion modelling. Modified version of SLAB
In ADAM, the SLAB algorithm has been rewritten with more efficient code, that increases significantly the number of points to produce consequence maps with better spatial resolution. All environmental data, which are coded and fixed in the original software, are taken directly from the ADAM database that correspond to the specific environmental conditions of the scenario under study. In addition to these modifications, ADAM introduces some major modelling improvements, and specifically: Fig. B1. Catastrophic releases in presence of rainout. The total mass released Q is given by the sum of a vapour, Q V = x f Q and a liquid, Q L = (1-x f )Q, phase. After rainout a pool is formed, Q pool = X L rain Q L , whilst part of the liquid droplets remain airborne Q aerosols = (1-X L rain ) Q L .
• alternative calculus of the average concentration for instantaneous releases; • calculus for time-varying releases; • inclusion of the contribution from pool evaporation in case of rainout; Alternative calculus of the average concentration for instantaneous releases The calculus of the time-average concentration is performed by using the following equation: where C is the instantaneous concentration, recalculated to account the meander effect, t av is the averaging time and t pk the time of peak concentration. This calculus is rather trivial for continuous or finite duration releases, but the full integration has to be done in the case of puff dispersion, typical of instantaneous releases. The original SLAB code simplified this calculus by performing a variable transformation from time to downwind distance via the velocity of the puff centre-of-mass, and resolving the above integral. However, and as demonstrated by Fabbri et al. (2017), this approximation has the inherent error that the peak concentration time at a certain location does not necessarily occur when the centre-of-mass of the puff is centred in such a location. Besides, the centre-of-mass of the puff will never be located for negative values of x, since it propagates downwind. Consequently, the estimated average concentration will be always zero backwind that is obviously not possible since a small part of the toxics will always diffuse backwind, for catastrophic releases. Overall, this approximation will tend to produce a correct average concentration only in the far field, a lower concentration in the near region, and complete wrong results both in proximity of the emitting source and backwind (Fabbri et al., 2017). For this reason, the SLAB approximation is not applied in ADAM, and the full integral is calculated using a special algorithm based on both Romberg and Gaussian integration methods to optimise the calculus time.

Calculus for time-varying releases
For scenarios involving finite duration releases (i.e. horizontal/vertical jets, and pool evaporation), SLAB is based on the assumption that the flow rate (or evaporation rate) is constant within the release time range, and zero outside. Since ADAM calculates the actual source term, which includes its time dependence, it is necessary to couple the time-varying release with the dispersion model. In order to achieve this, the time-varying release is divided into a number of discrete time segments, and the dispersion associated therewith is calculated accordingly. As suggested in the Purple Book (2005), the segments used to approximate the flow rate are characterised by constant values, which correspond to the outflow average in the selected time segment. The segment duration is defined in such a way that all segments have equal areas (i.e. the released mass is equal in each segment time) and the sum of all segment areas is the same as the area under the flow rate curve (i.e., the total released mass). ADAM allows splitting the timevarying release source term into a maximum number of twenty segments, with a default value of five, and it produces the calculus of the concentration by using the following procedure: 1 The release is replaced by N segments with outflow rates q i (i refers to the i-th segment) and time duration t i . = Q tot /(N q i ) , where Q tot is the total mass released. 2 N different dispersion calculations are separately conducted for hypothetical releases with outflow rates q i and durations t Di = min( t i , Q tot /q i ) , where t i is the total duration of the release. 3 Since the concentration produced by the time varying release, has to be in between the concentrations produced by the hypothetical releases with higher and lower outflow rates, the overall concentration is estimated by a weighted average of those associated to the single releases, with the released masses Q i as the reference weights.
A more exhaustive explanation of the above procedure and limitations is reported in the ADAM technical guidance (Fabbri et al., 2017).

Contribution from pool evaporation in case of rainout
Since ADAM provides an estimate of the rainout, the dispersion is calculated by using a recombination process. In particular, the dispersion of the vapour jet (or the vapour part in a catastrophic release) is combined with the vapour resulting from the evaporation of the rainout pool. These two phenomena are considered as independent, and ADAM separately estimates the effects associated with each of them. The overall concentration is conservatively estimated by adding up each contribution. In order to apply the recombination method, the dispersion calculation is applied twice. The first is applied to the primary phenomenon, i.e., the one associated with the direct vapour source, whilst the second is associated with the secondary phenomenon, i.e., the evaporation for the pool resulting from the rainout. When the release involves a pure liquid, the primary dispersion phenomenon is driven by the vapours originating from the pool evaporation whilst the secondary and minor phenomenon results from the jet vapours produced by the mechanical brake-up of liquid droplets. The input parameters to apply are selected according to the simplified scheme depicted in Fig. B1, which typically refers to a two-phase catastrophic release. Since after rainout, the droplets that remain airborne will be added to the vapour phase in the dispersion-related phenomenon, the masses used as SLAB input variables for the primary and secondary dispersions are: where Q is the total mass released, and x rain is given by: where X L rain the liquid fraction after rainout (i.e. Q pool /Q L in the figure), and x f the vapour quality after flash.
For the part of dispersion associated with the direct vapour, since in SLAB the substance aerosols are considered as embedded/transported by the cloud, the liquid fraction to be fed to SLAB in the recombination process is provided by ADAM as follows: Clearly for the part of the dispersion associated with pool evaporation mechanism (i.e. generally the secondary dispersion), this parameter will be fixed to zero as the evaporated phase has no liquid portions. For continuous releases, the above relations are also valid, with the condition of substituting the mass with the flow rate.