What is the value of health emergency preparedness exercises? A scoping review study

Emergency exercises involving the health community are considered an important and integral part of emergency preparedness activities. However, little is known about whether these exercises are e ﬀ ective at improving individual and/or organisational preparedness for responding to emergencies. This paper reviews and summarises published evidence on the e ﬀ ectiveness and bene ﬁ ts of exercises to prepare health emergency professionals for responding to emergencies and disasters. A literature search strategy was designed to identify the relevant publications from four major medical databases (Medline, Embase, Global Health and CINAHL). Studies meeting our inclusion criteria were analysed in detail (N=86). Both qualitative and quantitative data were reviewed. Data analysis involved a descriptive summary and a thematic analysis. Health emergency preparedness exercises were found to be e ﬀ ective (post exercise) at improving participants ’ knowledge of emergency activities, policies and procedures and improving overall competence and con ﬁ dence. Other immediate individual bene ﬁ ts included improved perceptions of preparedness and improved understanding of individual roles and roles of partners. Whether these improvements persist over time and translate into improved emergency response is not clear. The most commonly reported post-exercise organisational bene ﬁ ts were: identifying gaps or limitations in plans, protocols or procedures and providing opportunities to share lessons. Only a few identi ﬁ ed studies reported the long-term impact of exercises on job practices and on real emergency response. Published evidence of exercises impact on individual and organisational levels of emergency preparedness and response over the long term is very limited. The majority of the studies included (n=65, 76%) were US-based.


Introduction
Recent acts of international terrorism, the increased frequency of extreme weather events and newly emerging health threats (e.g. Ebola Virus Disease (EVD)) highlight the importance of effective emergency response plans and capabilities for responding to large-scale health emergencies. Emergency preparedness is a key activity in the health agenda in the UK and the Civil Contingencies Act placed a legal responsibility for emergency preparedness with NHS organisations [1,2].
Emergency preparedness activities comprise many components and include a complex cycle of planning, equipment, training, exercises and improvement [3] with emergency preparedness exercises often considered the most vital part of the cycle [4,5]. There are different types of emergency preparedness exercises which can broadly be combined into two major groups and which test different aspects of an organisation and/or systems' emergency preparedness: discussion-based exercises (often referred to by different names, including tabletop or desktop exercises, workshops or seminar-based exercises) and operation-based exercises (such as drills, functional exercises/ command post exercises, and field exercises) [6].
Discussion-based exercises can be used to familiarise participants with their plans, roles, and procedures (which is often the focus in a workshop or seminar-based exercises), or to allow participants to practice their roles and emergency plans through taking part in a facilitated discussion of simulated emergency situations (which is more commonly a focus in tabletop exercises). Discussion-based exercises are typically led by facilitators and presenters to keep participants on track in meeting the exercise objectives.
Operation-based exercises typically involve responding to a scenar-io designed to simulate an emergency situation under more realistic conditions. Such exercises can range from smaller activities such as drills to practice specific skills or procedures (such as triage, evacuation or communication) to field-based exercises designed to replicate as closely as possible a response to a real emergency. Operation-based exercises are more difficult to conduct and they demand significantly more resources than discussion-based exercises. However, they allow the testing of tools, plans, procedures, resources, technologies, interagency coordination, and command centres, under conditions closely matched to a real emergency event.
[ Table 1 provides a summary of different types of emergency preparedness exercises].
Since health emergencies, that require a major response, happen quite infrequently, organisations and staff need to exercise the procedures and skills for these events in order to be prepared to respond. Scenarios may include a simulated response to a mass casualty incident, hospital or community evacuation, initiating and operating a point of dispensing, as well as practising response to chemical, biological, radiological and nuclear (CBRN) "public health threats". Regulatory bodies often mandate or recommend that organisations draw up long-term exercise plans, which are developed in accordance with organisational, regional, national, or supra-national priorities and include a combination of different exercise types to accomplish specific aims and objectives [7].
Even though universally accepted as an important component of health emergency preparedness activities, there is little research that provides a comprehensive overview of the benefits of emergency preparedness exercises themed around the health response. Two previously published systematic literature reviews, which attempted to look at the effectiveness of emergency exercises as a training opportunity, failed to produce conclusive evidence [8,9]. A statistically significant change in knowledge, measured by a test score, was used as the outcome measure to demonstrate the effectiveness of emergency preparedness training, including drills for health-care providers [9]. Studies with quantitative data, such as pre-and post-knowledge scores or statistics showing significant improvement in performance from exercising were not identified in sufficient numbers to produce conclusive evidence about the effectiveness of exercises. Heterogeneity of methods used to evaluate the outcomes of exercises and poor quality of the studies have also been acknowledged as part of the issue. Therefore, the present study sought to address these shortcomings by presenting a comprehensive review of the literature related to the perceived benefits of health emergency-preparedness exercises using a scoping study approach. A scoping review is warranted where insufficient quantitative evidence is available, and can include both quantitative and qualitative studies [10]. We aimed to synthesise existing literature reporting the benefits to both individuals and organisations of health emergency preparedness exercises, as well as any lasting impact these exercises have on emergency preparedness and response.

Methodology
This study uses a scoping review approach, which is intended to allow the breadth of knowledge and practice in an emerging domain to be explored and documented [11]. It is especially valuable when there is not a clear consensus on the boundaries of the domain, or on the   1991  1992  1993  1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  methodological and conceptual attributes which constitute the domain. This is the case in the dispersed and varied evidence base of emergency preparedness exercises, which to date uses varied data collection methods, utilises a great variety of evaluation methodologies and reports many different types of evidence. A scoping review is considered no less systematic than any other approach to mapping the literature in a particular field, but unlike meta-analyses, it does not seek to combine quantitative studies statistically nor claim to produce clear outcomes from that analysis. Rather, by including qualitative in addition to quantitative studies, this approach can allow mapping of different types of evidence and is particularly valuable if insufficient quantitative evidence is available [10].
To complete the scoping review process, we followed the "five stages framework" suggested by Arksey and O'Malley [12]. The final optional consultation stage six was not undertaken in this study, although the findings were presented to a group of the project stakeholders and their comments were considered in preparing this paper.

Framework Stage 1: identifying a research question
This study focused on the following broad research question: what is known from the existing literature about the effectiveness of public health emergency preparedness exercises? More specific research questions were: what are the benefits from participating in public health emergency exercises for personnel involved; what are the benefits from participating in public health emergency exercises for organisations, and finally, what kind of impact does an emergency exercise have in the immediate, short, and longer-term on emergency preparedness of individuals, teams and organisations?

Framework Stage 2: identifying relevant studies
The search strategy aimed for published studies in medical databases. A two-step search strategy was used in this review. An initial limited search of Medline and CINAHL was undertaken followed by the analysis of the text words contained in the title and abstracts, and of the index terms used to describe article. A second search using all identified key words and index terms was then undertaken across all included databases. Four major medical databases (Medline, EMBASE, CINAHL and Global Health) were searched using terms related to different types of emergency situations (disaster, natural disaster, weapons of mass destruction, bioterrorism, CBRN, terrorism, and pandemic) and different descriptive names for types of emergency exercises (tabletop, desktop, functional exercise, drill, and command post). We also used the related MeSH/subject terms, if offered. The search string was based on a Boolean approach; synonyms for different types of emergency situations were combined by the operator "AND" with synonyms for different types of emergency exercises. The search included literature published from 1990 through September 2015 in English.

Framework Stage 3: selecting studies for inclusion
The literature search across medical databases identified 602 studies (Embase -224; Medline -160, CINAHL -121 and Global Health -97) using the search strategy outlined above. Almost half of those papers (N=270; 49%) were discarded as duplicates. Titles and abstracts of 332 papers were screened to assess if they were reporting exercise outcomes such as evaluation of performance or response in emergency exercises, participants' perceptions about the potential benefits of emergency exercises, lessons learned from exercises, and exercises' impact on subsequent actual disaster response. At this stage the inclusion/exclusion criteria continued to be refined with the aim of achieving the breadth of available evidence, which is recommended by scoping study methodology [11]. Both qualitative and quantitative studied were considered.
Only papers that were clearly irrelevant were removed at the title/ abstract screening stage and the full text of 151 papers was further reviewed including five more potentially relevant papers that were identified through a review of relevant papers' citations.
Excluded studies were: a) reports that only include the experience of developing and conducting emergency exercises, without any evaluation data; b) reports of the development of emergency policies or plans; c) the results of tests of emergency equipment or tools; d) emergency preparedness surveys and literature reviews about emergency preparedness; e) any exercises conducted as a part of emergency training courses or curricula; f) exercises conducted to practice response to a clinical crisis, like cardiac arrest, for example and g) studies with unclear or undefined aims and objectives were excluded from further analysis.
Papers that were discarded at this stage were reviewed by another researcher (PR) to ensure they met the exclusion criteria. The final cohort included 86 studies which underwent further detailed analysis Fig. 2. In line with the general guidelines for a scoping review, the 86 accepted studies were not assessed for quality [12]. The data from the accepted 86 studies was tabulated by one researcher (ES) as follows: reference number, year of publication, country of publication, type of study, exercise type, aim and objectives, type of threat, evaluation criteria and outcomes (knowledge, skills, benefits, attitudes and lessons learned).

IdenƟficaƟon
Analysis of the data involved a descriptive numeric summary and a thematic analysis. For the thematic analysis, each article was coded inductively by explicit or surface explanation of the data (based only on the stated purposes and benefits), and without trying to fit into the preexisting coding frame. This approach allowed as many codes to form as was needed [13]. The codes were then inductively analysed to form an initial framework in support of the themes, which were further discussed and agreed by the research group.

Description of included studies
Most of the papers which were included and reviewed were either qualitative studies sharing experience with emergency exercises and reporting benefits or lessons learned (N=47, 55%), or descriptive quantitative studies (N=27, 31%) mainly quantifying participants' perceptions of exercise outcomes. Eight studies (9%) reported quantitative outcomes from pre-and post-exercise evaluation [14][15][16][17][18][19][20][21] with only one study using a control group to assess the impact of the emergency exercise on participants' perceptions of team work, training adequacy, response network, job risk and equipment adequacy [19]. Four studies reported results from multiple emergency exercises [22][23][24][25].
Type of intervention: The most commonly reported emergency exercises were operation-based exercises such as drills or functional exercises (N=51; 59%); including four drills involving paediatric departments/patients [54,58,61,63]; see Table 3. Twenty-three studies (27%) reported results from discussion-based tabletop or desktop exercises and three studies included reports of the outcomes from the combined discussion-based (mainly tabletop) and operation-based exercises [23,46,61]. In two reports outcomes of the real disaster response and the role of preparatory emergency exercises in this response were discussed [51,64]. Four studies reported outcomes from a large scale field exercise [3,54,57,65] and three studies reported on large scale command post exercises (CPX) [39,47,66].

Qualitative summary (thematic analysis)
All 86 papers were subjected to qualitative data extraction and thematic analysis of data relating to both the purposes of emergency exercises and the reported benefits of exercises was undertaken.

Purposes of emergency exercises
Two major themes emerged among the included studies in terms of the purpose of exercises (the mapping of included studies is provided in Supplementary material, Table 6). Testing, assessing, and evaluating appeared to be the most common purpose of emergency exercises, with most of the exercises reportedly being designed to validate various emergency plans, both general [43,44,46] and specific (PICU surge plan [54], polio outbreak preparedness [24] or decontamination [67]), as well as to test various protocols and procedures (vertical evacuation [62], mass screening [68] and decontamination [58,69]).

Effectiveness and benefits of emergency exercises
Research Question 1: what are the benefits from participating in emergency exercises for personnel involved?
Very few of the studies included in this review outlined education and training as the purpose of emergency exercises. As such, exercises rarely included educational or personal outcomes as evaluation criteria. However the studies that did inform this research question largely reported the perspectives and experiences of exercise participants. A complete listing of the studies whose results consider the benefits of exercises for individuals involved are included in Table 4, with outcomes presented thematically.
These studies identified several benefits from this self-reported data, including increased confidence [32,45], an improved perception of preparedness [44,45], and an improved understanding of their own roles [57,73], the roles of partners [57,73,74] and the role of the Incident Command System [57].
Some studies reported data collected from participants using preand post-exercise instruments. There were reports of significant improvements in participants' post-exercise knowledge of emergency activities [16], policies and procedures [21], and the hospital disaster plan, as well as an overall perception of the level of departmental  preparedness [14]. Significant improvements in competence [20], confidence [21], and perceptions of network effectiveness, training, equipment adequacy [19] and teamwork [18] were also reported.
Across the studies, practical lessons learned were the most often reported-and reported to be some of the most valuable outcomes for organisations that conducted emergency preparedness exercises. These are summarised in Supplementary material, Table 7 but of particular note are: • Practical recommendations for developing protocols for situations such as mass decontamination [67,71], vertical evacuation of neonates [61], and infection control protocol for pandemic response [32].
• Hospital emergency preparedness and response to simulated situations is difficult to assess from exercises due to the lack of robust and validated tools [86]. However, improvements in hospital exercise performance from repeat exercise participation was reported when assessed by a standardised assessment tool [53] and improvements in facilities to deal with emergencies from repeat exercises were also noted [87].
• Pre-exercise community training can improve exercise performance [45,82] and contribute to building trust and sustainable relationships with community partners [31]. Necessity to complete training in emergency procedures annually was recognised by 69% of participants of a community-based drill [82].
Research Question 3: what kind of impact an emergency exercise has in the immediate, short, and longer term on emergency preparedness of individuals, teams and organisations?
Evidence of exercises' impact on individuals in the short and longer term is very limited and only a few papers reported any effects on organisations over longer period. Improvements in emergency plans three months after exercises were reported in one study [15], improved facilities [87] and improved hospital performance from repeat exercises suggested a possibility of better response to real emergencies [53]. The "enormous" help of emergency preparedness exercises to deal with actual disaster responses was also acknowledged [15] and the successful evacuation of 947 patients was attributed to the existence of a plan that was practiced and refined through full-scale evacuation exercises [51]. As for the personal benefits, progressively improved performance, as well as improved attitude towards fire risk and safety issues from repeat exercising had been reported by members of public [40], and improved performance and engagement in subsequent exercises was reported by health-care staff who had previously engaged in tabletop exercises [16].

Discussion
In this paper we present an analysis of the previously published studies that describe how emergency exercises contribute to improving public health emergency preparedness. We reviewed both quantitative and qualitative studies of the effects of exercises on participants and organisations.
This study attempted to address three major research questions: the benefits of exercises for individuals, the benefits for organisations and Table 4 How do individuals benefit from emergency exercises? Included studies organised by thematic analysis categories.

Collaboration
• Brought together multiple departments that had not worked together [16,75,81] • New community partnership was formed [52] • Improved working relationships of the university and local community ("most unified effort ever displayed by the community") [52] • More than 50% felt more certain about interaction between involved agencies in case of chemical mass casualty event [74] • Acquaintance with members from supporting departments [61] • Provided opportunities to meet to share good practices [53] identify needs in international cooperation and further preparation [47] • Increase awareness of challenges and possible solutions [53] • Discuss pandemic influenza [81] 7 Training • Provided interdisciplinary training [15,34] • Highlighted importance of individual emergency plans [15] • Increased awareness of disaster preparedness methods and resources [41] • Identified future training needs [20,45] 6

Tested
• The readiness and capacity to implement the disaster plan [36,45] • Time to set up and activate a public dispensing point [99] • • The effectiveness of emergency plan [54] • Hospital preparedness [22,100] 3

Team building
• Useful team building exercise [34,61] • Provided a collaborative team based environment, created a realistic sense of urgency, had a very effective way to learn about surge capacity strategies and build IP skills and organisational capacity [34] 2 Response to real disaster • Training (tabletop) helped to respond in real disaster [15] • Successful response due to practiced and refined emergency plans [51] 2

Great value
• Public health preparedness programme [83] systems, and any longitudinal impacts of exercises. In this discussion section, we will address each of these in turn.

Benefits for individual participants
Individual benefits can be demonstrated by assessing participants' knowledge, skills, attitudes, perceptions, and intended behaviours before and after the exercise to measure for change. A positive change in knowledge, skills and attitudes (KSA) would indicate that an exercise is effective and indicate that learning has taken place [19]. However it can be argued that immediate post-exercise measures are imperfect indicators of learning, as they may indicate only temporary change rather than lasting learning [88].
Most studies included in this review were focused on organisational and system testing, and so did not collect data on individuals pre-and post-exercise. Only a few papers reported post-exercise responses of participants, and improved understanding, knowledge and perceptions were the most common benefits of emergency exercises reported by participants.
Several quasi-experimental studies attempted to measure individual change as a result of participation in emergency exercises. Those studies differed in terms of the exercise type (drills and tabletops), responders (healthcare workers [14,16,17,20]; community [15,21], firefighters [18,19], police and civilians [18]), scenarios, and threats. However, all sought to discover the impact of exercises on individuals and reported positive individual outcomes: discussion-based tabletop exercises reported significant improvements in participants knowledge and competencies [15,16,20,21] and drills reported improvements in basic knowledge of emergency plans and response [14,17]. Also reported were improvements in self-reported perceptions of other team capabilities, personal abilities, and teamwork, as well as reduced levels of self-reported stress associated with emergency response [18,19]. Assuming that the positive changes are produced by successful exercises [19], and taking into account most participants' positive attitudes towards these exercises [16,20], it is worth understanding the ways in which these exercises seem to produce the reported benefits.
Firstly, most of the studies reported a pre-exercise preparation element as a part of the exercise, which included either an audio visual lecture [14] or a live presentation of a didactic lecture [17,20,21]. Although in two of these studies a pre-exercise evaluation was conducted after the introduction, the authors suggest that any such introductory lecture will contribute to learning by addressing the importance and relevance of the issues for learners to attend to during the exercise. Indeed, an intensive and versatile preparedness programme, including lectures and posters, produced clear improvements for participants in one study [89]. Conversely, the value of a drill on its own without any pre-exercise preparation was found to be limited in improving knowledge of physicians in anthrax bioterrorism [29].
The pervasiveness of poor staff emergency preparedness and training [14,28,78,90] and reported difficulties in attempting to educate staff in a hospital disaster plan and their emergency roles prior to exercise "because of the lack of interest or other priorities" [14] emphasise the importance of pre-exercise preparation as an educational opportunity that can improve the outcomes and preparedness [21].
Secondly, addressing the principles of adult learning in the design of health emergency exercises produces significant benefits, including improved knowledge and skills, as well as improved understanding of strategies to communicate, coordinate and collaborate, which can, in turn, contribute to effective response in a real life event [20]. Although there are many approaches to designing learning for adults, some general guidelines include: making the content meaningful and jobrelated; providing opportunities to practice new skills in a low-stress and safe environment; giving specific task-related feedback; and giving participants opportunities to observe and interact with each other [91]. Discussion-based tabletop exercises which used those elements in their design and implementation reported positive outcomes for participants [16,20,21].
Thirdly, discussion-based exercise outcomes can be affected by the quality of facilitators and lecturers, as discussions are largely supported by facilitators. Facilitators need to be good at facilitating small group discussion as well as knowledgeable about the local healthcare system. The satisfaction ratings and qualitative comments provided by participants highlighted the importance of these individuals in the effectiveness of the exercise [20,74].
Fourthly, a discussion-based tabletop exercise provides a forum for different emergency responders to be brought together to practice emergency response situations. The importance of including a mix of participants and agencies in tabletop exercises was highlighted and the level of discussion and networking that naturally occurs from the varied mix of participants and agencies was cited as the most valuable aspect of a tabletop exercise [20,72,74]. The importance of inviting key response agencies to take part in an emergency exercise with key senior level players present was stressed by the group of experts [92].
Fifthly, the importance of practicing emergency skills under pressure has been emphasised as multiple issues and errors have been identified from operation-based exercises, which have not been picked up in normal practice or from discussion-based exercises [83,93]. The inability of a discussion-based exercise to expose operational and logistic gaps has been acknowledged [3]. A few studies advocated conducting an operation-based exercise after a discussion-based exercise [23,65].

Organisational benefits
Emergency preparedness exercises are believed to help identify gaps in emergency plans and procedures that, when addressed, will improve an organisation's or system's emergency preparedness [6]. As expected, testing emergency plans and protocols appeared to be the most common purpose of emergency exercises. Most studies reviewed here used objective evaluation criteria to assess organisational performance in line with the aims of the exercise. The most commonly reported benefits from exercises were in identifying gaps or limitations in existing plans and protocols, and in sharing lessons learned.
Identifying gaps and limitations is important and a few studies reported introducing modifications (improvements) in emergency plans following exercise participation [15,23,70,94]. However, the most reported "lessons learned" should have been considered "lessons identified", as there was limited evidence presented that the challenges and limitations identified had been addressed or "learned". We did not identify any follow-up studies which have reported the outcomes from another exercise that tested modified plans designed to address the gaps or lessons identified in previous exercises. Such a design would demonstrate the exercise's benefits in not just identifying the gaps, but in actually improving emergency preparedness, which could be an indication of the exercise effectiveness.
Multiple barriers in addressing limitations and challenges identified from emergency exercises were acknowledged [95] and generating an action report immediately after the exercise to prompt further actions has been discussed as one option to stimulate after-exercise actions [38,58,76]. Commonly reported and recurring challenges must be addressed by emergency planners, and including those challenges in the emergency exercise objectives is recommended [96]. Publicising the changes and improvements that result from public health emergency preparedness exercises helps to sustain interest in the exercise programmes [6] and to increase their public credibility [19].

Longitudinal impact of emergency preparedness exercises
Overall, multiple personal benefits from exercise participation have been reported, with evidence of positive post-exercise change in participants knowledge, perceptions and attitudes. This can be an indication of the exercises' immediate positive effect. It has also been reported that individuals who participated in repeat tabletop exercises tend to engage in more in-depth problem solving and focus on more system-wide issues [16]. Whether the impact of these benefits remains over time or translates into everyday job practices is unknown and evidence from the learning sciences as applied to emergency training could facilitate the retention and transferability of knowledge obtained from emergency exercises [95].
Published evidence relating to the long-term benefits of exercising on public health organisations' emergency preparedness and response is also limited. A few studies reported actions undertaken as a result of the exercise participation which, as claimed, resulted in improved emergency plans and procedures. However the vast majority of the studies reported the lessons identified in exercises concerning limitations and challenges as main benefits, and it is not clear, from the literature to date, if the lessons are taken forward or addressed, and if these in turn result in improved emergency preparedness or response. One way to understand the long-term effect of emergency preparedness exercises is to study their impact on the real life emergency response. Success with evacuation of 947 patients when using a plan that was practiced and refined through full-scale evacuation exercises [51] can be used as an example to demonstrate the importance of refining an emergency protocol through exercises to produce a workable document which can be put into practice in case of major public emergency to save lives.
Although multiple immediate post-exercise benefits have been reported, it is difficult to establish their effectiveness at an organisational level without any further evidence of any lasting positive change in public health emergency preparedness.

Study limitations
Our search was limited to only four medical databases (Medline, EMBASE, CINAHL and Global Health), while such major databases as Scopus and Web of Science might have also included relevant studies. We also did not search across unpublished studies and grey literature, but only considered studies identified through database searches. All identified studies reporting public health emergency exercise benefits, including case studies, were included without any rigorous assessment of their scientific quality. This approach was taken deliberately due to the recognised lack of the high quality scientific evidence in this area and is allowed by the scoping study methodology. High quality studies researching personal and organisational benefits of public health emergency preparedness exercises are in much need. Another limitation of this study is that most of studies were carried out in the US; unique local regulatory and cultural issues mean that care must be taken when extrapolating or applying to other contexts.

Conclusion
This study sought to review the literature on the effectiveness of public health emergency preparedness exercises and what benefits they provide to individuals and organisations in the immediate and longerterm. The studies included in this review show clearly that the most reported benefits to emergency preparedness exercises were in identifying gaps (in emergency plans, procedures, resources, communication) and in sharing the lessons from emergency exercises. However, from the literature to date, it is not clear if the lessons identified in exercises concerning limitations and challenges are taken forward or addressed, and if these in turn result in improved emergency preparedness or response. Only a few studies reported positive long-term impact of exercises on job practices and on real emergency response; likely this is due to the difficulty associated with longitudinal follow-up studies and the relative infrequency of the emergencies that these exercises prepare healthcare professionals to deal with. Because the primary purpose of most exercises is addressing organisational emer-gency preparedness (such as testing plans or procedures), personal benefits from exercising of the staff involved were reported less often. However, a few quantitative studies included reports of improved knowledge, competencies, improved confidence and understanding. Again, there is no evidence about whether these positive changes persist over time.
One purpose of a scoping review is to map the territory in an emerging domain, so that both the boundaries and the terrain can more confidently identified. We argue that despite the lack of evidence these exercises hold tremendous benefits for individual participants, and that exploring these should be a prime part of research in the field moving forward. The overall analysis of studies reported here show that individuals enjoy and see value in participating in these exercises, but this should be explored further using both empirical and interpretive approaches. Personal outcome measures could assess the impact of the exercise on the individuals involved, as well assess the transfer of learning and skills acquired through emergency exercises to their day-to-day performance at work. Further, the impact of exercise participation on real-world emergency response is still poorly researched; this would directly relate to the exercise effectiveness. Follow-up studies to look at the implementation of actions identified through the exercise, as well as turning lessons identified into lessons learned and acted upon, would add significantly to our understanding about how and why emergency preparedness exercises can be valuable and effective.

Funding
This study was funded by the National Institute for Health Research Health Protection Research Units HPRU-2012-10414.