Article Text
Abstract
Objectives This study investigated the costs of 2-hour multiprofessional in situ hospital trauma team simulation training and its effects on teams’ non-technical skills using the T-NOTECHS instrument.
Background Simulation is a feasible and effective teaching and learning method. Calculating the costs of simulated trauma team training in medical emergency situations can yield valuable information for improving its overall cost-effectiveness.
Design A prospective cohort study.
Setting Trauma resuscitation room in Central Finland Hospital, Finland.
Participants 475 medical professionals in 81 consecutive, simulated trauma teams.
Primary and secondary outcome measures Team simulation training costs in 2017 and 2018 were analysed in the following two phases: (1) start-up costs and (2) costs of education. Primary outcome measures were training costs per participant and training costs per team. Secondary outcome measures were non-technical skills, which were measured on a 5–25-point scale using the T-NOTECHS instrument.
Results The annual mean total costs of trauma team simulation training were €58 000 for 40 training sessions and 238 professionals. Mean cost per participant was €203. Mean cost per team was €1220. The annual costs of simulation training markedly decreased when at least 70–80 teams participated in the training. Mean change in T-NOTECHS score after simulation training was +2.86 points (95% CI 1.97 to 3.75;+14.5%).
Conclusions The greater the number of teams trained per year, the lower the costs per trauma team. In this study, we developed an activity-based costing method to calculate the costs of trauma team simulation training to help stakeholders make decisions about whether to initiate or increase existing trauma team simulation training or to obtain these services elsewhere.
- medical education & training
- trauma management
- health economics
Data availability statement
Data are available upon reasonable request. Technical appendix and statistical code and data set available from the corresponding author at eerika.rosqvist@ksshp.fi.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
All the cost-related data were obtained from reliable sources, yielding reliable estimates of real-world costs.
The cost components of the trauma team simulation training course were calculated in detail and thoroughly analysed in the following two phases: start-up costs and costs of education.
These cost analyses cannot be readily generalised to other countries or simulation environments, as other countries and centres are likely to have different cost structures and price levels.
The T-NOTECHS scale has demonstrated strong structural validity and acceptable reliability for assessing performance in simulation-based multiprofessional trauma team resuscitations.
Adhering to rigorous translation and cross-cultural adaptation guidelines for instruments rating non-technical skills, a Finnish version of the T-NOTECHS scale has been developed.
Introduction
Simulation training is effective in educating healthcare professionals.1 2 It is time-efficient, shortens the learning curve and leads to fewer treatment errors, all of which reduce long-term healthcare costs.3 Simulation is feasible and effective in specialised acute settings, such as emergency departments.4
Reports on the costs of simulation-based medical education research are few.5 6 In their review, Zendejas et al5 found that costs were mentioned in only 59/967 studies on simulation-based medical education. Thus, the literature fails to provide consistent and interpretable information on the relative costs of simulation-based education.6 The resources required to implement simulation-based education have led some stakeholders to question its overall value.2
The primary aims of a trauma team are to rapidly resuscitate and stabilise a potentially critically injured patient, prioritise and determine the nature and extent of the injuries, and, according to the predetermined protocol, prepare the patient for transport to the site of definitive care.7 8 In this study, the educational goals of the trauma team simulation course focused on improving team members’ non-technical skills.9
Computerised patient simulator-based multiprofessional trauma team training started in 2009 in the Central Finland Central Hospital (CFCH) hospital and has since been used as a regular teaching and learning method to improve and maintain teams’ performance. The positive effects of this short-structured 2-hour in situ trauma team simulation training course in improving the non-technical skills of hospital trauma teams has been confirmed.9 However, the real costs of the course have not previously been calculated. Moreover, to the best of the authors’ knowledge, no previous studies have been published on the costs of hospital trauma team simulation training courses or programmes. Calculating the real costs can yield valuable information for improving overall cost-effectiveness, for example, the number of teams trained per year required to reduce the costs of these courses.
This study, implemented in the Central Finland Hospital (CFH), investigated the costs of a 2-hour multiprofessional in situ hospital trauma team simulation training course and its effects on teams’ non-technical skills using the T-NOTECHS instrument.
Materials and methods
The CFH, Jyväskylä, Finland, is the only hospital with an around-the-clock emergency department and the only provider of public secondary care in its catchment area, which has a population of 275 000. As defined by the American Trauma Society, CFH is a level II/III trauma centre with 24-hour coverage by emergency medicine physicians, residents mastering the field of general surgery, orthopaedic surgery, gastrointestinal surgery, anaesthesiology, radiology and critical care.
CFH’s Center of Medical Expertise is one of the Network of Accredited Skills Centres in Europe. In CFH, trauma team simulation training is being developed for specialists and residents as well as for trauma and other nurses working in the emergency department, intensive care unit (ICU) and postanaesthesia care unit. Minimally, the trauma team includes a surgeon, an anaesthesiologist, a radiologist, a trauma nurse and another nurse paired with the anesthesiologist. Surgeons, anaesthesiologists and nurses participate in simulation training once or twice per year.
The structured 2-hour trauma team simulation courses reported on in this study were conducted in a real hospital environment (‘in situ’) in CFH’s emergency department and ICU. All the simulations performed in 2017 and 2018 were included. Participants included surgeons, anaesthesiologists, radiologists and nurses, and they acted in their real-life professional roles. The 2-hour course comprised either one simulation per course (pregnant patient scenario in spring 2017 and emergency laparotomy scenario in spring 2018) or two simulations per course (thoracic injury scenarios in autumn 2017 and car crash scenarios in autumn 2018). The course included either (1) clarifying the method, (2) an introductory lecture, (3) assuming roles, (4) first simulation and debriefing and (5) second simulation and debriefing or (1) clarifying the method, (2) an introductory lecture, (3) assuming roles and (4) simulation and debriefing. The following three different computer-based adult patient simulators were used: the HAL S3201 (Gaumard, Nordic Simulators Oy), SimMom (Laerdal) and Ambu Man (Ambu).
The main simulation training instructor was a senior anaesthesiologist and intensivist who had participated in the European Trauma Course, the Finnish simulation instructor basic course and Generic Instructor Course. He was paired (one at a time) with three nurse teachers (specialised in anaesthesia, ICU and ER, respectively), who acted as simulator pilots. Nurse teachers have also participated in the Finnish basic simulation instructor course. Each team also includes a technical assistant, who conducts the simulations.
The simulated patient scenarios were changed twice a year. The introductory lecture topics and two simulated scenarios are described in table 1. We used the data on these two simulation scenarios (four cases in total).
After the study was approved by the Central Finland Health Care District’s institutional review board, the data collection started. Data were collected prospectively and anonymously between 2017 and 2018 by expert raters (an anaesthesiologist and a nurse teacher) using the T-NOTECHS scale.10 Answering the self-assessment questionnaire was voluntary. The courses were attended by 475 participants in 81 teams. The cost-related data were collected between January and March 2019.
Patient and public involvement
No patient involvement.
Instrument
T-NOTECHS scale
T-NOTECHS scale was developed to assess non-technical skills in trauma team resuscitation.11 The T-NOTECHS scale comprises the following five categories of teamwork behaviours: leadership, cooperation and resource management, communication and interaction, assessment and decision-making, and situation awareness/coping with stress. Each category is scored on a 5-point scale (range: 5–25 points). The lowest score indicates the total absence of the teamwork behaviour and the highest score flawless teamwork behaviour.
Adhering to rigorous translation and cross-cultural adaptation guidelines for non-technical skills rating instruments, a Finnish version of the T-NOTECHS scale has been developed.12 The scale has proven to have strong structural validity and acceptable reliability for assessing performance in simulation-based multiprofessional trauma team resuscitations.10 It has also been widely adopted for assessing simulation performance.9 11 13–23
Cost components
Team simulation training was analysed in two phases: (1) start-up costs and (2) costs of education. The start-up costs included the costs of equipment, the training courses attended by the training instructors, planning, and the salaries of instructors, nurse teachers, educational designers and technical specialists before education. The education costs included the salaries of the participants and instructors during the education and the costs of materials, maintenance of equipment and hiring spaces.
Data were obtained from multiple sources: the use of personnel resources was estimated by interviewing the training instructor and educational designer. Salaries (including indirect costs) and the costs of equipment, materials and hiring spaces were collected from CFCH databases.
The costs of personnel and spaces were analysed using simplified activity-based costing logic. The cost of each activity was calculated by multiplying its duration by its unit costs. Equipment, maintenance and material costs were extracted directly from CFCH databases. Depreciation over 10 years with a 3% interest rate was used for equipment. Costs of salaries and spaces were determined by the duration of the simulation. For example, if a room was reserved for 8 hours, the cost per hour was multiplied by number of hours actually used. Correspondingly, personnel costs were calculated by multiplying the duration of the simulation in hours by total salary costs per hour, including overheads.
The primary outcome measures were training costs per participant and training cost per team. Training costs were divided into two categories: participants’ salaries and training costs.
Statistical analysis
The T-NOTECHS-related data were analysed with SPSS, V.25. The results were presented as frequency and percentage distributions with SD. A CI of 95% and paired-samples t-test were used as analytical methods. A p value of <0.05 was considered significant. Inter-rater reliability between the expert raters was calculated using intraclass correlation coefficients (ICCs). The ICC value was classified according to Cicchetti et al24 as poor (<0.40), fair (0.40–0.59), good (0.60–0.74) or excellent (0.75–1.00).
Results
Costs
In total, we analysed 81 teams, 124 simulations and 475 participants. The mean total costs of simulation training were €58 000 per year (€53 000 in 2017 and €63 000 in 2018; figure 1). Education costs accounted for 53% and start-up costs for 47% of total costs. In the start-up phase, equipment accounted for 85% and the instructor-training courses for 15% of costs. Personnel costs accounted for 94% of the costs in the education phase.
Mean cost per participant was €203 and mean cost per team €1220. Participant salaries formed 32% and training costs 68% of total costs. Since the initial investment costs were high, the mean cost per team was reduced by increasing the number of teams trained (figure 2).
T-NOTECHS mean score
Online supplemental files 1 and 2 demonstrate the trauma teams’ non-technical skills after the first and second simulations (N=41 teams, 225 participants). Tables 2 and 3 show the improvement in the T-NOTECHS mean scores after the first and the second simulations.
Supplemental material
According to the evaluation of the four raters, the mean change in the T-NOTECHS score was +2.86 points (95% CI 1.97 to 3.75; +14.5 %). The cost per one-point increase in teamwork performance (T-NOTECHS score) was €427.
Inter-rater reliability for T-NOTECHS
The inter-rater agreement between two raters for T-NOTECHS after the one-simulation (n=76) and two-simulation (n=55) courses was good and fair (ICC=0.67, 95% CI 0.48 to 0.79 and ICC=0.53, 95% CI 0.20 to 0.72, respectively). The inter-rater agreement between two raters for T-NOTECHS after all the simulations (N=131) was good (ICC=0.69, 95% CI 0.56 to 0.78).
Discussion
This study analysed the costs trauma team simulation training and its effects on non-technical skills as evaluated with the T-NOTECHS instrument. Mean cost per participant was €203 and the mean cost of the whole team simulation was €1220. The simulation training costs markedly decreased when at least 70–80 teams participated annually in the training. The mean change in the T-NOTECHS score after the simulation training was +2.86 points. The cost per one-point increase in teamwork performance (T-NOTECHS score) was €427. These results can be used as a benchmark for further studies.
Personnel salaries per year amounted to €20 300. The one-simulation case in spring 2018 (emergency laparotomy) increased salary costs, as the simulation ended in the ICU and almost all the teams had three instructors (90 hours of specialists teaching in spring 2018 compared with 34 hours in spring 2017). This indicates that the more complex the patient case, the greater the number of specialists needed to teach multiprofessional teams.
The results show that the present training volume (81 simulations) in the CFH was not optimal with regard to costs, as the costs per team fell, the greater the number of trauma teams trained per year. Thus, it might lower costs to increase the number of trauma teams receiving simulation training or to sell the simulation concept, teachers, and equipment to other hospitals. This accords with the finding of Jensen et al,25 who studied the implementation barriers related to simulation-based training for trauma resuscitation in centres using and not using simulation-based training. They found that funding faculty or staff time to participate in the session and funding to run the sessions were commonly rated barriers in both groups. Funding to purchase simulators was ranked as the greatest barrier in centres not using simulation-based training. These funding barriers were negatively associated with number of annual simulation-based training sessions. They concluded that strategies to share resources, thereby decreasing costs, may improve the usage of simulation-based training.
In this study, the non-technical skills of the participating trauma teams improved statistically significantly after the simulation training course when evaluated using the T-NOTECHS instrument. Non-technical skills training has been identified as an important contributor to improved patient safety.26 Good non-technical skills allow healthcare providers to concentrate more on the technical side of the task. For example, delegating subtasks to team members reduces one’s own workload and introduces an additional safety layer in technical performance.26 Good non-technical performance is also associated with a significant decrease in disposition time.16 However, certain combinations of non-technical skills are more effective than others in different contexts.26 To avoid these possible confounding effects, we used T-NOTECHS, an instrument specifically developed for evaluating trauma team resuscitations.
Trauma team simulation training per se, has many positive influences on patient-related outcomes. Trauma teams have been shown to reduce time to CT scan and secondary survey of the patient; in other words, assessment time has been reduced.27 Knobel et al28 found a significant decrease in time spent on real trauma resuscitation from patient arrival at the emergency department to CT scan after simulation training. As emergency department length of stay is an independent predictor of hospital mortality following trauma activation, team-based simulation training has the potential to improve patient outcomes.27
In sum, the economic evaluation of simulation-based training programmes or curricula is required to determine whether the improvement effected in trainee performance (knowledge, skills and attitudes) and health outcomes justifies the investment cost.6 When deciding whether to purchase potentially expensive simulation training programmes, it is important that administrators are well informed about their clinical and economic effects.29 Future work should focus on both costs and outcomes, with robust measures of resource investments, provider performance, patient outcomes and impact on the healthcare organisation.2 It must, however, be conceded that measuring real-life patient outcomes is challenged by many confounding factors.
Limitations and strengths of the study
A strength of this study is that all the cost-related data were obtained from reliable sources, yielding reliable estimates of real-world costs. Another strength is that the cost components of the trauma team simulation training programme were calculated in detail and thoroughly analysed in two phases: start-up costs and education costs. A properly conducted economic evaluation can help stakeholders to determine the optimal use of resources when selecting, for example, the simulation modality or method of assessment.6
A limitation of this study is that these cost analyses cannot be readily generalised to other countries or simulation environments. The obvious reason for this is that other countries and centres are likely to have different cost structures and price levels. Nevertheless, the information yielded by this study may be of value in designing trauma team simulations and estimating their costs. This in turn may help to provide estimations of the true costs of simulation training for policymakers and stakeholders.
Conclusion
The results showed that the greater the number of teams trained per year, the lower the costs per structured 2-hour session of in situ hospital trauma team simulation training. This study developed an activity-based costing method to calculate the costs of trauma team simulation training, helping stakeholders to make decisions about whether to initiate or increase existing trauma team simulation training or to obtain these services elsewhere.
Data availability statement
Data are available upon reasonable request. Technical appendix and statistical code and data set available from the corresponding author at eerika.rosqvist@ksshp.fi.
Ethics statements
Ethics approval
This study was approved by the Central Finland Healthcare District’s institutional review board and scientific research application form was signed by Medical Director of the Central Finland Healthcare District, 2 October 2015. There is no number/ID in approved application form.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors ER, MY, PT, JPR and JP: substantial contribution to the concept and design of the work; and acquisition, analysis and interpretations of data; have drafted the article and revised it critically for important intellectual content; have approved the version to be published and have participated sufficiently in the work to take public responsibility for appropriate portions of the content.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.