Developing a novel framework for non-technical skills learning strategies for undergraduates: A systematic review

Objectives There is substantial lack of guidance when it comes to the implementation of non-technical skills (NTS) in undergraduate medical education. This review aimed to identify and critically evaluate published literature on learning strategies for NTS in undergraduate medical education and to derive a training framework targeted towards standardizing future training interventions. Methods A systematic review of the MEDLINE database was performed using a prospective protocol following PRISMA guidelines. Studies evaluating undergraduate medical students exposed to NTS interventions, which measured subjective or objective outcomes in selected attributes, were included. Results Initial systematic search yielded a total of 5079 articles, out of which 68 fulfilled the inclusion criteria. A total of 24 NTS were identified, with communication skills being the most commonly reported skill evaluated (n = 37). A variety of educational tools were used (n = 32), noteworthy being the use of simulated patients. Great heterogeneity was also observed in measured outcomes and methods of assessment. A ‘triad of outcomes’ in NTS training was devised (knowledge, skill performance and attitude towards skills) and used for classification of all reported outcomes. Extracted data were used to design a non-technical skill training framework. Conclusions The existing literature describes a plethora of NTS interventions in undergraduate medical education, with varied outcomes and assessments. We hereby propose the ‘NTS Training Framework’, in an attempt to coordinate future research and catalyze the identification of an ideal NTS course structure to form tomorrow's physicians.


Introduction
In an era of globalized medicine and increased public pressure for high-quality care, the need to form medical professionals with greater adaptability to social environments is ever-growing. Whilst knowledge and technical skills remain indispensable pillars of medical education, non-technical skills (NTS) training has attracted considerable attention in recent decades [1], aiming to contribute to a more holistic model of medical education. NTS can be defined as a mélange of 'soft skills' allowing doctors to self-evolve as part of a 'learning organization' capable of adapting in volatile environments [2][3][4]. Increasing use of the term "soft skills" pertains to a paradigm shift from the medical profession's traditional notions of internalized norms and implicit standards towards a culture of audits, transparency and self-surveillance.
Inspired by its original application in the aviation sector and air safety, NTS training implementation has expanded to many multidisciplinary fields, including healthcare, to prevent adverse outcome related to human factors errors [5,6]. NTS training aspires to resolve healthcare failures precipitated by errors often conceived at the organizational level. For instance, narratives surrounding failures such as the Mid-Staffordshire scandal, revealed conditions that often lay the groundwork for errors, favouring quick fixes and 'blame games' over https://doi.org/10.1016/j.amsu.2018. 10 learning and transparency within teams [7,8]. Such systemic 'defects' have been the focus of numerous high-impact reports such as 'To Err is Human' and 'A promise to learn-a commitment to act' [7,9]. Yet preventable harm continues to occur, exacerbating both patient suffering and healthcare costs [10]. Even at the undergraduate level, lack of practice, anxiety and reduced confidence are all factors contributing to students' under performance when interacting with patients or operating within multidisciplinary environments [11]. These shortcomings may be traceable to the lack of a universal NTS training framework, highlighting the need for a unified and focused training approach part of medical school curricula [12][13][14].
NTS training efforts are the product of two main driving forces. The first, arising in the 1970s and based on the concept of 'dehumanization', sustains that medical students become progressively 'estranged' and detached from patients throughout their training, leading to compromised patient care -a theory also supported by more recent studies [15,16]. The second relates to the potential of NTS to influence clinical outcomes independently of technical skills [17]. Indeed, studies suggest that NTS, such as effective doctor-patient communication, ensure better health outcomes, patient safety, satisfaction and compliance [18], and decreased patient distress [19]. Equally, doctors benefit from a gratifying work environment and reduced malpractice lawsuits [20,21].
Despite the widely recognized advantages of comprehensive NTS training, research surrounding its implementation remains elusive. Whilst many studies evaluate NTS interventions, at present there are no clear guidelines for implementing NTS learning strategies. With this in mind, we performed a systematic review (SR) to identify and critically evaluate published literature on learning strategies for NTS in undergraduate medical education. Additionally, we outline comprehensive NTS intervention outcomes and derive a NTS training framework targeted for standardizing future training interventions.

Methods
We performed a SR following PRISMA guidelines to identify studies evaluating NTS interventions in undergraduate medical education.
Studies were hand-searched to find additional papers not included in the initial search.

Registration
This systematic review has been registered with Research Registry (registration code: reviewregistry608).

Assessment of methodological quality of the systematic review (AMSTAR 2)
We completed the AMSTAR 2 checklist to assess the quality of our methodology [22].

Search strategy
The search strategy focused on pooling studies published on the MEDLINE database, including targeted 'non-technical skills' intervention strategies. We utilized an extensive list of keywords obtained from MeSH terms pertaining to all qualities and skills-other than technical proficiency-believed to play a role in the development of future doctors; a complete list of all included keywords can be viewed in Appendix 1. This list was compiled following detailed review of key precedent studies [14,23] as well as the General Medical Council's (GMC) "Outcomes for graduates" guidelines, outlining key competencies expected of newly qualified doctors [24].

Selection criteria
As part of the SR protocol we agreed to a "Population, Intervention, Comparison, Outcome -PICO" strategy ( Fig. 1). Inclusion criteria limited selected studies to only those exposing primarily undergraduate medical students or mixed medical with other healthcare students (P, population) to any non-technical skills training approach incorporated within the curriculum of, or offered by, a Medical Institution or any other provider (I, intervention). Included studies were furtherly limited Fig. 1. PRISMA flow diagram [25]. to those measuring perceived or actual changes in attributes such as skills, knowledge or attitudes in randomized or quasi-experimental study designs (C, comparison). Outcomes sought were the objective or perceived improvement in participants' attributes. In borderline cases, study inclusion was made by default. Exclusively qualitative studies, reporting students' perceptions towards intervention (NTS module) were not considered. Such studies did not seem to affect any of our primary results. We also excluded studies not reporting baseline performance of the subjects, and therefore unable to comment on the impact of the intervention (performance improvement).

Data extraction
To maximize the homogeneity of extracted information from shortlisted studies, we used a pilot, prospectively designed worksheet, structured around the PICO headings. Extracted fields were based on the subheadings as indicated in Table 1. Whenever study location was not provided, the presumed location was the corresponding authors' affiliation country. A third reviewer (I.T.) was involved in the full-text articles assessment and resolved any disagreement between the two reviewers (M.N., L.C.), cross-checking and confirming the validity of extracted data. Any further disagreement was discussed and dissolved by the senior authors (M.S., A.P.). The final extraction sheet was standardized to provide refined results amenable to more accurate qualitative analysis and subsequent synthesis of results.

Selected studies
The initial systematic search yielded a total of 5079 records from MEDLINE. Following removal of duplicates, 5050 records were screened against our inclusion criteria. A total of 100 studies were selected from primary screening, and the full-text articles retrieved for eligibility assessment. A total of 68 studies were eligible for inclusion in our SR. (Fig. 1).  , followed closely by Europe (n = 24) . Specifically, the majority were carried out in the United States (US) (n = 23) , whereas the United Kingdom (UK) ranked second with 10 studies [51][52][53][54][55][56][57][58][59][60]. Of 68 included studies, 48 reported both qualitative and quantitative measures, whilst the remainder were solely quantitative.

Study population
Study population was homogeneous, with medical students forming the main participant group in selected studies. 11 studies also involved other healthcare students [28,32,34,39,46,47,56,57,62,66,75], with nursing students being the most represented group (n = 7) [28,39,47,56,57,66,75]. As summarized in Table 3, the total sample size ranged from 7 to 373 medical students, with an average of 111 ± 87.1 per study. Two papers were not considered for such average given that participant number was not provided. Indeed, tutors' number and profession were provided by only 24 and 22 studies respectively; both of these factors were considered while designing the 'NTS Training Framework'.

Intervention
After exploring the learning outcomes of the included studies, we defined an NTS intervention as 'any teaching strategy aiming to improve an individual's performance, knowledge and attitude towards a non-technical skill'. A total of 24 discrete NTS interventions were identified, with communication and empathy skills being featured a total of 37 and 9 times respectively (Table 3). We described studies as either long or short, based on intervention duration greater than 40 hours (active time) or 12 weeks (total time). When duration was defined in days, we assumed one working day equals to 8 hours. Considering these parameters, we computed an average study duration of 33 ± 94.1 hours and described 25 studies as long (Table 3). Of the 68 studies, 15 did not specify the exact intervention time and were therefore omitted from the aforementioned calculations [30,32,33,38,42,43,50,53,54,57,72,77,82,83]. Courses were either (a) implemented in the medical schools' core curricula and made compulsory to all attending students (n = 46) or (b) optional to internal medical students (n = 22).
As anticipated, a variety of educational tools were utilized (32 in total). Simulated patients (SP) were used in 31 studies and of those, 16 (52%) utilized SP feedback either as an educational tool or as a method of assessment. Didactic lectures, video-assisted learning and role play were recurrently integrated in the courses generating a combination of both traditional (didactic lectures) and non-traditional methods of teaching in most NTS courses. Table 2 summarizes the 16 most  commonly used educational tools with corresponding frequency, whilst the remaining (n = 16) were only utilized once each.

Learning outcomes
We noted an extensive variation in learning outcomes amongst selected studies, possibly owing to the broad pool of courses, each one assessing different NTS. Inspired by the apparent gap in literature concerning universal NTS outcomes, we devised a 'triad of outcomes'. This approach encapsulates a step-wise model by which students potentially acquire new NTS, with 1) knowledge of the NTS providing the initial context and scaffolding to the subsequent 2) performance of the NTS, followed by 3) self-reflection on the learning accomplished. Indeed, our 'triad' represents three potentially overlapping, albeit distinct, steps which both medical students and educators should focus upon when reflecting on their performance and designing interventions, respectively (Fig. 3). An overlap between assessed outcomes is inevitable as some skills impact various improvement areas.
We used the following definitions for classification of the reported triad of outcomes: 1. Knowledge: theoretical understanding of the principles surrounding a specific NTS. 2. Skill performance: quantifiable adeptness in a specific NTS. 3. Attitude towards skill: subjective perception about the usefulness and purpose of a specific NTS.
By qualitatively synthesizing (Table 4) the reported outcomes in accordance with the aforementioned 'improvement areas', we were able to deduce some descriptive statistics. For example, skill performance was encountered most frequently, amounting to 60 times, while attitude towards skill and knowledge appeared on 31 and 16 occasions, respectively.

Assessment
Our analysis yielded a total of 49 individual methods of assessment (Table 4), which we classified as either objective or subjective. Objective methods of assessment reflect the actual performance (n = 23), whilst subjective methods of assessment reflect the perceived competence of the participant in the relevant skill (n = 26). Of the 68 studies, 53 included a subjective method, either on its own or in conjunction with an objective one. In 30 studies (44%), a validated method of assessment was used, nevertheless many of the remaining studies failed to report whether methods used were validated.
Overall study results after NTS intervention were classified as having either a positive or negative impact on the study population, based upon the provided data and conclusions drawn by the authors (Table 4). Only six papers reported negative findings, with a decline in measured outcomes following the intervention enacted. This may suggest the presence of publication bias towards positive results.

Limitations of the included studies
Included studies reported several limitations, the most common being the absence of a control group (n = 19). This was followed by a small sample size (n = 16) and the application of the NTS intervention in question to a single institution only (n = 12). Few studies reported on the long-term retention of the intervention undertaken, and this was mentioned as a limitation to 10 studies. Further to this, based on AMSTAR 2 checklist the methodological quality is low, which can be attributed to significant heterogeneity in the studies included.

Discussion
The modern shift of patient care to the biopsychosocial approach demands a coincident change in medical education, particularly in nontechnical skills teaching [94,95]. This review has compiled substantial evidence supporting early exposure of undergraduate medical students to NTS training. Evaluation of included studies has been complicated by the heterogeneity of reported participant and educator demographics, interventions, outcomes, and their assessment. This underscores the need to unify NTS teaching implementation and reporting. To this effect, we propose a singular NTS teaching framework.

Parameter 1: Participants and educators
NTS training was compulsory in the majority of studies included, and we suggest it be a mandatory element of medical school curricula (Table 5). Results showed participants were mostly in the clinical years of their studies, however it is unclear whether introduction of NTS training in clinical versus pre-clinical years has particular advantages. Few studies specified students' socio-demographics and previous experience in NTS, complicating appraisal of the important link between such and NTS acquisition [96][97][98]. This hinders our efforts to optimize future course designs, particularly if neglected student minorities exist undetected by the current literature. Similarly, the optimum number of students per intervention cannot be established reliably due to: 1) variation in group size dependent on the educational tool used and 2) variation in the student to tutor ratio. Acknowledging these ambiguities, our framework helps overcome them by standardizing the reporting of results such that future studies can be compared more accurately (Table 5).
Educator information is also poorly represented, with most studies failing to provide sufficient analyzable data. Given the pivotal role of educators in NTS training, we propose to alleviate any potential disparities in educators' backgrounds with a uniform pre-course training ( Table 5).

Parameter 2: Intervention
The vast number of different NTS intervened for in the analyzed studies reinforces our argument to unify training by defining a core set of NTS for the undergraduate curriculum. To accomplish this, we propose a 'frequency-based' approach, selecting the most common NTS and training strategies from the existing evidence base. This assumes educational demand has spurred coincidental development of relevant NTS study interventions. For example, the frequent use of communication skills interventions (37 times) may suggest recognition of the importance of this NTS and students' need for improvement in this domain. Indeed, the literature demonstrates a shift away from 'classic'  areas of intervention previously incorporated in surgical, anesthetic and health education frameworks, towards skills such as empathy and breaking bad news [3,4,99]. Whilst this approach guarantees catering for the current educational needs, it's effectiveness is dependent upon continued monitoring of NTS learning. Currently, the majority of reported interventions are limited to domain-specific training, focusing on singular aspects of the medical profession such as the operating room (OR) [3,4,100]. This observation highlights a future pathway for NTS training, where movement out of the OR and into surgical wards can be mirrored in other areas of the medical profession [101].
Present educational methods combine traditional and novel training tools. Simulation-Based Learning (SBL) has been implemented in a variety of medical disciplines [102,103], with recent evidence of its role in NTS training [104][105][106][107][108][109]. Despite its sparse use in the selected studies, evaluating its time advantages and high fidelity, we consider SBL a fundamental part for all future NTS interventions [103,110,111]. Also, we appraise the increasing use of feedback from SP noted in our study. Though a recent review [112] was inconclusive regarding feedbacks' effectiveness in aiding medical students, it has been suggested that feedback can positively affect communication skills when used in conjunction with other education tools, warranting further investigation [113]. Taking into consideration the above points, we encourage use of simulation in NTS training in conjunction with other widelyaccepted traditional or non-traditional methods of teaching (Table 5).
To the best of our knowledge, no existing studies correlate NTS active intervention duration to participant outcomes. However, most authors support a longitudinal model, involving a course extending over weeks or months as opposed to singular timepoints [114,115]. Positive outcomes for participants are generally reported by studies, independently of intervention time or frequency. This may indicate that duration and frequency are not pivotal factors towards the success of NTS training. However, we exercise caution when drawing scheduling suggestions, as our calculation of study duration was based on assumptions (see results), whilst publication bias towards positivelyskewed results may also exist. Nonetheless, due to the time constraints pre-existing in medical school curricula, we propose that the ideal NTS intervention duration should verge towards our average of 20-30 hours, distributed longitudinally.
Whilst most studies show NTS training leads to participant learning, retention of skills can degrade over time [43,78,116]. Maintenance of high quality NTS throughout a physician's career has the potential to improve clinical outcomes and care standards; hence, we argue it is vital to periodically revisit NTS throughout undergraduate education. This also addresses the natural decline in students' NTS, including empathy, experienced over the duration of medical school [117][118][119]. This can be achieved by scheduling regular NTS training over the academic years, building upon previous NTS taught, whilst increasing student proficiency through further, more complex exercises [120].

Parameter 3: Outcomes
The extensive variation in learning outcomes reported in the reviewed studies constituted a barrier to interpreting the relative impact of the NTS interventions on students. To simplify this, we developed a "triad of outcomes", unifying measured outcomes in three areas: 1) knowledge, 2) skill performance, 3) attitude towards skill. Whilst this provides some basis for analysis, it is inherently limited in attempting to normalize outcomes disparate as the NTS they refer to. To eliminate this in future studies, we propose the ecumenical utilization of unified outcomes such as our "triad", on the footsteps of the subdivision of NTS in tools such as the surgeons' NOTSS and anesthetists' ANTS [2,3].

Parameter 4: Nature and method of assessment
The commonest rationale for study exclusion in this review was the lack of student assessment before and after the NTS intervention (Fig. 1). Also, many studies failed to utilize a control group not participating in any NTS intervention (n = 50). These limitations denote the inherent complexity of implementing a well-designed and controlled NTS intervention trial to a cohort of medical students. This is chiefly impeded by preexisting time constraints and the educational disadvantage of offering NTS training to only a cohort of students. Hence, variables such as students' previous exposure to NTS training and to other curricular activities that may contribute to their NTS development cannot be compounded for. To obviate this shortcoming in future studies, we suggest the use of baseline and post-intervention assessments, which can also be "formative" assessments to students [121,122]. We also suggest the use of control groups, which can receive  [76] Interviewing skills Long Yes 60 ✓ Tsai et al. [92] Interviewing skills Short Yes 27 ✓ Usherwood [59] Interviewing    the NTS intervention subsequently, thus also permitting improved tutor to student ratios (Table 5). Whilst many studies evaluated the same NTS, different assessment methods were often used, complicating the comparison of training effectiveness. To address this, we propose that future studies utilize unified assessment methods. A possible solution is to reaffirm assessment methods already commonly utilized, such as the Communication Skills Attitude Scale and the Jefferson Scale of Physician Empathy [123,124]. Amongst the reviewed studies these tools were the most frequently occurring, suggesting they may be easily extended to all future NTS intervention studies [4]. Importantly, these scales have been extensively validated, demonstrating good test-retest reliability and internal consistency, both in their English and non-English translations [88,[123][124][125][126][127][128][129][130][131]. Likert scales were also utilized by a majority of the included studies. This suggests a second route to implement a common NTS assessment method, mimicking tools such as the NOTSS and the ANTS, grading NTS by Likert scales in key domains [2,3]. Whilst an attractive option, the success of this unifying method is dependent on reaching a consensus on the domains to be included. Moreover, subdivision of NTS into domains could allow assessment through feedback and self-evaluation, methods invaluable to capture the social skills involved in many NTS [4]. Applied to our proposed "triad" of outcomes, attitude towards skill could be best assessed via a subjective scale, such as a Likert scale or one of the validated tools aforementioned, whilst skill performance and knowledge could be measured via more objective tools, including multiple-choice questions, as exemplified by some included studies [42,45,64,84] (Table 5).

NTS in the transition from a personal to an organizational level
NTS should be viewed as a catalyst for improving personal competence and performance in multi-disciplinary teams and settings. Such personal gains lead to improved organizational performance and allow the individual to 'mature' and form a key prerequisite for organizational gains. We suggest that effective plurality can lead to innovation, resilience, sustainability, collaboration, productivity and growth (Fig. 4) all key principles of high quality and cost-efficient care. On the basis of evidence from the compiled studies, and considering the damaging repercussions in the absence of such elements, we strongly support the early introduction of NTS training in undergraduate medical education.

Limitations
We have performed a systematic review of the literature on the MEDLINE database only. Albeit 68 articles were included for full text retrieval, this is a restriction of our study, and future work should include studies in other databases. Furthermore, the intrinsic diversity of NTS teaching studies included prejudiced the use of currently approved tools for appraisal of quality of evidence. Whilst we believe this would not have altered our conclusions, we recognize this as a limitation which should be addressed in further developments of our framework.

Conclusion
It is evident that integration of NTS training in undergraduate education continues to face challenges, illustrated in part by the huge increase in emerging studies but more importantly by the striking inconsistency between them. We propose a unified framework for NTS training, with the objective of guiding future research, facilitating comparison between interventions, and spurring the creation of a standardized NTS course. Although this review focuses solely on personal gains from the medical perspective, coordinated efforts to achieve similar gains in allied healthcare professionals are expected to trigger multiplier effects. Future studies will be required to elucidate the current theories on NTS teaching, in the endeavor to enhance the education of tomorrow's doctors.

Provenance and peer review
Not commissioned, externally peer reviewed.

Ethical approval
Not required.

Sources of funding
Self funded.

Author contribution
MN drafted the manuscript and is the lead author; parts of the manuscript have been drafted and edited by LC and IT. MN, LC, IT, JH performed data extraction and synthesis of evidence. MS, AP GT, TA are the senior authors with input in several steps of the authorship. MS, MN and AP conceived study, search strategy and data synthesis. All authors have approved final manuscript.

Conflicts of interest
None declared.

NA.
Research registration number reviewregistry608.

Guarantor
MS and AP are the guarantors of this work. MN is the lead author.

Availability of data and material
The datasets supporting the conclusions of this article are included within the article (and its additional files). The raw data extraction from the reviewed studies can be shared by the corresponding author upon reasonable request.