Situational On Call Survival (SOS) Guide-qualitative assessment of a novel peer induction initiative

Background In the UK there is a 6% increase in the mortality of emergency patients in the week following ‘Black Wednesday’. On this day, there is an abrupt change in the dynamic of medical teams; medical students become Foundation Year 1 doctors (FY1s) and existing junior doctors step up a grade in seniority. Attempts have been made to develop induction initiatives to better prepare FY1 doctors for their new role. Methods FY1 focus groups noted a lack of dedicated teaching in existing induction programmes from the unique perspective of a FY1. To address this, the SOS Guide was developed; a one day course, created and taught exclusively by FY1s for FY1s. The course has been delivered five times, in four hospitals, over two consecutive years to 175 FY1 doctors. Cascade teaching was employed to recruit and support tutor teams of FY1s to create their own hospital specific SOS Guide. All attendees completed pre/post-course questionnaires. Results Qualitative questionnaire feedback demonstrated high satisfaction and a significant average increase of 26.8% (95% CI:2.25-3.11, p=<0.0001) in the average confidence levels of FY1 attendees. Conclusion This innovative initiative should continue to be developed in order to maximise benefits to FY1s; students and teachers alike. Lane V, White J MedEdPublish https://doi.org/10.15694/mep.2016.000038 Page | 2


Introduction Problem and background
In the United Kingdom, the phenomenon of "Black Wednesday" makes media headlines every year. This is unsurprising as the risk of emergency patients dying in the week following Black Wednesday is 6% higher than the previous week in July (Jen, Bottle, Majeed, Bell, and Aylin, 2009). This significant annual increase in mortality has been attributed to the abrupt change that occurs in the dynamic of every single medical team in the country; final year medical students become Foundation Year 1 (FY1) doctors and every other existing junior doctor steps up a grade in seniority over the course of a single day (Jen et al, 2009;Young et al, 2011;Sukcharoen, Matthew, and Van Hemel, 2014).
The transition from medical student to FY1 is undoubtedly an enormous challenge; many doctors are shaped personally and professionally by the experiences they have, particularly whilst on call. One way of easing this transition and reducing the number of adverse events in the first week of August is to ensure new doctors are familiar with their environment, confident in the initial steps needed to manage emergency situations and comfortable in calling for help (Sukcharoen et al, 2014). To achieve this attempts have been made to develop appealing induction initiatives for new FY1 doctors.
In 2013 the Department of Health introduced a compulsory four day shadowing period for all new FY1 doctors, immediately prior to changeover day. Hospital trusts were asked to provide an experience that not only allowed new doctors to become more familiar with their working environments but also generated an improved understanding of their role, responsibilities and limitations (NHS Employers, 2014). No national curriculum was developed and anecdotal evidence from across the UK suggests that experiences of this induction programme vary. Some positive outcomes have been identified, for example the University Hospital Bristol NHS Foundation Trust demonstrated that following the introduction of the new shadowing programme, the number of self-reported critical incidents identified by FY1 doctors reduced by 45% (Sukcharoen et al, 2014).
Many hospitals have generated their own induction programmes to complement the mandatory shadowing period. A review of the induction initiatives in the Anglia region found that the most frequent score given to denote the quality of the induction was 3/10. Online induction modules, which were commonly incorporated, were seen as "long winded and annoying". The overall cynical impression of the programme is summarised by the comments of one junior doctor; "that the aim of the induction was to protect the Trust, rather than to help patients or make me feel more welcome" (Stanton & Lemer, 2010).
An example of a successful hospital induction was designed by junior doctors and delivered over the course of one day by a multitude of different healthcare professionals including consultants, pharmacists and specialist nurses. The aims of the day were compiled by studying the needs identified by FY1s both in local feedback and in a national survey. Substantial improvements were noted in confidence levels in many areas including; knowing how to request investigations, understanding the equipment, prescribing insulin, and dealing with unwell patients (Sukcharoen et al, 2014).

Focus group
In 2014, the original SOS team, consisting of five FY1 doctors, held focus groups with ten of their peers, working throughout the West Yorkshire Deanery, to reflect upon and discuss the deficiencies they experienced in their FY1 induction programmes. It was noted that although the existing inductions made reference to the management of common on call situations, few of them outlined the specific roles and responsibilities of new FY1 doctors and none of them addressed these important details from the unique perspective of a junior doctor. It was unanimously agreed that expanding the role FY1 doctors play in creating and delivering the induction programme, would act to increase the value of the experience for new-starters and indirectly, might also act to improve patient safety in August.
Having experienced the lack of dedicated teaching firsthand and having recently witnessed multiple on call scenarios from the unique perspective of an FY1, the SOS team were in an ideal position to develop a teaching initiative to tackle these insufficiencies. The SOS Guide was subsequently created; a novel peer-delivered teaching experience that focuses on common on call situations and passes on key, hospital specific learning points thought to be fundamental to being successful and safe as a newly qualified doctor.

Population and setting
The SOS Guide is designed and delivered exclusively by outgoing FY1 doctors for the new cohort of FY1s starting in their hospital trust. It has been delivered five times to 175 new FY1 doctors across four hospital sites in West Yorkshire, by thirty FY1 tutors, over 2014/15. For each course, recruitment was conducted six months in advance using an electronic mailing list of all of the new FY1 doctors due to start working in the hospital in August.
In order for the SOS Guide to be of maximal benefit to new starters it is designed to be delivered in the final days of the medical student summer holidays, immediately prior to the compulsory FY1 shadowing period and the mandatory hospital induction programme. Permission to hold the course in the educational department was given by each of the participating hospital trusts. FY1 attendance was voluntary and on an extracurricular basis.
No ethical approval was required for the SOS Guide.

Course design: phase one (2014)
In 2014, the first SOS Guide was designed and delivered by the original SOS Team for Mid-Yorkshire NHS Hospital (MYH) (40 FY1 attendees). Research into different methods of teaching and reflection upon personal experiences of medical education led to a decision to combine small group, interactive seminars with short lectures. This approach aimed to cover large amounts of information in a way that would appeal to multiple individual learning styles (Murphy, Gray, Straja, and Bogert, 2004). The SOS Guide 2014 consisted of; Small group, interactive seminars built around the hospital specific personal experiences of the FY1 tutors. Topics included; chest pain, shortness of breath, abdominal pain, gastrointestinal bleeding, and sepsis. Where possible the hospital specific IT systems, charts, guidelines and protocols were utilised in simulated situations.
Short, targeted, evidence-based lectures that focused on the following commonly encountered, often confusing topics; prescribing IV fluids, electrolyte abnormalities, falls, and an interactive FY1 top tips session. The SOS Guide buddy handbook was provided free of charge to all attendees. The first edition of this concise, pocket sized handbook covered the recognition, investigation and management of thirty-seven key on call scenarios. Guidance was divided using an innovative traffic light system which identified the level of care expected by each grade of doctor (FY1/SHO/Register) and made it clear when to call for help. Hospital specific practical details such as maps, bleep numbers, logistical information for on calls and antibiotic/VTE guidelines were also included.
A short tour of the hospital from an FY1 perspective to aid orientation.
In order to create a friendly environment for social networking, breakfast and lunch were provided and an informal social event was organised for the evening.
Prior to the delivery of the course, all FY1 attendees were required to complete a qualitative pre-course questionnaire. This included demographic data and an assessment of confidence in a variety of skills using the Likert scale (score from 0-10). In order to reduce and assess acquiescence bias, some questions were asked twice in both a positive and a negative format. To ascertain a broader understanding of the fears and expectations of the FY1 attendees, open questions were asked.
Feedback was collected following each session. This provided personalised feedback on the performance of each tutor that could be used for reflection and personal development.
The effect of the course was evaluated using a qualitative post-course questionnaire completed at the end of the day. It included an assessment of self-perceived confidence in the skills previously measured in the pre-course questionnaire and satisfaction ratings for different components of the day.
Prior to delivery, the original SOS Guide was piloted to a team of FY1 doctors. This provided an opportunity for feedback, which was acted upon accordingly to ensure quality control.
To minimise barriers to attendance, the course (including the buddy handbook, refreshments and the social) was delivered free of charge to all attendees. All costs were covered by the Educational Department at MYH.

Course design: phase two (2015)
Building on the positive reception of the original SOS Guide, in 2015 the course was disseminated throughout West Yorkshire. It was taught to the new cohort of MYH (43 FY1 attendees), as well as to FY1s in three new hospital trusts; Airedale General Hospital(17 FY1 attendees), Bradford Teaching Hospitals NHS Foundation Trust (36 FY1 attendees) and Calderdale and Huddersfield Foundation Trust (39 FY1 attendees).
The principle of cascade teaching was used to spread the course throughout the region and ensure the maintenance of the original ideology; that the SOS Guide should be created and taught exclusively by FY1s for FY1s. Nine months in advance of the course, a faculty comprising of three key members of the original SOS team recruited new FY1 tutors by sending an advertisement and an application form to every FY1 working in the participating hospitals using an electronic mailing list. Applicants were required to outline the reasons for their interest, the topics they would like to teach and their experience of medical education to date. For each hospital 5-10 of the highest quality, most enthusiastic applicants were selected to form a new SOS team. Each new team benefited from a wealth of different teaching experiences and was headed by a team lead who was recruited to act as a point of contact between the original faculty and the new SOS teams. Team leads had a copy of the entire day's activities and were responsible for covering sessions in the unlikely event that their tutors did not attend.
Each new SOS team developed their own hospital specific version of the SOS Guide, using the same format and Lane V, White J MedEdPublish https://doi.org/10.15694/mep.2016.000038 Page | 5 principles as the original. They were supported by the SOS faculty, who provided advise (including lessons learnt and suggested improvements from the 2014 course), time-keeping, ensured quality control and updated the SOS Guide buddy handbook (which was modified to include a different hospital specific chapter for each trust and a wider range of topics). Each of the courses were cost neutral; funding for the direct costs of the 2015 courses was supplied by the Educational Department at MYH and by sponsorship from the Medical Defence Union (£525). Again, attendance at all courses was free. Each of the participating hospitals provided rooms, simulation resources (such as SimMan) and hospital specific resources such as drug charts.

Data analysis
All feedback data was collected using standard pre/post-course questionnaires allowing collation and analysis of the entire data set. For the five courses, the overall average score for each question was calculated and a comparison was made between pre/post-course questionnaire results.
The total number of responses from each course for the pre-course questionnaire, with the average responses to each question can be seen in Table 1 and Table 2, respectively. The total number of responses from each course for the post-course questionnaire, with the average responses to each question can be seen in Table 3 and Table 4, respectively.

Baseline measurement using pre-course questionnaire
The pre-course questionnaire was completed by 94% (n=165) of FY1 attendees (refer to Table 1 and Table 2 for full  results) All question responses were rated out of ten. On average, prior to attending an SOS Guide course, new doctors rated their confidence about; becoming an FY1 as 4.67, making an ABC assessment of an acutely unwell patient as 5.95, knowing when to escalate for senior help as 5.60 and prioritising jobs and patients whilst on call as 5.41. They rated their ability to; order urgent investigations for unwell patients as 4.79 and prescribe unsupervised as 5.25. The overall level of anxiety on starting FY1 was 6.00. Although following graduation, to comply with the GMCapproved curriculum 'Good Medical Practice', FY1 doctors are expected to be confident and competent in all of the skills assessed in the questionnaire, in no area did the average level of perceived ability exceed 6.06 (General Medical Council, 2016).
In the open questions, the main concerns identified were similar across all hospital trusts. They included; a perceived lack of support from seniors, the high risk of making a mistake, the possibility of missing a sign, night shifts, the chance of making a fatal error, prescribing (including e prescribing), not being able to make a diagnosis, attending crash calls and not being organised enough.

Post course questionnaire
The questionnaire was completed by 94% (n=165) of attendees (refer to Table 3 and Table 4  to seniors, a 15.1% increase in ability to prioritise jobs and patients whilst on call, a 19.5% improvement in ability to order investigations within that hospital trust and a 16.1% increase in the perceived ability to prescribe unsupervised. Detailed feedback was collected after every seminar/lecture and this was compiled for the benefit of each tutor, to allow them to develop their teaching skills and reflect on their experiences. Student satisfaction with the SOS Guide was extremely high. Scores were given out of 10. The average overall satisfaction rating for all five courses was 9.07. Course organisation was scored particularly highly at 9.41. The SOS Guide buddy handbook was very well received and when students were asked to score how useful they felt it would be, it received 9.20. Every student who attended the SOS Guide said they would recommend it to their peers and it was unanimously agreed that the course should continue to be designed and taught by FY1 doctors.
Suggestions for improvement identified in the 2014 questionnaire, including; increased use of the IT systems, more simulation and the use of SimMan, were acted upon and incorporated in the 2015 courses.
An additional two month post-course questionnaire was completed by 70% (n=28) of the FY1 doctors who attended the original MYH SOS Guide in 2014. Overall, 100% agreed that the SOS Guide had improved their confidence over starting FY1 and felt the course had reduced their anxiety levels. The SOS Guide buddy handbook was used regularly by 92% of respondents. Many FY1s (69%) expressed their interest in being involved in the next year's SOS Guide.

Discussion
The SOS Guide was designed to rectify the deficiencies recognised in the current mandatory induction programmes that exist for new junior doctors in the UK. The courses were held before the official hospital induction and therefore supplemented but did not interfere with compulsory activities. The course aimed to indirectly improve patient safety in August through the provision of dedicated teaching from outgoing FY1s to new FY1 starters, that focused on the hospital specific roles and responsibilities of FY1s, with particular regard to the approach, management and escalation of common on call situations. It aimed to produce measurable increases in the confidence levels of new starters in multiple, important skill domains that are outlined in the GMC-approved 'Good Medical Practice' curriculum as being essential to the practice of all doctors in the UK.
The SOS Guide achieved its stated aims; qualitative data from the feedback of 165 FY1 attendees demonstrated a significant increase in the overall confidence levels of all attendees, with improvements noted in all measured domains. Detailed instruction on the hospital specific management of common on call situations by FY1s was covered in the small group seminars. The use of hospital specific resources in interactive simulated scenarios throughout the day allowed FY1 attendees to become familiar with the paper-work and systems used in their new hospital and ask pertinent questions in a relaxed environment. The SOS team delivered short lectures on topics that had caused anxiety in their cohort and were therefore likely to cause anxiety in the new cohort, such as the prescription of IV fluids. The messages of the day were reinforced by the provision of the SOS buddy handbook, which included hospital specific orientation advice and a succinct summary of the recognition, investigation and management of on call situations. FY1s reported that the traffic light system employed by the books made them particularly useful as they clearly identify the specific roles and responsibilities of each grade of doctor and make it very clear when FY1s should escalate and call for help. Many FY1s who attended SOS courses commented that the SOS Guide buddy handbook was their constant companion throughout their first year. As only FY1 doctors were involved in teaching the courses, attendees had the opportunity to explore the hospital and network with their new colleagues in a relaxed and friendly environment. This was promoted by the provision of free refreshments and Lane V, White J MedEdPublish https://doi.org/10.15694/mep.2016.000038 Page | 7 evening social events. These aspects of the course were particularly beneficial to the new FY1s who were moving into the region and did not yet have a support network in place in West Yorkshire.
There was no difficulty in recruiting enthusiastic FY1 tutors; following the initial recruitment e-mail, the SOS faculty were inundated with applications from FY1s who were willing to volunteer their personal time, for free, to participate. No problems were encountered with tutor attendance or availability; tutors were recruited many months prior to the course and had ample time to discuss their involvement with their supervisors in order to arrange clinical cover for their duties. In some cases, tutors were so keen to be involved they took a day of annual leave in order to teach. Tutors reported their interest was due to the multiple invaluable opportunities the SOS Guide provided for them to get involved in medical education, including experience on the logistics of setting up a regional teaching course and most importantly, the chance to deliver meaningful teaching in a structured, non-threatening, supported environment. Tutors valued the dedicated individual feedback they received, following each session, on their personal performance. As well as using this feedback to develop further as medical educators, FY1 tutors were also able to use their certificates of participation to boost their applications for Speciality Training. This model of cascade teaching could be utilised in multiple ways to encourage an early interest in medical education in the next generation of young doctors.
The main challenge of the SOS course was time. In 2014, the idea for the course went from concept to completion in only two months. This required a huge input from the entire team. In 2015, lessons were learnt and preparation started nine months in advance. Recruiting tutors was easy. Keeping those tutors on schedule as they developed a teaching course that broadly followed the original format of the SOS Guide was more challenging. As many of the tutors never attended the original SOS Guide, it was difficult to clearly pass on the underlying concepts. It was quickly realised that it was unsustainable for the original team to troubleshoot. This problem was overcome by allocating team leads and having regular contact with them.
Several possible limitations to the design of the SOS Guide have been identified. It should be noted that the data collected is purely qualitative; it represents the perceived ability and confidence of FY1 doctors and does not demonstrate their true level of competence. Although it is hoped that increased confidence in FY1s will translate indirectly to improved patient care, no data was collected to demonstrate this. As the SOS Guide is created and taught by FY1 doctors, there is a high turnover of tutors. Although personal development with regard to teaching is possible, tutors are not able to improve their SOS session by having a second attempt. It is therefore imperative that the SOS faculty maintains a degree of continuity in the management of the SOS Guide, ensuring lessons learnt from the previous years are passed on. A third limitation is the lower response rate for the two month post-course questionnaire following the 2014 course (70%). As the tutors of the original course had moved to a new trust with the August changeover, the survey was delivered remotely using SurveyMonkey. Despite attempts to boost rates by e-mail reminders and the provision of a course certificate to all responders, higher levels of response were difficult to achieve.
In order to run the SOS Guide sustainably in the future over multiple sites to improve the induction experience of junior doctors, the role of the team lead needs to be developed further. If possible a training day should be held in order to give each team lead: a detailed plan on how to organise their own SOS Guide course, a timetable of when each component needs to be completed (for example; booking rooms, organising food, recruiting tutors, and advertising for students), useful templates (for example; posters, e-mails, example lectures, and seminars), and feedback from previous courses. The buddy handbook could be transferred onto a Smartphone application, to improve its accessibility on the ward and reduce printing costs. Regular contact should be made by the SOS faculty to offer support. In order to ensure quality control and give a chance for feedback, a run through of the entire course should be held one month prior to the delivery date. To further refine this concept, feedback should be collected Lane V, White J MedEdPublish https://doi.org/10.15694/mep.2016.000038 Page | 8 from all the FY1 tutors about their experiences.
The true value of the SOS Guide could be reviewed in greater depth through the collection of objective qualitative data on FY1 knowledge and performance. To achieve this, a short examination could be delivered before and after the course to assess attendee knowledge on hospital specific systems and the recognition, investigation, management and escalation of common on call scenarios. The two results could be compared and statistically analysed to provide information on the qualitative impact of the course on FY1 performance.

Conclusion
Existing induction programmes for FY1 doctors fail to adequately identify the specific roles and responsibilities of new doctors and make no attempt to address these important details from the unique perspective of an FY1. The SOS Guide was created to tackle these insufficiencies. It is a one day, peer-delivered, hospital specific induction initiative that focuses on how to manage common on call situations from the perspective of a junior doctor. Designed and taught exclusively by FY1's for FY1's, it has been delivered successfully five times, in four separate locations, over two consecutive years. Consistently, feedback has demonstrated a significant increase in the overall confidence levels of FY1 attendees, with improvements demonstrated in all measured domains. The principle of cascade teaching was used to disseminate the SOS Guide throughout the West Yorkshire region and provide invaluable experiences and opportunities in medical education for thirty FY1 tutors.
This sustainable, innovative teaching initiative should continue to be developed, in conjunction with the existing induction programmes, to maximise benefits to FY1s; students and teachers alike.

Take Home Messages
The SOS Guide is a sustainable, innovative induction initiative designed and delivered by FY1s for FY1s. It is a one day, peer-delivered, hospital specific course that focuses on how to manage common on call situations from the unique perspective of a junior doctor. It consists of tutorials, a buddy handbook and opportunities for social networking and hospital orientation.
The SOS Guide has been delivered five times, in four seperate locations over two consecutive years. Results consistently demonstrated a significant increase in the confidence levels of new doctors, with improvements noted in all measured domains.
The principle of cascade teaching was used to disseminate the SOS Guide throughout the West Yorkshire region and provide invaluable experiences and opportunities in medical education for thirty FY1 tutors.