What do trainee doctors and their trainers say about workplace based assessments? A survey of UK based junior doctors and their supervisors in a rural hospital

Purpose: To determine the opinion of recently-qualiﬁed trainee doctors and their assessors on workplace based assessments in postgraduate training in one UK rural hospital. Design: Two 10-item questionnaires (one for trainers and one for trainees) were completed anonymously by 22 trainers conducting the review and 17 trainees (postgraduate year 1 & 2) in the Northern Devon Hospital. Results: Most supervisors had formal medical education training (68%) and in giving structured feedback to trainees (82%). Most trainers (81%) spent 10 – 20 minutes on the assessment and almost all (95%) conducted it through face-to-face meeting. Most trainers (67%) believed that the assessment was useful for facilitating feedback, determining engagement with the training programme and assisting a poorly performing trainee (52%). Eighty-two percent of trainers believed the assessment was a good use of their time. Poor IT service and lack of time (50%) were believed to be the biggest challenges for trainers. Trainees rated the assessment useful for engagement with the training programme (94%), assessment of overall performance (88%), recording training and achievements (88%) and allowing structured feedback (82%). Most trainees believed feedback helped boost their conﬁdence (58%) and made them reﬂect on their performance (52%). A majority (52%) found it easy to arrange the assessment and many were satisﬁed with the process (76%). Conclusion: This study provides insight into an important workplace based assessment for the ﬁrst time. A majority of trainees and trainers viewed the assessment useful. This survey suggests that good IT infrastructure and


Introduction
As part of Modernising Medical Careers (MMC) programme for postgraduate training in the UK, since 2005 newlyqualified doctors enter a two-year structured Foundation Programme (FP), during which they must complete six 4month placements. [1][2] In order to progress through the FP, foundation doctors must undergo mandatory portfoliobased reviews to assess performance in the workplace at the end of each placement. These competency-based summative assessments take account of a wide of range of skills, behaviours and performance based on the FP curriculum. These include workplace-based assessments (WBAs) known as Supervised Learning Events (SLEs) [mini-clinical evaluation exercises (mini-CEXs), case-based discussions (CbDs) and direct observation of procedural skills (DOPS)], 360-degree multi-source feedback from colleagues, trainer's own observations and records of trainees' other continuing professional development work. 3 During the assessment meeting an online report must be completed on an online system called e-portfolio (figures 1-2).
There are two assessments at the end of each placement, one with clinical supervisor and one with educational supervisor. Although there is general guidance on how these assessment meetings should be conducted and used for professional development, [3][4] in practice there is likely to be considerable variation. At the start of their posts, each foundation trainee is allocated an educational supervisor for the whole 12-month period who sets objectives and overviews the overall progress of the trainee. In addition, each trainee has a clinical supervisor in each placement who is responsible for training, support and assessment for the 4-month period.
Previous studies on this area have focused on the role and impact of WBAs on trainee doctors' education and performance [5][6] and some have reported trainees' views on the use and usefulness of portfolio-based learning 7-8 and assessments such as the Annual Review of Competence Progression (ARCP) in specialty training 9 . There is no study, to our knowledge, that has consulted either trainees or trainers for their views on how end of placement assessments are conducted and used in doctors' professional development in the first two years post graduation. The aim of this study is to explore foundation trainers' and trainees' perspectives on the mandatory workplace assessments of overall performance.

Method
A 10-item questionnaire was sent to all foundation trainers (clinical and educational supervisors) undertaking the end of placement assessments and another 10-item questionnaire was sent to all doctors in their first two years post graduation (Foundation year 1 & 2) in the Northern Devon Hospital, following the final end of placement assessments in the summer of 2015. Questionnaires were completed anonymously online.
Trainers were asked about whether they had training for their roles as supervisors, providing feedback to trainees and for their views about the process of the assessment. Trainers were also asked how useful they found the assessment and the challenges they faced in completing it using a four-point Likert scale with textual anchors. Three questions collected demographic data.
Trainees were asked to rate the usefulness of the assessment and feedback on a three-point Likert scale with textual anchors. In addition, the questionnaire comprised of items related to trainees' experience of the assessment process, understanding and perception of its significance to them and whether they were satisfied with the outcome in each placement. One question collected demographic data. Data from both questionnaires were analysed using simple descriptive statistics, frequencies and percentages on Microsoft Excel SpreadSheet.
Ethical approval: This study was a review of a current process; therefore, it did not require formal ethical approval. Electronic surveys were completed anonymously by the participants, which was taken to indicate consent to participate. There was no need to access confidential data. All information about the study was included in the introduction page of the survey.

Results
There was 61% (22/36) response rate amongst trainers to the questionnaire and 48% (17/36) response rate amongst the foundation trainees. Most trainers were male consultant (55%) and most were physicians (63%). Thirty two percent were surgeons and one was both physician and surgeon. Most trainee respondents were female UK graduate (53%), there were two female international graduates and one male international graduate.

Trainers' Questionnnaire:
All 22 respondents had undertaken end of placement assessments as clinical supervisors and 14 out of 22 also as educational supervisors. Sixty-eight percent of trainers had training for their roles as clinical and educational supervisors and 82% in giving structured feedback to foundation trainees. Most trainers (67%) believed assessments were useful or very useful for facilitating feedback and determining engagement with the Foundation Programme (FP) and a majority reported (52%) that it would assist a poorly performing trainee. Lack of access to online eportfolio due to poor internet and IT service and lack of time occasionally or often (58%) were the biggest challenges reported by the trainers. Most trainers (81%) spent 10 -20 minutes on assessment with the trainee and all conducted the assessment through face-to-face meeting. Ninety percent of trainers believed the assessment was a good use of their time.

Trainees' Questionnnaire:
Foundation trainees rated the assessment useful for engagement with the FP (94%), assessment of overall performance (88%), recording training and achievements (88%) and allowing structured feedback (82%). Most believed feedback helped boost their confidence (58%) and made them reflect on their performance (52%). Most trainees regarded the assessment moderately important (53%) and a majority (52%) found it easy to arrange it with their supervisors and many were satisfied with the process (76%). For most trainees the assessment process was before end of placement (67-100%), conducted through face-to-face meeting (93-100%), with more than 5 minutes spent on feedback and discussion (75-93%) across the three placements for the whole year.

Views on the process
It is surprising that in spite of reported lack of time by trainers, most spent more than 10 minutes for the assessment through a face-to-face meeting with the trainee. In addition, the trainees did not find it difficult to arrange an assessment meeting before the end of placement (table 2). Furthermore, the time spent on feedback and discussion of performance with the trainee also reported to be more than 5 minutes. This is despite the reported poor IT and

Perspectives on the outcome and usefulness
Most trainees regarded the assessment moderately important (53%), this is in spite of the fact that a large number rated it useful for a wide range of professional development domains. The perceived level of importance of this assessment mechanism is significant as it may determine trainees' engagement with the process and affect the outcome. One reason may be lack of awareness of the role the assessment plays in progression through the foundation programme (FP) and satisfactory outcome at the end of the year.
Almost a quarter of trainees were never given any plans for future development during the assessment feedback and discussion by the trainers; however, the proportion who reported that it was given occasionally or often increased in the second and third placement (table 2). There is some overlap between trainers' and trainees' views as both reported the assessment useful in determining engagement with the FP, facilitating structured feedback and discussion on the performance in the workplace. It is significant that almost all trainers regarded the assessment as good use of their time in spite of the challenges discussed below.

Challenges and issues
Although a majority of supervisors reported to have the required training for their roles, a significant number (32%) did not have any formal training to meet the GMC and Academy of Medical Educators standards 10 (table 1). It is therefore important to address this aspect to ensure a consistent training experience by all foundation trainees. Furthermore, support for trainers could also address problems with 'lack of time' as reported by a significant number of trainers (more than 50% as occasionally or often). Protected teaching time for trainers may be a useful step in improving the quality of supervision, training and feedback. Poor IT infrastructure such as slow internet connection, incompatible software, missing login details for access to online system (eportfolio) and other training material are significant problems reported by the trainers (table 1) that could affect the validity of the assessment mechanism. Some of these challenges including consultants computer literacy are issues that have previously been reported by other studies 9 .

Study limitations and further work
The authors acknowledge that the study provides a snapshot of recently-qualified (foundation year 1 & 2) doctors in one hospital; therefore, the results may not be generalisable to a wider population. The second important limitation is that although the response rate is reasonably good for this kind of study, the sample size is small, which could potentially skew the results. Therefore, we do not draw firm conclusions from this opinion survey. Nonetheless, the survey provides valuable insight into previously unexplored summative assessment of overall performance from both trainers' and trainees' perspectives. We believe this survey is not only of interest locally but will be useful to wider including international readership. This is because the assessment mechanisms of overall performance in the workplace is a widely used method in postgraduate training in the UK and elsewhere. However, further regional and national surveys exploring the reliability, consistency and validity of assessments such as this is required for developing and evaluating postgraduate trainees' assessment mechanisms.

Conclusion
Workplace assessments are important in postgraduate training as they determine trainees suitability for progression. This study provides evidence for the first time that both trainers and trainees have positive views of the process leading to more satisfied outcome for supervisors and doctors in training. To better equip trainers to harness the opportunities that the assessment offers better IT infrastructure and protected teaching time for trainers could be considered in improving the quality of this assessment mechanism.

Take Home Messages
The assessment is a good opportunity to provide useful, structured and specific feedback that would improve trainees' performance in the workplace. However, better infrastrutcture such as IT service and more protected time as well as guidance for trainers will significantly improve the quality of the process.

Notes On Contributors
Mohammad Razai contributed to the conception and design of the study as well as with the data collection, interpretation and writing up of this article.
Byron Theron contributed to the conception and design of the study, data interpretation and writing up of this article.