The effect of COVID-19 on medical student clinical skill practice and self-perceived proficiency

Background: The coronavirus disease 2019 (COVID-19) pandemic significantly impacted medical education. This study aimed to determine how COVID-19 affected students’ opportunity to practice core clinical skills across specialty rotations and their self-perceived proficiency at performing these. Methods: Routinely administered surveys of fifth year medical student’ experiences and perceptions of medical training from 2016 to 2021 were analysed. Number of times core clinical skills were performed and self-perceived proficiency of each skill were compared pre- (years 2016-2019) and during-COVID (years 2020-2021). Results: Data from 219 surveys showed a reduction in the opportunity to perform “a cervical screen test” (p<0.001), “a mental health assessment” (p=0.006), “assess the risk of suicide” (p=0.004) and “bladder catheterisation” (p=0.007) during-COVID. Self-reported skill proficiency was also less during-COVID for performance of: “a mental health assessment” (p=0.026) and “an ECG” (p=0.035). Conclusions: The impact of COVID-19 on mental health skills was greatest, potentially due to a shift toward telehealth services and consequent reduced ability for students to engage in consultations. In a time of potential long-term change in the healthcare landscape, it is imperative to ensure adequate opportunity to practice all core clinical skills during medical training. Inclusion of telehealth earlier into the curriculum may benefit student confidence.


Introduction
After its emergence in China, the novel coronavirus disease 2019  was soon declared a global pandemic by the World Health Organisation in March 2020 (World Health, 2020). Many countries worldwide quickly implemented public policies to reduce the spread of COVID-19 including: international/national travel restrictions; closures of workplaces, schools and public transport; public gathering cancellations/restrictions; stay-at-home requirements; contact tracing; mandatory vaccination and facial coverings (Hale et al., 2022).
COVID-19 has significantly impacted all aspects of society, including the education sector. Australian medical schools, like most schools across the globe, were required to rapidly transform and restructure medical education to comply with fast-changing government policies whilst maintaining a quality, valid, reliable curriculum. Online learning, digital teaching and modified assessments (Heal et al., 2022) were fast-tracked to ensure continuity of student education and clinical training.
The full impact of COVID-induced changes to medical education is yet to be quantified. International studies have shown that while some students have welcomed the flexibility of online learning, many felt the disruptions to the curriculum, particularly clinical placements, would negatively affect clinical competencies and learning quality (Dost et al., 2020;Huddart et al., 2020;Martin et al., 2022;Olmes et al., 2021). In Australia, COVID-imposed restrictions resulted in reduced elective surgeries and emergency department presentations, and a move towards telehealth consultations for general practitioners and medical specialists in 2020 (Australian Institute of Health & Welfare, 2021). These changes to health services are likely to have a flow-on effect on medical training programs and education, particularly clinical rotations which cannot be easily replaced by online or virtual environments.
The aim of the current study was to assess the effect of COVID-induced changes to healthcare services and medical education on students' exposure to and ability to practice fundamental clinical skills and their self-perceived proficiency of each skill at the end of the fifth academic year in a rural setting.

Methods
Fifth year medical students at James Cook University (JCU) undertake medical rotations across six core clinical disciplines as a part of standard medical training. Exit examinations are undertaken at the end of 5 th year, and the 6 th year of the course is considered to be a student internship. Surveys of student experiences and perceptions of medical training have been routinely administered to fifth year medical students since 2005. These are completed at the end of each academic year to allow students to highlight strengths and weaknesses in the program. Senior clinicians from each rotation nominated three clinical skills believed to be fundamental to the curriculum (Table 1) and these were integrated into the survey. It should be noted that students were asked to rate the nominated clinical skill based on performance during the relevant rotation, rather than across the entire year e.g., performance of a mental health assessment only during the mental health rotation. Students indicated, using 5-point Likert scales, the number of times the identified fundamental clinical skills were performed in each rotation (1 = never, 2 = Once, 3 = Twice, 4 = three times, 5 = more than three times) and rated their self-perceived proficiency for each skill (1 = very poor, 2 = poor, 3 = indifferent, 4 = good, 5 = very good). This study analyses data from all surveys conducted from 2016 to 2021 (to allow comparison of pre-COVID to COVID data). No identifiable data information was collected to maintain student anonymity.

Data analysis
Data were collated and analysed using Excel (Microsoft Office 365, v2202, 2022. Data on the number of times skills were performed in the associated rotation was not considered linear and therefore were condensed into two categories, less than three times, and three times or more. The decision for the cut-off point for this dichotomy was based on the pragmatic "See One, Do One, Teach One" teaching method traditionally used in medical training in which the three most optimal strategies for retaining information are employed i.e. discussion, practice and teaching (Lalley & Miller, 2007). Hence, the minimum number of times a skill should be practiced with adequate supervision, guidance and critical feedback was considered three. Similarly, self-rated proficiency scores were categorised into "lacks proficiency" (inclusive of scores of very poor, poor, and indifferent) or "proficient" (scores of good and very good). Years 2016 to 2019 inclusive were considered "pre-COVID" and 2020 to 2021 as "COVID".
The chi-square test was used to investigate the association between year of rotation (pre-COVID and COVID) and the categorical variables of number of times a skill was performed (less than three; three or more), and self-proficiency rating (lacks proficiency; proficient). Statistical significance was considered as P < 0.05.

Ethical considerations.
This study was approved by the James Cook University Human Research Ethics Committee (H5595 -approved 6 th May 2014, and H6921 -approved 8 th May 2017). Participants were informed that completion of the survey implied consent for the study, as per the ethical clearance stated above.

Results
A total of 219 surveys were sent out to 5 th year medical students across the 6-year data collection period (pre-COVID n=150) of which 188 were returned (pre-COVID n=134) and REVISED analysed. The overall response rates pre-and during COVID were 89.3% and 78.3%, respectively.
The number of times students were able to perform four of the 18 identified fundamental clinical skills reduced during COVID. These skills were: "perform a cervical screen test" (p<0.001), "perform a mental health assessment" (p=0.006), "assess the risk of suicide" (p=0.004) and perform bladder catheterisation" (p=0.007) ( Table 1).
Despite a reduction in the four skills listed above, self-reported skill proficiency was less during COVID than the four years pre-COVID for two skills only: "Perform a mental health assessment" (p=0.026) and "Perform an ECG" (p=0.035) ( Table 2).

Discussion
This study showed that changes due to COVID-19 were correlated with a reduction in the number of times students were able to perform four of their 18 (22%) fundamental clinical competencies. There was a reduction in self-perceived skill proficiency in two of the 18 skills, and in only one clinical skill, completion of a mental health assessment, was there an alignment, with a reduction in both measures.
Self-perceived proficiency is subjective, and prone to many sources of variation, which may explain the lack of alignment between measures. Performance of all the clinical skills are impacted by variables causing potential confounding, such as temporal changes in the use and appropriateness of the skill, changes in co-ordinating staff, and changes in student cohorts. A reduction in the frequency of bladder catheterisation may have been a reflection of reduced elective surgery due to COVID-19. A reduction in the number of times performing electrocardiograms (ECGs) may have been a reflection in the reduction in normal inpatient care, although this skill could be easily simulated.
Of the four clinical skills performed with reduced frequency in the covid period, cervical screening is the easiest to The decrease in students completing mental health assessments and suicide risk assessments may have been due to a number of factors. First, nationwide restrictions were introduced including social distancing measures and limits on the number of people permitted in hospitals and medical clinics. These are known to impact clinical session time, patient consultation access and patient interaction in rural clinical settings (Hoang et al., 2022). Fewer inpatients in the mental health rotation will have impacted student experience. Third, students' mental health and well-being may have impacted their self-reported proficiency and participation in performing mental health clinical skills, and lack of this data may be considered a limitation. Many causes of COVID-related stressors have been identified including transition to online learning and assessment, relocation and personal and family health and finances (O'Byrne et al., 2021). Psychological distress (stress, anxiety or depression) is associated with poor selfperception of performance as well as impaired concentration levels (Chandavarkar et al., 2007;Yamada et al., 2014). It is also reasonable to infer that students who were vulnerable and experiencing symptoms of mental health disorders may have avoided practising and studying this aspect of their syllabus, as part of a maladaptive avoidance coping mechanism (Ball & Gunaydin, 2022). A previous study reported increased stress and anxiety in a large percentage (84.1%) of medical students surveyed due to disruptions in their medical education from the COVID-19 pandemic (Harries et al., 2021). This was supported by another survey that showed mental health was negatively impacted by pandemic-induced curriculum changes in more than half of medical students (68%) at an Australian university (Lyons et al., 2020). Thus, students' mental health and wellbeing is a key element that may have contributed to the reduction in performance of mental health clinical skills combined with a drop in self-reported proficiency. The setting of health care in general has shifted somewhat towards telehealth, a proportion of which will undoubtedly remain after the pandemic. It is important to emphasise that students must have the ability to participate and practice their skills when consultations are shifted to telehealth and that their skills in this advancing field be built upon early in their medical training. Whilst this is important across all clinical rotations, particular attention should be given to mental health as this has been identified as one specialty in which students perceive to have deficits in opportunity and proficiency. It is also imperative that health professionals and medical educators are aware of the increase in mental health concerns amongst medical students, particularly during the pandemic, as this may to contribute to poor clinical learning experiences and outcomes.
Ironically, at a time in which mental health is a major global concern, medical students were least able to practice the skill of performing a mental health assessment within their mental health rotation and also felt their proficiency was lower in this area than students in the preceding four years before COVID-19 began. The number of times they were able to assess the risk of suicide was also reduced in the two COVID-affected academic years. Thankfully, despite dire warnings of a COVID-19 suicide tsunami, with the exception of Japan (Okada et al., 2022), suicide rates across the world have remained stable during the pandemic (Tandon, 2021). While this is encouraging, suicide rates in general remain high and it is the fourth leading cause of death in people aged 15-29 years (World Health Organization, 2021). The teaching of mental health assessment skills may need to be reassessed in a post-COVID landscape where non-face-to-face methods of clinical delivery of mental health are likely to be retained, at least in part. Simulation may have a role in teaching of mental health skills, as well as reassessing how to engage medical students in telehealth consultation.
There are some limitations of this study. First, as mentioned above, self-perceived proficiency is a subjective measure of skill performance that can be affected by a myriad of factors pertaining both to the student personally and their learning environment. An objective measure of skill performance (e.g., clinical grading for each of the identified skills) would have provided additional information and served as a good comparator to self-proficiency. This was not possible as the surveys were made anonymous to increase student response rates and gain more honest feedback about performance, therefore negating our ability to cross-reference subjective and objective measures of performance.
Second is its lack of generalisability to other Australian regions as it was conducted in a rural North Queensland setting. COVID-19 restrictions in Queensland were less severe and shorter-lived than those experienced in southern states such as New South Wales and Victoria. With fewer and shorter lockdowns, especially in North Queensland, it is possible the impact of COVID-19 on medical clinical training was less than southern states.

Conclusion
Medical student clinical training was negatively impacted by restrictions imposed during the COVID-19 pandemic. The number of times students were able to perform or practice four of the 18 fundamental clinical skills was reduced compared to students on clinical rotations in the 4 years prior to COVID-19. Their self-perceived proficiency was also reduced for two skills ("Perform a mental health assessment" and "Perform an ECG"). Skills related to mental health assessment were most affected. With the increased focus on mental health since the beginning of the pandemic, it is imperative that students are provided sufficient opportunity to practice and build confidence in core skills in this specialty.

Open Peer Review
This article reports the findings of a study conducted at a rural university in Australia. This study explores how social distancing regulations during the COVID-19 pandemic affected medical students during their clinical rotations. The authors look at 22 clinical skills and present changes to how many times these skills were performed and students' self-reported skill proficiency. The study findings identify only 4 statistically significant clinical skills that were performed fewer times.
Furthermore, students' self-reported proficiency was reduced in only one clinical skill.
There are a few points that need clarification. While students showed a decrease in the number of times they performed 4 clinical skills, they only showed a decrease in self-reported proficiency of one of the skills. It would be helpful to explore this difference. What implications does this difference have for teaching ○ clinical skills?
This study provides a snapshot of what happened during a specific point in time -COVID-19. The interest and implications of this study are very bound by this context. The authors need to move this study beyond the COVID context and consider larger implications for practice. Without this, the study leaves the reader questioning the significance of this study.
medical students during their clinical rotations. The authors look at 22 clinical skills and present changes to how many times these skills were performed and students' selfreported skill proficiency. The study findings identify only 4 statistically significant clinical skills that were performed fewer times. Furthermore, students' self-reported proficiency was reduced in only one clinical skill. There are a few points that need clarification. While students showed a decrease in the number of times they performed 4 clinical skills, they only showed a decrease in self-reported proficiency of one of the skills. It would be helpful to explore this difference. What implications does this difference have for teaching clinical skills?
Response: Thank you for your time in reviewing our paper and feedback which allows us to improve the article content. There was a reduction in 2 of the proficiencies, but only one of these was accompanied by a reduction in frequency of skill performance. It is difficult to postulate the implications for teaching as we move back into an environment that more resembles pre-COVID medical training. We have attempted to address this comment by adding the following text to the discussion. "The teaching of mental health assessment skills may need to be reassessed in a post-COVID landscape where non-face-to-face methods of clinical delivery of mental health are likely to be retained, at least in part. Simulation may have a role in teaching of mental health skills, as well as reassessing how to engage medical students in telehealth consultation".
This study provides a snapshot of what happened during a specific point in time -COVID-19. The interest and implications of this study are very bound by this context. The authors need to move this study beyond the COVID context and consider larger implications for practice. Without this, the study leaves the reader questioning the significance of this study.
Response: The idea of this paper was to identify the impact of changes at a specific timepoint, i.e. during COVID-19 We have attempted to address this in part with the comment above on using the experience of COVID-19 in which face-to-face training was not always possible to help direct changes in teaching where applicable. It is difficult to simulate many clinical skills despite advances in simulation techniques therefore current methods of medical training are most likely required to be retained into the future both to enhance clinical skill ability and to allow students to feel more confident in their abilities to perform them in clinical practice.
The only limitation identified by the authors is that the study was conducted in a rural Australian setting and might not be generalizable. However, there are other limitations as well. For example, the reliance on self-reported proficiencies and the COVID-19 context.
Response: We have included other limitations throughout the discussion. E.g. "Third, students' mental health and well-being may have impacted their self-reported proficiency and participation in performing mental health clinical skills, and lack of this data may be considered a limitation" and "Self-perceived proficiency is subjective, and prone to many sources of variation, which may explain the lack of alignment between measures.". We have amended the text to add other limitations as follows. "First, as mentioned above, selfperceived proficiency is a subjective measure of skill performance that can be affected by a myriad of factors pertaining both to the student personally and their learning environment.