A novel method for identifying continuing medical education among faculty in anesthesia

Introduction Identifying learning needs and gaps among faculty is an important and crucial step in the educational process. A well designed and targeted needs assessment provides information useful to identify and meet learners needs in medical education and professional improvement. In reflecting on one’s practice in general, the ability to identify weaknesses by self-assessment can serve the function of helping the clinician identify learning needs and set appropriate learning goals. In this study, we utilized a competence score based on self-assessment to identify and prioritize the needs in clinical case management and procedural skills among pediatric anesthesiologists ahead of the opening of a new pediatric hospital in the middle-east. Methods Institutional approval was obtained. Content areas covering clinical management and procedural skills were developed based on a systematic literature review and a modified Delphi process. Questionnaires listing the content areas were sent to respondents. The respondents indicated on a modified Dreyfus self-rated expertise scale of 1-10, their current level of expertise versus their ideal level of expertise for each content area. A need score for each content area was calculated from the difference between ideal and current level of expertise with a maximum need score of 9. Descriptive statistics were performed. Results The responses rate was 100 % (21/21). Two thirds of respondents were anesthesia specalists (fellows) and the rest staff anesthesiologists. The modified Dreyfus scale was found to be a relevant, internally consistent (Cronbach’s alpha = 0.85) and valid tool for assessing perceived levels competency/expertise of anesthesiologists and anesthesia Matava C, Naser B, Crawford M MedEdPublish https://doi.org/10.15694/mep.2017.000078 Page | 2 specialists. Management of malignant hyperthermia, premature infants, difficult airway demonstrated the highest needs with scores; 5.3, 4.4 and 3.8 respectively. Ultrasound use, one lung ventilation and use of difficult airway devices had highest need scores for procedural skill; 5.3, 5.2 and 4.4 respectively. Respondents preferred one to one discussions and workshops as effective methods for improving skills. Conclusions We have demonstrated the use of a novel tool, the need score to identify and prioritize needs in a needs assessment for a new program. The information we have gathered is preliminary and further assessments are needed to validate our findings by external reviews and use of focus groups. Our study methodology provides information for immediate in this context but should be translatable for use in other clinical situations in medicine.


Introduction
Identifying learning needs and gaps among faculty is an important and crucial step in the educational process. A well designed and targeted needs assessment provides information useful to identify and meet learners needs in medical education and professional improvement. In reflecting on one's practice in general, the ability to identify weaknesses by self-assessment can serve the function of helping the clinician identify learning needs and set appropriate learning goals.
In this study, we utilized a competence score based on self-assessment to identify and prioritize the needs in clinical case management and procedural skills among pediatric anesthesiologists ahead of the opening of a new pediatric hospital in the middle-east.

Methods
Institutional approval was obtained. Content areas covering clinical management and procedural skills were developed based on a systematic literature review and a modified Delphi process. Questionnaires listing the content areas were sent to respondents. The respondents indicated on a modified Dreyfus self-rated expertise scale of 1-10, their current level of expertise versus their ideal level of expertise for each content area. A need score for each content area was calculated from the difference between ideal and current level of expertise with a maximum need score of 9. Descriptive statistics were performed.

Introduction
'The highest expression of the art of the adult educator is skill in helping adult learners to discover their own learning needs' (1). Learning needs is an important and crucial step in the educational process. A well designed and targeted needs assessment provides information useful to identify and meet learners needs in medical education and professional improvement (2). Self-assessment has been defined broadly as the involvement of learners in judging if learner-identified standards have been met and works as a mechanism for identifying both one's weaknesses and one's strengths (3,4). Self-assessment is also useful in clinical practice as a tool that allows the clinician to practice self-limit and restrain in areas of limited competence (4). In reflecting on one's practice in general, the ability to identify weaknesses can serve the function of helping the clinician identify learning needs and set appropriate learning goals (2).
In this study we utilized a novel needs score based on self-assessment to identify and prioritize the needs in clinical case management and procedural skills among pediatric anesthesiologists ahead of the opening of a new pediatric hospital in a middle east country.

Methods
Following study approval by the Research Ethics Board at The Hospital for Sick Children, a preliminary inventory of skills and clinical management areas expected of competent pediatric anesthesiologists was developed from a search of relevant medical/educational literature. Using a modified Delphi technique involving 6 staff pediatric anesthesiologists, all of whom had completed pediatric anesthesia fellowships and were active in teaching anesthesia trainees, we reached consensus on a list of skills and clinical areas deemed to be essential to the practice of a pediatric anesthesiologist.

Need Score
We adapted the validated Dreyfus' 5-point scale of expertise into a 10-point scale (1 as Novice, and 10 as Expert). A 10-point scale would allow us to further separate each level of the Dreyfus scale into more discrete components. Each clinical management and procedural skill item on the inventory was rated twice by respondents using the modified scale with one rating for current perceived level of expertise and another rating for ideal/desired level of Matava C, Naser B, Crawford M MedEdPublish https://doi.org/10.15694/mep.2017.000078 Page | 3 expertise. The difference between the ideal/desired level of expertise and the current level of expertise were taken as the need score for each inventory item.
Following testing and piloting on 5 pediatric anesthesia fellows, the survey tool was finalized to elicit information on: Current and ideal/desired level of expertise in clinical management and procedural skill areas 1.
Educational formats for enhancing skills perceived to be effective among respondents 2.
Demographics of the respondents 3.
The survey was constructed using the online tool Qualtrics.com (http://www.qualtrics.com) and distributed on paper.

Data Collection
The sampling frame was purposive and consisted of anesthesiologists and anesthesia specialists who worked in the current existing mixed adult and children hospital. The surveys were distributed by the administrative assistant and once completed were posted into a drop box. All responses were anonymous. Responses were entered into a Microsoft Excel spreadsheet for analysis.
We defined high priority areas as those with a mean need score > 3 and those with a mean need score <3 as low priority areas.

Data Analysis
Data analysis was performed using GraphPad Prism version 5.0b for Mac OS X, GraphPad Software, San Diego California USA, www.graphpad.com. Descriptive statistics were used to summarize the data. Two-tailed Fisher's exact test was used to analyze nominal data. P < 0.05 was considered statistically significant.

Demographics and characteristics of respondents
One hundred percent (21/21) of anesthesiologists responded to the survey. Two thirds (14/21) were specialist anesthesiologists (Table 1). Fifty-two percent (11/21) of the respondents had been in practice for 11-20 years. The majority (15/21) of respondents reported practicing pediatric anesthesia less than 25% of time while 81% (17/21) also provided cover for emergency pediatric anesthesia. A third of respondents (6/21) reported asking colleagues for help in anesthesizing complicated pediatric patients.

Validation of Scale
The modified Dreyfus scale was found to be a relevant, internally consistent (Cronbach's alpha = 0.85) and valid tool for assessing perceived levels competency/expertise of anesthesiologists and anesthesia specialists.

Clinical case management
Management of malignant hyperthermia, management of premature patients were identified as high priority areas with high need scores of 5.3 and 4.4 respectively (Figure 1). Respondents reported strengths in areas such as use of inhalation anesthesia for inductin in children, VP shunt insertion and anesthesia for CT Scan, MRI and Radiology ( Figure 1

Procedural Skills
The use of ultrasound for regional anesthesia, single lung ventilation and use of advanced airway device for difficult airways in children were all high priority areas with mean needs scores of 5.3, 5.2 and 4.4 respectively (Figure 2).

Preferred educational formats
One to one discussions, workshops and operating room based learning were highly ranked as effective tools for enhancing skills in both clinical case management and procedures ( Table 2). Didactic lectures without opportunity for questions were regarded ineffective for skills enhancement (Table 2).

Discussion
Our study has utilized a novel approach for identifying and prioritizing educational need based on a needs score defined as the gap between respondents' perceived current level of competency and their desired/ideal levels of competency in various knowledge and skills areas.
The needs identified by our respondents can be explained in part by their clinical background experience and local context. The respondents in our study are mostly anesthesiology staff and specialists who have worked predominantly in an adult setting for many years. In the process of preparing to staff a pediatric anesthesia service, our study has provided data on the their perceived needs in the areas of skills and case management. As expected, the areas unique to pediatric anesthesia were identified as having the highest need score, such as the management of severe premature patients, malignant hyperthemia and the difficult airway. Similarly, procedural skills such as use of ultrasound and the use of difficult airway devices in pediatric patients had the highest need scores.
Our ranking of the areas with high need scores reveal management of malignant hyperthermia, premature infants and difficult airway as priority areas. These finding may be explained by the relatedly low incidence of malignant hyperthermia in this population. However the perceived need for more training in managing neonates and difficult airway is likely to be associated with the challenging nature of these clinical cases. Prioritizing learning needs alongside likelihood of managing such cases is important for appropriate utilization of educational resources and personnel (5).
In order to maximize learning motivation, topics ranked as high priority should be considered along with the respondents suggestions for effective teaching strategies. This needs to be balanced with appropriate pedagogical approaches. Our respondents ranked one on one teaching and workshops as the most effective method for improving both clinical and procedural skills. This may be explained by that our respondents are previously experienced anesthesiologists and as adult learners expect the educational activities to be tailored to the each respondent's different level of experience perhaps best offered by these modalities (6). A major development in the field of adult education is the premise that the successful and effective educational initiatives do not rest on the efforts of the tutors, but depend on learners ability to self-direct the experiential learning experience (7). This is reflected in the respondents' high rating of simulation for learning both clinical management and procedural skills. Low need areas suggest the respondents are well versed in these areas and focusing on these may reduce the level of respondents' motivation to participate in the program.
Our study has some limitations. The data generated through the survey tool assesses perception of expertise based on self-ratings of performance. It is possible that respondent's may have inflated their perceived expertise on areas known to be basic in the practice of pediatric anesthesia. Respondent's may also not accurately recognize the limitations of their skills as clinicians. Respondents may have also over-stated the effective of various educational formats while motivation for engaging in these once established may be affected by other factors such as institutional or cultural issues.

Take Home Messages
We have demonstrated the novel use of a competence scale to identify and prioritize learning needs in a skills enhancement program for anesthesiologists. The information we have gathered is preliminary and further assessments are needed to validate our findings by external reviews and use of focus groups. Our study methodology provides information for immediate in this context but should be translatable for use in other clinical situations in medicine.

Notes On Contributors
CM, BN and MC participated in the study design, review and manuscript writeup and revisions.