Use of a point-of-care app increases patient satisfaction with residents performing minor procedures

This study investigated patient satisfaction of residents’ ability after using a Point-of-Care (POC) app for minor procedures in Family Medicine. Patients were asked to a) evaluate their satisfaction of a resident performing a minor procedure, b) rate their skill level, and c) decide if they explained the procedure clearly. The results show that a) use of the POC app improved patient satisfaction (p<0.05), but had no effect on either b) residents’ skill level (p>0.05) or c) their communication skills. This study demonstrates a possible role for a POC app in teaching minor procedures in Family Medicine.


Introduction
As teachers of minor procedures in Family Medicine, we identified a gap that existed in teaching our Family Medicine resident learners; specifically, that gap was insufficient exposure to minor procedures. To better understand how we might address this learning gap, we examined learning theory to assist in developing a way to increase the exposure of residents to procedures without increasing risk to patients. To that end, we developed a Point-Of-Care (POC) App, ('Proceducate'), to serve as a repository of short videos and information on the procedures we perform in clinic. Consistent with the literature, videos were developed in accordance with best practice (ie. applying multimedia and contiguity principles (Clark & Mayer, 2011). Thus, our goal was to build on the foundation of the traditional textbook by introducing a more modern technology-based experience. The app was developed to demonstrate concepts and ideas for novices as well as jog your memory of previously learned materials. Historically, patients have been used to assess a number of different areas of medicine including i) the performance of health systems, ii) in measuring the empathy of health professionals, and iii) evaluating communication skills of medical doctors (Birhanu, Assefa, Woldie, & Morankar, 2012;Kerns, Krist, Woolf, Flores, & Johnson, 2008). We recruited patients to serve as evaluators of residents, since, as the persons having the procedure done upon them, they have the most to lose when it is not performed correctly. While a number of patient factors, including, age, socioeconomic status, gender and ethnicity can influence a patient's evaluation, the exact effect seems to be difficult to ascertain with a number of studies offering contradictory results (Sixma, Spreeuwenberg, & van der Pasch, 1998;Hall & Dornan, 1990;Haviland, Morales, Dial, & Pincus, 2005).
To say that someone is competent at something means that have the necessary ability, knowledge and skill to do something and have an acceptable and satisfactory outcome. More specifically, when it comes to Family Medicine minor procedures, one must possess both the technical and cognitive skills required to perform a procedure from start to finish. Whereas the cognitive component comprises the ability to know when to do the procedure and how to properly communicate the risks, benefits and side effects of the given procedure, to the patient, the technical skill is the ability to safely and accurately complete the procedure. While direct observation of the patient-resident interaction is one method of evaluating resident competency, we used the patient's evaluation of the resident as a means of measuring if our POC app had any impact on our patient's satisfaction with our resident's procedural skills.

Methods
Family Medicine residents at Sunnybrook Health Sciences Centre were recruited to participate in this observational study. Resident learners who consented to participate in the study were randomized to either the control group (standard medical teaching) or intervention group (standard medical teaching plus access to 'Proceducate') prior to attending a Family Medicine procedures clinic. Patients who were referred to the procedures clinic by other family doctors at Sunnybrook Health Sciences Centre or University of Toronto associated clinics were recruited as evaluators of the resident that performed a procedure on them.
In clinic, patients were examined by a resident learner who took a focused history, did the required physical exam, and obtained written consent before reviewing the case with 1 of 2 Physician Instructors. After case review, if it was appropriate for a procedure to be performed, and if the resident was allocated to the intervention group, they were given access to 'Proceducate' so they could review the desired procedure on the app. After the patient gave informed consent to have a procedure done, the study was explained to the patient, and they were asked to consent to participate in study as an evaluator of the resident. The patient was told that the study involved a comparison of two different methods of resident teaching but patients were blinded to the exact nature of the teaching methods and to which group their resident was assigned.
After the procedure was completed by the resident under the supervision of the Physician Instructor, the patient was given a survey to complete. The survey recorded which procedure the patient had undergone (e.g. cryotherapy, shoulder injection, punch biopsy, or Van den Bos procedure) and the patient was asked to a) evaluate their satisfaction of the resident performing the procedure, b) rate the resident's skill level, and c) decide if the resident explained the procedure, risks and benefits in a way that they were able to understand. (Figure 1). The completed survey was sealed in an envelope by the patient and handed to the Physician Instructor. At no time did the resident observe the questionnaire. Survey results were compiled and the relevant statistical analysis completed.
Ethics Protocol Identification was 426-2012 for Ethics Approval at Sunnybrook Health Sciences Centre, University of Toronto

Results
When the data were pooled for all procedures performed (Table 1), we found a statistically significant increase in overall patient satisfaction with the procedure for the intervention group vs the control group (p=0.0261). A subanalysis of the individual procedures (Table 2) showed no statistically significant difference for patient satisfaction between control and intervention groups. For example, patients were not more satisfied with residents in the intervention group in performing cryotherapy than their control counterparts (p> 0.05).
As the procedure clinic utilized two physician instructors, we examined whether the physician instructor impacted patients' satisfaction with the resident who performed the procedure ( Table 3). The analysis demonstrated no statistically significant difference .
In comparison to patient satisfaction, when all individual procedure results were pooled for analysis of the patientperceived skill level of our resident learners, there was no difference between the control group (6.146 +/-1.152) and intervention group (6.450 +/-0.746) (p = 0.308). The sub-analysis of the skill level for individual procedures (Table 4) also yielded no significant differences (p>0.05), as did a comparison on instructor-led clinics (Table 5).
Overall satisfaction with the clinic today:

Discussion
People acquire skills and knowledge in different ways and the way we facilitate that learning can depend on the target audience. Traditionally, teaching of procedures employs a "See one, do one, teach one" approach and exemplifies the social cognition theory of learning that is best exemplified in hands-on-experiences and demonstrations. Over time, many teachers in areas of study including physiotherapy, have felt this type of learning can be improved through changes in teaching methods (Rossettini et al., 2017;Vozenilek et al., 2004;Mason & Strike, 2003). Furthermore, in our teaching centre, residents requested more exposure to Family Medicine procedures during the residency program, thus a gap was identified in their training that we wanted to address.
In medical education we lay the foundation of knowledge through events like reading course materials or attending lectures or rounds and, as we progress in our training, we rely on those long-term memories to diagnose and treat patients. By utilizing technology, we can expand both the time and space of the learning environment, thus allowing us to simultaneously incorporate different types of learning into our curriculum. With these ideas in mind, we created the mobile POC app to 1) address an identified gap in training to perform those procedures and 2) serve as a repository of minor procedural skills information for Family Medicine resident learners when they perform minor procedures common to an urban general practice.
In designing of this study, we contemplated asking patients to rate the competency of our Family Medicine residents. However, to ask this question directly, we would have had to first determine if patients fully understood the cognitive and technical skills required to reach competency. By asking patient participants to a) rate their overall satisfaction of the procedure, b) the skill level of the resident, and c) the ability of the resident to communicate the procedure to them, we were able to efficiently examine the individual components of competency from the perspective of those who have the most to gain or lose, i.e. those individuals undergoing procedures.
As the number of specific procedures performed was quite small, we pooled the results for all procedures done and examined patient satisfaction. Analysis of the pooled data produced a statistically significant difference between the control and intervention groups (p < 0.05). Thus, while our study was limited in the small sample size for each individual procedure, the collective data of 100 performed procedures gave us more statistical power (Table 1).
When we examined patient satisfaction on individual procedures performed, (cryotherapy, excisional biopsy, ganglion cyst removal, joint injection, and punch biopsy), (Table 2), analysis yielded no statistically significant difference between control and intervention groups (p> 0.05).
In addition, because our clinic utilized two instructors, we looked at the results from each instructor's clinic and found no effect of instructor selection on the results (Control p = 0.956; Intervention p= 0.765), supporting pooling of the data between clinics to increase our sample size (Table 3). We also performed the same statistical analysis on patient-perceived resident skill level and found no statistical differences with the pooled data (Table 4) or the individual procedure data ( Table 5).
The differences between patient-perceived satisfaction and patient-perceived resident skill level highlights there is a difference in the way patients view those two concepts. In the world of business, customer satisfaction can be linked to the pre-conceived expectation and the resulting quality of the product (Anderson & Sullivan, 1993). In the delivery of health care, factors that influence patient satisfaction include wait time to treatment, cost, race, and age (Morgan et al., 2015). While we did record demographics in our study, our sample size is very small and studies examining the impact of these factors typically involve thousands of patients. Furthermore, we did not record the wait time and overall time for the procedure and this is a subject for future analysis. Regardless, much needs to be done to determine which components of the patient-resident encounter contributed to better satisfaction scores in the intervention group but had no effect on perceived skill level.