Impact of a learning portfolio on reflective, clinical and communication skills in undergraduate dental students- A pilot study

Student portfolios are a collection of evidence, prepared by the student and evaluated by the faculty member to demonstrate mastery, comprehension, application, and synthesis of a given set of concepts. The purpose of this study was to investigate the effect of portfolio on student’s self-reflection, clinical competency and communication skills in daily clinical practice and to know students perception on faculty feedback. A randomized controlled, triangulation study was conducted in Melaka-Manipal Medical College with thirty consenting Year IV BDS students as participants. There was statistically significant improvement for all the outcomes tested in the portfolio group compared to traditional logbook group.


Introduction
In 1990, Miller wrote that no tools were available for assessment of what a learner does when functioning independently at the clinical workplace (Miller,1990;Tartwijk and Driessen, 2009). Since then portfolios have filled this gap and found their way into medical education, not only as a tool for assessment of performance in the workplace, but also as tools to stimulate learning from experience.

The following objectives were formulated for the study:
To assess improvement in student's ability to practice reflective learning in prosthodontics after 1.
intervention To compare student's competency in prosthodontic clinical skills (case history, intra oral 2. examination, primary impression in dentate and edentate patients) in the intervention and control groups To compare student's communication skills in intervention and control groups based on multi source 3.
feedback To compare student perceptions on faculty feedback in intervention and control groups based on a 4.
To know student perception on portfolio and traditional log book implementations after the study 5.

Methods
The ethical approval was obtained from Research Ethics Committee, Faculty of Dentistry, Melaka-Manipal Medical College. (Ref No: MMMC/FOD/AR/EC 2016 (F-02)). Consent to use the images was obtained from the participants.
The study was divided into three phases: Phase 1: Development of learning Portfolio Phase 2: Baseline evaluation using conventional logbook (Control) Phase 3: Evaluation using learning Portfolio (Intervention)

Phase 1: Development of Portfolio
A focus group discussion was conducted in three sessions to finalise the contents of the portfolio and grading criteria.  Item content validity index was calcualted by five experts.  Phase 2: Baseline evaluation using conventional logbook Faculty and students were trained and sensitized to the proper use of logbook. Allocation concealment and blinding procedures were ensured. The following method was implemented.  Students in each group repeated the procedure for four weeks. The student competency was assessed using checklist and global index scores. Verbal feedback was given based on their performance in the procedures done.
Assessment of communication skills of the students was done by faculty, peer and Dental Surgery Assistant (DSA) (Multisource feedback) using a pre-validated questionnaire adopted from Joshi et al. (Joshi, R  The following method was implemented.  Student's competency was assessed similar to that done in conventional logbook group. Additionally student's selfreflection and written faculty feedback on the student's competency and communication skills were recorded each time after the procedure. The student's reflection was graded using the technique adopted from Wong et al. (Wong, F.K., Kember, D. and Chung, L.Y. 1995) Assessment of communication skills and students perception on the faculty feedback were also collected.

Results/Analysis
Friedman's test was done to assess improvement in student's ability to reflect after intervention. Statistically significant difference was found from week two to week four, indicating an improvement in the reflective writing.  Two way mixed ANOVA of competency scores was done to compare student's competency in the intervention and control groups for all the four procedures over a period of four weeks. There was a statistically significant difference between the groups for all the procedures with portfolio group showing more improvement in competency scores from week one to week four.  Wilcoxon signed rank test for the student communication skills in portfolio and traditional logbook using multi source feedback is shown below. A statistically significant difference between the groups was found in faculty and DSA feedback indicating that the portfolio group exhibited better communication skills. However, peer feedback did not show any statistically significant difference for both the groups. Peer feedback for both conventional group and portfolio group was 4 (4,5) p> 0.05 for both the group. Wilcoxon signed rank test was used to compare student perception on faculty feedback in the intervention and control groups. Except for one question all other questions showed a statistically significant difference between the groups indicating a better faculty feedback in the portfolio group.  Spearman's correlation showed a negative correlation between the feedback, reflection and competency scores. However, the results were not statistically significant. Qualitative analysis of the student perceptions on portfolio implementation was done using the thematic analysis by Braun and Clark. (Clark, V and Braun, V, 2013) Five themes were generated.
Strengths and weakness 5.

Discussion
We adopted the technique given by Wong, et al. (Wong, et. al., 1995) to grade the student's into reflectors, nonreflectors and critical reflectors. Over the period of four weeks, there was a gradual increase in the number of reflectors and critical reflectors similar to our reference study (Wong, et al., 1995). These results could be attributed to the influence of an effective written feedback, which is the strength of portfolio (Snadden, D and Thomas, M.L, 1998).
Competency of four procedures were assessed in the current study and in an attempt to establish reliability of the results , the procedures were repeated and assessed weekly for four weeks period in both the groups. The results showed a significant improvement in the portfolio group over the four weeks and this could be due to the written feedback by the faculty and self-reflection by the students. Tochel C et al. (Tochel et al., 2009) reported a systematic review on the effectiveness of using portfolios. In this article the author discusses that according to Kirkpatrick's Hierarchy (McCullan et al., 2003), most included studies were found to impact on the learning of the portfolio user (a level 2 impact). This indicates that, the reflections and feedback help to bring about a modification of knowledge/ skills relating to acquisition of concepts, procedures and principles.
Multi source feedback regarding the communication skills was rated higher in the portfolio group compared to the conventional group by the DSA and faculty. These results could be attributed to the self reflection of the student and a written feedback by the faculty. This is supported by the systematic review by Tochel C et. al. (Tochel et. al, 2009) who reported studies showing an improvement in the attitude and perception of the students due to the component of self reflection and feedback. The qualitative analysis of the students revealed that they had an opportunity to relook into the previous week's reflection and comments before beginning the procedure thereby taking care not to repeat the mistakes done earlier. Peer feedback did not show any statistical difference and was rated high in both the groups. As reported by Robinson (Li, L and Gao, F, 2016) this could be because some students feel uneasy and reluctant to assess their peers' work and 'marking could be easily affected by friendship, cheating, ego or low self-esteem'. Hence the rating is same for both the groups and consistently high (Driessen et. al, 2005) Students felt the need for an elaborate written feedback from faculty, which was possible in the portfolio group. The results reflect the same showing a better satisfaction rate for the portfolio group.
We tried to correlate the three components: reflective writing, competency scores and communication skills by using the Spearman's correlation. However, no correlation was found between any of the parameters. With the small sample we have in the study, establishing correlations between the different parameters may be difficult. This aspect could be looked into in future studies.
Qualitative analysis revealed five main themes. The students felt that through self-reflection they had better clarity of thoughts. The written feedback provided by the faculty was very helpful to identify the areas they needed improvement. They opined that before starting the procedure they had an opportunity to relook into the previous week's mistakes and feedback which helped them avoid the same. The tips given by lecturers could be implemented effectively due to the possibility of reading the written feedback a number of times. This automatically led to an improvement in their performance in contrast to the traditional logbook group where verbal feedback is easily forgotten .The students perceived that they were able to better identify their strengths and weakness through selfreflection and feedback.

Conclusion
There was a statistically significant improvement in student's ability to practice reflective learning in 1.
prosthodontics after intervention. Student's competency in prosthodontic clinical skills (case history, intra oral examination, primary 2. impression in dentate and edentate patients) in the intervention group was better than the control group and statistically significant. Faculty and DSA feedback on communication skills showed a statistically significant difference with 3.
the portfolio group showing better results. Portfolio group showed better scores for student perceptions on faculty feedback than the control 4.
group and the results were statistically significant. Five themes were generated by qualitative analysis of student perceptions on portfolio 5.
implementation and showed a positive impact.

Take Home Messages
With the results obtained in our study, portfolio has a significant impact on student competency in selected prosthodontic procedures and communication skills. Over the four weeks period a drastic improvement in the reflective skills was also noted. The main strength of the study lies in reflective learning which is a way of allowing students to step back from their learning experience to help them develop critical thinking skills and improve on future performance by analyzing their experience. The explosive growth of learning portfolios in higher education as a compelling tool for enhanced student learning, assessment, and career preparation is a sign of the increasing