Experiences in Teaching and Learning
Comparing student self-assessments of global communication with trained faculty and standardized patient assessments

https://doi.org/10.1016/j.cptl.2018.03.012Get rights and content

Abstract

Background and purpose

Assess the reliability of first year pharmacy student assessments completed by faculty members in comparison with a standardized patient (SP), and student self-assessments during a structured educational module on communication.

Educational activity and setting

Pharmacy students completed four stations focused on communication with an SP. During each encounter, students completed a self-assessment and were evaluated by a faculty member and a trained SP. A five point Likert scale was used to evaluate student performance. Faculty assessments were compared against all others. A Pearson correlation coefficient for total scores was used and a Cohen's kappa was used to compare inter-rater reliability. Agreement and correlation was performed with student results categorized into poor, adequate, and exceptional performance based on faculty evaluation.

Findings

Twenty-four students participated. In all stations, student self-assessments were graded higher than corresponding faculty and SP assessments. Agreement between faculty, SP, and self-assessment was fair to slight (k < 0.4) for all comparisons but only significant (p < 0.05) between the faculty and self-assessment. After categorization, there was a small, non-significant correlation between faculty and self-assessment (r = 0.13, p = 0.21) and moderate and significant correlation between faculty and SP (r = 0.32, p = 0.001). Categorized inter-rater agreement was fair for all comparisons (k < 0.2) and only significant (p < 0.05) between faculty and SP assessment.

Discussion

Pharmacy students in their first professional year assess their communication skills more positively than other evaluators. Further instruction for students and reflection may be required to build understanding of global assessment in communication.

Summary

There is high incongruity between student self-assessment and faculty appraisal.

Section snippets

Background and purpose

Communication skills are found throughout all levels of Bloom's taxonomy of learning, incorporating lower and higher orders of thinking.1 Pharmacy students are taught how to communicate professionally from an early point in their pharmacy program to provide them with the skills for effective pharmaceutical care. At the College of Pharmacy, Qatar University (CPH-QU), the Association of Faculties of Pharmacy of Canada (AFPC) educational outcomes and the National Association of Pharmacy Regulatory

Educational activity and setting

The CPH-QU offers a four year, undergraduate degree program that has been accredited by the Canadian Council for Accreditation of Pharmacy Programs (CCAPP). The CPH-QU curriculum incorporates a mandatory Professional Skills series through all four years of the program and aims to develop the practical skills of a pharmacist. Within this Professional Skills course, a Communications Module focused on patient counseling was introduced to first-year pharmacy students (P1) at CPH-QU in fall semester

Findings

All twenty-four P1 students at CPH-QU, four faculty members and four SPs participated in the communications module evaluation. All students were female as CPH-QU only enrolls females. All faculty members were full-time with an adjunct appointment at a local hospital. Two faculty members were female (50%) while three of the SPs were female (75%).

Faculty member and SP evaluators assigned 96 communication grades divided across four stations. The mean (±SD) communication scores from faculty member

Discussion

In this study, a standardized global communication assessment was used to assess the communications skills of P1 students, evaluate the ability of students to self-assess and compare their performance with those of faculty members and SPs. Student self-evaluated communication skills resulted in higher scores than those assigned by both faculty and SP evaluators. These results suggest that students early in their education may not fully realize their own strengths and, more particularly,

Summary

Having first year pharmacy students assess their own communication skills resulted in poor correlation and agreement with SP and faculty evaluators when using a quantitative evaluation scale. Standardized patient assessment of communication also resulted in unfavorable correlation and agreement with faculty assessment and student self-assessment. Further training of first year pharmacy students is needed to improve their ability to self-assess their communication skills after an interaction

Financial disclosures

None.

Conflicts of interest

None.

Acknowledgments

The authors would like to thank the faculty of pharmacy for supporting this project and the pharmacy students who participated in this study.

References (23)

  • N Fjortoft

    Self-assessment in pharmacy education

    Am J Pharm Educ

    (2006)
  • Cited by (0)

    View full text