Abstract
There is an increasing focus on factors that influence the variability of rater-based judgments. First impressions are one such factor. First impressions are judgments about people that are made quickly and are based on little information. Under some circumstances, these judgments can be predictive of subsequent decisions. A concern for both examinees and test administrators is whether the relationship remains stable when the performance of the examinee changes. That is, once a first impression is formed, to what degree will an examiner be willing to modify it? The purpose of this study is to determine the degree that first impressions influence final ratings when the performance of examinees changes within the context of an objective structured clinical examination (OSCE). Physician examiners (n = 29) viewed seven videos of examinees (i.e., actors) performing a physical exam on a single OSCE station. They rated the examinees’ clinical abilities on a six-point global rating scale after 60 s (first impression or FIGR). They then observed the examinee for the remainder of the station and provided a final global rating (GRS). For three of the videos, the examinees’ performance remained consistent throughout the videos. For two videos, examinee performance changed from initially strong to weak and for two videos, performance changed from initially weak to strong. The mean FIGR rating for the Consistent condition (M = 4.80) and the Strong to Weak condition (M = 4.87) were higher compared to their respective GRS ratings (M = 3.93, M = 2.73) with a greater decline for the Strong to Weak condition. The mean FIGR rating for the Weak to Strong condition was lower (3.60) than the corresponding mean GRS (4.81). This pattern of findings suggests that raters were willing to change their judgments based on examinee performance. Future work should explore the impact of making a first impression judgment explicit versus implicit and the role of context on the relationship between a first impression and a subsequent judgment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ambady, N., Bernieri, F., & Richeson, J. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. In Advances in experimental social psychology (pp. 201–271).
Bown, M. H., Regehr, G., & Reznick, R. (1996). the effect of early performance on examiners’ marking patterns during an oral examination. Academic Medicine, 71(1), s73–s75.
Carney, D., Colvin, C., & Hall, J. (2007). A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, 41(5), 1054–1072.
Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: Construct alignment improves the performance of workplace-based assessment scales. Medical Education, 45, 560–569.
Gauthier, G., St-Onge, C., & Tavares, W. (2016). Rater cognition: Review and integration of research findings. Medical Education, 50, 511–522.
Gingerich, A., Kogan, J. R., Yeates, P., Govaerts, M. J. B., & Holmboe, E. S. (2014). Seeing the “Black Box” differently: Assesssor cognition from three research perspectives. Medical Education, 48, 1055–1068.
Gingerich, A., Regehr, G., & Eva, K. W. (2011). Rater-based assessments as social judgments: Rethinking the etiology of rater errors. Academic Medicine, 86(10 Suppl), S1–S7.
Govaerts, M. J. B., Schuwirth, L. W. T., Van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2011). Workplace-based assessment: Effects of rater expertise. Advances in Health Sciences Education, 16(2), 151–165.
Govaerts, M. J. B., Van de Wiel, M. W. J., Schuwirth, L. W. T., Van der Vleuten, C. P. M., & Muijtjens, a M. M. (2013). Workplace-based assessment: Raters’ performance theories and constructs. Advances in Health Sciences Education, 18(3), 375–396.
Harris, M., & Garris, C. (2008). You never get a second chance to make a first impression. In N. Ambady & J. Skowronski (Eds.), First impressions (pp. 147–168). New York: Guilford Press.
Hilton, J. L., Klein, J. G., & von Hippel, W. (1991). Attention allocation and impression formation. Personality and Social Psychology Bulletin, 17(5), 548–559.
Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32, 676–682.
Iramaneerat, C., & Yudkowsky, R. (2007). Rater errors in a clinical skills assessment of medical students. Evaluation and the Health Professions, 30(3), 266–283.
Kogan, J., Conforti, L., Bernabeo, E., Iobst, W., & Holmboe, E. (2011). Opening the black box of clinical skills assessment via observation: A conceptual model. Medical Education, 45(10), 1048–1060.
Macan, T. H., & Dipboye, R. L. (1990). The relationship of interviewers’ preinterview impressions to selection and recruitment outcomes. Personnel Psychology, 43(4), 745–768.
Mann, T. C., & Ferguson, M. J. (2015). Can we undo our first impressions? The role of reinterpretation in reversing implicit evaluations. Journal of Personality and Social Psychology, 108(6), 823–849.
Murphy, K. R., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985). Effects of previous performance on evaluations of present performance. Journal of Applied Psychology, 70(1), 72–84.
Skowronski, J. J., & Carlston, D. E. (1989). Negativity and extremity biases in impression formation: A review of explanations. Psychological Bulletin, 105(1), 131–142.
Sumer, H. C., & Knight, P. A. (1996). Assimilation and contrast effects in performance ratings: Effects of rating the previous performance on rating subsequent performance. The Journal of Psychology, 81(4), 436–442.
Tavares, W., & Eva, K. W. (2013). Exploring the impact of mental workload on rater-based assessments. Advances in Health Sciences Education, 18(2), 291–303.
Williams, R. G., Klamen, D. A., & McGaghie, W. C. (2003). Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine, 15(4), 270–292.
Wood, T. J. (2014). Exploring the role of first impressions in rater-based assessments. Advances in Health Sciences Education: Theory and Practice, 19(3), 409–427.
Wood, T. J., Chan, J., Humphrey-Murto, S., Pugh, D., & Touchie, C. (2017). The influence of first impressions on subsequent ratings within an OSCE station. Advances in Health Sciences Education, 22, 969–983.
Wyer, N. A. (2010). You never get a second chance to make a first (implicit) impression: The role of elaboration in the formation and revision of implicit impressions. Social Cognition, 28(1), 1–19.
Yaphe, J., & Street, S. (2003). How do examiners decide? A qualitative study of the process of decision making in the oral examination component of the MRCGP examination. Medical Education, 37(9), 764–771.
Yeates, P., O’Neill, P., Mann, K., & Eva, K. (2013). Seeing the same thing differently: Mechanisms that contribute to assessor differences in directly-observed performance assessments. Advances in Health Sciences Education: Theory and Practice, 18(3), 325–341.
Yeates, P., Moreau, M., & Eva, K. (2015). Are examiner’s judgments in OSCE-style assessments influenced by contrast effects? Academic Medicine, 90(7), 975–980.
Acknowledgements
This research was supported in part by a research grant from the Medical Council of Canada and from the Department of Medicine, University of Ottawa. The authors would like to acknowledge Lesley Ananny, Meredith Mackay, and Katherine Scowcroft for their help on this study as well as the Department of Innovation in Medical Education.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Time point 1
What would be your prediction for this examinee’s final performance on this station relative to other trainees who have recently graduated from a Canadian medical school?
Unacceptable | Acceptable | ||||
|---|---|---|---|---|---|
Inferior | Poor | Borderline | Borderline | Good | Excellent |
Time point 2
Please view the remainder of the station and then rate the examinee’s performance on the following competencies relative to other trainees who have recently graduated from a Canadian medical school (OSCE Scale).
Physical examination skills
As compared to other trainees who have recently graduated from a Canadian medical school, this examinee’s history taking skills are:
Unacceptable | Borderline unacceptable | Borderline acceptable | Acceptable | Above expectations |
Organization
As compared to other trainees who have recently graduated from a Canadian medical school, this examinee’s organizational skills are:
Unacceptable | Borderline unacceptable | Borderline acceptable | Acceptable | Above expectations |
Diagnosis
As compared to other trainees who have recently graduated from a Canadian medical school, this examinee’s diagnosis and/or differential diagnoses are:
Unacceptable | Borderline unacceptable | Borderline acceptable | Acceptable | Above expectations |
Data interpretation
As compared to other trainees who have recently graduated from a Canadian medical school, this examinee’s ability to interpret data is:
Unacceptable | Borderline unacceptable | Borderline acceptable | Acceptable | Above expectations |
Management
As compared to other trainees who have recently graduated from a Canadian medical school, this examinee’s ability to manage priorities is:
Unacceptable | Borderline unacceptable | Borderline acceptable | Acceptable | Above expectations |
Overall performance
Based on what you have just seen, please rate the examinee’s overall performance relative to other trainees who have recently graduated from a Canadian medical school (GRS).
Unacceptable | Acceptable | ||||
|---|---|---|---|---|---|
Inferior | Poor | Borderline | Borderline | Good | Excellent |
Rights and permissions
About this article
Cite this article
Wood, T.J., Pugh, D., Touchie, C. et al. Can physician examiners overcome their first impression when examinee performance changes?. Adv in Health Sci Educ 23, 721–732 (2018). https://doi.org/10.1007/s10459-018-9823-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-018-9823-4