Skip to main content
SpringerLink
Log in

How well do faculty evaluate the interviewing skills of medical students?

  • Original Articles
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

Objective:To study the reliability and validity of using medical school faculty in the evaluation of the interviewing skills of medical students.

Design:All second-year University of North Carolina medical students (n=159) were observed interviewing standardized patients for 5 minutes by one of eight experienced clinical faculty. Interview quality was assessed by a faculty checklist covering questioning style, facilitative behaviors, and specific content. Twenty-one randomly chosen students were videotaped and rated: by the original rater as well as four other raters; by two nationally recognized experts; and according to Roter’s coding dimensions, which have been found to correlate strongly with patient compliance and satisfaction.

Setting:Medical school at a state university in the southeastern United States.

Participants:Faculty members who volunteered to evaluate second-year medical students during an annual Objective Structured Clinical Exam.

Interventions:Interrater reliability and intrarater reliability were tested using videotapes of medical students interviewing a standardized patient. Validity was tested by comparing the faculty judgment with both an analysis using the Roter Interactional Analysis System and an assessment made by expert interviewers.

Measurements and main results:Faculty mean checklist score was 80% (range 41–100%). Intrarater reliability was poor for assessment of skills and behaviors as compared with that for content obtained. Interrater reliability was also poor as measured by intraclass correlation coefficients ranging from 0.11 to 0.37. When compared with the experts, faculty raters had a sensitivity of 80% but a specificity of 45% in identifying students with adequate skills. The predictive value of faculty assessment was 12%. Analysis using Roter’s coding scheme suggests that faculty scored students on the basis of likability rather than specific behavioral skills, limiting their ability to provide behaviorally specific feedback.

Conclusions:To accurately evaluate clinical interviewing skills we must enhance rater consistency, particularly in assessing those skills that both satisfy patients and yield crucial data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Platt FW, McMath JC. Clinical hypocompetence: the interview. Ann Intern Med. 1979;91:898–902.

    PubMed  CAS  Google Scholar 

  2. Duffy DL, Hamerman D, Cohen MA. Communication skills of house officers: a study in a medical clinic. Ann Intern Med. 1980;93:354–7.

    PubMed  CAS  Google Scholar 

  3. Beckman HB, Frankel RM. The effect of physician behavior on the collection of data. Ann Intern Med. 1984;101:692–5.

    PubMed  CAS  Google Scholar 

  4. Kaplan SH, Greenfield S, Ware JE. Assessing the effects of physician-patient interaction on the outcomes of chronic disease. Med Care. 1989;27(March suppl):S110-S127.

    Article  PubMed  CAS  Google Scholar 

  5. Hulka BS, Kupper LL, Cassel JC, Mayo F. Doctor-patient communication and outcomes among diabetic patients. J Community Health. 1975;1:15–27.

    Article  PubMed  CAS  Google Scholar 

  6. Hulka BS, Cassel JC, Kupper LL, Burdette JA. Communication, compliance, and concordance between physicians and patients with prescribed medications. Am J Public Health. 1976;66:847–53.

    PubMed  CAS  Google Scholar 

  7. Starfield B, Wray C, Hess K, Gross R, Birk PS, D’lugoff BD. The influence of patient-practioner agreement on outcome of care. Am J Public Health. 1981;71:127–31.

    PubMed  CAS  Google Scholar 

  8. Stiles WB, Putnam SM, Wolf MH, James SA. Interaction exchange structure and patient satisfaction with medical interviews. Med Care. 1979;17:667–81.

    Article  PubMed  CAS  Google Scholar 

  9. DiMatteo MR, Hays RD, Prince LM. Relationship of physicians’ nonverbal communication skill to patient satisfaction, appointment noncompliance, and physician workload. Health Psychol. 1986;5(6):581–94.

    Article  PubMed  CAS  Google Scholar 

  10. Engel GL. Are medical schools neglecting clinical skills? JAMA. 1976;236:861–3.

    Article  PubMed  CAS  Google Scholar 

  11. Seegal D, Wertheim AR. On the failure to supervise students’ performance of complete physical examinations. JAMA. 1962;180:476–7.

    Google Scholar 

  12. Blank LL, Grosso LJ, Benson JA. A survey of clinical skills evaluation practices in internal medicine residency programs. J Med Educ. 1984;59:401–6.

    PubMed  CAS  Google Scholar 

  13. Ende J. Feedback in clinical education. JAMA. 1983;250:777–81.

    Article  PubMed  CAS  Google Scholar 

  14. Hart I, Harden R, Walton H (eds). Newer developments in assessing clinical competence. Montreal: Can-Heal, 1986.

    Google Scholar 

  15. Petrusa ER, Blackwell TA, Ainsworth MA. Reliability and validity of an objective structured clinical examination for assessing the clinical performance of residents. Arch Intern Med. 1990;150:573–7.

    Article  PubMed  CAS  Google Scholar 

  16. Hoole AJ, Kowlowitz V, McGaghie WC, Sloane PS, Colindres RE. Using the Objective Structured Clinical Exam at the University of North Carolina. N C J Med. 1987;48(9):463–7.

    CAS  Google Scholar 

  17. Inui T, Carter WB, Problems and prospects for health services research on provider-patient communication. Med Care. 1985;23:521–38.

    Article  PubMed  CAS  Google Scholar 

  18. Mishler EG, Clark JA, Ingelfinger J, Simon MP. The language of attentive patient care: a comparison of two medical interviews. J Gen Intern Med. 1989;4:325–35.

    Article  PubMed  CAS  Google Scholar 

  19. Novack DH. Therapeutic aspects of the clinical encounter. Clin Rev. 1987;2:346–55.

    CAS  Google Scholar 

  20. Larsen LM, Smith CK. Assessment of nonverbal communication in the patient-physician interview. J Fam Pract. 1981;12:481–8.

    PubMed  CAS  Google Scholar 

  21. Roter D. The Roter method of interaction process analysis. Baltimore, MD: Johns Hopkins University Press, 1989.

    Google Scholar 

  22. Wasserman RC, Inui TS. Systematic analysis of clinician-patient interactions: a critique of recent approaches with suggestions for future research. Med Care. 1983;21:279–93.

    Article  PubMed  CAS  Google Scholar 

  23. Inui TS, Carter WB, Kukull WA, Haigh VH. Outcome-based doctor-patient interaction analysis: I. Comparison of techniques. Med Care. 1982;20:535–49.

    Article  PubMed  CAS  Google Scholar 

  24. Carter WB, Inui TS, Kukull WA, Haigh VH. Outcome-based doctor-patient interaction analysis: II. Identifying effective provider and patient behavior. Med Care. 1982;20:550–66.

    Article  PubMed  CAS  Google Scholar 

  25. Wolraich ML, Albanese M, Stone G, et al. Medical communications behavior system: an interactional analysis system for medical interaction. Med Care. 1986;24:891–903.

    Article  PubMed  CAS  Google Scholar 

  26. Kramer MS, Feinstein AR. Clinical biostatistics: LIV. The biostatistics of concordance. Clin Pharmacol Ther. 1981;1:111–23.

    Article  Google Scholar 

  27. Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley and Sons, 1986.

    Google Scholar 

  28. Nunnally JS. Psychometric theory. New York: McGraw Hill, 1978.

    Google Scholar 

  29. Herbers JE Jr, Noel GL, Cooper GS, Harvey J, Pangaro LN, Weaver MJ. How accurate are faculty evaluations of clinical competence? J Gen Intern Med. 1989,4:202–8.

    PubMed  Google Scholar 

  30. Branch W. Office practice of medicine. In: Lipkin M. The medical interview and related skills. Social-psychiatric problems. Philadelphia: W. B. Saunders, 1982.

    Google Scholar 

  31. Hampton JR, Harrison MJG, Mitchell JRA, Prichard JS, Seymour C. Relative contributions of history-taking, physical examination, and laboratory investigation to diagnosis and management of medical outpatients. Br Med J. 1975;31:486–9.

    Article  Google Scholar 

  32. Wolliscroft JO, Calhoun JG, Billiu GA, Stross JK, MacDonald M, Templeton B. House officer interviewing techniques: impact on data ellicitation and patient perceptions. J Gen Intern Med. 1989;4:108–14.

    Google Scholar 

  33. Wartman SA, Morlock LL, Malitz FE, Palm E. Do prescriptions adversely affect doctor-patient interactions? Am J Public Health. 1981;71:1358–61.

    PubMed  CAS  Google Scholar 

  34. Dirks JF, Schraa JC, Robinson SK. Patient mislabeling of symptoms: implications for patient-physician communication and medical outcome. Int J Psychiatry Med. 1982;12:15–27.

    Article  PubMed  CAS  Google Scholar 

  35. Maguire P, Rutter DR. History-taking for medical students: deficiencies in performance. Lancet. 1976;ii:556–8.

    Article  Google Scholar 

  36. Valente CM, Antlitz AM, Boyd MD, Troisi AJ. The importance of physician-patient communication in reducing medical liability. Med Mutual J. 1988;January:75–8.

    Google Scholar 

  37. Winslow R. Sometimes, talk is the best medicine for physicians, communication may avert suits. Wall Street Journal. October 5, 1989:B1.

    Google Scholar 

  38. Eisenberg JM. Evaluating internists’ clinical competence. J Gen Intern Med. 1989;4:139–43.

    PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Medicine, Gouverneur Diagnostic and Treatment Center, Room 479, 227 Madison Street, 10002, New York, NY

    Adina Kalet MD, MPH

Authors
  1. Adina Kalet MD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jo Anne Earp ScD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vicki Kowlowitz PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Received from the Robert Wood Johnson Clinical Scholars Program, Division of General Medicine, Department of Medicine, Department of Health Behavior and Health Education, School of Public Health, Office of Academic Affairs, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kalet, A., Earp, J.A. & Kowlowitz, V. How well do faculty evaluate the interviewing skills of medical students?. J Gen Intern Med 7, 499–505 (1992). https://doi.org/10.1007/BF02599452

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02599452

Key words

Search

Navigation