Abstract
Changes in certification requirements and examinee characteristics are likely to influence the validity of the evidence associated with interpretations made based on test data. We examined whether changes in Educational Commission for Foreign Medical Graduates (ECFMG) certification requirements over time were associated with changes in internal medicine (IM) residency program director ratings and certification examination scores. Comparisons were made between physicians who were ECFMG-certified before and after the Clinical Skills Assessment (CSA) requirement. A multivariate analysis of covariance was conducted to examine the differences in program director ratings based on CSA cohort and whether the examinees emigrated for undergraduate medical education (national vs. international students). A univariate analysis of covariance was conducted to examine differences in scores from the American Board of Internal Medicine (ABIM) Internal Medicine Certification Examination. For both analyses, United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores were used as covariates. Results indicate that, of those certified by ECFMG between 1993 and 1997, 17 % (n = 1,775) left their country of citizenship for undergraduate medical education. In contrast, 38 % (n = 1,874) of those certified between 1999 and 2003 were international students. After adjustment by covariates, the main effect of cohort membership on the program director ratings was statistically significant (Wilks’ λ = 0.99, F 5, 15391 = 19.9, P < 0.001). However, the strength of the relationship between cohort group and the ratings was weak (η = 0.01). The main effect of migration status was statistically significant and weak (Wilks’ λ = 0.98, F 5,15391 = 45.3, P < 0.01; η = 0.02). Differences in ABIM Internal Medicine Certification Examination scores based on whether or not CSA were required was statistically significant, although the magnitude of the association between these variables was very small. The findings suggest that the implementation of an additional evaluation of skills (e.g., history-taking, physical examination) as a prerequisite to postgraduate medical education (residency) provides some additional, relevant data to those who select ECFMG-certified residents.
Similar content being viewed by others
References
Boulet, J. R., Cooper, R. A., Seeling, S. S., Norcini, J. J., & McKinley, D. W. (2009). U.S. citizens who obtain their medical degrees abroad: An overview, 1992–2006. Health Affairs, 28(1), 226–233.
Gary, N. E., Sabo, M. M., Shafron, M. L., Wald, M. K., Ben-David, M. F., & Kelly, W. C. (1997). Graduates of foreign medical schools: Progression to certification by the Educational Commission for Foreign Medical Graduates. Academic Medicine: Journal of the Association of American Medical Colleges, 72(1), 17–22.
Hallock, J. A., & Kostis, J. B. (2006). Celebrating 50 years of experience: An ECFMG perspective. Academic Medicine: Journal of the Association of American Medical Colleges, 81(12 Suppl), S7–16. doi:10.1097/01.ACM.0000243344.55996.1e.
Hallock, J. A., McKinley, D. W., & Boulet, J. R. (2007). Migration of doctors for undergraduate medical education. Medical Teacher, 29(2–3), 98–105.
Holland, P. W., & Dorans, N. J. (2006). Linking and equating. In R. Brennan (Ed.), Educational measurement (4th ed., pp. 187–220). Westport, CT: Praeger Publishers.
IBM. (2010). SPSS. Chicago, IL.
Kane, M. T. (1994). Validating interpretive arguments for licensure and certification examinations. Evaluation and the Health Professions, 17(2), 133–159.; discussion 236–241.
Kane, Michael T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. doi:10.1111/jedm.12000.
Melnick, D. E. (2006). From defending the walls to improving global medical education: Fifty years of collaboration between the ECFMG and the NBME. Academic Medicine, 81(12 Suppl), S30–S35. doi:10.1097/01.ACM.0000243462.05719.e1.
Norcini, J., Anderson, M. B., & McKinley, D. W. (2006). The medical education of United States citizens who train abroad. Surgery, 140(3), 338–346.
Rosnow, R. L., & Rosenthal, R. (2003). Effect sizes for experimenting psychologists. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale, 57, 221–237. doi:10.1037/h0087427.
Shea, J. A., Norcini, J. J., & Kimball, H. R. (1993). Relationships of ratings of clinical competence and ABIM scores to certification status. Academic Medicine, 68(10 Suppl), S22–S24.
Tamblyn, R., Abrahamowicz, M., Dauphinee, W. D., Hanley, J. A., Norcini, J., Girard, N., et al. (2002). Association between licensure examination scores and practice in primary care. JAMA, 288(23), 3019–3026.
Van Zanten, M., Boulet, J. R., McKinley, D. W., De Champlain, A., & Jobe, A. C. (2007). Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills (CS) Exam. Academic Medicine, 82(10 Suppl), S65–S68.
Whelan, G. P., Boulet, J. R., McKinley, D. W., Norcini, J. J., Van Zanten, M., Hambleton, R. K., et al. (2005). Scoring standardized patient examinations: Lessons learned from the development and administration of the ECFMG clinical skills assessment (CSA). Medical Teacher, 27(3), 200–206.
Whelan, G. P., Gary, N. E., Kostis, J., Boulet, J. R., & Hallock, J. A. (2002). The changing pool of international medical graduates seeking certification training in US graduate medical education programs. JAMA, 288(9), 1079–1084.
Acknowledgments
This research was supported by the Foundation for Advancement of International Medical Education and Research and the American Board of Internal Medicine. The findings and conclusions do not necessarily reflect the opinions of the organizations.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
McKinley, D.W., Hess, B.J., Boulet, J.R. et al. Examining changes in certification/licensure requirements and the international medical graduate examinee pool. Adv in Health Sci Educ 19, 19–28 (2014). https://doi.org/10.1007/s10459-013-9456-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-013-9456-6