Abstract
Passing scores for licensure and certification tests are justified by showing that decisions based on the passing score achieve the purposes of the credentialing program while avoiding any serious negative consequences. The standard should be high enough to provide adequate protection for the public, and not so high as to unnecessarily restrict the supply of qualified practitioners or to exclude competent candidates from practicing. This paper begins by examining the intended outcomes of licensure and certification programs and by outlining the interpretive argument that is typically used for written credentialing examinations. Criteria are then developed for evaluating standard-setting methods in terms of how well they serve the goals of protecting the public, maintaining an adequate supply of practitioners, and protecting the rights of candidates. Finally, the criteria are used to evaluate the Angoff method. This analysis identifies two potential sources of bias in the Angoff method, and suggests ways to control these weaknesses.
References
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational Measurement, 2nd edn., pp. 508–600. American Council on Education, Washington, DC.
Berk, R. (1986). A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research 56: 137–172.
Busch, M. I. and Jaeger, R. M. (1990). Influence of type of judge, normative information, and discussion on standards recommended for the National Teacher Examination. Journal of Educational Measurement 27: 145–163.
Clauser, B. and Clyman, S. (1994). A contrasting groups approach to standard setting for performance assessments of clinical skills. Academic Medicine, RIME Supplement 69(10): S42-S44.
Clauser, B. E. and Nungester, R. J. (1997). Setting standards on performance assessments of physician's clinical skills using contrasting groups and receiver operating characteristic curves. Evaluation and the Health Professions 20: 215–238.
Ebel, R. L. (1972). Essentials of Educational Measurement. Prentice-Hall, Englewood Cliffs, NJ.
Harker, J. and Cope, R. (1988). The Effects of Several Variables on Judgementally-Obtained Cut Scores. Paper presented at the annual meeting of the National Council on measurement in Education, New Orleans.
Jaeger, R. M. (1982). An iterative structured judgment process for establishing standards on competency tests: Theory and application. Educational Evaluation and Policy Analysis 4: 461–476.
Jaeger, R. M. (1989). Certification of Student Competence. In R. L. Linn (Ed.), Educational Measurement, 3rd edn., pp. 485–514. American Council on Education and Macmillan, New York.
Kane, M. (1992). An argument-based approach to validity. Psychological Bulletin 112: 527–535.
Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research 64: 425–461.
Kane, M. (1995). Examinee-centered vs. task-centered standard setting. In Joint Conference on Standard Setting for Large-scale Assessments, Proceedings, Vol. II, pp. 119–141.
Kane, M. and Wilson, J. (1984). Errors of measurement and standard setting in mastery testing. Applied Psychological Measurement 8: 107–115.
LaDuca, A. (1994). Validation of professional licensure examinations: Professions theory, test design, and construct validity. Evaluations and the Health Professions 17(2): 178–197.
Livingston, S. A. and Zieky, M. J. (1982). Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. Educational Testing Service, Princeton, NJ.
McLaughlin, D. H. (1993). Validity of the 1992 NAEP Achievement-Level-Setting Process. In Setting Performance Standards for Student Achievement: Background Studies. The National Academy of Education, Stanford, CA.
Mehrens, W. A. (1995). Methodological issues in standard setting for educational exams. In Joint Conference on Standard Setting for Large-Scale Assessments, Proceedings, Volume II. US Government Printing Office.
Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement 14: 3–19.
Norcini, J. and Shea, J. (1997). The credibility and comparability of standards. Applied Measurement in Education 10: 39–59.
Norcini, J. (1994). Research on standards for professional licensure and certification examinations. Evaluations and the Health Professions 17(2): 160–177.
Norcini, J., Shea, J. and Ping J. (1988). A note on the application of multiple matrix sampling to standard setting. Journal of Educational Measurement 25(2): 159–164.
Norcini, J., Shea, J. and Kanya, D. (1988). The effect of various factors on standard setting. Journal of Educational Measurement 25: 7–65.
Plake, B. (1996). Setting Performance Standards for Professional Licensure and Certification. Paper presented at workshop on NAEP Achievement Levels: Setting Consensus Goals for Academic Achievement, Washington, DC.
Shepard, L. (1984) Setting performance standards. In R. A. Berk (Ed.), A Guide to Criterion-Referenced Test Construction. Johns Hopkins Press, Baltimore.
Shepard, L., Glaser, R., Linn, R. and Bohrnstedt, G. (1993). Setting Performance Standards for Student Achievement. National Academy of Education, Stanford University, Stanford, CA.
Sireci, S. A. and Biskin, B. M. (1992). Measurement practices in national licensing examination programs: A survey. CLEAR Exam Review 3: 21–35.
Tamblyn, R. (1994). Is the public being protected? Suboptimal medical practice through training programs and credentialing examinations. Evaluations and the Health Professions 17(2): 198–221.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kane, M., Crooks, T. & Cohen, A. Designing and Evaluating Standard-Setting Procedures for Licensure and Certification Tests. Adv Health Sci Educ Theory Pract 4, 195–207 (1999). https://doi.org/10.1023/A:1009849528247
Issue Date:
DOI: https://doi.org/10.1023/A:1009849528247