Skip to main content
SpringerLink
Log in

Construct Validation: View from the “Trenches”

  • Chapter
  • First Online:
Validating Psychological Constructs

Abstract

A review is given of the major construct validation frameworks falling roughly within three historic periods: mid-1950s; late 1950s to mid-1980s; and 1990s to current. The chapter then shifts in focus from scholarship prescribing how to validate constructs to a description of construct validation research as actually lived and practiced. The major findings from a fairly recent body of empirical research dedicated to the examination of psychometric reporting and validation practices are summarized in light of the prescriptions implied by the major validation frameworks. The chapter aims to get to the “ground floor” of CVT by examining the practices of researchers engaged in construct validation research so that recommendations can be appropriately targeted to those researchers.

Many types of evidence are relevant to construct , validityincluding content validity, interitem correlations, intertest correlations, test-“criterion” correlations, studies of stability over time, and stability under experimental intervention.… The investigation of a test’s construct validity is not essentially different from the general scientific procedures for developing and confirming

theories. —Cronbach and Meehl (1955, p. 300)

The concept that validity theorists are concerned with seems strangely divorced from the concept that working researchers have in mind when posing the question of validity

—Borsboom et al. (2004, p. 1061)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

    Google Scholar 

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1966). Standards for educational and psychological tests and manuals. Washington, DC: American Psychological Association.

    Google Scholar 

  • American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1974). Standards for educational and psychological tests. Washington, DC: American Psychological Association.

    Google Scholar 

  • Anastasi, A. (1961). Psychological testing. New York: Macmillan.

    Google Scholar 

  • Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1–15.

    Article  Google Scholar 

  • Barry, A. E., Chaney, B. H., Piazza-Gardner, A. K., & Chavarria, E. A. (2014). Validity and reliability reporting practices in the field of health education and behavior: A review of seven journals. Health Education and Behavior, 41(1), 12–18.

    Article  PubMed  Google Scholar 

  • Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071.

    Article  PubMed  Google Scholar 

  • Brennan, R. L. (Ed.). (2006). Educational measurement (4th ed.). Washington, DC: American Council on Education/Praeger.

    Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.

    Article  PubMed  Google Scholar 

  • Chapelle, C. A., Enright, M. K., & Jamieson, J. (2010). Does an argument-based approach to validity make a difference. Educational Measurement: Issues and Practice, 29(1), 3–13.

    Article  Google Scholar 

  • Cizek, G. J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justifications of test use. Psychological Methods, 17, 31–43.

    Article  PubMed  Google Scholar 

  • Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–412.

    Article  Google Scholar 

  • Crisp, V., & Shaw, S. D. (2012). Applying methods to evaluate construct validity in the context of A level assessment. Educational Studies, 38, 209–222.

    Article  Google Scholar 

  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity and psychological tests. Psychological Bulletin, 52, 281–302.

    Article  PubMed  Google Scholar 

  • Embretson, S. E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179–197.

    Article  Google Scholar 

  • Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380–396.

    Article  Google Scholar 

  • Garcia, D. A., Slaney, K.L., & Power, W. (2013, June). Construct validation: Does philosophical stance matter? Paper presented at the Canadian Psychological Association Annual Convention, Quebec City.

    Google Scholar 

  • Goldman, B. A., Mitchel, D. F., & Egelson, P. E. (1997). Directory of unpublished experimental mental measures (Vol. 7). Washington, DC: American Psychological Association.

    Google Scholar 

  • Goldstein, J., & Behuniak, P. (2011). Assumptions in alternate assessment: An argument-based approach to validation. Assessment for Effective Intervention, 36, 179–191.

    Article  Google Scholar 

  • Green, C. E., Chen, C. E., Helms, J. E., & Henze, K. T. (2011). Recent reliability reporting practices in psychological assessment: Recognizing the people behind the data. Psychological Assessment, 23, 656–669.

    Article  PubMed  Google Scholar 

  • Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of unidimensionality. Educational and Psychological Measurement, 37, 827–838.

    Article  Google Scholar 

  • Hogan, T. P., & Agnello, J. (2004). An empirical study of reporting practices concerning measurement validity. Educational and Psychological Measurement, 64, 802–812.

    Article  Google Scholar 

  • Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of various types. Educational and Psychological Measurement, 60, 523–531.

    Article  Google Scholar 

  • Jonson, J. L., & Plake, B. S. (1998). A historical comparison of validity standards and validity practices. Educational and Psychological Measurement, 58, 736–753.

    Article  Google Scholar 

  • Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.

    Article  Google Scholar 

  • Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319–342.

    Article  Google Scholar 

  • Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed.). Washington, DC: American Council on Education/Praeger.

    Google Scholar 

  • Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73.

    Article  Google Scholar 

  • Kane, M. T. (2016). Explicating validity. Assessment in Education: Principles, Policy & Practice, 23, 198–211.

    Article  Google Scholar 

  • Linn, R. L. (Ed.). (1989). Educational measurement (3rd ed.). New York: American Council on Education/Macmillan Publishing Company.

    Google Scholar 

  • Lissitz, R. W., & Samuelsen, K. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437–448.

    Article  Google Scholar 

  • Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 635–694.

    Google Scholar 

  • Martin, C., & Savage-McGlynn, E. (2013). A “good practice” guide for the reporting of design and analysis for psychometric evaluation. Journal of Reproductive and Infant Psychology, 31, 449–455.

    Article  Google Scholar 

  • McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100–117.

    Article  Google Scholar 

  • Mehrens, W. A. (1998). Consequences of assessment: What is the evidence? Education Policy Analysis Archives, 6, 1–30.

    Article  Google Scholar 

  • Meier, S. T., & Davis, S. R. (1990). Trends in reporting psychometric properties of scales used in counseling psychology research. Journal of Counselling Psychology, 37, 113–115.

    Article  Google Scholar 

  • Messick, S. (1965). Personality measurement and the ethics of assessment. American Psychologist, 20, 136–142.

    Article  PubMed  Google Scholar 

  • Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35, 1021–1027.

    Google Scholar 

  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: MacMillan.

    Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific enquiry into score meaning. American Psychologist, 50, 741–749.

    Article  Google Scholar 

  • Mislevy, R. J. (2009). Validity from the perspective of model-based reasoning. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 83–108). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  • Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessment. Measurement: Interdisciplinary Research and Perspectives, 1, 3–62.

    Google Scholar 

  • Newton, P. E. (2013). Two kinds of argument. Journal of Educational Measurement, 50, 105–109.

    Article  Google Scholar 

  • Newton, P. E., & Shaw, S. (2014). Validity in educational and psychological assessment. London: Sage.

    Book  Google Scholar 

  • Qualls, A. L., & Moss, A. D. (1996). The degree of congruence between test standards and test documentation within journal publications. Educational and Psychological Measurement, 56, 209–214.

    Article  Google Scholar 

  • Schilling, S. G., & Hill, H. C. (2007). Assessment measures of mathematical knowledge for teaching: A validity argument approach. Interdisciplinary Research and Perspectives, 5, 70–80.

    Article  Google Scholar 

  • Shaw, S. D., & Crisp, V. (2012). An approach to validation: Developing and applying an approach for the validation of general qualifications. Research Matters, Special Issue 3, 1–44.

    Google Scholar 

  • Shaw, S. D., Crisp, V., & Johnson, N. (2012). A framework for evidencing assessment validity in large-scale, high-stakes international examinations. Assessment in Education: Policy, Principles and Practice, 19, 159–176.

    Article  Google Scholar 

  • Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405–450.

    Google Scholar 

  • Sireci, S. G. (2013). Agreeing on validity arguments. Journal of Educational Measurement, 50, 99–104.

    Article  Google Scholar 

  • Slaney, K. L. (2013). Numeric coding data for construct validation methods and research aims/interpretations. Unpublished raw data.

    Google Scholar 

  • Slaney, K. L., Garcia, A., & Power, W. (2013, February). What’s so valid about construct validation research? Paper presented at the Annual Meeting for the Society of Theoretical and Philosophical Psychology, Austin, Texas.

    Google Scholar 

  • Slaney, K. L., & Maraun, M. D. (2008). A proposed framework for conducting data-based test analysis. Psychological Methods, 13, 376–390.

    Article  PubMed  Google Scholar 

  • Slaney, K. L., Storey, J. E., & Barnes, J. (2011). “Is my test valid?”: Guidelines for the practicing psychologist for evaluating the psychometric properties of measures. International Journal of Forensic Mental Health, 10, 261–283.

    Article  Google Scholar 

  • Slaney, K. L., Tkatchouk, M., Gabriel, S. M., Ferguson, L. P., Knudsen, J. R. S., & Legere, J. C. (2010). Assessment and reporting practices: An examination of measurement-oriented versus non-measurement-oriented domains. Canadian Journal of School Psychology, 25, 246–259.

    Article  Google Scholar 

  • Slaney, K. L., Tkatchouk, M., Gabriel, S. M., & Maraun, M. D. (2009). Psychometric assessment and reporting practices: Incongruencies between theory and practice. Journal of Psychoeducational Assessment, 27, 465–476.

    Article  Google Scholar 

  • Slaney, K. L., Tkatchouk, M., & Malange, R. (2014). Psychometric assessment and reporting practices: A quasi-replication involving four journals. Unpublished Manuscript. Burnaby, Canada: Department of Psychology, Simon Fraser University.

    Google Scholar 

  • Spearman, C. (1904a). The proof and measurement of association between two things. American Journal of Psychology, 15, 72–101.

    Article  Google Scholar 

  • Spearman, C. (1904b). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15, 201–292.

    Article  Google Scholar 

  • Thompson, B., & Snyder, P. A. (1998). Statistical significance and reliability analyses in recent Journal of Counseling & Development research articles. Journal of Counseling and Development, 76, 436–441.

    Article  Google Scholar 

  • Thorndike, R. L. (Ed.). (1971). Educational measurement (2nd ed.). Washington, DC: American Council on Education.

    Google Scholar 

  • Vacha-Haase, T., Henson, R. K., & Caruso, J. C. (2002). Reliability generalization: Moving toward improved understanding and use of score reliability. Educational and Psychological Measurement, 62, 562–569.

    Article  Google Scholar 

  • Vacha-Haase, T., Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity and score reliability inductions. Educational and Psychological Measurement, 60, 509–522.

    Article  Google Scholar 

  • Vacha-Haase, T., Ness, C., Nilsson, J., & Reetz, D. (1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335–341.

    Article  Google Scholar 

  • Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44, 159–168.

    Article  Google Scholar 

  • Vassar, M., & Hale, W. (2009). Reliability reporting across studies using the Buss Durkee Hostility Inventory. Journal of Interpersonal Violence, 24, 20–37.

    Article  PubMed  Google Scholar 

  • Weiss, D. J., & Davison, M. L. (1981). Test theory and methods. Annual Review of Psychology, 32, 629–659.

    Article  Google Scholar 

  • Whittington, D. (1998). How well do researchers report their measures? An evaluation of measurement in published educational research. Educational and Psychological Measurement, 58, 21–37.

    Article  Google Scholar 

  • Willson, V. L. (1980). Research techniques in AERJ articles: 1969–1978. Educational Researcher, 9, 5–10.

    Google Scholar 

  • Wolming, S., & Wikström, C. (2010). The concept of validity in theory and practice. Assessment in Education: Principles, Policy and Practice, 17, 117–132.

    Article  Google Scholar 

  • Zientek, L. R., Capraro, M. M., & Capraro, R. M. (2008). Reporting practices in quantitative teacher education research: One look at the evidence cited in the AERA Panel Report. Educational Researcher, 37, 208–216.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Simon Fraser University, Burnaby, BC, Canada

    Kathleen Slaney

Authors
  1. Kathleen Slaney
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathleen Slaney .

Copyright information

© 2017 The Author(s)

About this chapter

Cite this chapter

Slaney, K. (2017). Construct Validation: View from the “Trenches”. In: Validating Psychological Constructs. Palgrave Studies in the Theory and History of Psychology. Palgrave Macmillan, London. https://doi.org/10.1057/978-1-137-38523-9_9

Download citation

Publish with us

Policies and ethics

Search

Navigation