Abstract
Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. Available from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1100705.
Bauer, D.J., & Hussong, A.M. (2009). Psychometric approaches for developing commensurate measures across independent studies: traditional and new models. Psychological Methods, 14, 101–125.
Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.
Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park: Sage.
Dolan, C.V., & van der Maas, H.L.J. (1998). Fitting multivariate normal finite mixtures subject to structural equation modeling. Psychometrika, 63, 227–253.
Emmons, R.A. & McCullough, M.E. (Eds.) (2004). The psychology of gratitude. New York: Oxford University Press.
Ferguson, T.S. (1996). A course in large sample theory. London: Chapman & Hall.
Froh, J.J., Fan, J., Emmons, R.A., Bono, G., Huebner, E.S., & Watkins, P. (2011). Measuring gratitude in youth: assessing the psychometric properties of adult gratitude scales in children and adolescents. Psychological Assessment, 23, 311–324.
Genz, A., Bretz, F., Miwa, T., Mi, X., Leisch, F., Scheipl, F., et al. (2012). mvtnorm: multivariate normal and t distributions [Computer software manual]. Available from http://CRAN.R-project.org/package=mvtnorm (R package version 0.9-9992).
Hjort, N.L., & Koning, A. (2002). Tests for constancy of model parameters over time. Nonparametric Statistics, 14, 113–132.
Hothorn, T., & Zeileis, A. (2008). Generalized maximally selected statistics. Biometrics, 64(4), 1263–1269.
Huber, P.J. (1967). The behavior of maximum likelihood estimation under nonstandard conditions. In L.M. LeCam & J. Neyman (Eds.), Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Berkeley: University of California Press.
Lubke, G.H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21–39.
McCullough, M.E., Emmons, R.A., & Tsang, J.-A. (2002). The grateful disposition: a conceptual and empirical topography. Journal of Personality and Social Psychology, 82, 112–127.
Mellenbergh, G.J. (1989). Item bias and item response theory. International Journal of Educational Research, 13, 127–143.
Merkle, E.C., & Zeileis, A. (2013). Tests of measurement invariance without subgroups: a generalization of classical methods. Psychometrika, 78, 59–82.
Millsap, R.E. (2011). Statistical approaches to measurement invariance. New York: Routledge.
Molenaar, D., Dolan, C.V., Wicherts, J.M., & van der Mass, H.L.J. (2010). Modeling differentiation of cognitive abilities within the higher-order factor model using moderated factor analysis. Intelligence, 38, 611–624.
Purcell, S. (2002). Variance components models for gene-environment interaction in twin analysis. Twin Research, 5, 554–571.
R Development Core Team. (2012). R: a language and environment for statistical computing [Computer software manual]. URL http://www.R-project.org/. Vienna, Austria. (ISBN 3-900051-07-0)
Rosseel, Y. (2012). lavaan: an R package for structural equation modeling. Journal of Statistical Software, 48(2), 136. Available from http://www.jstatsoft.org/v48/i02/.
Satorra, A. (1989). Alternative test criteria in covariance structure analysis: a unified approach. Psychometrika, 54, 131–151.
Satorra, A., & Bentler, P.M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514.
Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–384.
Strobl, C., Kopf, J., & Zeileis, A. (2013). A new method for detecting differential item functioning in the Rasch model. Psychometrika. doi:10.1007/s11336-013-9388-3.
Thomas, M., & Watkins, P. (2003). Measuring the grateful trait: development of the revised GRAT. Poster presented at the Annual Convention of the Western Psychological Association, Vancouver, BC.
Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.
Yuan, K.-H., & Bentler, P.M. (1997). Mean and covariance structure analysis: theoretical and practical improvements. Journal of the American Statistical Association, 92, 767–774.
Zeileis, A. (2006a). Implementing a class of structural change tests: an econometric computing approach. Computational Statistics & Data Analysis, 50(11), 2987–3008.
Zeileis, A. (2006b). Object-oriented computation of sandwich estimators. Journal of Statistical Software, 16(9), 1–16. Available from http://www.jstatsoft.org/v16/i09/.
Zeileis, A., & Hornik, K. (2007). Generalized M-fluctuation tests for parameter instability. Statistica Neerlandica, 61, 488–508.
Zeileis, A., Leisch, F., Hornik, K., & Kleiber, C. (2002). strucchange: an R package for testing structural change in linear regression models. Journal of Statistical Software, 7, 1–38. Available from http://www.jstatsoft.org/v07/i02/.
Acknowledgements
This work was supported by National Science Foundation grant SES-1061334.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Merkle, E.C., Fan, J. & Zeileis, A. Testing for Measurement Invariance with Respect to an Ordinal Variable. Psychometrika 79, 569–584 (2014). https://doi.org/10.1007/s11336-013-9376-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-013-9376-7