Skip to main content
SpringerLink
Log in

Raw Data Maximum Likelihood Estimation for Common Principal Component Models: A State Space Approach

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

The specifications of state space model for some principal component-related models are described, including the independent-group common principal component (CPC) model, the dependent-group CPC model, and principal component-based multivariate analysis of variance. Some derivations are provided to show the equivalence of the state space approach and the existing Wishart-likelihood approach. For each model, a numeric example is used to illustrate the state space approach. In addition, a simulation study is conducted to evaluate the standard error estimates under the normality and nonnormality conditions. In order to cope with the nonnormality conditions, the robust standard errors are also computed. Finally, other possible applications of the state space approach are discussed at the end.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The term “eigenvalue” used here (and frequently in Flury’s work) may not be the same as that in the traditional sense. In case \(K = 2\), it is possible that the eigenvalues subsumed by \({\varvec{\Lambda }}_{kk}\) may not be strictly ranked in descending order.

  2. The block-diagonal matrix is obtained by permutation of columns of a parallel-diagonal matrix. Details of permutation can be found in Flury and Neuenschwander (1995).

  3. For each square, but asymmetric, off-diagonal block \({\varvec{\Sigma }}_{kh} (k \ne h)\), such diagonalization requires two different orthogonal matrices, a process conceptually similar to singular value decomposition of an arbitrary square matrix. Because there are \(K(K - 1)/2\) unique off-diagonal blocks for the general cases, \(K(K - 1)\) orthogonal matrices are required.

References

  • Anderson, T. W. (2003). An introduction to multivariate statistical analysis (3rd ed.). New York: Wiley.

    Google Scholar 

  • Bauer, D. J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics, 28, 135–167.

    Article  Google Scholar 

  • Bechger, T. M., Blanca, M. J., & Maris, G. (2014). The analysis of multivariate group differences using common principal components. Structural Equation Modeling, 21, 577–587.

    Article  Google Scholar 

  • Browne, M. W., & Shapiro, A. (1988). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematical and Statistical Psychology, 41, 193–208.

    Article  Google Scholar 

  • Caines, P. E., & Rissanen, J. (1974). Maximum likelihood estimation of parameters in multivariate Gaussian stochastic processes. IEEE Transactions on Information Theory, 20, 102–104.

    Article  Google Scholar 

  • Curran, P. J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38, 529–569.

    Article  PubMed  Google Scholar 

  • Dolan, C. V. (1996). Principal component analysis using LISREL 8. Structural Equation Modeling, 2, 307–322.

    Article  Google Scholar 

  • Dolan, C. V., Bechger, T. M., & Molenaar, P. C. M. (1999). Using structural equation modeling to fit models incorporating principal components. Structural Equation Modeling, 6, 233–261.

    Article  Google Scholar 

  • Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532.

    Article  Google Scholar 

  • Flury, B. N. (1984). Common principal components in k groups. Journal of the American Statistical Association, 79, 892–898.

    Google Scholar 

  • Flury, B. N. (1986). Asymptotic theory for common principal component analysis. The Annals of Statistics, 14, 418–430.

    Article  Google Scholar 

  • Flury, B. N. (1987). Two generalizations of the common principal component model. Biometrika, 74, 59–69.

    Article  Google Scholar 

  • Flury, B. N. (1988). Common principal components and related multivariate models. New York: Wiley.

    Google Scholar 

  • Flury, B. N., & Constantine, G. (1985). Algorithm AS 211: The F-G algorithm. Journal of the Royal Statistical Society Series C (Applied Statistics), 34, 177–183.

    Google Scholar 

  • Flury, B. N., & Gautschi, W. (1986). An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. SIAM Journal on Scientific and Statistical Computing, 7, 169–184.

    Article  Google Scholar 

  • Flury, B. D., & Neuenschwander, B. E. (1994). Simultaneous diagonalization algorithms with applications in multivariate statistics. In R. V. M. Zahar (Ed.), Approximation and computation: A Festschrift in honor of Walter Gautschi (pp. 179–205). Boston: Birkhäuser.

    Chapter  Google Scholar 

  • Flury, B. D., & Neuenschwander, B. E. (1995). Principal component models for patterned covariance matrices with application to canonical correlation analysis of several sets of variables. In W. J. Krzanowski (Ed.), Recent advances in descriptive multivariate analysis (pp. 90–112). New York: Oxford University Press.

    Google Scholar 

  • Goria, M. N., & Flury, B. D. (1996). Common canonical variates in \(k\) independent groups. Journal of the American Statistical Association, 91, 1735–1742.

    Google Scholar 

  • Gu, F., Preacher, K. J., Wu, W., & Yung, Y.-F. (2014). A computationally efficient state space approach to estimating multilevel regression models and multilevel confirmatory factor models. Multivariate Behavioral Research, 49, 119–129.

    Article  PubMed  Google Scholar 

  • Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 417–441.

    Article  Google Scholar 

  • Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, 221–233.

    Google Scholar 

  • Jones, R. H. (1993). Longitudinal data with serial correlation: A state-space approach. London: Chapman & Hall.

    Book  Google Scholar 

  • Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering (Series D), 82, 35–45.

    Article  Google Scholar 

  • Kiers, H. A. L., & ten Berg, J. M. F. (1989). Alternating least squares algorithms for simultaneous components analysis with equal component weight matrices in two or more populations. Psychometrika, 54, 467–473.

    Article  Google Scholar 

  • Kiers, H. A. L., & ten Berg, J. M. F. (1994). Hierarchical relations between methods for simultaneous component analysis and a technique for rotation to a simple simultaneous structure. British Journal of Mathematical and Statistical Psychology, 47, 109–126.

    Article  Google Scholar 

  • Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R., Kirkpatrick, R. M., & Boker, S. M. (2015). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika. doi:10.1007/s11336-014-9435-8.

  • Neuenschwander, B. E. (1991). Common principal components for dependent random vectors. Unpublished doctoral dissertation. Switzerland: Department of Statistics, University of Berne.

  • Neuenschwander, B. E., & Flury, B. D. (1995). Common canonical variates. Biometrika, 82, 553–560.

    Article  Google Scholar 

  • Neuenschwander, B. E., & Flury, B. D. (2000). Common principal components for dependent random vectors. Journal of Multivariate Analysis, 75, 163–183.

    Article  Google Scholar 

  • Nevitt, J., & Hancock, G. R. (2001). Performance of bootstrapping approaches to model test statistics and parameter standard error estimation in structural equation modeling. Structural Equation Modeling, 8, 353–377.

    Article  Google Scholar 

  • Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2, 559–572.

    Article  Google Scholar 

  • R Development Core Team. (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org.

  • Schweppe, F. (1965). Evaluation of likelihood functions for Gaussian signals. IEEE Transactions on Information Theory, 11, 61–70.

    Article  Google Scholar 

  • Timmerman, M. E., & Kiers, H. A. L. (2003). Four simultaneous component models for the analysis of multivariate time series from more than one subject to model intraindividual and interindividual differences. Psychometrika, 68, 105–121.

    Article  Google Scholar 

  • Trendafilov, N. T. (2010). Stepwise estimation of common principal components. Computational Statistics and Data Analysis, 54, 3446–3457.

    Article  Google Scholar 

  • Yamada, T. (2008). The efficiency of the asymptotic expansion of the distribution of the canonical vector under nonnormality. Journal of The Japan Statistical Society, 38, 451–474.

    Article  Google Scholar 

  • White, H. (1980). A heteroscedasticity-consistent covariance matrix estimator and a direct test for heteroscedasticity. Econometrica, 48, 817–838.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Yiu-Fai Yung for his valuable comments and help with the programming in SAS/IML.

Author information

Authors and Affiliations

  1. McGill University, Montreal, Quebec, Canada

    Fei Gu

  2. Boston College, Chestnut Hill, MA, USA

    Hao Wu

Authors
  1. Fei Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Gu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, F., Wu, H. Raw Data Maximum Likelihood Estimation for Common Principal Component Models: A State Space Approach. Psychometrika 81, 751–773 (2016). https://doi.org/10.1007/s11336-016-9504-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-016-9504-2

Keywords

Search

Navigation