Skip to main content
SpringerLink
Log in

Linear Mixed Models: Part I

  • Chapter
  • First Online:
Linear and Generalized Linear Mixed Models and Their Applications

Part of the book series: Springer Series in Statistics ((SSS))

Abstract

The best way to understand a linear mixed model , or mixed linear model in some earlier literature, is to first recall a linear regression model. The latter can be expressed as y = Xβ + 𝜖, where y is a vector of observations, X is a matrix of known covariates, β is a vector of unknown regression coefficients, and 𝜖 is a vector of (unobservable random) errors. In this model, the regression coefficients are considered as fixed, unknown constants. However, there are cases in which it makes sense to assume that some of these coefficients are random.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Allen, H. L., Estrada, K., Lettre, G., Berndt, S. I., Weedon, M. N., Rivadeneira, F., and et al. (2010). Hundreds of variants clustered clustered in genomic loci and biological pathways affect human height. Nature, 467, 832–838.

    Google Scholar 

  • Anderson, R. D. (1979), Estimating variance components from balanced data: Optimum properties of REML solutions and MIVQUE estimators, in Variance Components and Animal Breeding (L. D. VanVleck and S. R. Searle, eds.), 205–216, Dept. of Animal Sci., Cornell Univ.

    Google Scholar 

  • Anderson, T. W. (1969), Statistical inference for covariance matrices with linear structure, Proc. 2nd Internat. Symp. Multivariate Anal. (P. R. Krishnaiah, ed.), 55–66, Academic Press, New York.

    Google Scholar 

  • Anderson, T. W. (1971a), Estimation of covariance matrices with linear structure and moving average process of finite order, Tech. Report No. 6, Dept. of Statist., Stanford Univ.

    Google Scholar 

  • Anderson, T. W. (1971b), The Statistical Analysis of Time Series, Wiley, New York.

    MATH  Google Scholar 

  • Barndorff-Nielsen, O. (1983), On a formula for the distribution of the maximum likelihood estimator, Biometrika 70, 343–365.

    Article  MathSciNet  MATH  Google Scholar 

  • Bates, D., Mächler, M., Bolker, B. and Walker, S. (2015), Fitting linear mixed-effects models using lme4, J. Statist. Software 67, 1–48.

    Article  Google Scholar 

  • Brown, K. G. (1976), Asymptotic behavior of MINQUE-type estimators of variance components, Ann. Statist. 4, 746–754.

    Article  MathSciNet  MATH  Google Scholar 

  • Cochran, W.G. (1977), Sampling Techniques, 3rd ed., Wiley, New York.

    MATH  Google Scholar 

  • Cramér, H. (1946), Mathematical methods of statistics, Princeton Univ. Press, Princeton, NJ.

    MATH  Google Scholar 

  • Cressie, N. and Lahiri, S. N. (1993), The asymptotic distribution of REML estimators, J. Multivariate Anal. 45, 217–233.

    Article  MathSciNet  MATH  Google Scholar 

  • Das, K. (1979), Asymptotic optimality of restricted maximum likelihood estimates for the mixed model, Calcutta Statist. Assoc. Bull. 28, 125–142.

    Article  MathSciNet  MATH  Google Scholar 

  • Datta, G. S. and Lahiri, P. (2000), A unified measure of uncertainty of estimated best linear unbiased predictors in small area estimation problems, Statist. Sinica 10, 613–627.

    MathSciNet  MATH  Google Scholar 

  • Demidenko, E. (2013), Mixed Models—Theory and Application with R, 2nd ed., Wiley, New York.

    MATH  Google Scholar 

  • Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), Maximum likelihood from incomplete data via de EM algorithm (with discussion), J. Roy. Statist. Soc. B 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  • Diggle, P. J., Heagerty, P., Liang, K. Y., and Zeger, S. L. (2002), Analysis of Longitudinal Data, 2nd ed., Oxford Univ. Press.

    MATH  Google Scholar 

  • Efron, B. and Hinkley, D. V. (1978), Assessing the accuracy of the maximum likelihood estimator: observed versus expected Fisher information, Biometrika 65, 457–487.

    Article  MathSciNet  MATH  Google Scholar 

  • Fan, J., Guo, S. and Hao, N. (2012), Variance estimation using refitted cross-validation in ultrahigh dimensional regression, J. Roy. Statist. Soc. Ser. B 74, 37–65.

    Article  MathSciNet  MATH  Google Scholar 

  • Fay, R. E. and Herriot, R. A. (1979), Estimates of income for small places: An application of James-Stein procedures to census data, J. Amer. Statist. Assoc. 74, 269–277.

    Article  MathSciNet  Google Scholar 

  • Fisher, R. A. (1922a), On the mathematical foundations of theoretical statistics, Phil. Trans. R. Soc. Lond., A 222, 309–368.

    Article  MATH  Google Scholar 

  • Gan, L. and Jiang, J. (1999), A test for global maximum, J. Amer. Statist. Assoc. 94, 847–854.

    Article  MathSciNet  MATH  Google Scholar 

  • Goldstein, H. (1986), Multilevel mixed linear model analysis using iterative generalized least squares, Biometrika 73, 43–56.

    Article  MathSciNet  MATH  Google Scholar 

  • Hand, D. and Crowder, M. (1996), Practical Longitudinal Data Analysis, Chapman and Hall, London.

    Book  MATH  Google Scholar 

  • Hartley, H. O. and Rao, J. N. K. (1967), Maximum likelihood estimation for the mixed analysis of variance model, Biometrika 54, 93–108.

    Article  MathSciNet  MATH  Google Scholar 

  • Harville, D. A. (1974), Bayesian inference for variance components using only error contrasts, Biometrika 61, 383–385.

    Article  MathSciNet  MATH  Google Scholar 

  • Harville, D. A. (1977), Maximum likelihood approaches to variance components estimation and related problems, J. Amer. Statist. Assoc. 72, 320–340.

    Article  MathSciNet  MATH  Google Scholar 

  • Harville, D. A. and Fenech, A. P. (1985), Confidence intervals for a variance ratio, or for heritability, in an unbalanced mixed linear model, Biometrics 41, 137–152.

    Article  MathSciNet  MATH  Google Scholar 

  • Henderson, C. R. (1953), Estimation of variance and covariance components, Biometrics 9, 226–252.

    Article  MathSciNet  Google Scholar 

  • Heyde, C. C. (1994), A quasi-likelihood approach to the REML estimating equations, Statist. & Probab. Letters 21, 381–384.

    Article  MathSciNet  MATH  Google Scholar 

  • Heyde, C. C. (1997), Quasi-likelihood and Its Application, Springer, New York.

    Book  MATH  Google Scholar 

  • Jiang, J. (1996), REML estimation: Asymptotic behavior and related topics, Ann. Statist. 24, 255–286.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, J. (1997a), Wald consistency and the method of sieves in REML estimation, Ann. Statist. 25, 1781–1803.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, J. (1998b), Asymptotic properties of the empirical BLUP and BLUE in mixed linear models, Statistica Sinica 8, 861–885.

    MathSciNet  MATH  Google Scholar 

  • Jiang, J. (2003b), Empirical method of moments and its applications, J. Statist. Plann. Inference 115, 69–84.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, J. (2004), Dispersion matrix in balanced mixed ANOVA models, Linear Algebra Appl. 382, 211–219.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, J. (2005a), Partially observed information and inference about non-Gaussian mixed linear models, Ann. Statist. 33, 2695–2731.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, J. and Lahiri, P. (2004), Robust dispersion tests for longitudinal generalized linear mixed models using Jackknife method, unpublished manuscript.

    Google Scholar 

  • Jiang, J., Lahiri, P. and Wan, S. (2002), A unified jackknife theory for empirical best prediction with M-estimation, Ann. Statist. 30, 1782–1810.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, J., Li, C., Paul, D., Yang, C., and Zhao, H. (2016), On high-dimensional misspecified mixed model analysis in genome-wide association study, Ann. Statist. 44, 2127–2160.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang. J., Luan, Y. and Wang, Y.-G. (2007), Iterative estimating equations: Linear convergence and asymptotic properties, Ann. Statist. 35, 2233–2260.

    Article  MathSciNet  MATH  Google Scholar 

  • Kackar, R. N. and Harville, D. A. (1984), Approximations for standard errors of estimators of fixed and random effects in mixed linear models, J. Amer. Statist. Assoc. 79, 853–862.

    MathSciNet  MATH  Google Scholar 

  • Khuri, A. I. and Sahai, H. (1985), Variance components analysis: A selective literature survey, Internat. Statist. Rev. 53, 279–300.

    Article  MathSciNet  MATH  Google Scholar 

  • Laird, N. M. and Ware, J. M. (1982), Random effects models for longitudinal data, Biometrics 38, 963–974.

    Article  MATH  Google Scholar 

  • Lange, N. and Ryan, L. (1989), Assessing normality in random effects models, Ann. Statist. 17, 624–642.

    Article  MathSciNet  MATH  Google Scholar 

  • Lee, S. H., DeCandia, T. R., Ripke, S., Yang, J., Sullivan, P. F., Goddard, M. E., and et al. (2012), Estimating the proportion of variation in susceptibility to schizophrenia captured by common SNPs, Nature Genetics, 44, 247–250.

    Google Scholar 

  • Littell, R. C., Milliken, G. A., Stroup, W. W., and Wolfinger, R. D. (1996), SAS System for Mixed Models, SAS Institute Inc.

    Google Scholar 

  • Loh, P.-R. et al. (2015a), Efficient Bayesian mixed model analysis increases association power in large cohorts, Nature Genetics 47, 284–290.

    Article  Google Scholar 

  • Loh, P.-R. et al. (2015b), Contrasting genetic architectures of schizophrenia and other complex diseases using fast variance components analysis, Nature Genetics 47, 1385–1392.

    Article  Google Scholar 

  • Loh, P.-R., Kichaev, G., Gazal, S., Schoech, A. P. and Price, A. L. (2018), Mixed-model association for biobank-scale datasets, Nature Genetics 50, 906–908.

    Article  Google Scholar 

  • Luenberger, D. G. (1984), Linear and Nonlinear Programming, Addison-Wesley, Reading, MA.

    MATH  Google Scholar 

  • Maher, B. (2008). Personal genomes: The case of the missing heritability. Nature, 456, 18–21.

    Article  Google Scholar 

  • Miller, J. J. (1977), Asymptotic properties of maximum likelihood estimates in the mixed model of analysis of variance, Ann. Statist. 5, 746–762.

    Article  MathSciNet  MATH  Google Scholar 

  • Neyman, J. and Scott, E. (1948), Consistent estimates based on partially consistent observations, Econometrika 16, 1–32.

    Article  MathSciNet  MATH  Google Scholar 

  • Patterson, H. D. and Thompson, R. (1971), Recovery of interblock information when block sizes are unequal, Biometrika 58, 545–554.

    Article  MathSciNet  MATH  Google Scholar 

  • Quenouille, M. (1949), Approximation tests of correlation in time series, J. R. Statist. Soc. B 11, 18–84.

    MATH  Google Scholar 

  • Rao, C. R. and Kleffe, J. (1988), Estimation of Variance Components and Applications, North-Holland, Amsterdam.

    MATH  Google Scholar 

  • Rao, J. N. K. and Molina, I. (2015), Small Area Estimation, 2nd ed., Wiley, New York.

    Book  MATH  Google Scholar 

  • Richardson, A. M. and Welsh, A. H. (1994), Asymptotic properties of restricted maximum likelihood (REML) estimates for hierarchical mixed linear models, Austral. J. Statist. 36, 31–43.

    Article  MathSciNet  MATH  Google Scholar 

  • Robinson, D. L. (1987), Estimation and use of variance components, The Statistician 36, 3–14.

    Article  Google Scholar 

  • Scheffé, H. (1959), The Analysis of Variance, Wiley, New York.

    MATH  Google Scholar 

  • Searle, S. R., Casella, G., and McCulloch, C. E. (1992), Variance Components, Wiley, New York.

    Book  MATH  Google Scholar 

  • Shao, J. and Tu, D. (1995), The Jackknife and Bootstrap, Springer, New York.

    Book  MATH  Google Scholar 

  • Speed, D., Hemani, G., Johnson, M. R., and Balding, D. J. (2012), Improved heritability estimation from genome-wide SNPs, Amer. J. Human Genetics 91, 1011–1021.

    Article  Google Scholar 

  • Speed, T. P. (1997), Restricted maximum likelihood (REML), Encyclopedia of Statistical Sciences 1, 472–481.

    Google Scholar 

  • Thisted, R. A. (1988), Elements of Statistical Computing—Numerical Computation, Chapman and Hall, London.

    MATH  Google Scholar 

  • Thompson, W. A., Jr. (1962), The problem of negative estimates of variance components, Ann. Math. Statist. 33, 273–289.

    Article  MathSciNet  MATH  Google Scholar 

  • Tukey, J. (1958), Bias and confidence in not quite large samples, Ann. Math. Statist. 29, 614.

    Google Scholar 

  • Vattikuti, S., Guo, J., and Chow, C. C. (2012), Heritability and genetic correlations explained by common snps for metabolic syndrome traits, PLoS genetics, 8, e1002637.

    Article  Google Scholar 

  • Verbyla, A. P. (1990), A conditional derivation of residual maximum likelihood, Austral. J. Statist. 32, 227–230.

    Article  Google Scholar 

  • Visscher, P. M., Hill, W. G., and Wray, N. R. (2008), Heritability in the genomics era - concepts and misconceptions, Nature Reviews Genetics, 9, 255–266.

    Article  Google Scholar 

  • Visscher, P. M., Brown, M. A., McCarthy, M. I., and Yang, J. (2012), Five years of GWAS discovery, Amer. J. Human Genetics, 90, 7–24.

    Article  Google Scholar 

  • Wald, A. (1949), Note on the consistency of the maximum likelihood estimate, Ann. Math. Statist. 20, 595–601.

    Article  MathSciNet  MATH  Google Scholar 

  • Yang, J., Benyamin, B., McEvoy, B. P., Gordon, S., Henders, A. K., Nyholt, D. R., and et al. (2010), Common SNPs explain a large proportion of the heritability for human height, Nature Genetics 42, 565–569.

    Article  Google Scholar 

  • Yang, J., Lee, S. H., Goddard, M. E., and Visscher, P. M. (2011), GCTA: a tool for genome-wide complex trait analysis, Amer. J. Human Genetics 88, 76–82.

    Article  Google Scholar 

  • Yang, J., Zaitlen, N. A., Goddard, M. E., Visscher, P. M. and Price, A. L. (2014), Advantages and pitfalls in the application of mixed-model association methods, Nature Genetics 46, 100–106.

    Article  Google Scholar 

  • Zaitlen, N., Kraft, P., Patterson, N., Pasaniuc, B., Bhatia, G. Pollack, S., and Price, A. L. (2013), Using extended genealogy to estimate components of heritability for 23 quantitative and dichotomous traits, PLoS Genetics 9, e1003520.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, University of California, Davis, CA, USA

    Jiming Jiang

  2. School of Public Health, Oregon Health & Science University, Portland, OR, USA

    Thuan Nguyen

Authors
  1. Jiming Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thuan Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Science+Business Media, LLC, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jiang, J., Nguyen, T. (2021). Linear Mixed Models: Part I. In: Linear and Generalized Linear Mixed Models and Their Applications. Springer Series in Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-0716-1282-8_1

Download citation

Publish with us

Policies and ethics

Search

Navigation