Skip to main content
SpringerLink
Log in

Shrinkage estimation in linear mixed models for longitudinal data

  • Published:
Metrika Aims and scope Submit manuscript

Abstract

This paper is concerned with the selection and estimation of fixed effects in linear mixed models while the random effects are treated as nuisance parameters. We propose the non-penalty James–Stein shrinkage and pretest estimation methods based on linear mixed models for longitudinal data when some of the fixed effect parameters are under a linear restriction. We establish the asymptotic distributional biases and risks of the proposed estimators, and investigate their relative performance with respect to the unrestricted maximum likelihood estimator (UE). Furthermore, we investigate the penalty (LASSO and adaptive LASSO) estimation methods and compare their relative performance with the non-penalty pretest and shrinkage estimators. A simulation study for various combinations of the inactive covariates shows that the shrinkage estimators perform better than the penalty estimators in certain parts of the parameter space. This particularly happens when there are many inactive covariates in the model. It also shows that the pretest, shrinkage, and penalty estimators all outperform the UE. We further illustrate the proposed procedures via a real data example.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Ahmed SE, Rohatgi VJ (1996) Shrinkage estimation in a randomized response model. Metrika 43:17–30

    Article  MathSciNet  MATH  Google Scholar 

  • Ahmed SE, Hussein A, Sen PK (2006) Risk comparison of some shrinkage M-estimators in linear models. J Nonparametr Stat 18(3):401–415

    Article  MathSciNet  MATH  Google Scholar 

  • Ahmed SE, Hossain S, Doksum KA (2012) LASSO and shrinkage estimation in Weibull censored regression models. J Stat Plan Inference 142(6):1273–1284

    Article  MathSciNet  MATH  Google Scholar 

  • Ahn M, Zhang HH, Lu W (2012) Moment-based method for random effects selection in linear mixed models. Stat Sin 22(4):1539–1562

    MathSciNet  MATH  Google Scholar 

  • Bondell HD, Krishna A, Ghosh SK (2010) Joint variable selection for fixed and random effects in linear mixed-effects models. Biometrics 66(4):1069–1077

    Article  MathSciNet  MATH  Google Scholar 

  • Datta G, Ghosh M (2012) Small area shrinkage estimation. Stat Sci 27(1):95–114

    Article  MathSciNet  MATH  Google Scholar 

  • Diggle PJ, Heagerty P, Liang KY, Zeger SL (2002) Analysis of longitudinal data. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96(456):1348–1360

    Article  MathSciNet  MATH  Google Scholar 

  • Fan YY, Li RZ (2012) Variable selection in linear mixed effects models. Ann Stat 40(4):2043–2068

    Article  MathSciNet  MATH  Google Scholar 

  • Fitzmaurice GM, Laird NM, Ware JH (2011) Longitudinal data analysis. Wiley, New York

    MATH  Google Scholar 

  • Gao X, Ahmed SE, Fen Y (2017) Post selection shrinkage estimation for high-dimensional data analysis. Appl Stoch Models Bus Ind 33:97–120

    MathSciNet  Google Scholar 

  • Gumedze FN, Dunne TT (2011) Parameter estimation and inference in the linear mixed model. Linear Algebra Appl 435(8):1920–1944

    Article  MathSciNet  MATH  Google Scholar 

  • Gupta A, Saleh AKME, Sen PK (1989) Improved estimation in a contingency table: independence structure. J Am Stat Assoc 84(406):525–532

    Article  MathSciNet  MATH  Google Scholar 

  • Harville D (1977) Maximum likelihood approaches to variance component estimation and to related problems. J Am Stat Assoc 358:320–340

    Article  MathSciNet  MATH  Google Scholar 

  • Hedeker D, Gibbons RD (2006) Longitudinal data analysis. Wiley, New York

    MATH  Google Scholar 

  • Hossain S, Doksum KA, Ahmed SE (2009) Positive-part shrinkage and absolute penalty estimators in partially linear models. Linear Algebra Appl 430:2749–2761

    Article  MathSciNet  MATH  Google Scholar 

  • Hossain S, Ahmed SE, Doksum KA (2015) Shrinkage, \(l_1\) penalty and pretest estimators for generalized linear models. Stat Methodol 24:52–68

    Article  MathSciNet  Google Scholar 

  • Hossain S, Ahmed SE, Yi Y, Chen B (2016) Shrinkage and pretest estimators for longitudinal data analysis under partially linear models. J Nonparametr Stat 28(3):531–549

    Article  MathSciNet  MATH  Google Scholar 

  • Ibrahim JG, Zhu HT, Garcia R, Guo RX (2011) Fixed and random effects selection in mixed effects models. Biometrics 67(2):495–503

    Article  MathSciNet  MATH  Google Scholar 

  • Judge GG, Bock ME (1978) The statistical implication of pretest and Stein-rule estimators in econometrics. North-Holland, Amsterdam

    MATH  Google Scholar 

  • Judge GG, Mittelhammaer RC (2004) A semiparametric basis for combining estimation problem under quadratic loss. J Am Stat Assoc 99(466):479–487

    Article  MathSciNet  MATH  Google Scholar 

  • Kemper HCG (1995) The Amsterdam Growth Study: a longitudinal analysis of health, fitness and lifestyle. HK Sport Science Monograph Series, vol 6. Human Kinetics Publishers, Champaign

  • Laird NM, Ware JH (1982) Random-effects models for longitudinal data. Biometrics 38(4):963–974

    Article  MATH  Google Scholar 

  • Lindstrom MJ, Bates DM (1988) Newton–Raphson and EM algorithms for linear mixed-effects models for repeated measures data. J Am Stat Assoc 83(404):1014–1022

    MathSciNet  MATH  Google Scholar 

  • Longford NT (1993) Random coefficient models. Oxford University Press, Oxford

    MATH  Google Scholar 

  • McCulloch CE, Searle SR, Neuhaus JR (2008) Generalized, linear, and mixed models, 2nd edn. Wiley, Hoboken, New Jersey

    MATH  Google Scholar 

  • Müller S, Scealy JL, Welsh AH (2013) Model selection in linear mixed models. Stat Sci 28(2):135–167

    Article  MathSciNet  MATH  Google Scholar 

  • Pan J, Shang J (2017) Adaptive LASSO for linear mixed model selection via profile log-likelihood. Commun Stat Theory Methods. https://doi.org/10.1080/03610926.2017.1332219

    MATH  Google Scholar 

  • Peng H, Lu Y (2012) Model selection in linear mixed effect models. J Multivar Anal 109:109–129

    Article  MathSciNet  MATH  Google Scholar 

  • Schelldorfer J, Bühlmann P, van de Geer S (2011) Estimation for high-dimensional linear mixed effects models using \(l_1\)-penalization. Scand J Stat 38(2):197–214

    Article  MathSciNet  MATH  Google Scholar 

  • Searle SR, Casella G, McCulloch CE (2006) Variance components. Wiley, New York

    MATH  Google Scholar 

  • Thomson T, Hossain S, Ghahramani M (2015) Application of shrinkage estimation in linear regression models with autoregressive errors. J Stat Comput Simul 85(16):1580–1592

    Article  MathSciNet  Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58(1):267–288

    MathSciNet  MATH  Google Scholar 

  • Twisk JWR (2003) Applied longitudinal data analysis for epidemiology—a practical guide. Cambridge University Press, New York

    Google Scholar 

  • Verbeke G, Molenberghs G (2009) Linear mixed models for longitudinal data. Springer Series in Statistics: corrected edition, New York

  • Zeng T, Hill RC (2016) Shrinkage estimation in the random parameters logit model. Open J Stat 6:667–674

    Article  Google Scholar 

  • Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101(476):1418–1429

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to thank the referees, editor and associate editor for their valuable suggestions in the revision of this paper. The research of Shakhawat Hossain and Ejaz Ahmed was supported by the Natural Sciences and the Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, University of Winnipeg, Winnipeg, Canada

    Shakhawat Hossain & Trevor Thomson

  2. Department of Mathematics and Statistics, Brock University, St. Catharines, ON, Canada

    Ejaz Ahmed

Authors
  1. Shakhawat Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trevor Thomson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ejaz Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shakhawat Hossain.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hossain, S., Thomson, T. & Ahmed, E. Shrinkage estimation in linear mixed models for longitudinal data. Metrika 81, 569–586 (2018). https://doi.org/10.1007/s00184-018-0656-1

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00184-018-0656-1

Keywords

Search

Navigation