On decompositions of estimators under a general linear model with partial parameter restrictions

Abstract A general linear model can be given in certain multiple partitioned forms, and there exist submodels associated with the given full model. In this situation, we can make statistical inferences from the full model and submodels, respectively. It has been realized that there do exist links between inference results obtained from the full model and its submodels, and thus it would be of interest to establish certain links among estimators of parameter spaces under these models. In this approach the methodology of additive matrix decompositions plays an important role to obtain satisfactory conclusions. In this paper, we consider the problem of establishing additive decompositions of estimators in the context of a general linear model with partial parameter restrictions. We will demonstrate how to decompose best linear unbiased estimators (BLUEs) under the constrained general linear model (CGLM) as the sums of estimators under submodels with parameter restrictions by using a variety of effective tools in matrix analysis. The derivation of our main results is based on heavy algebraic operations of the given matrices and their generalized inverses in the CGLM, while the whole contributions illustrate various skillful uses of state-of-the-art matrix analysis techniques in the statistical inference of linear regression models.


Introduction
Consider a partitioned linear model with partial parameter restrictions M W ( y D XˇC " " " D X 1ˇ1 C C X kˇk C " " "; A 1ˇ1 D b 1 ; : : : ; A kˇk D b k ; E." " "/ D 0; D." " "/ D 2 † † †; where y is an n 1 vector of observable response variables, X D OE X 1 ; : : : ; X k is an n p matrix of arbitrary rank, X 1 ; : : : ; X k are k known n p 1 ; : : : ; n p k matrices with p D p 1 C C p k , A 1 ; : : : ; A k are given m 1 p 1 ; : : : ; m k p k matrices, respectively, with m D m 1 C C m k , b 1 ; : : : ; b k are m 1 1; : : : ; m k 1 known vectors, respectively. The system of linear equations in M is often available as extraneous information for the unknown parameter vectoř to satisfy which is an integral part of the constrained general linear model (CGLM) about the parameter space, and thus should ideally be utilized in any estimation procedure of the parameter space in (1). Associated with M are the following k submodels M i W y D X iˇi C " " " i ; A iˇi D b i ; E." " " i / D 0; D." " " i / D 2 † † †; i D 1; : : : ; k: ( Obviously, these models can be considered as reduced versions of M by deleting k 1 regressors except X iˇi , i D 1; : : : ; k. It has been realized that estimators of the unknown parameters in M and M i have some intrinsic connections, and people are interested in establishing certain additive decomposition of estimators under the partitioned model and its submodels. For convenience of representation, denote to rewrite linear models as certain partitioned forms, and then to make estimation and statistical inference under the partitioned linear models. One of the main objectives in the statistical inference of linear models is to establish various estimators of the parameter spaces in the models and to characterize mathematical and statistical properties and features of these estimators under various model assumptions. In this approach statisticians are often interested in the connections of different estimators and especially in establishing possible equalities between estimators. There have been various attempts to establish additive decomposition equalities for estimators under linear models. Under the assumptions in (9) and (10), it is natural to consider relations among the best linear unbiased estimators (BLUEs) of b Xˇin (9) and b X iˇi in (9) and (10). In this paper, we first verify or prove that under the assumptions that X 1ˇ1 ; : : : ; X kˇk , b X 1ˇ1 ; : : : ; b X kˇk are estimable in (9), the BLUE of Xˇin b M admits the following two additive decomposition identities In view of the above observations, we propose the following two additive decomposition equalities for the BLUEs of Xˇand b Xˇin b M: and then derive identifying conditions for the equalities to hold, respectively. These estimator decomposition identities have many different statistical interpretations and are not rare to see in statistical analysis of CGLMs. The problem on additive decompositions of BLUEs under general liner models was approached in [4,5]. Zhang and Tian [6] recently investigated the above two decomposition identities for k D 2 by using some effective algebraic methods of dealing with additive decompositions of matrix expressions and ranks/ranges of matrices. Before proceeding, we introduce the notation to the reader and explain its usage in this paper. R m n stands for the collection of all m n real matrices. The symbols A > , r.A/, and R.A/ stand for the transpose, the rank, and the range (column space) of a matrix A 2 R m n , respectively; I m denotes the identity matrix of order m. The Moore-Penrose inverse of A, denoted by A C , is defined to be the unique solution G satisfying the four matrix equations AGA D A, GAG D G, .AG/ > D AG, and .GA/ > D GA. Further, let P A , E A , and F A stand for the three orthogonal projectors (symmetric idempotent matrices) P A D AA C , E A D A ? D I m AA C , and F A D I n A C A. Two symmetric matrices A and B of the same size are said to satisfy the inequality A < B in the Löwner partial ordering if A B is nonnegative definite. Further information about the orthogonal projectors P A , E A , and F A with their applications in the linear statistical models can be found in [7][8][9]. Also, it is well known that the Löwner partial ordering is a surprisingly strong and useful property between two symmetric matrices. For more results about the Löwner partial ordering of symmetric matrices and applications in statistical analysis see, e.g., [8]. Generalized inverses of matrices are common tools to deal with singular matrices, which now are a fruitful and core part in current matrix theory and have profound impact in the field of statistics.

Some preliminaries in linear algebra
Statistical inference for linear models, as is well known, is entirely based on computations with the given vectors and matrices in the models, and formulas and algebraic tricks for handling matrices in linear algebra and matrix theory play an important role in the derivations of these estimators and the characterization of their performance. Because BLUEs of parameter spaces in linear models are calculated from given matrices and vectors in the models and are often represented by certain formulas composed by given matrices and vectors in linear models, the approach we take to the above problems is in fact to establish and characterize matrix equalities composed by matrices and their generalized inverses, and thus we need to use many influential and effective mathematical tools in order to characterize the above equalities of estimators and their covariance matrices under CGLMs. Many mathematical methods in statistical science require algebraical computations with vectors and matrices. In particular, formulas and algebraic techniques for handling matrices in linear algebra and matrix theory play important roles in the derivations and characterizations of estimators and their performances under linear models. As remarked in [10], a good starting point for the entry of matrices into statistics was in 1930s, while it is now a routine procedure to use given vectors, matrices and their generalized inverses in statistical models to formulate various estimators of parameter spaces in linear models and to make the corresponding statistical inferences.
As the study of additive decompositions of estimators in the contexts of linear regression models requires more effective mathematical analysis tools, it is forced toward algebraic questions that overlap with precise description and characterization of matrix decomposition identities in linear algebra. The scope of this section is to introduce various formulas for ranks of matrices in linear algebra suitable for establishing and characterizing various possible equalities for estimators under CGLMs. In this section, we first introduce some fundamental formulas for calculating ranks of matrices that will be used in the statistical analysis. Recall that the rank of matrix is conceptual foundation in matrix theory and is the most significant finite nonnegative integer in reflecting intrinsic properties of matrices, while the mathematical prerequisites for understanding the rank of matrix are minimal and do not go beyond elementary linear algebra. The intriguing connections between generalized inverses of matrices and rank formulas of matrices were recognized in 1970s, and a seminal work on establishing formulas for calculating matrices and their generalized inverses was presented in [11]. It has been known that matrix rank formulas are direct and effective tools of simplifying matrix expressions and equalities. The whole work in this paper is based on the effective use of the matrix rank methodology (MRM), which is a set of quantitative description techniques that encompass: I. establishing non-trivial analytical formulas for calculating the maximum and minimum ranks of a matrix expression, and using the ranks to determine the singularity and nonsingularity of the matrix expression, the rank invariance of the matrix expression, the dimension of the row/column space of the matrix expression; II. establishing formulas for calculating the rank of the difference of two matrix expressions, and using them to derive necessary and sufficient conditions for the two matrix expressions to be equal, i.e., proving matrix equality by matrix rank formulas; III. characterizing relations between two linear subspaces, or two matrix sets by matrix rank formulas.
The above assertions show that there are important and peculiar consequences of establishing various formulas for calculating ranks of matrices from theoretical point of view. Thus, the MRM in fact provides us with a specified algebraic framework for tackling matrix expressions and matrix equalities, and gives a glimpse into a very broad and interesting field of matrix mathematics. But it was not until a few decades ago that the MRM was essentially recognized as an effective and influential tool in the field of mathematics and was extensively applied in matrix theory and applications. Because matrices are common objects in linear regression analysis, the advent of the MRM has greatly extended from the domain of matrix theory into statistical areas, some seminal work on the fundamental theory of the MRM and its applications in statistics can be found in e.g. in [11][12][13]. Some recent work on the MRM in the analysis of additive decompositions of BLUEs under linear models were presented in [4][5][6], while some contributions on MRM in the statistical analysis of CGLMs can be found in [14][15][16][17][18][19][20][21][22][23][24].
In order to establish and characterize various possible equalities for estimators in the context of linear models and to simplify various matrix equalities composed by Moore-Penrose inverses of matrices, we will need the following well-known rank formulas involving Moore-Penrose inverses to make the paper self-contained.
Furthermore; the following results hold: With the support of the formulas in Lemmas 2.1-2.3, we are able to covert the problems in (11)- (14) into certain algebraic problems characterizing matrix equalities composed by the given matrices in the models and their generalized inverses, and to derive analytical solutions of the problems by using the methods of matrix equations, matrix rank formulas, and various skillful partitioned matrix calculations.

Estimability of parameter spaces under CGLMs
We take 2 D 1 in (1)-(10) for the convenience of presentation below, because it doesn't play any role in the main results in this paper. In what follows, we assume that the model in (9) is consistent, i.e., see [27,28]. We next introduce the definitions of the estimability of parameter spaces in CGLMs.
M be as given in (9) and let K 2 R k p be given. Then; the vector Kˇof the unknown parameters It is well known in statistical theory that the unbiasedness of linear statistics with respect to given parameter spaces in linear models is an important property. Considerable literature exists on estimability of parameter spaces in linear models; see e.g. [29][30][31][32][33][34][35][36][37][38] for some excellent expositions. We next present some classic and new results on the estimability of the parameter space in (9) and give their proofs. 29]). Let b M be as given in (9) and let K 2 R k p be given: Then; the following results hold: M; i D 1; : : : ; k: M be as given in (9): Then; the following statements are equivalentW (a) All X 1ˇ1 ; : : : ; X kˇk are estimable under b M: Proof. It is obvious from (7) that Hence if (c) holds, we obtain from (22) that which means that (a) and (b) hold by Lemma 3.2. The equivalence of (c) and (23) can be proved by induction, we leave it to the reader.

BLUEs' computations
Theoretical and applied researches of a CGLM seek to develop various possible estimators of the parameter space in the CGLM. When there exist unbiased estimators for a given parameter space, there are usually many unbiased estimators for the parameter space. Thus, it is natural to seek such an unbiased estimator that has the smallest dispersion matrix among all the unbiased estimators, that is to say, the unbiasedness and smallest dispersion matrices of estimators are most intrinsic requirements in statistical analysis and inference. The concepts of BLUEs of parameter spaces in the contexts of (1)-(10) are given below.
Definition 4.1. Let b M be as given in (9); and assume that Kˇis estimable under b M for K 2 R k p . If there exists an L 2 R k .mCn/ such that E.Lb y Kˇ/ D 0 and D.Lb y Kˇ/ D min (24) hold in the Löwner partial ordering, the linear statistic Lb y is defined to be the BLUE of Kˇunder b M, and is denoted by Estimators of the parameter spaces in linear models are usually formulated from mathematical operations of the observed response vectors, the given model matrices, and the covariance matrices of the error terms in the models. Hence, the standard inference theory of linear statistical models can be established from the exact algebraic expressions of estimators, which is easily acceptable from both mathematical and statistical points of view. In fact, linear statistical models are the only type of statistical models that have complete and solid support from linear algebra and matrix theory. Observing that (9) is a special case of GLMs, the following lemma follows from the well-known results on the BLUEs under linear models; see e.g. [28, p. 282] and [39, p. 55].
Lemma 4.2. Let b M be as given in (9); assume that Kˇis estimable under b M for K 2 R k p ; and denote t D n C m: Then; the following results hold: (a) The following implication M can be written as Cov Cov where V i 2 R t t is arbitrary; i; j D 1; : : : ; k: (c) The following two decomposition identities hold Proof. Results (a) and (b) follow directly from (8) and (28) by letting K D Y i ; b Y i ; respectively. Result (c) follows directly from (7), (29), and (30).
In what follows, we use fBLUE b M .Kˇ/g to denote the collection of all BLUE b M .Kˇ/ in (28).

Additive decompositions of BLUEs under a full CGLM and its submodels
For convenience of representation, we adopt the notation in this section. 0 X 2 X k X 1 0 X k : : : : : : : : : : : : 5; i ¤ j; i; j D 1; : : : ; k: The misspecified BLUEs under the submodels in (10) are given below. where H i 2 R n t and G i 2 R t t are arbitrary matrices; i D 1; : : : ; k: It should be pointed out that under the assumptions in (9), the k submodels in (10) are misspecified versions of (9). So that the estimators in (44) and (45) are not true BLUEs of X iˇi and b X iˇi under the models in (10), that is to say, they neither are unbiased for X iˇi and b X iˇi under (9), nor have the smallest covariance matrices in the Löwner sense. In such a case, the sums of the BLUEs may, however, be the BLUEs of Xˇand b Xˇunder some conditions. In this section, we derive some algebraical and statistical properties and features of the BLUEs under (9) and (10), and then give necessary and sufficient conditions for the equalities in (13) and (14) to hold. Although the results in the last section present exact formulas of BLUEs under various assumptions, we have to pay more attention to the mathematical manipulations hidden behind the BLUE formulas in order to establish the connections among the BLUEs. During this process, many skillful calculations of matrix ranks and elementary block matrix operations will be conducted in establishing and simplifying matrix equalities and expressions.

Concerning the relations between BLUE
. b X iˇi /, i D 1; : : : ; k; we have the following conclusions.
It can be seen from (47) and (49) that neither the sum BLUE b . b X kˇk / is necessarily unbiased for b Xˇunder (1). Concerning the unbiasedness of the two sums and the corresponding BLUE decompositions, we have the following general conclusions. . b X iˇi /; i D 1; : : : ; k; such that It should be pointed out that many exclusive and tricky methods for establishing and simplifying matrix expressions and matrix equalities have been developed in linear algebra and matrix theory, which have greatly benefited both mathematics and applications. In particular, these new methodologies have also found essential applications in statistical analysis, such as establishing various intriguing and sophisticated formulas, equalities, and inequalities associated with estimators under linear statistical models.