The minimum coverage probability of confidence intervals in regression after a preliminary F test
Introduction
Consider the linear regression model , where is a random n-vector of responses, is a known matrix with linearly independent columns, is an unknown parameter p-vector and where is an unknown positive parameter. Suppose that the parameter of interest is where is a given p-vector (). We seek a confidence interval for .
Let the s-dimensional parameter vector be defined to be where is a specified matrix () with linearly independent columns and is a specified s-vector. Suppose that does not belong to the linear subspace spanned by the columns of . Also suppose that we carry out a preliminary F test of the null hypothesis against the alternative hypothesis . It is then common statistical practice to construct a confidence interval for with nominal coverage , using the same data, based on the assumption that the selected model had been given to us a priori (as the true model). We call this the naive confidence interval for . In Section 2, we provide a convenient description of this confidence interval. This assumption is false and it can lead to the naive confidence interval having minimum coverage probability far below , making it completely inadequate. Our aim is to compute this minimum coverage probability. For s=1, the preliminary F test is equivalent to a t test. The case of a single preliminary t test has been dealt with by Kabaila and Giri (2009b, Theorem 3). So, in the present paper, we restrict attention to the case that .
Straightforward application of the methodology of Farchione (2009, Ph.D. thesis, Section 5.7), leads to an expression for the coverage probability of the naive confidence interval, for a given value of an s-dimensional parameter vector, that is a multiple integral of dimension . Finding the minimum coverage probability using this formula becomes increasingly cumbersome as s increases due to both the need to (a) evaluate multiple integrals of dimension and (b) the need to search for the minimum over a space of dimension s.
In Section 3, by a careful consideration of the geometry of the situation, we derive a new elegant and computationally convenient formula for the coverage probability of this confidence interval for given parameter values. For s=2 this formula is a sum of a triple and a double integral and for all this formula is a sum of a quadruple and a double integral. This formula also shows that the coverage probability is a function of a two-dimensional parameter vector, irrespective of how large s is. This makes it easy to compute the minimum coverage probability of the naive confidence interval, irrespective of how large s is. Another important aspect of this formula is that it can be used to delineate general categories of , and for which the naive confidence interval has poor coverage properties.
A very important practical application of this formula is to the analysis of covariance. In this context, can be defined so that H0 expresses the null hypothesis of “parallelism”. In the applied statistics literature on the analysis of covariance it is commonly recommended that a preliminary F test of the null hypothesis of “parallelism” be carried out. See, for example, Kuehl (2002, p. 563), Milliken and Johnson (2002, pp. 14–17) and Freund et al. (2006, pp. 363–368). For an analysis of covariance, we can choose so that the parameter is the difference in expected responses for two specified treatments, for the same specified values of the covariates.
In Section 4, we illustrate the application of the results of the paper with a real-life analysis of covariance data set and a preliminary F test for “parallelism”. We define to be (expected response to treatment 1)−(expected response to treatment 2), evaluated at the same specified value of the covariate. We show that the naive 0.95 confidence interval for has minimum coverage probability 0.0846, for this specified value of the covariate. This shows that this confidence interval is completely inadequate, for this specified value of the covariate.
Section snippets
Description of the naive confidence interval
In this section we provide a convenient description of the naive confidence interval constructed after the preliminary F test. Let denote the least squares estimator of . Define . Let . Define . Also, define and . We suppose that the columns of the matrix are linearly independent. We also suppose that does not belong to the linear subspace spanned by the columns of . Now define the matrix:
The coverage probability of the naive confidence interval
Define and . Let fW denote the probability density function of W. Define . ThusSince , . The assumption that the vector does not belong to the linear subspace spanned by the columns of implies that . So, we may assume that . Now define where , and
Application to a real-life data set
In this section we consider the real-life analysis of covariance data set due to Chin et al. (1994) and analysed by Yandell (1997, Chapter 17), who makes this data available at the website http://www.stat.wisc.edu/∼yandell/pda/. This data is listed in Table 1. It consists of the observed response (weight gain) for a given treatment and value of the covariate (feed intake). There are four possible treatments, numbered 1–4.
We use the following linear regression model for this data:
Discussion
Discussion 5.1 The poor coverage properties of naive confidence intervals found in this paper are presaged by the poor coverage properties of naive confidence intervals found in the context of a preliminary best subset variable selection by minimizing an AIC-type criterion, see e.g. Kabaila (2005), Kabaila and Leeb (2006) and Kabaila and Giri, 2009a, Kabaila and Giri, 2009b (cf Kabaila, 2009). Apart from the form of preliminary model selection used, minimum AIC versus an F test, these papers differ from the
Acknowledgments
The authors are grateful to an anonymous reviewer for some comments and suggestions that helped to improve the paper.
References (15)
- et al.
Conjugate linoleic acid is a growth factor for rats as shown by enhanced weight gain and improved feed efficiency
Journal of Nutrition
(1994) - et al.
Confidence intervals in regression utilizing prior information
Journal of Statistical Planning and Inference
(2009) - et al.
Number-Theoretic Methods in Statistics
(1994) - Farchione, D., 2009. Interval Estimators that Utilize Uncertain Prior Information. Unpublished Ph.D. Thesis, Department...
- et al.
Regression Analysis: Statistical Modeling of a Response Variable
(2006) Theory and Application of the Linear Model
(1976)On the coverage probability of confidence intervals in regression after variable selection
Australian & New Zealand Journal of Statistics
(2005)