Abstract
The normalized maximum likelihood (NML) formulation of the stochastic complexity Rissanen ([10]) contains two components: the maximized log likelihood and a component that may be interpreted as the parametric complexity of the model. The stochastic complexity for the data, relative to a suggested model, serves as a criterion for model selection. The calculation of the stochastic complexity can be considered as an implementation of the minimum description length principle (MDL) (cf. Rissanen [12]). To obtain an NML based model selection criterion for the Gaussian linear regression, Rissanen [11] constrains the data space appropriately. In this paper we demonstrate the effect of the data constraints on the selection criterion. In fact, we obtain various forms of the criterion by reformulating the shape of the data constraints. A special emphasis is placed on the performance of the criterion when collinearity is present in data.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Akaike, H.: Information theory as an extension of the maximum likelihood principle. In: Petrov, B.N., Csaki, F. (eds.) Second International Symposium on Information Theory, pp. 267–281. Akademiai Kiado, Budapest (1973)
Belsley, H.: Conditioning Diagnostics. Wiley, New York (1991)
Burnham, K.P., Anderson, D.R.: Model Selection and Multi-model Inference. Springer, New York (2002)
Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)
Draper, N.R., Smith, H.: Applied Regression Analysis, 2nd edn. Wiley, New York (1981)
Grünwald, P.D.: The Minimum Description Length Principle. MIT, London (2007)
Hansen, A.J., Yu, B.: Model Selection and the Principle of Minimum Description Length. J. Am. Stat. Assoc. 96, 746–774 (2001)
Liski, E.P.: Normalized ML and the MDL Principle for Variable Selection in Linear Regression. In: Festschrift for Tarmo Pukkila on his 60th Birthday, pp. 159–172, Tampere, Finland (2006)
Miller, A.: Subset Selection in Regression, 2nd edn. Chapman & Hall/CRC, New York (2002)
Rissanen, J.: Fisher Information and Stochastic Complexity. IEEE Trans. Inf. Theory, IT-42, 1, 40–47 (1996)
Rissanen, J.: MDL Denoising. IEEE Trans. Inf. Theory, IT-46, 1, 2537–2543 (2000)
Rissanen, J.: Information and Complexity and in Statistical Modeling. Springer, New York (2007)
Schwarz, G.: Estimating the Dimension of a Model. Ann. Stat. 6, 461–464 (1978)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Physica-Verlag Heidelberg
About this chapter
Cite this chapter
Liski, E.P., Liski, A. (2009). Minimum Description Length Model Selection in Gaussian Regression under Data Constraints. In: Schipp, B., Kräer, W. (eds) Statistical Inference, Econometric Analysis and Matrix Algebra. Physica-Verlag HD. https://doi.org/10.1007/978-3-7908-2121-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-7908-2121-5_14
Publisher Name: Physica-Verlag HD
Print ISBN: 978-3-7908-2120-8
Online ISBN: 978-3-7908-2121-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)