Abstract
We consider the problem of estimating the noise level σ2 in a Gaussian linear model Y = Xβ+σξ, where ξ ∈ ℝn is a standard discrete white Gaussian noise and β ∈ ℝp an unknown nuisance vector. It is assumed that X is a known ill-conditioned n × p matrix with n ≥ p and with large dimension p. In this situation the vector β is estimated with the help of spectral regularization of the maximum likelihood estimate, and the noise level estimate is computed with the help of adaptive (i.e., data-driven) normalization of the quadratic prediction error. For this estimate, we compute its concentration rate around the pseudo-estimate ||Y − Xβ||2/n.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aster, R.C., Borchers, B., and Thurber, C.H., Parameter Estimation and Inverse Problems, Amsterdam: Elsevier/Academic, 2013, 2nd ed.
Kneip, A., Ordered Linear Smoothers, Ann. Statist., 1994, vol. 22, no. 2, pp. 835–866.
Engl, H.W., Hanke, M., and Neubauer, A., Regularization of Inverse Problems, Dordrecht: Kluwer, 1996.
Golubev, G. and Levit, B., On the Second Order Minimax Estimation of Distribution Function, Math. Methods Statist., 1996, vol. 5, no. 1, pp. 1–31.
Dalalyan, A.S., Golubev, G.K., and Tsybakov, A.B., Penalized Maximum Likelihood and Semiparametric Second-Order Efficiency, Ann. Statist., 2006, vol. 34, no. 1, pp. 169–201.
Laurent, B. and Massart, P., Adaptive Estimation of a Quadratic Functional by Model Selection, Ann. Statist., 2000, vol. 28, no. 5, pp. 1302–1338.
Kolmogoroff, A., Über das Gesetz des iterierten Logarithmus, Math. Ann., 1929, vol. 101, no. 1, pp. 126–135.
Green, P.J. and Silverman, B.W., Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach, London: Chapman & Hall, 1994.
Demmler, A. and Reinsch, C., Oscillation Matrices with Spline Smoothing, Numer. Math., 1975, vol. 24, no. 5, pp. 375–382.
Utreras, F., Cross-validation Techniques for Smoothing Spline Functions in One or Two Dimensions, Smoothing Techniques for Curves Estimation (Proc. Workshop Held in Heidelberg, Germany, April 2–4, 1979), Gasser, Th. and Rosenblatt, M., Eds., Lect. Notes Math., vol. 757, New York: Springer, 1979, pp. 196–232.
Speckman, P., Spline Smoothing and Optimal Rates of Convergence in Nonparametric Regression Models, Ann. Statist., 1985, vol. 13, no. 3, pp. 970–983.
Kolmogoroff, A., Über die beste Annäherung von Funktionen einer gegebenen Funktionenklasse, Ann. of Math. (2), 1936, vol. 37, no. 1, pp. 107–110. Russian transl. in A.N. Kolmogorov. Izbrannye trudy. Matematika i mekhanika (A.N. Kolmogorov. Selected Works. Mathematics and Mechanics), Moscow: Nauka, 1985, pp. 186–189.
Efromovich, S. and Low, M., On Optimal Adaptive Estimation of a Quadratic Functional, Ann. Statist., 1996, vol. 24, no. 3, pp. 1106–1125.
Golubev, G.K., On Risk Concentration for Convex Combinations of Linear Estimators, Probl. Peredachi Inf., 2016, vol. 52, no. 4, pp. 31–48 [Probl. Inf. Trans. (Engl. Transl.), 2016, vol. 52, no. 4, pp. 344–358].
Author information
Authors and Affiliations
Corresponding author
Additional information
Original Russian Text © G.K. Golubev, E.A. Krymova, 2018, published in Problemy Peredachi Informatsii, 2018, Vol. 54, No. 4, pp. 60–81.
Supported in part by the Russian Foundation for Basic Research, project no. 15-07-09121, and the German Research Foundation (DFG), project SFB 823: Statistical Modelling of Nonlinear Dynamic Processes.
Rights and permissions
About this article
Cite this article
Golubev, G.K., Krymova, E.A. Noise Level Estimation in High-Dimensional Linear Models. Probl Inf Transm 54, 351–371 (2018). https://doi.org/10.1134/S003294601804004X
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S003294601804004X