Abstract
A general result concerning the strong universal consistency of local averaging regression estimates is presented, which is used to extend previously known results on the strong universal consistency of kernel and partitioning regression estimates. The proof is based on ideas from Etemadi’s proof of the strong law of large numbers, which shows that these ideas are also useful in the context of strong laws of large numbers for conditional expectations in \(L_2\).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
29 August 2018
There is a gap at the end of the proof of Theorem 1, since there the application of the conditional McDiarmid inequality yields.
References
Biau, G., Devroye, L. (2015). Lectures on the nearest neighbor method. Heidelberg: Springer.
Devroye, L. (1981). On the almost everywhere convergence of nonparametric regression function estimates. Annals of Statistics, 9, 1310–1319.
Devroye, L., Györfi, L., Krzyżak, A., Lugosi, G. (1994). On the strong universal consistency of nearest neighbor regression function estimates. Annals of Statistics, 22, 1371–1385.
Devroye, L., Krzyżak, A. (1989). An equivalence theorem for \(L_1\) convergence of the kernel regression estimate. Journal of Statistical Planning and Inference, 23, 71–82.
Devroye, L. P., Wagner, T. J. (1980). Distribution-free consistency results in nonparametric discrimination and regression function estimation. Annals of Statistics, 8, 231–239.
Epanechnikov, V. A. (1969). Non-parametric estimation of a multivariate probability density. Theory of Probability and Its Applications, 14, 153–158.
Etemadi, N. (1981). An elementary proof of the strong law of large numbers. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 55, 119–122.
Greblicki, W., Krzyżak, A., Pawlak, M. (1984). Distribution-free pointwise consistency of kernel regression estimate. Annals of Statistics, 12, 1570–1575.
Györfi, L. (1991). Universal consistencies of a regression estimate for unbounded regression functions. In G. Roussas (Ed.), Nonparametric functional estimation and related topics (pp. 329–338)., NATO ASI series Dordrecht: Kluwer.
Györfi, L., Walk, H. (1996). On the strong universal consistency of a series type regression estimate. Mathematical Methods of Statistics, 5, 332–342.
Györfi, L., Walk, H. (1997). On the strong universal consistency of a recursive regression estimate by Pál Révész. Statistics and Probability Letters, 31, 177–183.
Györfi, L., Kohler, M., Krzyżak, A., Walk, H. (2002). A distribution-free theory of nonparametric regression. New York: Springer.
Györfi, L., Kohler, M., Walk, H. (1998). Weak and strong universal consistency of semi-recursive partitioning and kernel regression estimates. Statistics & Decisions, 16, 1–18.
Irle, A. (1997). On consistency in nonparametric estimation under mixing conditions. Journal of Multivariate Analysis, 60, 123–147.
Kohler, M. (1997). On the universal consistency of a least squares spline regression estimator. Mathematical Methods of Statistics, 6, 349–364.
Kohler, M. (1999). Universally consistent regression function estimation using hierarchical B-splines. Journal of Multivariate Analysis, 67, 138–164.
Kohler, M. (2003). Universal consistency of local polynomial kernel regression estimates. Annals of the Institute of Statistical Mathematics, 54, 879–899.
Kohler, M., Krzyżak, A. (2001). Nonparametric regression estimation using penalized least squares. IEEE Transactions on Information Theory, 47, 3054–3058.
Kohler, M., Krzyżak, A., Walk, H. (2003). Strong consistency of automatic kernel regression estimates. Annals of the Institute of Statistical Mathematics, 55, 287–308.
Kozek, A. S., Leslie, J. R., Schuster, E. F. (1998). On a universal strong law of large numbers for conditional expectations. Bernoulli, 4, 143–165.
Lugosi, G., Zeger, K. (1995). Nonparametric estimation via empirical risk minimization. IEEE Transactions on Information Theory, 41, 677–687.
McDiarmid, C. (1989). On the method of bounded differences. In J. Siemons (Ed.), Surveys in combinatorics (pp. 148–188). Cambridge: Cambridge University Press.
Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Applications, 9, 141–142.
Spiegelman, C., Sacks, J. (1980). Consistent window estimation in nonparametric regression. Annals of Statistics, 8, 240–246.
Stone, C. J. (1977). Consistent nonparametric regression. Annals of Statistics, 5, 595–645.
Tukey, J. W. (1947). Nonparametric estimation II. Statistically equivalent blocks and tolerance regions—The continuous case. Annals of Mathematical Statistics, 18, 529–539.
Walk, H. (2001). Strong universal pointwise consistency of recursive regression estimates. Annals of the Institute of Statistical Mathematics, 53, 691–707.
Walk, H. (2002). Almost sure convergence properties of Nadaraya–Watson regression estimates. In M. Dror, P. L’Ecuyer, F. Szidarovszky (Eds.), Modeling uncertainty: An examination of stochastic theory, methods, and applications (S. Yakowitz memorial volume) (pp. 201–223). Dordrecht: Kluwer.
Walk, H. (2005). Strong universal consistency of smooth kernel regression estimation. Annals of the Institute of Statistical Mathematics, 57, 665–685.
Walk, H. (2008). A universal strong law of large numbers for conditional expectations via nearest neighbors. Journal of Multivariate Analysis, 99, 1035–1050.
Watson, G. S. (1964). Smooth regression analysis. Sankhya Series A, 26, 359–372.
Acknowledgements
The authors thank Editors and Reviewers for their comments, which led to a generalization of a result and to an improvement of the presentation.
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Hansmann, M., Kohler, M. & Walk, H. On the strong universal consistency of local averaging regression estimates. Ann Inst Stat Math 71, 1233–1263 (2019). https://doi.org/10.1007/s10463-018-0674-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-018-0674-9