Shai Ben-David
ABSTRACT
We investigate the PAC learnability of classes of {0,…,n}-valued functions. For n = 1, it is known that the finiteness of the Vapnik-Chervonenkis dimension is necessary and sufficient for learning. In this paper we present a general scheme for extending the VC-dimension to the case n > 1. Our scheme defines a wide variety of notions of dimension in which several variants of the VC-dimension, previously introduced in the context of learning, appear as special cases. Our main result is a simple condition characterizing the set of notions of dimension whose finiteness is necessary and sufficient for learning. This provides a variety of new tools for determining the learnability of a class of multi-valued functions. Our characterization is also shown to hold in the “robust” variant of PAC model.
- Alo83.N. Alon. On the density of sets of vectors. Discrete Mathematics, 24:177-184, 1983.Google Scholar
- BEHW89.A. Blumer, A. Ehrenfeucht, D. Haussler, and M.K. Warmuth. Learnability and the Vapnik-Chervonenkis dimension. JA CM, 36(4):929-965, 1989. Google ScholarDigital Library
- Dud87.R.M. Dudley. Universal donsker classes and metric entropy. Ann. Prob., 15(4):1306- 1326, 1987.Google ScholarCross Ref
- EHKV89.A. Ehrenfeucht, D. Haussler, M. Kearns, and L.G. Valiant. A general lower bound on the number of examples needed for learning. Information and Computation, 82(3):247- 251, 1989. Google ScholarDigital Library
- Hau91.D. Haussler. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Technical Report UCSC-CRL-91-02, University of California at Santa Cruz, 1991. Google ScholarDigital Library
- HL90.D. Haussler and P.M. Long. A generalization of Sauer's lemma. Technical Report UCSC-CRL-90-15, University of California at Santa Cruz, 1990. Google ScholarDigital Library
- Nat89.B.K. Natarajan. On learning sets and functions. Machine Learning, 4:67-97, 1989. Google ScholarDigital Library
- Pol84.D. Pollard. Convergence of Stochastic Processes. Springer Verlag, 1984.Google Scholar
- Pol90.D. Pollard. Empirical Processes: Theory and Applications. Institute of Mathematical Statistics, 1990.Google Scholar
- Val84.L.G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134- 1142, 1984. Google ScholarDigital Library
- Vap89.V.N. Vapnik. Inductive principles of the search for empirical dependences (methods based on weak convergence of probability measures). The 1989 Workshop on Computational Learning Theory, 1989. Google ScholarDigital Library
- VC71.V.N. Vapnik and A.Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2):264-280, 1971.Google ScholarCross Ref
Index Terms
- Characterizations of learnability for classes of {O, …, n}-valued functions
Recommendations
Characterizations of Learnability for Classes of {0, ..., n)-Valued Functions
We investigate the PAC learnability of classes of {0, ..., n}-valued functions (n < ∞). For n = 1 it is known that the finiteness of the Vapnik-Chervonenkis dimension is necessary and sufficient for learning. For n > 1 several generalizations of the VC-...
Fat-Shattering and the Learnability of Real-Valued Functions
We consider the problem of learning real-valued functions from random examples when the function values are corrupted with noise. With mild conditions on independent observation noise, we provide characterizations of the learnability of a real-valued ...
Comments