Attribute-Efficient Learning

Years and Authors of Summarized Original Work

• 1987; Littlestone

Problem Definition

Given here is a basic formulation using the online mistake-bound model, which was used by Littlestone [9] in his seminal work.

Fix a class C of Boolean functions over n variables. To start a learning scenario, a target function f_∗ ∈ C is chosen but not revealed to the learning algorithm. Learning then proceeds in a sequence of trials. At trial t, an input $\boldsymbol{x}_{t} \in \{ 0,1\}^{n}$ is first given to the learning algorithm. The learning algorithm then produces its prediction$\hat{y}_{t}$, which is its guess as to the unknown value f_∗(x _t ). The correct value y _t  = f_∗(x _t ) is then revealed to the learner. If $y_{t}\neq \hat{y}_{t}$, the learning algorithm made a mistake. The learning algorithm learns C with mistake-bound m, if the number of mistakes never exceeds m, no matter how many trials are made and how f_∗ and $\boldsymbol{x}_{1},\boldsymbol{x}_{2},\ldots$ are chosen.

Variable (or...