Superquantiles at Work: Machine Learning Applications and Efficient Subgradient Computation

Abstract

R. Tyrell Rockafellar and his collaborators introduced, in a series of works, new regression modeling methods based on the notion of superquantile (or conditional value-at-risk). These methods have been influential in economics, finance, management science, and operations research in general. Recently, they have been subject of a renewed interest in machine learning, to address issues of distributional robustness and fair allocation. In this paper, we review some of these new applications of the superquantile, with references to recent developments. These applications involve nonsmooth superquantile-based objective functions that admit explicit subgradient calculations. To make these superquantile-based functions amenable to the gradient-based algorithms popular in machine learning, we show how to smooth them by infimal convolution and detail numerical procedures to compute the gradients of the smooth approximations. We put the approach into perspective by comparing it to other smoothing techniques and by illustrating it on toy examples.

Appendices

Appendix A: Proof of Theorem 1

In this appendix, we provide a complete proof of Theorem 1. For classical results in this spirit, we refer to the monograph [8]. For discussions on statistical aspects of statistical learning, we refer to e.g., [11, 24, 28].

The key step in the proof of Theorem 1 is to show the uniform convergence

$$ S^{p}_{n}(w) \to S^{p}(w) \text{ almost surely for all }w \in W. $$

(38)

Indeed, once we have this, the result immediately follows as

$$ \begin{array}{@{}rcl@{}} 0 \le S^{p}(w_{n}^{\star}) - S^{p}(w^{\star}) &=& S^{p}(w_{n}^{\star}) - S^{p}_{n}(w_{n}^{\star}) + S^{p}_{n}(w_{n}^{\star}) - S^{p}_{n}(w^{\star}) + S^{p}_{n}(w^{\star}) - S^{p}(w^{\star})\\ &\le& 2 \underset{w \in W}{\sup} |S^{p}_{n}(w) - S^{p}(w)| \to 0, \end{array} $$

where we use $S^{p}_{n}(w_{n}^{\star }) \le S^{p}_{n}(w^{\star })$ in the second inequality.

In order to prove (38), we use the variational expression of the superquantile (4). We define

$$ \bar{S}^{p}(w, \eta) = \eta + \frac{1}{1-p} \mathbb{E}_{(x, y)\sim P} [\max(\ell(y, \varphi(w, x))-\eta,0)] , $$

so that, using that the loss is bounded by B, we can write

$$ S^{p}(w) = \underset{\eta \in [0, B]}{\min} \bar{S}^{p}(w, \eta). $$

We define the analogous empirical version $\bar {S}^{p}_{n}(w, \eta )$ so that $S^{p}_{n}(w) = \min \limits _{\eta \in [0, B]} \bar {S}^{p}_{n}(w, \eta )$. Note that $\bar {S}^{p}_{n}(w, \eta )$ is measurable for each fixed (w, η) and $S^{p}_{n}(w)$ is measurable for each fixed w.

Claim 1

Under Assumption 2, the random variable

$$ \delta_{n}(w, \eta) := \bar{S}^{p}_{n}(w, \eta) - \bar{S}^{p}(w, \eta) $$

has mean zero, lies almost surely in [−B, B], and satisfies

$$ |\delta_{n}(w, \eta) - \delta_{n}(w^{\prime}, \eta^{\prime})| \le 2M/(1-p) \text{dist}_{\varphi}(w, w^{\prime}) + 2(1+1/(1-p)) |\eta - \eta^{\prime}| . $$

(39)

Note first that $\mathbb {E}[\bar {S}^{p}_{n}(w, \eta )] = \bar {S}^{p}(w, \eta )$ and that the boundedness of δ_n comes from the boundedness of the loss function. The Lipschitzness of δ_n also comes from the one of the loss function, as follows. Using that $\max \limits \{\cdot , 0\}$ is 1-Lipschitz and that the loss ℓ is M-Lipschitz, we get

$$ \begin{array}{@{}rcl@{}} |\max\{\ell(y, \varphi(w, x)) - \eta, 0\} &-& \max\{\ell(y, \varphi(w^{\prime}, x)) - \eta^{\prime}, 0\}| \\ &&\le |\ell(y, \varphi(w, x)) - \ell(y, \varphi(w^{\prime}, x))| + |\eta - \eta^{\prime}| \\ &&\le M \|\varphi(w, x) - \varphi(w^{\prime}, x)\| + |\eta - \eta^{\prime}| \\ &&\le M\text{dist}_{\varphi}(w, w^{\prime}) + |\eta - \eta^{\prime}| . \end{array} $$

Then, Eq. 39 simply follows from the triangle inequality, and Claim 1 is proved.

The next step in the proof is, for a given ε > 0

• to construct a cover T of W × [0, B], and then

• to control the convergence over the points of T, more precisely to control the probability of the event

  $$ E_{n}(\varepsilon) = \underset{(w, \eta) \in T}{\bigcap} \left\{ \delta_{n}(w, \eta) \le \varepsilon / 2 \right\} . $$

First, using Assumption 1, we consider T₁ a (ε(1 − p)/(8M))-cover of W with respect to dist_φ. We also consider T₂ a uniform discretization of the line segment [0, B] at width ε(1 + 1/(1 − p))/8. We can introduce the cover of W × [0, B]

$$ T = T_{1} \times T_{2} \subset W \times [0, B]. $$

Since, |T₂| = 8B/((1 + 1/(1 − p))ε), we have that |T| = (8B/((1 + 1/(1 − p))ε))N(ε(1 − p)/(8M)). Note that the event $\left \{ \delta _{n}(w, \eta ) \le \varepsilon / 2 \right \}$ for fixed (w, η) since δ_n(w, η) is measurable, and therefore, E_n(ε) is measurable since it is a finite intersection.

To get uniform convergence, it is sufficient to control what happens at points of T. Indeed, for any (w, η), there exists a point $(w^{\prime }, \eta ^{\prime }) \in T$ such that $\text {dist}_{\varphi }(w, w^{\prime }) \le \varepsilon (1-p)/(8M)$ and $|\eta - \eta ^{\prime }| \le \varepsilon (1+1/(1-p))/ 8$. As a consequence, if the event E_n(ε) holds, then

$$ \begin{array}{@{}rcl@{}} \delta_{n}(w, \eta) &\le& \delta_{n}(w^{\prime}, \eta^{\prime}) + |\delta_{n}(w, \eta) - \delta_{n}(w^{\prime}, \eta^{\prime})| \\ &\overset{(39)}{\le}& \delta_{n}(w^{\prime}, \eta^{\prime}) + 2M/(1-p) \text{dist}_{\varphi}(w, w^{\prime}) + 2(1+1/(1-p)) | \eta - \eta^{\prime}|\\ &\le& \frac{\varepsilon}{2} + \frac{\varepsilon}{4} + \frac{\varepsilon}{4} = \varepsilon. \end{array} $$

This implies that events of interest are included in $\overline {E}_{n}(\varepsilon )$, the complement of E_n(ε); we have indeed

$$ \left\{\underset{w \in W}{\sup} |S^{p}_{n}(w) - S^{p}(w)| > \varepsilon\right\} \subset \left\{\underset{(w,\eta) \in W \times [0, B]}{\sup} \delta_{n}(w, \eta) >\varepsilon\right\} \subset \overline E_{n}(\varepsilon) . $$

Postponing the proof of measurability of these events to Claim 3 at the end of this proof, we have the following bound on the sum of probabilities

$$ \sum\limits_{n=1}^{\infty} \mathbb{P}\left( \underset{w \in W}{\sup} |S^{p}_{n}(w) - S^{p}(w)| > \varepsilon\right) \le \sum\limits_{n=1}^{\infty} \mathbb{P}\left( \overline E_{n}(\varepsilon)\right). $$

(40)

Claim 2

The probabilities of the complements of E_n(ε) are summable, i.e.,

$$ \sum\limits_{n=1}^{\infty} \mathbb{P}\left( \overline E_{n}(\varepsilon)\right) < \infty . $$

This is a direct application of the Hoeffding’s inequality (see e.g. [55, Theorem 2.2.2]) as follows. For any fixed (w, η) ∈ W × [0, B], the Hoeffding’s inequality gives

$$ \mathbb{P}(|\delta_{n}(w, \eta)| > \varepsilon/2) \le 2 \exp\left( - \frac{n\varepsilon^{2}}{2B^{2}}\right) . $$

Applied to all (w, η) ∈ T, this yields

$$ \mathbb{P}\big(\overline E_{n}(\varepsilon)\big) \le 2|T|\exp\left( - \frac{n\varepsilon^{2}}{2B^{2}}\right) = \frac{16B}{((1+1/(1-p))\varepsilon} N\left( \frac{\varepsilon(1-p)}{8M}\right) \exp\left( - \frac{n\varepsilon^{2}}{2B^{2}}\right) . $$

and proves Claim 2.

We conclude on the uniform convergence (38) with the Borel-Cantelli Lemma by the classical rationale (see e.g. the textbook [37, Chap. 2, Sec. 6]): the bound (40) and Claim 2 give that the probabilities for any 𝜖 are summable; applying Borel-Cantelli with the sequence 𝜖_k = 1/k gives the uniform convergence (38), which completes the proof of the theorem.

Finally, it remains to show measurability of some events of interest.

Claim 3

The following events are measurable for each ε > 0:

$$ \begin{array}{@{}rcl@{}} E^{\prime}_{n}(\varepsilon) &:=& \left\{\underset{w \in W}{\sup} |S^{p}_{n}(w) - S^{p}(w)| > \varepsilon\right\} , \\ E^{\prime\prime}_{n}(\varepsilon) &:=& \left\{\underset{(w,\eta) \in W \times [0, B]}{\sup} \delta_{n}(w, \eta) >\varepsilon\right\} . \end{array} $$

We prove the claim for $E^{\prime }_{n}(\varepsilon )$ and the second one is entirely analogous. Since the set $\mathbb {Q}^{d}$ of d-dimensional rationals is dense in $\mathbb {R}^{d}$ and the map $w \mapsto |S^{p}_{n}(w) - S^{p}(w)|$ is continuous, we have that

$$ \underset{w \in W}{\sup} |S^{p}_{n}(w) - S^{p}(w)| = \underset{w \in W \cap \mathbb{Q}^{d}}{\sup} |S^{p}_{n}(w) - S^{p}(w)| . $$

Since the latter term is a supremum over a countable set of measurable random variables, we get that $E^{\prime }_{n}(\varepsilon )$ is measurable.

Appendix B: Numerical Illustrations

We provide simple illustrations of the interest of using superquantile for machine learning. More precisely, we reproduce the experimental framework of the computational experiments of [9] and we solve the superquantile optimization problems with the approach depicted here, by combining smoothing and quasi-Newton. For additional experiments with other datasets, metrics, and contexts, we refer to [9].

We consider two basic machine learning tasks (regression and classification) with linear prediction functions φ(w, x) = w^⊤x and with two standard datasets, from the UCI ML repository. Denoting these datasets P_n = {(x_i, y_i)}_1≤i≤n, we introduce the (regularized) empirical risk minimization

$$ \underset{w \in \mathbb{R}^{d}}{\min} ~~\mathbb{E}_{(x,y)\sim P_{n}}\left[\ell(y, {w}^{\top} x)\right] +\frac{1}{2n}\|w\|^{2} , $$

and its smoothed superquantile analogous

$$ \underset{w \in \mathbb{R}^{d}}{\min} ~~{[\mathbb{S}^{\nu}_{p}]}_{(x,y)\sim P_{n}}\left[\ell(y, {w}^{\top} x) \right] +\frac{1}{2n}\|w\|^{2}. $$

We solve these problems using L-BFGS via the toolbox SPQR [22] offering an simple user-interface and implementing the oracles (with the Euclidean smoothing of Example 5 for the smoothed approximation).

Regression and Least-Squares

We consider a regularised least square regression on the dataset Abalone from the UCI Machine learning repository. We perform a 80%/20% train-test split on the dataset. We minimize the least-squares loss on the training set both in expectation and with respect to the superquantile (with p = 0.98 and ν = 0.1).

We report on Fig. 7 the distribution of errors $|y_{i} - {w}^{\top } x_{i}|$ for the testing dataset for both models w (standard in blue and superquantile in red). We observe that the superquantile model exhibits a thinner upper tail than the risk-neutral model, which is quantified by the shift to the left of 0.98 quantile. This comes at the price of lower performance in expectations than the model trained with expectation, which is clear visible on the picture and quantified by the shift to the right of the mean.

Fig. 7

Regression: histogram of the regression errors on the testing dataset for the model learning by the superquantile approach (red) compared to the one of the classical empirical risk minimization (violet). We see a reshaping of the histogram of errors and a gain on worst-case errors

Classification and Logistic regression

We consider a logistic regression on the Australian Credit dataset. We randomly split the dataset with a 80%/20% train-test split for 5 different seeds. For each seed, we perform a pessimistic distributional shift on the training dataset by downsampling the majority class (similarly to what is done in [9, Sec. 5.2]). More precisely, we remove an important fraction of the majority class, randomly selected, so that it counts afterward for only 10% of the minority class. We tune then the safety level parameter p by a k-cross validation on the shifted dataset and select the safety parameter yielding the best validation accuracy. The grid we use for tuning this parameter is [0.8, 0.85, 0.9, 0.95, 0.99] We finally compute with this parameter the testing accuracy and the testing precision.

We report the testing accuracy and the testing precision averaged over the 5 different seeds on the table of Fig. 8 with the associated standard deviation. We observe that the superquantile model brings better performance for both in terms of accuracy and precision than the standard model.

Fig. 8

Classification: better testing accuracy and precision for the superquantile approach, in the case of distributional shifts