Tackling Algorithmic Bias in Neural-Network Classifiers using Wasserstein-2 Regularization

Abstract

The increasingly common use of neural network classifiers in industrial and social applications of image analysis has allowed impressive progress these last years. Such methods are, however, sensitive to algorithmic bias, i.e., to an under- or an over-representation of positive predictions or to higher prediction errors in specific subgroups of images. We then introduce in this paper a new method to temper the algorithmic bias in Neural-Network-based classifiers. Our method is Neural-Network architecture agnostic and scales well to massive training sets of images. It indeed only overloads the loss function with a Wasserstein-2-based regularization term for which we back-propagate the impact of specific output predictions using a new model, based on the Gâteaux derivatives of the predictions distribution. This model is algorithmically reasonable and makes it possible to use our regularized loss with standard stochastic gradient-descent strategies. Its good behavior is assessed on the reference Adult census, MNIST, CelebA datasets.

Appendices

A Proof of Gâteaux Differentiability

1.1 A.1 Proof of Proposition 1

Proof

Recall from the duality of the transport problem that it can be rewritten as the optimization problem

$$\begin{aligned} {\mathcal {W}}^2_2(\mu ,\nu ){=}\sup _{f}\int x^2{-}2f(x)d\mu (x){+}\int y^2-2f^*(y)d\nu (y), \end{aligned}$$

(20)

where the optimization is over the set of convex functions. Following [50], we then denote here f a convex function and $f^*$ its conjugate. Let $f_0$ and $f_{t}$ be the solutions for ${\mathcal {W}}^2_2(\mu ,\nu )$ and ${\mathcal {W}}^2_2(\mu +t\alpha ,\nu +t\beta )$. We then have

$$\begin{aligned}&{\mathcal {W}}^2_2(\mu +t\alpha ,\nu +t\beta )-{\mathcal {W}}^2_2(\mu ,\nu )\\&\le t\left( \int x^2-2f_t(x)d\alpha (x)+\int y^2-2f_t^*(y)d\beta (y)\right) \end{aligned}$$

and

$$\begin{aligned}&{\mathcal {W}}^2_2(\mu +t\alpha ,\nu +t\beta )-{\mathcal {W}}^2_2(\mu ,\nu )\\&\ge t\left( \int x^2-2f_0(x)d\alpha (x)+\int y^2-2f_0^*(y)d\beta (y)\right) . \end{aligned}$$

Note that, following [51, 52], the convergence $\mu +t\alpha \xrightarrow {w} \mu$ implies that $f_t\rightarrow f_0$ uniformly on the compact sets of the support of $\mu$ if the support is connected. As a consequence, if ${\text {Supp}}(\alpha )\subset {\text {Supp}}(\mu )$ and ${\text {Supp}}(\beta )\subset {\text {Supp}}(\nu )$, then under the assumption of compact supports we have

$$\begin{aligned} D{\mathcal {W}}^2_2(\mu ,\nu )(\alpha ,\beta )= \int x^2-2f_0(x)d\alpha (x)+\int y^2-2f_0^*(y)d\beta (y). \end{aligned}$$

(21)

Recall from [53] (Section 2.2.) that in the real line case the potential of the transport, solution of Eq. (20), is a primitive of the transport map, which is the map $F^{-1}_{\mu }(F_{\nu })$. Then, the conclusion becomes straightforward. $\square$

Fig. 5

Observations with the highest differences of predictions with or without regularization. We recall that Y represents the subjective variable Attractive, which represents who is attractive for the persons who labelled the data, and S is the variable Young

1.2 A.2 Proof of Lemma 1

Proof

Set $t>0$ and $\beta$, then compute

$$\begin{aligned} L(\mu +t\beta )-L(\mu )= \int l d \mu + t\int l d \beta - \int l d \mu =t\int l d \beta . \end{aligned}$$

In consequence, we have that

$$\begin{aligned} \frac{L(\mu +t\beta )-L(\mu )}{t}= \int l d \beta . \end{aligned}$$

$\square$

B Extension to Error Rates Eq. (19)

By using the same strategy as in Proposition 1 with $(f_{\theta }(x_i)-y_i)^2$ and the ${\tilde{H}}_g$ instead of $f_{\theta }(x_i)$ and the $H_g$, we can compute:

$$\begin{aligned}&\displaystyle { \varDelta _{\tau } \left[ \mathbb {1}_{s_i=0} \frac{ (f_{\theta }(x_i)-y_i)^2 - cor_1((f_{\theta }(x_i)-y_i)^2) }{ n_0 \left( {\tilde{H}}_{0}^{j_i+1} - {\tilde{H}}_{0}^{j_i} \right) } \right. } \nonumber \\&\displaystyle { \left. - \mathbb {1}_{s_i=1} \frac{ cor_0((f_{\theta }(x_i)-y_i)^2) - (f_{\theta }(x_i)-y_i)^2 }{ n_1 \left( {\tilde{H}}_{1}^{j_i+1} - {\tilde{H}}_{1}^{j_i} \right) } \right] . } \end{aligned}$$

(22)

This equation specifically expresses how to back-propagate the impact of a squared error $(f_{\theta } (x_i)-y_i)^2$, and not an output observation $f_{\theta } (x_i)$, on $W_2^2 (\tilde{\mu }^n_{\theta ,0},\tilde{\mu }^n_{\theta ,1})$. From Lemma 1, we know that the chain rule applies on the Gâteaux differentiability which is used to back-propagate the impact of $f_{\theta } (x_i)$ on the distributions. As

$$\begin{aligned} \frac{\partial (f_{\theta }(x_i)-y_i)^2}{\partial f_{\theta }(x_i)} = 2 (f_{\theta }(x_i)-y_i) \,, \end{aligned}$$

(23)

we can then simply deduce that the impact of an observation $f_{\theta } (x_i)$ on $W_2^2 (\tilde{\mu }^n_{\theta ,0},\tilde{\mu }^n_{\theta ,1})$ can be back-propagated using Eq. (19):

$$\begin{aligned}&\displaystyle { 2 \varDelta _{\tau } \left[ \mathbb {1}_{s_i=0} \frac{ (f_{\theta }(x_i)-y_i)^2 - cor_1 \left( (f_{\theta }(x_i)-y_i)^2\right) }{n_0 \left( {\tilde{H}}_{0}^{j_i+1} - {\tilde{H}}_{0}^{j_i}\right) \left( f_{\theta }(x_i)-y_i\right) ^{-1}} \right. } \nonumber \\&\displaystyle { \left. - \mathbb {1}_{s_i=1} \frac{ cor_0 \left( (f_{\theta }(x_i)-y_i)^2 \right) - (f_{\theta }(x_i)-y_i)^2 }{n_1 \left( {\tilde{H}}_{1}^{j_i+1} - {\tilde{H}}_{1}^{j_i}\right) \left( f_{\theta }(x_i)-y_i\right) ^{-1} } \right] \,. } \end{aligned}$$

(24)

C Extensions of the Method of Section 3.4

1.1 C.1 Wasserstein-1 Distances

Our approach can be straightforwardly extended to approximate Wasserstein-1 distances. By using the same reasoning as in Sect. 3.4, it can be shown that the impact of an observation $f_{\theta } (x_i)$ on $W_1 (\tilde{\mu }^n_{\theta ,0},\tilde{\mu }^n_{\theta ,1})$ can be back-propagated using

$$\begin{aligned} \varDelta _{\tau } \displaystyle { \left[ \mathbb {1}_{s_i=0} \frac{ sign \left( \eta ^{j_{i}} - \eta ^{j_{i}'} \right) }{n_0 (H_{0}^{j_i+1} - H_{0}^{j_i})} - \mathbb {1}_{s_i=1} \frac{ sign \left( \eta ^{j_{i}'} - \eta ^{j_{i}} \right) }{n_1 (H_{1}^{j_i+1} - H_{1}^{j_i})} \right] } \,, \end{aligned}$$

(25)

instead of Eq. (13), where sign(x) is equal to $+1$ or $-1$ depending on the sign of x. We emphasize that the distances between the cumulative densities are therefore not taken into account when computing the gradients of the Wasserstein-1, although this is the case for Wasserstein-2 distances.

Fig. 6

Impact of $\lambda$ on the convergence of the neural-network predictions on the CelebA dataset, when using the experimental protocol of Section . All results were obtained using the mean squared error as a loss, except for the sub-figure on the top-left, which were obtained using Binary Cross Entropy. Blue and red curves represent the accuracy for the observations with $S=1$ and $S=0$, respectively. Green curves represent the disparate impact. Continuous and dashed curves were finally obtained on the training and test sets. The iterations represented in abscissa are in mini-batches and not in epochs

1.2 C.2 Logistic Regression

We now show how to simply implement our regularization model for Logistic Regression. We minimize:

$$\begin{aligned} \hat{\theta } = {{\,\mathrm{arg\,min}\,}}_{\theta }&\frac{1}{n} \sum _{i=1}^n \log \left( f_{\theta } (x_i)^{y_i} \left( 1 - f_{\theta } (x_i) \right) ^{1-y_i} \right) \nonumber \\&+ \lambda W_2^2 (\mu ^n_{\theta ,0},\mu ^n_{\theta ,1}) \,, \end{aligned}$$

(26)

where $f_{\theta } (x_i) = (1+\exp {( -\theta ^0 -\theta ' x_i)})^{-1}$ is the logistic function and $\theta = (\theta ^1, \ldots , \theta ^p)'$ is a vector in ${\mathbb {R}}^p$ representing the weights given to each dimension. The derivatives of the whole energy Eq. (26) with respect to each $\theta ^j$, $j=0, \ldots , p$, can be directly computed using finite differences here.

We emphasize that a fundamental difference between using our Wasserstein-based regularization model in Sect. 3.4 and here is that p derivatives of the minimized energy are approximated using Logistic Regression (derivatives w.r.t. the $\theta ^j$, $j=0, \ldots , p$), while n derivatives are required when using Neural Networks with a standard gradient descent (derivatives w.r.t. the $f_{\theta } (x_i)$, $i=0, \ldots , n$). As a cumulative histogram is computed each time, the derivative of a Wasserstein-2 distance is approximated, this task can be bottleneck for common Neural-Networks applications where n is large. This fully justifies the proposed batch-training regularization strategy of Sect. 3.4.

1.3 C.3 Automatic Tuning of $\lambda$

The minimized energy Eq. (5) depends on a weight $\lambda$ which balances the influence of the regularization term $W_2^2 (\mu ^n_{\theta ,0},\mu ^n_{\theta ,1})$ with respect to the data attachment term $R(\theta )$. A simple way to automatically tune $\lambda$ is the following. Compute the average derivatives of $W_2$ and R after a predefined warming phase of several epochs, where $\lambda =0$. We denote $d_{W_2}$ and $d_{R}$ these values. Then, tune $\lambda$ as equal to $\alpha \frac{g_{R}}{g_{W_2}}$, where $\alpha$ is typically in [0.1, 1]. This makes it intuitive to tune the scale of $\lambda$.

In the disparate impact case (Sect. 3.4), it can be interesting to accurately adapt $\alpha$ to the machine learning problem, in order to finely tune $\lambda$ with regards to the fact that we simultaneously want fair and accurate predictions. Inspired by the hard constraints of [19] to enforce fair predictions, we update $\alpha$ based on measures of the Disparate Impact (DI), Eq. (1), and average Prediction Accuracy (Acc) at the beginning of each epoch. Remark that lowering the Wasserstein-2 distance between the predictions $f_{\theta }(x_i)$ in groups 0 and 1 naturally tend to make decrease $\mathbb {1}_{f_{\theta }(x_i,s_i=0)>0.5}-\mathbb {1}_{f_{\theta }(x_i,s_i=1)>0.5}$, which we empirically verified. The disparate impact therefore tends to be improved. We believe that hard constraints based on other fairness measures could also be used. Establishing a clear relation of causality between the Wasserstein-2 distance and different fairness measures is, however, out of the scope of this paper and hence considered as future work. Note that the same technique holds in the squared error case (Sect. 3.5), but the Disparate Mean Squared Error (DMSE) index, Eq. (14), is used instead of the DI.

In the experiments of Sect. 4.1, our hard constraints are for instance: If the prediction accuracy is too low (Acc$<0.75$), then $\alpha$ is slightly decreased to favor the predictions accuracy ($\alpha =0.9\alpha$). If the prediction accuracy is sufficiently high and the DI is too low (DI$<0.85$), then $\alpha$ is slightly increased ($\alpha =1.1\alpha$) to favor fair decisions. We empirically verified in our experiments that $\alpha$ converges to satisfactory values using this method, if the classifier is able to learn classification rules leading to sufficiently high PA. Parameter $\alpha$ converges to zero otherwise.

D Impact of $\lambda$ on the Convergence

1.1 D.1 Results

In this section, we extend the results of Sect. 4.3.1 by discussing the convergence of the training algorithm on the training set and the test set for different values of $\lambda$. Each sub-figure specifically represents the accuracy of the prediction model mini-batch after mini-batch, where we distinguish the average accuracy obtained on observations with $S=0$ and those with $S=1$. The disparate impact is also given. We additionally compare the convergence of the training algorithm without regularization when using a mean squared error (MSE) loss, as in other experiments, and a binary cross entropy (BCE) loss, which is known to be less sensitive than MSE to outlier observations. The regularization was applied on the output predictions directly. Results are detailed in Fig. 6.

1.2 D.2 Discussion

We can first notice that the convergence properties of the training algorithm with $\lambda =0$ are very similar when using the MSE and the BCE loss. The potential outlier observations seem therefore to have no or little impact here. In both cases and both for $S=0$ and $S=1$, the observed level of accuracy is similar for the training set and the test set until about the 30th iteration. Then, the accuracy becomes clearly higher on the training set than on the test set, which means that the trained model overfits the training data. This phenomenon is particularly strong for the observations in the group $S=1$, which have a lower accuracy than in the group $S=0$ before overfitting. In all cases, the disparate impact is close to 0.25 at convergence.

Now, when increasing $\lambda$ from $1.e-3$ to $5.e-3$ the level of accuracy becomes increasingly similar between $S=0$ and $S=1$, and the disparate impact is closer and closer to 0.4. It can be remarked that for $\lambda =4.e-3$ and $\lambda =5.e-3$, the curves start oscillating when the training algorithm overfits the training data. The regularization therefore seems to be strong compared with the data attachment term. This phenomenon is stronger, even before overfitting, for $\lambda =6.e-3$. This explains why the results become unstable in Fig. 4-(top) for high values of $\lambda$ and confirms that a reasonable trade-off should be searched between regularization and prediction accuracy when tuning $\lambda$.