Learning from noisy label proportions for classifying online social data

Abstract

Inferring latent attributes (e.g., demographics) of social media users is important to improve the accuracy and validity of social media analysis methods. While most existing approaches use either heuristics or supervised classification, recent work has shown that accurate classification models can be trained using supervision from population statistics. These learning with label proportion (LLP) models are fit on bags of instances and then applied to individual accounts. However, it is well known that many social media sites such as Twitter are not a representative sample of the population; thus, there are many sources of noise in these label proportions (e.g., sampling bias). This can in turn degrade the quality of the resulting model. In this paper, we investigate classification algorithms that use population statistical constraints such as demographics, names, and social network followers to fit classifiers to predict individual user attributes. We propose LLP methods that explicitly model the noise inherent in these label proportions. On several real and synthetic datasets, we find that combining these enhancements together can significantly reduce averaged classification error by 7%, resulting in methods that are robust to noise in the provided label proportions.

Appendix: Partial derivatives of LR cost function

We use logistic function derivative, i.e.,

$$\begin{aligned} \frac{\partial }{\partial \theta } \sigma (f) = \sigma (f)(1 - \sigma (f)) \frac{\partial }{\partial \theta } f \end{aligned}$$

(23)

to compute the derivative of hypothesis as:

$$\begin{aligned} \frac{\partial }{\partial \theta } h_{u, i} = h_{u, i}(1 - h_{u, i}) X_{u, i} \end{aligned}$$

(24)

Now we can compute the partial derivative of cost function:

$$\begin{aligned} \begin{aligned} \frac{\partial }{\partial \theta } J(\varTheta )&= -\,\sum _i \left( \tilde{h_i} \frac{\partial }{\partial \theta } \log \bar{h_i} + (1 - \tilde{h_i}) \frac{\partial }{\partial \theta } \log (1 - \bar{h_i})\right) + \lambda \theta \\&= -\,\sum _i \left( \frac{\tilde{y_i}}{\bar{h_i}} \frac{\partial \bar{h_i}}{\partial \theta } - \frac{1 - \tilde{y_i}}{1 - \bar{h_i}} \frac{\partial \bar{h_i}}{\partial \theta }\right) + \lambda \theta \\&= \sum _i \frac{\bar{h_i} - \tilde{y_i}}{\bar{h_i}(1 - \bar{h_i})} \frac{\partial \bar{h_i}}{\partial \theta } + \lambda \theta \\&= \sum _i \frac{\bar{h_i} - \tilde{y_i}}{\bar{h_i}(1 - \bar{h_i})} \frac{\partial }{\partial \theta } \frac{1}{|T_i|} \sum _{u \in T_i} h_{u, i} + \lambda \theta \\&= \sum _i \frac{\bar{h_i} - \tilde{y_i}}{\bar{h_i}(1 - \bar{h_i})|T_i|} \sum _{u \in T_i} \frac{\partial }{\partial \theta } h_{u, i} + \lambda \theta \\&= \sum _i \frac{\bar{h_i} - \tilde{y_i}}{\bar{h_i}(1 - \bar{h_i})|T_i|} \sum _{u \in T_i} h_{u, i}(1 - h_{u, i}) X_{u, i} + \lambda \theta \\&= \sum _{u,i} \frac{(\bar{h_i} - \tilde{y_i})h_{u, i}(1 - h_{u, i})}{\bar{h_i}(1 - \bar{h_i})|T_i|} X_{u, i} + \lambda \theta \\&= \sum _{u,i} e_{u,i} X_{u, i} + \lambda \theta \end{aligned} \end{aligned}$$

(25)

where $e_{u,i}$ is defined in Eq. 10. The partial derivative of other variables in LRBF model is computed similarly.