Skip to main content
SpringerLink
Log in

Simple and effective complementary label learning based on mean square error loss

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

A complementary label specifies one of the classes that an instance does not belong to. Complementary label learning only uses training instances each assigned a complementary label to train a classifier that can predict a ground-truth label for each testing instance. Though many surrogate loss functions have been proposed for complementary label learning, the mean square error (MSE) surrogate loss function, widely used in the standard classification paradigm, cannot provide classifier consistency in complementary label learning. However, classifier consistency not only guarantees the converged model is the optimal classifier that can be found in the searching space but also indicates that standard backpropagation is enough to search for the optimal classifier without needing model selection. This paper designs an effective square loss for complementary label learning under unbiased and biased assumptions. We also theoretically demonstrate that our method assurances that the optimal classifier under complementary labels is also the optimal classifier under ordinary labels. Finally, we test our method on different benchmark datasets with biased and unbiased assumptions to verify the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  2. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc (2017)

  3. Yang, W., Cao, Z., Chen, Q., Yang, Y., Yang, G.: Confidence calibration on multiclass classification in medical imaging. In: 2020 IEEE International Conference on Data Mining (ICDM), pp. 1364–1369 (2020)

  4. Yang, W., Yang, Y.: A stabilized dense network approach for high-dimensional prediction. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2021)

  5. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth \(16\times 16\) words: Transformers for image recognition at scale. In: International Conference on Learning Representations (2021)

  6. Li, W., Gao, Y., Zhang, M., Tao, R., Du, Q.: Asymmetric feature fusion network for hyperspectral and SAR image classification. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3149394

    Article  Google Scholar 

  7. Gao, Y., Li, W., Zhang, M., Wang, J., Sun, W., Tao, R., Du, Q.: Hyperspectral and multispectral classification for coastal wetland using depthwise feature interaction network. IEEE Trans. Geosci. Remote Sens. 60, 1–15 (2022). https://doi.org/10.1109/TGRS.2021.3097093

    Article  Google Scholar 

  8. Gao, Y., Zhang, M., Li, W., Song, X., Jiang, X., Ma, Y.: Adversarial complementary learning for multisource remote sensing classification. IEEE Trans. Geosci. Remote Sens. 61, 1–13 (2023). https://doi.org/10.1109/TGRS.2023.3255880

    Article  Google Scholar 

  9. Li, Y., Yang, J., Song, Y., Cao, L., Luo, J., Li, L.-J.: Learning from noisy labels with distillation. In: Proceedings of the IEEE International Conference on Computer Vision (2017)

  10. Lee, K.-H., He, X., Zhang, L., Yang, L.: Cleannet: transfer learning for scalable image classifier training with label noise. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  11. Hu, M., Han, H., Shan, S., Chen, X.: Weakly supervised image classification through noise regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)

  12. Xia, X., Liu, T., Wang, N., Han, B., Gong, C., Niu, G., Sugiyama, M.: Are anchor points really indispensable in label-noise learning? In: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 6838–6849. Curran Associates Inc, Red Hook (2019)

    Google Scholar 

  13. Liu, D., Zhao, J., Wu, J., Yang, G., Lv, F.: Multi-category classification with label noise by robust binary loss. Neurocomputing 482, 14–26 (2022)

    Article  Google Scholar 

  14. Liu, D., Yang, G., Wu, J., Zhao, J., Lv, F.: Robust binary loss for multi-category classification with label noise. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1700–1704 (2021)

  15. Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine Learning, pp. 912–919 (2003)

  16. Kingma, D.P., Mohamed, S., Jimenez Rezende, D., Welling, M.: Semi-supervised learning with deep generative models. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 3581–3589. Curran Associates Inc, Red Hook (2014)

    Google Scholar 

  17. Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: Mixmatch: a holistic approach to semi-supervised learning. In: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32, pp. 5049–5059. Curran Associates Inc, Red Hook (2019)

    Google Scholar 

  18. Rasmus, A., Berglund, M., Honkala, M., Valpola, H., Raiko, T.: Semi-supervised learning with ladder networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28, pp. 3546–3554. Curran Associates Inc, Red Hook (2015)

    Google Scholar 

  19. Miyato, T., Maeda, S., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1979–1993 (2019)

    Article  Google Scholar 

  20. Zhai, X., Oliver, A., Kolesnikov, A., Beyer, L.: S4l: self-supervised semi-supervised learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

  21. Sakai, T., du Plessis, M.C., Niu, G., Sugiyama, M.: Semi-supervised classification based on classification from positive and unlabeled data, vol. 70. PMLR, pp. 2998–3006 (2017)

  22. Chapelle, O., Scholkopf, B., Zien, Eds., A.: Semi-supervised learning (chapelle, O. et al., eds.; 2006) [book reviews]. IEEE Trans. Neural Netw. 20(3), 542–542 (2009)

  23. Yan, Y., Guo, Y.: Partial label learning with batch label correction. In: AAAI, pp. 6575–6582 (2020)

  24. Xu, N., Lv, J., Geng, X.: Partial label learning via label enhancement. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5557–5564 (2019)

  25. Zhang, M.-L., Yu, F.: Solving the partial label learning problem: an instance-based approach. In: IJCAI, pp. 4048–4054 (2015)

  26. Gong, C., Shi, H., Yang, J., Yang, J.: Multi-manifold positive and unlabeled learning for visual analysis. IEEE Trans. Circuits Syst. Video Technol. 30(5), 1396–1409 (2020)

    Article  Google Scholar 

  27. Ishida, T., Niu, G., Sugiyama, M.: Binary classification from positive-confidence data. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 5917–5928. Curran Associates Inc, Red Hook (2018)

    Google Scholar 

  28. Elkan, C., Noto, K.: Learning classifiers from only positive and unlabeled data. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 213–220. Association for Computing Machinery (2008)

  29. du Plessis, M.C., Niu, G., Sugiyama, M.: Analysis of learning from positive and unlabeled data. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 703–711. Curran Associates Inc, Red Hook (2014)

    Google Scholar 

  30. Kiryo, R., Niu, G., du Plessis, M.C., Sugiyama, M.: Positive-unlabeled learning with non-negative risk estimator. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 1675–1685. Curran Associates Inc, Red Hook (2017)

    Google Scholar 

  31. Bao, H., Niu, G., Sugiyama, M.: Classification from pairwise similarity and unlabeled data, vol. 80. PMLR, pp. 452–461 (2018)

  32. Lu, N., Niu, G., Menon, A.K., Sugiyama, M.: On the minimal supervision for training any binary classifier from only unlabeled data. In: International Conference on Learning Representations (2018)

  33. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709 (2020)

  34. Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.: Big self-supervised models are strong semi-supervised learners. arXiv preprint arXiv:2006.10029 (2020)

  35. Ishida, T., Niu, G., Hu, W., Sugiyama, M.: Learning from complementary labels. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 5639–5649. Curran Associates Inc, Red Hook (2017)

    Google Scholar 

  36. Liu, D., Ning, J., Wu, J., Yang, G.: Extending ordinary-label learning losses to complementary-label learning. IEEE Signal Process. Lett. 28, 852–856 (2021)

    Article  Google Scholar 

  37. Yu, X., Liu, T., Gong, M., Tao, D.: Learning with biased complementary labels. In: Proceedings of the European Conference on Computer Vision, pp. 68–83 (2018)

  38. Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., Qu, L.: Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)

  39. Ishida, T., Niu, G., Menon, A., Sugiyama, M.: Complementary-label learning for arbitrary losses and models, vol. 97. PMLR, pp. 2971–2980 (2019)

  40. Liu, D., Yang, G.: Robust loss functions for complementary labels learning (2021). https://openreview.net/forum?id=LhAqAxwH5cn

  41. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  42. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747 (2017)

  43. Krizhevsky, A.: Learning multiple layers of features from tiny images. Master’s thesis, University of Tront (2009)

  44. Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)

    Article  Google Scholar 

  45. Cour, T., Sapp, B., Taskar, B.: Learning from partial labels. J. Mach. Learn. Res. 12, 1501–1536 (2011)

    MathSciNet  MATH  Google Scholar 

  46. Chou, Y.-T., Niu, G., Lin, H.-T., Sugiyama, M.: Unbiased risk estimators can mislead: a case study of learning with complementary labels. In: International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119. PMLR, pp. 1929–1938 (2020)

  47. Feng, L., Kaneko, T., Han, B., Niu, G., An, B., Sugiyama, M.: Learning with multiple complementary labels. In: International Conference on Machine Learning. PMLR, pp. 3072–3081 (2020)

  48. Feng, L., Lv, J., Han, B., Xu, M., Niu, G., Geng, X., An, B., Sugiyama, M.: Provably consistent partial-label learning. In: Advances in Neural Information Processing Systems, pp. 10948–10960. Curran Associates Inc, Red Hook (2020)

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for the careful reading of this paper and the constructive comments they provided. This paper is supported by the China Postdoctoral Science Foundation (NO. 2019TQ0051).

Author information

Authors and Affiliations

  1. Southwest China Institute of Electronic Technology, Chengdu, 610036, Sichuan, China

    Chenggang Wang

  2. University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, China

    Xiong Xu, Defu Liu, Xinyu Niu & Shijiao Han

Authors
  1. Chenggang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Defu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinyu Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shijiao Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shijiao Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Xu, X., Liu, D. et al. Simple and effective complementary label learning based on mean square error loss. Machine Vision and Applications 34, 118 (2023). https://doi.org/10.1007/s00138-023-01469-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01469-0

Keywords

Search

Navigation