Skip to main content
SpringerLink
Log in

Sample separation and domain alignment complementary learning mechanism for open set domain adaptation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Open Set Domain Adaptation (OSDA) reduces domain shift and semantic shift by dividing the known/unknown target samples and aligning the known target samples with the source samples. Unfortunately, either separating or aligning first will cause the negative shift to the other side. Moreover, numerous methods do not utilize the sample knowledge of the target domain. In this study, a new method is put forward to address the issue called Sample Separation and Domain Alignment Complementary Learning Mechanism (CLM) for Open Set Domain Adaptation. Specifically, this work proposes a complementary learning mechanism that jointly trains two complementary learning structures including Sample Separation Module (SSMod) and Domain Alignment Module (DAMod). SSMod and DAMod are performed simultaneously, exchanging training experiences during the learning process using the self-supervised pseudo-labeling method. In addition, we introduce a novel sample separation method, which not only facilitates the distinction between known and unknown classes of target samples but also enriches the semantic knowledge of the model by employing the unlabeled data in an unsupervised manner. Extensive experiments demonstrate that our method realizes significant performance on four standard Digits, Office-31, Office-Home and VisDA-2017 benchmarks. For example, CML achieves 89.2% accuracy of HOS on Office-31 and increases by 1.8% than the second best method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The datasets used in this study are publicly available online.

References

  1. Xu Y, Chen L, Duan L, Tsang IW, Luo J (2021) Open set domain adaptation with soft unknown-class rejection. IEEE Transactions on Neural Networks and Learning Systems

  2. Shermin T, Lu G, Teng SW, Murshed M, Sohel F (2020) Adversarial network with multiple classifiers for open set domain adaptation. IEEE Transactions on Multimedia

  3. Zhong L, Fang Z, Liu F, Yuan B, Zhang G, Lu J (2021) Bridging the theoretical bound and deep algorithms for open set domain adaptation. IEEE Transactions on Neural Networks and Learning Systems

  4. Liu J, Jing M, Li J, Lu K, Shen HT (2021) Open set domain adaptation via joint alignment and category separation. IEEE Transactions on Neural Networks and Learning Systems

  5. Fu J, Wu X, Zhang S, Yan J (2019) Improved open set domain adaptation with backpropagation. In: 2019 IEEE international conference on image processing (ICIP), IEEE pp 2506–2510

  6. Jing M, Li J, Zhu L, Ding Z, Lu K, Yang Y (2021) Balanced open set domain adaptation via centroid alignment. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 8013–8020

  7. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning, PMLR pp 97–105

  8. Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE international conference on data mining (ICDM), IEEE pp 1129–1134

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Advances in neural information processing systems, p 27

  10. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning, PMLR pp 1180–1189

  11. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. Advances in neural information processing systems, p 31

  12. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 7167–7176

  13. Loghmani MR, Vincze M, Tommasi T (2020) Positive-unlabeled learning for open set domain adaptation. Pattern Recogn Lett 136:198–204

    Article  Google Scholar 

  14. Panareda Busto P, Gall J (2017) Open set domain adaptation. In: Proceedings of the IEEE International conference on computer vision, pp 754–763

  15. Saito K, Yamamoto S, Ushiku Y, Harada T (2018) Open set domain adaptation by backpropagation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 153–168

  16. Liu H, Cao Z, Long M, Wang J, Yang Q (2019) Separate to adapt: Open set domain adaptation via progressive separation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2927–2936

  17. Bucci S, Loghmani MR, Tommasi T (2020) On the effectiveness of image rotation for open set domain adaptation. In: European conference on computer vision, pp 422–438. Springer

  18. Wang Q, Meng F, Breckon TP (2021) Progressively select and reject pseudo-labelled samples for open-set domain adaptation. arXiv:2110.12635

  19. Gao Y, Chen P, Gao Y, Wang J, Pan Y, Ma AJ (2022) Hierarchical feature disentangling network for universal domain adaptation. Pattern Recogn 127:108616

    Article  Google Scholar 

  20. Kirichenko P, Izmailov P, Wilson AG (2020) Why normalizing flows fail to detect out-of-distribution data. arXiv:2006.08545

  21. Yan X, Zhang H, Xu X, Hu X, Heng P-A (2021) Learning semantic context from normal samples for unsupervised anomaly detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 3110–3118

  22. Sun Y, Guo C, Li Y (2021) React: out-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems, p 34

  23. Zaeemzadeh A, Bisagno N, Sambugaro Z, Conci N, Rahnavard N, Shah M (2021) Out-of-distribution detection using union of 1-dimensional subspaces. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 9452–9461

  24. Huang R, Geng A, Li Y (2021) On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, p 34

  25. Liang S, Li Y, Srikant R (2017) Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv:1706.02690

  26. Liu W, Wang X, Owens J, Li Y (2020) Energy-based out-of-distribution detection. Adv Neural Inf Process Syst 33:21464–21475

    Google Scholar 

  27. Wang H, Liu W, Bocchieri A, Li Y (2021) Can multi-label classification networks know what they don’t know? Advances in Neural Information Processing Systems, p 34

  28. Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. J R Stat Soc Series C (applied statistics) 28(1):100–108

    MATH  Google Scholar 

  29. Lee D-h et al (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, vol 3, p 896

  30. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  31. Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning

  32. Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  33. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision, pp 213–226. Springer

  34. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5018–5027

  35. Peng X, Usman B, Kaushik N, Hoffman J, Wang D, Saenko K (2017) Visda: The visual domain adaptation challenge. arXiv:1710.06924

  36. You K, Long M, Cao Z, Wang J, Jordan MI (2019) Universal domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2720–2729

  37. Jain LP, Scheirer WJ, Boult TE (2014) Multi-class open set recognition using probability of inclusion. In: European conference on computer vision, pp 393–409. Springer

  38. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  39. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  40. Smith LN, Topin N (2019) Super-convergence: very fast training of neural networks using large learning rates. Artificial intelligence and machine learning for multi-domain operations applications, vol 11006, International Society for Optics and Photonics p 1100612

  41. Lian Q, Li W, Chen L, Duan L (2019) Known-class aware self-ensemble for open set domain adaptation. arXiv:1905.01068

  42. Fang Z, Lu J, Liu F, Xuan J, Zhang G (2020) Open set domain adaptation: Theoretical bound and algorithm. IEEE Trans Neural Netw Learn Syst 32(10):4309–4322

    Article  MathSciNet  Google Scholar 

  43. Gao Y, Ma AJ, Gao Y, Wang J, Pan Y (2020) Adversarial open set domain adaptation via progressive selection of transferable target samples. Neurocomputing 410:174–184

    Article  Google Scholar 

  44. Kundu JN, Venkat N, Revanur A, Babu RV et al (2020) Towards inheritable models for open-set domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12376–12385

  45. Mendes Júnior PR, De Souza RM, Werneck RdO, Stein BV, Pazinato DV, de Almeida WR, Penatti OA, Torres RdS, Rocha A (2017) Nearest neighbors distance ratio open-set classifier. Mach Learn 106(3):359–386

    Article  MathSciNet  MATH  Google Scholar 

  46. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  47. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE International conference on computer vision, pp 2200–2207

  48. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1859–1867

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (No. 2020YFA0714103), the Innovation Capacity Construction Project of Jilin Province Development and Reform Commission (2021FGWCXNLJSSZ10, 2019C053-3) and the Fundamental Research Funds for the Central Universities, JLU.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, 2699 Qianjin Street, Changchun, 130012, Jilin, China

    Long Sifan, Wang Shengsheng, Zhao Xin, Fu Zihao & Wang Bilin

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, 2699 Qianjin Street, Changchun, 130012, Jilin, China

    Long Sifan, Wang Shengsheng, Zhao Xin, Fu Zihao & Wang Bilin

Authors
  1. Long Sifan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wang Shengsheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhao Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fu Zihao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wang Bilin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wang Shengsheng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sifan, L., Shengsheng, W., Xin, Z. et al. Sample separation and domain alignment complementary learning mechanism for open set domain adaptation. Appl Intell 53, 18790–18805 (2023). https://doi.org/10.1007/s10489-022-04262-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04262-0

Keywords

Search

Navigation