Skip to main content
SpringerLink
Log in

Bi-directional class-wise adversaries for unsupervised domain adaptation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Unsupervised domain adaptation relies on well-labeled auxiliary source domain information to get better performance on the unlabeled target domain. It has shown tremendous importance for various classification and segmentation problems. Classical methods rely on diminishing the domain discrepancy in the latent space but ignore class-wise information, which will lead to elimination of the inherent data structure. To avoid destroying the inherent structure during unsupervised domain adaptation, we propose a Bi-Directional Class-level Adversaries cross-domain model (BDCA) with two symmetric classifiers interpolating two latent spaces to build a tunnel between the source domain and target domain. Specifically, we propose a class-level discrepancy metric to enforce domain consistency during the trend of domain adaption. We also employ two symmetric classifiers that are collectively optimized to maximize the discrepancy on target sample prediction. Extensive experiments are conducted on four publicly available datasets (i.e. office-31, office-home, GTAV and Cityscapes) and two challenging computer vision prediction problems, i.e., image classification and semantic segmentation. Quantitative and qualitative results demonstrate the effectiveness of our proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010, pp. 177–186. Springer

  2. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3722–3731

  3. Chen C, Xie W, Huang W, Rong Y, Ding X, Huang Y, Xu T, Huang J (2019) Progressive feature alignment for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 627–636

  4. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40(4):834–848

    Article  Google Scholar 

  5. Chen MH, Kira Z, AlRegib G, Yoo J, Chen R, Zheng J (2019) Temporal attentive alignment for large-scale video domain adaptation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6321–6330

  6. Chen X, Wang S, Long M, Wang J (2019) Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In: International conference on machine learning, pp. 1081–1090. PMLR

  7. Chen YC, Lin YY, Yang MH, Huang JB (2019) Crdoco: Pixel-level domain transfer with cross-domain consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1791–1800

  8. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213–3223

  9. Cui S, Wang S, Zhuo J, Li L, Huang Q, Tian Q (2020) Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3941–3950

  10. Cui S, Wang S, Zhuo J, Su C, Huang Q, Tian Q (2020) Gradually vanishing bridge for adversarial domain adaptation. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 12455–12464

  11. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: IEEE Conference on computer vision and pattern recognition, pp. 248–255. Ieee

  12. Ding Z, Fu Y (2016) Robust transfer metric learning for image classification. IEEE Trans Image Process 26(2):660–670

    Article  MathSciNet  Google Scholar 

  13. Ding ZM, Fu Y (2018) Deep transfer low-rank coding for cross-domain learning. IEEE transactions on neural networks and learning systems 30(6):1768–1779

    Article  MathSciNet  Google Scholar 

  14. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp. 647–655

  15. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17(1):2096–2030

    MathSciNet  MATH  Google Scholar 

  16. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2066–2073. IEEE

  17. Gong R, Li W, Chen Y, Gool LV (2019) Dlow: Domain flow for adaptation and generalization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2477–2486

  18. Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B., Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(Mar):723–773

    MathSciNet  MATH  Google Scholar 

  19. Guan D, Huang J, Lu S, Xiao A (2021) Scale variance minimization for unsupervised domain adaptation in image segmentation. Pattern Recogn 112:107764

    Article  Google Scholar 

  20. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  21. Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Efros A, Darrell T (2018) Cycada: Cycle-consistent adversarial domain adaptation. In: International conference on machine learning, pp. 1989–1998. PMLR

  22. Hoffman J, Wang D, Yu F, Darrell T (2016) Fcns in the wild:, Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649

  23. Huang J, Lu S, Guan D, Zhang X (2020) Contextual-relation consistent domain adaptation for semantic segmentation. In: European conference on computer vision, pp. 705–722. Springer

  24. Kang G, Jiang L, Yang Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4893–4902

  25. Kingma DP, Ba J (2014) Adam:, A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  26. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105

  27. Kurmi VK, Kumar S, Namboodiri VP (2019) Attending to discriminative certainty for domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 491–500

  28. Lee CY, Batra T, Baig MH, Ulbricht D (2019) Sliced wasserstein discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10285–10295

  29. Li J, Wu Y, Lu K (2016) Structured domain adaptation. IEEE Transactions on Circuits and Systems for Video Technology 27(8):1700–1713

    Article  Google Scholar 

  30. Li M, Zhai YM, Luo YW, Ge PF, Ren CX (2020) Enhanced transport distance for unsupervised domain adaptation. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 13936–13944

  31. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791

  32. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. In: Advances in neural information processing systems, pp. 1640–1650

  33. Long M, Wang J, Ding G, Pan SJ, Philip SY (2013) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089

    Article  Google Scholar 

  34. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp. 2200– 2207

  35. Long M, Zhu H, Wang J, Jordan MI (2016) Unsupervised domain adaptation with residual transfer networks. In: Advances in neural information processing systems, pp. 136–144

  36. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: Proceedings of the International Conference on Machine Learning, pp. 2208–2217. JMLR. org

  37. Luo Y, Zheng L, Guan T, Yu J, Yang Y (2019) Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2507–2516

  38. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the international conference on machine learning, vol. 30, p. 3

  39. Mancini M, Bulò SR, Caputo B, Ricci E (2019) Adagraph: Unifying predictive and continuous domain adaptation through graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6568–6577

  40. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks 22(2):199–210

    Article  Google Scholar 

  41. Pan Y, Yao T, Li Y, Wang Y, Ngo CW, Mei T (2019) Transferrable prototypical networks for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2239–2247

  42. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. In: AAAI Conference on artificial intelligence

  43. Peng X, Usman B, Kaushik N, Hoffman J, Wang D, Saenko K (2017) Visda:, The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924

  44. Pinheiro PO (2018) Unsupervised domain adaptation with similarity learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8004–8013

  45. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  46. Richter SR, Vineet V, Roth S, Koltun V (2016) Playing for data: Ground truth from computer games. In: European conference on computer vision, pp. 102–118. Springer

  47. Ros G, Sellart L, Materzynska J, Vazquez D, Lopez AM (2016) The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3234–3243

  48. Roy S, Siarohin A, Sangineto E, Bulo SR, Sebe N, Ricci E (2019) Unsupervised domain adaptation using feature-whitening and consensus loss. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9471– 9480

  49. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision, pp. 213–226. Springer

  50. Saito K, Ushiku Y, Harada T (2017) Asymmetric tri-training for unsupervised domain adaptation. In: Proceedings of the International Conference on Machine Learning, pp. 2988–2997. JMLR. org

  51. Saito K, Watanabe K, Ushiku Y, Harada T (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3723–3732

  52. Sankaranarayanan S, Balaji Y, Castillo CD, Chellappa R (2018) Generate to adapt: Aligning domains using generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8503–8512

  53. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Proceedings of AAAI Conference on Artificial Intelligence

  54. Tsai YH, Hung WC, Schulter S, Sohn K, Yang MH, Chandraker M (2018) Learning to adapt structured output space for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7472–7481

  55. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167–7176

  56. Unar S, Wang X, Wang C, Wang M (2019) New strategy for cbir by combining low-level visual features with a colour descriptor. IET Image Process 13(7):1191–1200

    Article  Google Scholar 

  57. Unar S, Wang X, Wang C, Wang Y (2019) A decisive content based image retrieval approach for feature fusion in visual and textual images. Knowl-Based Syst 179:8–20

    Article  Google Scholar 

  58. Unar S, Wang X, Zhang C (2018) Visual and textual information fusion using kernel method for content based image retrieval. Information Fusion 44:176–187

    Article  Google Scholar 

  59. Unar S, Wang X, Zhang C, Wang C (2019) Detected text-based image retrieval approach for textual images. IET Image Process 13(3):515–521

    Article  Google Scholar 

  60. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5018–5027

  61. Vu TH, Jain H, Bucher M, Cord M, Pérez P (2019) Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517–2526

  62. Wang C, Wang X, Xia Z, Ma B, Shi YQ (2019) Image description with polar harmonic fourier moments. IEEE Transactions on Circuits and Systems for Video Technology 30(12):4440– 4452

    Article  Google Scholar 

  63. Wang C, Wang X, Xia Z, Zhang C (2019) Ternary radial harmonic fourier moments based robust stereo image zero-watermarking algorithm. Inf Sci 470:109–120

    Article  Google Scholar 

  64. Wang X, Li L, Ye W, Long M, Wang J (2019) Transferable attention for domain adaptation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5345– 5352

  65. Wang X, Wang Z (2013) A novel method for image retrieval based on structure elements’ descriptor. J Vis Commun Image Represent 24(1):63–74

    Article  Google Scholar 

  66. Wang X, Wang Z (2014) The method for image retrieval based on multi-factors correlation utilizing block truncation coding. Pattern recognition 47(10):3293–3303

    Article  Google Scholar 

  67. Wang XY, Chen ZF, Yun JJ (2012) An effective method for color image retrieval based on texture. Computer Standards & Interfaces 34(1):31–35

    Article  Google Scholar 

  68. Wu L, Hong R, Wang Y, Wang M (2019) Cross-entropy adversarial view adaptation for person re-identification IEEE Transactions on Circuits and Systems for Video Technology

  69. Xie GS, Zhang XY, Yan S, Liu CL (2015) Hybrid cnn and dictionary-based models for scene recognition and domain adaptation. IEEE Transactions on Circuits and Systems for Video Technology 27(6):1263–1274

    Article  Google Scholar 

  70. Xu X, He H, Zhang H, Xu Y, He S (2019) Unsupervised domain adaptation via importance sampling IEEE Transactions on Circuits and Systems for Video Technology

  71. Yang G, Xia H, Ding M, Ding Z (2020) Bi-directional generation for unsupervised domain adaptation. In: Proceedings of AAAI Conference on Artificial Intelligence, pp. 6615–6622

  72. Yang X, Jiang X, Tian C, Wang P, Zhou F, Fujita H (2020) Inverse projection group sparse representation for tumor classification: a low rank variation dictionary approach. Knowl-Based Syst 196:105768

    Article  Google Scholar 

  73. Yao T, Pan Y, Ngo CW, Li H, Mei T (2015) Semi-supervised domain adaptation with subspace learning for visual recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2142–2150

  74. Ye X, Li Z, Sun B, Wang Z, Xu R, Li H, Fan X (2019) Deep joint depth estimation and color correction from monocular underwater images based on unsupervised adaptation networks IEEE Transactions on Circuits and Systems for Video Technology

  75. Zellinger W, Grubinger T, Lughofer E, Natschläger T., Saminger-Platz S (2017) Central moment discrepancy (cmd) for domain-invariant representation learning. arXiv preprint arXiv:1702.08811

  76. Zhang L, Wang P, Wei W, Lu H, Shen C, van den Hengel A, Zhang Y (2018) Unsupervised domain adaptation using robust class-wise matching. IEEE Transactions on Circuits and Systems for Video Technology 29(5):1339–1349

    Article  Google Scholar 

  77. Zhang Y, Liu T, Long M, Jordan MI (2019) Bridging theory and algorithm for domain adaptation. arXiv preprint arXiv:1904.05801

  78. Zhang Y, Qiu Z, Yao T, Liu D, Mei T (2018) Fully convolutional adaptation networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6810–6818

  79. Zhang Y, Tang H, Jia K, Tan M (2019) Domain-symmetric networks for adversarial domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5031–5040

  80. Zhou F, Yang S, Fujita H, Chen D, Wen C (2020) Deep learning fault diagnosis method based on global optimization gan for unbalanced data. Knowl-Based Syst 187:104837

    Article  Google Scholar 

  81. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp. 2223–2232

Download references

Author information

Authors and Affiliations

  1. School of Instrument Science and Engineering, Harbin Institute of Technology (HIT), Harbin, China

    Guanglei Yang, Mingli Ding & Yongqiang Zhang

Authors
  1. Guanglei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingli Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongqiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingli Ding.

Ethics declarations

Conflict of Interests

We would like to note that in the manuscript entitled “Bi-Directional Class-wise Adversaries for Unsupervised Domain Adaptation”, no conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, G., Ding, M. & Zhang, Y. Bi-directional class-wise adversaries for unsupervised domain adaptation. Appl Intell 52, 3623–3639 (2022). https://doi.org/10.1007/s10489-021-02609-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02609-7

Keywords

Search

Navigation