Skip to main content
SpringerLink
Log in

Target adaptive extreme learning machine for transfer learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Extreme learning machines (ELM) have been applied in several fields due to their simplicity and computational efficiency. However, ELM hurts the performance in cross-domain learning problems similar to most machine learning algorithms. In this paper, we mainly focus on the semi-supervised transfer learning algorithm under ELM framework. Unlike other transfer learning methods employed both source and target domains, we propose a target adaptive ELM (TAELM) of learning a high-quality target-specific classifier with less resources. We formulate a novel objective function to obtain a target-specific classifier by introducing a knowledge transfer term on a pre-trained source model and a graph laplacian-based manifold regularization term on the target domain, while its solution are analytically determined without loss of the computing efficiency and learning ability of traditional ELM. In our experiments, we verify the effectiveness of the proposed approach by using a deep neural network model as feature extractor for both domains. Experimental results demonstrate that our method with less resources significantly outperforms other state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

References

  1. Chang P, Zhang J, Wang J, Fei R (2020) ELMAENet: a simple, effective and fast deep architecture for image classification. Neural Process Lett 51:129–146

    Article  Google Scholar 

  2. Chen C, Jiang B, Jin X (2018) Parameter transfer extreme learning machine based on projective model, in Proceedings of the International Joint Conference on Neural Networks (IJCNN 2018), pp. 18, IEEE, Rio de Janeiro, Brazil

  3. Chen Y, Song S, Li S, Yang L, Wu C (2019) Domain space transfer extreme learning machine for domain adaptation. IEEE Trans Cybern 49(5):1909–1922

    Article  PubMed  Google Scholar 

  4. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2013) DeCAF: a deep convolutional activation feature for generic visual recognition, arXiv: 1310.1531

  5. Eshtay M, Faris H, Obeid N (2018) Improving extreme learning machine by competitive swarm optimization and its application for medical diagnosis problems. Expert Syst Appl 104:134–152

    Article  Google Scholar 

  6. Ghoneim A, Muhammad G, Hossain MS (2020) Cervical cancer classification using convolutional neural networks and extreme learning machines. Futur Gener Comput Syst 102:643–649

    Article  Google Scholar 

  7. Huang G-B, Zhu Q-Y, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  8. Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybernetics) 42(2):513–529

    Article  Google Scholar 

  9. Huang G, Song S, Gupta JND, Wu C (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 44(12):2405–2417

    Article  PubMed  Google Scholar 

  10. Jing M-M, Zhao J-D, Li J-J, Zhu L, Yang Y, Shen H-T (2021) Adaptive component embedding for domain adaptation. IEEE Trans Cybern 51(7):3390

    Article  PubMed  Google Scholar 

  11. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  12. Li H, Yang X, Li Y, Hao L-Y, Zhang T-L (2020) Evolutionary extreme learning machine with sparse cost matrix for imbalanced learning. ISA Trans 100:198–209

    Article  PubMed  Google Scholar 

  13. Li S, Song S, Huang G, Wu C (2018) Cross-domain extreme learning machines for domain adaptation. IEEE Trans Syst Man Cybern 49(6):1194–1207

    Article  Google Scholar 

  14. Li C-L, Lin L, Zuo W-M, Tang J, Yang M-H (2019) Visual tracking via dynamic graph learning. IEEE Trans Pattern Anal Mach Intell 41(11):2770–2782

    Article  PubMed  Google Scholar 

  15. Liu O, Chen G-L, Lin H-C, Zhang W-P, Liu J-Q (2021) Prediction of IGBT junction temperature using improved cuckoo search-based extreme learning machine. Microelectron Reliab 124:114267

    Article  Google Scholar 

  16. Lu J-L, Zhou J, Chen Y-D, Pedrycz W, Hung K-W (2022a) Asymmetric Transfer Hashing with Adaptive Bipartite Graph Learning, arXiv:2206.12592v2

  17. Lu J-L, Wang H-L, Zhou J, Chen Y-D, Lai Z-H, Hu Q-H (2021) low-rank adaptive graph embedding for unsupervised feature extraction. Pattern Recogn 113:107758

    Article  Google Scholar 

  18. Lu S-Y, Zhang Z, Zhang Y-D, Wang S-H (2022) CGENet: a deep graph model for COVID-19 detection based on chest CT. Biology 11(11):33

    CAS  Google Scholar 

  19. Lu S-Y, Wang S-H, Zhang Y-D (2021) Detection of abnormal brain in MRI via improved AlexNet and ELM optimized by chaotic bat algorithm. Neural Comput Appl 33(17):10799–10811

    Article  Google Scholar 

  20. Ma Z-Y, Luo G-C, Qin K, Wang N, Niu W-N (2018) Weighted domain transfer extreme learning machine and its online version for gas sensor drift compensation in E-Nose systems. Wireless Commun Mobile Comput. https://doi.org/10.1155/2018/2308237

    Article  Google Scholar 

  21. Muhammad AK, Seifedine K, Zhang Y-D, Tallha A, Muhammad S, TanzilaSaba RA (2021) Prediction of COVID-19—Pneumonia based on selected deep features and one class kernel extreme learning machine. Comput Electr Eng 90:106960

    Article  Google Scholar 

  22. Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. NIPS Workshop Deep Learning Unsupervised Feature Learn 2011:5

    Google Scholar 

  23. Noh Y, Sugiyama M, Liu S, Plessis MCD, Park FC (2018) Bias reduction and metric learning for nearest neighbor estimation of Kullback-Leibler divergence. Neural Comput 30(7):1930–1960

    Article  MathSciNet  PubMed  Google Scholar 

  24. Zyurt FO (2020) A fused CNN model for WBC detection with MRMR feature selection and extreme learning machine. Soft Comput 24:8163–8172

    Article  Google Scholar 

  25. Pan S-J, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  26. Raghuwanshi BS, Shukla S (2020) SMOTE based class-specific extreme learning machine for imbalanced learning. Knowl Based Syst 187:229–242

    Article  Google Scholar 

  27. Rajabi A, Mousavi S, Mohamadian N, Wood DA, Ghorbani H, Davoodi S, Alvar MA, Shahbazi K (2021) Hybrid machine learning algorithms to predict condensate viscosity in the near wellbore regions of gas condensate reservoirs. J Nat Gas Sci Eng 95:104210

    Article  Google Scholar 

  28. Ri J-H, Liu L, Liu Y, Wu H-F, Huang W-L, Kim H (2018) Optimal weighted extreme learning machine for imbalanced learning with differential evolution. IEEE Comput Intell Mag 13(3):32–47

    Article  Google Scholar 

  29. Ri J-H, Tian G-Z, Liu Y, Xu W-H, Lou J-G (2020) Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning. Int J Mach Learn Cybern 11(9):2007–2020

    Article  Google Scholar 

  30. Song G, Dai Q, Han X, Guo L (2020) Two novel ELM-based stacking deep models focused on image recognition. Appl Intell 50:345–1366

    Article  Google Scholar 

  31. Tian X-H, Jiao W-L, Liu T-J, Ren L-M, Song B (2021) Leakage detection of low-pressure gas distribution pipeline system based on linear fitting and extreme learning machine. Int J Press Vessels Pip 194:104553

    Article  Google Scholar 

  32. Tommasi T, Orabona F, Caputo B (2014) Learning categories from few examples with multi model knowledge transfer. IEEE Trans Pattern Anal Mach Intell 36(5):928–941

    Article  PubMed  Google Scholar 

  33. Wu C, Li Y, Zhao Z, Liu B (2020) Extreme learning machine with autoencoding receptive fields for image classification. Neural Comput Appl 32:8157–8173

    Article  Google Scholar 

  34. Yan H, Ding Y, Li P, Wang Q, Xu Y, Zuo W (2017) Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation, in proceedings of the IEEE International conference on computer vision (CVPR), Hawaii United States. p. 945–954

  35. Zang S-F, Cheng Y-H, Wang X-S, Yan Y-Y (2021) Transfer extreme learning machine with output weight alignment. Comput Intel Neurosci. https://doi.org/10.1155/2021/6627765

    Article  Google Scholar 

  36. Zhang J, Li Y, Xiao W, Zhang Z (2020) Non-iterative and fast deep learning: multilayer extreme learning machines. J Franklin Inst 357:8925–8955

    Article  MathSciNet  Google Scholar 

  37. Zhang L, Zhang D (2015) Domain adaptation extreme learning machines for drift compensation in E-nose systems. IEEE Trans Instrum Meas 64(7):1790–1801

    Article  ADS  Google Scholar 

  38. Zhang L, He Z, Liu Y (2017) Deep object recognition across domains based on adaptive extreme learning machine. Neurocomputing 239:194–203

    Article  Google Scholar 

  39. Zhu H-H, Liu G-J, Zhou M-C, Xie Y, Abusorrah A, Kang Q (2020) Optimizing weighted extreme learning machines for imbalanced classification and application to credit card fraud detection. Neurocomputing 407:50–62

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Information Technology, Hightech Research and Development Center, Kim Il Sung University, Pyongyang, Democratic People’s Republic of Korea

    Jong Hyok Ri, Tok Gil Kang & Chol Ryong Choe

Authors
  1. Jong Hyok Ri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tok Gil Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chol Ryong Choe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jong Hyok Ri.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest and no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ri, J.H., Kang, T.G. & Choe, C.R. Target adaptive extreme learning machine for transfer learning. Int. J. Mach. Learn. & Cyber. 15, 917–927 (2024). https://doi.org/10.1007/s13042-023-01947-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01947-x

Keywords

Search

Navigation