Skip to main content
SpringerLink
Log in

A novel approach of decoding four-class motor imagery tasks via wavelet transform and 1DCNN-BiLSTM

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Electroencephalogram (EEG)-based human-computer interaction (HCI) has become a major research direction in the field of brain-computer interface (BCI). Although EEG research has made progress, motor imagery (MI) EEG decoding remains a challenge due to a lack of sample data, a lower signal noise ratio (SNR), and individual differences. Recently, many deep learning methods have been widely used in EEG classification tasks. Our work presents a novel approach to decoding four-class MI tasks by utilizing one-dimensional convolutional neural network (1DCNN). We use 1D multiscale CNN (1DMCNN) block, residual attention mechanism (RAM) block and bidirectional long-short-term memory (BiLSTM) networks for EEG decoding. We named it the 1DMRCNN-BiLSTM model, which can achieve good accuracy in decoding human intentions. The highlights include: (1) Based on the wavelet transform (WT), we use 1D wavelet denoising and 1D wavelet reconstruction methods not only to improve the SNR of EEG signals but also to enhance the number of EEG samples. (2) We fused the 1DMCNN block and the RAM block with dropout layers to design a new 1DMRCNN model for EEG feature extraction. (3) Based on the 1DMRCNN-BiLSTM structure, an effective end-to-end framework for MI classification is built. We trained and tested our proposed method on the BCI competition IV datasets 2a (BCICID-2a) and PhysioNet. The experimental results show that the method demonstrates excellent ability in EEG decoding.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7

Similar content being viewed by others

Data availability statement

The data used to support the findings of this study have been deposited in the repository ([http://bnci-horizon-2020.eu/database/data-sets]) and ([https://physionet.org/static/published-projects/eegmmidb/eeg-motor-movementimagery-dataset-1.0.0.zip]).

References

  1. Saibene A, Caglioni M, Corchs S et al (2023) EEG-based BCIs on motor imagery paradigm using wearable technologies: a systematic review. Sensors 23(5):2798. pp 221-237

  2. Wang K, Zhai DH, Xiong Y et al (2021) An MVMD-CCA recognition algorithm in SSVEP-based BCI and its application in robot control. IEEE Trans Neural Netw Learn Syst 33(5):2159–2167

    Article  Google Scholar 

  3. Khademi S, Neghabi M, Farahi M et al (2022) A comprehensive review of the movement imaginary brain-computer interface methods: challenges and future directions. Artif Intell Based Brain Comput Interface 23–74

  4. Wu Y (2023) Applications of EEG-based brain-computer interface devices in rehabilitation. Highlights Sci Eng Technol 39:809–815

    Article  Google Scholar 

  5. Tang X, Yang C, Sun X et al (2023) Motor imagery EEG decoding based on multi-scale hybrid networks and feature enhancement. IEEE Trans Neural Syst Rehabil Eng 31:1208–1218

    Article  Google Scholar 

  6. Pillette L, Roc A, N’kaoua B et al (2021) Experimenters’ influence on mental-imagery based brain-computer interface user training. Int J Human-Comput Stud 149:102603

  7. Alzahrani SI, Alsaleh MM (2023) The influence of smoothing filtering methods on the performance of an EEG-based brain-computer interface. IEEE Access

  8. Aggarwal S, Chugh N (2019) Signal processing techniques for motor imagery brain computer interface: a review. Array 1:100003

    Article  Google Scholar 

  9. Zhuang J, Geng K, Yin G (2019) Ensemble learning based brain-computer interface system for ground vehicle control. IEEE Trans Syst Man Cybern Syst 51(9):5392–5404

    Article  Google Scholar 

  10. Xie L, Deng Z, Xu P et al (2018) Generalized hidden-mapping transductive transfer learning for recognition of epileptic electroencephalogram signals. IEEE Trans Cybern 49(6):2200–2214

    Article  Google Scholar 

  11. Zhang T, Zheng W, Cui Z, Li Y (2019) Spatial-temporal recurrent neural network for emotion recognition. IEEE Trans Cybern 49(3):839–847

    Article  Google Scholar 

  12. Velasco-álvarez F, Fernández-Rodríguez á (2022) Brain-computer interface (BCI)-generated speech to control domotic devices. Neurocomputing 509:121–136

    Article  Google Scholar 

  13. Padfield N, Zabalza J, Zhao H et al (2019) EEG-based brain-computer interfaces using motor-imagery: techniques and challenges. Sensors 19(6):1423

    Article  Google Scholar 

  14. Noureddin B, Lawrence P, Birch G (2012) Online removal of eye movement and blink EEG artifacts using a high-speed eye tracker. IEEE Trans Biomed Eng 59(8):2103–2110

    Article  Google Scholar 

  15. Grobbelaar M, Phadikar S et al (2022) A survey on denoising techniques of electroencephalogram signals using wavelet transform. Signals 3(3):577–586

    Article  Google Scholar 

  16. Li X, Zhang Y, Tiwari P et al (2022) EEG based emotion recognition: a tutorial and review. ACM Comput Surv 55(4):1–57

    Article  Google Scholar 

  17. Feng JK, Jin J, Daly I et al (2019) An optimized channel selection method based on multifrequency CSP-rank for motor imagery-based BCI system. Comput Intell Neurosci

  18. Mammone N, Ieracitano C, Adeli H et al (2023) AutoEncoder filter bank common spatial patterns to decode motor imagery from EEG. IEEE J Biomed Health Inform

  19. Khademi Z, Ebrahimi F, Kordy HM (2022) A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals. Comput Biol Med 143:105288

    Article  Google Scholar 

  20. Tuncer T, Dogan S, Naik GR et al (2021) Epilepsy attacks recognition based on 1D octal pattern, wavelet transform and EEG signals. Multimed Tools Appl 80:25197–25218

    Article  Google Scholar 

  21. Swain CMK, Singh A, Raghu I (2023) Electroencephalogram (EEG) signal denoising using optimized wavelet transform (WT) a study. In: Computational intelligence in medical decision making and diagnosis, CRC Press, pp 169–182

  22. Jiao Z, Gao X, Wang Y et al (2018) Deep convolutional neural networks for mental load classification based on EEG data. Pattern Recogn 76:582–595

    Article  Google Scholar 

  23. Schirrmeister RT, Springenberg JT, Fiederer LDJ et al (2017) Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp 38(11):5391–5420

    Article  Google Scholar 

  24. Bang JS, Lee MH, Fazli S et al (2021) Spatio-spectral feature representation for motor imagery classification using convolutional neural networks. IEEE Trans Neural Netw Learn Syst 33(7):3038–3049

    Article  Google Scholar 

  25. Zhang R, Li Y, Yan Y et al (2015) Control of a wheelchair in an indoor environment based on a brain-computer interface and automated navigation. IEEE Trans Neural Syst Rehabil Eng 24(1):128–139

    Article  Google Scholar 

  26. Hou Y, Zhou L, Jia S et al (2020) A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng 17(1):016048

    Article  Google Scholar 

  27. Xu G, Ren T, Chen Y et al (2020) A one-dimensional cnn-lstm model for epileptic seizure recognition using eeg signal analysis. Front Neurosci 14:578126

    Article  Google Scholar 

  28. Li H, Ding M, Zhang R et al (2022) Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network. Biomed Signal Process Control 72:103342

    Article  Google Scholar 

  29. Ma W, Gong Y, Xue H et al (2022) A lightweight and accurate double-branch neural network for four-class motor imagery classification. Biomed Signal Process Control 75:103582

    Article  Google Scholar 

  30. Ma W, Xue H, Sun X et al (2022) A novel multi-branch hybrid neural network for motor imagery EEG signal classification. Biomed Signal Process Control 77:103718

    Article  Google Scholar 

  31. Wang X, Gong G, Li N (2019) Automated recognition of epileptic EEG states using a combination of symlet wavelet processing, gradient boosting machine, and grid search optimizer. Sensors 19(2):219

    Article  MathSciNet  Google Scholar 

  32. Lopez KL, Monachino AD, Morales S et al (2022) HAPPILEE: HAPPE in low electrode electroencephalography, a standardized pre-processing software for lower density recordings. NeuroImage 260:119390

    Article  Google Scholar 

  33. Boubchir L, Boashash B (2013) Wavelet denoising based on the MAP estimation using the BKF prior with application to images and EEG signals. IEEE Trans Signal Process 61(8):1880–1894

    Article  MathSciNet  MATH  Google Scholar 

  34. Ivaylov I, Lazarova M, Manolova A (2020) EEG classification for motor imagery mental tasks using wavelet signal denoising. 2020 28th National conference with international participation (TELECOM). IEEE, pp 53–56

  35. Qu W, Wang Z, Hong H et al (2020) A residual based attention model for eeg based sleep staging. IEEE J Biomed Health Inform 24(10):2833–2843

    Article  Google Scholar 

  36. Chu C, Xiao Q, Zhang Y et al (2022) Multi-modal fusion sign language recognition based on residual network and attention mechanism. Int J Pattern Recognit Artif Intell 36(12):2250036

    Article  Google Scholar 

  37. Ang K, Chin Z, Wang C, Guan C, Zhang H (2012) Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Frontiers Neurosci 6:39

    Article  Google Scholar 

  38. Zhao X, Liu D, Ma L et al (2022) Deep CNN model based on serial-parallel structure optimization for four-class motor imagery EEG classification. Biomed Signal Process Control 72:103338

    Article  Google Scholar 

  39. Lun X, Yu Z, Chen T et al (2020) A simplified CNN classification method for MI-EEG via the electrode pairs signals. Front Hum Neurosci 14:338

    Article  Google Scholar 

  40. Wang J, Cheng S, Tian J et al (2023) A 2D CNN-LSTM hybrid algorithm using time series segments of EEG data for motor imagery classification. Biomed Signal Process Control 83:104627

    Article  Google Scholar 

  41. Jeong JH, Shim KH, Kim DJ et al (2020) Brain-controlled robotic arm system based on multi-directional CNN-BiLSTM network using EEG signals. IEEE Trans Neural Syst Rehabil Eng 28(5):1226–1238

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Nature Science Foundation of China (Nos. 61671362 and 62071366).

Author information

Authors and Affiliations

  1. Department of Mechanical and Electrical Engineering, Xi’an Technological University, Xi’an, Shaanxi, 710021, People’s Republic of China

    Chaoqin Chu & Qinkun Xiao

  2. Department of Electrical and Information Engineering, Xi’an Technological University, Xi’an, Shaanxi, 710021, People’s Republic of China

    Qinkun Xiao, Jianing Shen, Leran Chang, Na Zhang, Yu Du & Hui Gao

Authors
  1. Chaoqin Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinkun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leran Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Na Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hui Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Qinkun Xiao or Hui Gao.

Ethics declarations

Competing Interest

The authors report no declarations of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chu, C., Xiao, Q., Shen, J. et al. A novel approach of decoding four-class motor imagery tasks via wavelet transform and 1DCNN-BiLSTM. Multimed Tools Appl 82, 45789–45809 (2023). https://doi.org/10.1007/s11042-023-17396-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17396-1

Keywords

Search

Navigation