Abstract
Human activity recognition (HAR) is a research hotspot in the field of artificial intelligence and pattern recognition. The electrooculography (EOG)-based HAR system has attracted much attention due to its good realizability and great application potential. Focusing on the signal processing method of the EOG-HAR system, we propose a robust EOG-based saccade recognition using the multi-channel convolutional independent component analysis (ICA) method. To establish frequency-domain observation vectors, short-time Fourier transform (STFT) is used to process time-domain EOG signals by applying the sliding window technique. Subsequently, we apply the joint approximative diagonalization of eigenmatrix (JADE) algorithm to separate the mixed signals and choose the “clean” saccadic source to extract features. To address the problem of permutation ambiguity in a case with a six-channel condition, we developed a constraint direction of arrival (DOA) algorithm that can automatically adjust the order of eye movement sources according to the constraint angle. Recognition experiments of four different saccadic EOG signals (i.e. up, down, left and right) were conducted in a laboratory environment. The average recognition ratios over 13 subjects were 95.66% and 97.33% under the between-subjects test and the within-subjects test, respectively. Compared with “bandpass filtering”, “wavelet denoising”, “extended infomax algorithm”, “frequency-domain JADE algorithm” and “time-domain JADE algorithm, the recognition ratios obtained relative increments of 4.6%, 3.49%, 2.85%, 2.81% and 2.91% (within-subjects test) and 4.91%, 3.43%, 2.21%, 2.24% and 2.28% (between-subjects test), respectively. The experimental results revealed that the proposed algorithm presents robust classification performance in saccadic EOG signal recognition.
Acknowledgments
The authors would like to thank the volunteers who participated in this study and anonymous reviewers for comments on the draft of this article.
Author Statement
Research funding: The research work is supported by National Natural Science Fund of China under Grant 61401002 and Anhui Provincial Natural Science Research Project of Colleges and Universities Fund under Grant KJ2018A0008.
Conflict of interest: The authors declare that they have no competing interests.
Informed consent: All the subjects have signed an informed consent for allowing the authors to open their physiological parameters, and the subjects whose photographs appear in figures have given written permission to publish their photographs.
Ethics approval: The research related to human use complied with all the relevant national regulations and institutional policies, was performed in accordance with the tenets of the Helsinki Declaration, and the experiments had received approval by the ethics committee of Anhui University.
Author Contributions: BBZ, NB, ZC and XPG mainly focused on the development and verification of the proposed method. LZ collected, organized the literature and supervised all the process. All authors read and approved the final manuscript.
References
[1] Ou Y, Lv Z, Wu X. An algorithm for reading activity recognition based on electrooculogram. In Proc Int Inf Commun Signal Proc Conf 2015;1–5.Search in Google Scholar
[2] Huda K, Hossain M, Ahmad M. Recognition of reading activity from the saccadic samples of electrooculography data. In Proc Int Elect Elect Eng Conf 2015;73–6.10.1109/CEEE.2015.7428296Search in Google Scholar
[3] Bulling A, Ward J, Gellersen H, Troster G. Eye movement analysis for activity recognition using electrooculography. IEEE Trans Pattern Anal Mach Intell 2011;33:741–53.10.1109/TPAMI.2010.86Search in Google Scholar PubMed
[4] Tapia E, Intille S, Larson K. Activity recognition in the home using simple and ubiquitous sensors. In Proc Int Pervasive Computing Conf 2004;4:158–75.10.1007/978-3-540-24646-6_10Search in Google Scholar
[5] Li F, Kolakowski S, Pelz J. Using structured illumination to enhance video-based eye tracking. In Proc Int Image Process Conf 2007;1:I-373–I-376.10.1109/ICIP.2007.4378969Search in Google Scholar
[6] Lv Z, Zhang C, Zhou B, Gao X, Wu X. Design and implementation of an eye gesture perception system based on electrooculography. Expert Syst Appl 2018;91:310–21.10.1016/j.eswa.2017.09.017Search in Google Scholar
[7] Becerra-García RA, García-Bermúdez RV, Joya-Caparrós G, Fernández-Higuera A, Velázquez-Rodríguez C, Velázquez-Mariñoa M, et al. Data mining process for identification of non-spontaneous saccadic movements in clinical electrooculography. Neurocomputing 2017;250:28–36.10.1016/j.neucom.2016.10.077Search in Google Scholar
[8] Arthi SV, Norman SR. Analysis of electrooculography signals for the interface and control of appliances. Int J Multidiscip Curr Res 2015;3:87–90.10.1007/978-81-322-2656-7_100Search in Google Scholar
[9] Vidal M, Bulling A, Gellersen H. Analysing EOG signal features for the discrimination of eye movements with wearable devices. International Workshop on Pervasive Eye Tracking & Mobile Eye-Based Interaction. ACM 2011;15–20.10.1145/2029956.2029962Search in Google Scholar
[10] Lisberger SG. Visual guidance of smooth pursuit eye movements. Annu Rev Vis Sci 2015;1:447–68.10.1146/annurev-vision-082114-035349Search in Google Scholar PubMed
[11] Keegan J, Burke E, Condron J. An electrooculogram-based binary saccade sequence classification (BSSC) technique for augmentative communication and control. In Proc IEEE Eng Med Biol Soc Conf 2009;2604–7.10.1109/IEMBS.2009.5335325Search in Google Scholar PubMed
[12] Bulling A, Ward JA, Gellersen H, Troster G. Robust recognition of reading activity in transit using wearable electrooculography. In Proc Int Pervasive Computing Conf 2008;19–37.10.1007/978-3-540-79576-6_2Search in Google Scholar
[13] Jiang P, Zhou R. De-noising and recognition of EOG signal based on mathematical morphology. In Proc Int Comput Intel Design Conf 2013;2:351–4.10.1109/ISCID.2013.201Search in Google Scholar
[14] Reddy M, Chander K, Rao K. Complex wavelet transform driven de-noising method for an EOG signals. In Proc IEEE India Conf 2011;1–5.10.1109/INDCON.2011.6139477Search in Google Scholar
[15] Bigirimana A, Siddique N, Coyle D. A hybrid ICA-wavelet transform for automated artefact removal in EEG-based emotion recognition. In Proc Int Syst Man Cyb Conf 2016;004429–34.10.1109/SMC.2016.7844928Search in Google Scholar
[16] Noori S, Mikaeili M. Detecting driving drowsiness using EEG, EOG and driving quality signals. In Proc Int Biomed Eng Conf 2015;210–5.10.1109/ICBME.2015.7404144Search in Google Scholar
[17] Zhang B, Ou Y, Zhang C, Lv Z. A research on suppressing blink signal in reading activity recognition algorithm based on EOG. J Signal Process (In Chinese) 2017;33:236–44.Search in Google Scholar
[18] Kaur C, Singh P. EEG artifact suppression based on SOBI based ICA using wavelet thresholding. In Proc Int Recent Adv Eng Computat Sci Conf 2015;1–4.10.1109/RAECS.2015.7453319Search in Google Scholar
[19] Grover P, Weldon J, Kelly S, Venkatesh P, Jeong H. An information theoretic technique for harnessing attenuation of high spatial frequencies to design ultra-high-density EEG. In Proc Int Allerton Commun Control Computing Conf 2015;901–8.10.1109/ALLERTON.2015.7447102Search in Google Scholar
[20] Nunez P, Srinivasan R. Electric fields of the brain: the neurophysics of EEG. Phys Today 1982;35:59–9.10.1063/1.2915137Search in Google Scholar
[21] Lopez A, Ferrero F, Valledor M, Campo JC, Postolache O. A study on electrode placement in EOG systems for medical applications. In Proc Int Med Meas Appl Conf 2016;1–5.10.1109/MeMeA.2016.7533703Search in Google Scholar
[22] Tamura H, Miyashita M, Tanno K, Fuse Y. Mouse cursor control system using electrooculogram signals. In Proc Int World Automat Congress Conf 2010;1–6.Search in Google Scholar
[23] Hagemann D, Naumann E. The effects of ocular artifacts on (lateralized) broadband power in the EEG. Clin Neurophysiol 2001;112:215–31.10.1016/S1388-2457(00)00541-1Search in Google Scholar PubMed
[24] Turnip A. JADE-ICA algorithm for EOG artifact removal in EEG recording. In Proc Int Technol Informatics Conf 2014;270–4.10.1109/TIME-E.2014.7011630Search in Google Scholar
[25] Turnip A, Soetraprawata D, Kusumandari D, Siahaan M, Setiawan IR, Saepulloh S, et al. Extraction of mental task in recorded EEG signal using ICA-JADE algorithm. In Proc Int Technol Informatics Conf 2014;275–80.10.1109/TIME-E.2014.7011631Search in Google Scholar
[26] Hu J, Fan LH. Application of JADE to separate complex-valued sources. In Proc Int Comput Sci Service Syst Conf 2014;275–80.Search in Google Scholar
[27] Moreau E. An any order generalization of JADE for complex source signals. In Proc Int Acoust Speech Signal Process Conf 2001;5:2805–8.10.1109/ICASSP.2001.940229Search in Google Scholar
[28] Mazur R, Mertins A. A sparsity based criterion for solving the permutation ambiguity in convolutive blind source separation. In Proc Int Acoust Speech Signal Process Conf 2011;1996–9.10.1109/ICASSP.2011.5946902Search in Google Scholar
[29] Mazur R, Jungmann J, Mertins A. A new clustering approach for solving the permutation problem in convolutive blind source separation. In Proc Int Applications of Signal Processing to Audio and Acoustics Conf 2013;1–4.10.1109/WASPAA.2013.6701852Search in Google Scholar
[30] Lima PFC, Miranda RK, Costa JPCLD, Zelenovsky R, Yuan Y, Del Galdo G. Low complexity blind separation technique to solve the permutation ambiguity of convolutive speech mixtures. In Proc Int Conf Signal Process Commun Syst IEEE 2017;1–10.10.1109/ICSPCS.2016.7843325Search in Google Scholar
[31] Nikunen J, Virtanen T, Pertilä P, Vilermo M. Permutation alignment of frequency-domain ICA by the maximization of intra-source envelope correlations. In Proc Int Signal Process Conf 2012;1489–93.Search in Google Scholar
[32] Ikram M, Morgan D. A beamforming approach to permutation alignment for multichannel frequency-domain blind speech separation. In Proc Int Acoust Speech Signal Process Conf 2002;1:I-881–I-884.10.1109/ICASSP.2002.1005881Search in Google Scholar
[33] Mazur R, Jungmann JO, Mertins A. A new clustering approach for solving the permutation problem in convolutive blind source separation. In Proc Int Appl Signal Process Conf 2013;1–4.10.1109/WASPAA.2013.6701852Search in Google Scholar
[34] Brandstein M, Adcock J, Silverman H. A closed-form location estimator for use with room environment microphone arrays. IEEE Trans Speech Audio Process 1997;5:45–50.10.1109/89.554268Search in Google Scholar
[35] Jiang H, Marot J, Fossati C, Bourennane S. Fuzzy contour characterization by subspace based methods of array processing and direct method. In Proc Int Signal Process Conf 2010;1344–8.10.1109/SAM.2010.5606727Search in Google Scholar
[36] Alibi D, Javed U, Wen F, He D, Liu P, Zhang Y, et al. 2D DOA estimation method based on channel state information for uniform circular array. In Proc Int Ubiquit Positioning, Indoor Navigation and Location Based Services Conf 2016;68–72.10.1109/UPINLBS.2016.7809952Search in Google Scholar
[37] Lv C, Wang C, Cao X, Xie C, Zhuang H. Sensor uncertainty analysis of MIMO sonar. In Proc Int Ocean Conf 2016;1–5.10.1109/OCEANSAP.2016.7485726Search in Google Scholar
[38] Araki S, Sawada H, Mukai R, Makino S. DOA estimation for multiple sparse sources with normalized observation vector clustering. In Proc Int Acoust Speech Signal Process Conf 2006;5:V–V.10.1109/ICASSP.2006.1661205Search in Google Scholar
[39] Athi M, Zekavat S. Real-time root-MUSIC DOA estimation via a parallel polynomial rooting method. Int Proc Int Pers Indoor Mob Radio Commun Conf 2014;290–5.10.1109/PIMRC.2014.7136177Search in Google Scholar
[40] Suleiman W, Pesavento M, Zoubir A. Performance analysis of direction-of-arrival estimation using the decentralized root-MUSIC. In Proc Int Comput Adv Multi Sensor Adapt Process Conf 2015;329–32.10.1109/CAMSAP.2015.7383803Search in Google Scholar
[41] Hwang H K, Aliyazicioglu Z, Grice M, Yakovlev A. Direction of arrival estimation using a Root-MUSIC Algorithm. Lect Notes Comput 2008;2169:353–60.Search in Google Scholar
[42] Banerjee A, Pal M, Tibarewala D, Konar A. Electrooculogram based blink detection to limit the risk of eye dystonia. In Proc Int Adv Pattern Recogn Conf 2015;1–6.10.1109/ICAPR.2015.7050712Search in Google Scholar
[43] Peng Y, Lu BL. Discriminative extreme learning machine with supervised sparsity preserving for image classification. Neurocomputing 2017;261:242–52.10.1016/j.neucom.2016.05.113Search in Google Scholar
[44] Vapnik V. The nature of statistical learning theory. Berlin/Heidelberg, Germany: Springer Science and Business Media, 2013.Search in Google Scholar
[45] Chavhan Y, Yelure B, Tayade K. Speech emotion recognition using RBF kernel of LIBSVM. In Proc Int Elect Commun Syst Conf 2015;1132–5.10.1109/ECS.2015.7124760Search in Google Scholar
[46] Saurabh B, Kumar Y, Patidar S, Vasantha MH. Design and implementation of tunable bandpass filter for biomedical applications. In Proc Int Sym Nanoelectronic Inform Syst Conf 2016;43–6.Search in Google Scholar
[47] Srivastava M, Anderson C, Freed J. A new wavelet denoising method for selecting decomposition levels and noise thresholds. IEEE Access 2016;4:3862–77.10.1109/ACCESS.2016.2587581Search in Google Scholar PubMed PubMed Central
[48] Kaczorowska M, Plechawska-Wojcik M, Tokovarov M, Dmytruk R. Comparison of the ICA and PCA methods in correction of EEG signal artefacts. In Proc Int Adv Top Elect Eng Conf 2017;262–7.10.1109/ATEE.2017.7905095Search in Google Scholar
[49] Chen Z, Chan L. New approaches for solving permutation indeterminacy and scaling ambiguity in frequency domain separation of convolved mixtures. In Proc Int Neural Networks Conf 2011;911–8.10.1109/IJCNN.2011.6033319Search in Google Scholar
[50] Congedo M, Gouy-Pailler C, Jutten C. On the blind source separation of human electroencephalogram by approximate joint diagonalization of second order statistics. Clin Neurophysiol 2017;119:2677–86.10.1016/j.clinph.2008.09.007Search in Google Scholar PubMed
[51] Barthélemy Q, Mayaud L, Renard Y, Kim D, Kang SW, Gunkelman J, et al. Online denoising of eye-blinks in electroencephalography. Neurophysiol Clin 2017;47:371–91.10.1016/j.neucli.2017.10.059Search in Google Scholar PubMed
©2019 Walter de Gruyter GmbH, Berlin/Boston
