计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (4): 291-298.doi: 10.11896/jsjkx.230300158

• 人工智能 • 上一篇    下一篇

基于可穿戴设备的开放集动作识别技术研究

王佳昊1, 闫航1, 胡鑫1, 赵德鑫2   

  1. 1 电子科技大学信息与软件工程学院 成都610051
    2 军事科学院国防科技创新研究院 北京100071
  • 收稿日期:2023-03-20 修回日期:2023-06-30 出版日期:2024-04-15 发布日期:2024-04-10
  • 作者简介:(wangjh@uestc.edu.cn)
  • 基金资助:
    电子科技大学-智小金-智能家居联合研究中心项目(H04W210180);内江市科技孵化和成果转化专项资金(2021KJFH004);四川省科技支撑计划项目(2022YFG0212,2021YFG0024)

Study on Open Set Activity Recognition Technology Based on Wearable Devices

WANG Jiahao1, YAN Hang1, HU Xin1, ZHAO Dexin2   

  1. 1 School of Information and Software Engineering,University of Electronic Science and Technology,Chengdu 610051,China
    2 National Innovation Institute of Defense Technology,Academy of Military Sciences,Beijing 100071,China
  • Received:2023-03-20 Revised:2023-06-30 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    UESTC-ZHIXIAOJING Joint Research Center of Smart Home(H04W210180),Neijiang Technology Incubation and Transformation Funds(2021KJFH004) and Science and Technology Program of Sichuan Province,China(2022YFG0212,2021YFG0024).

PDF (PC)

48

摘要： 随着智能手表、手环等可穿戴设备的普及,将其用于人体行为识别领域并从中解码出人类行为活动,对于健康监测、日常行为分析、智能家居等应用具有重要意义。然而,传统的动作识别算法存在特征提取困难、识别准确率较低等问题,并且均基于封闭集假设,即所有的训练数据和测试数据均来自同一个标签空间,而现实世界中大多都是开放集(Open-Set)场景,在测试阶段可能会将未知标签样本送入模型,从而导致分类错误。文中针对人体动作识别问题,提出了多通道自适应卷积网络(Multi-channel Adaptive Convolutional Network,MCACN),针对传统CNN网络特征提取仅局限于一个小范围内的问题,自适应卷积模块能够使用不同大小的卷积核提取不同时间跨度的特征,并自动计算权重求和。此外MCACN的多通道结构使各传感器数据得以分头进行处理,获得能够区分相近动作的特征细节。最后,设计了基于标签的多元变分自编码器,提出了用于开放集识别的模型MCACN-VAE。该模型能够通过计算重建误差来识别未知类,聚焦于已知类别动作,提高了模型的健壮性。实验结果表明,在封闭集实验中,MCACN模型能够有效地对动作进行识别,对7种日常动作的识别准确率均达到了91%以上,总体准确率达到了95%。在开放集实验中,MCACN-VAE在不同开放度下对于已知类别的总体识别准确率均达到了89%以上,对于未知动作片段的识别准确率也保持在75%以上,证明了所提模型能够有效拒绝未知类,识别已知类。

关键词: 可穿戴设备, 动作识别, 自适应卷积, 开放集识别

Abstract: With the popularity of wearable devices such as smart watches and bracelets,using them for human activity recognition and decoding human behavior is of great significance for health monitoring,daily behavior analysis,smart home and other applications.However,traditional action recognition algorithms have problems such as difficult feature extraction and low recognition accuracy,and are all based on the close set assumption,that is,all training data and test data come from the same label space,while most of the real world is open.In the open-set scene,unknown label samples may be sent to the model during the test phase,resulting in incorrect classification.This paper proposes a multi-channel adaptive convolutional network(MCACN) for human acti-vity recognition.For the problem that the traditional CNN network feature extraction is limited to a small range,the adaptive convolution module can use convolution kernels of different sizes to extract features of different time spans,automatically calculate the weights and sum them up.In addition,the multi-channel structure of MCACN enables each sensor data to be processed separately to obtain feature details that can distinguish similar actions.Finally,this paper designs a label-based multivariate variational autoencoder,and proposes MCACN-VAE for open set recognition.The model can identify unknown classes by calculating recons-truction loss,focusing on known class actions,and improving the robustness of the model.Experimental results show that in the closed set experiment,the MCACN model can effectively recognize the actions,and the accuracy of the recognition of seven daily actions has reached more than 91%,the overall accuracy has reached 95%.In the open set experiment,the overall recognition accuracy of MCACN-VAE for known categories has reached more than 89% at different degrees of openness,and the recognition accuracy of unknown action segments has also remained above 75%.It proves that the proposed model can effectively reject unknown classes and identify known classes.

Key words: Wearable devices, Activity recognition, Adaptive convolution, Open set recognition

中图分类号: 

  • TP181
[1]SINGH S P,SHARMA M K,LAY-EKUAKILLE A,et al.Deep ConvLSTM with self-attention for human activity decoding using wearable sensors[J].IEEE Sensors Journal,2020,21(6):8575-8582.
[2]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[3]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrent neural network based language model[C]//Interspeech.2010:1045-1048.
[4]KINGMA D P,WELLING M.Auto-encoding variational bayes[C]//Proceedings of the International Conference on Learning Representations(ICLR).Banff,Canada:ICLR,2014.
[5]PARKKA J,ERMES M,KORPIPAA P,et al.Activity classification using realistic data from wearable sensors[J].IEEE Transactions on Information Technology in Biomedicine,2006,10(1):119-128.
[6]KWAPISZ J R,WEISS G M,MOORE S A.Activity recognition using cell phone accelerometers[J].ACM SigKDD Explorations Newsletter,2011,12(2):74-82.
[7]LI J.Research on Motion Capture Algorithm Based on 3D Static Model [D].Beijing:North China University of Technology,2019.
[8]MOTTAGHI A,SORYANI M,SEIFI H.Action recognition in freestyle wrestling using silhouette-skeleton features[J].Engineering Science and Technology,an International Journal,2020,23(4):921-930.
[9]LAPTEV I,MARSZALEK M,SCHMID C,et al.Learning rea-listic human actions from movies[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-8.
[10]MAHBUB U,IMTIAZ H,AHAD M A R.An optical flowbased approach for action recognition[C]//14th International Conference on Computerand Information Technology(ICCIT 2011).IEEE,2011:646-651.
[11]LIU Z,ZHANG C,TIAN Y.3D-based deep convolutional neuralnetwork for action recognition with depth sequences[J].Image and Vision Computing,2016,55:93-100.
[12]DU Y,WANG W,WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:1110-1118.
[13]XIA K,HUANG J,WANG H.LSTM-CNN architecture for human activity recognition[J].IEEE Access,2020,8:56855-56866.
[14]SENYUREK V Y,IMTIAZ M H,BELSARE P,et al.A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors[J].Biomedical Engineering Letters,2020,10(2):195-203.
[15]ORDÓÑEZ F J,ROGGEN D.Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition[J].Sensors,2016,16(1):115.
[16]AGAC S,SHOAIB M,INCEL O D.Context-aware and dynamically adaptable activity recognition with smart watches:A case study on smoking[J].Computers & Electrical Engineering,2021,90:106949.
[17]PHILLIPS P J,GROTHER P,MICHEALS R.Evaluation me-thods in face recognition[M]//Handbook of Face Recognition.2011:551-574.
[18]SCHEIRER W J,DE REZENDE ROCHA A,SAPKOTA A,et al.Toward open set recognition[J].IEEE Transactionson Pattern Analysis and Machine Intelligence,2012,35(7):1757-1772.
[19]DONG H,FU Y,SIGAL L,et al.Learning to separate domains in generalized zero-shot and open set learning:a probabilistic perspective[J].arXiv.1810.07368,2018.
[20]NEIRA M A C,JÚNIOR P R M,ROCHA A,et al.Data-fusion techniques for open-set recognition problems[J].IEEE Access,2018,6:21242-21265.
[21]YANG H M,ZHANG X Y,YIN F,et al.Convolutional prototype network for open set recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,44(5):2358-2370.
[22]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[23]BENDALE A,BOULT T E.Towards open set deep networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1563-1572.
[24]OZA P,PATEL V M.C2ae:Class conditioned auto-encoder for open-set recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:2307-2316.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发