ABSTRACT
Human Activity Recognition (HAR) is a relevant inference task in many mobile applications. State-of-the-art HAR at the edge is typically achieved with lightweight machine learning models such as decision trees and Random Forests (RFs), whereas deep learning is less common due to its high computational complexity. In this work, we propose a novel implementation of HAR based on deep neural networks, and precisely on Binary Neural Networks (BNNs), targeting low-power general purpose processors with a RISC-V instruction set. BNNs yield very small memory footprints and low inference complexity, thanks to the replacement of arithmetic operations with bit-wise ones. However, existing BNN implementations on general purpose processors impose constraints tailored to complex computer vision tasks, which result in over-parametrized models for simpler problems like HAR. Therefore, we also introduce a new BNN inference library, which targets ultra-compact models explicitly. With experiments on a single-core RISC-V processor, we show that BNNs trained on two HAR datasets obtain higher classification accuracy compared to a state-of-the-art baseline based on RFs. Furthermore, our BNN reaches the same accuracy of a RF with either less memory (up to 91%) or more energy-efficiency (up to 70%), depending on the complexity of the features extracted by the RF.
- Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge L Reyes-Ortiz. 2012. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In International workshop on ambient assisted living. Springer, 216--223.Google ScholarDigital Library
- Ferhat Attal, Samer Mohammed, Mariam Dedabrishvili, Faicel Chamroukhi, Latifa Oukhellou, and Yacine Amirat. 2015. Physical human activity recognition using wearable sensors. Sensors 15, 12 (2015), 31314--31338.Google ScholarCross Ref
- Serkan Balli, Ensar Arif Sağbaş, and Musa Peker. 2019. Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm. Measurement and Control 52, 1--2 (2019), 37--45.Google ScholarCross Ref
- Akram Bayat, Marc Pomplun, and Duc A Tran. 2014. A study on human activity recognition using accelerometer data from smartphones. Procedia Computer Science 34 (2014), 450--457.Google ScholarCross Ref
- Valentina Bianchi, Marco Bassoli, Gianfranco Lombardo, Paolo Fornacciari, Monica Mordonini, and Ilaria De Munari. 2019. IoT wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment. IEEE Internet of Things Journal 6, 5 (2019), 8553--8562.Google ScholarCross Ref
- G. Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal of Software Tools (2000).Google Scholar
- Jiasi Chen and Xukan Ran. 2019. Deep learning with edge computing: A review. Proc. IEEE 107, 8 (2019), 1655--1674.Google ScholarCross Ref
- Francesco Conti, Pasquale Davide Schiavone, and Luca Benini. 2018. XNOR Neural Engine: A Hardware Accelerator IP for 21.6-fJ/op Binary Neural Network Inference. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37, 11 (2018), 2940--2951. Google ScholarCross Ref
- Francesco Daghero, Daniele Jahier Pagliari, and Massimo Poncino. 2020. Energy-efficient deep learning inference on edge devices. (2020).Google Scholar
- Lin Fan, Zhongmin Wang, and Hai Wang. 2013. Human activity recognition model based on decision tree. In 2013 International Conference on Advanced Cloud and Big Data. IEEE, 64--68.Google ScholarDigital Library
- Eric Flamand, Davide Rossi, Francesco Conti, Igor Loi, Antonio Pullini, Florent Rotenberg, and Luca Benini. 2018. GAP-8: A RISC-V SoC for AI at the Edge of the IoT. In 2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP). IEEE, 1--4.Google ScholarCross Ref
- Francesco Daghero. 2021. Ultra-compact BNNs. https://github.com/francescodaghero/ultracompactBNNGoogle Scholar
- Angelo Garofalo, Manuele Rusci, Francesco Conti, Davide Rossi, and Luca Benini. 2020. PULP-NN: accelerating quantized neural networks on parallel ultra-low-power RISC-V processors. Philosophical Transactions of the Royal Society A 378, 2164 (2020), 20190155.Google Scholar
- Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan. 2015. Deep Learning with Limited Numerical Precision. CoRR abs/1502.0 (2015). http://arxiv.org/abs/1502.02551Google Scholar
- Nils Y Hammerla, Shane Halloran, and Thomas Plötz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. 1533--1540.Google ScholarDigital Library
- Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized neural networks. In Advances in Neural Information Processing Systems. 4114--4122. arXiv:1602.02505Google Scholar
- Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2017. Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations. J. Mach. Learn. Res. 18, 1 (jan 2017), 6869--6898.Google ScholarDigital Library
- Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.Google ScholarCross Ref
- Daniele Jahier Pagliari, Roberta Chiaro, Enrico Macii, and Massimo Poncino. 2020. CRIME: Input-Dependent Collaborative Inference for Recurrent Neural Networks. IEEE Trans. Comput. (2020), 1--1. Google ScholarCross Ref
- Daniele Jahier Pagliari, Enrico Macii, and Massimo Poncino. 2018. Dynamic Bit-width Reconfiguration for Energy-Efficient Deep Learning Hardware. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED '18). ACM, New York, NY, USA, 47:1--47:6. Google ScholarDigital Library
- Wei Mao, Zhihua Xiao, Peng Xu, Hongwei Ren, Dingbang Liu, Shirui Zhao, Fengwei An, and Hao Yu. 2020. Energy-Efficient Machine Learning Accelerator for Binary Neural Networks. In Proceedings of the 2020 on Great Lakes Symposium on VLSI (Virtual Event, China) (GLSVLSI '20). Association for Computing Machinery, New York, NY, USA, 77--82. Google ScholarDigital Library
- Daniela Micucci, Marco Mobilio, and Paolo Napoletano. 2017. Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Applied Sciences 7, 10 (2017), 1101.Google ScholarCross Ref
- Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.Google ScholarCross Ref
- Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. 2016. Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision. Springer, 525--542.Google ScholarCross Ref
- Davide Rossi, Francesco Conti, Andrea Marongiu, Antonio Pullini, Igor Loi, Michael Gautschi, Giuseppe Tagliavini, Alessandro Capotondi, Philippe Flatresse, and Luca Benini. 2015. PULP: A parallel ultra low power platform for next generation IoT applications. In 2015 IEEE Hot Chips 27 Symposium (HCS). IEEE, 1--39.Google ScholarCross Ref
- Pasquale Davide Schiavone, Davide Rossi, Antonio Pullini, Alfio Di Mauro, Francesco Conti, and Luca Benini. 2018. Quentin: An ultra-low-power PULPissimo SoC in 22nm FDX. In 2018 IEEE SOI-3D-Subthreshold Microelectronics Technology Unified Conference (S3S). IEEE, 1--3.Google Scholar
- Semidynamics. 2021. Avispado Core. https://www.semidynamics.com/amp/products/avispadoGoogle Scholar
- Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge computing: Vision and challenges. IEEE internet of things journal 3, 5 (2016), 637--646.Google Scholar
- SiFive. 2021. SiFive Core IP. https://www.sifive.com/risc-v-core-ipGoogle Scholar
- STMicroelectronics. 2019. iNEMO inertial module: always-on 3D accelerometer and 3D gyroscope. Website. www.st.com/resource/en/datasheet/lsm6dsox.pdf.Google Scholar
- Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. 2017. Efficient processing of deep neural networks: A tutorial and survey. Proc. IEEE 105, 12 (2017), 2295--2329.Google ScholarCross Ref
- The PULP Platform. 2020. GVSOC: PULP Virtual Platform. https://github.com/pulp-platform/gvsocGoogle Scholar
- Haojin Yang, Martin Fritzsche, Christian Bartz, and Christoph Meinel. 2017. Bmxnet: An open-source binary neural network implementation based on mxnet. In Proceedings of the 25th ACM international conference on Multimedia. 1209--1212.Google ScholarDigital Library
- Jianhao Zhang, Yingwei Pan, Ting Yao, He Zhao, and Tao Mei. 2019. dabnn: A super fast inference framework for binary neural networks on arm devices. In Proceedings of the 27th ACM International Conference on Multimedia. 2272--2275.Google ScholarDigital Library
Index Terms
- Ultra-compact binary neural networks for human activity recognition on RISC-V processors
Recommendations
Self-Gated Recurrent Neural Networks for Human Activity Recognition on Wearable Devices
Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017This paper develops a self-gated recurrent neural network (SGRNN), and applies it to human activity recognition (HAR), using time-series signals collected from embedded sensors of wearable devices. Recurrent neural networks (RNNs) are very powerful for ...
BP Neural Network for Human Activity Recognition in Smart Home
CSSS '12: Proceedings of the 2012 International Conference on Computer Science and Service SystemIn this paper, BP (back propagation) neural network is applied to represent and recognize human activities from observed sensor sequences. The proper features for activity recognition are selected. Then, the model of BP neural network for activity ...
Benchmarking Reservoir and Recurrent Neural Networks for Human State and Activity Recognition
Advances in Computational IntelligenceAbstractMonitoring of human states from streams of sensor data is an appealing applicative area for Recurrent Neural Network (RNN) models. In such a scenario, Echo State Network (ESN) models from the Reservoir Computing paradigm can represent good ...
Comments