Skip to main content
SpringerLink
Log in

Mixed-dependency models for multi-resident activity recognition in smart homes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recent growing interest in ambient intelligent environments has driven a desire for effective models to reason about activities of multiple residents. Such models are the keystone for the future of smart homes where occupants can be assisted with non-intrusive technologies. Much attention has been put on this research, however current works tend to focus on developing statistical algorithms for prediction, whilst there still lacks a study to fully understand the relations of residents’ behaviours and how they are reflected through the sensors’ states. In this paper we investigate the dependencies of the activities from residents and their interaction with the environments. We represent such dependencies in Bayesian networks that leads to construction of six variants of Hidden Markov Models (HMMs). Furthermore, we argue that a complete model should embody more than one type of dependency. Therefore, we propose an ensemble of HMMs, and then generalize it to a novel mixed-dependency model. In the experiments we perform intensive evaluation of our study on multi-resident activity recognition task. The results show that the proposed models outperform other models in three smart home environments, thus asserting our hypothesis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Here p denotes p(a1:T,o1:T) to save the presentation space.

  2. http://ailab.eecs.wsu.edu/casas/

  3. http://www.cmpe.boun.edu.tr/aras/

References

  1. Alemdar H, Ertan H, Incel OD, Ersoy C (2013) Aras human activity datasets in multiple homes with multiple residents. In: ICPCTH, PervasiveHealth ’13, pp 232–235

  2. Aminikhanghahi S, Cook D (2019) Enhancing activity recognition using cpd-based activity segmentation. Pervasive Mob Comput 53:75–89

    Article  Google Scholar 

  3. Benmansour A, Bouchachia A, Feham M (2015) Multioccupant activity recognition in pervasive smart home environments. ACM Comput Surv 48(3):34:1–34:36. http://doi.acm.org/10.1145/2835372

    Google Scholar 

  4. Brand M, Oliver N, Pentland A (1997) Coupled hidden markov models for complex action recognition. In: CVPR, CVPR ’97, pp 994–

  5. Chen R, Tong Y (2014) A two-stage method for solving multi-resident activity recognition in smart environments. Entropy 16(4):2184

    Article  Google Scholar 

  6. Chiang YT, Hsu KC, Lu CH, Fu LC, Hsu JYJ (2010) Interaction models for multiple-resident activity recognition in a smart home. In: IEEE/RSJ international conference on IROS, pp 3753–3758

  7. Chung J, Gülçehre Ç, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. ArXiv:1412.3555

  8. Cook D (2012) Learning setting-generalized activity models for smart spaces. IEEE Intell Syst 27(undefined):32–38

    Article  Google Scholar 

  9. Cook D, Crandall A, Singla G, Thomas B (2010) Detection of social interaction in smart spaces. Cybern Syst 41(2):90–104

    Article  Google Scholar 

  10. Crandall AS, Cook D (2008) Resident and caregiver: handling multiple people in a smart care facility. In: Proceedings of AAAI fall symposium: AI in Eldercare: new solutions to old problems, AAAI Technical Report, vol FS-08-02. AAAI

  11. Dahmen J, Cook D (2019) Synsys: a synthetic data generation system for healthcare applications. Sensors, 19

  12. Dahmen J, Thomas BL, Cook D, Wang X (2017) Activity learning as a foundation for security monitoring in smart homes. Sensors, 17

  13. Davis RIA, Lovell BC (2004) Comparing and evaluating HMM ensemble training algorithms using train and test and condition number criteria. Pattern Anal Appl 6 (4):327–335

    Article  MathSciNet  Google Scholar 

  14. Feuz KD, Cook D (2017) Collegial activity learning between heterogeneous sensors. Knowl Inf Syst 53(2):337–364

    Article  Google Scholar 

  15. Ghahramani Z, Jordan MI (1997) Factorial hidden Markov models. Mach Learn 29(2–3):245–273

    Article  Google Scholar 

  16. Ghods A, Cook D (2019) Activity2vec: learning adl embeddings from sensor data with a sequence-to-sequence model. In: KDD Workshop on applied data science for healthcare

  17. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  18. Hsu KC, Chiang YT, Lin GY, Lu CH, Hsu JYJ, Fu LC (2010) Strategies for inference mechanism of conditional random fields for multiple-resident activity recognition in a smart home. Springer, Berlin, pp 417–426

  19. Mehr HD, Polat H, Cetin A (2016) Resident activity recognition in smart homes by using artificial neural networks. In: 2016 4th International Istanbul smart grid congress and fair (ICSG). https://doi.org/10.1109/SGCF.2016.7492428, pp 1–5

  20. Minor B, Doppa JR, Cook D (2015) Data-driven activity prediction: algorithms, evaluation methodology, and applications. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’15. http://doi.acm.org/10.1145/2783258.2783408. ACM, New York, pp 805–814

  21. Minor BD, Doppa JR, Cook D (2017) Learning activity predictors from sensor data: algorithms, evaluation, and applications. IEEE Trans Knowl Data Eng 29 (12):2744–2757

    Article  Google Scholar 

  22. Natani A, Sharma A, Peruma T, Sukhavasi S (2019) Deep learning for multi-resident activity recognition in ambient sensing smart homes. In: 2019 IEEE 8th Global conference on consumer electronics (GCCE), pp 340–341

  23. Prossegger M, Bouchachia A (2014) Multi-resident activity recognition using incremental decision trees. In: Adaptive and intelligent systems - third international conference, ICAIS 2014, Bournemouth, UK, September 8-10, 2014. Proceedings, pp 182–191

  24. Rabiner LR (1990) Readings in speech recognition. chap. A tutorial on hidden Markov models and selected applications in speech recognition. Morgan Kaufmann Publishers Inc., San Francisco, pp 267–296. http://dl.acm.org/citation.cfm?id=108235.108253

    Google Scholar 

  25. Singla G, Cook D, Schmitter-Edgecombe M (2010) Recognizing independent and joint activities among multiple residents in smart environments. J Ambient Intell Humaniz Comput 1(1):57–63. https://doi.org/10.1007/s12652-009-0007-1

    Article  Google Scholar 

  26. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Proceedings of the 27th international conference on neural information processing systems - volume 2, NIPS’14. http://dl.acm.org/citation.cfm?id=2969033.2969173. MIT Press, Cambridge, pp 3104–3112

  27. Tan T, Gochoo M, Huang S, Liu Y, Liu S, Huang Y (2018) Multi-resident activity recognition in a smart home using rgb activity image and dcnn. IEEE Sensors J 18(23):9718–9727

    Google Scholar 

  28. Tran SN, Zhang Q (2004) Towards multi-resident activity monitoring with smarter safer home platform. In: Chen F, García-Betances RI, Chen L, Cabrera-Umpiérrez MF, Nugent CD (eds) Smart assisted living, chap. 12. Springer International Publishing, Springer, pp 249–267

  29. Tunca C, Alemdar H, Ertan H, Incel OD, Ersoy C (2014) Multimodal wireless sensor network-based ambient assisted living in real homes with multiple residents. Sensors 14:9692–9719

    Article  Google Scholar 

  30. Wang L, Gu T, Tao X, Chen H, Lu J (2011) Recognizing multi-user activities using wearable sensors in a smart home. Pervasive Mob Comput 7(3):287–298. https://doi.org/10.1016/j.pmcj.2010.11.008

    Article  Google Scholar 

  31. Wang A, Chen G, Shang C, Zhang M, Liu L (2016) Human activity recognition in a smart home environment with stacked denoising autoencoders. In: WAIM Workshops

Download references

Author information

Authors and Affiliations

  1. The Discipline of ICT, University of Tasmania, Newnham Campus, Launceston, Australia

    Son N. Tran

  2. Department of Computer Science, FPT University, Hanoi, Vietnam

    Tung-Son Ngo

  3. The Australian E-Health Research Centre, CSIRO, Royal Brisbane and Women’s Hospital, Level 5, UQ Health Science building, Brisbane, Australia

    Qing Zhang & Mohan Karunanithi

Authors
  1. Son N. Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tung-Son Ngo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohan Karunanithi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Son N. Tran.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tran, S.N., Ngo, TS., Zhang, Q. et al. Mixed-dependency models for multi-resident activity recognition in smart homes. Multimed Tools Appl 79, 23445–23460 (2020). https://doi.org/10.1007/s11042-020-09093-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09093-0

Keywords

Search

Navigation