Skip to main content
SpringerLink
Log in

From speech acts to assistance acts for cognitive assistance in ambient assisted living: how to nudge cognitively impaired people to act independently

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Technology-driven cognitive assistance is often either excessive or insufficient, and then not necessarily helpful, because of the inadaptability to the needs and abilities of people with cognitive impairments. Cognitive assistance is usually a dialogue (an interaction) where the caregiver provides assistive cues to which the assisted person replies with behaviours. This research proposes an ontology-based cognitive assistance model for ambient assisted living (AAL) systems. It aims to characterize the situations where assistance is needed and to provide an adaptive dialogue between the AAL system and individuals to help them cope with problematic situations in a minimally assisted manner, in so far as that is possible. The theory of speech acts is extended to translate cognitive assistance into adaptive assistive messages. The model enables AAL systems to identify when to assist according to the sensor observations and to deliver progressive minimal guidance messages with stops, or adjustments and replays, depending on the behaviours of people living with cognitive impairments, such as traumatic brain injury. Such user-centered assistance considers both the abilities, and the assistance needs of individuals, to give the right level of assistance that just nudges them towards carrying out tasks “on their own”. The model builds on cognitive assistance as used within clinical practice in the field of TBI and extends it to enable AAL systems to deliver assistance acts in due course through various actuators during the person’s everyday life activities. The model is formalized in an ontology to enable reasoning capability, extension, and reuse in different AAL systems. The illustration using a real-life scenario shows its usability and applicability in the provision of cognitive assistance through graded guidance message spreading.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In this work, the terms “user”, “assisted person”, “patient” and “resident” are used to refer to the same entity, namely the resident of an AAL environment who has cognitive impairments.

  2. https://www.merriam-webster.com/dictionary/dialogue [Accessed August 2021].

  3. http://www.fipa.org/repository/aclspecs.html [Accessed August 2021].

  4. http://ontologydesignpatterns.org/wiki/Ontology:DOLCE+DnS_Ultralite [Accessed August 2021].

  5. The Application Programming Interface that directly accesses the ontology.

  6. https://www.openhab.org/ [Accessed August 2021].

  7. Message Queuing Telemetry Transport https://mqtt.org/ [Accessed August 2021].

  8. https://www.w3.org/Submission/2004/SUBM-SWRL-20040521/ [Accessed August 2021].

  9. https://www.w3.org/TR/sparql11-query/ [Accessed August 2021].

  10. World Wide Web Consortium.

  11. https://protege.stanford.edu/ [Accessed: august 2021].

  12. https://jena.apache.org [Accessed: august 2021].

  13. National Fire Protection Association (NFPA): www.nfpa.org [Accessed: august 2021].

  14. https://www.usherbrooke.ca/domus/en/ [Accessed: august 2021].

References

  • Abowd GA, Bobick A, Essa I, Mynatt E, Rogers W (2002) The Aware Home: A Living Laboratory for Technologies for Successful Aging. In: National Conference on Artificial Intelligence, January, pp 1–7

  • Abras C, Maloney-Krichmar D, Preece J (2004) User-centered design. In: Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications

  • Alirezaie M, Renoux J, Köckemann U, Kristoffersson A, Karlsson L, Blomqvist E, Tsiftes N, Voigt T, Loutfi A (2017) An ontology-based context-aware system for smart homes: E-care@home. Sensors (switzerland) 17(7):1–24. https://doi.org/10.3390/s17071586

    Article  Google Scholar 

  • Andreadis S, Stavropoulos TG, Meditskos G, Kompatsiaris I (2016) Dem@Home: Ambient intelligence for clinical support of people living with dementia. CEUR Workshop Proc 1588:27–36

    Google Scholar 

  • Andrieu S, Grand A (2002) Place de l’aidant informel dans la consommation médicamenteuse de patients atteints de la maladie d’Alzheimer. Gérontologie Et Société 25(103):187–193

    Article  Google Scholar 

  • Austin JL (1962) How to do things with words. Oxford University Press, Oxford

    Google Scholar 

  • Ayimdji Tekemetieu A, Haidon C, Bergeron F, Ngankam HK, Pigot H, Gouin-Vallerand C, Giroux S (2021) Context Modelling in Ambient Assisted Living: Trends and Lessons. In: García Márquez FP, Lev B (eds) Internet of Things: Cases and Studies. Springer, Cham, pp. 189–225. https://doi.org/10.1007/978-3-030-70478-0_10

  • Ayimdji Tekemetieu A, Pigot H, Bottari C, Gagnon-Roy M, Giroux S (2021) Modeling an adaptive resident-system interaction for cognitive assistance in ambient assisted living. Hum Agent Interact (HAI’21). https://doi.org/10.1145/3472307.3484166

  • Barman A, Chatterjee A, Bhide R (2016) Cognitive impairment and rehabilitation strategies after traumatic brain injury. Indian J Psychol Med 38(3):172–181. https://doi.org/10.4103/0253-7176.183086

    Article  Google Scholar 

  • Bauchet J, Pigot H, Giroux S, Lussier-Desrochers D, Lachapelle Y, Mokhtari M (2009) Designing judicious interactions for cognitive assistance. In: 11th International ACM SIGACCESS Conference on Computers and Accessibility, 11–18. https://doi.org/10.1145/1639642.1639647

  • Bellifemine F, Caire G, Greenwood D (2007) Developing multi-agent systems with JADE. Wiley, Hoboken

    Book  Google Scholar 

  • Blackman S, Matlo C, Bobrovitskiy C, Waldoch A, Fang ML, Jackson P, Mihailidis A, Nygård L, Astell A, Sixsmith A (2016) Ambient assisted living technologies for aging well: a scoping review. J Intell Syst 25(1):55–69. https://doi.org/10.1515/jisys-2014-0136

    Article  Google Scholar 

  • Bottari C, Dassa C, Rainville CM, Dutil É (2010) The IADL profile: development, content validity, intra- and interrater agreement. Can J Occup Ther 77(2):90–100. https://doi.org/10.2182/cjot.2010.77.2.5

    Article  Google Scholar 

  • Bottari C, Iliopoulos G, Shun PLW, Dawson DR (2014) The clinical reasoning that guides therapists in interpreting errors in real-world performance. J Head Trauma Rehabil 29(6):E18–E30. https://doi.org/10.1097/HTR.0000000000000029

    Article  Google Scholar 

  • Bottari C, Dutil É, Auger C, Lamoureux J (2020) Structural validity and internal consistency of an ecological observation-based assessment, the Activities of Daily Living (ADL) Profile. Aust Occup Ther J 67:407–416. https://doi.org/10.1111/1440-1630.12654

    Article  Google Scholar 

  • Camp CJ, Bird MJ, Cherry KE (2000) Retrieval Strategies as a Rehabilitation Aid for Cognitive Loss in Pathological Aging. In: Hill RD, Bäckman L, Neely AS (eds.), Cognitive rehabilitation in old age

  • Chahuara P, Portet F, Vacher M (2017) Context-aware decision making under uncertainty for voice-based control of smart home. Expert Syst Appl 75:63–79. https://doi.org/10.1016/j.eswa.2017.01.014

    Article  Google Scholar 

  • Colledanchise M, Almeida D, Ögren P (2019) Towards blended reactive planning and acting using behavior trees. In: Proceedings - IEEE International Conference on Robotics and Automation, 2019-May, 8839–8845. https://doi.org/10.1109/ICRA.2019.8794128

  • Cook DJ, Youngblood M, Edwin O Heierman I, Gopalratnam K, Sira Rao AL, Khawaja F (2003) MavHome: an agent-based smart home. In: Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003), 521–524. https://doi.org/10.1109/percom.2003.1192783

  • Dey AK, Abowd GD (1999) Towards a better understanding of context and context-awareness. Comput Syst 40(3):304–307

    Google Scholar 

  • Díaz Rodríguez N, Cuéllar MP, Lilius J, Delgado Calvo-Flores M (2014) A fuzzy ontology for semantic modelling and recognition of human behaviour. Knowl-Based Syst 66:46–60. https://doi.org/10.1016/J.KNOSYS.2014.04.016

    Article  Google Scholar 

  • Francillette Y, Bouchard B, Bouchard K, Gaboury S (2020) Modeling, learning, and simulating human activities of daily living with behavior trees. Knowl Inf Syst 62(10):3881–3910. https://doi.org/10.1007/s10115-020-01476-x

    Article  Google Scholar 

  • Gagnon-Roy M, Bier N, Couture M, Giroux S, Pigot H, Zarshenas S, Bottari C (2020) Facilitators and obstacles to the use of a cognitive orthosis for meal preparation within the homes of adults with a moderate to severe traumatic brain injury: Informal caregivers and health-care professionals’ perspectives. Assist Technol. https://doi.org/10.1080/10400435.2020.1809552

    Article  Google Scholar 

  • Gagnon-Roy M, Bier N, Boulé-Riley S, Keurentjes H, LamWaiShun P, LeDorze G, Bottari C (2021) Providing verbal assistance when assessing individuals living with a traumatic brain injury. Can J Occup Therapy. https://doi.org/10.1177/00084174211034263

    Article  Google Scholar 

  • Garcia Marquez FP, Lev B (eds) (2021) Internet of Things - Cases and studies. Springer, NewYork. https://doi.org/10.1007/978-3-030-70478-0

  • Giroux S, Bier N, Pigot H, Bouchard B, Bouzouane A, Levasseur M, Couture M, Bottari C, Swaine B, Therriault P-Y, Bouchard K, Le Morellec F, Pinard S, Azzi S, Olivares M, Zayani T, Le Dorze G, De Loor P, Thépaut A, Le Pévédic B (2015) Cognitive Assistance to Meal Preparation: Design, Implementation, and Assessment in a Living Lab. In: 2AAAI 2015 Spring Workshop Symposium, Ambient Intelligence for Health and Cognitive Enhancement, 14–16

  • Greber C, Ziviani J, Rodger S (2007) The four quadrant model of facilitated learning: a clinically based action research project. Aust Occup Ther J 54(2):149–152. https://doi.org/10.1111/j.1440-1630.2006.00558.x

    Article  Google Scholar 

  • Griol D, Molina JM, Callejas Z (2014) Modeling the user state for context-aware spoken interaction in ambient assisted living. Appl Intell 40(4):749–771. https://doi.org/10.1007/s10489-013-0503-z

    Article  Google Scholar 

  • Haskins E, Cicerone K, Dams-O’Connor K, Eberle R, Langenbahn D, Shapiro-Rosenbaum A (2012) Cognitive Rehabilitation Manual, Translating Evidence-Based Recommendations into Practice ACRM Publishing

  • Hassan MK, El Desouky AI, Elghamrawy SM, Sarhan AM (2018) Intelligent hybrid remote patient-monitoring model with cloud-based framework for knowledge discovery. Comput Electr Eng 70:1034–1048. https://doi.org/10.1016/j.compeleceng.2018.02.032

    Article  Google Scholar 

  • Imbeault H, Bier N, Pigot H, Gagnon L, Marcotte N, Fulop T, Giroux S (2014) Electronic organiser and Alzheimer’s disease: fact or fiction? Neuropsychol Rehabil 24(1):71–100. https://doi.org/10.1080/09602011.2013.858641

    Article  Google Scholar 

  • Imbeault H, Gagnon L, Pigot H, Giroux S, Marcotte N, Cribier-Delande P, Duval J, Bocti C, Lacombe G, Fülöp T, Bier N (2016) Impact of AP@LZ in the daily life of three persons with Alzheimer’s disease: long-term use and further exploration of its effectiveness. Neuropsychol Rehabil 28(5):755–778. https://doi.org/10.1080/09602011.2016.1172491

    Article  Google Scholar 

  • Janowicz K, Haller A, Cox SJD, Le Phuoc D, Lefrançois M (2019) SOSA: a lightweight ontology for sensors, observations, samples, and actuators. J Web Semant 56:1–10. https://doi.org/10.1016/j.websem.2018.06.003

    Article  Google Scholar 

  • Kenfack Ngankam H (2019) Modèle sémantique d’intelligence ambiante pour le développement « Do-It-Yourself » d’habitats intelligents [Université de Sherbrroke]. https://savoirs.usherbrooke.ca/handle/11143/14941

  • Langrial S (2014) Developing assistive interventions: Promises and challenges. In: Theng LB (ed.), Assistive Technologies for Physical and Cognitive Disabilities (pp 136–151). IGI Global. https://doi.org/10.4018/978-1-4666-7373-1.ch007

  • Le Dorze G, Villeneuve J, Zumbansen A, Masson-Trottier M, Bottari C (2014) Verbal assistance within the context of an IADL evaluation. Open J Therapy Rehabil 02(04):182–198. https://doi.org/10.4236/ojtr.2014.24024

    Article  Google Scholar 

  • Ledvinka M, Křemen P (2015) JOPA: Accessing ontologies in an object-oriented way. In: ICEIS 2015 - 17th International Conference on Enterprise Information Systems, Proceedings, 2, 212–221. https://doi.org/10.5220/0005400302120221

  • Lee SG, Sathikh P (2013) a Framework for Effective Human-To- Machine Communication for Artificial Interactive Systems. In: International Conference on Engineering Design, Iced13, August, 1–9

  • López-Cózar R, Callejas Z (2010) Multimodal Dialogue for Ambient Intelligence and Smart Environments. In: Handbook of Ambient Intelligence and Smart Environments, pp 559–579. https://doi.org/10.1007/978-0-387-93808-0_21

  • Merdivan E, Singh D, Hanke S, Holzinger A (2019) Dialogue systems for intelligent human computer interactions. Electron Notes Theor Comput Sci 343:57–71. https://doi.org/10.1016/j.entcs.2019.04.010

    Article  Google Scholar 

  • Nihei, M., & Fujie, M. G. (2012). Proposal for a New Development Methodology for Assistive Technology Based on a Psychological Model of Elderly People. In: Cheein FAA (ed), Assistive Technologies. InTech. https://doi.org/10.5772/30407

  • Noy NF, McGuinness DL (2001) Ontology development 101: a guide to creating your first ontology. Tech. rep., Stanford Knowledge Systems Laboratory, Stanford University

  • O’Neill B, Best C, O’Neill L, Ramos SDS, Gillespie A (2018) Efficacy of a micro-prompting technology in reducing support needed by people with severe acquired brain injury in activities of daily living: a randomized control trial. J Head Trauma Rehabil 33(5):E33–E41. https://doi.org/10.1097/HTR.0000000000000358

    Article  Google Scholar 

  • Olivares M, Giroux S, De Loor P, Thépaut A, Pigot H, Pinard S, Bottari C, Le Dorze G, Bier N (2016) An ontology model for a context-aware preventive assistance system: reducing exposition of individuals with Traumatic Brain Injury to dangerous situations during meal preparation. In: 2nd IET International Conference on Technologies for Active and Assisted Living (TechAAL 2016), 8. https://doi.org/10.1049/ic.2016.0052

  • Pinard S, Bottari C, Laliberté C, Pigot H, Olivares M, Couture M, Giroux S, Bier N (2019) Design and usability evaluation of COOK, an assistive technology for meal preparation for persons with severe TBI. Disabil Rehabil Assist Technol. https://doi.org/10.1080/17483107.2019.1696898

    Article  Google Scholar 

  • Radziszewski R, Ngankam HK, Grégoire V, Lorrain D, Pigot H, Giroux S (2017) Designing calm and non-intrusive ambient assisted living system for monitoring nighttime wanderings. Int J Pervasive Comput Commun 13(2):211–234. https://doi.org/10.1108/IJPCC-02-2017-0015

    Article  Google Scholar 

  • Rashidi P, Mihailidis A (2013) A survey on ambient-assisted living tools for older adults. IEEE J Biomed Health Inform 17(3):579–590. https://doi.org/10.1109/JBHI.2012.2234129

    Article  Google Scholar 

  • Rodríguez-Muro M, Kontchakov R, Zakharyaschev M (2013) Ontology-Based Data Access: Ontop of Databases. In: Alani H, Kagal L, Fokoue A, Groth P, Biemann C, Parreira JX, Aroyo L, Noy CW, Janowicz K (eds.) The Semantic Web – ISWC, vol 8218. Springer, Berlin Heidelberg, pp 558–573. https://doi.org/10.1007/978-3-642-41335-3_35

  • Rogers JC, Holm MB, Burgio LD, Granieri E, Hsu C, Hardin JM, McDowell BJ (1999) Improving morning care routines of nursing home residents with dementia. J Am Geriatr Soc 47(9):1049–1057. http://www.ncbi.nlm.nih.gov/pubmed/10484245

  • Searle J (1969) Speech acts an essay in the philosophy of language. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Searle J (1979) Studies in the theory of speech acts. In philosophy. Cambridge University Press, Cambridge. https://doi.org/10.1213/01.ane.0000250225.96165.4b

    Book  Google Scholar 

  • Searle J (1976) A classification of illocutionary acts. Lang Soc 5(1):1–23. https://sites.duke.edu/conversions/files/2014/09/Searle_Illocutionary-Acts.pdf

  • Seelye AM, Schmitter-Edgecombe M, Das B, Cook DJ (2012) Application of cognitive rehabilitation theory to the development of smart prompting technologies. IEEE Rev Biomed Eng 5:29–44. https://doi.org/10.1109/RBME.2012.2196691

    Article  Google Scholar 

  • Serna A, Pigot H, Bauchet J, Giroux S, Rialle V, L-Desrochers D, Mokhtari M (2010) Recommandations ergonomiques pour la conception de systèmes d’assistance cognitive dans les habitats intelligents. 3. https://doi.org/10.1145/1868650.1868652

  • Shahrestani S (2017) Internet of things and smart environments: assistive technologies for disability, dementia, and aging. In: Internet of Things and Smart Environments: Assistive Technologies for Disability, Dementia, and Aging. https://doi.org/10.1007/978-3-319-60164-9

  • Skillen KL, Chen L, Nugent CD, Donnelly MP, Burns W, Solheim I (2014) Ontological user modelling and semantic rule-based reasoning for personalisation of Help-On-Demand services in pervasive environments. Futur Gener Comput Syst 34:97–109. https://doi.org/10.1016/j.future.2013.10.027

    Article  Google Scholar 

  • Sohlberg MM, Mateer CA (1989) Training use of compensatory memory books: a three stage behavioral approach. J Clin Exp Neuropsychol 11(6):871–891. https://doi.org/10.1080/01688638908400941

    Article  Google Scholar 

  • Talmelli LFS, Gratão ACM, Kusumota L, Rodrigues RAP (2010) Functional independence level and cognitive deficit in elderly individuals with Alzheimer’s disease. Rev Esc Enferm USP 44(4):933–939. https://doi.org/10.1590/S0080-62342010000400011

    Article  Google Scholar 

  • Wang J, Mahajan HP, Toto PE, McCue MP, Ding D (2019) The feasibility of an automatic prompting system in assisting people with traumatic brain injury in cooking tasks. Disabil Rehabil Assist Technol 14(8):817–825. https://doi.org/10.1080/17483107.2018.1499144

    Article  Google Scholar 

  • Zarshenas S, Gagnon-Roy M, Couture M, Bier N, Giroux S, Nalder E, Pigot H, Dawson D, Poncet F, LeDorze G, Bottari C (2021a) Potential of using an assistive technology to address meal preparation difficulties following acquired brain injury: clients’ and caregivers’ perspectives. Disabil Rehabil Assist Technol. https://doi.org/10.1080/17483107.2020.1867244

    Article  Google Scholar 

  • Zarshenas S, Couture M, Bier N, Giroux S, Nalder E, Lemsky C, Pigot H, Dawson DR, Gosselin N, LeDorze G, Gagnon-Roy M, Hendryckx C, Bottari C (2021b) Implementation of an assistive technology for cognition within a supported residence for adults with acquired brain injury: a mixed-methods single case (in press). Disabil Rehabil Assist Technol

Download references

Funding

This study was funded by Centre de recherche sur le viellissement.

Author information

Authors and Affiliations

  1. Laboratoire Domus, Université de Sherbrooke, Sherbrooke, Québec, Canada

    Armel Ayimdji Tekemetieu, Hélène Pigot & Sylvain Giroux

  2. Institut Universitaire de Technologie (IUT) de Douala, Université de Douala, Douala, Cameroun

    Armel Ayimdji Tekemetieu

  3. Centre de Recherche Sur Le Vieillissement, Sherbrooke, Québec, Canada

    Hélène Pigot & Sylvain Giroux

  4. École de Réadaptation, Université de Montréal, Montreal, Québec, Canada

    Carolina Bottari

  5. Institut Universitaire Sur La Réadaptation en Déficience Physique de Montréal, Montreal, Canada

    Carolina Bottari

Authors
  1. Armel Ayimdji Tekemetieu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hélène Pigot
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carolina Bottari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sylvain Giroux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Armel Ayimdji Tekemetieu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ayimdji Tekemetieu, A., Pigot, H., Bottari, C. et al. From speech acts to assistance acts for cognitive assistance in ambient assisted living: how to nudge cognitively impaired people to act independently. J Ambient Intell Human Comput 14, 11767–11793 (2023). https://doi.org/10.1007/s12652-022-03735-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-03735-x

Keywords

Search

Navigation