Abstract
In the context of self-reported health, where the subjective perception of the patients is reported through simple yet effective questionnaires, information gathering is very important to obtain consistent and meaningful data analysis. Smart phones are a good tool to gather self-reported variables, but the interaction with the user should go ahead of scheduled notifications. We develop an intelligent notification system that learns by exploration the most adequate time to perform a questionnaire, while using just the answers of the notification messages from the user. We address the smart notification as a Reinforcement Learning (RL) problem, considering several states representations and reward functions for the Upper Confidence Bound, Tabular Q-learning and Deep Q-learning. We evaluate the algorithms on a simulator, followed by an initial prototype where the approach with better performance in simulation is selected for a small pilot. The simulator acts as a person, accepting or discarding the notifications according to the behavior of a three typical users. From the simulation experiments the UCB algorithm showed the most promising results, so we implemented and deployed the RL algorithm in a smartphone application with several users. On this initial pilot with four users, the UCB algorithm was able to find the adequate hours to send notifications for quiz answering.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bellman, R.: Dynamic Programming, 1st edn. Princeton University Press, Princeton (1957)
Developers, A.: Manager, alarm (2022). https://developer.android.com/reference/android/app/AlarmManager.html. Accessed 14 Nov 2022
Fiordelli, M., Diviani, N., Schulz, P.J.: Mapping mhealth research: a decade of evolution. JMIR 15(5), 1–14 (2013). https://doi.org/10.2196/jmir.2430
Fukazawa, Y., Yamamoto, N., Hamatani, T., Ochiai, K., Uchiyama, A., Ohta, K.: Smartphone-based mental state estimation: a survey from a machine learning perspective. JIP 28, 16–30 (2020). https://doi.org/10.2197/ipsjjip.28.16
Geneva: WHO: noncommunicable diseases progress monitor 2020 (2020). https://doi.org/10.5005/jp/books/11410_18
Google: Firebase (2022). https://firebase.google.com/. Accessed 14 Nov 2022
Ho, B.J., Balaji, B., Koseoglu, M., Srivastava, M.: Nurture: notifying users at the right time using reinforcement learning. In: UBICOMP, pp. 1194–1201 (2018). https://doi.org/10.1145/3267305.3274107
Lin, L.J.: Reinforcement learning for robots using neural networks. Ph.D. thesis (1993)
Machado., A., Cardoso., H., Moreno., P., Bernardino., A.: Active data collection of health data in mobile devices. In: Proceedings of the 3rd International Conference on Deep Learning Theory and Applications - DeLTA, pp. 160–167. INSTICC, SciTePress (2022). https://doi.org/10.5220/0011300700003277
Mehrotra, A., Musolesi, M.: Intelligent notification systems: a survey of the state of art and research. Challenges 1(1), 1–26 (2017)
Mehrotra, A., Musolesi, M., Hendley, R., Pejovic, V.: Designing content-driven intelligent notification mechanisms for mobile applications. In: UBICOMP, pp. 813–824 (2015). https://doi.org/10.1145/2750858.2807544
Mehrotra, A., Pejovic, V., Vermeulen, J., Hendley, R.: My phone and me: understanding people’s receptivity to mobile notifications. In: CHI, pp. 1021–1032 (2016). https://doi.org/10.1145/2858036.2858566
Mnih, V., Kavukcuoglu, K.: Silver: human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Richard S. Sutton, A.G.B.: Reinforcement Learning : an Introduction. MIT Press, Cambridge 2\(^\circ \) edn. (2018). https://doi.org/10.5555/3312046
Vaizman, Y., Ellis, K., Lanckriet, G.: Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Comput. 16(4), 62–74 (2017). https://doi.org/10.1109/MPRV.2017.3971131
Vishwanath, S., Vaidya, K., Nawal, R.: Touching lives through mobile health-assessment of the global market opportunity. In: PwC (2012)
Watkins, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992). https://doi.org/10.1109/ICCC49849.2020.9238991
Whitmore, C., et al.: The relationship between multimorbidity and self-reported health among community-dwelling older adults and the factors that shape this relationship: a mixed methods study protocol using clsa baseline data. Int. J. Qual. Methods 20, 16094069211050166 (2021)
WHO Global Observatory for eHealth: mHealth: new horizons for health through mobile technologies: second global survey on eHealth (2011)
Acknowledgements
This work has been partially funded by the project LARSyS - FCT Project UIDB/50009/2020 and the project and by the project IntelligentCare - Intelligent Multimorbidity Managment System (Reference LISBOA-01-0247-FEDER-045948), which is co-financed by the ERDF - European Regional Develpment Fund through the Lisbon Portugal Regional Operational Program - LISBOA 2020 and by the Portuguese Foundation for Science and Technology - FCT under CMU Portugal.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Marques, J., Faria, F., Machado, R., Cardoso, H., Bernardino, A., Moreno, P. (2023). Active Collection of Well-Being and Health Data in Mobile Devices. In: Fred, A., Sansone, C., Gusikhin, O., Madani, K. (eds) Deep Learning Theory and Applications. DeLTA 2022. Communications in Computer and Information Science, vol 1858. Springer, Cham. https://doi.org/10.1007/978-3-031-37317-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-37317-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37316-9
Online ISBN: 978-3-031-37317-6
eBook Packages: Computer ScienceComputer Science (R0)