Healthy Aging: A Deep Meta-Class Sequence Model to Integrate Intelligence in Digital Twin

Objective: The behavior monitoring of older adults in their own home and enabling daily-life activity analysis to healthcare practitioner is a key challenge. Methods and procedures: Our framework replicates the elderly home in digital space which can provide an unobtrusive way to monitor the resident&ahat;s daily life activities. The learning challenges posed by different performed activities at home are solved by introducing the deep meta-class sequence model. The notion is to group the set of activities into a single meta-class according to the nature of the activities. It helps the learning process, which is based on long short-term memory (LSTM) to learn feature space abstraction. Each meta-class abstraction is further decomposed to an individual activity performed by the elderly at home. Results: The experiments are carried out over the Center for Advanced Studies in Adaptive Systems dataset and proposed model outperforms as compared to baseline models. Clinical impact: Our findings demonstrate a robust framework to digitally monitor the elderly behavior, which is beneficial for healthcare practitioners to understand the level of support the elderly needed to perform the daily tasks or potential risk of an emergency in their own homes.


I. INTRODUCTION
The aging population is growing globally at an unprecedented rate, which is associated with the challenges of declines in health, reduces the quality of life, and leads to loss of independence. This emerging demographic introduces drastic changes to our society and increases the complex needs of the aging population [1], [2]. The existing health care systems may not be sufficiently prepared to respond to this age shift. The United Nations (UN) emphasize the worldwide need to enhance the lives of elder individuals to combat ageism. It established a mission on Decade of Healthy Aging (2021-2030) for a better tomorrow for elder people, their families, and societies [3].
Long-term care facilities (LTCFs) or nursing dwellings are renowned solutions for elderly people. Most elderly who lives in these care facilities become depressed due to the lack of independence. While a large population cannot bear the financial burden to reside in such facility units. Similarly, the elderly residing in LTCFs face significant challenges and growing concerns have been observed during the pandemic time because of the weakened immunity and most vulnerable to infection [4]. Such solutions are not sustainable to fulfill the needs of future generations of the elderly population. In contrast, a home is a place where individuals feel secure and high level of comfort because they lived in their own homes for ages. There is a need to transform the homes into smart spaces which can assist the residents to perform daily routine tasks without another person's interference. It can be seen as a sustainable solution that provides a sense of independence, prevent social isolation, and self-sufficiency in their own homes.
The advancement in sensor technology without disturbing the privacy of residents (such as motion, door, and temperature sensor placement in-home setting) [5], abilities of deep learning models to recognize daily routines [6], and information communication technology (ICT) [7] between the household and caregivers enable assisted living. Although the concept of assistive living successfully contributes to providing a level of independence for elderly residing in their own homes to perform daily routines [8]. There are still gaps that exist to connect caregivers with elderly homes where the elderly live independently. First, a digital transformation is required to develop a virtual replica, where caregivers can interact in digital space in real-time. The digital twin has ability of digital transformation, where interaction with knowledge graph provide a mechanism to interact in digital space. Secondly, a robust learning scheme, which enables accurate context recognition in the elderly home to avoid emergencies and assist the individual when needed.
To solve the aforementioned challenges, we proposed a framework based on digital twin technology to create an accurate virtual abstraction of an elderly home connected in the digital space with integrated intelligence. Our deep meta-class sequence model efficiently learns the sensor events sequence to classify the performed activities. It can facilitate the caregivers to enable the provision of remote monitoring and need-based interventions for elderly residents living alone at home. The caregivers can interact with virtual space and query the digital twin to know the status of elderly activities. Consequently, it can alleviate the burdens of caregivers and reduce the cost associated with care facility units.
We identify the three key contributions: (1) Design a novel digital twins' framework to connect elderly homes as a new-age technology to interact with the physical home in digital space; (2) We introduce and develop a deep meta-class sequence model to produce accurate results as compared to existing counterparts; (3) Our framework brings a new sustainable solution to respond to the age shift.
The rest of the paper is organized as follows. Section II discusses the related research work and its limitations. Section III introduces the technical details of the proposed framework. Section IV, presents the result and discussion with performance analysis. Lastly, Section V concludes our findings.

II. RELATED WORK
The relevant research details in the domain of digital twin technology for the healthcare and embedded sensor-based home intelligence are provided in this section. We also identify and discuss the related issues in this section. The related work is divided into the following subsections.

A. DIGITAL TWIN IN HEALTHCARE
The digital twin is fast growing technology, which has many potential applications in smart healthcare ranging from the planning of hospital resources [9], personalized medicine [10], and well-being [11], [12]. In early work, Lehrach et al. [13] proposed a personalized healthcare system ''virtual-patient virtual-self'', which is based on clinical molecular imaging and multi-scale sensor data of European citizens. The objective was to create a digital system that provides a way to create personalized models and practitioners can optimize the individual treatment strategy on a detailed computer model in a cost-effective manner.
Xing et al. [14] presented a deep learning framework integrated with digital twins for cardiac analysis. The approach is utilized for the synthesis of myocardial velocity maps in imaging. They experimentally evaluate the model performed well for the task of precise left ventricle segmentation. This work is the first one to generate synthetic data from digital twin in the cardiac analysis domain. Similarly, Elayan et al. [15] suggested a digital twin-based system to monitor the fitness of the heart by analyzing the electrocardiogram (ECG). Their model is based on LSTM which outperforms as compared to classical machine learning approaches. Liu et al. [16] proposed a cloud-based framework for senior citizen healthcare using a digital twin. Their case study demonstrates the applicability of digital twins as well as medication reminder services. They consider a person with symptoms of arrhythmia, and reminders are sent to balance the heart rate. In industry, Siemens [17] developed a digital twin of a patient-focused healthcare system, which has the ability to tailor the interaction at mater private hospital (MPH) in Dublin, Ireland. It creates an individual patient workflow to reduce the wait time for patients and increase equipment utilization, which leads to a better patient experience as well as reduces the cost.
The digital twin provides real-time access to medical appliances, workflows, and synthetic data generation in digital healthcare. The role of AI is crucial to embed the intelligence to solve the challenges. In our work, we integrate a deep learning model to provide services in the elderly home.

B. EMBEDDED SENSOR-BASED HOME INTELLIGENCE
The embedded sensor in the home setting has many potential applications including monitoring daily life activities and behavior patterns of the aging population. In early research, Chen et al. [18] proposed a data-driven approach based on ontology to recognize the daily life activities inside the home setting. Similarly, Zhang et al. [19] presented a novel model which is based on an agent to comprehend the knowledge of the human activity and automatically decode it into knowledge. Other noticeable work is published in [20], [21], and [22] using classical machine learning approaches. In classical approaches, the pre-segmented sensor streams are mapped with hidden Markov models, Conditional Random fields, and Naive Bayes Classifier. The classical approaches work well in specific scenarios and are unable to map the long sequences of sensor streams. Due to these reasons, the research community is developing deep learning models, which are robust and has the ability to consider long sequences and temporal information for prediction. The details about the developed deep learning models are presented in [6]. Due to the intrinsic complexity of human behavior, deep learning models suffer from rare data samples. Liciotti et al. [23], suggested a sequential deep learning model based on LSTM units. Their model allows the learning of spatial-temporal information from the sensor data stream as well as temporal evolution due to recurrent connections. They successfully applied the deep model to process the ambient sensor data. VOLUME 11, 2023 Our model adds-on this existing model by adding the meta-class concept and novel architecture to classify the sensor event sequence at the micro activity level. In the elderly home, bathing, and dressing are critical to the well-being and independence of the aging population [2]. Such activities have fewer training examples (i.e., due to the nature of the activity) known as rare data samples, and usually get confused with other household activities. In the previous research, such activities are neglected/merged with the macro context. Our framework solves these issues by introducing a metaclass structure. One of the possible existing solutions is to solve this problem by adding synthetic data, weight the rare activities, or ensemble learning [24], [25]. The model weights to rate activities require domain knowledge. In our approach, we grouped the activities into a meta-class based on their nature. This helped the learning process during the training phase of the model and outperforms it as compared to its counterparts.
We integrate a robust intelligent model in digital twin to understand the elderly behavior at home to provide the required assistance and services. The developed framework is described as follows.

III. THE PROPOSED FRAMEWORK
The proposed framework consists of physical spaces (i.e., elderly home and healthcare services), digital space, and healthcare center as shown in Fig. 1. The details about the framework are provided in the following subsections.

A. PHYSICAL SPACES
The physical spaces include homes and hospitals to provide healthcare services. We consider the home as an elderly-centered approach to provide assisted services based on ambient sensors (i.e., motion, temperature, and door sensors). These sensors sense the covered location and data is intelligently processed to recognize the individual daily life activities. We assume that installed sensors have secure connectivity with the internet for further process over the cloud computing infrastructure. Suppose the set S represents installed sensors in the home at locations L, which generates data D to sense the environment. Assume a sensor s d ∈ S as a discrete value sensor, or s c ∈ S as continuous value sensor placed at some location l ∈ L, and generates data d ∈ D at time instant t is presented by tuple given by P t = {t, s, l, d}. We define a sensor value v (s,l) which have: Note that when a sensor changes the value, it is recorded as an event, and the sequence of sensor events is associated with an elderly person's interaction with objects in the home. This interaction has an association with the residents perform daily life activities at home with activity class c ∈ C. The |C| denotes the number of performed activities in the home and is expressed as: A training example maps a k sequence of ambient sensor events. Where examples are independent and identically distributed (i.i.d.) according to some unknown distribution (·, ·). Let DT represent the digital twin, to model physical space (i.e., elderly home), which depends on N to find the elderly behavior in digital space and provide healthcare services in physical space with the interaction of healthcare center. The learning scheme is defined as a function that maps the sequence of sensor events v (s α ,l) (t) to an activity label c, Such that: where is a parameter of the deep meta-class sequence model to embed the intelligence in the digital twin. The details are provided in the subsections.

B. DIGITAL SPACE
It consists of a digital twin and embedding intelligence layer. Healthcare center practitioners interact with digital space to identify the clinical nuances related to health-critical activities in the elderly home. The details are provided as follows:

1) DIGITAL TWIN
In case of traditional activity recognition, sensor data is sent to machine learning model to recognize the activity. If AI model detects an emergency at home, then there is no way to interact with home to find the behavioral patterns. For instance, the elderly at home spends too much time in cooking activity and the model generates an emergency alert. In case of digital twin, a home is virtual represented with installed sensors. The digital twin facilitates bidirectional communication between the home and its digital representation. This digital replica can facilitate the caregivers to check the home environment virtually inside the digital twin. The digital twin provides real-time access to an elderly home while preserving privacy.
The major firms like Amazon, Eclipse, and Microsoft start providing the platform for creating and operating digital twins in cloud environment. In our case, we utilized Microsoft Azure Digital Twin (ADT) [26] as a platform as a service (PaaS) to replicate the physical environment. Digital Twin for smart home environment is coded using a digital twin definition language (DTDL). It is a based on JavaScript Object Notation for Linked Data (JSON-LD) and is programminglanguage independent.
Let's define a 5D digital twin model as: where PE is physical entities, VR is virtual representation, DC is data curation, CS is communication scheme  and Ss for services. A detailed information to build the Microsoft Azure-based digital twins model is presented in our paper [27]. We consider the all attached sensors in a home environment as a PE and virtually present as a knowledge graph as shown in Fig. 2. In Fig. 2, the knowledge graph attaches the sensors in digital space by defining each sensor using digital twin definition language, and a relationship is established with a sensor stream. For instance in Fig. 2, H represents the elderly home, T 01 presents the temperature sensor, M 03 presents the motion sensor, and D01 represents the door sensor. The knowledge graph provides real-time access to the sensors to know the status of sensor readings and reduced the false positive rate. The caregivers in the healthcare center can understand the resident's behavior by interacting with the house virtually in digital twins using a knowledge graph. To provide such a facility, data is curated. The curation consists of cleaning and transforming the data into a format that provide an access to query the virtual home in realtime. The representational state transfer (REST) APIs are used to connect the physical spaces, which supports hypertext transfer protocol (HTTP) methods to access the digital twin resources. Security is an important concern that is provided by Microsoft Azure by registering healthcare center applications (i.e., client applications) with the Azure active directory to secure the REST request.

2) EMBEDDING INTELLIGENCE
This one is the core of our proposed framework to process the sensory data stream intelligently and provide information about the performed activities at home.

3) DATA PREPROCESSING
The data is preprocessed to transform the sensor events into a series of sequences which enables it to process in a deep learning model. The elderly person in the home performed daily life activities and different sensor events are generated which are stored in the database. The activity pattern is annotated as a start time with sensor events and an end time at the completion of the activity. In order to capture the temporal and spatial features of the data, it is aggregated as a feature vector with all its states during each activity. This feature vector is fed into the deep meta-class sequence model to recognize the context of the activity. The details about the model are provided in the following section.

4) DEEP META-CLASS SEQUENCE MODEL
Our hypothesis is to combine a subset of classes into a metaclass, which helps the deep sequence model to partition the feature space into disjoint regions effectively. Consequently, VOLUME 11, 2023 FIGURE 3. The personal hygiene meta class (m 1 ) to presents primary bathroom (c u1 ), guest bathroom (c u2 ), and bed to toilet (c uk ) activity.
it will help the learning scheme to recognize the activities accurately. Let's consider ( ) represents a deep meta-class sequence model as a composition of two levels and defined as follows.
At first level, we introduced meta-classes as: where m i represents a single meta-class from a set of n metaclasses. Each m i combines the different sensor streams based on spatial characteristics of the activity class, thus, resulting in a set of N ′ training examples. Fig. 3, presents the personal hygiene meta class, which combines primary bathroom activity, guest bathroom activity, and bed-to-toilet activity. The idea is to shift the learning scheme to a higher level (i.e., ( )), where it can partition the feature spaces easily. The model ( ) accepts a sequence of temporal input x = (x (1) , x (2) , x (3) , . . . , x (T ) ). Note that x (T ) represents v (s α ,l) (t) and processed by LSTM units layer, followed by two dense layers (i.e., as shown in Fig. 1). The LSTM cell is connected with its connection to the previous context as well as with hidden states that are presented in Fig. 4.
The LSTM cell uses a gating mechanism, which is calculated by the following equations.
The details about the LSTM cell computation are provided in the following Table. 1.
Thus one challenge is solved and each partition is assigned to a meta-class. While each class retains the information to  classify the individual activity. The output can be described as follows: where θ represents the parameters of the model, and ( * ) presents the whole model with LSTM and dense layers. The output is further processed by ( ).
At this stage, each meta-class is classified into an individual subclass. The constructed deep sequence network used sparse cross entropy to calculate the network loss, which can be formulated as follows: for c classes (16) The model is trained and validated according to Algorithm 1, after that, the trained model is integrated with DT .
Transform the input into model x → ( ) 8: Minimize the loss c i=1 y i log(ŷ) 9: model { ( )} update the model 10: end for 11: save the model { ( )} 12: end for 13: for each epoch do 14: for each batch X i N ′′ i=1 do 15: Transform the input into model x → ( ) 16: Minimize the loss c i=1 y i log(ŷ) 17:

IV. EXPERIMENT AND ANALYSIS
In this section, experimental details with hyperparameters and comparative analysis with similar work are presented to confirm the usefulness of our presented model.

A. DATASET
We conducted experiments on a publicly available dataset from the repository of the center for advanced studies in adaptive systems (CASAS) at Washington state university [28]. The dataset was composed in the dwelling of one volunteer adult. The residents in the home were a female and a dog. The motion detectors, door closure sensors, and temperature sensors are placed at different locations in the home, and the placement of sensors can be seen in Fig. 5 and dataset characteristics is presented in Table. 2. A detailed description of the dataset is available in [29], while a sample sensor state sequence is presented for kitchen activity in Fig. 6.  In Fig. 6, motion sensor 'M08' and 'M017' is not located in the kitchen as well as performed activity in the kitchen activity but still recorded as ON/OFF state. It might be possible the dog is wandering in the house. Such sensor event can make the activity recognition challenging, while a provided annotation is Kitchen activity. Our proposed model is robust to deal with such noise that leads to an accurate recognition of household activities. The activities are grouped into meta-classes and presented in Table. 3.

B. RESULTS AND DISCUSSION
This section presents the model parameters, evaluation metrics, and obtained results to assess the performance of meta-class sequence model. The model optimal parameters for ( ) and ( ) are presented in Table. 4.
During the training phase, we fine-tuned the model parameters until their convergence as shown in Fig. 7.
In Fig. 7, the convergence graphs show the optimal parameters selection of the model. The training and validation set curves present the learning of the model is smooth while loss value is minimized in each individual meta-class. Data augmentation based on Random OverSampling (ROS) [25]    our case it was effective to train the model. It can be seen the training curves presented in Fig. 7. The experiments are carried out on a Macbook Pro machine with a 10-core central processing unit, 32-core graphical processing unit, 16-core neural engine, and 64 gigabytes of RAM using Python.
The proposed framework considers the number of physical sensors reading as a sequence of sensor events along with annotated labels. The training of the deep meta-class sequence model enabled the unseen sensor stream to classify them into daily life activities. The model is based on supervised learning mechanism, which is applicable on different sensor streams and generalized to behavior monitoring in home setting. It is developed for elderly persons living alone in the home to maintain their independence, hence the number of users will be limited to only one. Furthermore, the living spaces can be easily transformed into intelligent spaces with the considered simple tapped sensors, without disturbing the privacy and aesthetics of the home. To evaluate the model, precision, recall, and F1-Score are calculated, and reported the obtained results in Table. 5 to 336 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.      Table. 11. The single inhabitant activities are recognized accurately, except for the mediate activity. In the case of mediating our model didn't recognize this activity. One of the possible reasons is the small number of sensor events generated during this activity. While all activities performed in a home setting from personal hygiene, kitchen, leisure, medicines, sleep, leaving home, and work is recognized successfully. It shows that model can be integrated into the digital twin for healthcare services.

C. COMPARATIVE ANALYSIS
We compared our model results with sequence modeling algorithm as well as classical approaches from supervised learning models. In case of sequence modeling algorithm, VOLUME 11, 2023 337 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.     Recurrent Neural Network (RNN) is considered for comparison. While in case of classical approaches such as Random Forest (RF) and Gradient Boosting (GB) algorithms are considered. The choice of these models is based on their performance as compared to existing models. The RNNs are constructed with high dimensional hidden states with non-linear dynamics [30]. These high dimensional hidden states worked as a memory units of network. For fair comparison, we used the same dataset, same number of units, batch size, optimizer, and epochs. (i.e., presented in Table. 12). The model is implemented in Python using Keras library [31]. In case of classical approaches, the random forest is a meta estimator which is based on multiple decision trees and sub-samples the training data. Later, it uses averaging to improve the classification   results. Similarly, gradient boosting is an additive model for classification that combine the weak learners to make a strong model. These models are implemented in Python using an open-source machine learning library scikit-learn, and parameters are presented in Table. 12.
The experiments are performed on the same dataset. In contrast, our model is a meta-class sequence model based on a deep learning to classify the behavior of residents. The comparison with such models is presented in terms of precision, recall, and F1-score. The Fig. 8 presents the personal hygiene, where our model has accurate results as compared to RNN, RF, and GB.
In kitchen activities, 'dining room activity' is not recognized by baseline sequence model RNN and RF while GB results are low in the case of recall and F1-score, while our model has the F1-score 100% as well as 99% in case of kitchen activity. Similarly, leave home/sleep and leisure activities are recognized accurately as compared to its counterpart in Fig. 14 and Fig. 10 except 'mediate' activity.
In the case of bedroom activities (i.e., Fig. 12), our model performs well as compared to existing counterparts.   The existing models F1-score is zero to classify the sensor events as bed-to-toilet activity. In the case of 'chores' activity, RF and RNN couldn't classify it while GB results is low. Taking Medicines is an important activity and our model is able to recognize it correctly in both cases of morning and evening medicines as shown in Fig. 13

V. CONCLUSION
Digital twin has ability to replicate the elderly residence in virtual home to provide enhance healthcare services. In this paper, we proposed a framework to support healthy aging which is based on physical and digital spaces. The physical spaces are elderly home enhanced with off-the-shelf sensor technology. While digital space is constructed in digital twin to monitor the behaviour and a deep meta-class sequence model is proposed to embed the intelligence. The proposed model has ability to monitor the daily life activities for understanding routine patterns of elderly population. In healthcare, one of the critical metrics is the robustness to deploy the models in real healthcare system. Our proposed model is robust in terms of behaviour monitoring. It is evaluated on publicly available dataset and compared with the base-line models to present the achieved improvements. The obtained results allow the real-time data-analysis in digital twin and behaviours monitoring.
Our future work is to include personal wearable devices to provide wide range of healthcare services.