TENSORFLOW-BASED SMART HOME USING SEMI-SUPERVISED DEEP LEARNING WITH CONTEXT-AWARENESS

In recent years, Smart homes play a significant role in improving the quality of human life due to the rapid proliferation of the Internet of Things (IoT) technology. The previous research works on the smart home system have adopted the machine learning and deep learning algorithms to predict the sequential activities in the smart home. This work presents a model of SMART home automation with Context-Awareness using Stacked AutoEncoder (SAE) Long Short-Term Memory (LSTM) in TensorFlow (SMART-CAST). The SMART-CAST approach comprises three main processes, including the integration of internal and external home data, SAE-assisted unsupervised learning, and LSTM with back propagation-assisted supervised learning. By inter-linking the spatial and temporal attribute-values, the SMART-CAST unifies the smart home and weather data for facilitating decision-making. It employs the SAE to generate the compressed representation of the unified smart home data from the unlabeled information. In consequence, the SMART-CAST approach applies the LSTM with the extracted compressed representation for learning the labeled data and updates the weight of the LSTM through backpropagation to predict the sequential activities in the smart home system. To further improve the decision-making performance, the experimental model executes the proposed semi-supervised deep learning algorithm in the TensorFlow deep learning framework.


INTRODUCTION
In recent years, the rapid proliferation of the Internet of Things (IoT) technology [1] has gained significant attention among the researchers to provide smart services to Internet users. Owing to the continuous generation of the stream data from the IoT devices, the accuracy and speed of the processing become very crucial factors in the different smart applications [2] such as smart city, smart agriculture, smart transportation, and smart home. Smart home [3] includes a different kind of IoT applications such as lighting control, temperature modulation, security access, and home automation. In contrast to the traditional home, the main aim of the smart home is to provide smart services to the users for their satisfaction and enjoyment. Despite the dramatic growth in IoT technologies, the researchers confront several constraints while developing smart solutions due to the massive generation of IoT data [4]. To extract the knowledge from the data generated by the IoT devices for smart services, the existing research works [5] have applied machine learning and deep learning techniques. In the IoT applications, the scarcity of the class labels in the IoT generated data leads to the difficulty in developing the solutions for the smart services. Hence, several researchers have employed the semi-supervised learning models [6] for handling the paucity of the unlabeled data streams in smart applications. Nowadays, deep learning or deep structured learning [7] plays a vital role by providing the classification, prediction, and function approximation capabilities to deal with large-scale data. It enables the computational models with multiple processing layers to precisely learn the data representations based on the multiple levels of abstraction. Deep learning algorithms [8] have been widely used in different processes of smart applications such as object detection, speech recognition, visual object recognition, and so on. It enforces the decision making process in the IoT applications through deep extraction of information from the vast amount of data streams.
Deep learning algorithms provide higher accuracy in recognizing the inherent data patterns over the heterogeneous and abundance of data streams. To easily implement the deep learning algorithms, Google's TensorFlow deep learning framework [9,10] has been widely utilized by the researchers. It relies on the data flow model to perform the computations, which supports the scaling of the training to the large-scale data by parallelizing the execution and replicating the core data flow model. Hence, the SMART-CAST approach implements the proposed model using the TensorFlow framework. Moreover, automating the smart home system with the consideration of only indoor activities is ineffective due to the high level of inter-relations between the smart home 3 TENSORFLOW-BASED SMART HOME activities and the environmental conditions such as lighting and temperature control. Hence, instead of extracting the information only from the indoor smart home devices, this work targets to utilize the information from the environmental conditions and employ the semi-supervised deep learning model such as Stacked AutoEncoder (SAE) and Long-Short-Term-Memory (LSTM) to attain benefits from the massive unlabeled data streams generated from the smart home IoT devices.
Furthermore, this fuses the proposed algorithm with the TensorFlow model to effectively improve the performance of smart home automation.
The significant contributions of the SMART home automation with Context-Awareness using SAE-LSTM in TensorFlow (SMART-CAST) are summarized as follows.
• This work presents a TensorFlow based smart home system that effectively handles both the unlabeled and labeled data with the contextual information, which improves the performance of the decision-making.
• By integrating the internal home data with the external home data based on the spatial and temporal information, the SMART-CAST approach enriches the input knowledge of the smart home system and facilitates the smart activity prediction.
• In the semi-supervised deep learning SAE-LSTM model, the SAE algorithm is the unsupervised learning model that extracts the knowledge from the unlabeled input data and reconstructs the activity patterns.
• After extracting the compressed smart home activity patterns from the SAE, the proposed approach learns the labeled smart home data using the LSTM model with the backpropagation for better decision making.
• Thus, the experimental model executes the semi-supervised deep learning model based on the computational graph of the TensorFlow model for the smart home system.

RELATED WORKS
Nowadays, with the rapid increase of the massive and diverse data in real-time, especially IoT applications, deep learning techniques have been extensively used in the large-scale environment.
Hence, this section briefly reviews the TensorFlow deep learning framework based approaches and deep learning approaches designed for the various smart applications.

TensorFlow based Deep Learning Approaches
To support smart tourism in an art city, the research work [11] presents the mobile application with smart activities based on the neural networks. By employing the visual recognition strategies in Google TensorFlow, it effectively recognizes the artistic heritage in an urban environment through the smartphone's camera. An efficient infrastructure paradigm, TensorFlow agents [12] allow the computation of the neural network that is reinforcement learning on the batch processing.
By modeling the Batch Proximal Policy Optimization (BatchPPO), it performs the parallel computation without interference in the neural network processing. The research work [13] applies the Convolutional Neural Network (CNN) based on TensorFlow for the application of face recognition. By handing the uncontrolled features such as illumination, facial expression, occlusion, and pose, it recognizes the face from the images. Owing to the repeated access to the smart devices for the same purpose in the smart home, the research work [14] designs the proactive smart home manager to predict and suggest the next steps to the users with the help of the logistic classification algorithm. It exploits the TensorFlow to implement the logistic algorithm and reduces the processing steps through the analysis of user behavior patterns. To improve the reliability and efficiency of the spacecraft, and improve the autonomous coordination among the spacecraft, the research work [15] executes the TensorFlow graph on the SmallSat platforms. It develops the space applications based on both the TensorFlow and TensorFlow Lite. To address the energy wastage and high traffic in the smart home network, the research work [16] designs an intelligent model for the IoT application services in a smart home. It comprises three intelligent models, such as Intelligence Awareness Target as a service (IAT), Intelligence Energy Efficiency as a service (IE 2 S), and Intelligence Service TAS (IST). By applying three intelligent models, it alleviates the unnecessary network tasks based on the IoT usage patterns in the smart home. An integrated model [17] combines the CitySim energy simulator with the TensorFlow with the assistance of the Keras API. It enables the investigation of the learning algorithms to exemplify the significance of the diverse applications and illustrate the robustness of the deep reinforcement learning controller. TensorFlow-based smart parking model [18] minimizes the wastage of time by allocating the free parking spaces and navigating the driver in the parking area along with the dependency of the Google Cloud. To design the smart parking application, it implements the CNN algorithm and develops the navigation bot using a deep reinforcement learning algorithm using TensorFlow. A smart parking approach [19] detects the availability of the parking space by continuously monitoring the status of the vehicle through different IoT devices such as Passive 5 TENSORFLOW-BASED SMART HOME Infrared Sensor. It employs the TensorFlow for implementing the machine learning algorithm, which is responsible for determining the collection of vehicles, whether located in the parking slot or availability in the parking slot.

Deep Learning Approaches for Smart Applications
The semi-supervised deep reinforcement learning model [20] improves the accuracy and performance of the learning agent in the smart city applications by utilizing both the labeled and unlabeled data. It generalizes the optimal policies by applying the inference engine, such as the Variational Autoencoders (VAE). An intelligent transportation system [21] optimizes the traffic light control by applying the Deep Q-Network (DQN). Initially, in a single inter-section, it obtains the 'thresholding' policy by establishing the DQN algorithm and then, across several intersections, it analyzes the scalability and performance of the DQN algorithm in a linear network topology. To resolve the complexity in urban traffic control over the multiple intersections, the research work [22] employs a deep reinforcement learning algorithm with a suitable control strategy in a short period. It mitigates the human invention and assumption of the fixed traffic patterns while tuning the parameters in the deep learning model. Smart agriculture IoT system [23] integrates artificial intelligence techniques such as the deep reinforcement learning algorithm and cloud computing to increase food production. It comprises four smart layers such as the data collection layer, edge computing layer, data transmission layer, and cloud computing layer for making the smart agricultural decisions involve the identification of the amount of required water for the crop growth improvement. To predict the sequential activities of the humans in the smart home system, the work [24] explores the performance of the sequence prediction techniques with the real-life smart home datasets. From the analysis of the sequence prediction techniques, it is determined that the LSTM model provides better performance in predicting the sequential activity as well as the timestamp of the subsequent event in the smart home.

AN OVERVIEW OF THE PROPOSED METHODOLOGY
The smart home management system plays a significant role in the day-to-day life of humans.
The primary aim of the home automation system is to ensure easier, and high quality lives for the people. Modeling a completely automated system is crucial for the smart home system. In the smart home system, there is a massive amount of data continuously generated from different human activities. Instead of only considering the indoor activities in the home for the decision-making, utilizing the outdoor information such as environmental changes in the smart home decision-making is crucial. It is because the home automation system heavily relies on outdoor factors such as weather conditions significantly impact the triggering of the IoT devices inside the smart home system. Thus, the SMART-CAST approach integrates the indoor IoT data with the outdoor IoT data and then applies the semi-supervised deep learning for the home automation using the TensorFlow. The proposed semi-supervised deep learning model is the combination of the stacked autoencoder and LSTM model, which plays an essential role in predicting the flow of operations in the smart home system.

Contextual Feature Integration:
Initially, the SMART-CAST approach analyzes the spatial and temporal fields in the smart home data source and the environmental data source to build the integrated features of the input data. After analyzing the spatial-temporal features, it applies the attribute-level matching to integrate the inter-linked instances across two data sources. In essence, the SMART-CAST approach extends the features in the smart home data sources with the features in the environmental data sources.

THE PROPOSED METHODOLOGY
Owing to the massiveness of the IoT data, the SMART-CAST approach focuses on extracting the insights with the context awareness using the machine learning techniques. With the target of incorporating the context recognition in the smart home, the SMART-CAST approach integrates 7 TENSORFLOW-BASED SMART HOME the events and activities that occurred in the home with the changes occurred in the environment.
In essence, the interaction between the smart home and environmental conditions ensures a low cost, sustainable and personalized service, and increased flexibility in the home control. Thus, this work suggests the solution to the automated home system even without user intervention.  Hence, the SMART-CAST approach focuses on applying the semi-supervised deep learning method to accurately as well as efficiently extract the patterns from the data for home automation in the TensorFlow framework. Consequently, the SMART-CAST approach controls the smart home devices based on the surrounding environmental conditions and delivers the required service at the right time

Integrating the Features of the Indoor and Outdoor Data
Initially, the SMART-CAST approach utilizes the IoT data from both the sensors in the home and sensors outside the home to generate unified data for precisely predicting the activities in the smart home environment. The sequential activities in the indoor environment heavily rely on the outdoor environment in the smart home system, such as the environment changes based triggering activities inside the home creates a more significant impact on home automation. To measure the home conditions, a smart home consists of a set of measured activities such as light, proximity, object recognition, clock-time recognizer, and so on.

S. UMAMAGESWARI, M. KANNAN
In the context of the proposed system, to enable the right activities at the right time, the proposed model gathers both the internal and external home data to measure the home state and provide the required response to the smart home system. In a nutshell, the proposed smart home model exploits two different information sources, such as the smart home data source comprising the data generated from the smart IoT devices in the home and the environmental data source comprising the data generated from the IoT devices related to the weather conditions. It utilizes the monitored information of both the house and the outdoor environment. To determine the aspect or the context of the required activity in the smart home, extracting and exploiting the information from the data of environmental changes is crucial. Accordingly, the SMART-CAST approach considers two essential features of both the house IoT data and weather IoT data, such as spatial and temporal features, while inter-linking such two data sources. IoT systems have continuously TENSORFLOW-BASED SMART HOME evolved naturally over time. Hence, to support the adaptive learning mechanism, the SMART-CAST approach considers the temporal information along with the geospatial information during the integration of two data sources. The spatial-temporal characteristics play a significant role in identifying the interaction between the data that co-occur in both space and time. For the integration process, the SMART-CAST approach employs the attribute-level linking method with the contemplation of the geospatial and temporal features or attributes. In the proposed system, data integration is also referred to as the reconciliation of the different instances based on the

Extracting the Representative Patterns using a Stacked Autoencoder
The SMART-CAST approach employs the TensorFlow deep learning model to enforce the decision-making based on the semi-supervised deep learning model for the smart home system.
As depicted in Figure 1

Generating the Reconstructed Patterns
In the smart home system, the unsupervised learning model employs the stacked autoencoders to facilitate the feature learning for decision making and effective representation of the integrated smart home inputs. Initially, the SMART-CAST approach analyzes the features and generates the sample that represents the input data process, which is performed as the pre-training process in the TensorFlow. By extracting the potential features of the smart home data using the stacked autoencoder, the SMART-CAST approach generates the latent discriminative features based on the analysis of multiple hidden layers.
The autoencoder method is one of the deep learning techniques, which is an unsupervised learning model targeting the conversion of the inputs into outputs with a minimum possible variation. It comprises two main parts, such as an encoder and decoder, converting the input data into low-dimensional data representations and generating the reconstruction data from the data representations, respectively. In the semi-supervised deep learning model, the supervised learning model utilizes the extracted features of the original smart home input data, which is extracted from the encoder to obtain the inherent data structure. In the autoencoder model, the decoding is only In the stacked autoencoder, the sequence of the input data is represented as xi = {x1, x2, ..., xk}.
The compressed sample that is the representation of the input data extracted from the encoder of the stacked autoencoder model, which is denoted as xs' = {x'1, x'2, ..., x'n}. Where 'i' and 's' refer to the input of the smart home data and the sample of the smart home data, respectively. The output of the autoencoder model is fed as input into the decoder of the autoencoder model in the smart home system. The stacked autoencoder is the composition of the multiple layers in which a single autoencoder learns each layer. Initially, the first autoencoder learns the smart home input data and obtains the first-level features. In subsequence, the first-level features or representations are used as the input to the second autoencoder to obtain the second-level parameters and representations.
In the stacked autoencoder model, such procedure repeats until reaching the last autoencoder, which results in the high-level representations with low dimensions. The SMART-CAST approach employs the stacked autoencoder model to learns the unlabeled data and validates the reconstructed output using the cost function. (2)

Predicting the Sequential Activities in the Smart Home using LSTM
To accomplish effective prediction in the smart home system, the SMART-CAST approach applies the unsupervised learning on the unlabeled data for feature extraction and supervised learning as the fine-tuning on the minimum amount of labeled data along with the knowledge of the extracted sequential characteristics. It yields the combination of the autoencoder and the LSTM model over the input of the integrated internal home and external home data. With the target of improving the prediction performance of the LSTM, the SMART-CAST approach computes the prediction error and applies the backpropagation over the LSTM model.

Decision-making using LSTM Back Propagation
To support the sequences of the smart home input data, the SMART-CAST approach employs the LSTM network, which can learn the complex dynamics with the consideration of temporal ordering of the sequential input. Moreover, the LSTM network remembers the information over the long sequence of the input using its internal memory. It improves the hidden layer based on the concept of the RNN, which addresses the gradient disappearance problem in RNN. With the accumulation of the input gate, forget gate, and output gate in the LSTM, the memory units decide which data to be deleted or to be retained. indicates the error in the ith memory block of the LSTM model. ɑ and β denote the weighted parameters, 0<ɑ, β< 1 and ɑ+β =1. The SMART-CAST approach provides higher importance to the weight calculation of the first term that is 'ɑ' compared to the second weight term that is 'β'.
According to the non-linear activation function in the LSTM layer, it computes the error function and updates the weights based on the partial derivative of the current weight at a particular iteration and the error value. The SMART-CAST approach leverages the LSTM model to learn the labeled data through forwarding propagation over the sequence of k1 timesteps and updates the activation function based weight through backpropagation at k2 timesteps. This weight updation process continues until reaching the best convergence in the decision-making.

CONCLUSION
This work proposed the smart home automation model, the SMART-CAST approach that predicts the sequential activities from the combination of internal and external home data using the semi-supervised deep learning model, SAE-LSTM. Initially, the SMART-CAST approach integrates the internal smart home data attributes with the external weather attributes by matching the spatial and temporal information to enhance decision-making performance. The SMART-CAST approach employs the SAE for unsupervised learning that generates the compressed smart home activities from the unlabeled data and also employs the LSTM for supervised learning that predicts the sequential activities in the smart home from the labeled data along with the compressed data knowledge. It dynamically updates the weight of the LSTM model through backpropagation over time, which improves the accuracy of the activity prediction. The SMART-CAST approach employs Google's TensorFlow to implement the semi-supervised deep learning model with the target of improving efficiency.