COMPARISON OF DEEP LEARNING SEQUENCE-TO-SEQUENCE MODELS IN PREDICTING INDOOR TEMPERATURE AND HUMIDITY IN SOLAR DRYER DOME

: Solar Dryer Dome (SDD), which is an agriculture facility for preserving and drying agriculture products, needs an intelligent system for predicting future indoor climate conditions, including temperature and humidity. An accurate indoor climate prediction can help to control its indoor climate conditions by efficiently scheduling its actuators, which include fans, heaters, and dehumidifiers that consume a lot of electricity. This research implemented deep learning architectures to predict future indoor climate conditions such as indoor temperature and indoor humidity using a dataset generated from the SDD facility in Sumedang, Indonesia. This research compared adapted sequenced baseline architectures with sequence-to-sequence (seq2seq) or encoder-decoder architectures in predicting sequence time series data as the input and output of both architecture models which are built based on Recurrent Neural Network (RNN) layers such as Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM). The result shows that the adapted sequence baseline model using GRU is the best model, whereas seq2seq models yield bigger Mean Absolute Error (MAE) values by almost ten times. Overall, all the proposed deep learning models are categorized as extremely strong with 𝑅 2 ≥ 0.99 . Their results show that


INTRODUCTION
Indonesia has implemented the Indonesia Agriculture 4.0 programs, which means that the agriculture system should consist of Artificial Intelligence (AI) or Machine Learning (ML), the Internet of Things (IoT), and cyber-physical systems. One of those programs is Smart Dome 4.0, a low-cost, eco-friendly, and sophisticated program to support Indonesian farmers in saving their agricultural products [1]. The purpose of building a Solar Dryer Dome (SDD) is for food preservation and maintaining the product's nutritional content because agricultural products require a long time to process before they are delivered to consumers [2]. SDD overcomes the many shortcomings of traditional drying methods under the sun in the outdoors, such as longer drying processes, potential rain, dust impact, bird and other flying animal droppings, the growth of fungi, inappropriate humidity, and color change.
One weakness of SDD is the need for a power source for running the system continuously to provide suitable indoor environmental conditions with a constant supply of electricity for operating the actuators such as fans, heating systems, and dehumidifiers [1] [3] [4]. SDD uses green energy by collecting solar energy using a solar panel during the day and storing it in a battery for use at night. Indonesia, a country with two seasons, has various solar radiation distributions, so it can become a problem for SDD for solar energy absorption [5]. When the weather is dark and rainy throughout the day, it also becomes a problem. In a mountain area, dramatic weather changes also affect indoor SDD significantly. Another study on SDD concludes that controlling indoor climate by increasing indoor temperature and decreasing indoor humidity consumes the most power [6].
It makes predicting environmental parameters for scheduling the actuators an important thing for SDD. The application of the actuator scheduling can reduce SSD power consumption by using Predicting indoor climate for controlling SDD environmental conditions in order to achieve power consumption efficiency and the best quality of dried agricultural products is one of the most important and difficult tasks to perform for SDD [7]. Deep learning (DL), a method that learns the distribution of the data to be modeled automatically, can be applied to address these challenges in the agriculture sector, especially DL with RNN [8] [9]. Many reports show how amazing RNN can solve various challenges where the data is sequential [10] [11]. RNN can learn historical information in time series data with the aim of predicting future results [12]. Later, special improved RNNs such as LSTM and GRU appear, which are intended for long-term learning. [13].
Sequence-to-sequence (seq2seq) or encoder-decoder is one of the deep learning architectures which is popularly implemented in Natural Language Processing (NLP), which can output sequence data with sequence data input [14]. Since the input and output data are sequence, this research compared both the adapted sequence baseline architecture and the seq2seq architecture, which applied RNN layers on it, so the proposed 4 models are adapted sequence baseline with stacked GRU, adapted sequence baseline with stacked LSTM, seq2seq with GRU, and seq2seq with LSTM.

RELATED WORKS
For many years, DL has been a major improvement in ML research, solving high-dimensional data problems. [15]. DL is used in many domains of science, business, and government. The most popular deep learning methods which are used for predicting indoor climate problems are Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), a simplified LSTM.
There is also some research which used deep learning to predict indoor climate problems. The closest work to our research is the research done by Gunawan et al. [16]. Another indoor climate prediction research was done by Liu et al [19]. Their research implemented time sliding window to their LSTM model for learning the change of environment climate over short a period of time. Their datasets were tomato, cucumber, and spicy greenhouse with indoor temperature and humidity, light intensity, carbon dioxide concentration, soil temperature, and soil humidity. Their modified LSTM outperformed the GRU model. Elhariri and Taie conducted a similar study to SDD in which they experimented with Heating, Ventilation, and Air Conditioning (HVAC), an indoor system similar to SDD [20]. They compared LSTM and GRU models to predict the future microclimate inside smart buildings by using UCI Machine Learning Repository SML2010 datasets containing indoor temperature and humidity, carbon dioxide concentration, and outdoor temperature and humidity. The result showed that the GRU model was the best in their case.
The research that is closest to ours, which implemented the seq2seq architecture as a model time series prediction, was done by Fang et al. [21]. They predicted an indoor climate inside the GreEn-ER building in the center of Grenoble, France, with the datasets containing indoor temperature and carbon dioxide. They proposed 3 seq2seq models, such as LSTM-Dense, LSTM-LSTM, and LSTM-Dense-LSTM, which outperform the LSTM and GRU baseline models.

Datasets
The dataset which was used in this experiment was generated from a SDD facility in Sumedang, a town in Western Java, Indonesia. The facility can be seen in Figure 1.  This research only addressed indoor temperature and indoor humidity, even though all models forecasted all six features by using all six features as input and output. Because the two primary factors that affect SDD, which need to be monitored and controlled, are indoor temperature and indoor humidity. So, the results of the outdoor temperature and the outdoor humidity were ignored. 7

Long Short-Term Memory
Due to its capacity for memorizing temporal information over a large number of timesteps, Long Short-Term Memory (LSTM) is frequently employed in classification and regression tasks involving sequential data [23]. LSTM is composed of three gates, which are the input gate, output gate, and forget gate. An LSTM was designed well to handle time series predictions and is also a solution for problems which require temporal memory [24].
The input gate in equation (1) is denoted as ( ) where ( ) , ( −1) , and ( −1) are the representations for the input data, last iteration output data, and last iteration cell value respectively with , , and as weight values. The bias vector of input gate in LSTM is indicated by the symbol of . The symbol of denotes the sigmoid activation function.
The forget gate in equation (2) is denoted as ( ) which eliminates the information from previous cell state where , , and symbolize the weight values for input data, last iteration output data, and last iteration cell value respectively. The bias vector of forget gate in LSTM is symbolized as .
The cell value in equation (3) is denoted as where ( ) is the block input.
The output gate in equation (4) is denoted as ( ) where , , and are the weight values for input data, last iteration output data, and last iteration cell value respectively.
The block output of LSTM in equation (5) is denoted as ( ) which combine current cell value and the output gate in LSTM where ( ) is hyperbolic tangent function.  [25]. GRU mostly outperforms LSTM in many cases [26]. If LSTM has 3 gates, which are input gate, forget gate, and output gate, GRU has 2 gates, which are reset gate symbolized as and update gate symbolized as [27].

Gated Recurrent Unit
where: ⊙ is element-wise multiplier; , , and are weight value; is input data; ̃ is candidate state; is output; , , are constants; and ℎ are sigmoid and tanh activation function.

Adapted Baseline Sequence Models
This research modified the models implemented by Gunawan et al. to handle sequence inputs and sequence outputs [16]. The adapted baseline sequence models consisted of the two most popular RNN layers, which are LSTM and GRU, which can be seen in Figure 3. Since the datasets in this research were processed as 3-dimensional (3D) data because of the sliding window process, the 3D data input as ( , , ) of adapted baseline sequence models were reshaped to 2D data ( , × ) with representing the amount of sliding window process, representing timestep, and representing the number of features. The reshaping process is illustrated in Figure 4. Because the output of adapted baseline models is 2D data ( , × ), the output needed to be reshaped back to 3D data ( , , ).

Sequence-to-sequence (Seq2seq) or Encoder-Decoder Models
The encoder-decoder or sequence-to-sequence (seq2seq) is also part of deep learning, which originated from machine translation problems, where at the beginning of its appearance, the seq2seq architecture could empirically perform well for translation tasks from English to French [28]. Seq2seq consists of two Recurrent Neural Networks (RNN) which act as the encoder and decoder. The Seq2seq architecture is mostly used for language processing models [29] and has rarely been used for indoor climate forecasting [21]. The Seq2seq model also performed well in predicting time-series data, like predicting Beijing PM25 datasets, energy consumption in Sceaux, highway traffic in the UK, Italian air quality, and California traffic with PeMS-Bays datasets [30].  This research implemented two simple seq2seq architectures with RNN layers such as LSTM and GRU used in the encoder and decoder layers, which can be seen in Figure 5 and 6. Both seq2seq architectures implemented batch normalization between encoder and decoder. Batch normalization can improve the accuracy and generalization [12], and accelerate the training process, which makes it one of the favorite techniques in deep learning [31], because seq2seq is more complex than our adapted baseline models.
To obtain suitable hyperparameter settings for the seq2seq model, this research observed and understood the early training with a short run of 10 epochs by doing random search, because by doing that, it could be a clue for suitable model settings without consuming time and expensive computational resources [32]. The result of random search observation showed that 64 neurons for each GRU and LSTM layer and a learning rate with 0.00001 provided the best results. Then this research equated to 64 batch size, 100 epochs, and implemented Adam optimization algorithm.
Adam was chosen because it has advantages over other adaptive learning rate optimization algorithms, including the ability to handle non-stationary objectives like RMSProp and manage sparse gradients like AdaGrad [33].

Pearson Correlation
To find out the correlation between each parameter, Pearson Correlation Coefficient (PCC), which is denoted as was implemented where and are the compared parameters, ̅ and ̅ are the mean value of and respectively [34]. The PCC results will be in the range [-1,1] where = −1 means that the correlation is extremely negative and = 1 is conversely [35].

Performance Metrics
The most commonly used performance metrics which are implemented in regression analysis cases in machine learning studies are Mean Absolute Error (MAE), Mean Square Error (MSE), and Root Mean Square Error (RMSE) [36]. In fact, each error measurement has different disadvantages that can lead to inaccurate evaluation of forecasting results, which makes it not recommended to only use one measurement [37]. This research aimed to forecast indoor temperature and humidity in the future, which made MAE and RMSE an ideal choice for collecting error information in the model. This research also implemented the coefficient of determination ( 2 ) because of its potential to compare ground truth elements with predicted data considering its distribution [36].
In MAE, RMSE and 2 equations, is the predicted value at ℎ , is the ground truth data at ℎ and ̅ represent mean of ground truth data. Both MAE and RMSE results must be in range [0,∞) with the best value is closer to 0, meanwhile in 2 result will be in range (-∞, 1] with the best value is closer to 1.
2 value describes the proportion of variance in a variable which is affected by another variable [38]. 2 can be categorized as strong when 2 ≥ 0.75 and weak when 2 ≤ 0.25 [39]. Meanwhile between both strong and weak, there is moderate 2 value.

Experimental Environments
In

Preprocessing Dataset
The datasets, contained enormous datasets with appropriate time series data, which made them suitable for deep learning models [40] [41], as shown in Figure 2. This study divided the dataset into two parts: training data and test data, with a percentage of 80% and 20%, respectively. The model presented in this study consumed training data in the training step, with 80% of the training data being used to train the model and 20% of the training data being used to validate the model.
Because extremely high or low data values can trigger the models to overfit, data standardization was used in this experiment research to assist the models in learning the data [42].
In this study, Z-score standardization, designated as ( ) with mean ( ) and standard deviation as ( ) and, is used. Figures 8 show the outcomes of applying Z-score standardization to 14 SETIAWAN, ELWIREHARDJA, PARDAMEAN our datasets. Figure 8 show how standardized datasets were divided into two parts: training and testing.
Relative humidity data in orange and temperature data in blue were used to train models, while relative humidity data in red and temperature data in green were used to test models. represents the timesteps number of input or output data. This research aims to predict data in the future 5 timesteps based on 150 previous data, because five timesteps of predicted data should be enough to support SDD operational. Figure 9 illustrates the sliding process.

Model Results and Comparison
This research compared all prediction models by using testing data, which was untrained data, or a fifth of the original datasets. Untrained data, depicted in Figure 8, as red and green lines on the line chart, was fed into the model, yielding predicted data ( ). Then this study compared both predicted data ( ) and ground truth data ( ) with MAE, RMSE, and 2 as performance metrics.
Since the prediction result was standardized data with Z-score, the results needed to be converted back to real ranges. The prediction model results ( ) and ground truth data ( ) contained 6 features, including indoor temperature and humidity 1, indoor temperature and humidity 2, and outdoor temperature and humidity with 5 timesteps. Both sets of data were compared with MAE, RMSE, and 2 as performance metrics.   The results of indoor humidity prediction testing based on Tables 3 and 4    handle a dataset containing extremely strong positive PCC values between the same temperature parameters or humidity parameters and extremely negative PCC values between temperature parameters and humidity parameters. A quick glance at all the testing result tables shows that seq2seq models produced ten times higher error than adapted sequence baseline models, but all models were still good at predicting indoor climate with an average of ≤0.5 for indoor temperature prediction and ≤ 1.2 for indoor humidity prediction. With coefficient of determination, all of the models can be categorized as strong in with 2 ≥ 0.99 [39].
The dataset used in this research contained a large amount of real-time sensor data, which had the possibility of containing noise data [44]. So, for future work, the next research will implement Kalman filtering to correct the data from noise in order to increase the accuracy of all models.

CONCLUSION
The results show that in processing the dataset, which contained only extremely strong positive or extremely strong negative PCC values between each other parameter, both our adapted baseline models outperformed both our seq2seq models. All models were good at predicting indoor temperature and humidity because they had a relatively small error number in MAE and RMSE.
The coefficient determination values of all models were also categorized as strong, with 2 ≥ 0.99.
Based on this research, the curiosity arose because seq2seq models still have the potential to be improved, such as by implementing attention layers. In future research, there is a plan to improve seq2seq architectures by adding an attention layer and stacking some RNN layers inside both the encoder and decoder layers and improving the case to be a more complex problem, such as increasing the number of timesteps in both input and output models. Because of the dataset containing a large amount of time series data captured by sensors inside SDD, there will be a future study on reducing noise from the dataset by using a filtering technique such as Kalman filtering.