Activity Recognition for Smart Building Application Using Complex Event Processing Approach

— Activity recognition has become one of the most interesting and challenging subjects in performing surveillance or monitoring of smart building system. Although there are several systems already available in the market, limitations and several unresolved issues remain, especially when it involves complex engineering applications. As such, activity recognition is purposely incorporated in the smart system to detect simple and complex events that happen in the building. In all existing event detections, the complex event processing (CEP) approach has been used for the detection of complex events. The CEP is capable of abstracting meaningful events from various and heterogeneous data sources, filtering and processing both simple and complex events, as well as, producing fast mitigation action based on specific scenarios. The work reported in this paper intends to explain in detail on the development of activity recognition application using CAISER ™ and NESPER © platform as well as the complex event detection that uses the CEP approach. In assessing the system performance, Matthew Coefficient Correlation (MCC) has been used as the main performance parameter. Results obtained showed that the Temporal Constraint Template Match Detector (TCD) is more accurate, stable and better in complex event detection compared to NESPER © detector.


I. INTRODUCTION
The evolution of smart buildings is increasing at a tremendous phase.It unofficially began in 1950 [1] when pneumatic actuator was used for building automation purposes.In 1960, the technology changed from air compressors to the analog controller and then to the digital controller.Then, the home automation became popular from the year 1970 to 2010 [2].The open protocol has been introduced in 1990, whereas the wireless technology was introduced since the year 2000.From 2000 onwards, home or building automation was established and stabilised with the presence of the internet and ambient intelligence.Smart buildings are equipped with the automation system facility and integration technology [3] to measure, monitor, control and optimize operations and maintenance, life security, telecommunication, consumer system and facility management systems [4].
Activity recognition is an important part of the smart building system. it involves the use of image processing and artificial intelligence.Activity recognition monitors the environment or infrastructure automatically, with minimal intervention by manpower [5].This system involves an automatic detection and traction as well as advanced analysis.There are several components that contribute to input data acquisition such as sensor devices, computer vision and social messaging.
There are a number of challenges when developing an activity recognition system in smart buildings such as detection of a complex event, handling and managing huge amounts of data, controlling the mechanical and magnetic motor for the event actor, etc.It is obvious that there are lots of data produced from the network of sensors attached, especially in a building.The data produced contains information that can be used for prediction and action.It is needed so that the user can extract meaningful information / events wisely.
Currently, among the most popular techniques for event detection includes Hidden Markov Model (HMM), Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighbour (KNN), Conditional Random Fields, Naïve Bayes and Tree (NBTree).Artificial Neural Network (ANN) based technique such as Multi-Layer Perceptron (MLP) is also quite popular among researchers in this field.[6].To further

Complex Event Processing (CEP)
increase the accuracy of detection, some researchers fused several detection techniques and selected the best-predicted result using the ensemble-based approach.Some researchers also used CEP based framework utilizing tools such as Drools, NESPER © , Siddhi, and APAMA to detect meaningful events as well as produce fast mitigation action for related events.These tools mostly use the Sliding Windows (SW) technique to detect complex events from a sequence of simple events.This paper highlights the development of activity recognition, which is a form of a complex event, in smart buildings using the CEP based event detector.

A. Terms & Definition
There are many terms used in event processing.The important terms are highlighted in Table 1.

Event
Something significant happens in real-time and can lead to an increase in other events Event Stream A group of events that arrive in a context to a processor that has an event topic.Simple Event / Raw Event / Primitive Event Refers to any change in events, either in the form of value or things taken out directly from the real world.Events that occur directly and not from a combination of other events.Complex Event / Composite Event Extraction from a combination of simple events or when added to previous complex events Event Producer Detection of changes in circumstances that come from a variety of sources and is represented as an event.

Event Processor
Identify and process events as well as generate appropriate mitigation actions.

Event Consumer
Event User receives information or notification about events.

Event Actor
The final component that receives instructions to implement mitigation action.Event Channel / Event Communication Routes that connect the event producer to the event processor and from the event processor to the event consumer.

B. Complex Event Processing
CEP is defined as an approach to acquiring, analyzing, detecting and processing voluminous events in real-time and produces a quick mitigation action based on specific scenarios.CEP consists of three main parts, which are the event observer (sources), event processor, and event consumer (sinks).The CEP concept is illustrated in Fig. 1.CEP is responsible for filtering and combining messages/notifications that predict higher level events that are then notified to sinks.CEP uses an event processing language (EPL) for event detection.The event represents the entity that occurs in the CEP field or state change [7], [8].Events consist of two main types, which are the simple events (smoke_triggered, temperature_high) and complex events (smoke_triggered followed by temperature_high that predicts the Fire_detected).CEP correlates several simple events to detect higher level complex events [7].It is commonly applied in environmental monitoring, financial trading, business process automation, and control systems.

C. CEP Engine
There are lots of CEP platforms present in the market such as Drools, RuleCore, Progress Apama, Coral8, StreamBase, Esper, etc.The performance of these CEP engines is summarised in Table 2.

TABLE II PERFORMANCE SUMMARY OF CEP ENGINE
Esper is an open-source CEP engine with two implementations, which are Esper© for Java and NEsper© for .NET.Esper uses rule-based as an inference engine and API C# as well as API Java in the platform development.Esper is usually used in production environments, provides better documentation (well-written, extensive documentation spanning hundreds of pages), easy to understand, has clear concept of combining composition operators with data stream constructs, provides an Eclipse plugin and is easy to install and use, can access data using the API for implementing languages and can accept database query from event queries.

D. CEP Architecture
There are four main modules for CEP system, such as Event Producer, Communication Channel, Event Processor, and Event Consumer.Fig. 2   diagram for the CEP system.The event producer is responsible for identifying simple events as an input stream and send it to the Event Processing Network (EPN) or complex event processor via suitable communication channels.The communication channel connects the input or output adapter to the EPN or complex event processor.There are a number of communication channels, such as TCP/IP, database, JSON, XML, web service, etc.The complex event processor is used to detect the complex events by using several correlation techniques, such as temporal correlation, logical or causal correlation, and spatial as well as dimension-based correlations.Event consumers or event actors are responsible for connecting to the action gate.The mitigation actions initiated could be unified messaging or triggering the actuator.

E. CEP Event Detection
There are two main methods for event pattern detection in CEP, which are Exact Match and Similarity Matching.

1) Exact Match
In this technique, all elements in event sequences need to be matched [9].This technique has a low computational cost.However, it is exposed to detection failure due to the noise and system error that normally comes with event sequences.The clean sequences will result in high accuracy and precision with fast detection and low computational costs.

2) Similarity Matching
There are several criteria used in similarity matching such as Hamming distance [10] and Editing operational numbers (delete, add, edit), which are executed to make both sequences similar [11].Whereas, [12] uses distance in between patterns and sequences approach to determine the level of similarity matching.These matching approaches have been used in Non-Finite Automata (NFA) [13] and in the artificial intelligence technique, such as Hidden Markov Method (HMM) [14], [15] applications.

F. CEP Application
There are several CEP applications that have been implemented.The application highlighted in this study is the engineering field.The conventional data analysis and mining system need human operators to analyse the huge volumes of generated data.In addition, the system still lacks the unknown event pattern identification.Hence, the automated technique was developed by using a combination of statistical learning theory algorithms and event processing systems.The complex event processing is highly recommended to be deployed since it is capable of obtaining various sources of events, predict what would happen and can be applied in real-time data analysis.

1) Wireless Sensor Network
The increase in sensor networks has led to the need for managing and detecting meaningful events (simple and complex).There are two sensor network approaches, namely Centralized and Distributed processing [16].The centralized approach sends input sources to the gateway for processing.However, this approach uses more energy and transmission latency causes a delay in event detection.Whereas, the distributed approach allows evaluation on the node before sending the events to the gateway.These approaches reduce communication overhead and increase the system's performance.
In [17], an engine that allows detection of complex events and meaningful information from raw/primitive events is proposed.There are three main parts in this system architecture, which are the server side, sink side, and node side.Results show that their in-network complex event will increase the lifetime of the network compared to the centralized approach

2) Intrusion Detection
The intrusion detection system is a critical system.The system needs to have the ability to process various sources of data in real-time and produce multiple actions such as notifying engineering personnel and also providing relevant data to stop the attempts.Hence, the CEP methods are highly suitable for adoption in this kind of system.A generic Intrusion Detection and Diagnosis System (ID2S), has been implemented in [18].These systems are capable of detecting and diagnosing complex intrusion scenarios in Large-Scale Complex Critical Infrastructures using comprehensive alert correlation workflows.They also use hybrid and hierarchical approaches for on-line detection and diagnosis.In addition, a CEP has been used for complex event correlation.

3) Smart Homes
A smart home [16] can be described as a home/building that is equipped and integrated with surveillance and ambient devices with the purpose of creating safety, security, comfort, communication, entertainment and technical management.The technology used in a smart home can be the Bluetooth, bus operated systems, X10 standards, main borne communication systems, radio frequency transmission, infrared communication, and insteon.CEP technology uses a framework infrastructure work best suited for the smart home application.Smart homes can increase safety, automate the task, monitor home activity remotely, enhancing energy efficiency, technology accessibility and provide comfort to the user.Smart home systems still need further development in terms of providing security for all connected items and also needs to find efficient ways to reduce energy consumption as well as faster and efficient actions that need to be taken.
DIGIHOME [19] aims to provide comprehensive and simple solutions in a pervasive environment whenever dealing with context processing.DIGIHOME uses Esper with a Java open source stream event processing engine to process event management and make a decision.

4) Activity Recognition System
CEP has been used in an activity recognition system.[20] uses CAISER™ as a CEP engine to monitor and analyse the complex event produced by the CCTV and door sensors.The strength of the CEP platform is in its complex event detection.In addition, these platforms provide multiple builtin classifiers for comparison purposes.The system has an average accuracy of more than 90%.However, these systems are limited to single people tracking.In [7], event-driven architecture (EDA) is used for data processing and will process in near real-time.The communication between data and other entities is via messages.[7] deploys the CEP paradigm in its system since it allows all EDA features, filter information, and enable high-level information to be inferred.The system combines EDA and CEP engines in the overall system.[19], [21] uses NESPER © in the CEP engine for the SAPHE (Smart and Aware Pervasive Healthcare) project.This system aims to monitor vital signs and patient activity.However, this system has a limitation of discovery new services and unclear interaction between actuators and the engine.

A. Region of Interest
Event recognition infrastructure has been installed in the Innovation 2 Laboratory, Faculty of Engineering & Built Environment, Universiti Kebangsaan Malaysia.Fig. 3 shows the exact Region of Interest (ROI) inside the laboratory environment.There are 9 CCTV units, 4 door sensor units, a door actuator, a lamp switch and an air conditioner switch.Whereas, there are 14 ROIs that are marked in order to obtain the actor movement reading.
Table 4 depicts the sample of complex events (CE) description.For example, Door1_NotClosedProperly is triggered when the door is opened and not closed for a certain period of time (e.g., 3    source software available under the GNU General Public License (GPL).

D. Rules Language & Tool
In general, event detection rules are represented ontologically to ease rule creation.This approach is used not only for CEP applications but in other applications as well, especially when the data comes from heterogeneous sources and the data analysis is made based not only on current information but utilizes historical data patterns as well.[24].For example, NESPER© uses SQL-like query language, which is ESPER© Event Processing Language (EPL), to represent the rules; whereas, CAISERTM uses its CAISERTM Event Processing Language (CEPL) to represent its rules.

E. CEP Framework
Fig. 4 shows the CEP framework for this research.The status of movements in ROI's, sensors and triggered switches are sent as simple events directly to CAISER™ event detector to be processed in online mode.The event detector then analyses the pattern of past and present simple events and past complex events to identify current complex events.For comparison purposes, this study extracted the simple events from CAISER's database and re-annotated the complex events so that comparative offline analyses could be made between CAISER's Complex Event Detector (CED) and NESPER©.
Table 5 depicts the details of the number of complex events that occurred in the provided dataset for experimental purposes.There are 26 types of complex events that occurred in the experiments with the total sequences of 972 simple events.The data was taken from the real-time data with several manual annotations.

F. Complex Event Detector
There are 4 CE detectors used in this experimental study, which comprises the sliding window detector (SWD), weighted sub-window sliding window detector (WSD), temporally constrained template matching (TCD) and NESPER © [22].Both NESPER © and SWD uses similar sliding window techniques for detection but different ones in the rule language usage.NESPER © uses event processing language (EPL), whereas SWD uses CAISER ™ event processing language (CEPL).On the other hand, WSD algorithm is based on SWD, with the addition of weight function for matching the algorithm.Meanwhile, TCD adds a temporal constraint to its algorithm.All of these CE detectors produce different performances.
Fig. 5 shows an example of the SWD detection technique [22].For example, 5 sequences of events, such as based on a window size of 4. Exact matches with the current sub-rules are coloured in green and give a confidence factor value of CF=1.The unmatched sequences are coloured in red and produce CF=0.   e with a window size of 4. The green colour represents the exact matches rules, whereas the orange colour represents the matches rules for events detected before or after the current rules.Each detection will produce different CF based on the weighting function that has been set.Fig. 6 WSD detection technique Fig. 7 depicts the TCD detection technique.This method takes into account an alpha and beta parameter and is adjusted based on the needs of any temporal challenge that is detected.The alpha value represents the number of confidence factors, maximum overall waiting time and minimum overall waiting time.Whereas, beta values represent the sub-rules parameter, which is the waiting time before detection starts and the waiting time for a current subrule matching move to the next sub-rule if there is no detection in current time.For example, 7 sequences of events, such as e

#
In row 1, Beta (0,1) = 0,2 represents the waiting time before starting the detection that equals to zero and the waiting time for the current sub-rule matching the move to the next subrule that equals to two.Thus, the unmatched detection of the first sub-rule of a for the first sequence of e will wait until 2 seconds to move to the next sub-rule.The matches were found at the next second for sequences of b,a.Further explanation of these detection techniques is found in the main developer manuscript [22].
Fig. 8 shows the NESPER © detection technique.The matching process between event sequences and the rules template will take into account the desired window size, pattern matching used and quantifier search that has been set.All of these parameters are in the standard EPL parameter.For example, 5 sequences of events, such as a   e need to be matched with 4 sub-rules templates (a with the desired sliding window of 5 and the quantifier search used being *.In this case, the quantifier search used is written in this form A B* C D. This means that the sub-rule number 2 can occur more than once.The brief explanation is found in Table 6.Correlation (MCC).CM a method used to describe the classification model's performance with known true values on a set of test data.It is a common technique for the performance measure.However, CM has an unreliable metric for accuracy when there were unbalanced data sets that eventually led to misleading results.Hence, the MCC technique is used to support the detection of a balance measurement by considering true and false positives and negatives.
The results show that the performance for each activity is different.Table 7, Table 6, Table 9 represents activities such as "Door3.NotClosed", "Actor1.ExitBuilding FromRoom3" and "Actor1.PassedByRoom2".Based on the average results from the event analysis in Table 10, the fastest average operation time was by the TCD detector (2.355 ms) followed by SWD (8.689 ms), WSD (24.19 ms) and lastly, NESPER © (34111.88ms).TCD is the fastest because it has no weighting function.While SWD and NESPER © have similar detection technique are dependent on the size of the sliding window.The bigger window size, the longer the searching period.NESPER © has the longest detection due to bigger window size and complex rule structure.The event processing language (EPL) structure of NESPER © can be improved more so that the processing time can be smaller at least almost similar to the SWD detector.
For overall average accuracy and specificity value, all event detectors achieved the highest value of above 95% with the TCD detector (Acc.=99.5%)leading the accuracy performance.However, this study will also consider the MCC value to support the accuracy parameter.The acceptable MMC value is supposedly above 50%.The highest average MCC was obtained by the TCD detector (74.4%) followed by NESPER © (68.8%) and SWD (60.0%).The MCC value plays an important role in the performance parameter measure due to the balance of true / false positive and negative values.WSD had an average MCC of lower than 50% due to the unstable weighting function included in its algorithm.
The sensing system was evaluated based on human movement activities, which were entering and exiting a building / room.The overall highest average for precision and sensitivity values belonged to the TCD detector with 73.6% and 81.1% respectively.The results considered the amount of noise present in the background environment.For example, the ambient light present during daylight and night light might affect the movement detection.Hence, the user needs to set up a higher sensitivity level for both daylight and night light.In this study, we have applied the sensitivity from 0.8 to 0.95.Based on overall average performance from all CE detectors, it was found that the TCD detector was better than the NESPER© detector.Thus, it is recommended that the TCD detector becomes the baseline comparison detector for future algorithm development.

IV. CONCLUSIONS
This paper had provided a basic concept of activity recognition in smart buildings using the CEP approach.The experimental studies involved the use of two CEP engines, namely CAISER ™ and NESPER © .Several complex event detection techniques were also used such as TCD, SWD, WSD, and NESPER © .Results showed that TCD was the best detector for complex event detection with lower time latency, higher MCC, precision value as well as higher accuracy.NESPER © was also found to be good for complex event detection.However, NESPER © is limited to exact matching technique, and the parameter is restricted.TCD is better in terms of detailing in detection rule, and the parameter is easy to vary based on the situation.
It is recommended that future research use more complex activities and standard data using similar CEP detectors such as TCD.The TCD detector could be used as a baseline detector for future algorithm development.On the other hand, the NESPER © rule can be improved by managing the EPL structure and developing the weighting function for each sub-rule.To conclude, this study believes that the research in complex event processing will proliferate as the demand for an efficient surveillance system in smart buildings increases in order to ensure stability, security, safety and a comfortable environment for society and mankind.

Fig. 2
Fig. 2 Block diagram of CEP system

Fig. 3
Fig. 3 Region of interest in smart building

TABLE V DETAILS
ON THE NUMBERS OF COMPLEX EVENTS