An Epistemic-Deontic-Axiologic (EDA) agent-based Energy Management System in office buildings

(cid:129) A novel multi-agent Building Energy Management Systems is developed. (cid:129) The model meets dual-objectives of thermal comfort and energy e ﬃ ciency of the HVAC systems. (cid:129) The Epistemic-Deontic-Axiologic (EDA) agent model is applied to develop rational agents. (cid:129) E-component, D-component and A-component based multi-agent framework is described in details. (cid:129) The method could enhance the capacity of energy e ﬃ cient intelligent control of the HVAC system.


Introduction
Buildings have been regarded as one of the major carbon emission sources due to their high levels of energy consumption.It has been reported that, globally, more than 30% of total energy is consumed by buildings [1].Among that, the heating, ventilation and air-conditioning (HVAC) system contributes around 50% energy consumption in nondomestic buildings [2].Therefore, increasing the energy efficiency of HVAC systems is vital to reduce the carbon emission.The Building Energy Management System (BEMS) is regarded as an essential way in operations to achieve energy efficiency while maintaining occupants' thermal comfort [3].
Traditional BEMSs for the operation of heating and cooling systems are based on the designed or fixed range of thermal comfort in accordance with the recommendations by standards such as ANSI/ ASHRAE 55 and ISO7730 [4][5][6][7][8], which is based on the Predicted Mean Vote-Predicted Percentage of Dissatisfied (PMV-PPD) method from a large population based studies in the laboratory by Fanger [9].Such building energy management systems are usually not available for occupants to adjust the temperature range.However, the PMV/PPD index may not accurately reflect the occupants' actual thermal sensations in a certain air-conditioned environments [10][11][12].Moreover, due to the diversity of the occupants' thermal comfort demands, the index could not represent each individual's actual need [13,14].An appropriate temperature setting point is also important for the BEMS in terms of energy efficient.For example, it is revealed that decreasing the indoor air temperature setting-point by 1 °C may lead to 10% heating energy in a HVAC systems [15].Therefore it remains an open question how the energy operation system can satisfy occupants' diverse thermal comfort demand and behaviour adaptation at the meantime time to achieve energy efficiency.
The traditional BEMS based on the pre-fixed setting temperature has little capacities of interactions with occupants; thus not be capable to handle real situations of diverse demands and behaviours of thermal comfort from individuals in an open office.It poses growing challenges to solve the dual problems of (1) meeting onsite occupants' thermal comfort; and (2) energy efficiency of the energy systems.
Responding to this question, Yao and Zheng [16] proposed an advance BEMS called SMODIC (Smart Sensor, optimum Control and Intelligent Control) which aimed to close the gap of mismatching occupants' demand of thermal comfort and the energy supply of a HVAC system by intelligent control.Such advanced BEMS system is expected to have the function to predict occupants' real-time thermal sensations in order to perform dynamic control of the HVAC system.Furthermore such system is also expected being able to provide action advice or saying feedbacks to individuals in order to compromise with other occupants' needs as well as the limit set by the building standard when the conflict exists.This is a complex system because it requires (1) knowledge of individuals' needs; (2) function of the interaction between occupants and the energy system; (3) function of feedback to individuals with consideration of the group occupants' needs and the thermal regulation requirement; and (4) function of dynamite real-time control.
The multi-agent system has been recognised as an effective approach to solve such complex problem, which the traditional single controller method could not solve [17].The aim of this research is to invent a novel BEMS to tackle the challenges of the complexity of the sophisticated BEMS system by integrating four specific functions specified above that the traditional BEMS cannot solve.The research innovatively developed a combination of the problem decomposition method with the Epistemic, Deontic and Axiologic (EDA) agent model to form a holistic solution of multi-agent system design.In this research, the method of using EDA agent model to develop all the rational agents within the BEMS system is fully explored in the first time, which could enhance the capacity of energy efficient intelligent control of the HVAC system in the light of closely responding occupants' thermal comfort needs.

Multi-agent architecture
The 'agent' concept originates from artificial intelligence research [18].An agent is 'a computer system that is situated in some environment and that is capable of autonomous action in this environment in order to meet its delegated objectives' [19].The multi-agent system has been recognised with capabilities to solve complex problem in the building management.For example, research from the MIT intelligent room project employed different agents to realise different targets in the built environment [20].With the help of the agents, the intelligent room gains abilities such as speech recognition and machine-occupant interaction in the room.Sharples et al. [21] developed individual room agent to provide assistance to elderly and disabled people.Room agents are connected each other and share information such as occupancy information and fire alarm information.Liu et al. [22] proposed the Multi-agent System for Building Control (MASBO).In this system, personal agents has the function of representing occupants; the local agent has the function of control the environment parameters and the central agent has the function such as configuring the whole system.Wu and Noy [23] suggested a multi-agent based system to reconcile the occupants' well-being and energy consumption in domestic buildings.The proposed prototype system model is integrated with a wireless sensor actuator network (WSAN) to collect environmental information.Personal agents are recognised playing an important role in helping the system to fulfil individual requirements.Rogers et al. [24] proposed a home energy management agent to optimise the use of the heating system on behalf of the householder.The agent considers the comfort, carbon emissions and cost of energy to make control decisions.Feedbacks from the system sent to the occupants contains the cost and carbon emission information.Yang and Wang [25] developed a multiagent system for building energy management also including the personal agents, the local agents and the central agents.The agents are arranged in a hierarchical way with multiple local agents connected to one central agent while a local agent is serving more than one personal agent.The current multi-agent-based energy management model are suggested to be grouped into different levels such as master agents and slave agents [26].The master agents respond to energy efficiency and comfort issues in the building while slave agents negotiating to each other to avoid conflicts among controllers.A number of research projects have attempted to further extend the ability of multi-agent based building energy management systems by introducing energy resource side management into the function list of the system [27][28][29][30].The multi-agent energy management system is considered to work with a smart grid [28].It is demonstrated that the electronic grid can be controlled by an electronic agent working with heating/cooling agents and comfort agents [27].Alternatively, other renewable sources of power can be managed by a source agent [29,30].
It can be concluded from the literatures that very few systems tend to provide advisory information on the building energy management system.In some, the users' profiles and comfort models are developed to support the agents' decision making process.But the accuracy of such profiles and models needs to be verified and improved.Few researchers have tried to involve the behavioural adaptations in the decision making process as effective ways of changing the thermal condition for individuals.In this research a novel system featuring four abilities depicted in the introduction section is an original contribution to enhance the intelligent control of HVAC systems.Furthermore, in previous research, different multi-agent architectures were applied in BEMSs but there was no clear method to define the architecture, decide the functions of the agents within the system then develop these agents to realise their functions.To solve this problem, the 'problem decomposition' method is popularly used to define architecture of a multiagent BEMS [31,17].The process of this research can be described as: firstly to decompose main complex problem into sub-problems, secondly agents are assigned to allocate the sub-problems.The logics among the sub-problems and the logics between sub-problems and the main problem define the architecture of the multi-agent system.The function of an agent depends on the nature of the sub-problem/subproblems being faced.Once an agent's function is defined, the Epistemic, Deontic and Axiologic (EDA) agent models can be applied to build the agents.

EDA agent model
In the agent-based system, the agents are expected to be 'intelligent'.A 'rational agent' in the BEMS is expected to be able to realise the best possible solutions in a given situation' [17].The 'rational agent' is described as 'that acts so as to achieve the best outcome or, when there is uncertainty, the best expected outcome' [18].Filipe and Fred [32] theoretically explain that, the EDA agent model can guarantee the development of the 'rational agents'.The EDA (Epistemic, Deontic, Axiologic) agent model has been recognised as an effective method to develop the agents, in the BEMS [33].
The original definitions of EDA are 'Axiologic -to be disposed in favour or against something in value terms; 'Epistemic'to adopt a degree of belief or disbelief; 'Deontic'to be disposed to act in some way' [34].It is assumed that all the software and hardware of the BEMS use the same data format, so the input and output information of the agent does not need to be interpreted.Therefore, the perceptive interpreting and output interpreting components are not discussed in the agent structure.In the MASBO, an EDA agent model is used to define the structure of an agents [35,36].The focuses are mainly on the development of the D-component in an agent that defines the plans and the goals of the agent.It remains unclear the realisation of integrity functions of the E-component, D-component and A-component using software and hardware resources from the BEMS.

Thermal sensation model
As stated in the Introduction section, one of the expected function of the advanced BEMS is the prediction of an individual's real-time thermal sensation.Recently, the 'machine learning'-based methods are popularly used in modelling single or group people's thermal sensation.For example, the Extreme Learning Machine (ELM) is used to develop models to predict the thermal sensations of outdoor subjects [37].The Artificial Neural Network (ANN) was used to develop people's thermal sensations in the naturally-ventilated buildings [38].The ANN and support vector regression (SVR) methods have been applied to moderate the PMV values to reflect the true thermal sensations at the realtime.[39][40][41].It has been proved that C-Support Vector Classification (C-SVC) can successfully generate thermal sensation model for an occupant [42].As the PMV/PPD-based thermal comfort model could not reflect the onsite occupants' real-time thermal sensations, it is not ideal to be used in the advanced BEMS.In this case, SVR and C-SVC methods are chosen to generate thermal sensation models here.

Research design
The research problem is defined to solve the complex BEMS with dual objectives of achieving thermal comfort and energy efficiency of the indoor environment and energy system.The research problem defines the multi-agent system.
Firstly, the problem-decomposition method is used to design the architecture of the multi-agent system and define the functions of the agents; Secondly, the agents in the system are developed using the EDA agent model to generate the E-component, D-component, and A-component.The 'C-SVC' and SVR machine learning method is used to generate the thermal sensation models of the onsite occupants (personal and group) respectively that is used by the agents.The development process of the novel multi-agent-based BEMS at both the system and agent levels is illustrated in Fig. 1.Finally, performance of the application of the multi-agent BEMS are tested.

Problem decomposition
There are two steps to define the architecture of the multi-agent system and the functions of the agents [17,31].
Step 1: Decomposing the main problem into several sub-problems; Step 2: Allocating agents to solve sub-problems.
Once the main problem is decomposed into sub-problems, the logics among the sub-problems and logics between the sub-problem and main problem can be identified.These logics are thus used to form the architecture of the multi-agent system, as each sub-problem related to a particular agent.All solutions of the sub-problems are then aggregated to generate the solution of the main problem.When the function of an agent is determined, the agent can be developed following the EDA agent model.

EDA agent model
The EDA model provides the theoretical framework of the components in an intelligent agent.The research [36] attempts to interpret the framework of components in the EDA model in the BEMS context as follows: • The E-component represents the facts or knowledge the agent be- lieves, including the regulations and occupants' preferences.
• The D-component contains the set of available plans and goals.
• The A-component is an evaluating component.It evaluates the plans in the D-component and chooses the appropriate plan based on the knowledge in the E-component.
As illustrated in the introduction section, this research focuses on making the BEMS understand occupants' thermal comfort needs.These knowledges are generated and stored in the E-component in the agents.The methods used to generate these knowledge are introduced in Sections 3.4 and 3.5.
The BEMS system also requires the knowledge of the energy consumption of the HVAC system to be stored in the E-component in the certain agents.All the agents in the system need decision-making algorithms in their A-component and action plans in their D-components.Examples of them will be showed when the agents are built and tested in Section 4.

C-SVC based personal thermal sensation model
The personal thermal sensation is the essential knowledge of the Ecomponent.The C-SVC algorithm has been used to develop personal thermal sensation models.The developed model can realistically reflect an individual occupant's thermal sensation and expectation.The onsite occupants' personal thermal sensation predicted by the modelling method using C-SVC algorithm is more realistic than that by the PMV method.Therefore the C-SVC algorithm is embedded in the E-component to obtain the personal thermal sensation.The detailed modelling method of development of personal thermal sensation is introduced in reference [42].

SVR group-of-people-based thermal sensation model
The SVR is a modelling method from machine learning.Differing from the C-SVC method, it is a regression algorithm based on the support vector machine, which has been used for developing regression models by a number of researchers [43][44][45].The SVR algorithm is based on the ∊-insensitive loss function ∊ L f , which can be expressed according to Vapnik [46]: where y is desired output and the z is the model's prediction.The relationship between − y z and ∊ L f is depicted in Fig. 2. Assuming the total number of data sets is N, the input-output pairs can be expressed as (u ,y . M Let be the total number of training samples.The input vector u i contains environmental parameters and personal factors.
The targeted output y i only contains one element, which is the thermal sensation value under the particular circumstance, which is defined by u i .
Let z i represent the output value of the developed regression model when the input vector is u i .
The relationship between the output and the input pair can be expressed as: where ω is weights and b is bias.and ∅ u ( ) i is defined in the kernel function: In order to obtain the regression algorithms (SVR), the ER (Empirical Risk) need to be minimised.It is a constraint optimisation problem.
where e is a constant value.By introducing positive slack variables, the ∊-insensitive loss func- tion defined in Eq. ( 1) can be reformed as (reference to Xi et al. [48]): Then the constraint optimisation problem can be converted into: where C is a positive regularization parameter; Subject to the constrains of: Accordingly, Lagrangian function can be defined as [39,49]: Subject to the constraints of: where l i and ′ l j are Lagrangian multipliers.Finally, the regress algorithm can be expressed as: where l io ′ l io and b o are optimised coefficients.In this research, the ∊ -support vector regression (∊-SVR) tool, which is provided by the LibSVM library for Matlab software [50], is used to realise the SVR algorithm described above.The more detailed introduction of the basic principle of the SVR can be found in Vapnik [46] and Haykin [47].

The architecture of the multi-agent system
The dual-objective problem is solved by the problem decomposition method and the EDA agent model.By applying the method, the original problem can be divided into two sub-problems of (1) how to avoid the energy wastage of the HVAC system; (2) how to enable each occupant to acquire a thermally comfortable feeling.Applying the multi-agent model, the sub-problems can be solved by assigning two types of agents of (1) the local agent and (2) the personal agent.
The architecture of the multi-agent system is illustrated in Fig. 3. Assuming there are n number of occupants in the open plan office, n number of personal agents are assigned.From the figure we can see that the local agent and the personal agents work together in the BEMS.Personal agent is responsible for the dialogues with the occupants and the Local agent.Each personal agent communicates with its designated occupant namely Occupant 1, Occupant 2, … , Occupant n.It provides personalised advice to its occupants in terms of actions of regulating their thermal comfort.Their communications are realised through the human-machine interface and the sensors placed in the indoor environment.Technologies affecting the human-machine interfaces, sensor network and data storage are not discussed in detail in this paper.The local agent is responsible for the operation of the HVAC system.It provides signals of optimal temperature setting point to the actuator of the HVAC system.

Local agent
As the agents are expected to be rational, the EDA agent model are used to develop the theoretical framework of the rational agents in the BEMS.The functions of each EDA components in both personal and local agents need to be specified in the context of the BEMS, which is not fully realised within the existing literature.The key logic of the EDA-based agent is described here as: The agent makes decisions in the A-component applying decision-making method, which uses the action plans in the D-component and knowledge provided by the E-component.
The local agent is expected to have the ability of providing the optimal temperature setting point to the actuator in the HVAC system.The structure of the Local agent developed by the EDA model is illustrated in Fig. 4. The specification of functions of the E-, D-, and Acomponents in the Local agent is described in details here.
• The D-component needs to provide the plans regarding the different settings of the HVAC.
• The E-component contains (1) the models which evaluate group occupants' thermal comfort and the energy consumption; (2) the algorithms of generating models or the data of the actual energy consumption in practice; (3) the real-time environmental information from the sensor network and personal information from personal agents.
• The A-component contains decision-making algorithm to choose the appropriate settings from the set of available HVAC settings in the D-component based on the knowledge in the E-component.

Personal agent
The personal agent is expected to have the ability of providing the optimal suggestions on the behavioural adaptations to the occupant based on the pooled factors of the setting point of the HVAC system, the personal thermal sensation, environmental information and personal factors such as the clothing insulation level (Clo) and the activity level (MET).The structure of the personal agent developed by the EDA model is illustrated in Fig. 5.The specification of functions of the EDA components in the Personal agent is described in details here.
• The D-component provides the adaptive action plans.
• The E-component contains the model and the algorithm which evaluate the occupants' personal thermal comfort.It also contains the environmental information acquired from the sensor network, the personal factors from the human/machine interface and the HVAC setting point from the local agent, the energy consumption model or the data reflecting the actual energy consumption in real practice.
• The A-component contains decision making algorithm to provide suggestions on adaptive behavioural actions based on the action plans stored in the D-component and the knowledge in the E component.
Here, the personal agents not only play an assistant role of collecting, storing and transforming information in the multi-agent-based energy management system, but also make rational decision on any necessary actions to suggest to the occupants.
Once the HVAC temperature setting points are confirmed, it is to be transformed to the actuator for operation.

Experimental study
In order to fully develop the thermal sensation models in the Ecomponent in the novel personal agent and local agent, experimental studies were carried out in an open plan offices.The experiment collected the information of subjects' personal factors including clothing insulation levels and activity levels, subjects' sensations in the ambient environment and their reactions to the environment.
The experimental building is a four-storey building accommodating classroom, office and meeting rooms, which is located in the Whiteknights campus, University of Reading.The major experimental area is located in the ground floor of the building.The occupants of this area are administration staff, academic staff and PhD students.Three open plan office areas equipped with air conditioning system were selected.The air conditioning system operates at the fixed temperature setting point of 23 degrees centigrade from 9:00 to 17:00 all year around except at weekends and university closure days.
The experiment and data collection in the air-conditioned environment took place from October 2014 to August 2015.The questionnaire survey was conducted throughout the experimental period.The questionnaire design followed the current international standards: [4,5] and referred to previous research [12,38,51,52] consulted with the psychologist.
Letters were sent to all the potential candidates working in the studied area which explains the purpose and scope of the experiment.All the occupant in the open-plan office were volunteers who were ordinary healthy people.Consent forms were signed by all the subjects to meet the ethics requirement.Functions of the sensors and contents of the questionnaires were explained in detail to all of the subjects and other occupants who were not involved in the experimental area.Questionnaire surveys were conducted twice a day, two days a week in each zone.While the subjects were filling in the questionnaires, their ambient environmental parameters including air temperature, relative humidity and air velocity were recorded by sensors.Globe temperature and the air velocity values were measured by two hand-held instruments.The mean radiant temperature was derived by measured globe temperature.The specifications of the sensors and instruments are illustrated in Table 1.All of the sensors and instruments were new and calibrated by the manufacturers.
The sensor measuring points were set at 0.6 m (at waist level) above the ground close to the occupants.The method of data collection followed the specification of the class II data defined in Brager and de Dear [53], which is suitable to analyse the subjects' comfort influenced by the environment as well as their behavioural responses.Six volunteers  Table 1 The specification of the instruments and data loggers.were recruited and completed the whole survey process throughout a whole year.A total number of 247 effective samples were collected.Fig. 6 shows the total number of effective samples collected from each subject.AC1, AC2, AC3, AC4, AC5 and AC6 represent fix subjects participated in the experiment.

Personal thermal comfort model
By applying the C-SVC algorithm, personal thermal sensation models for subjects AC1, AC2, AC3 AC4 and AC5 are developed applying the developed C-SVC algorithm.As subjects are not sensitive to warm or hot environment, a simplified version of the thermal sensation models is developed for AC2, AC4 and AC5.Based on the structure of the selected data, the simplified version aims to categorise the thermal sensation into two types: the first type of sensation is feeling 'slightly cold' or colder, which is represented by '−1'; the second type is feeling 'neutral' or warmer, which is represented by '0'.As these models are able to decide boundary between the feeling 'neutral' and 'slightly cold', they are sufficient to be applied in the BEMS in an environment condition similar to the experimental condition.The situation for subject AC6 is special.During the whole experiment period, the reported thermal sensations from the subject are always 'neutral'.It indicates that this subject always satisfied with the thermal environment during the period, so it is not necessary to develop a personal thermal sensation model for AC6.These target output subjects AC1 and AC3 follows the ASHRAE seven-point thermal sensation scale.
Because of the limit of the total sample size, the leave-one-out crossvalidation (LOOCV) method is used to verify the performance of the developed models [54], which is shown in Fig. 7 separately.In these figures, the X axis presents the number of the experiment while The Y axis shows the thermal sensation vote (TSV) values.The crosses in the figure are the TSV values predicted by C-SVC-based personal thermal sensation models, and the circles represent the actual TSV data collected from the subjects.The cross covering the circle means the model makes a correct prediction.The developed personal thermal sensation model is represented by Fp and it is expressed in function (17).Where, the MET and the Clo values are the occupant's personal activity and clothing insulation level.( , , ,  ,  , )

= T T V RH MET Clo TSV Fp
a r a (17)

Group-of-people-based thermal sensation model
The SVR algorithm is used to develop the group-of-people-based thermal sensation model.The LOOCV method is used again here to verify the performance of the model developed using the entire data set.The linear kernel is selected when using LibSVM.The modelling process based on the SVR algorithm is illustrated in Fig. 8.The inputs of the modelling algorithm are environmental factors, personal factors and occupants' thermal sensation votes collected from the field study.The developed software program will automatically collect the input data from the database and input them into the modelling algorithm, then save the developed thermal sensation models.When making predictions, the 'Input Attributes' are input into the developed in the developed model and the predict target of the model is the actual mean vote (AMV) of these group of people.
As the predicted target is the AMV, the bin method is applied here to calculate the value of the actual mean thermal sensation vote.The method is used for similar purposes in previous research [55,56].The method here is realised by the average of the predicted values and the corresponding real sensation values in a range of 0.5 of an ASHRAE scale unit.The fitting between the average predicted thermal sensation values and the AMVs is illustrated in Fig. 9, which indicate high degree of linear fitting between these values.The developed group-of-peoplebased Thermal Sensation Model is represented by Fg and it is expressed in function (18).Where, the MET and the Clo values are the group of people's average activity and clothing insulation level.( , , ,  ,  , )

= T T V RH MET Clo AMV Fg
a r a (18) 4.6.The heating and cooling energy In the real practice of the application of BEMS, the actual energy consumption of a HVAC system could be stored in its E-component.Alternatively, in the simulated case studies, the energy consumption could be calculated using simulations.In this research, the main purpose is to test and evaluate the impact of the agent-based BEMS on energy saving comparing to the system without BEMS.Herein a simple energy load calculation algorithms is embedded in the E-component.The value of the total heat losses from the building are estimated by the 'Average Value Method' introduced in Brumbaugh [57] and Ansari et al. [58].The basic factors considered include the indoor and outdoor temperatures, factors of wall, contents and glazing.Let the length, width and height of the room be represented by the symbols L Wr , and H .The symbols Ww and Hw denote the width and height of the window respectively.The area of the wall exposed to the outside environment is * Wr H.The area of the window on the wall is * Ww Hw.Let DT present the indoor and outdoor air temperature difference, then: Where t o is the outdoor temperature; t i is the indoor temperature.
The values of the factors used in the calculation of the sensible cooling loads are listed in Table 2.The factor selection is based on the physical properties of the experimental built environment.The dimensional parameters of the environment are given in Table 3.
Cooling load values calculated using the factors listed in Table 2 are expressed as Eqs.( 20 The total cooling load of the sensible heat Ls of the room is: The latent heat allowance Lls is given by: The total heat for the four factors is: = + Las Ls Lls (26) It is noted that the table in Ansari et al. [58] did not give the factor for people, lights and equipment.The factor values for these three items in Table 2 are referenced from [59][60][61][62].
Let np represent the number of people in the room, Then the heating is shown as: The total heating load is calculated by: The cooling load can be calculated by a load estimation form provided by the Air-Conditioning and Refrigeration Institute (ARI), and the calculation method is also called the ARI method [57].The values of the three factors are defined in Table 4.
When the HVAC system is assumed to be working under the heating mode, the indoor and outdoor temperature difference is expressed as − t t i o .The heat loss through the glass HLG can be expressed as function (31): where * Ww Hw is the total area of the glass.The heat loss of the wall HLW is given by: where * W H is the total area of the wall.The heat loss caused by the contents of the space HLC is then: where * * L Wr H is the volume of the indoor space.Then the total estimated heating load HL is as:

The decision-making algorithm and the action plans
Both the local and the personal agent need the decision-making algorithms in their A-component to make rational decisions.In a local agent, the decision-making algorithms is to decide the optimal setting   point of temperature based on the onsite occupants' thermal comfort and energy efficiency requirements.The decision is based on the dualobjective of thermal comfort and energy efficiency.The constraint method is suggested to be an effective way to convert a dual-objective decision-making problem into a single objective problem [14].This method converts all but one of the objectives into constraints and then solves the constrained single-objective problem to obtain the solution to the original problem.The basic idea of the constraint method is leaving one of the objective functions and converting the rest of the objectives into constraints [63].The boundaries of the constraints can be defined by the user.Assuming that the two objectives in a dual-objective are presented as objective function x f ( ) In function (35), it can be found that object function x f ( ) b is used as the constraint and f a as the optimisation object.
Herein, the action plans for the HVAC system are the set points of the system, which are stored in the D-component in the local agent.When the local agent making decisions, the heating or cooling load illustrated in Eqs.(30) and ( 34) is used as objective function x f ( ) a and the group-of-people-based Thermal Sensation Model, which is expressed in Eq. ( 18) is chosen as objective function x f ( ) b .Once the dualobjective problem is covered, the x f ( ) b will be transferred into constrains, which means it selects all the possible set points that fit the thermal comfort requirements, then the A-components find the optimal set point giving the minimum value of the main objective function x f ( ) a from the points selected by x f ( ) b .
In the D-component in personal agents, the action plan includes the occupants' clothing insulation levels, which are also represented by discrete numbers.If the occupant is unsatisfied with the set temperature, the decision making algorithm decides the best clothing insulation level for a single occupant based on the selected set temperature and the occupants' personal thermal preference.The decision making logic of the personal agent is described as following: The personal thermal sensation model expressed in Eq. ( 17) is the only objective function considered by the algorithm and the Clo is one of the function's input.The algorithm then selects the Clo value which can guarantee the comfort condition for the occupant and meantime feedbacks such information to the occupant.If more than one value can be chosen, the algorithm will select the one which is closest to the current clothing level.
In operation, the local agent requests the information of occupants' Clo and MET values from the personal agents.The personal agents need the local agent to provide the current set point of HVAC system to make decision.The energy consumption data can also be transformed from the local agent to the personal agent so that the occupants will understand the current environmental conditions as well as the HVAC system's energy consumption.

Performance of the developed BEMS
The MATLAB software is used to realise the agents including modelling methods and the decision-making algorithm introduced in Section 4. The developed thermal sensation models are integrated into the personal agent and local agent to build up the proposed BEMS.As the agents with E-D-A-components are fully developed and the multiagent BEMS is constructed, the performance of the system is tested and illustrated in this section.
The testing process is arranged into two steps.In the first step, the test focuses on the performance of each agent.The operation of Ecomponent, D-component and A-component in all agents are examined.It is assumed that the outside temperature is 10 °C and the initial clothing insulation level is 1.0 Clo.The occupants' activity levels values equals to 1 MET.The indoor air velocity is 0.08 m/s and the relative humidity is 40%.All these knowledges are stored in the Ecomponents of the agents.In order to illustrate the effect of the thermal comfort model, three comfort temperature modes are proposed for the E-component in the local agent: the PMV model, the group-of-peoplebased thermal sensation model generated in Section 4.5 and a fixed set temperature at 23 °C.The energy consumptions of the HVAC system guided by the local agents equipped with in these three modes are compared.In the D-component of the local agent, the range of the HVAC system's setting points of temperature is between 18 °C and 27 °C within 0.5 °C intervals.The assumption is that the personal agents cannot change the settings of the HVAC but they can advise the occupants to adjust their clothing insulation level at 0.25 interval, which are integrated in the D-component in the personal agents.
Based on the knowledge in the E-component and the action plans in the D-component, the decision-making process in A-component in the local agent based on the PMV model are illustrated in Fig. 10.When the A-component in the local agent makes decisions following the guidelines provided by the ASHRAE standard and PMV index, the optimisation target for the thermal environment is to maintain the average value of the thermal sensation votes at between −0.5 and +0.5.Based on the constraint method, the thermal comfort goal is converted into the constraints of the energy consumption goal.In Fig. 10a, the area marked by the diamonds is the range that fits the thermal comfort requirement.The area defines constrains of the search area when considering the energy consumption in Fig. 10b.The final decision made by the local agent is highlighted with the red1 star in the figure.It can be found that the lowest possible set point predicted by the PMV is 22 °C.The heating load required to reach this setting is 6298.3W. Fig. 11 illustrates the decision-making outcomes of the local agent equipped with the newly-developed group-of-people-based thermal sensation model.The optimisation target for the thermal environment is also to maintain the average value of the thermal sensation votes at between −0.5 and +0.5.The acceptable range of indoor temperature is marked with blue diamonds in Fig. 11a.By considering the energy consumption, the final decision of the local agent is 21.5 °C in Fig. 11b, which is 0.5 degrees lower than the PMV prediction.In consequence, the required heating load is 6035.9W. The decision based on the developed model has a heating load 256 W smaller than its counterpart.
If the HVAC system is working under a fixed schedule policy with a 23 °C set point, as observed in the field study, the system needs to cover a 6823.2W heating load.In this case, the decision made by the group thermal sensation model requires a heating load 10% less than that required by the fixed schedule method required in the field study.
When the setting temperature is 21.5 °C, the personal agents find that AC1, AC3 and AC4 will feel slightly cool.Based on the integrated personal thermal sensation models, the personal agents suggest AC1, AC3 and AC4 to increase his/her clothing level to 1.25 Clo, 1.5 Clo and 1.25 Clo respectively via a human machine interface.The test step one proves that the operation of the local and personal agents in according with their design purposes.The test also practically proved that the EDA agent model is an effective way to generate the agents needed by the BEMS, which is not revealed before.
In the second step, the BEMS is tested by using a set of real outdoor climate data.The data is the hourly average outdoor temperature data from 9:00 am until 17:00 pm in March 2015.The data were collected by a meteorological station at the University of Reading.More details of the data collection can be referred to Brugge [65].The indoor air  velocity is 0.08 m/s.The relative humidity is 40%.For all the occupants, the default clothing level is 0.75 Clo and the activity level value is 1 MET.The knowledge on thermal comfort in the E-component and the action plans stored in the D-component in both the local agent and the personal agent are remain the same as they are in the step one.The hourly energy consumption of the HVAC system also guided by the local agent with the group-of-people-based model, the local agent with the PMV model and the fixed schedule are illustrated in Fig. 12.The decided set temperature and the monthly summary of the required heating and cooling energy is shown in Fig. 13 and in Table 5.It can be found that compared to the BEMS based on the PMV model and the fixed schedule, the BEMS with the developed thermal sensation model saves 7% and 3.5% of energy respectively compare to the system with the PMV and the fixed set temperature.For the personal agents, based on the set temperature decided by the group-of-people-based they suggest AC1, AC3 and AC5 to adjust their clothing insulation level into 1 Clo, 1.25 Clo and 1 Clo.The other occupants are satisfied with the set point.

Conclusion
This paper presents a newly developed novel multi-agent BEMS which aimed to meet dual-objects of reducing energy consumption of a HVAC system and in the meantime to satisfy occupants' thermal comfort.The methodology of the development of E-component, D-component and A-component based multi-agent framework is described in detail.The architecture of the BEMS is composed of two types of agents namely the local and the personal agents using the problem-decomposition method.The expected abilities of the agents are specifically defined.The function of the agents are specified using EDA component model.
The novel BEMS is intelligent as it contains rational local and personal agents who perform the designated roles and communicates among them within the multi-agent system.Personal agents gather the knowledge of individual's needs by learning the information collected from the human-machine dialogues system and they also provide feedbacks to individuals with consideration of the group occupants' needs and the thermal regulation requirement.The local agent communicates with personal agents to gain the knowledge provided form the local agents so to provide control signals to the actuator of the HVAC system performing dynamite real-time operation.The novelty of such BEMS is that the system overcomes the mismatching of thermal comfort and energy demand due to the lack of interaction of the actual occupants and the energy system.It solves the problem that the traditional single pre-set temperature control method could not solve, i.e. excessive heating and cooling supply causes overheating or overcooling.The advanced agent system can provide advice of adaptive behavioural actions for occupants and appropriate set temperature for HVAC system as the personal agent and the local agent contains the knowledge in the E-component learnt from the occupant and gathered form the best practice.The occupants' thermal sensation model and comfort requirement are established through the advanced machine learning SVR and C-SVC algorithm which is embedded in the BEMS which enhance the capacity of energy efficient intelligent control of the HVAC system.
The abilities of the developed multi-agent BEMS are verified by simulated case studies.The testing results demonstrate that the HVAC systems managed by the developed EDA agent model based multi-agent BEMS can save 3.5-10% energy comparing with that consumed by the pre-set control systems in the simulated built environments.
Future work includes the development of a web-based human-machine application or the mobile apps porotype BEMS and testing in real buildings for energy management.
Furthermore, the E-, D-and A-components in the local and personal agents can be further extended to meet more requirements relating to the occupant wellbeing such as humidity, air velocity, acoustic, lighting and air quality.

Fig. 1 .
Fig. 1.The development process of the EDA agent-based BEMS.

Fig. 3 .
Fig. 3.The architecture of the multi-agent system.

Fig. 8 .
Fig. 8.The modelling process based on the SVR algorithm.
the constraint method can be expressed as function(35) [64]:

Fig. 11 .
Fig. 11.Decision-making Outcomes in an Open Plan Office (group-of-people-based thermal sensation model -based BEMS).

Table 2
The factors used to calculate the cooling load (sensible heat only).

Table 3
Dimensional parameters of the environment.

Table 4
The factors used to calculate the heat loss.

Table 5
Optimised set point and the detailed monthly heating and cooling load values in an Open-Plan Office.