A decision support system for waste heat recovery and energy efficiency improvement in data centres

Data centre sector is emerging as one of the largest and fastest growing industrial sectors, accounting for 3% of the global electricity supply and contributing to 4% of total greenhouse gas emissions. A framework for waste heat energy recovery and its accompanying decision support system is presented with the ability to evaluate the waste heat (source) and potential demand (sink) compatibility, perform exergy and temporal availability analysis, and undertake cost-benefits and environmental impact analysis of available heat recovery technologies. This four stages framework is implemented through the case study of a medium-sized commercial data centre to provide evidence of its applicability. The results demonstrated that the framework and decision support system can deliver a streamlined and optimised heat recovery strategy for reducing overall energy requirement in the data centre. About 68% of the inevitable waste heat from IT equipment can be recovered by the recommended solution, allowing the recovered waste heat in the form of warm water to be fed back into the facility for applications such as space heating, achieving about 10% improvement in data centre power usage effectiveness. The solutions given by the decision support system are generally accessible for most data centres as the technologies are readily available and often lead to short payback time and substantial energy savings.


Introduction
There are currently 8 million private and commercial data centres globally, and these are projected to be doubling every four years [1]. The industry is experiencing exponential growth in data centre capacity with 1.94 billion square feet by 2018 up from 1.6 billion in 2016 alone [2]. By 2025, 75 billion Internet of Things (IoT) connected devices is expected to be sharing data worldwide [3]. The data centre sector is also emerging as one of the largest industrial sectors, responsible for 3% of global energy use and 4% of total greenhouse gas emission [4].
IT Equipment and space cooling are the most significant components of energy consumption, accounting for about 85% of data centre energy use as depicted in Fig. 1. The data centre must be adequately cooled during its operation which requires the use of large-scale cooling systems (e.g. cooling tower, air conditioning units) to keep the temperature of critical IT equipment in a safe operational range. Rising costs of energy coupled with targets for the reduction in greenhouse gas emissions have led to an impetus towards efficiency improvements within data centres. The incorporation of renewable energy technologies is becoming a more attractive option as costs fall due to improved technology maturity [5], but are still depending heavily on locations and infrastructure investments and not providing a reliable level of input power. Alternatively, a more cost-effective solution would be to reduce the overall energy demand of data centres, which includes the use of software tools to dynamically balance IT load across the facility to minimise power spikes and cooling load [6], or performing a computational fluid dynamics (CFD) analysis [7] to improve racks' heat dissipation for both air-cooled [8] and liquid-cooled [9], and also by recovering heat energy generated by IT equipment that is inevitably wasted.
In data centres, 90% of the electrical power is ultimately converted into low-grade waste heat, which creates the potential to recover and reuse it as useful input for other applications within proximity [10]. Daraghmeh and Wang [11] reviewed several promising technologies, such as thermosiphon heat exchangers and pulsating heat pipes that can facilitate low-quality waste heat recovery with small temperature difference. Wahlroos et al. [12] developed a costing model to simulate a district heating network with the integration of waste heat recovery from a nearby data centre. Depending on the location where it is generated and on cooling technology used in a data centre, captured waste heat temperature varies between 25°C and 35°C for air-cooled and between 50°C and 60°C for liquid submersion-cooled [12]. Most optimal locations for harnessing waste heat in a data centre have been discussed by Ebrahimi et al. [13] and Davies et al. [14]. The researchers have highlighted the waste heat recovery potential with heat exchanger and heat pump from hot aisles through the ventilation system in an aircooled data centre. However, in practice, the captured heat is usually in the form of warm water at 15-17°C which is used for cold outdoor air pre-heating [15]. Moreover, heat recovery could be undertaken closer to the heat-intensive hot spots, such as the processors. With a large temperature difference during heat transfer, liquid submersion cooling could yield a much higher potential for waste heat recovery [16]. Waste heat recovery in data centres has become an active area of investigation, and the number of commercial applications continues to increase [17]. In particular, waste heat may be used for absorption cooling [18] and district heating [19]. Besides, waste heat may be converted into electrical power through Organic Rankine Cycle (ORC) [20] or may be used to heat liquid nitrogen in a Dearman engine to provide combined power and cooling [21]. Although many researchers have identified the opportunities with waste heat in data centres and technologies for heat recovery, there does not appear to be a methodical approach to assess the potential of waste heat energy. Therefore, a decision support system for energy efficiency improvement within data centres can be a key to remove the barrier to practical implementation. This work aims to establish a framework for identification and evaluation of waste heat energy in data centres to provide a decision support mechanism that enables data centre managers to make an informed investment decision on the types of technology needed to recover and utilise waste heat and to achieve energy and cost savings ultimately.

Framework for waste heat recovery within data centres
The framework for waste heat recovery within data centre consists of four stages that are designed to outline a process for identification, quantity and quality assessment of waste heat streams, evaluation of compatible heat recovery technology and decision support as shown in Fig. 2.
The process of identifying opportunities and assessing feasibility for waste heat recovery within data centre facilities can be complicated, especially when the solution is to be compounded with the most benefit, such as energy and emission reduction, and return on investment. Due to the complexity of data centre infrastructure, it can be quite challenging to recognise the most suitable waste heat hot spots and potential uses, besides the process of assessing temporal and exergy availability between them can be quite complex too [23].
This framework has been developed to collect, collate, evaluate and generate relevant data to support the implementation of waste heat recovery technology within data centres.

Waste heat inventory (Stage 1)
The aim of the waste heat inventory stage in the framework is to identify waste heat sources and potential sinks within a data centre from both the IT equipment and supporting infrastructure perspectives. Data should be collected at the building, server room, server rack levels of data centre operation for the most significant waste heat generation activities, the number of waste heat sources identified are strictly related to the size of a data centre. The waste heat inventory is undertaken using a combination of invasive and non-invasive tools and techniques, i.e. thermocouples, ultra-sonic flowrate sensor and infrared thermography. To better understand the system, it is also recommended that data loggers to be fitted to obtain a continuous quantity of data. The data acquisition campaign can produce both numerical and descriptive output which is then converted and categorised to standard descriptors in the waste heat assessment stage of the framework that can be interpreted by a decision support algorithm, as summarised in Table 1.
The waste heat survey comprises data such as:

Fig. 2.
Overall framework scheme for waste heat recovery within data centres, including a four stages approach to identify, evaluate and compute for appropriate decision support.

Waste heat assessment (stage 2)
Having obtained the data in the waste heat survey, this part of the framework outlines a process for assessing the waste heat sources and potential sinks found within data centre facilities. Evaluating the quantitative and qualitative parameters of waste heat enables efficient comparison and match of the heat recovery strategy. A full list of the possible waste heat source and possible sink combinations are computed initially, followed by process of evaluating the correlation of each listed combination to enable the selection of most optimal pairing that could maximise heat recovery efficiency. Taking a standard data centre as an example [12], a waste heat survey may identify two waste heat sources and two potential sinks within the facility, the assessment listed all nine possible combinations available from the data entry, as shown in Table 2.
For each combination, the power profile of each individual waste heat source and sink is calculated as follows: where i represents a single source, j represents a single sink, s1 and s2 are numbers of sources and sinks respectively in the combination, m is mass flow [kg/s], c p is specific heat capacity [J/kg·K], T h,in and T h,out represent the temperatures for hot streams (sources) at inlet and outlet [°C] respectively, T c,in and T c,out represent the temperatures for cold streams (sinks) at inlet and outlet [°C] respectively, t [0, ] is a user-defined time window within which the heat recovery investigation is carried out.
The quantity and quality of waste heat source and sink are represented by an exergy value which can be calculated as the integral of the power over a specified time period of data provided, as shown in the formulas below: To maximise the potential of energy recovery, one of the critical factors to consider for is temporal availability between waste heat source and sink. A decent temporal alignment of waste heat source and sink can effectively reduce time leg and minimise heat energy loss. A methodical approach is used to select the most suitable waste heat source and potential sink match-up. For each of the listed waste heat source and sink combination, the exergy and temporal availability analysis is performed based on the user-defined time window and resolution to generate corresponding power profiles. The power profiles illustrate the waste heat availability and demand over a period of time, highlighting both the alignment in terms of duration and intensity.
Followed by the plotting of power profile, the next step is the computation of the exergy and temporal availability overlap function between sources and sinks, which is defined as: For a given waste heat source and sink combination, the maximum recoverable energy, i.e. exergy is defined as the integral of the overlap function in a specified time window [0, t] defined by the user. An example plot is shown in Fig. 3. In this plot, the waste heat source profile is displayed by green 1 line, and the potential sink profile is displayed by red line. The overlap function is represented by the yellow section underneath the blue dashed line.
Based on this operation, three unique indices, namely the Recovery Index, Waste Index and Utilisation Index are therefore defined to evaluate the quality of waste heat source and sink match-up.
Recovery Index, RI, is defined as the ratio of recoverable energy (Exergy source, sink ) to total waste heat energy available (Exergy source ) in the specified time window, representing the fraction of energy is captured from waste heat source.
For a given source and sink match-up, a higher value of RI (i.e. values closer to 1) is indicative of the larger percentage of exergy and temporal availability synchronisation, and therefore suggests a more suitable waste heat recoverability. The Waste Index, WI, is defined as the ratio of unrecoverable energy to total waste heat available in the specified time window, Utilisation Index, UI, is defined as the ratio of recoverable energy to total energy demand required (Exergy sink ) in the specified time window, representing the fraction of the heat energy demand is met by the recovered heat energy. It is to be noted that, a higher value of UI does not always represent a good source and sink match-up, e.g. during time periods that heat demand is low while waste heat availability is high, the match-up is inefficient.
Each source and sink pairing is computed for RI, WI and UI, for instance, waste heat profile of server room 1, Power source1 is computed against low-grade heat demand 1 and 2, Power sink1+2 , the source is insufficient to meet the demand on its own. In this case, RI is 1 and UI is 0.804, reflecting that waste heat from server room 1 can be fully captured while 80.4% of the heat demand can be met in terms of temporal and exergy availability. By computing these indices for each source and sink combination, it is possible to then rank them with respect to the RI and UI values as shown in Table 3.
To narrow down the good match and improve system efficiency, a threshold of RI and UI value may be applied for the next computation step, as such criteria can eliminate the inefficient source and sink match-ups and minimise unnecessary computation carried out. Also, qualitative assessment of the source and sink side conditions is performed considering the following parameters:

i. Technical properties for source and sink streams
The technical properties of the waste heat streams can strongly influence the selection of heat recovery methods, installation costs of heat recovery equipment and other requirements. A material properties library has therefore been built for the supply of physical and chemical properties of corresponding streams examined, and information accessed include density, viscosity, and specific heat capacity.
ii. Spatial availability Spatial availability assessment of the data centre facility is essential to survey possible constraints in respect to the room layout, servers and other IT equipment, evaluating potential locations where the heat recovery equipment needs to be installed. This assessment should consider the following factors: • Accessibility to the IT hot spots, such as servers, routers, and storage, for installation and maintenance of heat recovery units; • Pipework routing and positioning, i.e. underfloor or overhead pipework and pumping units for transporting hot and cold streams; • Relative distance between the waste heat source and sink: the aim is to minimise the heat loss during streams transportation, cost of pipework and pumping and maximise the heat recovery efficiency; iii.

Risk of contamination
Corrosion and particulate build-up are the leading causes of the performance reduction or failure of heat recovery units. Fluid leaks in the equipment can cause contamination, which highlights the need to select built materials very carefully to ensure their compatibility with the working fluid and to avoid other mechanical and chemical failures. Special attention must be paid to some of the critical properties corrosive sulphur content, acid number, and oxidisation stability.

Technology assessment and selection (stage 3)
Technical properties of the source and sink streams may dictate the selection and implementation of waste heat recovery technology. The selection criteria for the available technology options must be clearly defined, which includes four primary characteristics: heat transfer mechanism, stream media, equipment size and operating temperature range.
The quantitative and qualitative assessment of the waste heat and demand streams generate results in a form that can be used as filtering rules to cross-check against technology database to narrow down the number of technical options compatible with the waste heat source and sink match-up. This operation produces a list of possible heat recovery technologies that score similarity in the standard comparison.
In this study, a database was extracted from the Engineering Science Data Units (ESDU) 92013 selection and costing of heat exchangers [24], providing detailed guidance on the selection of appropriate heat exchanger types for various criteria. In this operation, non-compatible heat exchanger types are verified simultaneously and ruled out regarding a number of critical criteria: • Maximum and minimum pressure • Temperature range • Size range available • Fouling and cleanability A list of heat exchanger types is reported in Table 4.

Decision support (stage 4)
It is of interest to the data centre managers to have the ability to evaluate and visualise the impact of their decisions. A simplified Techno-Economic Assessment (TEA) is carried out in this stage of the framework to compare the investment quality of different technology applications. The TEA can produce results regarding the annualised net economic benefit and overall payback time in terms of energy saving. For a small-scale heat recovery technology with low capital and operational cost, a rough estimate of the economic return should be sufficient to serve as a guide and justify the investment, whilst for larger and more complex heat recovery units with high capital and operational costs, a full appraisal should be performed.
The heat load Q is determined from the heat balance between the hot and cold streams: The approximation of heat exchanger costing has been made in terms of the cost per unit area. The area (A) of a heat exchanger is calculated using: Hewitt and Pugh [25] reported the C value method to carry out a quick order-of-magnitude assessment of selected heat exchanger. The cost factor, C is defined as the cost per unit (Q T / m ) From the heat exchanger database, C 1 and C 2 are read off using logarithmical interpolation between the upper and lower level values of the (Q T / m ) given in the tables. In an example below, for identical fluid of waste heat source and sink, using a shell-and-tube heat exchanger, a C 1 value of 4.59 at (Q T / m ) 1 = 1000 and a C 2 value of 1.41 at (Q T / m ) 2 = 5000 are read from extracted ESDU database in Table 5. Here The process is repeated for double-pipe, and welded-plate heat exchangers, together with their estimated costs for each type are summarised in Table 5. In a comparison of three different heat exchangers, a double-pipe heat exchanger is likely to be the more cost-effective option for this particular application.
Having estimated the costs of the selected heat exchangers, the surface area required by heat exchanger can be calculated using Equation (12). In addition, by using guidelines for surface area per unit volume of 50 m 2 /m 3 (retrieved from the database) for shell-and-tube and double-pipe designs, heat exchanger volume V (m 3 ) can be estimated. This is regarded as a vital parameter to pay attention to, as nextgeneration data centres are built to have a higher level of complexity with more densely packed servers and allowable space for additional heat recovery technology is very limited.
Finally, it is possible to calculate the financial payback period given the capital investment cost of heat exchange, C HEX , provided by the quick sizing and costing calculation, the cost of auxiliary equipment, C aux and their operation, C o , overall maintenance, C m and the annual cost saving in terms of energy displaced, C annual . C aux is comprised of the costs for required pipework and corresponding pumps with reference to the accessibility to the IT hot spots, pipework routing and the relative distance between the waste heat source and sink. C o is determined by multiplying the equipment power rating by the annual hour of service and unit cost of electricity. C annual is reflected by the energy cost saving achieved when equivalent electricity and gas is to be drawn from the grid.
This structured approach can provide data centre managers with waste heat assessment of their facility, comparable results of waste heat recovery technology, cost and payback figures to support strategic investment. Again, it should be understood that for large data centre applications, the described empirical model may be considered to be complex to perform; however, the method may be implemented by specialised consultants or via commercial or governmental schemes.

Case study
To verify and demonstrate the applicability of the presented framework and its accompanying decision support system, the following case study is provided. The study is carried out with the regular operation of a 530 kW capacity data centre with about 400 servers, which are cooled using engineered dielectric fluid for maximum heat transfer and cooling efficiency. The engineered dielectric fluid is an end-to-end fluid submersion cooling solution for data centres, allowing any original equipment manufacturer (OEM) server, switch, firewall or router to be submersed within it for cooling. Fig. 4 shows side and top view of  An initial site survey of the data centre revealed three main sources of heat loss: 1. Heat losses from coolant circuit to the environment via two cooling towers located outdoor 2. Radiation losses from exposed pipes and openings by transporting the hot coolant 3. Heat losses from auxiliary equipment to support data centre operation Since almost 90% of electrical energy input into a data centre is ultimately converted into heat and total power required by the IT load comprises 47% of the utility input power [22], waste heat from the main servers were of largest quantify and therefore considered the highest priority. The data centre has its servers, routers, and other essential parts of IT operation fully submerged in a well-designed, and well-operated liquid cooling environment, whose specific heat capacity is about 1400 times larger than that of air [27] indicating much-improved heat absorbing capability hence better heat transfer. Besides, the coolant circuit is entirely isolated from the water cooling system, which by standard should be clean and free from particles of dust or condensable components that may require filtering. During the site survey, one potential sink for the waste heat from the data centre cooling stream was identified. This heat demand might be the preheating of boiler feedwater to supply space heating to the facility, such as office space, restrooms and kitchens. Temperature and flow rate measurements were carried out for the data centre cooling system using standard thermocouples and flow meter. Due to the nature of business, the operation is continuous throughout the year so as is the heat demand across the facility. Therefore the time window considered in this case study is 24 h and  resolution is set as one hour. The Data has been recorded over 24 hours as an accurate representation of the data centre's regular operation. Exergy evaluation for the selected waste heat streams is calculated based on the temperature and flow rate data from the survey in conjunction with physical properties of respective fluids. According to the assessment of the waste heat source and sink results summarised in Fig. 5, exergy and temporal availability computation were carried out leading to a Recovery Index of 0.68 and Utilisation Index of 1.0. A plot was automatically generated based on the computation results as reported in Fig. 5, which presents the power of waste heat source over 24 hours as a solid green line whereas potential heat sink is displayed in pink. The overlap function is represented by the blue dashed line.
By applying the criteria for heat recovery technology selection, the most compatible heat exchangers for this case study were identified: plate, shell-and-tube and double pipe respectively. Fig. 5 illustrates the model results and computed values for relevant parameters for the selected heat exchanger. To compare the chosen heat exchangers, a techno-economic analysis was undertaken with consideration of the computation of the recovery efficiency, heat transfer areas and heat exchanger volumes, the associated capital and operational costs, estimated payback period and the overall potential CO 2 reduction.
The application of the waste heat recovery framework and decision support system to the case study reveals three possible heat transfer options to enable recovery and utilisation within the data centre facility. It also highlights that provided sufficient space is available a double pipe heat exchanger would deliver the optimal return on financial investment and environmental benefit for the data centre. The implementation of recommended solution allows the recovered waste heat to be fed back into the data centre auxiliary systems; resulting in an improvement of 10% in PUE, annual energy saving of 281 MWh and payback time of 13 months.

Conclusions
The decision support system for waste heat energy recovery has been developed for data centre managers to take a systematic approach to selecting the most appropriate solution for energy efficiency improvement. The process for identifying waste heat recovery opportunities, analysing waste heat quantity and quality, evaluating exergy and temporal availability and providing decision support has been established. The applicability of the presented waste heat recovery framework and decision support system to an industrial problem has been demonstrated in the case study, with results showing a 68% recovery of waste heat from the IT equipment and 10% improvement in data centre's power usage effectiveness can be achieved by implementing the recommended heat recovery solution. It is considered that the decision support system could be widely applicable to existing and new build data centres and provides a method for relatively quickly deciding upon whether heat recovery might be a suitable investment and more importantly which technologies would be most beneficial.