Design and key technology of the energy consumption management system for the liquid cooling data center

In view of the serious problem of energy consumption waste in the application process of liquid cooling data center, a new energy consumption management system of liquid cooling data center is constructed in this research. Energy consumption predictor, resource controller and resource configurator are used to monitor and manage energy consumption and optimize resource allocation of liquid cooling data center. The S3C2440A microprocessor with the internal core of ARM920T is adopted as the core controller of the energy consumption data collection system, and the energy consumption sampling circuit is designed with the voltage transformer and the current transformer, and attenuation network is adopted to prevent frequency aliasing in data sampling. Particle swarm optimization algorithm is used to identify the parameters in the model estimator, and the resource coordinator is used to solve the power consumption and the performance models. When using multiple physical servers to simulate the data center environment, the experimental structure shows that the system in the research can control the overall energy consumption of the server within 260 W, and the prediction error of the model estimator is kept lower than 2.4%.


| INTRODUCTION
With the all-round start-up of the "East-to-West Computing Resource Transfer" project in China, the construction of data centers is speeding up, and at the same time, higher requirements for heat dissipation are put forward. Liquid cooling technology can meet the development requirements of high density and low energy consumption in data centers, and has become an inevitable trend of the current development. By combining the infrastructure management of the data center with IT service management, 1 the technology ofliquid cooling data centersuses liquid instead of air as the coolant for heat exchange with heating parts, so as to take away the heat in IT equipment. Although the liquid cooling technology can be used for efficient cooling to a certain extent to improve the using efficiency and the stability of the server, the current energy consumption management system 2 cannot achieve favorable control effect, with unfavorable information collection ability and energy evaluation ability, leading to serious waste in energy.
Aiming at the above problems, a load balancing method based on feedback dynamic weighting is proposed in Literature 3 in domestic research, to collect data with fixed time intervals, and evenly distributing the large amount of data traffic and the large number of concurrent accesses to multiple nodes for processing, or distributing the operation of a single node with heavy load to multiple nodes for processing can reduce, which can reduce the consumption of the energy of the server cluster, butthe control of energy consumption is unstable. The system in Literature 4 uses the dynamic voltage and frequency scaling (DVFS) technology to regulate the basic frequency and the voltage of the server in a dynamic way, to propose the threshold value unit algorithm based on power consumption perception, to conduct dynamic online dispatching to figures in the data center. The DVFS technique takes advantage of the characteristic that energy consumption is proportional to the square of voltage and the clock frequency to reduce the power consumption of general-purpose processors by reducing the clock frequency. However, just lowering the clock frequency does not save energy, because the performance degradation will lead to the increase of task execution time. Voltage regulation requires regulating the frequency in the same proportion so as to meet the signal propagation delay requirement. However, either voltage regulation or frequency regulation will cause the loss of the system performance and increase the response delay of the system. The system in Literature 5 in overseas research establishes the software defined network, and the entire network topology connecting method is adopted to reflect the connecting state and the flow state of various smart devices of the data center to the smart center for unifiedscheduling arrangement by the smart center. The system in Literature 6 establishes the deep learning network model, which is applied to network energy conservation of the data center, to make the equipment in the dormant and closed state with certain control measures, so as to reduce the wastes caused by energy consumption. However, in the current energy consumption management research, there are some defects, for example, the control level of the system is not accurate enough and the fineness of the algorithm needs to be further optimized, and the problem of energy consumption caused by remote input/output (I/O) has not been taken into consideration.

| ENERGY CONSUMPTION MANAGEMENT SYSTEM OF THE LIQUID COOLING DATA CENTER
In view of the deficiencies in the above research, an energy management system for the liquid cooling data center was designed in this research, andaccording to the functional demands of the data center, the research designed an energy consumption management system for the liquid cooling data center, to collect energy consumption data to the data center, so as to realize the self-adaptive management on energy consumption and the optimizing configuration of resources according to the analysis results of the energy consumption data. 7 The system collects energy consumption data of equipment such as the server with hardware equipment, and conducts real-time monitoring on energy consumption information of various systems, to provide important and reliable data basis for planning of the data center and improvement on operation and maintenance; in addition, it provides measurement standard for comparison on efficiency among different data centers. The framework of the energy consumption management system 8 for the liquid cooling data center of the research is shown in Figure 1.
From the perspective of system framework, it can be divided into the equipment layer, the server layer, the F I G U R E 1 System structure block diagram computing layer and the application layer. The bottom layer equipment of the system includes the resource coordinator, the resource allocator, the driver, the controller, the server, the monitoring equipment, the data collecting equipment, the UPS system and the air conditioning system. The server layer provides multiple-cloud server collection, which evaluates on the results of energy consumption statistics and analysis of the data center, to realize the quantization of the energy utilization level of the data center. The computing layer takes electric energy utilization efficiency, local electric energy utilization efficiency, cooling/power supply load coefficient as the basic energy efficiency indexes for measuring the energy efficiency of the data center. By conducting in-depth analysis on operating state and power utilization conditions of the equipment of the data center in combination of the data model, it realizes monitoring and management of energy consumption of the liquid cooling data center. In which the energy consumption estimation model and the energy consumption optimization model can conduct optimizing configuration to the resource configurator of the data center according to dynamic changes in loads of electric equipment of the data center. By utilizing multiple terminal equipment, the application layer shows the energy consumption information of the data center in a real-time way, to optimize and balance the load rate of the data server of the data center by receiving the power utilization conditions acquired from the computing layer, to avoid very low operating load of the storage server, reduction in the overall energy efficiency of the data center or electric energy loss.
The energy consumption management system of the liquid cooling data center can be divided into energy consumption data collection, energy consumption statistics and analysis, resource coordination and configuration, and energy efficiency evaluation and optimization. The energy consumption data is compared and analyzed, to find out the largest power utilization system of the data center. In combination of the energy consumption estimator and the performance estimator, the proportion of energy consumption of various systems 9 of the data center is calculated, to analyze the using conditions of the energy consumption equipment that can be optimized. The energy efficiency indexes of the liquid cooling data center can reflect the using conditions of the energy in operation of the liquid cooling data center in an objective way, and can provide basis for self-adaptive control to power utilization equipment and optimized distribution of resources. 10 3 | DESIGN OF KEY TECHNOLOGY

| Hardware system of the energy consumption data collection module
After the completion of the energy consumption data collection of the liquid cooling data center, real-time collection and data storage are conducted to the power utilization data of various equipment of the data center. The embedded microprocessor undertakes tasks such as coordination, control and monitoring of the whole energy consumption collection system. 11,12 The 32-bit S3C2440 microprocessor with the internal core of ARM920T based on the ARM framework is adopted for the research. The research optimizes the energy consumption collection accuracy of the collection module, and the modulation and demodulation module and the signal conditioning module are added, to reduce the interference of signal noises and further improve the accuracy of energy consumption collection. Besides, optimized design is conducted to the wireless communication interface, to increase the communication distance of the collection module. The main performance of the microcontroller is shown in Table 1.
The highest communication rate of the wireless communication interface of the module is 115,200 bps, and the farthest communication distance is 1200 m, which supports the communication and data transmission between the energy consumption management system and the equipment of the bottom layer. [13][14][15] The RS-485 conversion chip MAX485 is used to complete the level switch, and the balanced sending and different receiving method is adopted, to restrain the common mode interference, to support high speed data transmission. The hardware structure of the energy consumption data collection module is shown in Figure 2.
The research designs the energy consumption sampling circuit with the voltage and current transformer; in combination of the self-adaptive filter, it improves the data collection accuracy of the energy consumption data collection system, so as to collect and store multiple electric energy data types and parameters, to realize automatic data collection. 16 In addition, the wireless communication module is utilized to realize data transmission, so as to improve the flexibility of the data collection system; by supporting VPN communication and multiple encryption technologies, the wireless communication module ensures the security of user data during transmission in the wireless channel, 17,18 and supports dynamic refresh of the network, with strong adaptive ability of the data transmission module to environments. The signal frequency range of the highaccuracy current transformer is [40, 20k] Hz, and that of current and voltage transformer is [20, 10k] Hz, and the energy consumption sampling circuit is shown in Figure 3.
The voltage transformer and the current transformer in Figure 3 constitute the attenuation network, to realize the attenuation of voltage and current, so as to avoid frequency aliasing in data sampling. The circuit in the research uses six resistances of 6.8 K, with total resistance value of 40.8 K. 19 Frequency aliasing may lead to false signal frequency, leading to influences on measurement of the energy consumption data. The attenuation of load voltage and load current realizes the effect of anti-aliasing filter. The current of the main loop is 5.4 mA, and the current after the secondary loop of the transformer is 5.4 mA. In which the resistance R9 converts the output current to the voltage signal for A/D sampling; in the circuit, capacitances C5 and C7 play a role of filter, and D1 and D2 avoid the surge voltage in the circuit, which may lead to electrostatic damage to devices in the circuit. 20 3.2 | Energy consumption estimation and optimization method of the liquid cooling data center server Because of the nonlinear and time-varying characteristics of the energy consumption data of the liquid cooling data center, only static linear model is not enough for establishing energy consumption, performance and resource distribution model; to express the nonlinear characteristics in the data of the liquid cooling data center, the linear model with constantly updated parameters is adopted to simulate the time-varying process. The energy consumption optimization process of the liquid cooling data center is shown in Figure 4.
The resource optimizer and the model estimator in Figure 4 acquired the minimum value of parameters g k ( ) and H k ( ) i T based on the power consumption objective function and the performance objective function, and express the r k ( ) and P k ( ) of the next time interval by recognizing H k ( ) i T , a k ( ) and b k ( ) based on the kth time interval. The process for parameter identification of the model estimator is shown in Figure 5.
Identify parameters with the optimized algorithm, to change the parameter identification model into the F I G U R E 2 Hardware structure of the energy consumption data collection module F I G U R E 3 Energy consumption sampling circuit optimization model. The research constructs mathematical model with the autoregressive moving average model, to accurately express the energy consumption data of the liquid cooling data center at the second order, to establish the relation between energy consumption and the server resource of the data center, which can be expressed as follows: The k in the formula is the control interval of the server; P k ( ) is the average power consumption of all the data servers at k intervals; g k ( ) is the relation parameter between power consumption and VCPU threshold value; H k ( ) i T is the relation parameter between performance and VCPU threshold value; C k ( ) is the VCPU threshold value distributing to all the servers. The model between the energy consumption data of the data center and the resources allocated to the servers is established based on Formula (1). Then the relation model between the handling capacity of each server and the power resources distributed is established, expressed as follows: In the formula, r k ( ) is the handling capacity of the server; a k ( ) is the relation parameter between the handling capacity of each server and electric energy; b k ( ) is the relation parameter between the VCPU threshold value and the electric energy of each server. Formula (2) shows the relation between the handling capacity of the server and electric resources. Different values are adopted for parameters of the server during varied control intervals, and the power consumption and the measured value of the working performance of the server shall be updated according to the previous control interval.
The estimated value of the relation parameters among the handling capacity of the server, the VCPU threshold value and the electric energy is recognized by the system, and a quadratic function is used to estimate the deviation value: In the formula, J is the objective function to be optimized; y k ( − 1) is the actual output value of the energy consumption of the server; ȳ is the estimated value of the energy consumption. The deviation between the utilization of energy consumption and resource configuration can be estimated according to Formula (3), and J is used to measure the fitness function during energy consumption optimization. One time interval is adopted for solution, and the minimum values of g k ( ) and H k ( ) i T are adopted, to solve the parameters of the energy consumption model of the liquid cooling server, which can be expressed as follows: In the formula, J p is the objective function of the energy consumption of the server. The minimum value of the relation parameter between energy consumption, performance and VCPU threshold value can be solved based on Formula (4). The minimum values of a k ( ) and b k ( ) are adopted for solving the parameters of the performance model of the server: In the formula, J r is the performance objective function of the server. Dynamic parameters are identified through Formula (5), to acquire the model relation between the dynamic power consumption and the performance of the server. The resource coordinator is used to solve the model between the power consumption and the performance; to select the objective utility function of the liquid cooling data center, normalization is conducted to r k ( ) i by being divided by r i and r i , and the objective utility function is expressed as follows: In which U is the objective utility function; r i is the objective handling capacity of the server; p s is the objective energy consumption of the server; α is the weight relation between performance and energy consumption. When α is small, the system gives priority in meeting the handling capacity of the server; when α is large, the system optimization result gives priority in meeting the energy consumption demand. Acquire the minimum value of the objective utility function of energy consumption based on Formula (6), and the VCPU threshold value and the weight α distributed to the server, which is reflected to the resource distributor of the server; the electric energy using conditions can be distributed in a more efficient way based on the optimization of the distribution result with the resource optimizer; in this way, it reduces waste in energy consumption and improves effective utilization rate of electric power.

| APPLICATION TEST
In order to verify the performance of the energy consumption management system in the liquid cooling data center in this research, experiments are conducted to the system in Literature, 3 the system in Literature 4 and the system in the research, to make comparison on the three energy consumption management systems. The experiment environment framework is shown in Figure 6. The experiment environment parameters are shown in Table 2.
Five physical servers are adopted in Figure 6, and network connection is conducted among servers, and NFS server is responsible for communication among data servers; the data server is used to collect and store the energy consumption data and optimize the resource configuration of the server; the host server is used to simulate the energy consumption of the data center server.
F I G U R E 6 Experiment environment framework LAI ET AL.

| 1289
The work load characteristics collected by the energy consumption management system of the research is shown in Table 3.
The experiment of the research takes the power consumption of the CPU in loading state as the experiment subject, and collects the power consumption of the system in each 10 s with the power consumption measurement software Power Meter; related data is delivered to the controller for estimation of the model, to reduce the energy consumption of the system at the premise of ensuring service quality.
The power consumption of CPU in the loading state is 80-100W, and the power consumption of the three servers in loading state is 240-300W, in which the power consumption of the displayer is not included; the research selects 260 W as the objective energy consumption, and the energy consumption of the servers of the systems is shown in Figure 7.
In the system of Literature, 3 according to the load balancing method of feedback dynamic weighting, a number of servers form a server set in a symmetrical way, and each server can provide external services independently. The external requests are evenly distributed to a certain server in the symmetrical structure, and the server that receives the requests independently responds to the customer's requests. Although the energy consumption of the server cluster can be somewhat reduced, the energy consumption fluctuates greatly. The changing range of the energy consumption of the server in the energy consumption management system in Literature 3 is large, with unstable energy consumption control effect. The maximum energy consumption of the server can achieve 290, 30 W higher than the set objective energy consumption, resulting in waste. The energy consumption of the server reduces to 230 W at most at 40 s, and very low energy consumption may lead to influences on the normal working state of the server.
In Literature, 4 the real-time regulation is added into the system. The DVFS is essentially a plurality of low dropout linear regulators with different output voltages, which are bridged by a selection switch, and the DVFS control logic outputs the control vector of the required voltage to automatically complete the voltage source switching. The core is the strategy of dynamic regulation, the purpose of which is to make real-time regulation according to the system load at that time, so as to provide the lowest power meeting the performance requirements at that time, and thus achieve the lowest power consumption. The lowest energy consumption of the server in Literature 4 is 245 W, which increases to 297 W at 95 s and then decreases rapidly. The energy consumption of the server is stable at 280 W, with difference of 20 W with the energy consumption of the set objective, indicating that the energy consumption system in Literature 4 is not high, with very large changing ranges in energy consumption values and instable operating energy consumption of the server.
This research system simulates the time-varying process by continuously updating the parameters by using a linear model, thereby truly realizing the precise control. By making comparison on the changing curve of the energy consumption of the server, it can be known that the server of the energy consumption management system of this research has the smallest fluctuation range on its energy consumption curve, and the better control effect F I G U R E 7 Energy consumption of the system servers and the most stable changes of its energy consumption. The changing range of the energy consumption of the server is small, which achieves the highest point of 270 W at 50 s and achieves the lowest point of 245 W at 62 s. The objective power of the server set in this experiment is 260 W, and the energy consumption of the control server of the energy consumption management system of the research is stable at the objective energy consumption value; in addition, the energy consumption of the server is adjusted at any time according to the operating load of the server, to avoid excessive large or small energy consumption of the server. The three systems are utilized for performance evaluation, and the handling capacity refers to the quantity of data that can be successfully transmitted per unit time, which can reflect the speed and accuracy of data transmission. The average value of the handling capacity of all the virtual machine is taken as the standard, to acquire the estimated error of the three systems, as shown in Figure 8.
Because the data flow of the system changes with time, and the amounts of data processed at different times are different, the estimation error may change with time. By making comparisons on the estimated accuracy of the three systems, it can be known that the estimated error of the energy consumption of Literature 3 can be higher than 3% at most, with large changing ranges, and the estimation result is easily to be influenced by dynamic changes in loads. The estimation error of the system in Literature 4 is always higher than 3%, or even 3.6% at most, with unfavorable estimation effect to energy consumption, leading to unfavorable energy consumption management effect. The model estimator of the energy consumption management system of the research has the best effect, with lower estimation error, which is always lower than 2.4%, and the estimation can change with the true energy consumption change during the experiment process, which can be as low as 2.08% at 14 s.
According to the test delay time of the energy consumption management system obtained from the experiment, the operation performances of the three system management methods were compared based on the simulation data results, and the effectiveness of this research is verified according to the data calculated by the microcomputer. The experimental results are summarized in the data sheet, and the data sheet of the final total data amounts and the average delay time of the three systems is shown in Table 4.
The analysis of the data in Table 4 shows that the total test data size of the energy consumption management system of this research is up to 6848 MB, and the average test delay time in its operation is 268 ms; in the case of the system of Literature, 3 the total test data size is up to 6134 MB, and the average test delay time in its operation is 563 ms; and in the case of the system of Literature, 4 the total test data size is up to 5825 MB, and the average test delay time in its operation is 682 ms. According to the experimental results, the delay time curves of three different energy consumption management systems are made, and the performances of these systems are analyzed by using the simulation curves, which is as shown in Figure 9.
In Figure 9, the test delay time of the energy consumption management system of this research is affected by the total data size, and before the total data size reaches 5400 MB, the delay time gradually increases to and then remains at 280ms; the delay time of the system in Literature 3 keeps increasing, the fastest increase occurs in the range from 5400 to 6600 MB, and the highest test delay time is 563 ms; the overall delay time of the system in Literature 4 increases with the increase of the total data size, and the highest test delay time is 682 ms. Based on the above results, it can be concluded that the energy consumption management system of this research is more applicable.

| CONCLUSIONS
The energy consumption management system of the liquid cooling data center is designed in the research, which clarifies the system functions and introduces the optimized algorithm of the system resources in detail. By designing the energy consumption data collection circuit, it realizes monitoring and analysis of energy consumption data of the server. By conducting on-line recognition and analysis on server parameters, it distributes the server resources with the self-adaptive control research coordinator. The minimum energy consumption of the server is controlled to 245 W, and its overall energy consumption does not exceed 270 W, indicating that the energy consumption control is stable; compared with the average handling capacity of all virtual machines, the estimated error is always lower than 2.4%, and the data transmission is stable. The overall energy consumption control of the energy management system of the liquid cooling data center designed in this research is both stable and accurate, thereby greatly reducing the energy consumption of the existing system. It is necessary to further improve the hardware structure for collecting the energy consumption data of the research, by expanding more interfaces and making it applicable to multiple application scenarios of data center. There are different CPU loads and internal resource demands in actual application scenarios, and the influences of such resources on the energy consumption of the data center shall be taken into consideration in research in the future.