A Stochastic Approach to Energy Cost Minimization in Smart-Grid-Enabled Data Center Network

We propose a Lyapunov drift-plus-penalty(LDPP-) based algorithm to optimize the average power cost for a data center network. In particular, we develop an algorithm to minimize the operational cost using real-time electricity pricing with the integration of green energy resources from the smart grid. -e LDPP technique can achieve significant energy cost savings under quality of service (QoS) constraints. Numerical results are presented to evaluate and validate our solution. -ese results illustrate significant operational/energy cost reductions for a data center network over the conventional approach which optimizes the predicted values of stochastic parameters under a fixed QoS constraint.


Introduction
Data centers are primary information and communications technology (ICT) energy consumers.For example, data centers in the USA consume 100 billion kWh per year which costs approximately 7.4 billion dollars [1]. Recently, the nextgeneration power grid, known as the smart grid, has been introduced to facilitate the participation of power consumers and the integration of renewable generation to balance power supply and demand in real-time [2]. is provides consumers with the ability to dynamically track electricity price variations and efficiently manage their power consumption.In this case, we can improve system efficiency by shaping and balancing the power loads (e.g., servers).e smart grid also promotes the use of sustainable and green energy resources such as wind and solar panels (SPs).
e number of demand applications, particularly video streaming traffic, e.g., video on demand (VoD), web browsing, online gaming, and IPTV, has substantial impact on the energy consumption of data centers and can dynamically change according to the number of user demands.us, as emerging services grow as well as the number of connected smart devices rises, the energy consumption of data centers is anticipated to increase significantly.
In the current work, we propose and investigate the performance of a Lyapunov optimization method considering actual real-time prices.Experimental data and computer simulation are used to assess the performance of the LDPP-based energy cost reduction strategy.
e results obtained show that this approach can significantly outperform the expected value algorithm (e.g., linear programming or LP (the LP method is a well-known benchmark method for solving linear constrained optimization problems)) in terms of energy cost savings.In this case, we also derived the probability of violating the QoS (connection delay) and processing (connection handling capacity of the data center) constraints.It is also shown that the proposed LDPP algorithm has a lower probability of violating the QoS and processing constraints compared to the conventional linear programming (LP) approach.
In summary, the main contributions of the present paper are as follows.
(i) We consider energy and cost savings for a data center network in the context of the smart grid.A real-time energy management algorithm is proposed to reduce the cost of server cluster operation according to real-time pricing, energy demand, and power supply estimation.We also incorporate green power generators such as energy storage devices and solar panels as a power supply for the server clusters.(ii) We develop an algorithm based on optimizing the total energy consumption cost in data center networks.is allows us to intelligently route requests among data centers according to the type of user demand application (web/VoD) and energy efficiency considerations.(iii) LP and LDPP methods are investigated to solve the optimization problem. is leads to a tradeoff between operational cost saving and computational complexity.(iv) We examine the number of batteries required to support the average daily streaming traffic for a data center network.(v) Finally, a probabilistic model is given to model the complementary cumulative distribution function (CCDF) of violating the QoS and processing constraints for data center network users, and the performance with LP and LDPP is compared.
e remainder of this paper is organized as follows.Section 2 provides a review of the related work including system models and some assumptions used in the paper.Section 3 presents the formulation of a optimization problem for real-time cost reduction for a green data center.Section 4 describes the proposed LDPP-based algorithm.e performance evaluation results are presented in Section 5, and finally some conclusions are given in Section 6.

Related Work.
ere has been significant research on reducing data center power costs and emissions in the smart grid context [3][4][5][6][7][8][9][10][11][12].Previous approaches have focused on the design of real-time energy management systems which optimize the data center network power costs according to system uncertainties while considering a quality of service (QoS) constraint such as connection delay.
ese uncertainties are related to stochastic parameters including real-time pricing, power supplies from renewable sources, and data center workloads.To characterize these uncertainties, the expected values of the parameters during specific time periods have been employed under QoS constraints.us, the problem formulations have been deterministic and solved using techniques such as linear programming (LP) to minimize the power costs.However, this does not necessarily maximize the power cost savings.
Previous methods for saving data center power costs can be categorized as minimizing electricity costs [3][4][5][6][7], decreasing energy consumption [8,9], or using renewable energy resources [10,12].Methods in the first category typically use the temporal and spacial variations in electricity prices from the smart grid to exchange connections among data centers to reduce operational costs.e second class of techniques aims at lowering the energy consumption of a data center and/or a data center network by considering cloud computing to reduce the power consumption of end users.e impact of real-time pricing on the operational cost and energy efficiency of the cloud transport network infrastructure was examined in [10].e third class of techniques employs renewable energy from local sources to reduce emissions and operational costs.For example, the green star network (GSN) testbed [10] was used to examine the practicality of powering a network of data centers with solar and wind power.
In [13], a stochastic optimization problem was formulated to tackle the stochastic renewable generation and workload arrival processes.en, an online control algorithm based on Lyapunov optimization was proposed to solve it.In [14], Mao et al. proposed a low-complexity online algorithm to minimize the long-term average network service cost, namely, the Lyapunov optimization-based base station assignment and power control (LBAPC) algorithm.
e main advantage of this algorithm is that the decisions depend only on the instantaneous side information without requiring distribution information of channels and energy harvesting processes.In [15], a gamebased traffic exchange mechanism for green data center networks was proposed.Unlike in [15], the current work, a fast online optimization method based on the Lyapunov drift is employed which can reduce the system complexity more efficiently.
Different from the approaches in [13], we integrate smart grid concepts into the optimization formulation and use a QoS metric for user satisfaction in the proposed constraints.In [14], a similar Lyapunov optimization approach was used in a different context which is base station assignment and power control.
is method minimizes the long-term average power cost under QoS and processing constraints using the Lyapunov drift-plus-penalty (LDPP) algorithm.
is approach can reduce the optimization to a greedy minimization problem which provides a significant cost reduction.
In this paper, an optimization model is developed for a data center network based on stochastic parameters extracted from green data centers, and a drift-plus-penalty framework is used to optimize the time-averaged power cost reduction.
is model includes the dynamic and static power consumption of the servers, electricity prices, the availability of renewable generation, QoS and processing constraints, and service payments as well as penalties.

Green Data Centers.
e data center model is composed of servers, users, power management control units, the 2 Journal of Computer Networks and Communications electrical grid, renewable power sources, power storage, and batch schedulers, as illustrated in Figure 1. e total power consumption within a data center typically consists of the consumption by the facility infrastructure and the ICT infrastructure, as shown in Figure 1.In the facility infrastructure, a cooling system is the main source of energy consumption, while in the ICT infrastructure, computing resources (servers) and switches are the major consumers.e facility infrastructure also includes additional power loads such as uninterrupted power supplies (UPSs), batteries, lighting, and switchgear.e sensor network in Figure 1 is one of the smart grid components for measuring energy usage.e power consumption of the computing servers has two major components: static c static and dynamic c dynamic .e latter is associated with the CPUs and is directly related to user traffic demand.Static power consumption does not depend on this demand and can be expressed as [17] where c idle is the average idle power of a server, U is the power usage effectiveness [17], and c avg is the average server peak power when it processes application requests (i.e., user demand).e dynamic power consumption can be expressed as where the superscripts rt and nrt denote real-time and non real-time (elastic) applications, respectively.α rt and α nrt are two positive weighting constants where α rt + α nrt � 1, α nrt ≪ α rt (it is assumed that the real-time applications are broadband so they consume more resources than their narrowband counterparts).ζ rt represents the number of real-time connections (such as video on demand, gaming, and IPTV), and ζ nrt is the number of elastic connections which are served from the data center.Since static power consumption does not depend on user demand and the focus of this paper is on dynamic power consumption, only the latter is considered in the remainder of this paper.Electrical energy resources including renewable generators, local power generators (e.g., diesel generators), and batteries for storing energy are illustrated in Figure 1.Renewable energy sources are solar panels and wind turbine, while local generators include diesel generators and batteries.A renewable source generates a random amounts of power due to the weather conditions at different time slots.For example, wind energy depends on wind power, and solar energy depends on the level of sunlight.e data center has power storage (i.e., batteries), to store the power generated from renewable sources as well as power purchased from the electrical grid.An intelligent sensor network which is composed of sensors, actuators, and a control unit as shown in Figure 1 is essential for stable and reliable data center operation.Sensors/actuators are attached on rectifiers (in the case of wind generator), converters, power generators, and loads to sense and measure parameters such as active power and passive power for processing in the control unit.e control unit facilitates real-time energy management via energy consumption scheduling.An energy consumption scheduling module can work in disconnected or connected modes of operations according to the shortage or surplus of electrical power.In the former mode, it locally balances supply and demand and if there is a surplus of energy, there is the possibility to save this energy in the UPS batteries.In the latter mode, the power grid and local energy resources are employed to balance supply and demand.In this case, there is a capability for energy consumption scheduling to facilitate real-time energy management and consequently reduce operation costs.It also obtains real-time pricing via a gateway and bidirectional communications with the utility operations center.

Optimization Model for Minimizing Power Cost
Assume that the composite random λ(t) ∈Λ(t) at time slot t is defined as and Z(t) denote the set of powers generated from renewable sources, the set of grid spot power prices, and the set of user demands due to real-time and nonreal-time applications, respectively, within the data center at time step t. e random variables r(t), p(t), ζ rt (t), and ζ nrt (t) take values from these sets.e time step t is from the set indicates the number of applications.e minimization of the power cost can be formulated as an optimization problem with uncertainty λ(t).
e objective is to minimize the expected cumulative cost over all time steps, i.e., E[•] over random variables λ(t).To minimize the cost at time t, the objective function f t (•) is defined with random variables λ(t) and λ(t − 1).e cost minimization problem can then be expressed as arg min where subject to Journal of Computer Networks and Communications c buy (t), c sel (t), c ren (t), p buy (t), p sel (t), p ren (t) ≥ 0, ∀t ∈ T. ( We have considered the composite random λ at (T-1) because, in the real-time energy management methodology, the energy costs in the previous time slot will be used to estimate current costs.For simplicity, we assume that the set of purchase prices of power from the grid p buy (t) is equal to the set of prices of power sold to the electrical grid p sel (t).We also assume that the power generated c ren (t) is saved in batteries so it incurs the set of costs p sel (t) associated with battery losses due to charging and discharging.Equation (5) ensures that the number of available servers is sufficient to handle all user demands.β is a constant based on the user demand according to the number of available servers.Equations ( 4)-( 6) provide the optimal solutions for c buy (t) and c sel (t).

Proposed LDPP-Based Algorithm for Cost Reduction Optimization
To solve the uncertainty model defined in ( 4)-( 6), the deterministic equivalent model [18] given λ(t) is employed: where  (11) ensures that the power input is equal to the power output in time step t.
Instead of solving the stochastic constrained optimization model defined in ( 8)- (11), it can be rewritten with the random variables replaced by their expected values.is formulation has much lower complexity since the number of variables and constraints are reduced.e corresponding deterministic formulation is where the expected value of c buy (t and the other expected values are obtained similarly, and c ren max represents the maximum storage capacity of the batteries. Linear programming (LP) can be used as a solution to the expected value in (5).However, LDPP can provide a solution with large power cost savings as explained in the following section.

LDPP-Based Algorithm for a Data Center Network.
In order to employ the Lyapunov-drift algorithm for the above stochastic optimization problem, let { } represent the ith data center within a data center network where the number of data centers is I.It is also assumed that real-time connection arrivals within each data center have a Poisson distribution.e service rate is exponential with parameter Mμ m,i for application m at data center i, where μ m,i is the average service time [19].Further, the QoS and processing requirements must be satisfied, so the transmission delay from a front end server to data center i denoted as d m,i should be less than a delay threshold D m,i for real-time application m at data center i.For a data center network, the problem formulation ( 8)-( 11) is then given by arg min where M i is the maximum number of applications in data center i. 1 Table 1 summarizes the parameters and variables used in the paper.
Let Q 0 ℓ (t), ℓ � 1, 2, • • • , L, denote the stable mean rates for the queues associated with the network routers.We also assume that a ℓ (•) and b ℓ (•) denote the packet arrival and packet departure processes, respectively, for the ℓth queue.
We now propose a Lyapunov stochastic optimization for the problem formulated in ( 16)- (20).To achieve this, we define virtual queues for values of m and i so that e penalty function P(c(t)) incurred by the virtual queues at time step t is defined as Finally, the stochastic optimization model based on the drift-plus-penalty framework defining vectors Θ 1×(L+5I+ ] is given in the following steps [16].
Journal of Computer Networks and Communications 5 (1) Observe Θ(t) and greedily choose the power consumption vector c(t) � [c buy i (t), c sel i (t), c ren i (t)] such that the following expression is minimized: (2) Update the queues according to (21): Algorithm 1 describes the proposed LDDP algorithm for a green data center network.
As mentioned above, an advantage of the LDPP-based optimization algorithm is that the decision variables depend only on the instantaneous stochastic information without requiring their distribution information.To determine the power cost reduction, we only need to solve a deterministic per-time step problem which validates the effectiveness of the proposed algorithm.
In the proposed constrained optimization problem ( 16)-( 20), we have a nonlinear constraint (20) which is due to end user QoS requirements.So, in the proposed optimization, we have nonlinear constraints, and this is the main reason that the conventional LP method may be suboptimal with respect to its LDPP counterpart.Next, we derive the constraint violation probabilities for the processing capability constraint (14) and the QoS constraint (20).

Journal of Computer Networks and Communications
where * denotes convolution and g rt (•) and g nrt (•) are the PDFs of the real-time and nonreal-time connection arrivals, respectively.e processing violation probability (or CCDF process ) is then

Numerical Results and Discussion
In this section, numerical results are presented to demonstrate the performance of the proposed real-time management algorithm for both the expected value (LP) and Lyapunov optimization solutions.e power cost savings are determined with regards to the optimized amount of grid energy which can be bought and/or sold.To obtain realistic results, we use the actual hourly real-time prices for 24 hour periods from [19] for various dates.erefore, data centers at different geographic locations within the data center network employ different real-time pricing.e price of electricity is updated every two hours, while the real-time energy management algorithm is executed every one hour for both the LP and Lyapunov optimization solutions.All simulation results are averaged over 1000 iterations with real-time prices from different 24 hour periods.α rt and α nrt are set to 0.98 and 0.02, respectively.All the thresholds in Algorithm 1 are set to 0.01.e simulations were done using MATLAB on a Quad-Core Intel processor with a 3 GHz clock speed and 4 GB of RAM.
It is assumed that the packet generation process follows a Poisson distribution [20,21] for voice (real-time) and elastic (nonreal-time) applications.It is also assumed that the video packet generation process follows a heavy-tail distribution, namely, a Markov-modulated Poisson process (MMPP) [20,21].An MMPP is a doubly stochastic Poisson process whose average number of events in an interval, i.e., the event rate, varies according to a Markov process.ere are I � 7 data centers, and each employs different prices from [22].Practical 5 KWh batteries are used for storage of the energy from solar panels according to [23].We also assume that the data centers have different storage capacities, i.e., different numbers of solar panels.e proposed algorithm is evaluated for both LP and LDPP optimization.We compare a regular data center network which only uses the grid for power with a green data center network which uses both the grid and solar panels and balances the power supply and demand using real-time energy management.For simplicity, we only consider realtime traffic for streaming connections.Other types of traffic can be similarly examined.
e number of solar panels is allocated based on the variation in traffic within different data centers.In this case, the number of allocated batteries versus the average daily streaming traffic for a data center network is shown in Figure 2. is indicates that when the traffic within a data center network is increased, the number of batteries is also increased in order to use more green energy as well as to reduce the power costs.It is important that the marginal cost increases due to increasing battery costs is much smaller than the increasing costs related to a comparable fossil-based (nongreen) energy consumption scenario because the nonrenewable energy costs are typically much higher.
Figures 3(a) and 3(b) present the cost savings over a day (24 hours) versus the average number of streaming connections [ζ rt stream ] ave for the LP and Lyapunov optimization methods, respectively, when the total number of batteries in the data center network is 800.Figures 3(a) and 3(b) show that while considering a maximum amount of tolerable delay (D stream ≤ 10 s) for video streaming applications according to [24], algorithms with real-time energy management (LDPP) can achieve significant power cost savings over a conventional system without real-time energy management (LP) for different average real-time connection densities ([ζ rt stream ] ave ).Further, the LDPP-based optimization performs much better than the LP-based optimization while meeting the QoS or maximum tolerable delay constraint, i.e., D stream ≤ 10 s. ese cost savings are due to the real-time stochastic nature of the Lyapunov-based optimization method which can improve the performance in environments with uncertainty (random user arrival processes) and nonlinear characteristics (nonlinear QoS constraints).
e complementary cumulative distribution function (CCDF) of violating the QoS constraint, CCDF QoS , is shown in Figure 4 for both the LP and LDPP algorithms.Similarly, the CCDF of the probability of violating the processing constraint, CCDF Process , for both the LP and LDPP algorithms is illustrated in Figure 5.As expected, LDPP has a lower violation probability for both QoS and processing constraints compared to the LP algorithm.Figures 4 and 5 show that the difference between the two algorithms increases as the maximum tolerable delay D stream and average numbers of connections [ζ rt stream ] ave increase.Figures 4 and 5 indicate that, as the simultaneous average real time connection density ([ζ rt stream ] ave ) increases, the CCDF (QoS) and CCDF (process) increase because of the added uncertainty in the system due to larger user traffic behavior variations.On the other hand, by increasing the maximum tolerable delay D stream , for real-time connections, the CCDF (QoS) and CCDF (process) decrease because of lower QoS sensitivity of real-time applications.
Table 2 presents the time complexity of the conventional LP method and the proposed LDPP algorithm.is shows that the LDPP algorithm performs better than the LP method in terms of operational cost reduction.We define the operational cost reduction ratio for the proposed optimization (using LP and LDPP) as 1 − (Cost Proposed /Cost Conventional ).Here, we mean by Conventional an ordinary data center network without employing optimized real-time energy management.e LDPP algorithm can save between 55% and 60% of the operational costs over the conventional system for different numbers of solar panels (SPs). is saving is decreased between 22% and 41% if we consider LP over the conventional system.e computation time of the LDPP algorithm and LP approximation for different values of SP is shown in Table 2.We use the ratio of the LP optimization execution time for the proposed technique to that of the LDPP algorithm in  Journal of Computer Networks and Communications 9 order to compare the computational complexity, i.e., 1 − (time LP /time LDPP ). e optimization time for each algorithm is averaged over 1000 optimizations.Table 2 shows this ratio versus the total number of solar panels, SP. is indicates that, in the best scenario, the computation time of the LP approach can be as small as 88.5% (100-11.5) that of the LDPP algorithm.
us, the proposed approach can provide an operational cost saving versus complexity tradeoff for various total numbers of solar panels (SP).However, mean execution time for the LDPP algorithm is not much longer than the mean execution time for the LP algorithm.us, LDPP is superior to LP in terms of operational cost savings and has reasonable computational complexity.

Conclusion
In this paper, a Lyapunov drift-plus-penalty-(LDPP-) based algorithm was with green energy resources to improve the operational cost of data center networks.e LDPP technique can achieve significant power cost savings via minimizing the long-term average power cost under quality of service (QoS) constraints according to the drift-plus-penalty algorithm.Numerical results were presented which demonstrate the reduction in power cost and the robustness of the proposed LDPP-based technique.Moreover, the proposed method has reasonable computational complexity compared to the conventional LP approach.Further, the proposed LDPP method can deliver better quality of service compared to conventional techniques based on the CCDF metric.For future research, machine learning approaches (specifically deep learning) can be used to enhance the system performance in terms of convergence speed and overall cost savings.

Figure 1 :
Figure 1: Typical data center network including facility and ICT infrastructure with a green power supply including renewable sources (solar panels and wind turbines) and local sources (diesel generators and batteries), as well as sensor controls and internal communications systems.

8Figure 2 :
Figure 2: Number of batteries for a data center network versus the average number of streaming connections [ζ rt stream ] ave and daily hours.

Figure 3 :Figure 4 :
Figure 3: Daily cost savings versus the average number of streaming connections [ζ rt stream ] ave with a maximum tolerable delay of D stream ≤ 10 s real-time energy pricing and 800 5 kWh batteries with the (a) LDPP algorithm and (b) LP algorithm.

Figure 5 :
Figure 5: Probability of violating the processing constraints, CCDF Process , versus the number of streaming connections ζ rt stream

Table 1 :
Nomenclature.Number of real-time/elastic application m connections served by DC i at time slot t c buy (t)/c sel (t) Power amount bought/sold from/to electrical grid at slot t p buy (t)/p sel (t) Power price bought/sold from/to electrical grid at slot t c ren (t) Amount of renewable generation at slot t c buy (t)/c sel (t) Mean power amount bought/sold from/to electrical grid at slot t p buy (t)/p sel (t) Mean power price bought/sold from/to electrical grid at slot t c ren (t) j (t) Virtual queue j backlog at time slot t, j � 1, • • • 9 Violation (t) � 1 − P QoS (t), ∀t.(25) e probability that the processing constraints are satisfied in the proposed multiapplication/multidata center system, i.e.,  m∈M i [α rt ζ rt m,i (t) + α nrt ζ nrt m,i (t)] ≤ βM i , ∀i, is denoted by P Proc and is expressed for each time step t as

Table 2 :
Operational cost and computational complexity comparison of LP and LDPP algorithms for solving energy cost optimization.