Finding Optimal Allocation of Constrained Cloud Capacity Using Hyperbolic Voronoi Diagrams on the Sphere

We consider a network of computer data centers on the earth surface delivering computing as a service to a big number of users. The problem is to assign users to data centers to minimize the total communication distance between computing resources and their users in the face of capacity constrained datacenters. In this paper, we extend the classical planar Voronoi Diagram to a hyperbolic Voronoi Diagram on the sphere. We show that a solution to the distance minimization problem under capacity constraints is given by a hyperbolic spherical Voronoi Diagram of data centers. We also present numerical algorithms, computer implementation and results of simulations illustrating our solution. We note applicability of our solution to other important assignment problems, including the assignment of population to regional trauma centers, location of airbases, the distribution of the telecommunication centers for mobile telephones in global telephone companies, and others.


Introduction
Cloud Computing could transform the way we use computer hardware by eliminating the need to own a computer. Instead of placing expensive, fully loaded computers in every school, students all over the world could provision computing resources as they need them. A Gartner research report places the total savings of moving away from individually managed desktop computers at 41%. Behind the Public Cloud is a physical network of data centers with millions of servers, hidden from us through virtualization. Virtualization offers Public Cloud providers the flexibility to assign each application to run on any of its available physical servers.
The vision of less expensive Tablets powered by the remote computing resources of the Cloud is still in its early days. Combined, the emerging Tablet computers coupled with central Cloud Computing facilities to host applications and data will significantly improve schools ability to increase the use of computers in the classroom, allowing students to expand computing power on demand for educational applications anytime anywhere. However, with the transition to the cloud, students' desktops must be moved from local in-classroom computers to large, centralized datacenters. These facilities are typically hundreds, if not thousands, of miles away from the schools and the homes of students. This distance in-troduces network delay between the students and their desktops. The utility of these students' remote desktops depends on minimizing this network delay.
Presently, the approach cloud providers use to allocate users to a datacenter is referred to as Global Server Load Balancing, or GSLB. A user accesses a computer in the cloud using an Internet domain name. The domain name (for example, www.whitehouse.gov) is translated by a Domain Name Service (DNS) into an Internet Protocol (IP) address pointing to a specific computing device. Where there are multiple datacenters that could satisfy the request, the DNS is programmed to return the IP address of the closest computing device, or the one with the least network delay. The decision of where to put a user is made individually for each connection request. This approach works well for accessing websites where the content is replicated across all cloud locations, such as search engines, social network sites, news, and weather. For student's desktops, a more static assignment is needed. The CAP Theorem [1] states that a distributed system cannot simultaneously provide Consistency, Availability, and Partition Tolerance. Even if students could afford the cost of replicating their desktops continually between the multiple geographically disperse datacenters selected by a latency-based GSLB, the need for desktop consistency over a WAN would exacerbate the impact of Availability events or Partitioning events.
When based in a cloud, students' desktops require a continual static binding to the network access point of the student. The cloud-based desktop should be highly available (for example, using an Availability Zone [2] architecture such as employed by Amazon Web Services). Latency-based GSLBs are not useful for such constant binding of two locations on the earth's surface. A form of GSLB is required which can statically assign students to cloud datacenters.
One other problem remains. Cloud datacenters, despite their large capacity, are finite in size. They can become constrained in capacity. The solution for balancing students across cloud facilities must take into account occasional capacity constraints and allow weighting of assignment to potentially more distant facilities to compensate.

Problem Formulation
Combined, the emerging tablet computers coupled with central cloud computing facilities to host applications and data significantly improve schools' ability to increase the use of computing in the classroom. Cloud Computing can provide computing as a service to millions of students world-wide. These students would need tablet computers connected to the cloud via common Wi-fi networks and the Internet. The remaining technology required to connect these tablets is a placement service capable of considering proximity and any capacity constraints. Proximity is important due to network delay and the high correlation of this delay to physical distance. Because the relationship between students and their data is continuous, existing GSLB (as described above) will not meet this need. The new placement technology would need to consider occasional capacity constraints as datacenters are physical buildings limited by critical, longlead-time resources. These include electrical power, cooling capacity and servers, which require many weeks to order, ship, and install. The ideal placement technology could consider an entire population of students and optimally assign it to either the closest datacenter or an alternate should capacity in that datacenter be unavailable. We would like to develop an algorithm which could globally optimize for the variables of population, distance, and specific points of capacity on the earth's surface representing datacenters.
Our starting point in developing a mathematical framework is to model the network of computer datacenters as a set 1 n on the earth's surface, represented by a sphere . Each datacenter j S is described by its geographical coordinates, the latitude j  and the longitude j  . The population of students (users) is a set . Our task is to create an efficient algorithm to solve the mini-mization problem for the total distance between the data centers and the students. We assume that each student k is assigned to a data center    U j k , and the total distance is the sum of the distances between the users and the corresponding data centers, is the function assigning the user k to the data center j k S . For any two points x, y on the sphere we denote   , d x y the distance between x and y on the sphere, that is the length of the arc of the great circle connecting x to y. We also must solve the minimization of the total distance under certain constraints. In real life some data centers become capacity constrained, so the solution must take computing capacity into consideration. We will assume that each data center j S has a capacity j C which characterizes how many users it can service. For simplicity we assume here that all users are equal in terms of the requested volume of services. It is easy to extend our model to the case when different users request different volume of services.
The assumption of the limited capacity for the data centers leads to the minimization problem with constraints: find an assignment function such that is just the number of users assigned to the data center m . Since the number of users is usually large, it can be useful to consider a (nonnegative) measure of users on the sphere. In this case, the total distance functional looks like where j U is the assignment function of the users to the data centers. Respectively, constraint (3) takes the form To solve the minimization problem we explored the possibility of extending classic Voronoi Diagrams. Voronoi Diagrams are a well-known geometric tool for answering distance related queries. Informal use of Voronoi Diagrams can be traced back to Descartes in 1644. In 1854 British physician John Snow used a Voronoi Diagram to illustrate how the majority of people who died in the Soho cholera epidemic lived closer to the infected Broad Street pump. There have been numerous applications in science and technology since [3][4][5][6][7][8].
Voronoi Diagrams have the unique property that any point within a Voronoi cell is closer to the vertex of that cell than any other vertex. As we considered how the edges of a Voronoi cell behave in reaction to vertices constrained in the number of points they could service, we found another well-known geometric construct, the hyperbola, to be especially applicable. Our result is described below in Theorem 3.1, where we show that the solution to the minimization problem in the face of capacity constraints is given by a new, extended form of Voronoi Diagram, hyperbolic Voronoi Diagrams on the sphere. We believe this form of Voronoi Diagram could prove useful in solving many problems beyond the student-tablet assignment issue we are tackling here.

Solution of the Constrained Minimization Problem
Let us begin with the unconstrained minimization problem. In the absence of the capacity constraint, minimization problem (2) can be solved by assigning each user U to its closest data center j S , so that is the closest data center to U. Geometrically, we partition the ere Observe that the cells Voronoi Diagram. To describe the minimizing assignment in the presence of the constraint we will introduce hyperbolic Voronoi Diagrams on the sphere. To get an insight into the minimization problem, it's easiest to start by considering the case of a small number of data centers and then formulate a general result. Begin with two data centers. Two data centers. For two data centers 1 , 2 , the Voronoi Diagram is described by the spherical bisector of 1 , 2 . The bisector is the arc of the great circle perpendicular to the arc of the great circle through 1 , 2 at the middle point (Figure 2(a)). The bisector partitions the sphere into two hemispheres, 1 Suppose now that the data center 1 has a limited capacity 1 and the numbers 1 of the users in 1 S C N  is bigger than 1 . Some users must be redistributed in the cell 1 C  from the data center 1 to the one ( Figure  2(b)). This will increase the total distance by is the set of users redistributed from S 1 to 2 (Figure 3 This minimization condition implies that if , are any two users such that because otherwise we can switch back to S 1 and to and in this way decrease in (8). Let Then (10) means that 2 1 . Let be any number between and . Then Consider the spherical hyperbola on the sphere R  (cyan line on (Figure 3(b))), defined as a locus of points x satisfying the equation, into two regions 1 and 2 such that 1 1 and 2 2 (Figure 3(b)). The regions and are described as The regions 1 , 2 are the cells of the hyperbolic Voronoi Diagram of the points 1 , 2 with the parameter d. We obtain from (13) that to minimize the total distance we have to find such d that the number of users in the region is equal to the capacity and to assign all users in to the data center .
Now let's look at the case of three data centers. Three data centers. For three data centers 1 , 2 , 3 the above considerations prove the following. Suppose that 0 is the minimizing assignment, and let In general, the numbers ml are not unique, because they can be chosen from the intervals (17). I can state that it is possible to choose these numbers in such a way that  , be the set of users which assigns to m , . Then there exists numbers such that for , Let us move k from 1  to 2  , l from 2 U  to 3  , and p U from 3  to 1  . Then the number of users in 1  , 2  , 3  will not change but the total distance will decrease by  (Figure 4(c)).
hence 0 is not the minimal total distance assignment. This contradiction proves the possibility of choosing the numbers , and in such a way that (18) 3 . Then to minimize the total distance function assign the users in the cell to the data center , . S The parameters , 2 , 3 are determined by the capacities , 2 , 3 . We assume that the total capacity 1 2 3 of the data centers exceeds the number of users, because otherwise it would be impossible to service all users. The following three cases can appear: , so that m , , are the classical Voronoi cells on the sphere (Figure 4(a)).
2) One constraint is active, say, 1 C . In this case, 1 , 2 3 , and the parameter 1 is determined by the condition that the number 1 of users in the cell is equal to (Figure 4(b)). 3) Two constraints are active, say, 1 C , 3 . In this case, 1 , 2 , 3 , and the parameters 1 , are determined by the conditions that the number of users in the cell is equal to C for The subsequent sections present numerical algorithms, computer implementation and results of simulations illustrating the solution for 2, 3, and 4 data centers, and also, for a big number of data centers.
General case.

Numerical Algorithms and Computer Simulations
consists of the vertices and the edges , which are the curves separating neighboring cells.
ml  A solution to the constrained minimization problem can be obtained as follows. There is a natural assumption that the total capacity of all data centers is not less than the number of the users, 1) no constraints;

S S S S
2) limited number of users in the specific cells; .
(29) 3) equal numbers of users in each Voronoi cells.
In this section we discuss our algorithm in details and present results of our simulations under the constraints (a) and (b) for 2, 3, and 4 data centers. The case of many data centers will be discussed later.
Otherwise, it would be impossible to service all the users. Two data centers. 1 , 2 . As an illustration, start with 1 in Seattle (the magenta diamond in the left upper corner in Figure 5) and 2 in Atlanta (the red diamond in Figure 5), assuming that all users are located in the USA. Let's also assume that all citizens of the USA are the users. The arc   1 2 of the great circle connecting 1 to 2 is shown by the yellow line in A full proof of this theorem can be obtained by extending the logic described for three data centers to a general case. We omitted it here due to length.  Copy ht © 2012 S and the longitude 0  . Notice that the great circle connecting Seattle to Atlanta (the yellow line) is not a perfect straight line, and it is slightly curved in the projection.

 
The spherical bisector 0  of , , , (the cyan line in Figure 5) is a great circle through the midpoint of the arc But what if the data center in Atlanta has a capacity of only 80 million users? Then we need to redistribute some users from Atlanta to Seattle. Since the goal is to minimize the total distance, the answer is to find a hyperbola    1 : , such that the population in Atlanta  is close to 80 million, but not more. This can be found by successive approximations and by a multiscale analysis of the population distribution.
To analyze the population distribution, start with a large scale distribution, in which the states are represented by their geographic centers and the whole state population is put in the state geographic center. By successive approximations, the hyperbola (32) can be found which gives a large scale solution to the constrained minimization problem. Then, a finer scale is used in which the population of the states near the hyperbola of the large scale solution is represented by smaller units. This results in a solution to the constrained minimization problem at the finer scale. Then, if needed, a second finer scale can be considered, and so on. In the multiscale analysis of the population distribution, we consider big metropolitan areas like New York, Chicago, Houston, etc. as point masses, with all the population of the metropolitan area concentrated in one point.
For the large scale solution to the constrained minimization problem, use successive approximations of the parameter d in (32). In the first step , where (1) see a thin magenta line in Figure 5. An analysis of the population distribution gives that about 127 million citizens live in the region Seattle and about 182 million in the one Atlanta , with respect to the hyperbola 1  . It is still more than 80 million. Therefore, the second step is to take the hyperbola 2  through the point 2 X on the the second magenta line in Figure 5, resulting in about 209.6 million citizens living in the hemisphere Seattle and about 99.4 million in the hemisphere Atlanta , with respect to the hyperbola 2  . We continue with this, taking the third, fourth, and subsequent approximations. The first four approximations are shown by thin magenta lines in Figure 5. Observe that the successive hyperbolas j  are jumping back and forth.
We find that the 8-th approximation gives 75.1 million and the 9th one, 83.8 million. After that, all the subsequent approximations oscillate between these two numbers. The reason is the large scale representation of the population. The difference of 8.7 million comes from the state of New Jersey. Therefore, we consider a finer scale for the representation of the population distribution in New Jersey and other states near the hyperbola. In the finer scale we find, by successive approximations, a solution with 79.2 million population inside hyperbola (32). This solution is shown by a thick red line in Figure 5. In a similar way, if needed, subsequent approximate solutions with even better approximation to 80 million can be obtained.
Three data centers. The next step is to move on to three data centers 1 , 2 , and . As an illustration, we will consider 1 in Seattle, in Atlanta, and in New York, see (the cyan lines in Figure 6), which intersect at some point 0 X (the red point in Figure 6). The point 0 X is the vertex of the spherical Voronoi Diagram with the three points , 2 , and 3 . The bisectors ij  There is a nice relation between the spherical Voronoi D th iagram and the convex hull H of the points   1 , , n S S  in the 3D space. This relation was observed by ], [10] (see also [11]

Numerical Algorithm for Many Centers
In the case of many data centers   S . The following assumption is made, which seems plausible for the computer cloud n works: for most m S 's the capacity m C is big enough so i does not create any constraint. Mathematically, there is an assumption that for these m S 's we have an unlimited capacity, m C  . It is also plausible to assume that the cells with constraints are isolated. Both assumptions stem from the practical obs rvation that capacity constraints res lt suboptimal operational characteristics for computer clouds, and are therefore uncommon. In this case we need to change the Voronoi Diagram only locally. This can be done similarly to the described above procedure for a small number of data centers.
As an example, look at a model minimization problem for 10 data centers in Seattle, Atlanta, New York, Phoenix, San Francisco, Denver, Houston, Chicago, Boston, an et t  e u in d Miami (see Figure 10). The Voronoi Diagram on the sphere for these centers is shown by the cyan line. The distribution of the population in the Voronoi cells looks as follows: Seattle-13.8 million, Atlanta-48.5 million, New York-61.5 million, Phoenix-6.7 million, San Francisco-41.1 million, Denver-18.6 million, Houston -31.5 million, Chicago-57.6 million, Boston-10. 9 illion, and Miami-18.7 million. Suppose j S . m centers in Atlanta and San Francisco have limited capacities of 20 and 15 million users, respectively. Then their users must be redistributed to the neighboring centers. To solve the minimization problem with these constraints, the hyperbolic Voronoi Diagram is first constructed. Then the iterative method described above is applied together with the multiscale analysis of the population. As a result, the hyperbolic Voronoi Diagram on the sphere depicted in Figure 10 is arrived at. It differs from the initial Voronoi Diagram only locally, near Atlanta and San Francisco. The change from the initial Voronoi Diagram to the new one is shown in red in Figure 10.

Conclusions
In this work we stud total communicatio the condition of restricted capacity of the data centers. We assumed that the earth is a perfect sphere of radius R, so that the earth distance between two points is the length of the smaller arc of the great circle connecting these points. Our main result is Theorem 3.1, which shows that a solution to the minimization problem is given by a hyperbolic Voronoi Diagram constructed on the data centers 1  plementation for the construction of hyperbolic Voronoi Diagrams satisfying the capacity conditions. We conside the n ta centers, 2, 3, and 4, and for a large number of data centers. In the latter case we make a plausible assumption that most of the data centers have sufficient capacity to service the clients, and the data centers of insufficient capacity are isolated. This allows local construction of the hyperbolic Voronoi Diagram from a standard Voronoi Diagram.
Although we discuss the application to the computer cloud only, it is interesting to note that our solution and numerical algorithm can be used to solve other important assignment prob cation of air-bases [8], the assignment of population to regional trauma centers, the distribution of facilities in global Internet companies like Amazon.com, the distribution of the telecommunication centers for mobile telephones in global telephone companies, data collection centers, and others.