Analysis of cycle stealing with switching times and thresholds

Abstract

We consider two processors, each serving its own M/GI/1 queue, where one of the processors (the “donor”) can help the other processor (the “beneficiary”) with its jobs, during times when the donor processor is idle. That is the beneficiary processor “steals idle cycles” from the donor processor. There is a switching time required for the donor processor to start working on the beneficiary jobs, as well as a switching back time. We also allow for threshold constraints on both the beneficiary and donor sides, whereby the decision to help is based not only on idleness but also on satisfying threshold criteria in the number of jobs.

We analyze the mean response time for the donor and beneficiary processors. Our analysis is approximate, but can be made as accurate as desired, and is validated via simulation. Results of the analysis illuminate principles on the general benefits of cycle stealing and the design of cycle stealing policies.

Introduction

Since the invention of networks of workstations, systems designers have touted the benefits of allowing a user to take advantage of machines other than her own, at times when those machines are idle. This notion is often referred to as cycle stealing. Cycle stealing allows such a user, Betty, with multiple jobs, to offload one of her jobs to the machine of a different user, Dan, if Dan’s machine is idle, giving Betty two machines to process her jobs. When Dan’s workload resumes, Betty must return to using only her own machine. We refer to Betty as the beneficiary, to her machine as the beneficiary machine/server, and to her jobs as beneficiary jobs. Likewise, we refer to Dan as the donor, to his machine as the donor machine/server, and to his jobs as donor jobs.

Although cycle stealing provides obvious benefits to the beneficiary, these benefits come at some cost to the donor. For example, the beneficiary’s job may have to be checkpointed and the donor’s working set memory reloaded before the donor can resume, delaying the resumption of processing of donor jobs. In our model we refer to these additional costs associated with cycle stealing as switching times.

A primary goal of this paper is to understand the benefit of cycle stealing for the beneficiary and the penalty to the donor, as a function of switching times. A secondary goal is to derive parameter settings for cycle stealing. In particular, given non-zero switching times, cycle stealing may pay only if the beneficiary’s queue is “sufficiently” long. We seek to understand the optimal threshold on the beneficiary queue when switching to help, and the optimal threshold on the donor queue when switching back. More broadly, we seek general insights into which system parameters have the most significant impact on the effectiveness of cycle stealing.

We assume there are two queues, the beneficiary queue and the donor queue, with independent arrival processes and service time distributions operating as M/GI/1/FCFS queues. Jobs arrive at rate ${\lambda }_{\text{B}}$ (respectively, ${\lambda }_{\text{D}}$) at the beneficiary (respectively, donor) queue and have service requirement $X_{\text{B}}$ with distribution $G_{\text{B}}$ (respectively, $X_{\text{D}}$ with distribution $G_{\text{D}}$). The load made up by beneficiary (respectively, donor) jobs is denoted by ${\rho }_{\text{B}}$ (respectively, ${\rho }_{\text{D}}$) where ${\rho }_{\text{B}}={\lambda }_{\text{B}}E[X_{\text{B}}]$ and ${\rho }_{\text{D}}={\lambda }_{\text{D}}E[X_{\text{D}}]$. If the donor server is idle, and if the number of jobs at the beneficiary queue is at least $N_{\text{B}}^{\text{th}}$, the donor transitions into the switching state, for a random amount of time, $K_{\text{sw}}$. After $K_{\text{sw}}$ time, the donor server is available to work on the beneficiary queue and the beneficiary queue becomes an M/$G_{\text{B}}$/2 queue. When the number of donor jobs in queue reaches $N_{\text{D}}^{\text{th}}$ (either during $K_{\text{sw}}$, or during the time the donor is helping the beneficiary), the donor transitions into a switching back state, for a random amount of time, $K_{\text{ba}}$. After the completion of the switch back, the donor server resumes working on its own jobs until the donor queue is empty. The donor server cannot work on any job while the donor is in the switching or switching back state.

A few details are in order. First, in the above model, the donor processor continues to cooperate with the beneficiary even if there is no beneficiary work left for it to do—the donor processor can switch back only when a donor job arrives.¹ Second, we assume that if the donor processor is working on a beneficiary job and a donor job arrives, that beneficiary job is returned to the beneficiary queue and will be resumed by the beneficiary processor as soon as that processor is available. The work done on the job is not lost (i.e. preemptive resume).² Our performance metric throughout is mean response time (overall and for each class of jobs), where the response time of a job is the time from when the job arrives until it completes service. We assume the first three moments of the service times are finite, and queues are stable.

Consider the simplest instance of our problem—where the service requirements of all jobs are drawn from exponential distributions and the switching times and thresholds are zero. Even for this simplest instance the continuous time Markov chain, while easy to describe, appears computationally intractable. This is due to the fact that the stochastic process having state (number of beneficiary jobs, number of donor jobs) grows infinitely in two dimensions and contains no structure that can be easily exploited in practice to obtain an exact solution. While solution by truncation of the Markov chain is possible, the errors that are introduced by ignoring portions of the state space (infinite in two dimensions) can be significant, especially at higher traffic intensities. Thus truncation is neither sufficiently accurate nor robust for our purposes.

To our knowledge, there has been no previous analytical work on cycle stealing with switching times and thresholds. The analysis of cycle stealing without switching times and thresholds under exactly our model has been studied by Foley and McDonald [5]; they prove asymptotic, heavy-traffic bounds on the performance of cycle stealing under exponential job size distributions.

A related model to our cycle stealing model is the “coupled processor model,” which we elaborate below. In this model two processors each serve their own class of jobs, and if either is idle it may help the other, increasing the rate of the other processor. This help incurs no switching time and has a benefit even if only a single job is present (i.e. two processors can work on the single job). However, because the processors work in concert, rather than on different jobs, these systems will gain no multi-server benefit when serving highly variable jobs; short jobs may get stuck waiting behind long jobs in the single queue for each class. All works we mention below consider Poisson arrivals.

Early work on the coupled processor model includes papers by Fayolle and Iasnogorodski [4] and Konheim et al. [8]. Both papers assume exponential service times, deriving expressions for the stationary distribution of the number of jobs of each type. Fayolle and Iasnogorodski use complex algebra, eventually solving either a Dirichlet boundary value problem or a Riemann-Hilbert boundary value problem, depending on the accelerated rates of the servers. Konheim et al. assume that the accelerated rate is twice the original rate, which yields simpler expressions (still in the form of complex integrals). While it is possible to numerically evaluate these analytical expressions, they were not evaluated in either work; thus no intuition was provided on the performance of these systems.

The above work is extended by Cohen and Boxma [3] to the case of general service times. They consider stationary workload, which they formulate as a Wiener-Hopf boundary value problem. This leads to expressions involving either integrals or infinite sums; if the queues are symmetric simpler expressions for mean total workload are found, but not for mean response time. They again have the two processors working in concert, without a switching time, providing analytical expressions, rather than numerical values.

In more recent work, Borst et al. [2] apply a transform method to the expressions in [3], yielding asymptotic relations between the workloads and the service requirement distributions. This leads to the insight that if a processor has a load less than one, it is “insulated” from the heavy-tail of the other, as long periods without cooperation will not lead to large backlogs. This is not the case if the load is greater than one, as the queue now must rely on help to be stable. Borst et al. [1] consider a similar question under a related model of generalized processor sharing, where n classes of jobs can share a processor with arbitrary weights. Using probabilistic bounds, they show that different service rates can either insulate the performance of different classes from the others or not, again depending on whether the non-cooperative load is larger than one. Both of these papers are concerned with the asymptotic behavior of workload, whereas our work isolates mean response time. Our work is thus complementary to these results.

This paper presents the first analysis of cycle stealing under general service requirements with switching times and thresholds. Recall that the difficulty in analyzing cycle stealing is that the corresponding stochastic process has state space that grows infinitely in two dimensions (2D), making it computationally intractable. The key idea in our approach is to find a way to transform this 2D-infinite Markov chain into a Markov chain that is infinite in only one dimension (1D-infinite Markov chain), which can be analyzed efficiently. The questions in applying such a transformation are (i) what should the 1D-infinite Markov chain track, and (ii) how can all the relevant information from the 2D chain be captured in the 1D-infinite Markov chain. Our 1D-infinite Markov chain tracks the number of beneficiary jobs, yielding measures of beneficiary performance. For the donor jobs, our state space contains only limited knowledge, $0\text{,}1\text{,}\dots \text{,}{N}_{\text{D}}^{\text{th}}$, or $\ge$$N_{\text{D}}^{\text{th}}$ jobs. Nevertheless we are able to capture all necessary information by using special transitions in our Markov chain, where these transitions represent the lengths of an assortment of busy periods. We refer to this technique as dimensionality reduction. The difficulty in dimensionality reduction lies in specifying the right busy periods, some of which transcend the definition of the analytical model.

Once the 1D-infinite Markov chain is specified, the hard work is finished, since the limiting probabilities in this chain can be solved efficiently using known numerical (matrix analytic) techniques, which in turn gives mean response time for beneficiary jobs. The mean response time of donor jobs is analyzed as an M/GI/1 queue with generalized vacations, and all the necessary information is provided by the limiting probabilities of the 1D-infinite Markov chain. While a closed-form solution may be preferable, our chain is compact enough, and matrix analytic methods powerful enough, that only a couple of seconds are required to generate most of the results plots in this paper. Furthermore, our method generalizes to more complex models, e.g. multiple donors (Section 7).

Our analysis is approximate but can be made as accurate as desired. The primary approximation lies in representing the length of the busy periods by phase type (PH) distributions. The beneficiary service requirement ($X_{\text{B}}$) and the switching time to help ($K_{\text{sw}}$) must also be approximated by PH distributions, although $X_{\text{D}}$ and $K_{\text{ba}}$ can be general. In this paper, we use a PH distribution, shown in Fig. 1, to capture the first three moments of the busy periods, and verify that this is sufficient via simulation (see Section 5).³ For $X_{\text{B}}$ and $K_{\text{sw}}$, we assume throughout the paper that they have PH distributions, and hence no approximation is involved.

Our analysis yields many interesting results concerning cycle stealing, detailed in Sections 4 Stability, 6 Results of analysis. While cycle stealing obviously benefits the beneficiaries (beneficiary jobs) and hurts the donors (donor jobs), we find that when ${\rho }_{\text{B}}>1$, cycle stealing is profitable overall even under significant switching times, as it may ensure stability of the beneficiary queue. For ${\rho }_{\text{B}}<1$, we define load-regions under which cycle stealing pays. We find that in general the switching time is only prohibitive when it is large compared with $E[X_{\text{D}}]$. Under zero switching time, cycle stealing always pays.

Two counterintuitive results are that when ${\rho }_{\text{B}}<1$, the mean response time of the beneficiaries is surprisingly insensitive to the switching time, and also insensitive to the variability of the donor job size distribution. Even when the variability of the donor job sizes is very high, and donor help thus is very bursty, the beneficiaries still enjoy significant benefits.

Our analysis also allows us to investigate characteristics of the beneficiary and donor side thresholds, $N_{\text{B}}^{\text{th}}$ and $N_{\text{D}}^{\text{th}}$, both with respect to their impact on stability and their impact on mean response time. With respect to beneficiary stability, we find that $N_{\text{B}}^{\text{th}}$ has no effect, while increasing $N_{\text{D}}^{\text{th}}$ increases the stability region. Donor stability is not affected by either threshold. With respect to overall mean response time, we find that mean response time is far more sensitive to changes in $N_{\text{D}}^{\text{th}}$ than to changes in $N_{\text{B}}^{\text{th}}$. This extends the results of [12], where we study only the effect that $N_{\text{B}}^{\text{th}}$ has on mean response time. We find the optimal value of $N_{\text{B}}^{\text{th}}$ tends to be well above 1. The reason is that increasing $N_{\text{B}}^{\text{th}}$ does not appreciably diminish beneficiary gain, but it does alleviate donor pain. We find that the optimal setting of $N_{\text{B}}^{\text{th}}$ is an increasing function of ${\rho }_{\text{B}}$, ${\rho }_{\text{D}}$, and switching times. By contrast, we find that the optimal value of $N_{\text{D}}^{\text{th}}$ is often close to 1, provided ${\rho }_{\text{B}}<1$. Increasing $N_{\text{D}}^{\text{th}}$ significantly hurts the donor, although it may provide significant help to the beneficiary if ${\rho }_{\text{B}}$ is high. We find that the optimal $N_{\text{D}}^{\text{th}}$ is not a monotonic function of ${\rho }_{\text{D}}$, but is an increasing function of ${\rho }_{\text{B}}$ and switching times.

Our model deals with switching times in a general way, making the results both applicable to a shared-memory multiprocessor architecture and to a network of workstations (NOW). Our switching times, $K_{\text{sw}}$ and $K_{\text{ba}}$, can be viewed as the time for switching between job types in a shared-memory multiprocessor architecture. In a NOW, there is an additional overhead incurred from migrating jobs from one server to another, where jobs which have not started running require negligible overhead, whereas migrating a “running” job (in progress) requires high overhead, since all of its state must be migrated with it. Our model captures this notion in that the switching back time, $K_{\text{ba}}$, can be viewed as the time incurred for preempting an already running job and returning it to the beneficiary.