TLA: Temporal look-ahead processor allocation method for heterogeneous multi-cluster systems

Highlights

• The allocation simulation process used in TLA is novel and effective.

• TLA directly utilizes the performance metric to make allocation decisions.

• Extensive simulation has been carrying out to evaluate the performance of TLA.

• With precise runtime, TLA has up to an 87% performance improvement.

Abstract

In a heterogeneous multi-cluster (HMC) system, processor allocation is responsible for choosing available processors among clusters for job execution. Traditionally, processor allocation in HMC considers only resource fragmentation or processor heterogeneity, which leads to heuristics such as Best-Fit (BF) and Fastest-First (FF). However, those heuristics only favor certain types of workloads and cannot be changed adaptively. In this paper, a temporal look-ahead (TLA) method is proposed, which uses an allocation simulation process to guide the decision of processor allocation. Thus, the allocation decision is made dynamically according to the current workload and system configurations. We evaluate the performance of TLA by simulations, with different workloads and system configurations, in terms of average turnaround time. Simulation results indicate that, with precise runtime information, TLA outperforms traditional processor allocation methods and has up to an 87% performance improvement.

Introduction

This paper focuses on the issue of processor allocation for parallel jobs in heterogeneous multi-cluster (HMC) systems. An HMC system consists of multiple clusters, whose computational power, memory size, and communication capability can be varied for different clusters. Such systems are becoming more and more popular in grid computing and cloud computing  [20], [24], [23].

In an HMC system, a central job scheduler (also known as a meta-scheduler) is commonly used to dispatch all submitted jobs  [20], [23]. A central queue, called the waiting queue, is used to accommodate those submitted jobs. The central job scheduler usually involves two operations: job scheduling and processor allocation. Job scheduling decides the execution order of the jobs, while processor allocation decides which cluster(s) to allocate the job to  [20], [23].

The issues of processor allocation in a HMC system are more complicated than those in other parallel systems. In a homogeneous cluster system or a supercomputer, all processors have equal computation capability. Therefore, it makes no significant difference to allocate a job on different processors. For a homogeneous multi-cluster system, processor allocation methods can be divided into two categories, single-site allocation and multi-site co-allocation, depending on whether the system supports a job to be executed across different clusters  [12]. The single-site allocation algorithms need to take care of the resource fragmentation problem, which means the entire system has a sufficient amount of available processors for a job but no single cluster alone has enough free processors to accommodate it. For an HMC system, the heterogeneity of resources adds another dimension of complexity to the allocation problem  [23].

In this paper, we focus on the case of single-site allocation on HMC, since multi-site co-allocation is rarely seen in production systems  [21]. In addition, we assume the heterogeneity is only for computing speed (it is called speed heterogeneity hereafter in this paper). Because of speed heterogeneity, allocating a job onto different clusters could lead to a different job execution time. Moreover, different allocation decisions for a job may also affect the waiting time of the jobs behind it in the waiting queue because of the resource fragmentation. Therefore, processor allocation becomes a crucial issue in a HMC system because it affects both the waiting time and execution time of the job.

Traditional processor allocation heuristics on a HMC system aim to resolve the resource fragmentation problem or to leverage the speed heterogeneity property to improve system performance, which leads to heuristics such as Best-Fit (BF)  [10] and Fastest-First (FF)  [12]. As reported in  [23], their performance largely depends on the input workload and system configurations since both methods consider only a single performance factor.

This paper proposes a novel single-site allocation method called temporal look-ahead (TLA), which uses an allocation simulation process to guide the decision of processor allocation. Instead of focusing on resource fragmentation and speed heterogeneity directly, TLA tries to optimize the performance of entire system based on the specified performance metric. The idea behind TLA is that by considering the performance metric directly it will naturally take into account all relevant performance factors to that metric. This design gives TLA potential to optimize other performance metric, which may have completely different performance factors.

TLA works as follows. For each job $j$ to be allocated, TLA evaluates all possible allocations and picks the one that could result in best system performance. Each possible allocation, say allocate to cluster $c$, is evaluated through a simulation, which simulates the job scheduling, allocation, and execution of all subsequent jobs in the waiting queue, under the assumption that job $j$ is allocated to cluster $c$, and evaluates the consequent system performance of such an allocation.

The realization of TLA needs to know which job scheduling algorithm is used since it decides the execution order of jobs in the waiting queue. TLA is a general processor allocation algorithm which can work with different job scheduling algorithms. To simplify the presentation, this paper only demonstrate the capability of TLA with the well-known First-Come–First-Served (FCFS) job scheduling algorithm  [20], [23], [15].

To show the effectiveness of TLA, we compared TLA with existing processor allocation heuristics with the metric of average job turnaround time. Simulation experiments were conducted for various input workloads and system configurations. Simulation results show that the peak performance improvements made by TLA can be up to 87%, when the runtime estimation is accurate.

The rest of this paper is organized as follows. Section  2 presents the system model and reviews the related work. Section  3 illustrates the idea of TLA. Section  4 presents the results of experiments which are based on precise job runtime estimation. Section  5 presents and discusses the performance issue of TLA. Conclusions and future work are given in Section  6.