Flow Network Models for Online Scheduling Real-time Tasks on Multiprocessors

We consider the flow network model to solve the multiprocessor real-time task scheduling problems. Using the flow network model or its generic form, linear programming (LP) formulation, for the problems is not new. However, the previous works have limitations, for example, that they are classified as offline scheduling techniques since they establish a flow network model or an LP problem considering a very long time interval. In this study, we propose how to construct the flow network model for online scheduling periodic real-time tasks on multiprocessors. Our key idea is to construct the flow network only for the active instances of tasks at the current scheduling time, while guaranteeing the existence of an optimal schedule for the future instances of the tasks. The optimal scheduling is here defined to ensure that all real-time tasks meet their deadlines when the total utilization demand of the given tasks does not exceed the total processing capacity. We then propose the flow network model-based polynomial-time scheduling algorithms. Advantageously, the flow network model allows the task workload to be collected unfairly within a certain time interval without losing the optimality. It thus leads us to designing three unfair-but-optimal scheduling algorithms on both continuous and discrete-time models. Especially, our unfair-but-optimal scheduling algorithm on a discrete-time model is, to the best of our knowledge, the first in the problem domain. We experimentally demonstrate that it significantly alleviates the scheduling overheads, i.e., the reduced number of preemptions with the comparable number of task migrations across processors.


I. INTRODUCTION
Multicore or multiprocessor platforms are becoming prevalent in numerous digital devices and this advance has been accelerated by the increasing computational demands of various emerging high-quality services. Alongside this trend, there has been a vast amount of research into multiprocessor real-time scheduling theories [4], [9]. Real-time systems are computing systems where their correct behaviors depend not only on the value of the computation but also when the results are produced [6]. Most research problems related to multiprocessors involve far more than a simple theoretical extension from uniprocessors to multiprocessors and thus, The associate editor coordinating the review of this manuscript and approving it for publication was Laxmisha Rai . the real-time scheduling problems on multiprocessors are challenging.
Liu stated [20]: ''Few of the results obtained for a single processor generalize directly to the multiple processor case; bringing in additional processors adds a new dimension to the scheduling problem. The simple fact that a task can use only one processor even when several processors are free at the same time adds a surprising amount of difficulty to the scheduling of multiple processors''. This statement can be interpreted as saying that in uniprocessors, the constraint that each task is forbidden from executing simultaneously on more than one processor is implicit, because a single processor is the only processing capacity present in the system. By contrast, in multiprocessors, the constraint of no intra-task parallelism becomes not only explicit but also interrelated with the other constraints, which significantly increases the problem's complexity.

A. MOTIVATIONAL EXAMPLES
We provide several examples in this section to specifically illustrate our research motivation. First, we introduce some of the basic notions and terminology used throughout this study. We assume that each task τ i is characterized by (C i , D i , P i ) where C i , D i and P i are its worst-case execution time, deadline, and period, respectively. When D i is identical to P i , it is called the implicit deadline and τ i is then characterized by (C i , P i ). The hyper-period H of all tasks is defined as the least-common-multiple of all P i . The jth instance (or job) of the periodic task τ i is denoted by τ i,j and the arrival time of τ i,j is denoted by a i,j . The running rate is the ratio of the execution time of a job (or a part of the job) to the time interval for the execution. For example, when τ i,j has its execution time C i,j within its deadline D i , its the running rate r i,j is C i,j /D i .
A set B containing all a i,j of the given jobs is called a set of boundaries. For convenience, each element (or boundary) of B is denoted by b k where an earlier b k has a lower k. The individual utilization u i of τ i is defined as C i /P i and the total utilization U of the given task set is the sum of all u i . We assume that N tasks run on M processors.
Definition 1 (RT-Optimality): An optimal real-time schedule meets all the task deadlines when the total utilization demand U of a given task set does not exceed the total processing capacity M , which we call RT-optimal in this study.
Several classes of RT-optimal task scheduling algorithms on multiprocessors have been developed for periodic implicit-deadline tasks. One of the well-known classes is fluid schedule-based algorithms, where each task execution attempts to track the fluid schedule that is known to be RT-optimal [3], [32]. Here, the fluid schedule is defined as follows.
Definition 2 (Fluid Schedule): A schedule is said to be fluid if and only if at any time t ≥ 0, every instance τ i,j of task τ i that arrives at time a i,j has been executed for exactly (t − a i,j ) × C i /P i time units [26].
Thus, to achieve RT-optimality, the fluid schedule assigns each task τ i with a uniform running rate r i for all times, where r i is set to C i /P i (or u i ). The fluid schedule is known to be RT-optimal but unrealistic, since it allocates a fraction of the computational resource to each task during each time unit, e.g., 1/3 of the processing capacity is allocated to a task per unit time.
The first example shown in Table 1 includes five tasks running on two processors. Its fluid schedule is illustrated in Figure 1. Their total utilization demand is U = 2, which is equal to the total processing capacity 2 and thus, RT-optimality of this fluid schedule holds.
One advantage of this fluid schedule is that it satisfies all of the interrelated constraints of the multiprocessor real-time scheduling problem, including the constraint of no intra-task   Table 1.  Table 1. parallelism, by using a single parameter r i per task. Provided that each r i is not greater than one and the sum of all r i is not greater than the total processing capacity, all tasks satisfy the deadlines without violating their constraints of no intra-task parallelism within the permitted total processing capacity.
Several inheritors of the fluid schedule, which we refer to as the fluid schedule-based algorithms, attempt to track the fluid schedule in order to obtain RT-optimality. To track the fluid schedule, they restrict the difference between the actual computational resource allocation and the fluid-schedulebased resource allocation for each task. The first algorithm of this type, proportionate-fair(Pfair), strictly maintains the restriction at every time quantum [3], [5]; its several descendants relax the restriction to being maintained at every boundary [7], [10], [11], [34], [35]. Pfair and some descendants rely on a discrete-time model, since they allocate the integral units of the computational-resource to each task, e.g., integer multiples of the system time unit for execution. In addition, they are known to support fairness in the sense that the computationalresource allocated to each task is always proportionate to its u i during the time interval from zero to any time quantum for Pfair or to any boundary for its descendants. Figure 2 shows the task set in Table 1 scheduled by one of Pfair's descendants, boundary-fair scheduling (BF). It is observed that each task has the same amount of the computational resource within every time interval between two adjacent boundaries, i.e., the fairness is supported. For example, τ 1 executes for 2 time units within every time interval Recently, it was observed that the scheduling overheads, including the number of preemptions and task migrations across processors, decrease as the fairness is  Table 2. relaxed [24], [25]. Based on this observation, an unfair-butoptimal U-EDF was proposed. The unfairness of U-EDF implies that some of the task workload is allowed to be advanced or delayed across boundaries beyond fairness. It implies that unlike the fluid schedule-based approaches, all tasks do not even need to run for every time interval between two adjacent boundaries and thus, the number of preemptions can be reduced. U-EDF relies on a continuous-time model, since it may allocate a fractional computational-resource to a task, e.g., 1/3 processing capacity allocated to a task for a certain time interval.
In addition to the reduced scheduling overheads, we believe that the unfairness has several other significant advantages. For example, unfairness allows the total workload to be collected within certain time intervals and processors within the other idle intervals can turn into a different state to minimize their energy consumption. Figure 3 shows an unfair schedule for five tasks in Table 2. In this schedule, the workloads are aggregated around b 3 and thus, two processors have the chance to become slow or idle in both

B. CONTRIBUTION
In order to support the unfair-but-optimal scheduling, we build a framework that allows us to manipulate the task workload efficiently across boundaries while holding RT-optimality, which is the primary focus of this study. Our contributions include the following: (1) We formulate the problem for online-scheduling the periodic implicit-deadline tasks on multiprocessors by specifying its constraints and we propose a flow network model to solve the formulated problem. Again, using the flow network model or the LP formulation for multiprocessor real-time task scheduling is not new. However, the previous works have limitations, such as that some are used as offline scheduling techniques since they require a single flow network model or a single LP problem to be constructed considering a very long time interval from 0 to H of the given tasks [17], [18], [23] or that others are applicable only to the aperiodic tasks [16]. To overcome the limitations, we propose how to construct the flow network model only for the active instances of tasks at the current scheduling time while guaranteeing the existence of an RT-optimal schedule for the future instances of the tasks.
(2) Based on the framework, we introduce an unfair-butoptimal multiprocessor scheduling algorithm, called flow network-based Earliest-Deadline-First (fn-EDF), for periodic tasks on both continuous and discrete-time models. In particular, to the best of our knowledge, fn-EDF on a discrete-time model is the first online unfair-but-optimal scheduling algorithm in the given problem domain. We experimentally show that fn-EDF on the discrete-time model significantly reduces the number of preemptions with the comparable number of migrations against an existing BF algorithm. Table 3 compares fn-EDF with existing algorithms in terms of their problem domains. BF is fair-and-optimal on a discrete-time model and both U-EDF and RUN [30] are unfair-but-optimal on a continuous-time model. The original LP-based scheduling focused on constraineddeadline tasks, which have their deadlines less than their periods [17], [23]. In Table 3, it is assumed that LP-based scheduling can easily solve the implicit-deadline task scheduling problem by setting D i = P i . As discussed, the traditional LP-based scheduling is classified as an offline approach.
The remainder of this paper is organized as follows. Section II presents the problem formulation and the corresponding flow network model for scheduling implicitdeadline periodic tasks; the unfair-but-optimal scheduling algorithms on both continuous and discrete-time models are also discussed. Section III experimentally evaluate the performance of the proposed algorithm compared with an existing method. In Section IV, we discuss two issues about the proposed approach, complexity and extensibility. Section V summarizes the related work. In Section VI we give our conclusion.

II. SCHEDULING ALGORITHMS
In this section, we first define our system model. Based on the system model, we rigorously formulate the given problem as a linear optimization problem that eventually turns into a flow network problem. Under the formulation, we then propose our scheduling algorithms on both continuous and discrete time models.

A. SYSTEM MODEL
We consider the implicit-deadline task τ i characterized by (C i , P i ). We assume that the task parameters, C i and P i , are multiples of the system time unit. The active job τ i (t) of τ i at time t has its arrival time a i (t) subject to t ∈ [a i (t), a i (t)+P i ). At time t, τ i (t) has its remaining execution time c i (t), where 0 ≤ c i (t) ≤ C i . The active job set is denoted by τ (t). Both a i,j +P i and a i (t)+P i correspond to d i,j and d i (t), respectively and called absolute deadlines of the job. b k is the kth boundary of B. The earlier b k is assumed to have the lower index k. When we consider the active jobs only, B = {d i (t)|1 ≤ i ≤ N } ∪ {t} contains K absolute deadlines (boundaries) of the active jobs, where K is less than or equal to N . The window W k is the time interval ranging [b k ,b k+1 ). The length l k of W k is b k+1 − b k and the permitted processing capacity Cap(W k ) of W k is upperbounded by M ×l k . The allocated execution time for τ i within W k is denoted by X i,k .

B. PROBLEM FORMULATION WITH AN EXAMPLE
Assume that 5 tasks in Table 1 are running on 2 processors. To consider active jobs at the current time t, we focus on the time interval from the current time t to the latest deadline of all active jobs d max (t) = max ∀i {d i (t)}. In [t, d max (t)], the boundary set is B = {d i (t)|1 ≤ i ≤ N } ∪ {t}. For the given example, all active jobs at current time 0 are illustrated in white in Figure 4 and d max (0) is 9. For the active jobs, X i,k are assigned as shown in Figure 4.
Next, we formulate a linear programming problem for the active jobs at time 0. Equation 1 defines the constraint that each active job τ i (t) completes its execution within the permitted time interval, i.e., [a i (t), d i (t)). We call this the job completion constraint (JCC).
Inequality 2 is the constraint that the sum of the allocated execution times of the active jobs within a given window does not exceed the permitted processing capacity of the window. We call this the processing capacity constraint (PCC).
Inequality 3 is the constraint that each active job within a given window does not simultaneously occupy more than one processor. We call this the constraint of no intra-task parallelism (NIP).
To facilitate the discussion, the white region ranging [t, d max (t)) in Figure 4 is called the active job area at time t and it is denoted by AJA(t). In other words, AJA(t) is a collection of the maximum capacity per window that can be utilized to execute the active jobs at t. For each AJA(t), three types of constraints, JCC, PCC and NIP, are defined. No objective function is required and a feasible solution that satisfies all the constraints is sufficient for our purpose.
In the constraints above, all right-hand side values except Cap(W k ) can be determined easily when B for the active jobs is fixed. The rationale behind the determination of Cap(W k ) is explained using the following example. Figure 4 indicates that some capacity of W 2 is used for the active jobs, i.e., {τ 2,1 , τ 3,1 , τ 4,1 , τ 5,1 }, and the remainder is reserved for a future job τ 1,2 . Thus, each Cap(W k ) should be determined in order to support the schedulabilities of the current active jobs and future jobs as well. Therefore, we determine Cap(W k ) using the fluid schedule. For example, Cap(W 3 ) is calculated by reserving the execution time of {τ 1,3 , τ 2,2 , τ 3,2 } within W 3 from the permitted processing capacity of W 3 , e.g., 3 . When a schedulable implicit-deadline task set is given, since we reserve a suitable amount of processing capacity for future jobs based on the fluid schedule, the schedulability of future jobs remains valid regardless of our scheduling decision within the current AJA.
To schedule real-time tasks up to their hyper-period, we assume that AJA is constructed repeatedly at each boundary (scheduling event). Whenever AJA is constructed at time t, the right-hand sides of JCC equations are filled with the remaining execution time c i (t) of each τ i (t). PCCs and NIPs are formed in the same manner as shown for time 0. Since a new AJA is constructed at each boundary, some X i,k in the previous AJA are recalculated. In this example, at time 0, only X i,1 in W 1 are used for scheduling. The other allocated execution times X i,k are recalculated in the next AJA. Figure 5 shows both AJA(3) and AJA (6). We assume that the indices of b k and W k are updated for each AJA(t). Once all X i,k in W k are obtained, the actual schedule within W k can VOLUME 8, 2020 easily be determined, e.g., using McNaughton's wrap around algorithm [22].
We emphasize that a solution satisfying three types of constraints may yield the different running rates of a job between two adjacent windows, e.g., r i,k = r i,k+1 , which implies that this approach can generate an unfair schedule.

C. PROBLEM FORMULATION WITH THE GENERAL TASK MODEL
In this subsection, we generalize the problem formulation with the general task model. Figure 6 shows a set of general tasks with implicit deadlines. At the current time t 1 , AJA(t 1 ) is constructed with three types of constraints. We set the current time t 1 to the boundary b 0 . The first window W 1 then ranges from b 0 to b 1 (=t 2 ). The active job τ i (t 1 ) is assumed to have its remaining execution time c i (t 1 ) at t 1 . Within the time interval [t 1 , d max (t 1 )), a series of windows, {W 1 , . . . , W K }, is built, based on B. The number of windows K for AJA(t 1 ) is less than or equal to N .
For convenience, we define two sets for the current AJA as follows: K(t s , t e ) contains all indices of the window W k which are placed in the time interval [t s ,t e ]. J(k, t) contains all indices i of the active jobs at time t which are still active in W k . Using these two sets, the constraints for AJA(t 1 ) are defined as follows: For PCC, Cap(W k ) is set as follows: At a scheduling event (or boundary), the corresponding AJA is established and three types of constraints are defined. After a feasible solution is found for AJA, the part of the feasible solution for W 1 is used to allocate the computational resource to each task for their execution during W 1 . At the next scheduling event, the next AJA is established and a similar procedure follows. This iteration continues until H , which provides the schedulability for the given tasks. Note that the LP formulation includes no objective function and a feasible solution that satisfies all constraints is sufficient for our purpose.
RT-optimality of the proposed approach can be proved by showing that as long as the fluid schedule can be defined for a given task set, the proposed approach finds at least a feasible schedule.
Lemma 1: If a fluid schedule exists for the given periodic implicit-deadline task set, the fluid schedule satisfies the three types of constraints in the first AJA(0).
Proof: The fluid schedule guarantees schedulability by providing each task with the processing capacity based on its running rate r i that is equivalent to its individual utilization u i = C i /P i ≤ 1. For AJA(0), the fluid schedule ensures that every X i,k is r i × l k = u i × l k . First, JCCs for AJA(0) are satisfied as follows.
Second, NIPs for AJA(0) are satisfied since X i,k = r i × l k = u i × l k ≤ l k . Third, PCCs for AJA(0) are satisfied as follows. Since the fluid schedule is assumed, the following equation By the assumption of the fluid schedule, From Equation 9, ∀i∈J(k,0) X i,k ≤ Cap(W k ). Theorem 1: If a fluid schedule exists for the given periodic implicit-deadline task set, the proposed approach provides a feasible solution satisfying all constraints established by the task set.
Proof: The proof is obtained by induction on the increasing boundary. We assume that AJA(0) has a feasible solution that satisfies all constraints. Then, we try to show that if AJA(t 1 ) has a feasible solution, AJA(t 2 ) has at least one feasible solution, where t 2 is the next boundary to t 1 as shown in Figure 6. Note that the index k is not updated at the next scheduling event point t 2 simply for convenience in the proof.
Basis: From Lemma 1, AJA(t 1 ) in Figure 6 has at least one feasible solution, when t 1 = 0. Here, assume that the active jobs at t 1 are sorted in the increasing deadline order. Induction Step: Assume that X * is a feasible solution for AJA(t 1 ). After is still a feasible solution for the remaining area of AJA(t 1 ). Therefore, X * −1 satisfies the following equations.
At t 2 when a new job τ 1 (t 2 ) arrives, AJA(t 2 ) is established using the fluid schedule. For AJA(t 2 ) that covers the array of windows {W 2 ,W 3 ,. . . W K +1 }, we define three types of constraints as follows.
Since Equation 19 becomes the following.
The candidate solution satisfies the first case of Equation 20 since Equation 13 holds. It also satisfies the second case of Equation 20 as follows.
For PCCs, the left-hand side of Inequality 17 is modified as follows.
If the variables above are substituted with the candidate solution, it becomes the following due to Equation 14.
Therefore, the candidate solution satisfies PCCs. In addition, since X * −1 satisfies NIPs and X 1,k = u 1 l k ≤ l k , 2 ≤ ∀k ≤ K + 1, the candidate solution satisfies NIPs. VOLUME 8, 2020 In summary, the candidate solution satisfies all constraints of AJA(t 2 ). It implies that if a feasible solution exists for AJA(t 1 ), then at least one feasible solution for AJA(t 2 ) can be found. The feasible solution for every AJA from time 0 to H is ensured using the induction steps.
Corollary 1: The proposed approach is an RT-optimal real-time scheduling algorithm.
Proof: According to Theorem 1, if the fluid schedule exists for a given task set, our technique provides the feasible solutions for the repeated AJAs. It is also known that if U of the task set is less than or equal to M , RT-optimal fluid schedule exists. Therefore, if U of a task set is less than or equal to M , our technique provides the feasible solutions for the repeated AJAs, which implies its RT-optimality.

D. FLOW NETWORK MODEL
The objective of the maximum flow problem is to find the maximal flow from a single source to a single sink in the given flow network. To efficiently solve the LP problem for AJA, we transform it into the maximum flow problem by slightly changing JCCs as follows.
The maximum flow problem for AJA(t 1 ) consists of the constraints, including Inequalities 25, 7 and 8, and the objective function as follows.
Definition 3 (Maximum Flow Problem for AJA(t 1 )): s.t. (27) Note that when the maximum flow occurs in the given problem, the left-hand sides of all JCCs, i.e., Inequality 25, should be equal to c i (t 1 ), which satisfies the original JCC equations. We call it the complete maximum flow.
Definition 3 turns our problem into a maximum flow problem. Since all constraints become the upper bounded inequalities, each upper bound acts as the capacity of each edge in the flow network. Using these edges, the flow network is built as follows.
To construct the flow network, we add two additional nodes, the source n s and the sink n e . Between n s and n e in the network, two intermediate layers of nodes are placed, i.e., the first layer contains the nodes of all active jobs and the second layer contains the nodes of all windows. We name each node based on its corresponding job and window. Each edge is denoted by e(n 1 ,n 2 ) where n 1 and n 2 are the source and destination nodes, respectively. e(n 1 , n 2 ) ← {σ (n 1 , n 2 )} denotes that the edge e(n 1 , n 2 ) has its flow capacity σ (n 1 , n 2 ). From n s to node τ i , a directional edge is inserted and the flow capacity is determined as c i (t 1 ). From node W k to n e , a directional edge is inserted with the capacity Cap(W k ). Between node τ i to node W k , an edge whose flow is constrained by l k is inserted.
Formally, when AJA(·) is given, it is transformed into a capacitated network F = (V , E), where a network F contains a set of nodes V and a set of edges E. We call F the flow network for real-time scheduling (FNRT).
The actual flow on an edge (n 1 , n 2 ) from node n 1 to node n 2 is denoted by f (n 1 , n 2 ). In FNRT, f (τ i , W k ) corresponds to X i,k for the given AJA. In addition, the set of capacitated edges {e(n s , τ i )|∀i} represents JCCs. The sets of capacitated edges {e(τ i , W k )|∀i, k} and {e(W k , n e )|∀k} represent NIPs and PCCs, respectively. The complete maximum flow for FNRT satisfies all constraints even including the original JCCs in the linear programming problem. A flow network example for Figure 4 is shown in Figure 7.
A high-level description of the algorithm that is invoked at each scheduling event (boundary) is shown in Algorithm 1. Lines 1-12 shows how to construct FNRT. The maximum flow problem is solved by line 13. The computational complexity of constructing FNRT is proportional to the nested loops from lines 5-12 and though, the complexity of solving the maximum flow problem at line 13 is dominant.
Maximum flow problem has been intensively studied in the graph theory research community and several polynomial Algorithm 1 Schedule(t, τ ) Based on the Flow Network input : the current time t and the task set τ Data: F is the flow network to be constructed Data: C = {Cap(W k )| k is for all W k in AJA } Data: e(n 1 , n 2 ) is an edge from node n 1 to n 2 algorithms have been found [14]. In terms of complexity, the algorithms are categorized into two groups, i.e., weakly polynomial and strongly polynomial algorithms [14]. The complexity of weakly polynomial algorithms is upperbounded by a combination of |V |, |E| and the largest capacity among all edges. On the other hand, the complexity of the strongly polynomial algorithms is upper-bounded by a combination of |V | and |E| alone. A strongly polynomial algorithm with O(|V ||E|) complexity [27] was introduced recently and further improvement continues. We use the O(|V ||E|) complexity algorithm for line 13 in Algorithm 1. Since |V | and |E| are proportional to N and N 2 in the worst case respectively, the complexity of Algorithm 1 is O(N 3 ). Several other maximum flow algorithms with O(|V | 3 ) complexity are possible alternatives [12].

E. FN-EDF ON THE CONTINUOUS-TIME MODEL
The objective of maximum flow algorithms is to send the maximal flow from the source to the sink. In general, multiple sets of flows that achieve the goal could exist, which implies that the formulated problem could have multiple solutions. Since maximum flow algorithms simply find one of the solutions depending on the preference of the selected algorithm, the amount of flow on each individual edge is not controlled, but only upper-bounded. In a real-time scheduling context, this implies that a task schedule determined by a maximum flow algorithm may be, for example, non-workconserving or work-conserving from time to time. To a more carefully control the flow over FNRT, additional features of the flow network are required to use.
To control the flow over the network, we consider Minimum cost flow problem (MCFP). Each edge e(n 1 , n 2 ) on the flow network for MCFP contains one more parameter, called cost, w(n 1 , n 2 ). When the actual flow f is sent on an edge, the cost of the flow on the edge is calculated by w × f . The objective of MCFP is to find the set of the actual flow f on all edges that minimizes the total cost of the flow, when a total flow value is given. Since the total flow value is known to be the sum of the right-hand sides of JCCs for the given AJA, MCFP is easily applicable. The cost w to each edge allows us to control the actual flows in the network.
For example, assume that we try to generate an EDF-like schedule for the given AJA(t). Among all edges e(τ i , W 1 ) directed to node W 1 , to prioritize the task execution in EDF order, we assign the lower cost to the edges e(τ i , W 1 ) where τ i has the earlier deadline. For example, the cost {1, . . . , N } are assigned to all e(τ i , W 1 ) in EDF order.
In the following windows, to make the schedule workconserving, the costs are increasingly assigned to edges, i.e., the cost N + k − 1 is assigned to e(τ i , To implement this scheduling algorithm, line 8 in Algorithm 1 should be updated with the previous two Equations, 29 and 30, which yields Algorithm 2. We call this flow network-based EDF (fn-EDF). Figure 8 shows the flow network for fn-EDF that is extended from Figure 7.
To analytically describe the cost assignment, we introduce a notion, cost-slope, s i , at W k which is defined to be w(τ i , W k+1 ) − w(τ i , W k ). Assume that the costs {1, . . . , N } are assigned to all e(τ i , W 1 ) in EDF order and that the cost N + 1 is assigned to all e(τ i , W 2 ). Then, the earliest deadline task node has the highest cost-slope at W 1 . We prove that the steeper cost-slope imposes the higher priority.
Theorem 2: When two job nodes τ i and τ j are connected to W k and W k+1 with edges { (τ i , W k ), (τ i , W k+1 ), (τ j , W k ), VOLUME 8, 2020 Algorithm 2 Schedule(t,τ ) for Flow Control input : the current time t and the task set τ Data: τ s (t) is the active job set sorted by EDF Data: (τ j , W k+1 )} in the flow network, if s i > s j at W k , then the flow network gives higher priority to the flow from τ i to be sent to W k than that from τ j .
Proof: Assume that one time unit (or flow) is available within W k and that τ i and τ j compete for it. In the case where τ i occupies one time unit in W k and τ j is delayed to take one time unit in W k+1 , we assume that the total flow cost is u(F), where F is the set of actual flows on all edges. If the units of τ i and τ j are swapped between W k and W k+1 , the new cost u(F ) becomes u(F) + s i − s j . Since s i > s j , u(F ) > u(F). Therefore, MCFP algorithms prefer to assign the available single time unit in W k to τ i rather than τ j in order to minimize the total cost. All available time units are allocated in the same manner and thus, the flow from τ i achieves a higher priority of being sent to W k than that from τ j .
Note that the prioritization based on the cost slopes differs from the traditional prioritization in the real-time scheduling context. The assignment of traditional priorities to tasks may result in deadline misses, whereas prioritization using the cost slopes never attenuates the schedulability ensured by the flow network model. Thus, we can call this a weak priority if distinction is needed.
We also need to consider the maximum magnitude of costs, because the computational complexity of a certain class of MCFP algorithms depends on the cost. To generate the EDF-like schedule, the costs {1, . . . , N + K } are used. Since K is proportional to N , the maximum magnitude of costs is proportional to N . We emphasize that different cost assignments are also possible for various scheduling purposes.
For the line 17 in Algorithm 2, any minimum cost flow algorithm can be used. In terms of complexity, the algorithms are also categorized into two groups, i.e., weakly polynomial and strongly polynomial algorithms [15]. The complexity of the weakly polynomial algorithms is upperbounded by a combination of |V |, |E|, the largest cost w max and (or) the largest capacity of all edges. We restrict w max for our scheduling purposes, e.g., w max ∝ N for prioritizing task execution. Therefore, not only the strongly polynomial algorithms but also the polynomial algorithms bounded by a combination of |V |, |E| and w max are of our interest. Goldberg et. al introduced a cost-scaling algorithm with dynamic trees having O(|V ||E|log|E|log(|E|w max )) complexity [13]. Orlin introduced an enhanced capacity-scaling algorithm having O(|V |log|E|SP + (|V |, |E|)) complexity where SP + (|V |, |E|) denotes the time complexity of solving the single-source shortest path problem [28]. Dijkstra's algorithm with Fibonacci heaps is known to provide an O(|E| + |V |log|V |) bound for SP + (|V |, |E|) [21]. In our context, the complexity of the former algorithm is O(N 3 (logN ) 2 ) and that of the latter is O (N 3 logN ).

F. FN-EDF ON THE DISCRETE-TIME MODEL
One of our assumption in this study is that the task parameters, {C i , P i }, are multiples of the system time unit, i.e., integers. Despite the assumption, manipulating tasks for scheduling can generate non-integral numbers, which leads to two issues that we must consider.
First, several maximum flow and minimum cost flow algorithms assume that the parameters of the flow networks are integers. However, our flow network models permit noninteger network parameters. For example, in Algorithm 1, the edges e(W k , n e ) may have non-integral Cap(W k ), which is calculated using u i = C i /P i of several tasks. Nevertheless, since u i 's are rational numbers, this issue can be easily solved by linearly scaling all of the rational numbers in the flow network up to appropriate integers. It does not raise any concern about increasing complexity if a strongly polynomial algorithm is used. This simple technique is sufficient for scheduling tasks on the continuous-time model where the non-integral units of time are allowed to be allocated for task execution.
Second, on the discrete-time model, the allocation of the non-integral units of time for task execution is not allowed. To run tasks on the discrete-time model, all X i,k 's of the flow network solution should be integers.
RT-optimal scheduling on the discrete-time model has been studied by several researchers and BF algorithms were proposed recently [26]. To ensure RT-optimality, BF algorithms are designed to maintain their deviation from the fluid schedule to be less than one time unit at every boundary. Informally, when the fluid schedule yields a non-integral execution time c i between two adjacent boundaries for a task, the algorithm divides the non-integral execution time into mandatory and optional executions where the mandatory execution time is determined to be c i and the optional execution time is determined to be c i − c i . The optional execution times for all tasks are accumulated and additionally allocated to the tasks based on specific criteria.
In Algorithms 1 and 2, Cap(W k ) is calculated based on the fluid schedule of each task. For the discrete-time model, we instead use the BF schedule to determine Cap(W k ) to be integral. Specifically, in Equation 9, the subtrahend l k × ∀i / ∈J(k,t 1 ) C i /P i is replaced with a proper integer value produced by BF schedule.
Assume that BF invokes at the current time t. Then, BF returns a set S 1 that contains the (integral) execution time S i,1 of τ i for the time interval W 1 . The current time t is assumed to be b 0 without loss of consistency with our previous notations.
When AJA(t) is given, BF schedule within the time interval [t, d max ] is obtained using the following equation.
The fluid schedule on the continuous time model ensures that every task has a running rate r i = C i /P i all the time. On the other hand, BF on the discrete-time model has an individual rate r i that changes slightly for each W k . It is because if r i × l k for W k is non-integral, it should be adjusted to be integral. Therefore, all boundaries including those made by future jobs within [t,d max (t)] should be considered to calculate the integral execution time of each active job on the discrete-time model. Previously, the boundaries are defined as {d i (t)|1 ≤ i ≤ N } for the continuous-time model. For the discrete-time model, the boundaries are determined as follows.
Based on these notions, the flow network-based scheduling algorithm for the discrete-time model runs as follows. When a scheduling event occurs at time t, the range of AJA(t) is determined using d max (t). Within [t, d max (t)], all boundaries are determined using Equation 33. Then, for every W k within [t, d max (t)], all S i,k are determined using Equation 32. Note that the determination of all S i,k requires consideration of the active jobs and the future jobs within [t, d max (t)] as well. These S i,k are used to calculate Cap(W k ). In other words, BF runs in parallel with the actual flow network-based scheduling algorithm in order to calculate Cap(W k ). It is implemented as the function ComputeWindowCapacities() as shown in Algorithm 3.
Especially, we call the combination of Algorithm 2 and 3 fn-EDF on the discrete-time model. To the best of our knowledge, fn-EDF is the first unfair-but-optimal scheduling algorithm for periodic implicit-deadline tasks on the discrete-time model. RT-optimality of fn-EDF is shown in Appendix.
For fn-EDF, the number of boundaries of a task τ i in the time interval [t, d max (t)], which additionally

Algorithm 3 ComputeWindowCapacities(t, B) for Discrete-Time Model
Data: t is the current time Data: B is the set of boundaries of all jobs in [t, d max ] = max ∀i {P i } and thus, the number of windows K is proportional to ∀i P max /P i , where ∀i P max /P i is denoted by N . Since the complexity of runBF() is known to be O(N · P max ), the complexity of The increasing number of windows K also affects the complexity of Algorithm 2. The complexity of the dominant line 17 of Algorithm 2 is related to the number of nodes and edges in the flow network. The number of job nodes is N and the number of window nodes is at most N . Thus, |V | is proportional to N and |E| is proportional to N ·N . Therefore, if the cost-scaling algorithm with dynamic trees is selected for MCFP, its complexity becomes O(N 2 N (logN ) 2 ) in our context.
Considering ComputeWindowCapacities(), the complexity of fn-EDF is O(max{NN P max , N 2 N (logN ) 2 }). If the enhanced capacity-scaling algorithm is selected, its complexity becomes O (N 2 N logN ). Then, the complexity of fn-EDF is O(max{NN P max , N 2 N logN }).
Considering that H is the least-common-multiple of all P i , P max is usually assumed to be much smaller than H , which we also assume in this study. However, in the worst case when P max = H , the complexity of the proposed algorithm becomes dependent on the number of all jobs in time 0 to H .

III. EXPERIMENTS
We experimentally evaluated the performance of fn-EDF compared with BF on the discrete-time model, as BF is the latest technique in the problem domain. SimSo was used as the simulation environment [8]. We performed four sets of experiments for 2, 4, 6 and 8 processors where the task sets were randomly generated. The experimental procedure was the following. • For a task set, P i 's were randomly generated using the uniform distribution within [5,20]. VOLUME 8, 2020 • If H was greater than its upper-bound, 600, 000, P i 's were discarded and this procedure restarted. The upperbound helped avoid long time experiments.
• The total utilization U for tasks was set to M .
• u i 's were randomly generated based on the fixed U using the algorithm in [33]. The algorithm generates random numbers within their predefined ranges when their fixed sum is given.
• Based on each P i and u i , C i was calculated by max{1.0, u i × P i }. Then, all C i were determined to be integral and the final U became slightly less than or equal to M .
• We ran both BF and fn-EDF for the task set.
• We repeated this procedure until more than 100 task sets were tested for each result. Figure 9-10 shows the average numbers of preemptions and migrations per job with BF and fn-EDF. The figures show that the number of preemptions with fn-EDF is always less than that with BF, while the number of migrations with fn-EDF is comparable to or slightly less than that with BF. Moreover, the preemption reduction ratio of fn-EDF over BF, i.e., the number of preemptions with fn-EDF divided by that with BF, becomes more significant, when more tasks run on the system. For example, it reaches around 0.5 when 4M tasks run on M processors. It is caused by the difference between BF and fn-EDF. BF generally forces every task to consume its execution time for each window, whereas fn-EDF allows a task to skip its execution in some windows, which significantly reduces the number of preemptions.
In the current implementation of fn-EDF, the lower indexed processor is simply allocated to the active job with the earlier deadline. In order to reduce the number of preemptions and migrations further for fn-EDF, the processor should be allocated to jobs more carefully by an improved heuristic, that is one of our future works.

IV. DISCUSSION
In Section II-E and II-F, we have discussed the computational complexities of both versions of fn-EDF. Although the proposed algorithms have the polynomial complexity, the computationally lighter algorithms should be pursued. We believe that further reduction of their computational complexity is promising for two reasons. First, the organization of the flow network models in this study is rather fixed, e.g., four layers of nodes including two intermediate layers of job and window nodes. Therefore, there might exist an improved algorithm specialized for our flow network models. For example, if the source and sink nodes are aligned with the window and the job nodes respectively while retaining their connections, the flow network becomes a bipartite network having a single source and a single sink node. A bipartite network is defined as a network where the nodes are partitioned into two subsets and each edge has one endpoint in the one subset and the other endpoint in the other subset [1]. Thus, the maximum flow algorithms specialized for the bipartite networks are applicable to our network models. Second, research effort in the graph theory community to achieve better maximum flow algorithms continues. For example, one of the conclusion given by [14] states that an O(|V ||E|/|V | ) strongly polynomial algorithm may exist. As we have established the flow network models for the online multiprocessor real-time scheduling problems, we have the opportunity to directly utilize the advance in the modern graph theory.
We also explore the possibility of extending the proposed approach for the real-time tasks having different arriving patterns, e.g., sporadic tasks. On the continuous-time model, fn-EDF can be easily extended for sporadic tasks by letting each task own its portion u i out of the system's total processing capacity M , even when the task is inactive as proposed in [19]. On the discrete-time model, extending fn-EDF for sporadic tasks is not so simple. It is because fn-EDF on the discrete-time model is based on BF that has each task's portion (or the rate r i ) out of the system's total processing capacity slightly changing over windows. Thus, it is difficult to determine how much capacity portion should be given to a task when it is inactive. Extending fn-EDF for sporadic tasks is one of our future works.

V. RELATED WORK
There have been several attempts to formulate the multiprocessor real-time scheduling problem as an LP problem. Since these attempts considered all jobs of each task from time 0 to the hyper-period for problem formulation, they have been used as the preliminary offline technique to compensate for the shortcomings of the subsequent online algorithms [18], [23]. Lawler et al. formulated the multiprocessor constraineddeadline periodic task scheduling as an LP problem for the first time [17]. Transforming this linear programming problem into a network flow problem was considered in [23].
In addition, several RT-optimal solutions have been proposed for the multiprocessor implicit-deadline periodic task scheduling problem. Most of them rely on the notion of fairness which was first proposed by Baruah et al. [3], [5]. The perfect fairness can be obtained by the RT-optimal but nonrealistic fluid schedule and the fairness-based RT-optimal scheduling algorithms were designed to follow the fluid schedule as much as possible. The first algorithm of this kind was PFfair scheduler that invokes every time quantum to restrict the deviation of resource allocation for each task from its fluid schedule within one time quantum. Several variants were proposed later such as pseudo-deadline (PD) [31], PD 2 [2], etc. However, Pfair and its variants are known to lead to frequent preemption and migration of tasks, which motivated several alternatives to arise.
Zhu et al. introduced Boundary fair (BF) algorithm that invokes every boundary to restrict the deviation of resource allocation for each task from its fluid schedule within one unit time [34]. Since the scheduler-invocation frequency of BF is lower than that of PFair, the scheduling overheads are reduced. Recently, BF 2 algorithm was proposed as an extension of BF to handle sporadic tasks [26]. While BF and BF 2 were based on the discrete-time model, there was another branch of algorithms which supported a similar boundary fairness based on the continuous-time model [7], [11]. Readers who are interested in the fairness-based RT-optimal scheduling algorithms can refer to [9], [26]. Although these boundary fairness-based scheduling algorithms often outperform the proportionate fairness-based scheduling in terms of reducing the number of preemptions and migrations, the need for further minimizing the scheduling overhead remains.
Nelissen et al. observed that respecting the fairness causes the high overheads [24]. Thus, they proposed U-EDF, an unfair but still RT-optimal scheduling algorithm, and they showed that U-EDF performs better than the existing fairness-based algorithms in reducing the scheduling overhead. U-EDF is known to be based on the continuous-time model.

VI. CONCLUSION
In this study, we formulated the multiprocessor real-time task scheduling problems by identifying three types of constraints and suggested the flow network models to solve the formulated problems efficiently. We discussed the potential use of this framework for addressing several interesting scheduling problems without losing the scheduling optimality. Based on the framework, the unfair-but-optimal scheduling algorithm, fn-EDF, was proposed for both continuous and discrete-time models. Our experiments showed that fn-EDF outperforms an existing BF algorithm in terms of the number of preemptions while maintaining a comparable number of task migrations.
Formulating the multiprocessor real-time task scheduling problem by identifying its constraints is beneficial for expanding its possible applications. Simply by manipulating the constraints, this formulation can be flexibly extended to solve other related problems. One of the potential applications include the example of managing the processor idle time, shown in Section I-A. In addition to this horizontal expansion of its application, several existing techniques for the flow networks can be adapted for specific real-time scheduling purposes, e.g., load balancing techniques that aim at balancing the flow on the selected edges of a given network [29]. Our future research will consider these issues.

APPENDIX A RT-OPTIMALITY OF FN-EDF ON THE DISCRETE-TIME MODEL
In order to prove RT-optimality of fn-EDF on the discretetime model, we need some terminology for this section. B is defined to contain a i,j of all jobs in the time interval [0, H ] and each element of B is denoted by b k where an earlier b k has a lower k. The allocated execution time of τ i,j in W k is denoted by X i,j,k . J(k, t) is extended as follows.
(38) In FNRT as shown in F, we assume that τ i,j are sorted in the increasing arrival time order. For convenience of discussion, the sorted τ i,j is labeled as τ p , 1 ≤ p ≤ P. In addition, we assume that W k are sorted in the increasing starting time order. A set of edge flows, f, is assumed to contain all edge flows. The amount of the maximum flow sent from n s to n e is denoted by |f|. The complete maximum flow for F results in a feasible schedule where the flow f (τ i,j , W k ) corresponds to X i,j,k .
In the following lemma, we will define three flow networks F, F + , and F − . The elements for each network is  distinguished by using the superscript + or −, e.g., f (·) denotes the flow for F, σ + (·) denotes an edge capacity for F + .
Lemma 2: (Network decomposition) Assume that FNRT F is constructed for a task set τ where the total utilization of τ is less than equal to M . F is assumed to contain two disjoint task node sets, τ and τ where τ = τ ∪τ , τ = {τ p |1 ≤ p ≤ p }, and τ = {τ p |p + 1 ≤ p ≤ P} as shown in Figure 11. Then, the maximum flow f for F consists of two flows, f sent from n s through τ to n e and f sent from n s through τ to n e .
When W k = Child(τ ) ∩ Child(τ ), suppose that F is decomposed into F + and F − where F + contains τ and their child nodes and F − contains τ and their child nodes as shown in Figure 12. If the edge capacities σ (W k , n e ) for F + and F − are determined as follows, then, the maximum flow for F + is |f | and the maximum flow for F − is |f |.

Proof:
In F, In F − after decomposition, since σ − (W k , n e ) is sufficient to send f (τ , W k ) along e − (W k , n e ) and all the other edge capacities are preserved, the maximum flow |f − | is not lower than |f |.
The maximum flow |f − | is not greater than |f |, neither. First, the flow through the node W k is upper-bounded by the constraint of the edge capacity σ − (W k , n e ). Second, if the other flows detouring the node W k increase in F − compared to those in F, it implies that the flow f in F is not the maximum flow, which contradicts the assumption that f is the maximum flow for F. Therefore, |f | is the maximum flow for F − .
In F + after decomposition, the following inequality holds for the edge capacity based on Equation 40.
Thus, σ + (W k , n e ) is sufficient to send the flow f (τ , W k ) along e + (W k , n e ) and all the other edge capacities are preserved, the maximum flow |f + | is not lower than |f |. The maximum flow |f + | is not greater than |f |, neither. If |f + | is greater than |f |, it implies that the flow f in F is not the maximum flow, which contradicts the assumption that f is the maximum flow for F. Therefore, |f | is the maximum flow for F + . Lemma 2 only considers the case that the set Child(τ ) ∩ Child(τ ) contains a single window node W k . If it contains more nodes, then Lemma 2 can be easily extended by applying Equation 39 to each W k .
Lemma 3: (Node removal) Assume that FNRT F is given with its maximum flow f as shown in Figure 13. If an earliest window node W 1 with its parent nodes Parent(W 1 ) is removed and the edge capacities of F are adjusted as follows, then, F turns into F with its maximum flow |f | = |f| − ∀τ p ∈Parent(W 1 ) |f (τ p , W 1 )|. Proof: If |f | is not the maximum flow for F , the increased flow can be sent to n e in F . It implies that the increased flow can be sent through the other nodes W k than W 1 in F, which contradicts our assumption that f is the maximum flow for F.

Lemma 4 (Network Composition): Assume that two
FNRTs, F + and F − , are given with their complete maximum flows f + and f − respectively. In addition, assume that both F + and F − have their own window node W k . If two flow networks are combined to be F by merging the node W k and adjusting the edge capacity as follows, then F has its complete maximum flow |f + | + |f − |. Proof: In F, all edge capacities are preserved excluding σ (W k , n e ) on e(W k , n e ). σ (W k , n e ) is large enough to send f + (W k , n e ) and f − (W k , n e ) and thus, |f| is not lower than |f + | + |f − |. |f| is not greater than |f + | + |f − |, neither. Since both f + and f − are the complete maximum flow, |f| is upperbounded by the edge capacities ∀τ p σ (n s , τ p ). The following two propositions are straightforward by definition of FNRT. Proof: It is proved by induction on increasing boundaries from 0 to H . When a task set τ is given, BF generates a feasible schedule by determining the execution time S i,j,k allocated for each τ i,j in W k , where W k is the time window placed in [0, H ].
At time 0, according to Proposition 1, if BF provides a feasible schedule, FNRT F for [0, H ] can be constructed with its complete maximum flow f. f includes f (τ i,j , W k ) that corresponds to S i,j,k of BF.
According to Lemma 2, F can be decomposed into two flow networks F a and F b where F a contains the active job node set τ a within the time interval W 1 , i.e., {τ i,j |∀τ i,j ∈ Parent(W 1 )}, and F b contains the rest job nodes. When F is decomposed, the shared edges {e(W k , n e )|W k ∈ ∪ ∀τ i,j ∈τ a Child(τ i,j )} are split to F a and F b and their capacities VOLUME 8, 2020 are determined by Lemma 2.
Then, both F a and F b have their complete maximum flow f a and f b by Lemma 2.
Assume that fn-EDF finds a maximum flow f a for F a , where |f a | = |f a |. Note that all σ a (W k , n e ) for F a are still preserved and f a is upper-bounded by the capacities. Then, based on Lemma 3, the first window node and its parent nodes can be removed from F a , where we call the FNRT having the first window node removed F a . F a has its complete maximum flow |f a | − |f a (W 1 , n e )|. Then, F a with its complete maximum flow |f a | − |f a (W 1 , n e )| and F b with its complete maximum flow |f b | can be combined based on Lemma 4. The edge capacities σ (W k , n e ) then comes to Ml k . The combined FNRT has its complete maximum flow |f a | − |f a (W 1 , n e )| + |f b |. Since the final FNRT has its complete maximum flow, according to Proposition 2, at least a feasible schedule exists in the time interval [W 2 ,H ]. In addition, note that f b corresponding BF schedule is still preserved. Therefore, the same procedure above can be iteratively conducted up to H .
Corollary 2: fn-EDF on the discrete-time model is an RT-optimal real-time scheduling algorithm.
Proof: According to Theorem 3, if BF schedule exists for a given task set, fn-EDF provides a feasible solution. It is also known that if U of the task set is less than or equal to M , BF schedule exists. Therefore, if U of a task set is less than or equal to M , fn-EDF provides the feasible solutions, which implies its RT-optimality.  In NTU, his research focuses on the design and analysis of real-time and cyber-physical computing systems, including their application in domains such as automotive, manufacturing, and urban energy systems.