Analysis of the Tradeoffs Between Energy and Run Time for Multilevel Checkpointing

Abstract

In high-performance computing, there is a perpetual hunt for performance and scalability. Supercomputers grow larger offering improved computational science throughput. Nevertheless, with an increase in the number of systems’ components and their interactions, the number of failures and the power consumption will increase rapidly. Energy and reliability are among the most challenging issues that need to be addressed for extreme scale computing. We develop analytical models for run time and energy usage for multilevel fault-tolerance schemes. We use these models to study the tradeoff between run time and energy in FTI, a recently developed multilevel checkpoint library, on an IBM Blue Gene/Q. Our results show that energy consumed by FTI is low and the tradeoff between the run time and energy is small. Using the analytical models, we explore the impact of various system-level parameters on run time and energy tradeoffs.

Appendix

We first formalize our assumption on the checkpoint intervals of interest.

Assumption

(A1). We consider checkpoint intervals $\tau \in \mathbb {R}^L_+$ that satisfy (for $i=1, \ldots , L$): (i) $\tau _i>0$; (ii) $\tau _j >\tau _i / 2$ whenever $j> i$; and (iii) $\tau _i <4 /\sum _{j=1}^{i-1}\mu _j$.

The second condition says that the checkpoint at level $j$ cannot be that frequent relative to checkpoints at lower levels. The third condition says that the time between checkpoints needs to be sufficiently smaller than the expected time between any failure at a lower level.

Theorem 1

If (A1) holds, then the time $\mathbb {W}$ and energy $\mathbb {E}$ are convex functions of $\tau \in \mathbb {R}^L$.

Proof

Following (3) and (4), the second-order derivatives of $\mathbb {W}$ are given by

$$\begin{aligned} {\frac{\partial ^2\mathbb {W}}{\partial \tau _{i}^2} }= & {} \frac{c_{i}}{\tau _{i}^{3}} \left( 2+\sum _{j=i+1}^{L} \mu _{j}\tau _{j}\right) \\ \frac{\partial ^2\mathbb {W}}{\partial \tau _{i}\partial \tau _{j}}= & {} - \frac{c_{i}\mu _j}{2\tau _{i}^{2}}, \qquad j\ne i. \end{aligned}$$

We then have

$$\begin{aligned} \small \begin{array}{l} \frac{\partial ^2\mathbb {W}}{\partial \tau _{i}^2}-\sum _{j\ne i}\left| \frac{\partial ^2\mathbb {W}}{\partial \tau _{i}\partial \tau _{j}}\right| \\ = \frac{c_{i}}{\tau _{i}^{2}} \left( \sum \limits _{j=i+1}^{L} \mu _{j}\left( \frac{\tau _{j}}{\tau _i}-\frac{1}{2}\right) + \frac{2}{\tau _i}-\sum \limits _{j=1}^{i-1}\frac{\mu _j}{2} \right) \!, \end{array} \end{aligned}$$

(8)

which is positive by (A1). Equation (8) being positive for all $i$ means that the Hessian $\nabla ^2_{\tau \tau } \mathbb {W}(\tau )$ is diagonally dominant, and thus $\mathbb {W}$ is a convex function of $\tau$ over the domain prescribed by (A1).

The convexity of $\mathbb {E}$ follows by a similar argument, with the derivatives of $\mathbb {E}$ given by

$$\begin{aligned} \small \frac{\partial ^2\mathbb {E}}{\partial \tau _{i}^2}= & {} \frac{\mathcal {P}^{c} _{i}c_{i}}{\tau _{i}^{3}} \left( 2+\sum _{j=i+1}^{L} \mu _{j}\tau _{j}\right) \\ \frac{\partial ^2\mathbb {E}}{\partial \tau _{i}\partial \tau _{j}}= & {} - \frac{\mathcal {P}^{c} _{i}c_{i}\mu _j}{2\tau _{i}^{2}}, \qquad j\ne i. \end{aligned}$$

As a result, there are unique minimizers $\tau ^\mathbb {W}$ and $\tau ^\mathbb {E}$ over the domain prescribed by (A1).