Competitive online algorithms for resource allocation over the positive semidefinite cone

Abstract

We consider a new and general online resource allocation problem, where the goal is to maximize a function of a positive semidefinite (PSD) matrix with a scalar budget constraint. The problem data arrives online, and the algorithm needs to make an irrevocable decision at each step. Of particular interest are classic experiment design problems in the online setting, with the algorithm deciding whether to allocate budget to each experiment as new experiments become available sequentially. We analyze two greedy primal-dual algorithms and provide bounds on their competitive ratios. Our analysis relies on a smooth surrogate of the objective function that needs to satisfy a new diminishing returns (PSD-DR) property (that its gradient is order-reversing with respect to the PSD cone). Using the representation for monotone maps on the PSD cone given by Löwner’s theorem, we obtain a convex parametrization of the family of functions satisfying PSD-DR. We then formulate a convex optimization problem to directly optimize our competitive ratio bound over this set. This design problem can be solved offline before the data start arriving. The online algorithm that uses the designed smoothing is tailored to the given cost function, and enjoys a competitive ratio at least as good as our optimized bound. We provide examples of computing the smooth surrogate for D-optimal and A-optimal experiment design, and demonstrate the performance of the custom-designed algorithm.

A Additional proofs

Here, we provide additional proofs not given in detail in the body of the paper.

Proof

(of Lemma 1) By the definition of $D_{\text {sim}}$, the definition of $\tilde{x}_t$, and the concavity of $H_S$ and $G_S$, we have that

$$\begin{aligned} D_{\text {sim}}&={ \sum \limits _{t=1}^{m}}\left[ \langle {A_t\tilde{x}_t} , {\tilde{Y}_t} \rangle + c_t\tilde{x}_t\tilde{z}_t\right] - H^*(\tilde{Y}_m) - G^*(\tilde{z}_m)\\&\le { \sum \limits _{t=1}^{m}}\textstyle {\left[ H_S\left( \sum \limits _{s=1}^{t}A_s\tilde{x}_s\right) - H_S\left( \sum \limits _{s=1}^{t-1}A_s\tilde{x}_s\right) \right. }\\&\qquad {\left. +\, G_S\left( \sum \limits _{s=1}^{t}c_s\tilde{x}_s\right) - G_S\left( \sum \limits _{s=1}^{t-1}c_s\tilde{x}_s\right) \right] } - H^*(\tilde{Y}_m) - G^*(\tilde{z}_m)\\&= {H_S\left( \sum \limits _{s=1}^{m}A_s\tilde{x}_s\right) + G_S\left( \sum _{s=1}^{m}c_s\tilde{x}_s\right) } - H^*(\tilde{Y}_m) - G^*(\tilde{z}_m). \end{aligned}$$

The inequality follows from concavity of $G_S$ and $H_S$. The final equality holds by telescoping the sum and using the fact that $H_S(0) = 0 = G_S(0)$. For the sequential algorithm we can write:

$$\begin{aligned} D_{\text {seq}}&= {\sum \limits _{t=1}^{m}}\left[ \langle {A_t\hat{x}_t} , {\hat{Y}_{t-1}} \rangle + c_t\hat{x}_t\hat{z}_{t-1}\right] - H^*(\hat{Y}_m) - G^*(\hat{z}_m)\\&={\sum \limits _{t=1}^{m}} \left[ \langle {A_t\hat{x}_t} , {\hat{Y}_{t}} \rangle + c_t\hat{x}_t\hat{z}_{t}\right] - H^*(\hat{Y}_m) - G^*(\hat{z}_m) \\&\qquad \qquad +{\sum \limits _{t=1}^{m}} \left[ \langle {A_t\hat{x}_t} , {\hat{Y}_{t-1} - \hat{Y}_{t}} \rangle + c_t\hat{x}_t\left( \hat{z}_{t-1} - \hat{z}_{t}\right) \right] \\ \end{aligned}$$

Now, the rest follows similar to steps as the simultaneous case. $\square$

Proof

(of Lemma 2) We write out the argument for the inequality $D_{\text {sim}} \ge D^\star$. The argument showing that $D_{\text {seq}} \ge D^\star$ is identical. We first show that the PSD-DR assumption on $H_S$ implies

$$\begin{aligned} \sum _{t=1}^{m}\left( \langle {A_t} , {\tilde{Y}_t} \rangle +c_t\tilde{z}_t\right) _+ \ge \sum _{s=1}^{m}\left( \langle {A_s} , {\tilde{Y}_m} \rangle +c_s\tilde{z}_m\right) _+. \end{aligned}$$

(28)

Since $A_s \in S_+^n$ and $\tilde{x}_s \ge 0$ for all $s\in [m]$, it follows that $\sum _{s=1}^{t}A_s \tilde{x}_s \preceq \sum _{s=1}^{m}A_s\tilde{x}_s$ for all $t\in [m]$. Since $\tilde{Y}_t = \nabla H_S\left( \sum _{s=1}^{t}A_s\tilde{x}_s\right)$, if $H_S$ satisfies the PSD-DR assumption then $\tilde{Y}_t \succeq \tilde{Y}_m$ for all $t\in [m]$. By a similar argument, since $G_S$ is concave, $\tilde{z}_t \ge \tilde{z}_m$ for all $t\in [m]$. Since $A_t \in S_+^n$ and $c_t \ge 0$ for all $t\in [m]$,

$$\begin{aligned} \langle {A_t} , {\tilde{Y}_t} \rangle + c_t\tilde{z}_t \ge \langle {A_t} , {\tilde{Y}_m} \rangle + c_t\tilde{z}_t \ge \langle {A_t} , {\tilde{Y}_m} \rangle + c_t\tilde{z}_m \end{aligned}$$

for all $t\in [m]$. Taking the positive part and then summing establishes (28). To conclude that $D_{\text {sim}}\ge D^\star$, we need only observe that $D^\star$ is a lower bound on the dual objective (4) evaluated at $(\tilde{Y}_m,\tilde{z}_m)$. $\square$