A Convergent Iterative Support Shrinking Algorithm for Non-Lipschitz Multi-phase Image Labeling Model

Abstract

The non-Lipschitz piecewise constant Mumford–Shah model has been shown effective for image labeling and segmentation problems [33], where the non-Lipschitz isotropic $\ell _p$ ($0<p<1$) regularization term can possess strong abilities to maintain sharp edges. However, the Alternating Direction Method of Multiplier (ADMM)-based algorithm used in [33] lacks the convergence guarantee. In this work, we propose an iterative support shrinking algorithm with proximal linearization for multi-phase image labeling problems, which is theoretically proven to be globally convergent. A key step is that we prove a lower bound theory for the nonzero entries of the gradient of the iterative sequence when both box constraint and simplex constraint are involved in the target energy minimization problem. To the best of our knowledge, this is the first theoretical attempt at the non-Lipschitz piecewise constant Mumford–Shah model. Numerical experiments are conducted on both two-phase and multi-phase labeling problems to indicate the efficiency and effectiveness of the proposed algorithm.

Appendices

Appendix A

Proof of the Lemma 2

Proof

Let $\varvec{z}^* = (\varvec{u}_{ I^{1,*}}^{*},\varvec{u}_{ I^{2,*}}^{*},\dots ,\varvec{u}_{ I^{l,*}}^{*})$ and $R(\varvec{z}^*) = F(\varvec{u}^*)$. We prove that $\varvec{z}^*$ is a local minimizer of (6).

Since $\varvec{u}^*$ is a local minimizer of the model (3), there is $F(\varvec{u}) > F(\varvec{u}^*)$ for any $\varvec{u} \in \mathcal S$ and $\Vert \varvec{u} - \varvec{u}^*\Vert < \epsilon ^*$ with $\epsilon ^* > 0$. If $\varvec{z}^* = (\varvec{u}_{ I^{1,*}}^{*},\varvec{u}_{ I^{2,*}}^{*},\dots ,\varvec{u}_{ I^{l,*}}^{*})$ is not a local minimizer, then we can find a $\widetilde{\varvec{z}} \in \{\widetilde{\varvec{z}}:0< {\widetilde{z}}^{i}_{j} <1\}$ satisfying $\Vert \widetilde{\varvec{z}} - \varvec{z}^*\Vert < \epsilon ^*$, $(D_j)_{I^{i,*}} {\widetilde{z}}^{i} = 0$ and $\sum \limits _{i=i}^{l}{\widetilde{z}}_{j}^{i}=1,~\forall j \in { I}_{0}^{i,*}(=\varvec{\Omega }_{0}^{*} \cap I^{i,*})$ such that

$$\begin{aligned} R(\widetilde{\varvec{z}}) < R(\varvec{z}^*). \end{aligned}$$

(25)

Let $\widetilde{\varvec{u}} = (\widetilde{\varvec{z}};\varvec{u}_{B^{1,*}}^*,\varvec{u}_{B^{2,*}}^*,\dots ,\varvec{u}_{B^{l,*}}^*)$. We have $\widetilde{\varvec{u}} \in \mathcal S$, $\Vert \widetilde{\varvec{u}} - \varvec{u}^*\Vert < \epsilon ^*$ and thus $F(\widetilde{\varvec{u}}) > F(\varvec{u}^*)$. Now, we look into the relationship between $R(\widetilde{\varvec{z}})$ and $F(\widetilde{\varvec{u}})$. Because there is $t_{j}^{i,x,*} = t_{j}^{i,y,*} = 0$, $\forall j \in \varvec{\Omega }_{0}^{*}$ and $i \in [1,l]$, we have

$$\begin{aligned} F(\widetilde{\varvec{u}})&= R(\widetilde{\varvec{z}}) + \sum \limits _{j \in \varvec{\Omega }_{0}^{*}}\phi \Bigg (\sqrt{\sum _{i=1}^{l}\Big (((D_{j}^{x})_{I^{i,*}} {\widetilde{z}}^{i}+t_{j}^{i,x,*})^{2}+((D_{j}^{y})_{I^{i,*}} {\widetilde{z}}^{i}+t_{j}^{i,y,*})^{2}}\Big )\Bigg )\\&=R(\widetilde{\varvec{z}}) +\sum \limits _{j \in \varvec{\Omega }_{0}^{*}}\phi \Bigg (\sqrt{\sum _{i=1}^{l}\Big (((D_{j}^{x})_{I^{i,*}} {\widetilde{z}}^{i})^{2}+((D_{j}^{y})_{I^{i,*}} {\widetilde{z}}^{i})^{2}}\Big )\Bigg )\\&=R(\widetilde{\varvec{z}})+\sum \limits _{j \in \varvec{\Omega }_{0}^{*}}\phi \Bigg (\sqrt{\sum _{i=1}^{l}\Big (((D_{j}^{x})_{I^{i,*}} {\widetilde{z}}^{i})^{2}+((D_{j}^{y})_{I^{i,*}} {\widetilde{z}}^{i})^{2}}\Big )\Bigg ). \end{aligned}$$

By $t_{j}^{i,x,*} = t_{j}^{i,y,*} = 0,~\forall j \in \varvec{\Omega }_{0}^{*},~i \in [1,l]$, we get the second equation. The third equation is given by Lemma 1 (e), there is $(D_j^x)_{I^{i,*}} {\widetilde{z}}^{i} = 0$ and $(D_j^y)_{I^{i,*}} {\widetilde{z}}^{i} = 0$. Although the content of the second equation is the same as that of the third equation, $j \in \varvec{\Omega }_{0}^{*}$ in the third equation is reduced. Together with $(D_j)_{I^{i,*}} {\widetilde{z}}^{i} = 0$, we have

$$\begin{aligned} F(\widetilde{\varvec{u}})=R(\widetilde{\varvec{z}}). \end{aligned}$$

We observe that $R(\widetilde{\varvec{z}}) = F(\widetilde{\varvec{u}}) > F({\varvec{u}^*}) = R(\varvec{z}^*)$, which contradicts (25). Thus, $\varvec{z}^*$ is a local minimizer of the optimization problem (6). $\square$

Appendix B

Proof of the Theorem 1

Proof

By the first-order necessary condition of (6), for $i\in [1,l]$, we have

$$\begin{aligned} \langle \partial {R}(\varvec{z}^{*}),\hat{\varvec{z}}\rangle = 0,~~\forall \hat{\varvec{z}}\in \mathcal {K}({{I}^{1,*}_{0}}\times {{I}^{2,*}_{0}}\times \dots \times {{I}^{l,*}_{0}}), \end{aligned}$$

where $\mathcal {K}({{I}^{1,*}_{0}}\times {{I}^{2,*}_{0}}\times \dots \times {{I}^{l,*}_{0}}) = \big \{\varvec{z} \in \mathbb {R}^{|I^{1,*}\times I^{2,*}\times \dots \times I^{l,*}|}:(D_j)_{I^{i,*}} {z}^{i} = 0$ and  $\sum \limits _{i=i}^{l}{z}_{j}^{i}=1,~\forall j \in I_{0}^{i,*}\big \}$. By computing

$$\begin{aligned} \begin{aligned} \Big \langle \partial R(\varvec{z}^{*}),\hat{\varvec{z}}\Big \rangle =&\sum _{i=1}^{l}\sum _{j\in {\varvec{\Omega }^{*}_{1}}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\Big \langle \frac{D_j u^{i,*}}{\Vert D_j \varvec{u}^{*}\Vert },(D_j)_{I^{i,*}}{\hat{z}}^i\Big \rangle \\&+\lambda \sum _{i=1}^{l}\Big \langle (g^i)_{I^{i,*}},~~{\hat{z}}^{i}\Big \rangle , \end{aligned} \end{aligned}$$

we have

$$\begin{aligned} \begin{aligned} \sum _{i=1}^{l}\sum _{j\in {\varvec{\Omega }^{*}_{1}}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\Big \langle \frac{D_j u^{i,*}}{\Vert D_j \varvec{u}^{*}\Vert },(D_j)_{I^{i,*}}{\hat{z}}^i\Big \rangle \le \sum _{i=1}^{l}{\hat{\delta }}_i\Vert {\hat{z}}^{i}\Vert , \end{aligned} \end{aligned}$$

where ${\hat{\delta }}_i=|\lambda ||(g^i)_{I^{i,*}}| > 0$ is a constant independent of $\varvec{u}^*$. Next, we prove $\varvec{\Omega }_1^{*}=\varvec{\Omega }_1(\varvec{u}^{*})\subseteq \varvec{\Omega }_1(\hat{\varvec{u}})$.

We know that $\Vert D_j \varvec{u}^{*}\Vert >\frac{1}{m}$ for $\forall j \in \varvec{L}^{*}$ by Definition 7.5 in [48], and since $\varvec{u}^*$ is very near to $\hat{\varvec{u}}$, we can find a constant $v>0$ such that $\Vert D_j \hat{\varvec{u}}\Vert >v$, for $\forall j \in \varvec{L}^{*}$. Clearly, $\varvec{L}^{*}\subseteq \varvec{\Omega }_1(\hat{\varvec{u}})$, we just need to show ${\varvec{\Omega }}^{*}_{1}\backslash \varvec{L}^{*} \subseteq \varvec{\Omega }_1(\hat{\varvec{u}})$.

For all $j \in {\varvec{\Omega }}^{*}_{1}\backslash \varvec{L}^{*}$, we sort the values of $\Vert D_j \varvec{u}^{*}\Vert$ as

$$\begin{aligned} \Theta ^{*} = \{{\varpi ^{*}_{1}},{\varpi ^{*}_{2}},\cdots ,{\varpi ^{*}_{r}}\}, \end{aligned}$$

where ${\varpi ^{*}_{1}}> {\varpi ^{*}_{2}}> \cdots >{\varpi ^{*}_{r}}$ and $r \le \sharp ({\varvec{\Omega }}^{*}_{1}\backslash \varvec{L}^{*}) < m$. Let ${\varvec{E}}^{*}_{h} = \{j \in {\varvec{\Omega }}^{*}_{1}\backslash \varvec{L}^{*}:\Vert D_j \varvec{u}^{*}\Vert = {\varpi ^{*}_{h}}\}$, where $h \in [1,r]$.

First, we select a $\bar{\varvec{z}}$ such that

$$\begin{aligned} \begin{aligned}&(D_j)_{I^{i,*}}{\bar{z}}^{i} = \frac{D_j u^{i,*}}{{\varpi ^{*}_{1}}},\\&\forall j \in ({{I}^{i,*}_{0}}\cup {\varvec{\Omega }^{*}_{1}})\backslash \varvec{L}^{*},~\forall i \in [1,l],~\sum _{i=1}^{l}{\bar{z}}^{i}_{j}=1. \end{aligned} \end{aligned}$$

(26)

Since $\frac{\left| D_{j}^{x} u^{i,*}\right| }{\varpi ^{*}_{1}} \le 1$, $\frac{\left| D_{j}^{y} u^{i,*}\right| }{\varpi ^{*}_{1}} \le 1$ and $0\le u_j^i \le 1$, we can obtain $\left| {\bar{z}}^{i}\right| <1+m$ and $\left\| {\bar{z}}^{i}\right\| ^{2}<\frac{(m+1)(m+2)(2 m+3)}{6}$ similar Lemma 7.4 in [54]. Moreover, $\left| \left( D_{j}^{x}\right) _{I^{i,*}} {\bar{z}}^{i}\right| <2m$ and $\left| \left( D_{j}^{y}\right) _{I^{i,*}} {\bar{z}}^{i}\right| <2m$ for $\forall j \in J^{i}.$ Thus, $\left\| \left( D_{j}\right) _{I^{i,*}} {\bar{z}}^{i}\right\| ^{2}=$ $\left| \left( D_{j}^{x}\right) _{I^{i,*}} {\bar{z}}^{i}\right| ^{2}+\left| \left( D_{j}^{y}\right) _{I^{i,*}} {\bar{z}}^{i}\right| ^{2}<8m^{2}, \forall i \in J^{i} .$ By (26), it yields

$$\begin{aligned} \langle D_j u^{i,*},(D_j)_{I^{i,*}}{\bar{z}}^{i}\rangle = \frac{\Vert D_j u^{i,*}\Vert ^2}{{\varpi ^{*}_{1}}},~~\forall j \in {\varvec{\Omega }^{*}_{1}}\backslash \varvec{L}^{*}. \end{aligned}$$

When choosing $\hat{\varvec{z}} = \bar{\varvec{z}}$ in (25), we obtain

$$\begin{aligned} \begin{array}{rl} l{\hat{\delta }}\sqrt{\Gamma } &{}> \sum \limits _{i=1}^{l}{\hat{\delta }}_i\Vert {\bar{z}}^{i}\Vert \\ {} &{}\ge \sum \limits _{i=1}^{l}\sum \limits _{j\in {\varvec{\Omega }^{*}_{1}} \backslash \varvec{L}^{*} }\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\langle \frac{D_j u^{i,*}}{\Vert D_j \varvec{u}^{*}\Vert },(D_j)_{I^{i,*}}{\bar{z}}^i\rangle \\ {} &{}~+\sum \limits _{i=1}^{l} \sum \limits _{j\in {\varvec{\Omega }^{*}_{1}} \cap \varvec{L}^{*}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\langle \frac{D_j u^{i,*}}{\Vert D_j \varvec{u}^{*}\Vert },(D_j)_{I^{i,*}}{\bar{z}}^i\rangle \\ &{}\ge \sum \limits _{j\in {{\varvec{E}}_{1}^{*}}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\frac{\Vert D_j \varvec{u}^{*}\Vert }{{\varpi _{1}^{*}}}\\ {} &{}~-\sum \limits _{i=1}^{l}\sum \limits _{j\in {\varvec{\Omega }_{1}^{*}} \cap \varvec{L}^{*}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\frac{\Vert D_j u^{i,*}\Vert }{\Vert D_j \varvec{u}^{*}\Vert }\Vert (D_j)_{I^{i,*}}{\bar{z}}^i\Vert \\ &{}> \sum \limits _{j\in {{\varvec{E}}_{1}^{*}}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )- 8m^2l\sum \limits _{j\in {\varvec{\Omega }_{1}^{*}} \cap \varvec{L}^{*}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel ) \end{array} \end{aligned}$$

(27)

where $\Gamma := \frac{(m+1)(m+2)(2m+3)}{6}$ and ${\hat{\delta }} = \max \limits _{i=1}^{l}{\hat{\delta }}_i$. Then, there is

$$\begin{aligned} \begin{aligned} \sum _{j\in {{\varvec{E}}_{1}^{*}}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )&< l{\hat{\delta }}\sqrt{\Gamma } + 8m^2l\sum _{j\in {{\varvec{\Omega }}_{1}^{*}} \cap \varvec{L}^{*}}\phi '(\parallel {D_j \varvec{u}^{*}}\parallel )\\&< l{\hat{\delta }}\sqrt{\Gamma } + 8m^2l\sum _{j\in {{\varvec{\Omega }}_{1}^{*}} \cap \varvec{L}^{*}}\phi '(\frac{1}{m}). \end{aligned} \end{aligned}$$

(28)

We can follow a similar procedure as Theorem 1 in [54] for the rest proof. $\square$

Appendix C

Proof of the Lemma 3

Proof

According to Assumption 1(b), we have

$$\begin{aligned} \begin{array}{rl} \phi \left( \left\| D_{j}\varvec{u}\right\| \right) \le \phi \left( \left\| D_{j} \varvec{u}^{k}\right\| \right) +\phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left( \left\| D_{j} \varvec{u}\right\| -\left\| D_{j} \varvec{u}^{k}\right\| \right) , \forall j \in {\varvec{\Omega }^{k}_{1}}. \end{array} \end{aligned}$$

(29)

Then, we deduce

$$\begin{aligned} \begin{aligned} F_{k}(\varvec{u})+\frac{\rho }{2}\left\| \varvec{u}-\varvec{u}^{k}\right\| ^{2}&=\lambda \sum _{i=1}^{l}\langle g^i,u^i\rangle +\sum _{j\in {\varvec{\Omega }^{k}_{1}}}\phi (\parallel {D_j \varvec{u}}\parallel )+\frac{\rho }{2}\left\| \varvec{u}-\varvec{u}^{k}\right\| ^{2} \\&\quad \le \lambda \sum _{i=1}^{l}\langle g^i,u^i\rangle +\frac{\rho }{2}\left\| \varvec{u}-\varvec{u}^{k}\right\| ^{2}\\ {}&\qquad +\sum _{j\in {\varvec{\Omega }^{k}_{1}}}\bigg [\phi \left( \left\| D_{j} \varvec{u}^{k}\right\| \right) +\phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left( \left\| D_{j} \varvec{u}\right\| -\left\| D_{j} \varvec{u}^{k}\right\| \right) \bigg ] \\&=H_{k}(\varvec{u})+\sum _{j\in {\varvec{\Omega }^{k}_{1}}}\bigg [\phi \left( \left\| D_{j} \varvec{u}^{k}\right\| \right) -\phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left\| D_{j} \varvec{u}^{k}\right\| \bigg ]. \end{aligned} \end{aligned}$$

(30)

Therefore, we have

$$\begin{aligned}&F(\varvec{u}^{k+1}) +\frac{\rho }{2}\left\| \varvec{u}^{k+1}-\varvec{u}^{k}\right\| ^{2} =F_{k}(\varvec{u}^{k+1})+\frac{\rho }{2}\left\| \varvec{u}^{k+1}-\varvec{u}^{k}\right\| ^{2}\\&~~~~\le H_{k}(\varvec{u}^{k+1})+\sum _{j\in {\varvec{\Omega }^{k}_{1}}}\bigg [\phi \left( \left\| D_{j} \varvec{u}^{k}\right\| \right) -\phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left\| D_{j} \varvec{u}^{k}\right\| \bigg ] \\&~~~~\le H_{k}(\varvec{u}^k)+\sum _{j\in {\varvec{\Omega }^{k}_{1}}}\bigg [\phi \left( \left\| D_{j} \varvec{u}^{k}\right\| \right) -\phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left\| D_{j} \varvec{u}^{k}\right\| \bigg ] \\&~~~~=\lambda \sum _{i=1}^{l}\langle g^i,u^{i,k}\rangle + \sum _{j\in {\varvec{\Omega }^{k}_{1}}} \phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left\| D_{j} \varvec{u}^{k}\right\| \\ {}&~~~~~~+ \sum _{j\in {\varvec{\Omega }^{k}_{1}}}\bigg [\phi \left( \left\| D_{j} \varvec{u}^{k}\right\| \right) -\phi ^{\prime }\left( \left\| D_{j} \varvec{u}^{k}\right\| \right) \left\| D_{j} \varvec{u}^{k}\right\| \bigg ]\\&~~~~=\lambda \sum _{i=1}^{l}\langle g^i,u^{i,k}\rangle + \sum _{j\in {\varvec{\Omega }^{k}_{1}}}\phi (\parallel {D_j \varvec{u}^{k}}\parallel )\\ {}&~~~~=\lambda \sum _{i=1}^{l}\langle g^i,u^{i,k}\rangle +\sum _{j\in \varvec{J}}\phi (\parallel {D_j \varvec{u}^{k}}\parallel )= F(\varvec{u}^k).\\ \end{aligned}$$

By (12), (13) and (15), there is

$$\begin{aligned} {\overline{F}}^{\omega }\left( \varvec{u}_\omega ^{k}\right) -{\overline{F}}^{\omega }\left( \varvec{u}_\omega ^{k+1}\right) \ge \frac{\rho }{2}\left\| \varvec{u}^{k+1}-\varvec{u}^{k}\right\| ^{2} \ge \frac{\rho }{2}\left\| \varvec{u}_{\omega }^{k+1}-\varvec{u}_{\omega }^{k}\right\| ^{2}. \end{aligned}$$

We have $\lim \limits _{k\rightarrow \infty } \Vert \varvec{u}^{k+1}-\varvec{u}^{k}\Vert = 0$ from (15) by the boundness and convergence of ${F(\varvec{u}^k)}$. $\square$

Appendix D

Proof of the Theorem 2

Proof

The reasoning follows the similar procedure as Theorem 4.3 in [48]. Suppose that $\varvec{u}^{k+1}$ is a solution of (14). The index sets in Definition 1 and 3 for $\varvec{u}^{k+1}$ are simplified as ${\varvec{\Omega }^{K}_{1}}$, ${\varvec{\Omega }^{K}_{0}}$, $I^{i,k+1}$, $B^{i,k+1}$, $\varvec{J}_{\omega }^{k+1}$, $I_{\omega }^{i,k+1}$, $B_{\omega }^{i,k+1}$. We also represent $u^{i,k+1}$ as $\Big (u_{I^{i,k+1}}^{i,k+1};u_{B^{i,k+1}}^{i,k+1}\Big )$, and $u_{\omega }^{i,k+1}$ as $\Big ((u_{\omega }^{i,k+1})_{I_{\omega }^{i,k+1}};(u_{\omega }^{i,k+1})_{B_{\omega }^{i,k+1}}\Big )$. Because each row of $E_{\omega }^{i}$ only has one nonzero element 1 and $I^{i,k+1} \cap B^{i,k+1} = \emptyset$, we can take

$$\begin{aligned} \begin{aligned} u_{I^{i,k+1}}^{k+1}&=(E_{\omega }^{i})_{I^{i,k+1}}\left( u_{\omega }^{i,k+1}\right) _{I_{\omega }^{i,k+1}}, \\ u_{B^{i,k+1}}^{k+1}&=\left( E_{\omega }^{i}\right) _{B^{i,k+1}}\left( u_{\omega }^{i,k+1}\right) _{B_{\omega }^{i,k+1}}. \end{aligned} \end{aligned}$$

We define $D_j^x u^{i,k+1}$ and $D_j^y u^{i,k+1}$ as

$$\begin{aligned} \begin{array}{l} D_{j}^{x} u^{i,k+1}=\left( D_{j}^{x}\right) _{I^{i,k+1}}\left( \left( E_{\omega }^{i}\right) _{I^{i,k+1}}\left( u_{\omega }^{i,k+1}\right) _{I_{\omega }^{i,k+1}}\right) \\ ~~~~~~~~~~~~~~\qquad +\left( D_{j}^{x}\right) _{B^{i,k+1}}\left( \left( E_{\omega }^{i}\right) _{B^{i,k+1}}\left( u_{\omega }^{i,k+1}\right) _{B_{\omega }^{i,k+1}}\right) ; \\ D_{j}^{y} u^{i,k+1}=\left( D_{j}^{y}\right) _{I^{i,k+1}}\left( \left( E_{\omega }^{i}\right) _{I^{i,k+1}}\left( u_{\omega }^{i,k+1}\right) _{I_{\omega }^{i,k+1}}\right) \\ ~~~~~~~~~~~~~~\qquad +\left( D_{j}^{y}\right) _{B^{i,k+1}}\left( \left( E_{\omega }^{i}\right) _{B^{i,k+1}}\left( u_{\omega }^{i,k+1}\right) _{B_{\omega }^{i,k+1}}\right) . \end{array} \end{aligned}$$

Similar to the (5), let

$$\begin{aligned} t_{j}^{i,x, k+1}=\left( D_{j}^{x}\right) _{B^{i,k+1}} u_{B^{i,k+1}}^{i,k+1},~~\forall j \in J^{i}, \end{aligned}$$

and

$$\begin{aligned} t_{j}^{i,y, k+1}=\left( D_{j}^{y}\right) _{B^{i,k+1}} u_{B^{i,k+1}}^{i,k+1},~~\forall j \in J^{i}. \end{aligned}$$

Then there is $t_{j}^{i,x, k+1} = t_{j}^{i,y, k+1} = 0$, $\forall j \in {\varvec{\Omega }^{K}_{0}}$. To analyze $(\overline{\mathcal {H}}_k^{\omega })$ in (14), we construct the following optimization problem

$$\begin{aligned} \min \limits _{\varvec{z}_\omega }~~{\overline{R}}^{\omega }_k( \varvec{z}_\omega ):&=\lambda \sum \limits _{i=1}^{l}\langle (g^i)_{I^{i,k+1}},(E_{\omega }^{i})_{I^{i,k+1}} z_{\omega }^{i}\rangle \nonumber +\lambda \sum \limits _{i=1}^{l}\langle (g^i)_{B^{i,k+1}},u^{i,k+1}_{B^{i,k+1}}\rangle \\&\quad +\sum \limits _{i=1}^{l}\chi _{(0,1)}\Big (( E_{\omega }^{i})_{I^{i,k+1}} z_{\omega }^{i} + u_{B^{i,k+1}}^{i,k+1}\Big )\nonumber \\&\quad +\sum \limits _{j\in {\varvec{\Omega }^{k}_{1}}} \phi \left( \Vert (D_j)_{I^{k+1}}\bar{\varvec{z}}_\omega +(D_j)_{B^{k+1}}\varvec{u}_{B^{k+1}}^{k+1}\Vert \right) \nonumber \\&\quad +\frac{\rho }{2}\parallel {((\bar{\varvec{z}}_\omega }+\varvec{u}_{B^{k+1}}^{k+1}) -\varvec{u}^{k}\parallel ^2+ \chi _{\mathcal S}\Big (\bar{\varvec{z}}_\omega +\varvec{u}_{B^{k+1}}^{k+1}\Big )\\&\quad \text{ s.t. }\quad (D_j)_{I^{i,k+1}} (E_{\omega }^{i})_{I^{i,k+1}}z_{\omega }^{i} = 0,~~\forall j\in {\varvec{\Omega }^{k}_{0}}\cap I^{i,k+1}\nonumber , \end{aligned}$$

(31)

where $\bar{\varvec{z}}_\omega = (( E_{\omega }^{1})_{I^{1,k+1}} z_{\omega }^{1},\dots ,( E_{\omega }^{l})_{I^{l,k+1}} z_{\omega }^{l})$, $\Vert (D_j)_{I^{k+1}} \bar{\varvec{z}}_\omega +(D_j)_{ B^{k+1}}\varvec{u}_{B^{k+1}}^{k+1}\Vert = \sum \limits _{i=1}^{l} \Big (\big (D_{j}^{x}\big )_{I^{i,k+1}} (E_{\omega }^{i})_{I^{i,k+1}} z_{\omega }^{i} +t_{j}^{i,x, k+1}\Big )^{2}+ \sum \limits _{i=1}^{l}\Big (\big (D_{j}^{y}\big )_{I^{i,k+1}} (E_{\omega }^{i})_{I^{i,k+1}} z_{\omega }^{i}+t_{j}^{i,y, k+1}\Big )^{2}$ and $\bar{\varvec{z}}_\omega + \varvec{u}_{B^{k+1}}^{k+1} =\Big ((E_{\omega }^{1})_{I^{1,k+1}}z_{\omega }^{1}+u_{B^{1,k+1}}^{k+1}, \cdots , (E_{\omega }^{l})_{I^{l,k+1}}z_{\omega }^{l}+u_{B^{l,k+1}}^{k+1}\Big )$. By the first-order necessary condition of (31), we can follow a similar procedure in Theorem 1 as the rest proof. $\square$

Appendix E

Proof of the Lemma 4

Proof

When $k \ge {\widetilde{K}}$, $\varvec{u}_\omega ^{k+1}$ is a solution of$({\overline{H}}_k^\omega )$. By the first-order optimality condition, we have

$$\begin{aligned} \begin{array}{rl} \varvec{0} \in &{}\partial {\overline{H}}_{k}^{\omega }\left( \varvec{u}_{\omega }^{k+1}\right) \\ =&{}\bigg (\sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}\phi '(\parallel {D_j \varvec{u}^{k}}\parallel )\frac{(D_j E_{\omega }^{1})^\top D_j u^{1,k+1}}{\Vert D_j \varvec{u}^{k+1}\Vert } +(E_{\omega }^{1})^\top \lambda g^1\\ &{}+ \rho (E_{\omega }^{1})^\top ( u^{1,k+1}- u^{1,k})+ \partial \chi _{[0,1]}(u^{1,k+1}) +\partial \chi _{\mathcal S}(\varvec{u}),\\ &{}\dots , \\ &{}\sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}\phi '(\parallel {D_j \varvec{u}^{k}}\parallel )\frac{(D_j E_{\omega }^{l})^\top D_j u^{l,k+1}}{\Vert D_j \varvec{u}^{k+1}\Vert } +(E_{\omega }^{l})^\top \lambda g^l\\ &{}+ \rho (E_{\omega }^{l})^\top ( u^{l,k+1}- u^{l,k})+ \partial \chi _{[0,1]}(u^{l,k+1}) +\partial \chi _{\mathcal S}(\varvec{u})\bigg ), \end{array} \end{aligned}$$

where $\partial \chi _{[0,1]}(u^{i,k+1})$ is the subdifferential of $\chi _{[0,1]}(v)$ at $u^{i,k+1}$ and $\partial \chi _{\mathcal S}(\varvec{u})$ is the subdifferential of $\chi _{\mathcal S}(\varvec{u})$ at $u^{i,k+1}$. Assume that there exist $\xi ^{i,k+1} \in \partial \chi _{[0,1]}(u^{i,k+1})$ and $\varsigma ^{i,k+1} \in \partial \chi _{\mathcal S}(\varvec{u})$, such that

$$\begin{aligned} \begin{array}{rl} 0 =&{}\sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}\phi '(\parallel {D_j \varvec{u}^{k}}\parallel )\frac{(D_j E_{\omega }^{i})^\top D_j u^{i,k+1}}{\Vert D_j \varvec{u}^{k+1}\Vert }+(E_{\omega }^{i})^\top \lambda g^i\\ &{} + \rho ( E_{\omega }^{i})^\top (u^{i,k+1}- u^{i,k})+\xi ^{i,k+1} + \varsigma ^{i,k+1}. \end{array} \end{aligned}$$

(32)

Since $\Vert D_j \varvec{u}^{k+1}\Vert \ne 0$ for $\forall j \in {\varvec{\Omega }_{1}^{K}}$ and $k \ge {\widetilde{K}}$, the subdifferential of ${\overline{F}}^{\omega }(u_{\omega }^{i,k+1})$ is

$$\begin{aligned} \begin{array}{rl} \partial {\overline{F}}^{\omega }(u_{\omega }^{i,k+1})=&{}(E_{\omega }^{i})^\top \lambda g^i+\sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}\phi '(\parallel {D_j \varvec{u}^{k+1}}\parallel )\frac{(D_j E_{\omega }^{i})^\top D_j u^{i,k+1}}{\Vert D_j \varvec{u}^{k+1}\Vert }\\ &{}\quad + \partial \chi _{[0,1]}(u^{i,k+1}) + \partial \chi _{\mathcal S}(\varvec{u}). \end{array} \end{aligned}$$

In particular, there exists $\tau ^{i,k+1} \in \partial {\overline{F}}^{\omega }(u_{\omega }^{i,k+1})$, such that

$$\begin{aligned} \begin{aligned} \tau ^{i,k+1}&= (E_{\omega }^{i})^\top \lambda g^i + \xi ^{i,k+1} + \varsigma ^{i,k+1} \\&\quad + \sum _{j\in {\varvec{\Omega }_{1}^{K}}}\phi '(\parallel {D_j \varvec{u}^{k+1}}\parallel )\frac{(D_j E_{\omega }^{i})^\top D_j u^{i,k+1}}{\Vert D_j \varvec{u}^{k+1}\Vert }. \end{aligned} \end{aligned}$$

(33)

Combine (32) and (33), we obtain that

$$\begin{aligned} \begin{aligned} \tau ^{i,k+1}&=\rho ( E_{\omega }^{i})^\top (u^{i,k}-u^{i,k+1})\\&\quad + \sum _{j\in {\varvec{\Omega }_{1}^{K}}}\left( \phi '(\parallel {D_j \varvec{u}^{k+1}}\parallel )-\phi '(\parallel {D_j \varvec{u}^{k}}\parallel )\right) \frac{(D_j E_{\omega }^{i})^\top D_j u^{i,k+1}}{\Vert D_j \varvec{u}^{k+1}\Vert }. \\ \end{aligned} \end{aligned}$$

Therefore, we can deduce

$$\begin{aligned} \begin{aligned} \Vert \tau ^{i,k+1}\Vert&\le \rho \Vert E_{\omega }^{i}\Vert \Vert u^{i,k+1}-u^{i,k}\Vert \\&\qquad +\sum _{j\in {\varvec{\Omega }_{1}^{K}}}\left( \phi '(\parallel {D_j \varvec{u}^{k+1}}\parallel )-\phi '(\parallel {D_j \varvec{u}^{k}}\parallel )\right) \frac{\Vert D_j u^{i,k+1}\Vert }{\Vert D_j \varvec{u}^{k+1}\Vert }\Vert D_j E_{\omega }^{i}\Vert \\&\le \sum _{j\in {\varvec{\Omega }_{1}^{K}}}L_{\eta }\Vert D_j\Vert ^2\Vert u^{i,k+1}-u^{i,k}\Vert \Vert E_{\omega }^{i}\Vert +\rho \Vert E_{\omega }^{i}\Vert \Vert u^{i,k+1}-u^{i,k}\Vert . \end{aligned} \end{aligned}$$

Then, it gives

$$\begin{aligned} \begin{array}{rl} \Vert \tau ^{k+1}\Vert &{}\le \sum \limits _{i=1}^{l}\Vert \tau ^{i,k+1}\Vert \\ &{}\le \sum \limits _{i=1}^{l}\left( \sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}L_{\eta }\Vert D_j\Vert ^2\Vert u^{i,k+1}-u^{i,k}\Vert \Vert E_{\omega }^{i}\Vert + \rho \Vert E_{\omega }^{i}\Vert \Vert u^{i,k+1}-u^{i,k}\Vert \right) \\ &{}\le \sum \limits _{i=1}^{l}\left( \sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}L_{\eta }\Vert D_j\Vert ^2\Vert E_{\omega }^{i}\Vert ^{2} + \rho \Vert E_{\omega }^{i}\Vert ^{2}\right) \Vert \varvec{u}_{\omega }^{k+1}-\varvec{u}_{\omega }^{k}\Vert \\ &{}= (\sum \limits _{i=1}^{l}\Vert E_{\omega }^{i}\Vert ^{2})\Big (\sum \limits _{j\in {\varvec{\Omega }_{1}^{K}}}L_{\eta }\Vert D_j\Vert ^2 + \rho \Big )\Vert \varvec{u}_{\omega }^{k+1}-\varvec{u}_{\omega }^{k}\Vert , \end{array} \end{aligned}$$

which completes the proof. $\square$