Abstract
The forward–backward splitting technique is a popular method for solving monotone inclusions that have applications in optimization. In this paper, we explore the behaviour of the algorithm when the inclusion problem has no solution. We present a new formula to define the normal solutions using the forward–backward operator. We also provide a formula for the range of the displacement map of the forward–backward operator. Several examples illustrate our theory.
Similar content being viewed by others
Notes
We point out that the assumption A is firmly nonexpansive can be relaxed to A is cocoercive (see Remark 3.1).
For general conditions on strong convergence of the forward–backward algorithm we refer the reader to [13].
Let \({n\in {\mathbb N}}\). The positive orthant in \(\mathbb {R}^n\) is \(\mathbb {R}^n_{+}=[0,+\infty [^n\) and the strictly positive orthant in \(\mathbb {R}^n\) is \(\mathbb {R}^n_{++}=]0,+\infty [^n\). Likewise, we define the negative orthant and the strictly negative orthant \(\mathbb {R}^n_{-}\) and \(\mathbb {R}^n_{--}\), respectively.
Let \(B:X\rightrightarrows X\). Then and (see [27, Equation (10)]).
Recall that \(A:X\rightarrow X\) is cocoercive if \((\exists \alpha >0)\) such that \(\alpha A\) is firmly nonexpansive.
For detailed discussion and examples of paramonotone operators we refer the reader to [33].
For detailed discussion and examples of \(3^*\) monotone operators we refer the reader to [34].
Let \(h:X\rightarrow ]-\infty ,+\infty ]\) be proper. The set of minimizers of h, \(\left\{ x\in X~:~h(x)=\inf h(X)\right\} \), is denoted by \(\mathrm{argmin}\, h\).
Suppose that \(g:X\rightarrow ]-\infty ,+\infty ]\) is convex, lower semicontinuous and proper. Then \(\mathrm{Prox}_g\) is the Moreau prox operator associated with g defined by \(\mathrm{Prox}_g:X\rightarrow X:x\mapsto (\mathrm{Id}+{\partial }g)^{-1}(x) =\mathrm{argmin}_{y\in X} \left( g(y)+\tfrac{1}{2}||x-y||^2\right) \).
Let C be a nonempty closed and convex subset of X. We use \(d_C\) to denote the distance from the set C defined by \(d_C:X\rightarrow [0,+\infty [ :x\mapsto \min _{c\in C}||x-c||=||x-P_C x||\).
For convenience we shall use \(v_{\mathrm{FB}}\) and \(v_{\mathrm{DR}}\) to denote \(v_{T_{{}{\mathrm{FB}}}}\) and \(v_{T_{{}{\mathrm{DR}}}}\), respectively.
Let C be a subset of H. We use \(\mathrm{ri}C\) to denote the interior of C with respect to the affine hull of C.
For detailed discussion on the algebra of nearly convex sets, we refer the reader to [42, Section 3].
For detailed discussion on the properties of nearly equal and nearly convex sets, we refer the reader to [43].
Let \(B:X\rightrightarrows X\). Then, B is an affine relation if \(\mathrm{gra}B\) is an affine subspace of \(X\times X\).
Let \(f:X\rightarrow ]-\infty ,+\infty ]\) be convex, lower semicontinuous and proper. We use \(f^*\) to denote the convex conjugate (a.k.a. Fenchel conjugate) of f, defined by \(f^*:X\rightarrow ]-\infty ,+\infty ]:x\mapsto \sup _{u\in X}(\langle x,u\rangle -f(x))\).
Let C be a nonempty, closed and convex subset of X. The recession cone of C is \(\mathrm{rec}C:=\{x\in X~:~x+C\subseteq C\}\), and the polar cone of C is \({C^{\ominus }:=\big \{{u\in X}~:~{\sup _{c\in C}\langle c,u\rangle \le 0}\big \}, } \)
Suppose that U is a closed affine subspace of X. We use \(\mathrm{par}U\) to denote the parallel space of U defined by \(\mathrm{par}U:=U-U\).
References
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Berlin (2011)
Borwein, J.M., Vanderwerff, J.G.: Convex Functions. Cambridge University Press, Cambridge (2010)
Brezis, H.: Operateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert. North-Holland/Elsevier, Amsterdam (1973)
Burachik, R.S., Iusem, A.N.: Set-Valued Mappings and Enlargements of Monotone Operators. Springer-Verlag, Berlin (2008)
Simons, S.: Minimax and Monotonicity. Springer-Verlag, Berlin (1998)
Simons, S.: From Hahn-Banach to Monotonicity. Springer-Verlag, Berlin (2008)
Zeidler, E.: Nonlinear Functional Analysis and Its Applications II/A: Linear Monotone Operators. Springer-Verlag, Berlin (1990)
Zeidler, E.: Nonlinear Functional Analysis and Its Applications II/B: Nonlinear Monotone Operators. Springer-Verlag, Berlin (1990)
Zeidler, E.: Nonlinear Functional Analysis and Its Applications I: Fixed Point Theorems. Springer-Verlag, Berlin (1993)
Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Control Optim. 29, 119–138 (1991)
Lemaire, B.: Which fixed point does the iteration method select? In: Lecture Notes in Economics and Mathematical Systems, Vol 452, pp. 154–167. (1979)
Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53, 475–504 (2004)
Attouch, H., Briceño-Arias, L.M., Combettes, P.L.: A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 48, 3246–3270 (2010)
Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4, 1168–1200 (2005)
Combettes, P.L., Dũng, Đ., Vũ, B.C.: Dualization of signal recovery problems. Set-Valued Var. Anal. 18, 373–404 (2010)
Bauschke, H.H., Hare, W.L., Moursi, W.M.: Generalized solutions for the sum of two maximally monotone operators. SIAM J. Control Optim. 52, 1034–1047 (2014)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer-Verlag, Berlin (2009). corrected 3rd printing
Bruck, R.E., Reich, S.: Nonexpansive projections and resolvents of accretive operators in Banach spaces. Houst. J. Math. 3, 459–470 (1977)
Baillon, J.B., Bruck, R.E., Reich, S.: On the asymptotic behavior of nonexpansive mappings and semigroups in Banach spaces. Houst. J. Math. 4, 1–9 (1978)
Pazy, A.: Asymptotic behavior of contractions in Hilbert space. Israel J. Math. 9, 235–240 (1971)
Bauschke, H.H., Dao, M.N., Moursi, W.M.: On Fejér monotone sequences and nonexpansive mappings. Linear Nonlinear Anal. 1, 287–295 (2015)
Bauschke, H.H., Moursi, W.M.: The Douglas-Rachford algorithm for two (not necessarily intersecting) affine subspace. SIAM J. Optim. 26, 968–985 (2016)
Goebel, K., Reich, S.: Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings. Marcel Dekker, New York (1984)
Attouch, H., Théra, M.: A general duality principle for the sum of two operators. J. Convex Anal. 3, 1–24 (1996)
Svaiter, B.F.: On weak convergence of the Douglas-Rachford method. SIAM J. Control Optim. 49, 280–287 (2011)
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)
Bauschke, H.H., Boţ, R.I., Hare, W.L., Moursi, W.M.: Attouch-Théra duality revisited: paramonotonicity and operator splitting. J. Approx. Theory 164, 1065–1084 (2012)
Bauschke, H.H., Wang, X., Yao, L.: Rectangularity and paramonotonicity of maximally monotone operators. Optimization 63, 487–504 (2014)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)
Baillon, J.B., Haddad, G.: Quelques propriétés des opérateurs angle-bornés et n-cycliquement monotones. Israel J. Math. 26, 137–150 (1977)
Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)
Gossez, J.-P.: Opérateurs monotones non linéaires dans les espaces de Banach non réflexifs. J. Math. Anal. Appl. 34, 371–395 (1971)
Iusem, A.N.: On some properties of paramonotone operators. J. Convex Anal. 5, 269–278 (1998)
Brezis, H., Haraux, A.: Image d’une somme d’opérateurs monotones et applications. Israel J. Math. 23, 165–186 (1976)
Burachik, R.S., Jeyakumar, V.: A new geometric condition for Fenchel’s duality in infinite dimensional spaces. J. Convex Anal. 12, 279–290 (2005)
Moreau, J.-J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. Fr. 93, 273–299 (1965)
Zarantonello, E.H.: Projections on convex sets in Hilbert space and spectral theory. In: Zarantonello, E.H. (ed.) Contributions to Nonlinear Functional Analysis, pp. 237–424. Academic Press, New York (1971)
Eckstein, J.: Splitting Methods for Monotone Operators with Applications to Parallel Optimization, Ph.D. Thesis, MIT (1989)
Bauschke, H.H., Hare, W.L., Moursi, W.M.: On the range of the Douglas-Rachford operator. Math. Oper. Res. 41, 884–897 (2016)
Bauschke, H.H., Borwein, J.M.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory 79, 418–443 (1994)
Bauschke, H.H., Moffat, S.M., Wang, X.: Firmly nonexpansive mappings and maximally monotone operators: correspondence and duality. Set-Valued Var. Anal. 20, 131–153 (2012)
Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Bauschke, H.H., Moffat, S.M., Wang, X.: Near equality, near convexity, sums of maximally monotone operators, and averages of firmly nonexpansive mappings. Math. Program. (Ser. B) 139, 55–70 (2013)
Bauschke, H.H., Lukens, B., Moursi, W.M.: Affine nonexpansive operators, Attouch–Théra duality and the Douglas–Rachford algorithm, to appear in Set-Valued Var. Anal. arXiv:1603.09418 [math.OC]
Baillon, J.B.: Quelques propriétés de convergence asymptotique pour les contractions impaires. In: Comptes rendus de l’Académie des Sciences 238, Aii, pp. A587–A590 (1976)
Bauschke, H.H., Deutsch, F., Hundal, H., Park, S.-H.: Accelerating the convergence of the method of alternating projections. Trans. Am. Math. Soc. 355, 3433–3461 (2003)
Conway, J.B.: A Course in Functional Analysis. Springer-Verlag, Berlin (1990)
Pierra, G.: Decomposition through formalization in a product space. Math. Program. 28, 96–115 (1984)
Combettes, P.L., Vũ, B.C.: Variable metric forward-backward splitting with applications to monotone inclusions in duality. Optimization 63, 1289–1318 (2014)
Knopp, K.: Infinite Sequences and Series. Dover, New York (1956)
Moffat, S.M., Moursi, W.M., Wang, X.: Nearly convex sets: fine properties and domains or ranges of subdifferentials of convex functions. Math. Program. (Ser. A) 160, 193–223 (2016)
Acknowledgements
The author thanks Heinz Bauschke for his constructive comments and support. The author also thanks two anonymous referees for their careful refereeing of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Michel Théra.
Appendix
Appendix
Proof of Example 5.1
The claim about firm nonexpansiveness of A follows from, e.g. [37, Equation 1.6 on page 241] or [23, Section 3] and maximal monotonicity of B follows from Fact 3.2(i) applied to \(f^*\). Using Fact 3.2(ii) and [42, Example on page 218], we see that \(\mathrm{dom}{\partial }f=\mathrm{ran}({\partial }f)^{-1}= \mathrm{ran}{\partial }f^* =\mathrm{ran} B= \big \{{(\xi _1,\xi _2)}~:~{\xi _1>0, \xi _2\in \mathbb {R}}\big \} \) \( \cup \big \{{(0,\xi _2)}~:~{|\xi _2|\ge 1}\big \}\). Note that in view of Theorem 5.1(i) we have \( \big \{{(\xi _1,\xi _2)}~:~{\xi _1>0, \xi _2\in \mathbb {R}}\big \}=\mathrm{ri}(\mathrm{ran}A+\mathrm{ran}B) \) \(\subseteq \mathrm{ran}(\mathrm{Id}-T)\subseteq \mathrm{cl}{(\mathrm{ran}A+\mathrm{ran}B)} =\big \{{(\xi _1,\xi _2)}~:~{\xi _1\ge 0, \xi _2\in \mathbb {R}}\big \}\). Therefore, we only need to check the points in \(\big \{{(0,\beta )}~:~{\beta \in \mathbb {R}}\big \}\). To proceed further we recall that (see [51, Example 6.5])
Let \(\beta \in \mathbb {R}\). In view of Proposition 4.1 and Fact 3.2(ii) we have
We argue by cases using (28) and Example 29.
- Case 1: :
-
\(\xi _1\ge 0\) and \(\xi _2\ge 0\). Then \((0,\beta )\in \mathrm{ran}(\mathrm{Id}-T_{{}{\mathrm{FB}}})\) \(\Leftrightarrow (\exists (\xi _1,\xi _2)\in \mathbb {R}^2)~ (\xi _1,\xi _2-\beta )\in {\partial }f ( (0,\beta )-P_{\mathbb {R}_+^2} (\xi _1,\xi _2))\) \( ={\partial }f (-\xi _1,\beta -\xi _2))\) \(\Leftrightarrow \) [\((\exists (\xi _1,\xi _2)\in \mathbb {R}^2)~\xi _1=0, \xi _2-\beta =1 \text { and } \beta -\xi _2\ge 1\) or \(\xi _1=0, \xi _2-\beta =-1 \text { and } \beta - \xi _2\le -1\)], which is impossible.
- Case 2: :
-
\(\xi _1\le 0\) and \(\xi _2\le 0\). Then \((0,\beta )\in \mathrm{ran}(\mathrm{Id}-T_{{}{\mathrm{FB}}})\) \(\Leftrightarrow (\exists (\xi _1,\xi _2)\in \mathbb {R}^2)~ (\xi _1,\xi _2-\beta )\in {\partial }f ( (0,\beta )-P_{\mathbb {R}_+^2} (\xi _1,\xi _2))\) \( ={\partial }f (0,\beta ))\) \(\Leftrightarrow \) [\((\exists (\xi _1,\xi _2)\in \mathbb {R}^2)~ \xi _1\le 0, \xi _2-\beta =1 \text { and } \beta \ge 1\) or \(\xi _1\le 0, \xi _2-\beta =-1 \text { and } \beta \le -1\) ] \(\Leftrightarrow \) [ \((\exists (\xi _1,\xi _2)\) \(\in \mathbb {R}^2)~ \xi _1\le 0, \xi _2=\beta +1\ge 2 \) or \(\xi _1\le 0, \xi _2=\beta -1\le -2\)]. Since \(\xi _2\le 0\) we conclude that \(\beta \le -1\).
- Case 3: :
-
\(\xi _1> 0\) and \(\xi _2 < 0\). Then \((0,\beta )\in \mathrm{ran}(\mathrm{Id}-T_{{}{\mathrm{FB}}})\) \(\Leftrightarrow (\exists (\xi _1,\xi _2)\in \mathbb {R}^2)~ (\xi _1,\xi _2-\beta )\in {\partial }f ( (0,\beta )-P_{\mathbb {R}_+^2} (\xi _1,\xi _2)) \) \( ={\partial }f (-\xi _1,\beta )\) \(\Rightarrow \) [\( \xi _1>0\) and by (28) \(-\xi _1>0\)] which is impossible.
- Case 4: :
-
\(\xi _1< 0\) and \(\xi _2>0\). Then, \((0,\beta )\in \mathrm{ran}(\mathrm{Id}-T_{{}{\mathrm{FB}}})\) \(\Leftrightarrow (\exists (\xi _1,\xi _2)\in \mathbb {R}^2)~ (\xi _1,\xi _2-\beta )\in {\partial }f ( (0,\beta )-P_{\mathbb {R}_+^2} (\xi _1,\xi _2)) \) \( ={\partial }f (0,\beta -\xi _2)\) \(\Leftrightarrow \)[\(\xi _1<0, \xi _2-\beta =1 \text { and }\beta -\xi _2\ge 1 \text { or } \xi _1<0, \xi _2-\beta =-1 \text { and }\beta -\xi _2\le -1\)], which never occurs.
Altogether we conclude that \(\mathrm{ran}(\mathrm{Id}-T_{{}{\mathrm{FB}}})= \big \{{(\xi _1,\xi _2)}~:~{\xi _1>0, \xi _2\in \mathbb {R}}\big \} \cup \big \{{(0,\xi _2)}~:~{\xi _2\le -1}\big \}\), as claimed. \(\square \)
Rights and permissions
About this article
Cite this article
Moursi, W.M. The Forward–Backward Algorithm and the Normal Problem. J Optim Theory Appl 176, 605–624 (2018). https://doi.org/10.1007/s10957-017-1113-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-017-1113-4
Keywords
- Attouch–Théra duality
- Firmly nonexpansive mapping
- Fixed point
- Forward–backward splitting operator
- Normal problem