A globally convergent BFGS method for nonlinear monotone equations without any merit functions
HTML articles powered by AMS MathViewer
- by Wei-Jun Zhou and Dong-Hui Li PDF
- Math. Comp. 77 (2008), 2231-2240 Request permission
Abstract:
Since 1965, there has been significant progress in the theoretical study on quasi-Newton methods for solving nonlinear equations, especially in the local convergence analysis. However, the study on global convergence of quasi-Newton methods is relatively fewer, especially for the BFGS method. To ensure global convergence, some merit function such as the squared norm merit function is typically used. In this paper, we propose an algorithm for solving nonlinear monotone equations, which combines the BFGS method and the hyperplane projection method. We also prove that the proposed BFGS method converges globally if the equation is monotone and Lipschitz continuous without differentiability requirement on the equation, which makes it possible to solve some nonsmooth equations. An attractive property of the proposed method is that its global convergence is independent of any merit function.We also report some numerical results to show efficiency of the proposed method.References
- C. G. Broyden, A class of methods for solving nonlinear simultaneous equations, Math. Comp. 19 (1965), 577–593. MR 198670, DOI 10.1090/S0025-5718-1965-0198670-6
- C. G. Broyden, J. E. Dennis Jr., and Jorge J. Moré, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl. 12 (1973), 223–245. MR 341853
- Richard H. Byrd and Jorge Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal. 26 (1989), no. 3, 727–739. MR 997665, DOI 10.1137/0726042
- Richard H. Byrd, Jorge Nocedal, and Ya Xiang Yuan, Global convergence of a class of quasi-Newton methods on convex problems, SIAM J. Numer. Anal. 24 (1987), no. 5, 1171–1190. MR 909072, DOI 10.1137/0724077
- Yu-Hong Dai, Convergence properties of the BFGS algorithm, SIAM J. Optim. 13 (2002), no. 3, 693–701 (2003). MR 1972211, DOI 10.1137/S1052623401383455
- J. E. Dennis Jr. and Jorge J. Moré, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974), 549–560. MR 343581, DOI 10.1090/S0025-5718-1974-0343581-1
- J. E. Dennis Jr. and Jorge J. Moré, Quasi-Newton methods, motivation and theory, SIAM Rev. 19 (1977), no. 1, 46–89. MR 445812, DOI 10.1137/1019005
- John E. Dennis Jr. and Robert B. Schnabel, Numerical methods for unconstrained optimization and nonlinear equations, Prentice Hall Series in Computational Mathematics, Prentice Hall, Inc., Englewood Cliffs, NJ, 1983. MR 702023
- L. C. W. Dixon, Variable metric algorithms: necessary and sufficient conditions for identical behavior of nonquadratic functions, J. Optim. Theory Appl. 10 (1972), 34–40. MR 309305, DOI 10.1007/BF00934961
- Andreas Griewank, The “global” convergence of Broyden-like methods with a suitable line search, J. Austral. Math. Soc. Ser. B 28 (1986), no. 1, 75–92. MR 846784, DOI 10.1017/S0334270000005208
- Andreas Griewank, The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients, Math. Programming 50 (1991), no. 2, (Ser. A), 141–175. MR 1103931, DOI 10.1007/BF01594933
- Guang-Ze Gu, Dong-Hui Li, Liqun Qi, and Shu-Zi Zhou, Descent directions of quasi-Newton methods for symmetric nonlinear equations, SIAM J. Numer. Anal. 40 (2002), no. 5, 1763–1774. MR 1950621, DOI 10.1137/S0036142901397423
- Alfredo N. Iusem and Michael V. Solodov, Newton-type methods with generalized distances for constrained optimization, Optimization 41 (1997), no. 3, 257–278. MR 1473401, DOI 10.1080/02331939708844339
- Tamara G. Kolda, Dianne P. O’Leary, and Larry Nazareth, BFGS with update skipping and varying memory, SIAM J. Optim. 8 (1998), no. 4, 1060–1083. MR 1646119, DOI 10.1137/S1052623496306450
- Dong-Hui Li and Masao Fukushima, A derivative-free line search and global convergence of Broyden-like method for nonlinear equations, Optim. Methods Softw. 13 (2000), no. 3, 181–201. MR 1785195, DOI 10.1080/10556780008805782
- Donghui Li and Masao Fukushima, A globally and superlinearly convergent Gauss-Newton-based BFGS method for symmetric nonlinear equations, SIAM J. Numer. Anal. 37 (1999), no. 1, 152–172. MR 1742754, DOI 10.1137/S0036142998335704
- Dong-Hui Li and Masao Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim. 11 (2001), no. 4, 1054–1064. MR 1855221, DOI 10.1137/S1052623499354242
- Dong-Hui Li and Masao Fukushima, A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math. 129 (2001), no. 1-2, 15–35. Nonlinear programming and variational inequalities (Kowloon, 1998). MR 1823208, DOI 10.1016/S0377-0427(00)00540-9
- Jorge Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp. 35 (1980), no. 151, 773–782. MR 572855, DOI 10.1090/S0025-5718-1980-0572855-7
- J. M. Ortega and W. C. Rheinboldt, Iterative solution of nonlinear equations in several variables, Academic Press, New York-London, 1970. MR 0273810
- M. J. D. Powell, On the convergence of the variable metric algorithm, J. Inst. Math. Appl. 7 (1971), 21–36. MR 279977
- M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches, Nonlinear programming (Proc. Sympos., New York, 1975) SIAM-AMS Proc., Vol. IX, Amer. Math. Soc., Providence, R.I., 1976, pp. 53–72. MR 0426428
- Michael V. Solodov and Benav F. Svaiter, A globally convergent inexact Newton method for systems of monotone equations, Reformulation: nonsmooth, piecewise smooth, semismooth and smoothing methods (Lausanne, 1997) Appl. Optim., vol. 22, Kluwer Acad. Publ., Dordrecht, 1999, pp. 355–369. MR 1682755, DOI 10.1007/978-1-4757-6388-1_{1}8
- Ph. L. Toint, Global convergence of the partitioned BFGS algorithm for convex partially separable optimization, Math. Programming 36 (1986), no. 3, 290–306. MR 866412, DOI 10.1007/BF02592063
- Eberhard Zeidler, Nonlinear functional analysis and its applications. II/B, Springer-Verlag, New York, 1990. Nonlinear monotone operators; Translated from the German by the author and Leo F. Boron. MR 1033498, DOI 10.1007/978-1-4612-0985-0
- Yin Zhang and R. P. Tewarson, Quasi-Newton algorithms with updates from the preconvex part of Broyden’s family, IMA J. Numer. Anal. 8 (1988), no. 4, 487–509. MR 975609, DOI 10.1093/imanum/8.4.487
- Yun-Bin Zhao and Duan Li, Monotonicity of fixed point and normal mappings associated with variational inequality and its application, SIAM J. Optim. 11 (2001), no. 4, 962–973. MR 1855216, DOI 10.1137/S1052623499357957
Additional Information
- Wei-Jun Zhou
- Affiliation: College of Mathematics and Computational Science, Changsha University of Science and Technology, Changsha 410076, China
- Email: weijunzhou@126.com
- Dong-Hui Li
- Affiliation: College of Mathematics and Econometrics, Hunan University, Changsha 410082, China
- Email: dhli@hnu.cn
- Received by editor(s): March 14, 2006
- Received by editor(s) in revised form: March 30, 2007
- Published electronically: May 19, 2008
- Additional Notes: This work was supported by the NSF (10771057 and 10701018) of China.
- © Copyright 2008
American Mathematical Society
The copyright for this article reverts to public domain 28 years after publication. - Journal: Math. Comp. 77 (2008), 2231-2240
- MSC (2000): Primary 90C53
- DOI: https://doi.org/10.1090/S0025-5718-08-02121-2
- MathSciNet review: 2429882