Abstract
In this paper we propose new globalization strategies for the Barzilai and Borwein gradient method, based on suitable relaxations of the monotonicity requirements. In particular, we define a class of algorithms that combine nonmonotone watchdog techniques with nonmonotone linesearch rules and we prove the global convergence of these schemes. Then we perform an extensive computational study, which shows the effectiveness of the proposed approach in the solution of large dimensional unconstrained optimization problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. Barzilai and J.M. Borwein, “Two point step size gradient method,” IMA J. Numer. Anal., vol. 8, pp. 141–148, 1988.
D.P. Bertsekas, Nonlinear Programming, 2nd edn., Athena Scientific, 1999.
I. Bongartz, A. Conn, N. Gould, and P. Toint, “CUTE: constrained and unconstrained testing Environments,” ACM Transaction on Math. Software, vol. 21, pp. 123–160, 1995.
R.M. Chamberlain, M.J.D. Powell, C. Lemarechal, and H.C. Pedersen, “The watchdog technique for forcing convergence in algorithms for constrained optimization,” Math. Programming, vol. 16, pp. 1–17, 1982.
Y.H. Dai and L.Z. Liao, “R-Linear Convergence of the Barzilai and Borwein gradient method,” Research Report, 1999. Alsoin IMA J. Numer. Anal., vol. 22, pp. 1-10, 2002.
Y.H. Dai and H. Zhang, “An adaptive two-point stepsize gradient algorithm,” Research report, Chinese Academy of Sciences, 2000.
R. De Leone, M. Gaudioso, and L. Grippo, “Stopping criteria for linesearch methods without derivatives,” Math. Programming, vol. 30, pp. 285–300, 1984.
R. Fletcher, “Low storage methods for unconstrained optimization,” Lectures in Applied Mathematics (AMS), vol. 26, pp. 165–179, 1990.
R. Fletcher, “On the Barzilai-Borwein method,” Numerical Analysis Report NA/207, 2001.
A. Friedlander, J.M. Martinez, B. Molina, and M. Raydan, “Gradient method with retards and generalizations,” SIAM J. Numer. Anal., vol. 36, pp. 275–289, 1999.
A. Friedlander, J.M. Martinez, and M. Raydan, “A new method for large-scale box constrained convex quadratic minimization problems,” Optimization Methods and Software, vol. 5, pp. 55–74, 1995.
J. Gilbert and C. Lemaréchal, “Some numerical experiments with variable-storage quasi-Newton algorithms,” Math. Programming, Series B, vol. 45, pp. 407–435, 1989.
P. Gill, W. Murray, and M.H. Wright, Practical Optimization, Academic Press: San Diego, 1981.
W. Glunt, T.L. Hayden, and M. Raydan, “Molecular conformations from distances matrices,” J. Comput. Chem., vol. 14, pp. 114–120, 1993.
W. Glunt, T.L. Hayden, and M. Raydan, “Preconditioners for distance matrix algorithms,” J. Comput. Chem., vol. 15, pp. 227–232, 1994.
L. Grippo, F. Lampariello, and S. Lucidi, “Anonmonotone line search technique for Newton's method,” SIAM J. Numer. Anal., vol. 23, pp. 707–716, 1986.
L. Grippo, F. Lampariello, and S. Lucidi, “A class of nonmonotone stabilization methods in unconstrained optimization,” Numer. Math., vol, 59, pp. 779–805, 1991.
D.C. Liu and J. Nocedal, “On the limited-memory BFGS method for large scale optimization,” Math. Programming, vol. 45, pp. 503–528, 1989.
W. Liu and Y. H. Dai, “Minimization algorithms based on supervisor and searcher cooperation,” J. Optimization Theory and Applications, vol. 111, pp. 359–379, 1989.
B. Molina and M. Raydan, “Preconditioned Barzilai-Borwein method for the numerical solution of partial differential equations,” Numerical Algorithms, vol. 13, pp. 45–60, 1996.
J.M. Ortega and W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press: San Diego, 1970.
M. Raydan, “On the Barzilai and Borwein choice of the steplength for the gradient method,” IMA J. Numer. Anal., vol. 13, pp. 618–622, 1993.
M. Raydan, “The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem,” SIAM J. Optim., vol. 7, pp. 26–33, 1997.
M. Raydan and B.F. Svaiter, “Relaxed steepest descent and Chauchy-Barzilai-Borwein method,” Computational Optimization and Applications, vol. 21, pp. 155–167, 2002.
D.F. Shanno and K.H. Phua, “Matrix conditioning and nonlinear optimization,” Math. Programming, vol. 14, pp. 149–160, 1978.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Grippo, L., Sciandrone, M. Nonmonotone Globalization Techniques for the Barzilai-Borwein Gradient Method. Computational Optimization and Applications 23, 143–169 (2002). https://doi.org/10.1023/A:1020587701058
Issue Date:
DOI: https://doi.org/10.1023/A:1020587701058