Abstract
In this paper, we first discuss how the nearly exact (NE) method proposed by Moré and Sorensen [14] for solving trust region (TR) subproblems can be modified to solve large-scale “low-rank” TR subproblems efficiently. Our modified algorithm completely avoids computation of Cholesky factorizations by instead relying primarily on the Sherman–Morrison–Woodbury formula for computing inverses of “diagonal plus low-rank” type matrices. We also implement a specific version of the modified log-barrier (MLB) algorithm proposed by Polyak [17] where the generated log-barrier subproblems are solved by a trust region method. The corresponding direction finding TR subproblems are of the low-rank type and are then solved by our modified NE method. We finally discuss the computational results of our implementation of the MLB method and its comparison with a version of LANCELOT [5] based on a collection extracted from CUTEr [12] of nonlinear programming problems with simple bound constraints.
Similar content being viewed by others
References
Breitfeld M.G., Shanno D.F. (1994) Preliminary computational experience with modified log-barrier functions for large-scale nonlinear programming. In: Hager W., Hearn D., Pardalos P. (eds) Large Scale Optimization: State of the Art. Kluwer, Dordrecht, pp. 45–67
Breitfeld M.G., Shanno D.F. (1996) Computational experience with penalty-barrier methods for nonlinear programming. Ann. Oper. Res. 62, 439–463
Bunch J.R., Parlett B.N. (1971) Direct methods for solving symmetric indefinite systems of linear equations. SIAM J. Numer. Anal. 8, 639–655
Cline A.K., Moler C.B., Stewart G.W., Wilkinson J.H. (1979) An estimate for the condition number of a matrix. SIAM J. Numer. Anal. 16, 368–375
Conn A.R., Gould N.I.M., Toint P.L. (1992) LANCELOT: A Fortran package for large-scale nonlinear optimization. Springer Series in Computational Mathematics 17. Springer, Berlin New York
Conn A.R., Gould N.I.M., Toint P.L. (2000) Trust-region methods. SIAM Publications, Philadelphia
Dennis J.E., Mei H.W. (1979) Two new unconstrained optimization algorithms which use function and gradient values. J. Optim. Theor. Appl. 28, 453–482
Fletcher R. (1980) Practical Methods of Optimization. Unconstrained Optimization 1. John Wiley, New York
Fortin C., Wolkowicz H. (2004) The trust region subproblem and semidefinite programming. Optim. Methods Softw. 19(1): 41–67
Gay D.M. (1981) Computing optimal locally constrained steps. SIAM J. Sci. Stat. Comput. 4(2): 186–197
Gould N.I.M., Lucidi S., Roma M., Toint P.L. (1999) Solving the trust-region subproblem using the Lanczos method. SIAM J. Optimization 9(2): 504–525
Gould, N.I.M., Orban, D., Toint, P.L.: General CUTEr documentation. Technical Report TR/PA/02/13, CERFACS, Toulouse, France (2003)
Hebden, M.D.: An algorithm for minimization using exact second derivatives. Atomic Energy Research Establishment, Report T.P.515, Harwell, England (1973)
Moré J.J., Sorensen D.C. (1983) Computing a trust region step. SIAM J. Sci. Stat. Comput. 4(3): 553–572
Moré J.J. (1983) Recent developments in algorithms and software for trust region methods. In: Bachem A., Grotschel M., Korte B. (eds) Mathematical Programming: the State of the Art, University of Bonn. Springer, Berlin Heidelberg New York, pp. 258–287
Nocedal J., Wright S.J. (1999) Numerical optimization. Springer, Berlin Heidelberg New York
Polyak R. (1992) Modified barrier functions (theory and methods). Math. Program. 54, 177–222
Powell M.J.D. (1970) A hybrid method for nonlinear equations. In: Rabinowitz P. (ed) Numerical Methods for Nonlinear Algebraic Equations. Gordon and Breach, New York
Reinsch C.H. (1971) Smoothing by spline functions II. Numer. Math.16, 451–454
Shultz G.A., Schnabel R.B., Byrd R.H. (1985) A family of trust-region-based algorithms for unconstrained minimization with strong global convergence properties. SIAM J. Numer. Anal. 22, 47–67
Sorensen D.C. (1982) Newton’s method with a model trust region modification. SIAM J. Numer. Anal. 19, 404–426
Steihaug T. (1983) The conjugate gradient method and trust regions in large-scale optimization. SIAM J. Numer. Anal. 20, 626–637
Toint, P.L.: Towards an efficient sparsity exploiting newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, Institute of Mathematics and its Applications Conference Series, xii+387 Academic Press, Inc. London (1981)
Wang Z.H., Wen Z.W., Yuan Y. (2004) A subspace trust region method for large scale unconstrained optimization. In: Yuan Y. (ed) Numerical Linear Algebra and Optimization, Proceeding of the 2003 International Conference on Numerical Optimization and Linear Algebra. Science Press, Beijing, pp. 265–274
Yuan, Y.: A Subspace Trust Region Algorithm. In: Presented at the Conference of Multiscale Optimization Methods and Applications, Center for Applied Optimization, University of Florida, USA, 26–28, February 2004
Zhang J., Xu C. (1999) A class of indefinite dogleg path methods for unconstrained minimization. SIAM J. Optim. 9(3): 646–667
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lu, Z., Monteiro, R.D.C. A modified nearly exact method for solving low-rank trust region subproblem. Math. Program. 109, 385–411 (2007). https://doi.org/10.1007/s10107-006-0025-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-006-0025-0
Keywords
- Nearly exact method
- Trust region method
- Large-scale optimization
- Limited-memory BFGS method
- Sherman–Morrison–Woodbury formula