Data and performance profiles applying an adaptive truncation criterion, within linesearch-based truncated Newton methods, in large scale nonconvex optimization

In this paper, we report data and experiments related to the research article entitled “An adaptive truncation criterion, for linesearch-based truncated Newton methods in large scale nonconvex optimization” by Caliciotti et al. [1]. In particular, in Caliciotti et al. [1], large scale unconstrained optimization problems are considered by applying linesearch-based truncated Newton methods. In this framework, a key point is the reduction of the number of inner iterations needed, at each outer iteration, to approximately solving the Newton equation. A novel adaptive truncation criterion is introduced in Caliciotti et al. [1] to this aim. Here, we report the details concerning numerical experiences over a commonly used test set, namely CUTEst (Gould et al., 2015) [2]. Moreover, comparisons are reported in terms of performance profiles (Dolan and Moré, 2002) [3], adopting different parameters settings. Finally, our linesearch-based scheme is compared with a renowned trust region method, namely TRON (Lin and Moré, 1999) [4].


Subject area
Operations Research and Management Science More specific subject area

Nonlinear Optimization
Type of data Output data illuminate the comparison between the linesearch and the trust region approaches, as globalization methods.

Data
Data from different experimental settings are reported, along with performance profiles, which highlight the advantages of adopting the proposal in [1]. The use of the performance profiles [3] is typically advised in the community of Nonlinear Optimization, since they clearly summarize in one plot the comparison among several codes over an entire test set. We obtain such profiles after filtering the test set from CUTEst collection, in order to guarantee a fair comparison among different codes. In particular, for any test problem, we state that a code fails in solving such a problem whenever (i) a given stopping criterion is not satisfied within 100,000 outer iterations, or (ii) if the CPU time exceeds 900 s. Moreover, in comparing any two algorithms, we consider only those problems where the algorithms converge to the same stationary point. This is checked by using the test (see [5] being f Ã 1 , f Ã 2 the optimal function values obtained by the two algorithms. Finally, we discarded all the test problems where the compared algorithms required a CPU time below 0.1 s to solve them.

Experimental design, materials and methods
In order to assess the Adaptive Truncation Criterion proposed in [1] (named ATC), we consider a standard implementation of a truncated Newton method, namely the linesearch-based truncated scheme described in [6]. Inner iterations are performed using the Conjugate Gradient (CG) method. The novel criterion ATC is adopted in order to avoid over solving of the Newton equation at each outer iteration. In the ATC scheme (see [1]) the maximum number of CG inner iterations allowed at k-th outer iteration (max_it k ) is initialized to n, and then adaptively adjusted according to ATC. As regards the parameters in the ATC scheme, we set This choice is suggested by a preliminary coarse tuning on the chosen test set. Moreover, since we tested ATC both within the unpreconditioned and the preconditioned framework proposed in [6], the value of the parameter l is set to 7, in order to allow the construction of an effective preconditioner (see also the discussion about the choice of the parameter h max in [6]).
The algorithms were coded in FORTRAN 90 and the GFortran compiler under Linux Ubuntu 14.04 was used. The stopping criterion for the outer iterations is the standard one given by where x k denotes the k-th iterate, g k indicates the gradient of the objective function at x k and ‖•‖ stands for the Euclidean norm.  As regards the set of test problems, we selected all the unconstrained convex and nonconvex large problems available in the CUTEst collection [2], and when a problem is of variable dimension, we considered two different dimensions (usually 1000 and 10,000 variables). The resulting test set consists in 112 problems.
As regards the stopping criterion for the CG inner iterations, we tested both the criteria reported in Section 2 of [1]: a) the residual-based criterion; b) the quadratic model reduction-based criterion.  Since the criterion a) with proved to yield poorer performance in practice, we preferred to use the more reliable residual-based criterion adopted in [6]. This criterion sets & which both takes into account the size n of the problem and allows a coarser solution when far from a stationary point. The criterion b) adopts η k ¼ 0:5; as suggested in [7].
In the sequel we adopt the following terminology:  ATC-true stands for algorithms which use the ATC scheme; ATC-false stands for algorithms which do not use the ATC scheme.

Choice of C k in the ATC scheme
Two different formulae were adopted for the parameter C k in [1]:  Figs. 1-3 report performance profiles of the comparison among schemes where our proposal is adopted, with the two choices (1) and (2) for C k .

Numerical comparisons among different truncated Newton schemes
Figs. 4-7 report performance profiles of the comparison between the two algorithmic choices ATCtrue vs. ATC-false, where the residual-based criterion a) is adopted in the unpreconditioned and preconditioned cases.
Figs. 8 and 9 refer to the comparison, in terms of CPU time, between the adoption of the residualbased criterion a) and the quadratic model reduction-based criterion b) in the algorithm which uses ACT in the unpreconditioned and preconditioned cases.

Comparison with a trust region approach
Figs. 10-12 report performance profiles of the comparison between our proposal of a truncated Newton method, where ATC is adopted (ATC-true), and the trust region-based code TRON [4]. Table 1 reports comparisons among the outputs of different versions of TRON and our proposals, on a selection of test problems.
The output data reported show how the use of the Adaptive Truncation Criterion proposed in [1], enables to efficiently address the problem of "over-solving" the Newton equation, within linesearchbased truncated Newton methods. The adoption of this criterion could have important implications for future implementations of such methods, for solving large scale unconstrained optimization problems. Indeed, it leads to a noticeable reduction of the CG inner iterations, that is significant computational savings of the overall computational burden.   With the stopping criterion With ATC-true and With ATC-false and jjg k jj r 10 −5 jjg k jj r 10 −5 max 1; jjx k jj È É jjg k jj r 10 −5 max 1; jjx k jj È É jjg k jj r 10 −5 max 1; jjx k jj È É