Image Noise Reduction and Solution of Unconstrained Minimization Problems via New Conjugate Gradient Methods

: The conjugate gradient (CG) directions are among the important components of the CG algorithms. These directions have proven their effectiveness in many applications—more specifically, in image processing due to their low memory requirements. In this study, we derived a new conjugate gradient coefficient based on the famous quadratic model. The derived algorithm is distinguished by its global convergence and essential descent properties, ensuring robust performance across diverse scenarios. Extensive numerical testing on image restoration and unconstrained optimization problems have demonstrated that the new formulas significantly outperform existing methods. Specifically, the proposed conjugate gradient scheme has shown superior performance compared to the traditional Fletcher–Reeves (FR) conjugate gradient method. This advancement not only enhances computational efficiency on unconstrained optimization problems, but also improves the accuracy and quality of image restoration, making it a highly valuable tool in the field of computational imaging and optimization.


Introduction
Many real-world applications involve nonlinear optimization problems, particularly, problems of extremely enormous dimensions, and thus require the use of first-order schemes to obtain their solutions.Conjugate gradient (CG) methods, the most popular first-order approaches, have been widely demonstrated to be useful in handling challenging unconstrained and constrained problems, such as problems arising from imaging processing.This is due to the efficiency of the CG algorithm in handling sparse and large-scale systems, which are common in such tasks.These methods are iterative and thus allow for early termination criteria to save time, and can be adapted for nonlinear functions, making their algorithms versatile.In addition, CG schemes efficiently handle minimization problems with regularization, scale well with the size of the problem, and can control parallel processing capabilities, improving their efficacy and speed for high-resolution image processing tasks.
In [1], the authors propose a two-phase strategy that combines the advantages of the adaptive median filter with the variational method in a single approach.The adaptive median filter [2] is used in the initial phase of the processing for salt-and-pepper noise.In such methods, X represents the real image and A = {1, 2, 3, ..., M} × {1, 2, 3, ..., N} represents the index set of X and N ⊂ A denotes the set of indices of the noise pixels that were detected during the first phase of the analysis process.Thus, the issue is determining an effective method for minimizing the function as follows: where ζ is the regularization parameter, S 1 i,j = 2∑ (m,n)∈P i,j ∩N c ϕ α (u i,j − y m,n ), S 2 i,j = ∑ (m,n)∈P i,j ∩N ϕ α (x i,j − y m,n ) are the edge-preserving potential functions, and ϕ α = √ α + x 2 , α > 0 is the edge-preserving potential function.P i,j denotes the set of the four closest neighbors of the pixel at the location (i, j) ∈ A, and y i,j denotes the observed pixel value of the image at the position (i, j), with x i,j = x i,j (i,j)∈N in each row denoting a column vector c of one length sorted lexicographically after the other.
It has been demonstrated in [1,2] that the term x i,j − y i,j in (1) allows for the detection of noisy pixels, but it also adds a tiny bias in the restoration of corrupted pixels when used in conjunction with other techniques.Our method begins by detecting the set of all noisy pixels, which occurs in its initial phase.As a result, this phrase is no longer necessary during the restoration phase.This recommends that we remove it from consideration (1).Consequently, we can only analyze the functional aspects of the following: In this study, we are more interested in investigating the performance of a new Conjugate gradient method on image restoration and unconstrained optimization problem of the form: where f is continuously differentiable (see [3][4][5]).The CG algorithm generates a sequence of iterative points via [6]: where α k is a step length and the search direction d k+1 is generated as: For further information on the possible choices of the conjugate coefficient β k , see [7,8].In general, the global convergence characteristics of CG techniques are widely studied.
According to [9], the Fletcher-Reeves (FR) method has been identified as having the best convergence results, while the Hestenes-Stiefel (HS) method has been recognized as one of the most efficient CG methods with good numerical performance but failing to satisfy the global convergence properties under classical line search conditions [10].The particular choices for the FR and HS methods are: with y k = g k+1 − g k .The presentations in [11][12][13][14] provide excellent references to contemporary CG approaches that have yielded significant outcomes, in comparison to those in (6).Because the Hestenes-Stiefel formula meets the conjugacy criterion, it is particularly appealing to require that (see [15]) be satisfied when new methods are developed.The derivations in [16,17] represent an outstanding summary of the evolution of multiple variants of nonlinear conjugate gradient methods with a specific focus on global convergence qualities, as described in [17].In theory, based on Perry's conjugacy criterion (7), it may be rewritten as: In [14], the Hestenes-Stiefel (HS) method has been modified to produce: As revealed by the numerical findings, the Wu and Chen [17] approach numerically supersedes the HS method [10].
In [18], it is shown that for quadratic functions, the step size is determined exactly as: For non-quadratic problems, classical line searches, such as cubic interpolation, are employed to find a step size α k along a generated search direction.For convergence purposes, α k is usually required to satisfy the strong Wolfe-Powell (SWP) [19] line search conditions: where 0 < δ < σ < 1 [20,21].Such conditions are particularly beneficial in examining the convergence properties of CG methods.For a more recent study on the CG method, see [22][23][24][25][26][27].
Next, a new CG conjugacy parameter is developed by employing a quadratic model.This is followed by an analysis of the new method's convergence properties.The new derivation aims to further the numerical behavior of CG methods.

Deriving the New Parameter Based on the Quadratic Model
In this section, we present the derivation process of the new conjugate gradient formulas.The algorithm used for the computational experiment is further presented at the end of this section.The motivation for constructing novel conjugate gradient parameters via the quadratic model is to enhance the convergence rate and accuracy of the CG algorithm.By leveraging the second-order curvature and presenting modified formulas, the updated algorithm can provide improved direction and step length adjustments, and this would lead to more effective minimization, particularly in large-scale and complex problems in image restoration and unconstrained optimization, which need robust optimization procedures.The new CG parameter is derived using the quadratic model: where the corresponding gradient is given as: The second-order curvature is derived from ( 12) and ( 13) to obtain: Using the above equation to obtain: where Using ( 8) and (15) in the above equation, we obtain: Since d k+1 = −g k+1 + β k s k , this implies that: which yields: Additionally, for exact line search, ( 16) leads to two suggested expressions as follows: and We refer to the three alternatives of ω k as BT1, BT2, and BT3, respectively, as indicated in ( 16), (20), and (21).Introducing multiple forms of ω k , including BT1, BT2, and BT3, offers flexibility in selecting the most suitable parameter for different problem contexts, improving the approach's performance and adaptability.
The algorithmic steps (Algorithm 1) for the derived method are summarized as: Algorithm 1.The new conjugate gradient algorithm for minimizing.

Convergence Analysis of the Uniformly Convex Function
The global convergence analysis for the derived methods is considered in this section.The following assumptions are needed: Assumption 2. The gradient g is Lipschitz continuous, i.e., there exists a non-negative steady L such that ∥g(u Under these assumptions on the objective function, there exists a steady Γ ≥ 0, such that ∥∇ f (x)∥ ≤ Γ.More details can be found in [28,29].
We start by proving the descent property for the new algorithm in the following theorem.
Using the condition of Lemma 1, we can prove the following result.
Theorem 2. Assume that f is a uniformly convex function on R n , i.e., there exists a stable µ > 0, that satisfies: lim k→∞ If the conditions in Lemma 1 hold, then: Proof of Theorem 2: The proof is similar to the ones in [20,21].□

Numerical Results
In this section, numerical data is presented to demonstrate the efficiency of the BT1, BT2, and BT3 algorithms in reducing salt-and-pepper impulse noise by lowering the threshold in (3) and further solving unconstrained optimization problems.The parameters chosen for the line search in (11) for the BT1, BT2, and BT3 procedures are δ = 0.0001 and σ = 0.5.All simulations are conducted on a PC with MATLAB 2015a.The BT1, BT2, and BT3 techniques are compared to the FR method in terms of performance efficiency.It is vital to emphasize that the speed at which the obtained decrease in (3) is the primary focus of this research.The Signal-to-Noise Ratio is used to evaluate the quality of the recovered image: where u r i,j and u * i,j denote the pixel values of the restored image and the original image, respectively.The stopping criteria for both techniques are as follows: Table 1 reports the computed PSNR (peak signal-to-noise ratio), in addition to the iterations count (NI) and function evaluations (NF) for each of the tested methods, as opposed to the standard FR method.As demonstrated in Table 1, the BT1, BT2, and BT3 are more efficient as they require the fewest iterations and function evaluations compared to the FR method.Furthermore, the PSNR values generated by all three new approaches are superior.The restoration results achieved utilizing the FR, BT1, BT2, and BT3 algorithms are shown in Figures 1-4.These findings demonstrate that the recommended image-correcting procedures BT1, BT2, and BT3 are reliable and effective.As demonstrated in Table 1, the BT1, BT2, and BT3 are more efficient as they require the fewest iterations and function evaluations compared to the FR method.Furthermore, the PSNR values generated by all three new approaches are superior.The restoration results achieved utilizing the FR, BT1, BT2, and BT3 algorithms are shown in Figures 1-4.These findings demonstrate that the recommended image-correcting procedures BT1, BT2, and BT3 are reliable and effective.Next, the numerical performance of the proposed method was evaluated on a total of twenty-three unconstrained optimization problems under the strong Wolfe line search.Three metrics including the number of iterations (NOI), the number of function evaluation (NOF), and computational time (CPUT) are used to measure the efficiency of all algorithms.The termination condition was also set as: i.
If ‖ ‖ < 10 was not satisfied.Next, the numerical performance of the proposed method was evaluated on a total of twenty-three unconstrained optimization problems under the strong Wolfe line search.Three metrics including the number of iterations (NOI), the number of function evaluation (NOF), and computational time (CPUT) are used to measure the efficiency of all algorithms.The termination condition was also set as: i.
Table 2 presents a detailed performance of all the algorithms.The symbol (***) is used to denote the point where an algorithm fails to satisfy the above conditions.The symbol (##) means a failure to converge to a solution.
The above results are further analyzed using the performance profile tool introduced by Dolan and More [30].The following figures present the visual illustration of the results from Table 2.
As seen in Figures 5 and 6, a percent P(τ) of the problems for which the technique is within a factor τ of the optimal time is displayed for each method.A method's quickest percentage of test problems is indicated by the vertical axis on the left side of the curves.The figures indicate that the proposed algorithms outperformed the classical FR algorithm based on the number of iterations and function evaluation.This shows that the new methods are ranked out of the FR method, thus competing favorably with the existing method.The figures indicate that the proposed algorithms outperformed the classical FR algorithm based on the number of iterations and function evaluation.This shows that the new methods are ranked out of the FR method, thus competing favorably with the existing method.The figures indicate that the proposed algorithms outperformed the classical FR algorithm based on the number of iterations and function evaluation.This shows that the new methods are ranked out of the FR method, thus competing favorably with the existing method.

Conclusions
In conclusion, we have developed a new modified conjugate gradient formula and introduced three new conjugate gradient methods, BT1, BT2, and BT3, which propose different options for the conjugacy parameter.The new choices are designed to enhance image processing tasks, particularly in image restoration applications.By employing Wolfe line search conditions, we established the global convergence properties of these new methods.Our comprehensive simulation studies demonstrated that the BT1, BT2, and BT3 methods significantly reduce the number of iterations and function evaluations required, thus improving the computational efficiency of the methods on unconstrained optimization problems.Moreover, these methods were further shown to effectively restore image quality, surpassing the performance of the traditional conjugate gradient method.
The results highlight the potential of BT1, BT2, and BT3 methods to advance the field of image processing.Their ability to achieve higher accuracy with less computational effort makes them valuable tools for practitioners and researchers equally.Future work will focus on further optimizing these methods and exploring their application to a broader range of image processing challenges.

Figure 5 .
Figure 5. Performance profile based on NOI.

Figure 6 .
Figure 6.Performance profile based on NOF.

Figure 5 .
Figure 5. Performance profile based on NOI.

Figure 5 .
Figure 5. Performance profile based on NOI.

Figure 6 .
Figure 6.Performance profile based on NOF.

Figure 6 .
Figure 6.Performance profile based on NOF.

Table 2 .
Performance comparison based on NOI, NOF, and CPU time.