The forward-backward-forward algorithm with extrapolation from the past and penalty scheme for solving monotone inclusion problems and applications

In this paper, we propose an improved iterative method for solving the monotone inclusion problem in the form of $0 \in Ax + Dx + N_{C}(x)$ in real Hilbert space, where $A$ is a maximally monotone operator, $D$ and $B$ are monotone and Lipschitz continuous, and $C$ is the nonempty set of zeros of the operator $B$. Our investigated method, called Tseng's forward-backward-forward with extrapolation from the past and penalty scheme, extends the one proposed by Bot and Csetnek [Set-Valued Var. Anal. 22: 313--331, 2014]. We investigate the weak ergodic and strong convergence (when $A$ is strongly monotone) of the iterates produced by our proposed scheme. We show that the algorithmic scheme can also be applied to minimax problems. Furthermore, we discuss how to apply the method to the inclusion problem involving a finite sum of compositions of linear continuous operators by using the product space approach and employ it for convex minimization. Finally, we present a numerical experiment in TV-based image inpainting to validate the proposed theoretical theorem.


Introdunction
In the last decade, penalty schemes have become a popular approach for studying constrained or hierarchical optimization problems in Hilbert spaces, particularly for solving complex problems (see [2,3,6,[8][9][10]14]).In this work, we intend to study the general monotone inclusion problem in the following form: where A : H ⇒ H is a maximally monotone operator, D : H → H a (single-valued) monotone and η −1 -Lipschitz continuous operator with η > 0, and N C : H ⇒ H is the normal cone of the closed convex set C ⊆ H which is the nonempty set of zeros of another operator B : H → H, which is a (single-valued) µ −1 -Lipschitz continuous operator with µ > 0.
The problem (1) is the generalized version of the monotone inclusion problem where C = arg min Ψ ̸ = ∅, and Ψ : H → R is a convex differentiable function with a Lipschitz gradient satisfying min Ψ = 0, which introduces a penalization function with respect to the constraint x ∈ C. Implicit and explicit discretized methods to solve the problem in (2) were proposed in [2,3].Several subsequent penalty type numerical algorithms in the literature are inspired by the continuous nonautonomous differential inclusion investigated in [1].
In case A is the convex subdifferential of a proper, convex, and lower semicontinuous function Φ : H → R, then any solution of (2) solves the convex minimization problem min x∈H {Φ(x) : x ∈ arg min Ψ}. ( Consequently, a solution to this convex minimization can be approximated by using the same iterative scheme as in [2,3].Futhermore, there are other methods for solving such optimization problem which are studied by several authors in the literature (see, for instance, [6,10,14,16]).Normally, for the iterative algorithms for solving both monotone inclusion problems (1) and (2) in penalty scheme, weak ergodic convergence is proved (and strong convergence under the strong monotonicity of A) (see [2,3,6,8,9]).In order to achieve the (ergodic) convergent results, the following hypotheses need to be assumed.
• For solving the problem (2) in case C = arg min Ψ ̸ = ∅ with the algorithm proposed by Attouch et al. [2,3], the Fenchel conjugate function of Ψ (namely, Ψ * ) needs to fulfill some key hypotheses in this context, as shown below: (H) A + N C is maximally monotone and zer(A + N C ) ̸ = ∅; (ii) For every p ∈ ran(N C ), n∈N λ n β n Ψ * p βn − σ C p βn where (λ n ) n∈N and (β n ) n∈N are sequences of positive real numbers.Note that the hypothesis (ii) of (H) is satisfied when n∈N λn βn < +∞ and Ψ is bounded below by a multiple of the square of distances to C, as described in [3].One such case is when C = zerL, where L : H → H is a linear continuous operator with closed range and Ψ : H → R is defined as Ψ(x) = 1  2 ∥Lx∥ 2 .Further situations in which condition (ii) is fulfilled can be found in [2, Section 4.1].
• For solving the problem (1) with a forward-backward type or Tseng's type (or forwardbackward-forward) algorithm proposed by Bot , et al. in [6,8,9], the required hypotheses involve the Fitzpatrick function associated to the maximally monotone operator B and reads as: (iii where (λ n ) n∈N and (β n ) n∈N are sequences of positive real numbers.Note that when B = ∂Ψ and Ψ(x) = 0 for all x ∈ C, then by (4) one can see that condition (ii) of (H) implies the second condition of (H f itz ), see [8,9].
Bot , and Csetnek [9] relaxed the cocoercivity of B and D to monotonicity and Lipschitzian.In this setting, a forward-backward-forward penalty type algorithm based on a method proposed by Tseng [23] is introduced.The convergence properties of this algorithm are studied under (H f itz ) and the condition lim sup n→+∞ λnβn µ + λn η < 1.In recent years, Tseng's forward-backward-forward algorithm has been modified by many researchers in various versions depending on the setting of the operators (see [7,15,18,19,22], and references therein).One such modification is the Tseng's forward-backward-forward algorithm with extrapolation from the past, which is developed by using Popov's idea [17] for the extragradient method with the extrapolated technique.This algorithm can store and reuse the extrapolated term in the next step of the iterative scheme, illustrated as Tseng's algorithm and y n = J γA (x n − γB(y n−1 )), where γ satisfies some suitable condition for each method and J A is the resolvent operator of A. According to this scheme, it has the potential to reduce computational costs and energy consumption in practical applications.Motivated by these considerations, we investigate the Tseng's forward-backward-forward algorithm with extrapolation from the past involving a penalty scheme under certain hypotheses for solving the inclusion problem in (1).We prove its weak ergodic convergence and further the strong convergence when the operator A is a strongly monotone operator in Section 3. Furthermore, we can extend our results in Section 3 to solve minimax problems as elaborated in Section 4. In Section 5, our proposed algorithm can be extended to solving more intricate problems involving the finite sum of composition of linear continuous operator with maximally monotone operators by using the product space approach, and the convergence results for our modified iterative method are also provided.
To broaden the applicability of our scheme, we also provide the iterative scheme and its convergence result for the convex minimization problem expressed in Section 6.Finally, we demonstrate a numerical experiment in TV-based image inpainting to ensure theoretical convergence results in Section 7.

Preliminaries
In this paper, we introduce some notations used throughout.We denote the set of positive integers by N and a real Hilbert space with an inner product ⟨•, •⟩ and the associated norm is said to converge weakly to x, if for any y ∈ H, ⟨x n , y⟩ → ⟨x, y⟩, and we use the symbols ⇀ and → to represent weak and strong convergence, respectively.For a linear continuous operator L : H → G, where G is another real Hilbert space, we define the norm of L as ∥L∥ = sup{∥Lx∥ : x ∈ H, ∥x∥ ≤ 1}.We also use L * : G → H to denote the adjoint operator of L, defined by ⟨L * y, x⟩ = ⟨y, Lx⟩ for all (x, y) ∈ H × G.
For a function f : H → R (where R := R ∪ {±∞} is the extended real line), we denote its effective domain by dom f = {x ∈ H : f (x) < +∞} and say that f is proper if dom f ̸ = ∅ and f (x) ̸ = −∞ for all x ∈ H. Let f * : H → R, where f * (u) = sup x∈H {⟨u, x⟩ − f (x)} for all u ∈ H, be the conjugate function of f .We denote by Γ(H) the family of proper, convex, and lower semicontinuous extended real-valued functions defined on H.The subdifferential of f at x ∈ H, with f (x) ∈ R, is the set ∂f (x) := {v ∈ H : f (y) ≥ f (x) + ⟨v, y − x⟩ ∀y ∈ H}.We take, by convention, ∂f (x) := ∅ if f (x) ∈ ±∞.
The indicator function of a nonempty set S ⊆ H, denoted by δ S : H → R, is defined as follows: δ S (x) = 0 if x ∈ S and δ S (x) = +∞ otherwise.The subdifferential of the indicator function is called the normal cone of S, denoted by N S (x).For x ∈ S, N S (x) = {u ∈ H : ⟨y − x, u⟩ ≤ 0 ∀y ∈ S}, and It is worth noting that for x ∈ S, u ∈ N S (x) if and only if σ S (u) = ⟨x, u⟩, where σ S is the support function of S. The support function is defined as σ S (u) = sup y∈S ⟨y, u⟩.A cone is a set that is closed under positive scaling, i.e., λx ∈ S for all x ∈ S and λ > 0. The normal cone, which is a tool used in optimization, is always a cone itself.
Let A be a set-valued operator mapping from a Hilbert space H to itself.We define the graph of A as GrA = {(x, u) ∈ H × H : u ∈ Ax}, the domain of A as dom A = {x ∈ H : Ax ̸ = ∅}, and the range of A as ran A = {u ∈ H : ∃x ∈ H s.t.u ∈ Ax}.The inverse operator of A, denoted by A −1 : H ⇒ H, is defined as (u, x) ∈ GrA −1 if and only if (x, u) ∈ GrA.We also define the set of zeros of A as zerA = {x ∈ H : 0 ∈ Ax}.The operator A is said to be monotone if ⟨x − y, u − v⟩ ≥ 0 for all (x, u), (y, v) ∈ GrA.Moreover, a monotone operator A is said to be maximally monotone if its graph on H × H has no proper monotone extension.If A is maximally monotone, then zerA is a closed and convex set, and a point z ∈ H belongs to zerA if and only if ⟨u − z, w⟩ ≥ 0 for all (u, w) ∈ GrA.Conditions ensuring that zerA is nonempty can be found in [4,Section 23.4].
In optimization, the proximal operator is a common tool used to develope algorithms for solving problems that involve a sum of a smooth function and a nonsmooth function.The proximal operator of a function f at a point x is defined as the unique minimizer of the function y → f (y) + 1 2 ∥y − x∥ 2 , denoted by prox f (x) : H → H.The resolvent of A, denoted by J A : H ⇒ H, is defined as J A = (Id + A) −1 where Id : H → H, Id(x) = x for all x ∈ H, is the identity operator on H.Moreover, if A is maximally monotone, then J A : H → H is single-valued and maximally monotone (see [4,Corollary 23.10]).Notice that J ∂f = (Id + ∂f ) −1 = prox f (see also [4,Proposition 16.34]).
Fitzpatrick introduced the notation we use in this paper in [13], and it has proved to be a useful tool for investigating the maximality of monotone operators using convex analysis techniques (see [4,5]).For a monotone operator A, we define its associated Fitzpatrick function φ A : H × H → R as follows: This function is convex and lower semicontinuous, and it will be an important tool for investigating convergence in this paper.
If A is a maximally monotone operator, then its Fitzpatrick function is proper and satisfies φ A (x, u) ≥ ⟨x, u⟩ for all (x, u) ∈ H × H, with equality if and only if (x, u) ∈ GrA.In particular, the subdifferential ∂f of any convex function f ∈ Γ(H) is a maximally monotone operator (cf.[21]), and we have (∂f ) −1 = ∂f * .Furthermore, we have the inequality (see [5]) where f * is the convex conjugate of f .Before we enter on a detailed analysis of the convergence result, we introduce some definitions of sequences and useful lemmas that will be used several times in the paper.Let (x n ) n∈N∪{0} be a sequence in H and (λ k ) k∈N∪{0} a sequence of positive numbers such that k∈N∪{0} λ k = +∞.Let (z n ) n∈N∪{0} be the sequence of weighted averages defined as shown in [3,8,9]: The following lemma is the Opial-Passty Lemma, which serves as a key tool for achieving the convergence results in our analysis.
Lemma 1 (Opial-Passty).Let T be a nonempty subset of real Hilbert space H and assume that lim n→∞ ∥x n − x∥ exists for every x ∈ T .If every weak cluster point of (x n ) n∈N (respectively (z n ) n∈N ) belongs to T , then (x n ) n∈N (respectively (z n ) n∈N ) converges weakly to an element in T as n → ∞.
Another key lemma underlying our work is presented below, which establishes the existence of convergence and the summability of sequences satisfying the conditions of the lemma.This lemma has been sourced from [2].
3 Forward-backward-forward with extrapolation and penalty schemes In this section, we will begin by presenting a problem introduced in [8,9], which can be formulated as follows: Problem 1: Let H be a real Hilbert space, A : H ⇒ H a maximally monotone operator, D : H → H a monotone and η −1 -Lipschitz continuous operator with η > 0, B : H → H a monotone and µ −1 -Lipschitz continuous operator with µ > 0 and suppose that C = zerB ̸ = ∅.
The monotone inclusion problem that we want to solve is Some methods to solve Problem 1 have been already studied in [8,9] by using Tseng's algorithm and penalty scheme.The iterative scheme presented below for solving Problem 1 draws inspiration from Tseng's forward-backward-forward algorithm with extrapolation from the past (see [7,15,22]).
Algorithm 1: where (λ n ) n∈N∪{0} and (β n ) n∈N∪{0} are sequences of positive real numbers.Of course, in order to demonstrate the (ergodic) convergence results we need to assume the hypotheses (H f itz ) for every n ∈ N ∪ {0}.Remark 3.
1.When the operator B satisfies Bx = 0 for all x ∈ H (which implies N C (x) = 0 for every x ∈ H), the algorithm outlined in Algorithm 1 reduces to the error-free scenario with the identity variable matrices of the forward-backward-forward with extrapolation method, as introduced in [22].
2. Actually, we can choose y −1 as any starting point in H.
Prior to establishing the weak ergodic convergence result, we shall prove the following lemma, which serves as a valuable tool for our main outcome.Lemma 4. Let (x n ) n∈N∪{0} and (y n ) n∈N∪{0,−1} be the sequence generated by Algorithm 1 and let be (u, w) ∈ Gr(A Then the following inequlity holds for n ∈ N ∪ {0}: Proof.From Algorithm 1 and the definition of the resolvent, we can derive that ∀n ∈ N Thus, we have xn−yn λn − β n B(y n−1 ) − D(y n−1 ) ∈ A(y n ) for ever n ∈ N. Since v ∈ Au, then we can guarantee by using the monotonicity of A that By using the definition of x n+1 , we obtain that ∀n ∈ N, Then, it follows from (11), we have that Because v = w − p − Du and the fact that D is monotone, we have that for every n ∈ N ) By the Lipschitz continuity of B and D, it yields It follows from (12) and above inequality that By Parallelogram law, we know that 2∥x n − So, we can derive from ( 14) that Therefore Subsequently, we state the convergence of Algorithm 1 below.(c) every weak cluster point of (z n ) n∈N∪{0} lies in zer(A + D + N C ).
Every step of proof can be showed as follows.
(a) Let u ∈ zer(A + D + N C ).For n ∈ N, we take w = 0 in (8), so we have where M n = λnβn µ + λn η .Rearranging the inequality, we have that By the hypothesis, we have that lim inf n→+∞ Summing up for n = n 0 + 1, . . ., K the inequalities in (8) (it is nothing else than (16) with the additional term 2λ n ⟨u − y n , w⟩) with lim sup n→+∞ λnβn µ + λn µ < 1  2 , for n ∈ N and M n = λnβn µ + λn η we get that This leads to the following inequality.
Discarding the nonnegative term ∥x K+1 − u∥ 2 and dividing 2τ where R := R + 2⟨− n 0 n=1 λ n u + n 0 n=1 λ n x, w⟩ ∈ R. By passing to the limit as K → +∞ and using that lim K→+∞ τ K = +∞, we get lim inf Since z is a weak cluster point of (z ′ n ) n∈N∪{0} , we obtain that ⟨u − z, w⟩ ≥ 0. Finally as this inequality holds for arbitrary (u, w) ∈ Gr(A n ∥ = 0 and the statement of the theorem will be consequence of Lemma 1. Taking u ∈ zer(A+D+N C ) and w = 0 = v+p+D(u) where v ∈ A(u) and p ∈ N C (u), it follows from the proof of (a) and the proof of Lemma 4 (see (13)) that n∈N∪{0} ∥y n−1 − y n ∥ 2 < +∞ and ∥x n+1 − y n ∥ ≤ λnβn µ + λn η ∥y n−1 − y n ∥, respectively.For any n ∈ N ∪ {0} it holds Since lim sup As observed in the forward-backward-forward penalty scheme discussed in [8,9], strong monotonicity of the operator A guarantees the strong convergence of the sequence (x n ) n∈N∪{0} .However, it remains to be seen whether this result extends to the forward-backward-forward with extrapolation from the past in the penalty scheme under consideration.The following result provides the necessary clarification.Theorem 6.Let (x n ) n∈N∪{0} and (y n ) n∈N∪{0,−1} be the sequences generated by Algorithm 1.
2 and the operator A is γ-strongly monotone with γ > 0, then (x n ) n∈N∪{0} converges strongly to the unique element in zer(A + D + N C ) as n → +∞.
Proof.Let u be the unique element of zer(A + D + N C ) and w = 0 = v + p + D(u), where v ∈ A(u) and p ∈ N C (u).We can follow the proof of Lemma 4 under A is γ-strongly monotone with γ > 0. This means that we obtain xn−yn λn − D(y n−1 ) − β n B(y n−1 ) ∈ A(y n ) (see ( 9)), and one simply show that for each n ∈ N The hypothesis lim sup implies the existence of n 0 ∈ N such that for every Then, the inequality can used for finite summation as: for some Since ĒN ≥ 0, then we can omit this term on the left-hand side of above inequality.Hence, we derive the following inequality: It follows that n∈N From ( 13), we can derive Since we have already known from Theorem 5 (a) that n∈N ∥y n −y n−1 ∥ 2 < +∞ and is bounded from above by the hypothesis, then It follows from ( 19), (20), and λ n is bounded from above that n∈N Because n∈N λ n = +∞ and lim

Minimax problems
The minimax optimization problem is a classical problem, where the goal is to find a saddle point for a function in two variables.This problem arises in many applications, including game theory, control theory, and robust optimization.In recent years, there has been a growing interest in the study of minimax problems due to their importance in machine learning and data analysis.
In this section, we will consider a class of minimax problems with a convex-concave structure, which can be solved using Tseng's forward-backward-forward method with extrapolation from the past and penalty scheme.We will present the problem formulation, discuss the properties of the objective function, and introduce some numerical methods for solving the problem.Let f : H × G → R be convex-concave and differentiable, where f (•, y) is convex for all y ∈ G and f (x, •) is concave for all x ∈ H.The Min-Max problem (or minimax problem) is the problem in the form: A saddle point of ( 21) is a vector (x, ȳ) fulfilling: Let , where ∇ 1 , ∇ 2 are the derivative of f with respect to the first and second component, respectively.Then, writing the optimal conditions for the inequalities in (22), we get Moreover, F is monotone and also is Lipschitz continuous (if we impose Lipschitz continuous properties on ∇ 1 f, ∇ 2 f ), see [4,Proposition 17.10] and [20,Theorem 35.1].In constrained case, the problem in (21) can be written as where X ⊂ H, Y ⊂ G are nonempty closed convex sets and a saddle point (x, ȳ) ∈ X × Y satisfies: The characterization of saddle points of (24) follows from the considerations: Further we consider more involved minimax problems with constrained sets described by linear equations: where Q, Q ′ are other real Hilbert spaces, This kind of the minimax constrained problem was studied with a plenty of applications, for instance in [24].In this case we have the following characteristic of saddle points of (27) [4,Corollary 16.38]).Then . (29) We find the solution of by considering the functions Namely, the solution of (30) can be obtained by solving the following optimization problem: For every x ∈ H and y ∈ G, let Therefore, (29) can be written as (33) Due to the above setting, the problem (29) turns into the monotone inclusion in (6).The iterative pattern in Algorithm 1 can be transformed into for each n ≥ 0 as where [4,Example 23.4]).Therefore we are able to construct a relevant algorithm for solving the monotone inclusion scheme in (33) (i.e., for solving problem ( 27)) demonstrated as: Algorithm 2: where (λ n ) n∈N∪{0} and (β n ) n∈N∪{0} are positive real numbers.
We show that we can use condition (ii) in (H) on page 1 in order to get the convergent results.We define Ψ : Then, we have (see [4,Proposition 13.27]) From the considerations above we have: Furthermore, we also have Thus, if we impose for i = 1, 2 then, it follows that exactly the condition (ii) on page 1 that we need for convergence in (H).Hence, we can propose the ergodic convergence results which follow from Thorem 5 as below.
1. Considering the condition (i) in (H min−max ), notice that Ā + N C is maximally monotone if a regularity condition is fulfilled, for example (see [4,Corollary 24.4]) In addition, (33) has a solution if the (Condition A) holds and (27) has saddle points.
3. We can show the Lipschitz property of B as follows: 5 The forward-backward-forward algorithm with extrapolation from the past and penalty scheme for the problem involving composion of linear continuous operators In this section, we propose forward-backward-forward algorithm with extrapolation from the past and penalty scheme utilized to address the inclusion problem involving the finite sum of the compositions of monotone operators with linear continuos operators.To this end, we begin with the following problem: Problem 3: Let H be a real Hilbert space, A : H ⇒ H a maximally monotone operator, D : H → H a monotone and ν-Lipschitz continuous operator with ν > 0. Let m be strictly positive integer and for any i ∈ {1, . . ., m}, let G i be a real Hilbert space, B i : G i ⇒ G i a maximally monotone operator, and L i : H → G i a nonzero linear continuous operator.Assume that B : H → H is a monotone and µ −1 -Lipschitz continuous operator with µ > 0 and suppose The monotone inclusion problem, Problem 3, can be reformulated into the same form as Problem 1 using the product space approach with a pertinent setting.To address this problem, we work with the product space H × G 1 × • • • × G m , where the inner product and associated norm are defined for every element ( We proceed to define the operators on the product space Note that the maximal monotonicity of A and B i , i = 1, . . ., m, implies that the operator Ã is also maximally monotone, as stated in [4,Proposition 20.22 and 20.23].Moreover, according to [12, Theorem 3.1], it is possible to demonstrate that the operator D is monotone and β-Lipschitz continuous where β = ν + m i=1 ∥L i ∥ 2 (in this context, the monotonicity and Lipschitzian of D is a special case of [8,12]).Moreover, we also have that B is monotone and µ −1 -Lipschitz continuous with and One can show (see [8]) that x is a solution to Problem 3 if and only if there exists ).This means that determining the zeros of Ã + D + N C will automatically provide a solution to Problem 3.
By using the identity given in [4,Proposition 23.16], which provides that for any ( )), it can be observed that the iterations of Algorithm 1 are given for any n ∈ N ∪ {0} as follows: This leads to the algorithm below: Algorithm 3: where (λ n ) n∈N∪{0} and (β n ) n∈N∪{0} are positive real numbers.To consider the convergence of this iterative scheme, we need the following additionally hypotheses, similar to the hypotheses in [8, Section 2]: The Fitzpatrick function of B (i.e., φ B ) and the support function of C (i.e., σ C ) can be computed in the same way as demonstrated in [8, Section 2] for arbitrary elements Moreover, the satisfaction of condition (i) in (H sum f itz ) guarantees that Ã + N C is maximally monotone and zer( Ã + D + N C ) ̸ = ∅.As a result, we can apply Theorem 5 and 6 to the problem of finding the zeros of Ã + D + N C .Thus, we can establish the convergence results for the scheme of the sum of monotone operators and linear continuous operators, which are given below.Theorem 9. Let (x n ) n∈N∪{0} and (z n ) n∈N∪{0} be sequences generated by Algorithm 3. Assume (H sum f itz ) is fulfilled and lim sup n→+∞ ( λnβn µ + λ n β) < 1 2 , where Then (z n ) n∈N∪{0} converges weakly to an element in zer(A+ m i=1 L * i B i L i +D +N C ) as n → ∞.If, additionally, A and B −1 i , i = 1, . . ., m are strongly monotone, then (x n ) n∈N∪{0} converges strongly to the unique element in zer(A Remark 10.In case m = 1, our considered problem will become a solver of the monotone inclusion problems involing compositions with linear continous operators suggested in [9, Section 3.2] corresponding to the context of the iterative method based on Tseng's forward-backwardforward algorithm with extrapolation from the past and penalty scheme.

Convex minimization problem
In this section, we apply the obtained results by using the forward-backward-forward algorithm with extrapolation from the past and penalty scheme for monotone inclusion problems to the minimization of a convex function with a complex formulation subject to the set of minimizers of another convex and differentiable function with a Lipschitz continuous gradient.We consider the convex minimization problem proposed in [8], given by: Problem 4: Let H be a real Hilbert space, f ∈ Γ(H) and h : H → R a convex and differentiable function with ν-Lipschitz continous gradient for ν > 0. Let m be a strictly positive integer and for any i = 1, . . ., m, let G i be a real Hilbert space, g i ∈ Γ(G i ), and L i : H → G i a nonzero linear continuous operator.Furthermore, let Ψ : H → R be convex and differentiable with a µ −1 -Lipschitz continuous gradient, fulfilling min Ψ = 0.The convex minimization problem under investigation is inf In order to solve this problem in our context, we can follow the Problem 3 in section 5. We khow that any element belonging to zer(∂f + m i=1 L * i ∂g i L i + ∇h + N C ) is an optimal solution for (40) if we substitute all of operators with A = ∂f, B = ∇Ψ, C = arg min Ψ = zerB, D = ∇h, and where B is a monotone and µ −1 -Lipschitz continuous operator (see [4, Note that the strong quasi-relative interior for a convex set S in a real Hilbert space H is denoted by sqri S and is defined as the set of points x ∈ S such that the union of all positive scalar multiples of (S −x) is a closed linear subspace of H. Notice that sqri S always contains the interior of S, denoted by int S, but this inclusion can be strict.When H is finite-dimensional, sqri S coincides with the relative interior of S, denoted by ri S, which is the interior of S with respect to its affine hull.Moreover, the condition in (42) is fulfilled if • H and G i are finite dimensional and there exists x ∈ ri dom f ∩ ri C such that L i x ∈ ri dom g i , i = 1, . . ., m (see [12,Proposition 4.3]).
At this point, we are equipped to state the iterative method ground on Tseng's forwardbackward-forward algorithm with extrapolation from the past and penalty scheme and its convergent result.
Algorithm 4: where (λ n ) n∈N∪{0} and (β n ) n∈N∪{0} are positive real numbers.To analyze the convergence theorem, we need to assume the hypotheses similar to those in [8, Section 3], which are as follows: ∂f + N C is maximally monotone and (40) has an optimal solution; (ii) For every p ∈ ran(N For the hypothesis (H opt ), some observations have already been presented in [8], which are as follows: Remark 11.
2. The condition (ii) of (H opt ) is similar to the condition (ii) of (H) which implies to the second contion of (H f itz ) as we mentined in the Section 1.Hence, we can apply our proposed convergent results in Section 3 for this context.
With the same effort as in Section 5, we may continue and provide the convergence results for Algorithm 4 as shown below.
2. If Ψ(x) = 0 for all x ∈ H, then Algorithm 4 reduces to the error-free version of the iterative scheme proposed in [22] where all of the variable matrices involved are identity matrices.This scheme is used to solve the convex minimization problem given by

A Numerical Experiment in TV-Based Image Inpainting
In this section, we demonstrate the application of Algorithm 4 for solving an image inpainting problem, which involves recovering lost information in an image.The computations presented in this part were carried out with Python (version 3.7.9) on a Windows desktop computer powered by an Intel(R) Core(TM) i5-8250U processor that operated at speeds between 1.6 GHz and 1.8 GHz, and was equipped with 8.00 GB of RAM.We represent images of size M × N as vectors x ∈ R n, where n = M • N , and each pixel denoted by x i,j , 1 ≤ i ≤ M, 1 ≤ j ≤ N , takes values in the closed interval from 0 (pure black) to 1 (pure white).Given an image b ∈ R n with missing pixels (which are set to black in this case), we define P ∈ R n×n as the diagonal matrix with P i,i = 0 if the i-th pixel in the noisy image b is missing, and P i,i = 1 otherwise, for i = 1, . . ., n (noting that ∥P ∥ = 1).The original image is reconstructed by solving the following TV-regularized model: The function T V iso : R n → R, which we use as our objective function, is defined as the isotropic total variation: It is possible to express the problem presented in (44) as an optimization problem of the form of Problem 4 in (40).This can be achieved by introducing the set Y = R n × R n, and defining the linear operator L : R n → Y as L(x i,j ) = (L 1 x i,j , L 2 x i,j ),where The operator L represents a discretization of the gradient in the horizontal and vertical directions.It is worth noting that ∥L∥ 2 < 8, and its adjoint L * : Y → R n is as easy to implement as L itself (see [11,22]).Furthermore, the inner product on Y, given by ⟨(y, z), (p, q)⟩ = M i=1 N j=1 (y i,j p i,j + z i,j q i,j ), induces a norm defined as ∥(y, z)∥ × = M i=1 N j=1 y 2 i,j + z 2 i,j for (y, z), (p, q) ∈ Y.It can be shown that T V iso (x) = ∥Lx∥ × for every x ∈ R n.
Moreover, by considering the function Ψ : R n → R, Ψ(x) = q 1,n = prox λng * 1 [v 1,n + λ n L(y n−1 )], x n+1 = λ n β n P (y n−1 −y n ) +λ n L * (q 1,n−1 − q 1,n )+y n , v 1,n+1 = λ n L(y n − y n−1 ) + q i,n , To implement this iterative method, we need the following formulas: prox γf (x) = proj [0,1] n (x) ∀γ > 0 and ∀x ∈ R n, and prox γg * 1 (p, q) = proj S (p, q) ∀γ > 0 and ∀(p, q) ∈ Y, where S = {(p, q) ∈ Y : max 1≤i≤M 1≤j≤N p 2 i,j + q 2 i,j ≤ 1}.The projection operator proj S : Y → S is defined via (p i,j , q i,j ) → (p i,j , q i,j ) max{1, p 2 i,j + q 2 i,j } , The quality of the reconstructed images was compared using the Improvement in Signal-to-Noise Ratio (ISNR), defined as follows: where x, b and x n denote the original, the image with missing pixels and the recovered image at iteration n, respectively.We evaluated the performance of the algorithm on a 256 × 256 pixel image of the Pisa Tower, using the parameters λ n = 0.9 • n −0.75 , λ n = 0.9 • (2 • n) −0.75 and β n = n 0.75 for all n ∈ N. The original image, as well as an image with 80% randomly blacked-out pixels, were used in the test.Figure 1 displays the original image, the noisy image, the non-averaged reconstructed image x n , and the averaged reconstructed image z n after 2000 iterations using both the Forward-Backward-Forward (FBF) in the penalty scheme with λ n = 0.9 • n −0.75 , FBF in penalty scheme with λ n = 0.9•(2•n) −0.75 and Forward-Backward-Forward with Extrapolation from the Past(FBF-EP) in penalty scheme with λ n = 0.9•(2•n) −0.75 .Note that λ n = 0.9•n −0.75 can not be used for FBF-EP in penalty scheme because there is no theorem to support its convergence.
Figure 2 and 3 depict the evolution of the ISNR values for both the averaged and non-averaged reconstructed images using FBF and FBF-EP algorithms (in both two values of λ n in FBF, respectively).The theoretical outcomes concerning the sequences involved in Theorem 12 are illustrated in the figure, indicating that the averaged sequence exhibits better convergence properties than the non-averaged sequence.We notice that the behavior of ISNR values in both algorithms carry out similarly when we use the same value of λ n and the ISNR values of averaged FBF-EP have a bit outperform at 2000 iteration, even though we choose a bigger λ n in FBF for comparison.Table 1 reveals that the FBF-EP approach produced the best averaged reconstructed image with an ISNR value of 11.35116, while it was 11.32596 for maximally monotone and zer(A + D + N C ) ̸ = ∅; (ii) For every p ∈ ran(N C ), n∈N λ n β n sup u∈C φ B u, p βn − σ C p βn < +∞;

Figure 2 :
Figure 2: The figure illustrates the ISNR curves for both the averaged and non-averaged reconstructed images using the FBF method with λ n = 0.9n −0.75 and FBF-EP with λ n = 0.9(2 • n) −0.75 method.

Figure 3 :
Figure 3: The figure illustrates the ISNR curves for both the averaged and non-averaged reconstructed images using the FBF and FBF-EP methods with the same value of λ n = 0.9(2•n) −0.75 .
Proposition 17.10, Theorem 18.15]).However, the converse is true only if a suitable qualification condition is satisfied.