Inverse Scale Space Decomposition

We investigate the inverse scale space flow as a decomposition method for decomposing data into generalised singular vectors. We show that the inverse scale space flow, based on convex and absolutely one-homogeneous regularisation functionals, can decompose data represented by the application of a forward operator to a linear combination of generalised singular vectors into its individual singular vectors. We verify that for this decomposition to hold true, two additional conditions on the singular vectors are sufficient: orthogonality in the data space and inclusion of partial sums of the subgradients of the singular vectors in the subdifferential of the regularisation functional at zero. We also address the converse question of when the inverse scale space flow returns a generalised singular vector given that the initial data is arbitrary (and therefore not necessarily in the range of the forward operator). We prove that the inverse scale space flow is guaranteed to return a singular vector if the data satisfies a novel dual singular vector condition. We conclude the paper with numerical results that validate the theoretical results and that demonstrate the importance of the additional conditions required to guarantee the decomposition result.


Introduction
Regularisation methods are essential tools for the stable approximation of solutions of ill-posed inverse problems. Hence, their analysis is a vital part of inverse problems research. For linear regularisation methods a quite complete theory based on singular value decomposition has been available for a while (cf. [1,2]). Only recently though a more general definition of singular vectors for non-linear regularisation methods has been formalised in [3], despite many previous and recent works on generalised Eigenfunctions in the context of non-linear partial differential equations and functional inequalities (cf. [4,5,6,7,8,9,10,11,12,13,14]), control theory (cf. [15]), image processing (cf. [16,17,18,19,20,21,22]), and machine learning (cf. [23,24]). The generalised singular vector definition is based on the minimisation of convex, one-homogeneous but not necessarily differentiable functionals. Classical non-linear regularisation methods for which generalised singular vectors apply include the famous total variation (TV) regularisation model [25] and 1 -norm regularisation, which is a key tool in compressed sensing (cf. [26,27,28,29]). Newer advancements include the total generalised variation (TGV, cf. [30,31,32,33,34]), regularisations based on the 1 -norm in combination with the shearlet transform [35], vectorial total variation regularisation (cf. [36,37,38]), structure-tensor extensions of the total variation (cf. [39,40,41,42,43]), vector-field regularisation [44], de-biased total variation-type regularisations [45] and regularisations based on the 1 -norm in combination with tight wavelet frames [46,47,48], to name just a few. It has been shown that generalised singular vectors play a vital role in the understanding of those non-linear regularisation methods similar to the role of classical singular vectors for linear regularisations. Key results are that the non-linear regularisation methods are capable of recovering individual singular vectors from data given in terms of a forward model applied to a generalised singular vector and additional measurement errors, just with a systematic bias. Considering corresponding inverse scale space methods (cf. [49]) as regularisation methods instead, these methods are capable of recovering generalised singular vectors without systematic bias (cf. [3]). The latter also has significant impact on discretisations of the inverse scale space method, such as Bregman iteration (cf. [50,51]), linearised Bregman iterations (cf. [52,53,54,55]) and modifications (cf. [56,57]).
In [58] Gilboa initiated the idea of a non-linear spectral decomposition of singular vectors by defining a spectrum based on the forward-scale space formulation of the TV model. He was able to show that this non-linear spectrum can be converted into the original signal via a linear inverse transform. In [59] he made the connection that a single generalised singular vector (called Eigenfunction therein) of TV can be isolated via this spectral TV transform. In [60] the idea of the non-linear spectral transform has further been extended to arbitrary one-homogeneous functionals, and to variational regularisation as well as the inverse scale space method instead of just the forward TV flow.
The definition of the non-linear spectral transform immediately gave rise to the question of circumstances under which it would be possible to decompose a signal composed of singular vectors. In [61] this question has briefly been addressed, hinting that some sort of orthogonality condition is necessary for perfect separation (see [61,Proposition 1]). These initial ideas have been made more precise in [62]; given two singular vectors that are (fully orthogonal [62,Definition 5] and) linear in the subdifferential [62,Definition 7], the non-linear spectral transform can separate these singular vectors perfectly (see [62,Proposition 4]). In [63] it has further been shown that the non-linear spectral transform can indeed decompose data into a finite set of singular vectors, given that the corresponding regularisation functional is a one-norm concatenated with a linear matrix such that the matrix applied to its transpose is diagonally dominant (see [63,Theorem 9]).
Despite containing impressive results, we however also observe limitations of the previously mentioned works. First of all, all research is restricted to the non-linear spectral transform, which (in its current form) is not applicable to general inverse problems. Secondly, we either have theoretical conditions that allow us to separate only two singular vectors for general one-homogeneous regularisation functionals, or we have conditions for a specific family of regularisers that allow the decomposition of data into multiple singular vectors. With this work we want to fill in the knowledge gaps by introducing two conditions that guarantee the decomposition of sums of finitely many singular vectors via the inverse scale space method. The first condition will be an orthogonality condition on the finite set of singular vectors, while the second condition will ensure that partial sums of their subgradients are contained in the subdifferential of the corresponding regularisation functional at zero. Our main contribution will be a theorem which states that perfect decomposition of a finite linear combination of generalised singular vectors is guaranteed if these two conditions are satisfied. Subsequently, we are also going to investigate the case of arbitrary data that is not given in terms of a finite linear combination of generalised singular vectors.
The paper is organised as follows. First, we are going to introduce notation and mathematical preliminaries that are necessary throughout this paper. Then we formulate and discuss the major contribution of this work which is a result that states that finite sums of generalised singular vectors can be decomposed via the inverse scale space method under suitable conditions. Subsequently we also address the question of how to characterise the first inverse scale space step in case the given data is arbitrary. In this case we investigate under which conditions the first step is a generalised singular vector. Finally, we will present numerical results to support the theoretical results and conclude with an outlook of open questions and problems.

Mathematical Preliminaries
In this section we set the notation, define assumptions and give mathematical preliminaries. Throughout the paper we consider inverse problems of the form where K : U → H is a bounded linear operator from a Banach space U to a Hilbert space H. We will denote the space of bounded linear operators between two normed spaces U and V by L(U, V). We denote by U * the dual Banach space of U with norm where functional p(u) = p, u U * × U is the dual pairing between p and u that we will abbreviate by p, u throughout the course of this work. If U is a Hilbert space then the dual pairing can be identified with the inner product on U.
A popular variational framework for approximating solutions of (2.1) is the Tikhonovtype variational regularisation framework of the form u ∈ arg min u∈dom(J) where J : dom(J) ⊆ U → R ∪ {+∞} is a regularisation functional that incorporates prior information aboutû and α > 0 is a regularisation parameter that controls the impact of J onû.
Throughout this paper we assume that the regularisation functional J is proper, lower semi-continuous (l.s.c) and convex. We will further assume that J is absolutely onehomogeneous, i.e.
Such a functional is non-negative by definition and fulfils the triangle inequality see [63,Lemma 3.2]. The subdifferential ∂J(u) of J at some element u ∈ dom(J) will be used several times throughout this work and is defined by Since J is absolutely one-homogeneous the subdifferential can be characterised by see for instance [64,Lemma 3.12]. As a special case we have since J(0) = 0. We observe that the subdifferential for absolutely one-homogenous functionals is invariant with respect to positive scaling, i.e.
We also want to emphasise that for a subgradient p ∈ ∂J(u) we know by the characterisation (2.6) that p, v ≤ J(v) for all v ∈ dom(J). Now take v ∈ ker(J). Then p, v ≤ 0. Using −v we then get p, v ≥ 0. This shows that for absolutely one-homogeneous functionals: In [50] a contrast-enhancing alternative to (2.2) named Bregman iteration has been introduced. This is an iterative regularisation method for which Note that the generalised Bregman distance is always non-negative due to the convexity of J. Since the Bregman iteration has a semi-convergence behaviour a stopping criterium for the iteration is needed.
In the limit α → ∞ we can interpret ∆t = 1/α as a time step that tends to zero and the update (2.9) of p k+1 for the Bregman iteration can be interpreted as a forward Euler discretisation of the inverse scale space (ISS) flow (cf. [49]) with p(0) = 0 and u(0) = u 0 ∈ ker(J). If we assume u(0) = 0 ∈ ker(J), from the definition of the ISS flow we then obtain p(t) = tK * f for 0 ≤ t < t 1 for some t 1 > 0. Hence we would require p(t) = tK * f ∈ ∂J(0) for 0 ≤ t < t 1 , which by (2.8) implies K * f, v = 0 for all v ∈ ker(J). If this is not the case we can project K * f onto the space of elements p ∈ U * for which p, v = 0 for all v ∈ ker(J). In the end we can always add u 0 ∈ ker(J) to u(t). For the remainder of this work we therefore assume w.l.o.g. that u(0) = u 0 = 0 for the initial value of the ISS flow.
We want to point out that the solution u(t) for the ISS flow has to satisfy an orthogonality condition: Proposition 2.1 (Necessary Orthogonality Condition). Let u(t) be a solution of the inverse scale space flow (2.10). Then u(t) satisfies the orthogonality condition Proof. Using the chain rule for differentiation and p(t) ∈ ∂J(u(t)) we get On the other hand, since p(t) ∈ ∂J(u(t)) and by the absolute one-homogeneity of J we get Hence we see that a solution u(t) of (2.10) needs to satisfy From the definition of the inverse scale space flow we know ∂ t p(t) = K * (f − Ku(t)) and hence If we ignore the one-homogeneity of J for a moment it becomes evident that the ISS flow is a generalisation of Showalter's method [67], i.e.
where Ω ⊂ R m is some domain. It is well known that for compact K between Hilbert spaces, solutions of (2.12) -as those of other linear regularisation methods -can conveniently be expressed in terms of K's system of classical singular vectors. That is, for two orthonormal bases {v j } j∈N of R(K) and {u j } j∈N of R(K * ), where R(K) is the range of K and R(K * ) is the range of K * , and a null sequence {σ j } j∈N satisfying If we now assume that our data is given in terms of some linear combination of finitely many singular vectors, i.e. we have f = Ku † for where γ j ∈ R and n ∈ N, then (2.13) simplifies to Hence, Showalter's method allows us to recover a weighted linear combination of the singular vectors of u † from data f = Ku † .
One of our main goals throughout the course of this work is to study solutions of the ISS flow (2.10). With the example of Showalter's method we have seen that for the specific choice of J(u) = 1 2 u 2 L 2 (Ω) solutions of (2.12) are just simple transformations of the classical singular vectors if the data f is represented by a composition of those. However, as we are interested in (2.10) in combination with absolutely onehomogeneous regularisation functionals, classical singular vectors are of no use for our study. A remedy to overcome this issue seems to be the use of so-called generalised singular vectors as defined in [3]. Generalised singular vectors are based on the idea of minimising the generalised Rayleigh quotient (2.14) Here u g denotes the so-called non-trivial ground state of the functional J, following [4], for which we have J(u g ) ≤ J(u) for all u ∈ dom(J) with Ku H = 1. The set ker(J) ⊥ is the orthogonal complement of ker(J) for which we follow the definition of [3] and define ker(J) ⊥ = {u ∈ dom(J) : Ku, Kv = 0, ∀v ∈ ker(J)}.
Note that we have to ensure ker(K) ∩ ker(J) = {0} for this definition to make sense. Otherwise every element u ∈ ker(K) ∩ ker(J) would both satisfy u ∈ ker(J) and u ∈ ker(J) ⊥ .
We want to highlight that we require the normalisation constraint in (2.14); otherwise every function cu g with c ∈ R \ {0} is a solution of (2.14) given that u g is already a solution of (2.14). We could then make c arbitrarily close to zero to decrease J(cu g ) = |c|J(u g ) due to the absolute one-homogeneity of J.
Now let us reformulate (2.14) in terms of the Lagrange saddle-point problem with the corresponding Lagrange multipliers µ g and η g , for all v ∈ ker(J). We want to investigate the optimality condition of (2.15) with respect to u g more closely, i.e.
Computing the dual pairing of (2.16) with u g therefore yields the relation , which implies µ g = 0. If we further compute the dual product of (2.16) with v ∈ ker(J) we observe where p g , v = 0 for p g ∈ ∂J(u g ) due to (2.8). Hence, unless the kernel of J is trivial, we immediately observe η g = 0.
Investigating (2.16) for µ g = 0, η g = 0 and Ku g H = 0 leaves us with the condition λ g K * Ku g ∈ ∂J(u g ) , (2.17) for λ g := J(u g )/ Ku g 2 H and p g ∈ ∂J(u g ). Using the normalisation Ku g H = 1 and due to the absolute one-homogeneity (2.3) we conclude λ g = p g , u g = J(u g ).
Equation (2.17) now characterises the ground state that solves (2.14). However, there may exist other functions satisfying A function u λ that satisfies (2.18) and (2.19) is called a generalised singular vector with generalised singular value λ. We normally assume Ku λ H = Ku g H = 1.
Throughout the course of this work we will use the term singular vectors for generalised singular vectors as well. Note that the term vector does not refer to an element in R m but to an element in a possibly infinite dimensional space. Additionally, note that if u λ is a singular vector then also −u λ is a singular vector for absolutely one-homogeneous J.
Example 2.2. We want to briefly demonstrate that the concept of generalised singular vectors (2.18) is indeed a generalisation of classical singular vectors in inverse problems. If we consider the absolutely one-homogeneous regularisation functional J(u) = u L 2 (Ω) for U = L 2 (Ω) on some domain Ω ⊂ R m we obtain ∂J(u) = {u/ u L 2 (Ω) } for u with u L 2 (Ω) = 0. Hence, (2.18) reads as H . If we multiply this equation by 1/λ and define σ = Ku λ H / u λ L 2 (Ω) , we therefore obtain the classical singular vector condition which is simply the Eigenvalue problem of the Laplace operator.
Example 2.4. Let K = I : 1 (R m ) → 2 (R m ) be an embedding operator, and let Then every vector u λ ∈ R m making W u λ consist of peaks of the same magnitude is a singular vector: We need to show . If c is the magnitude of the peaks of W u λ and n is the number of peaks then If we further want u λ to be normalised we need c = ± 1 √ n . The singular value λ then simplifies to λ = √ n.
Example 2.5. Let J(u) = TV * (u) be the slightly modified total variation regularisation as defined in [3, Section 4.1]: where Ω ⊆ R m is a bounded domain. In contrast to the normal TV-functional we do not choose test functions ϕ with compact support, which yields an additional boundary term. For functions u ∈ W 1,1 be an embedding operator. It is then shown in [3, Theorem 2] that the Haar wavelet basis is an orthonormal set of singular vectors for K and J. Examples are for instance the one-dimensional Haar wavelets To see why generalised singular vectors are interesting in the context of inverse scale space regularisation, we recall the following theorem from [3]: Theorem 2.6 ([3, Theorem 9]). Let J be proper, convex, l.s.c and absolutely onehomogeneous and let u λ be a generalised singular vector with corresponding singular value λ. Then, if the data f is given by f = γKu λ for a positive constant γ, a solution of the inverse scale space flow (2.10) is given by Hence, the ISS flow (2.10) is able to recover u λ from data f = Ku λ with no systematic bias. In other words, generalised singular vectors are invariant under inverse scale space regularisation. This makes generalised singular vectors useful tools for the analysis of the ISS flow.
A second result that we want to recall is the following result in case of data f = γKu λ + g that can additively be decomposed into a singular vector and an arbitrary element g ∈ H.
Theorem 2.7 ([3, Theorem 10]). Let J be proper, convex, l.s.c and absolutely onehomogeneous and let u λ be a generalised singular vector with corresponding singular value λ. Then, if the data f is given by f = γKu λ + g for a positive constant γ, such that there exist constants µ and η satisfying with γ > µ/η, a solution of the inverse scale space flow (2.10) is given by The original rationale behind Theorem 2.7 was to allow for deterministic noise in the data, whilst showing that the singular vector can still be recovered. However, the additional term g can obviously also be interpreted as a correction term for the singular vector u λ to match data f .
In fact, (2.20) is nothing but a specific type of source condition. For a given function u † , the source condition is defined as It ensures the existence of a source element v ∈ H \{0} such that is satisfied. We immediately see that (2.20) is equivalent to (SC) for the specific choice v = µKu λ + ηg and u † = u λ .
The (generalised) strong source condition as introduced in [68], in contrast to (SC), is defined as which guarantees the existence of a source element w ∈ U \{0} with We want to highlight the following connections between the singular vector condition (2.18) and the two source conditions (SC) and (SSC). Starting with (SSC) it is obvious that any singular vector u λ satisfying (2.18) does also satisfy (SSC) with u † = u λ and w = λu λ . The converse is not true in general, which we want to demonstrate with the following example.
Then (SSC) reads as K * Kw = u † and we see immediately that w does not need to be a multiple of u † in this case. We now want to look more closely into the connection between (2.18) and (SC).
Proposition 2.8. Let A ∈ L(U, H) and f ∈ H, and define K ∈ L(U, R) with Ku := A * f, u for all u ∈ U. Then the singular vector condition (2.18) for K is equivalent to the source condition (SC) Proof. With the adjoint operator K * of K being given as for all c ∈ R, the singular vector condition (2.18) reads as for | A * f, u λ | = 1. Due to (2.19) we further conclude λ = J(u λ ), which verifies the proposition.
Remark 2.9. Note that we either have A * f, u λ = 1 or A * f, u λ = −1, due to the normalisation constraint | A * f, u λ | = 1. Without loss of generality we assume A * f, u λ = 1; in this case, the element v λ in the source condition simplifies to v λ = J(u λ )f .
We summarise the assumptions that will be used throughout this work: is a proper, lower semi-continuous, convex, and absolutely one-homogeneous functional.
. Then we can assume u(0) = u 0 = 0 for the initial value of the ISS flow.

Decomposition of Generalised Singular Vectors
Throughout this section we use Assumption 2.10. In addition we assume that the input data f for the inverse scale space flow (2.10) is represented by a linear combination of generalised singular vectors, i.e. f = n j=1 γ j Ku λj where {u λj } n j=1 is a finite set of generalised singular vectors for J and K and γ j ∈ R, j = 1, . . . , n. The main goal is to investigate if and under which conditions the ISS flow will give a perfect decomposition into the singular vectors representing f , i.e. when do we obtain for fixed t k > 0, k = 1, ..., n. We will introduce two conditions that will be imposed on the singular vectors in order for the ISS flow to behave this way. In addition we discuss the possibility of singular vector fusion, i.e. the possibility that two or more singular vectors can add up to another singular vector, which would make a decomposition impossible due to Theorem 2.6. Our results will be compared to the achievements of [62] which deals with signal denoising and data represented by two singular vectors.

Orthogonality Condition
We give a sufficient orthogonality condition on the generalised singular vectors {u λj } n j=1 representing the data f = n j=1 γ j Ku λj , for the ISS flow to satisfy the necessary orthogonality condition (2.11).
Let the data for the inverse scale space flow be given as f = n j=1 γ j Ku λj where {u λj } n j=1 is a set of generalised singular vectors. Assume that the inverse scale space flow gives a decomposition into the singular vectors of f with no loss of contrast, i.e. the solution of the ISS flow is given by (3.1). The necessary orthogonality condition (2.11) then states that for k = 1, 2, . . . , n − 1. A sufficient condition for this to hold is K-orthogonality between the singular vectors: This orthogonality condition will be used throughout the entire paper.

(SUB0) Condition
K-orthogonality between the singular vectors representing f will not be enough to guarantee that the solution of the ISS flow is given by (3.1). We therefore introduce an additional condition on the subgradients of the singular vectors that we name the (SUB0) condition. It is a condition on the sum of the subgradients of a set of singular vectors which turns out to be a kind of linearity condition for ∂J in the singular vectors. The (SUB0) condition furthermore implies linearity of J in the singular vectors.
Remark 3.2. In the definition of the (SUB0) condition we use the notation k ∈ {1; n} which means k = 1, 2, . . . , n. We will use this notation throughout the paper. We want to emphasise that in order for a set of singular vectors to satisfy the (SUB0) condition it is necessary that the inclusion is satisfied for all k ∈ {1; n}. It is not enough that the full sum over the subgradients is included in the subdifferential at zero. We will call the inclusions for k ∈ {1; n − 1} the partial sum conditions of the (SUB0) condition.
The (SUB0) condition is equivalent to a linearity condition for the subdifferential for K-orthogonal singular vectors: Proof. Proof of "⇐": This is straightforward using the subdifferential characterisation (2.6).
Proof of "⇒": Using the (SUB0) condition, the triangle inequality (2.4) for J and the orthogonality condition (OC) we get We would like to make a comparison to the (LIS) condition in [62,Definition 7]. LIS stands for linearity in the subdifferential. The (LIS) condition is in principle the same as the (SUB0) condition through Proposition 3.3, except that the (LIS) condition is only defined for a pair of two singular vectors and additionally requires the linearity in the subdifferential to hold for negative coefficients c j < 0 as well. (LIS) turns out to imply directly orthogonality between the singular vectors, see [62,Proposition 4].
The (SUB0) condition furthermore implies some kind of linearity of J for Korthogonal singular vectors:   In order to give an intuitive idea of what the orthogonality condition (OC) and the (SUB0) condition require of the singular vectors, we give three examples: Example 3.7. Let us again consider K = I : 1 (R m ) → 2 (R m ) and J(u) = W u 1 (R m ) as in Example 2.4. In order for two singular vectors u λ1 and u λ2 to be orthogonal we need Hence we can just check that W u λ1 and W u λ2 are orthogonal in 2 (R m ). For the (SUB0) condition we have If W u λ1 and W u λ2 have non-overlapping supports then and the (SUB0) condition is satisfied. Note that in this case also the orthogonality condition (OC) is satisfied.
Example 3.8. Let J(u) = u 1 (R m ) with ∂J(u) = sign(u), see Example 2.4. We look at a convolution problem where K : 1 (R m ) ⊇ 2 (R m ) → 2 (R m ) is the convolution operator defined by The adjoint operator K * is determined by Two examples of K-normalised singular vectors are We have The two singular vectors are K-orthogonal but they do not satisfy the (SUB0) condition since This example also shows that Proposition 3.4 only holds one way since for c 1 , c 2 ≥ 0 we have Hence the linearity of J in the singular vectors does not imply that the (SUB0) condition is satisfied. In Figure 1 we see the two singular vectors and their corresponding subgradients. We see that the subgradients contain values between −1 and 1 and are exactly −1 at the negative peaks and 1 at the positive peaks. In Figure 1c we see the sum of the singular vectors that contains three peaks. The sum of the subgradients contains values above one and is therefore not in the subdifferential of J at zero. The problem is that the two positive peaks of the singular vectors get too close, which makes the sum of the subgradients exceed one.
otherwise.  The total variation of the sum is then By Proposition 3.4, u λ1 and u λ2 do not satisfy the (SUB0) condition. In Figure 2a we see the two Haar wavelets plotted. In Figure 2b we see the sum of the two Haar wavelets. At x = 1/2 we have a second jump downwards. This is at the place where the two Haar wavelets have overlapping jumps in opposite directions. That is exactly what makes the TV * -functional non-linear in the singular vectors.
In the following we are going to provide an alternative dual norm formulation of the (SUB0) condition under a norm-inequality assumption on J.

Now we see that if
for k ∈ {1; n} then the (SUB0) condition is satisfied, since k j=1 λ j K * Ku λj , v = 0, ∀v ∈ ker(J) is already satisfied by (2.8) and λ j K * Ku λj ∈ ∂J(u λj ) for j = 1, ..., k. Hence we can use (3.3) as an alternative to the (SUB0) condition for numerical implementations when the norm-inequality assumption is satisfied.

Inverse Scale Space Decomposition
We now provide our main result which states that the ISS flow gives a perfect decomposition into the generalised singular vectors representing the data of the flow given that (OC) and (SUB0) are satisfied.
Theorem 3.14 (Inverse Scale Space Decomposition). Let {u λj } n j=1 be a set of Knormalised singular vectors of J with corresponding singular values {λ j } n j=1 . Assume that they satisfy the orthogonality condition (OC) and the (SUB0) condition. Then, if the data f is given by f = n j=1 γ j Ku λj for positive constants {γ j } n j=1 , a solution of the inverse scale space flow (2.10) is given by where t k = λ k /γ k and t k < t k+1 .
Proof. The proof is divided into the following three steps corresponding to the three different cases for u(t): Step 1: 0 ≤ t < t 1 .
For every step we have to show that the solution satisfies the inverse scale space flow.
Step 1: Let 0 ≤ t < t 1 . Then u(t) = 0, and from the inverse scale space flow we get Integrating t 0 ∂ τ p(τ )dτ yields where we have used the notation p λj = λ j K * Ku λj . We need to show that p(t) ∈ ∂J(u(t)) = ∂J(0). Using the characterisation (2.6) of the subdifferential we need to show that p(t), v ≤ J(v) for all v ∈ U. We have Hence we need to show γj λj p λj , v ≤ 0 then inequality (3.4) is satisfied since p λ1 + p λ2 ∈ ∂J(0) by the (SUB0) condition. If n j=3 γj λj p λj , v > 0 we continue the process. The process terminates when we reach n j=1 p λj , v , which is less than or equal to J(v) since n j=1 p λj ∈ ∂J(0) by the (SUB0) condition. In any case we have shown that p(t) ∈ ∂J(0) for 0 ≤ t < t 1 .
Step 2: Let t k ≤ t < t k+1 . Then u(t) = k j=1 γ j u λj and from the inverse scale space flow we get A continuous extension of p at t = t k is given by We require p(t) ∈ ∂J(u(t)) = ∂J k j=1 γ j u λj . Using the characterisation (2.6) of the subdifferential we need to show For the first criterion of (3.5) we get We have used the formula for p(t), the properties of singular vectors, the orthogonality condition (OC) and for the last equality Proposition 3.4. For the second criterion of (3.5) we can use the same process as for Step 1.
Step 3: Let t n ≤ t. Then u(t) = n j=1 γ j u λj and from the inverse scale space flow we get A continuous extension of p at t = t n is We need to show that p(t) ∈ ∂J(u(t)) = ∂J n j=1 γ j u λj . This follows directly from Proposition 3.3.
Remark 3.15. Note that for n = 2 the (SUB0) condition is both necessary and sufficient under the orthogonality condition (OC). In this case we can compute Hence we need p(t) ∈ ∂J(u(t)) for t 2 ≤ t, i.e.
This shows that the (SUB0) condition is necessary. In fact, for arbitrary n we always need n j=1 p λj ∈ ∂J(0) for the decomposition to happen. This is, however, not enough, as we will see in Example 3.18.
Remark 3.16. If u λj is a singular vector then also −u λj is a singular vector. Hence if γ j < 0, we can just pull the negative sign of γ j onto u λj and regard this as a singular vector with a positive contribution to f . Then Theorem 3.14 still holds true with t k = λ k /|γ k | and the (SUB0) condition being replaced by Remark 3.17. In Example 3.13 we have shown that the (SUB0) condition is never satisfied for n > 1 for J(u) = u L 2 (Ω) , U = L 2 (Ω). Since by Remark 3.15 the full sum over the subgradients of the singular vectors always has to be in the subdifferential of J at zero in order for the ISS flow to give a decomposition into the K-orthogonal singular vectors representing f , we then conclude that the inverse scale space decomposition can never happen in this case without re-weighting its coefficients. To emphasise that the partial sum conditions of the (SUB0) condition are necessary we look at the following example:      Hence {u λj } 5 j=1 is a set of five singular vectors. In Figure 3 we see all the singular vectors and their corresponding subgradients. We observe that two peaks of the same sign cannot get too close if we want to obtain a singular vector since the convolution kernel spreads out the peaks. We also see that the subgradients have their maximum magnitude 1 at the peaks.
We note that Ku λi , Ku λj = 1 λi p λi , u λj = 0 for i = j. Hence the orthogonality condition (OC) is satisfied. The singular vectors are also normalised such that Ku λj 2 (R m ) = 1 for j = 1, ..., 5 holds true. Now let {γ j } 5 j=1 be such that λ k /γ k < λ k+1 /γ k+1 for k = 1, ..., 4. Then if the (SUB0) condition is satisfied, Theorem 3.14 tells us that the ISS flow gives a decomposition of the data f = 5 j=1 γ j Ku λj into the singular vectors. But the (SUB0) condition is not satisfied since the partial sums over the subgradients are not in the subdifferential of J at zero: p λ1 + p λ2 = (0, 0, 1, 0, −2, 0, 1, 0, 0) T / ∈ ∂J(0), p λ1 + p λ2 + p λ3 = (0, 0, 3/2, 1, −1, 1, 3/2, 0, 0) T / ∈ ∂J(0), p λ1 + p λ2 + p λ3 + p λ4 = (−1/2, −1, 1, 1, −1, 1, 3/2, 0, 0) T / ∈ ∂J(0), Only the full sum over all the subgradients is in the subdifferential of J at zero. In Figure 4 we see some of the partial sums over the subgradients. In Figure 4a we see the sum over the first three subgradients. We see that the sum does not have its maximum magnitude at the peaks and it exceeds the limit of a magnitude of one. Hence the sum of the first three subgradients is not in the subdifferential of J at zero. In Figure 4b we see the sum over all five subgradients. This sum has its maximum magnitude at the peaks and does not exceed the magnitude limit of one. Hence the full sum is in the subdifferential of J at zero.   18. We see that the partial sum conditions of the (SUB0) condition are not satisfied. The sum over the first three subgradients seen in Figure 4a exceeds one and can therefore not be in the subdifferential of J at zero. In Figure  4b the full sum over the subgradients is seen and this is in the subdifferential of J at zero.
We are going to see in Section 5.2 that the first peak reconstructed by the ISS flow in this case is at the middle position. This peak does not match any of the singular vectors u λj , j = 1, . . . , 5, and we will not obtain the decomposition.

Singular Vector Fusion
One might think that if two or more singular vectors representing the data f add up to another singular vector, then the ISS flow is not able to give a complete decomposi-tion into the singular vectors. However, it turns out that under the (SUB0) condition and the orthogonality condition (OC), two or more singular vectors cannot add up to another singular vector.
Proposition 3.19 (Impossible Singular Vector Fusion). Let {u λj } n j=1 be a set of Knormalised singular vectors of J with corresponding singular values {λ j } n j=1 . Assume that they satisfy the orthogonality condition (OC) and the (SUB0) condition. Let the data f be given by f = n j=1 γ j Ku λj , where γ j > 0 and λ j /γ j < λ j+1 /γ j+1 . Let {u λj k } m k=1 , m ∈ {2; n}, be a subset of the singular vectors ordered so that λ j k /γ j k < λ j k+1 /γ j k+1 . Then no such subset of singular vectors can add up to another singular vector, i.e.
Proof. The proof is by contradiction. Hence we assume that m k=1 γ j k u λj k is a singular vector. Using (OC) and Proposition 3.4 we can explicitly compute the singular value: Using the characterisation (2.6) of the subdifferential we then get for all v ∈ dom(J). Set v = u λj 1 . Using J(u λj 1 ) = λ j1 and (OC) in the above we obtain Rearranging and writing out the first term of both sums, the above implies We let the term λ j1 γ 2 j1 on both sides cancel and rearrange again to obtain Using the ordering λ j k /γ j k < λ j k+1 /γ j k+1 then for k ≥ 3 we get λ j k > λj 2 γj 2 γ j k . Applying this in the above we reach

Source Condition
We emphasise the connection between the first non-trivial solution of the ISS flow (2.10) and the source condition (SC) for arbitrary data f . It turns out that the first solution satisfies the source condition, and under the special kind of source condition (2.20) the first non-trivial solution is also a singular vector.
We show that the first non-trivial solution of the ISS flow satisfies the source condition (SC). Since u(t) = 0 for 0 ≤ t < t 1 for some t 1 > 0, using the ISS flow we get Using p(0) = 0 and integration yields At t = t 1 we then require by the definition of the ISS flow. Setting v = t 1 f ∈ H we obtain K * v ∈ ∂J(u(t 1 )) which is exactly the source condition. Hence u(t 1 ) satisfies the source condition.
We also want to emphasise that (2.20) is in fact nothing but the source condition (SC) for the particular choice v = µKu λ + ηg. So we do know from Theorem 2.7 that under this specific kind of source condition the first non-trivial ISS solution is indeed a generalised singular vector. This can be of practical interest if we have arbitrary data f that can be decomposed into a singular vector and an element g such that (2.20) is satisfied. However, one has to be very careful that the conditions for a successful application of Theorem 2.7 are actually met, as the following example shows.
For the choice µ = γη we then obtain −q = µu λ + ηw, and further observe q , u λ = 4 = TV(u λ ). Hence, we have verified µu λ + ηw ∈ ∂TV(u λ ) for η = 2π and µ = γη. Unfortunately, we do not obtain µ/η < γ, which means that we cannot apply Theorem 2.7 to automatically conclude that the first non-trivial solution of the ISS flow is given via a singular vector, though it seems tempting at first glance.

Arbitrary input data
In this section we want to characterise the first non-trivial solution of the ISS flow for arbitrary given data. We investigate under which conditions the first solution is guaranteed to be a singular vector. This investigation leads to the definition of dual singular vectors in the data space.
Under the norm-inequality assumption (3.2) for J we can characterise the first nontrivial solution of the ISS flow (2.10) via the following lemma. where t 1 = c 0 / K * f U * (c 0 being the norm-inequality constant), c 1 = J(u 1 )/t 1 and Proof. For 0 ≤ t < t 1 we have u(t) = 0. The ISS flow then gives Using p(0) = 0 and integration we obtain p(t) = tK * f for 0 ≤ t < t 1 . This implies Together with the assumption that K * f, v = 0 for all v ∈ ker(J), Proposition 3.11 then implies p(t) ∈ ∂J(0). Hence the ISS flow is satisfied for 0 ≤ t < t 1 . At t = t 1 we need p(t 1 ) = t 1 K * f ∈ ∂J(c 1 u 1 ) = ∂J(u 1 ) = ∂J(v 1 ) for c 1 u 1 to be the solution at time t 1 . This is exactly the optimality condition of since the Lagrange multiplier for the constraint v ∈ ker(J) ⊥ is zero following the same procedure as of that for the generalised singular vectors in Section 2. Finally, we require u(t 1 ) to satisfy the necessary orthogonality condition (2.11): We have used that t 1 K * f ∈ ∂J(u 1 ) for the last equality. From the above we obtain The interesting question now is when the first ISS solution u 1 characterised via (4.1) is a singular vector. We need to investigate if J(u 1 )K * Ku 1 ∈ ∂J(u 1 ). Using the characterisation of the subdifferential (2.6) we need to show J(u 1 )K * Ku 1 , u 1 = J(u 1 ) and J(u 1 )K * Ku 1 ∈ ∂J(0).
The first condition is easy to verify since Ku 1 H = 1. For the second condition we use Proposition 3.11, thus we need J(u 1 )K * Ku 1 U * ≤ c 0 . For J(u 1 ) we use the estimate Then we obtain where K * is the standard operator norm Hence we see that if K * f U * = K * f H then the first ISS solution u 1 is actually a singular vector of J. We note here that one should be very careful using the correct norms. To emphasise this we consider the following example: Example 4.4. Let K = I : H 1 0 (Ω) → L 2 0 (Ω) be an imbedding operator and Ω a bounded domain in R 2 . Here subscript zero means that the functions are zero on the boundary of Ω. Let J(u) = ∇u L 2 (Ω) . Using Poincaré's inequality u L 2 (Ω) ≤ C ∇u L 2 (Ω) for some C > 0 and u ∈ ker(J) ⊥ , we can verify that J satisfies the norm-inequality assumption (3.2) for c 0 = 1/(1 + C 2 ) 1/2 . Let f ∈ H 1 (Ω) be the data for the ISS flow. If f satisfies the first non-trivial solution of the ISS flow is a singular vector. The equality is not always satisfied even though K is an embedding operator. We see that we need The condition K * f U * = K * f H gives rise to the following theorem: Theorem 4.5. Assume that J satisfies the norm-inequality assumption (3.2). If f ∈ H \ ker(K * ) is a dual singular vector, i.e.
and satisfies K * f, v = 0 for all v ∈ ker(J), then the first non-trivial solution u 1 of the ISS flow (2.10) characterised via (4.1) is a (primal) singular vector of J.
Proof. We have shown above that if f satisfies K * f U * = K * f H , then u 1 is a singular vector. Since we see that in order for K * f U * = K * f H to hold true, f has to satisfy The optimality condition for this is Remark 4.6. Setting J 2 (ω) = K * ω U * , ω ∈ H, we see from the above that f has to be a singular vector of J 2 (and I : H → H). J 2 is absolutely one-homogeneous and hence we can reuse all the results for absolutely one-homogeneous functionals.

Numerical Results
We conclude the paper with numerical results that demonstrate the sufficiency of the (SUB0) condition when we wish to decompose a linear combination of generalised singular vectors. In order to verify the theoretical results provided in Section 3, we need to be able to solve the ISS flow (2.10) numerically. In [69] it has been shown that solutions of (2.10) can be computed directly in case of J(u) = u 1 (R m ) via what has been termed the adaptive inverse scale space method (aISS). In [70] the aISS has been extended to general polyhedral functionals J and could therefore also be used to numerically verify the behaviour of functionals like the anisotropic total variation, for example. For arbitrary convex, absolutely one-homogeneous functionals J, however, there does not exist a way to efficiently compute the exact ISS solution at each time step numerically to the best of our knowledge. Therefore, for simplicity, we are going to restrict ourselves to the case of J(u) = u 1 (R n ) for the numerical results. We compute the inverse scale space solution for a composition satisfying the (SUB0) condition. We also demonstrate the need for the (SUB0) condition computing ISS solutions of compositions following Examples 3.8 and 3.18.

(SUB0) Violations
Following Example 3.8 we also want to verify numerically that the absence of the (SUB0) condition in fact can lead to results different from the decomposition of the data into the original singular vectors. We use the operator K, the functional J, and the two singular vectors u λ1 = (0, 0, 1, 0, 0) T and u λ2 = 1 √ 2 (0, 1, 0, −1, 0) T , from example 3.8. We know that the two singular vectors are K-orthogonal but violate the (SUB0) condition. We can then for instance define f = Ku λ1 + 3 2 √ 2 Ku λ2 as potential input data that we want to decompose. Note that we have ensured λ 1 /γ 1 = 1 < 4/3 = λ 2 /γ 2 . In order to compute a solution of the ISS flow (2.10) we again use the aISS method. The first non-trivial solution u(t 1 ) of the aISS method reads as u(t 1 ) = (0, 5/4, 0, 0, 0) T .     Figure 5a shows both the primal and the dual variable, i.e. u(t1), p(t1), at t = t1, whereas Figure 5b and 5c show the same for t2 and t3, respectively. It is easily observed that the aISS algorithm has decomposed the signal f into the original singular vectors with no loss of contrast, as stated by Theorem 3.14.
This solution is obviously as sparse as u λ1 and also a singular vector (with the same singular value after normalisation) but corresponds neither to u λ1 nor to u λ2 . More importantly, it is a better fit to the data f with respect to the 2 -norm, as we have Ku(t 1 ) − f 2 2 (R m ) = 9/16 as opposed to Ku λ1 − f 2 2 (R m ) = 9/8. Hence, it is logical that the first ISS solution is not u λ1 . The remaining ISS solutions are u(t 2 ) = (0, 1, 1/2, 0, 0) T and u(t 3 ) = (0, 3/4, 1, −3/4, 0) T , where u(t 3 ) satisfies u(t 3 ) = u λ1 + 3 2 √ 2 u λ2 . What is interesting about this example is that the ISS flow (2.10), whilst converging to a J-minimial solution of K * Ku = K * f , most importantly has to minimise the residual in each step. This dominates the behaviour of keeping J minimal at the same time, as we observe J(u(t 1 )) > J(u λ1 ) but Ku(t 1 ) − f 2 (R m ) < Ku λ1 − f 2 (R m ) .
The solutions at steps two and three cannot be written as any linear combination of the five singular vectors u λj , j = 1, . . . , 5. From step four the solutions can be written as linear combinations of the singular vectors with no loss of contrast. However, we are not able to separate u λ1 , u λ2 , and u λ3 .

Conclusions & Outlook
In this paper we have investigated the possibility of using the inverse scale space flow as a decomposition method for absolutely one-homogeneous regularisation functionals. We have formulated conditions under which the inverse scale space flow will give a decomposition of a linear combination of generalised singular vectors. Furthermore, we have investigated the behaviour of the flow for arbitrary data. We have given a characterisation of the first non-trivial solution and shown that this solution is a primal singular vector when the input data is what we have named a dual singular vector.
For the inverse scale space decomposition we require the generalised singular vectors representing the data to be orthogonal when mapped to the data space by the forward operator. This condition, however, is quite restrictive and in future research we aim to relax the condition. The idea is that the inverse scale space flow will then still give a decomposition into the singular vectors but with time-varying coefficients. Another subject of interest is to investigate whether the data can always be decomposed into a linear combination of singular vectors and a remainder that can be seen as noise satisfying special conditions such as (2.20). Extending the decomposition results to data containing noise in the same manner as in [3, Section 7.2] we then know that the inverse scale space flow will always give a decomposition into singular vectors.
Another direction for future research is the numerical computation of inverse scale space solutions as well as the numerical computation of generalised singular vectors. As mentioned earlier, the former is currently restricted to polyhedral regularisation functionals (cf. [69,70]). For the latter there has been made substantial progress in [71,72,73,74], but many open questions need yet to be addressed. How can we compute ground states as well as non-trivial singular vectors with larger singular values? How can we incorporate (linear) forward operators? How can we compute dual singular vectors in a robust fashion? These are only few of many open questions that need to be addressed in future research.