New estimations on the upper bounds for the nuclear norm of a tensor

Using the orthogonal rank of the tensor, a new estimation method for the upper bounds on the nuclear norms is presented and some new tight upper bounds on the nuclear norms are established. Taking into account the structure information of the tensor, an important factor affecting the upper bounds is discussed and some corresponding properties related to the nuclear norms are given. Meanwhile, some new results of the upper bounds on the nuclear norms are obtained.


Introduction
A tensor is a multidimensional array that can provide a natural and convenient way for representing the multidimensional data such as discrete forms of multivariate functions, images, video sequences, and so on [1,2,10,14]. With the successful and widespread use of the matrix nuclear norm (a sum of the singular values) in the information recovery, the research on the nuclear norm of the tensor (see Definition 2.3 in Sect. 2) has been a hot topic in both the theory and applications [5][6][7][8].
A natural problem is how to compute the nuclear norm of a tensor. Unfortunately, compared with the matrix nuclear norm, the nuclear norm of a tensor is closely related to the number field, and the computation of the tensor nuclear norm is NP-hard [5]. Thus, exploring some simple polynomial-time computable upper bounds on the nuclear norm is very important.
Relating to the nuclear norm of a tensor, Friedland and Lim established the following upper bound through the Frobenius norm of this tensor in [5]. Theorem 1.1 ([5]) Let X ∈ R n 1 ×···×n D . Then In [8], Hu established a tighter upper bound. Theorem 1.2 (Lemma 5.1 in [8]) Let X ∈ R n 1 ×···×n D . Then Furthermore, Hu established another upper bound on the nuclear norm of a tensor through the nuclear norms of its unfolding matrices. Theorem 1.3 (Theorem 5.2 in [8]) Let X ∈ R n 1 ×···×n D . Then In this paper, we present some new upper bounds on the nuclear norms through using the orthogonal rank of the tensor [9,12]. Furthermore, taking into account the structure information of the tensor, some new results of the upper bounds on the nuclear norms are obtained.
Since the spectral norm and nuclear norm of a tensor are closely related to the number field [6], for the sake of simplicity, we always assume that the tensors discussed are nonzero real tensors, and unless mentioned otherwise, we just discuss the spectral and the nuclear norm over the real field. Some corresponding notations are as follows: a tensor is denoted by the calligraphic letter (e.g., X ), while the scalar is denoted by the plain letter, and matrices and vectors are denoted by bold letters (e.g., X and x).
The rest of the paper is organized as follows. In Sect. 2, we recall some definitions and related results which are needed for the subsequent sections. In Sect. 3, we present the upper bounds on the nuclear norms of general tensors and discuss the factor affecting the upper bounds. Finally, some conclusions are made in Sect. 4.

Notations and preliminaries
This section is devoted to reviewing some conceptions and results related to tensors, which are needed for the following sections.
Firstly, we discuss the unfolding matrix or matrix representation of a tensor. Let X = (x i 1 ···i D ) ∈ R n 1 ×···×n D . Then, by organizing several indexes of X and the remaining indexes of X as the row index and the column index, respectively, the tensor X can be reshaped into a matrix form [13]. Especially, if the number of row index is equal to one, then we get the mode-d matricization X (d) , the columns of which are the mode-d fibers of the tensor (obtained by fixing every coordinate except one, , arranged in a cyclic ordering; see [3] for details. Contrary to the operation above, a matrix can also be reshaped into a tensor by using the opposite operation.
In what follows, we review some definitions of the tensor norms.

Definition 2.2 ([6]
) Let "•" denote the outer product operation. The spectral norm of X ∈ R n 1 ×···×n D is defined as Furthermore, X 2 is equal to the Frobenius norm of the best rank-one approximation of the tensor X .
Similar to the matrix case, the nuclear norm can be defined through the dual norm of the spectral norm.

Definition 2.3 ([6]
) Let X ∈ R n 1 ×···×n D , the nuclear norm of X is defined as the dual norm of the spectral norm. That is, For the nuclear norm defined by (3), it can be shown that Another important concept related to the matrix and the tensor is the mode-d multiplication.

Definition 2.4 ([3]
) Let X = (x i 1 ···i D ) ∈ R n 1 ×···×n D . Then the mode-d multiplication of X by the matrix U = (u ij ) ∈ R n d ×n d is defined by It should be mentioned that the mode-d multiplication is also available for n d = 1. Furthermore, let If for all 1 ≤ d ≤ D the matrices W (d) are orthogonal matrices (W (d) W (d) T is an identity matrix), then (W (1) , . . . , W (D) ) · X is called a multi-linear orthogonal transformation of the tensor X .
Finally in this section, we introduce the tool used in the paper for the estimation of the upper bounds.
Definition 2.5 ([9]) The orthogonal rank of X ∈ R n 1 ×···n D is defined as the smallest number R such that where U r (1 ≤ r ≤ R) are rank-one tensors such that U r 1 , U r 2 = 0, (r 1 = r 2 ) for 1 ≤ r 1 ≤ R and 1 ≤ r 2 ≤ R.
The decomposition of X given by (4) is also called the orthogonal decomposition of the tensor X .
For the orthogonal rank of a tensor, the following conclusion is true.
where r ⊥ (X ) denotes the orthogonal rank of X .
Noting the fact that indices relabeling does not change the tensor orthogonal rank, then by Theorem 2.1, we have that, for any X ∈ R n 1 ×···×n D , it holds Especially, for the third order tensor, the following result was established in [11].
Then, for any X ∈ R n×n×2 , the following holds: 2n, if n is even.

Upper bounds of the nuclear norm
In this section, we discuss the upper bounds on the nuclear norm of a tensor. Meanwhile, some properties and polynomial-time computable bounds related to the nuclear norm will be given.

Upper bounds given by the Frobenius norm
In this subsection, we use the orthogonal rank of a general tensor to establish the upper bounds on the nuclear norm through the Frobenius norm of this tensor. Then Proof Let Y ∈ R n 1 ×···×n D be an arbitrary nonzero tensor and the orthogonal rank of Y be R y . Suppose that Then, according to the properties of the orthogonal rank decomposition and the best rank-one approximation, we have and Without loss of generality, suppose that Then it follows from (7) and (8) Thus, according to the arbitrariness of Y and (9), we get Noting the definition of the nuclear norm (Definition 2.3), the conclusion is established.
Remark 3.1 Comparing the upper bound given by (6) with the upper bound given by (1), which is obtained in [8], the new upper bound given by (6) is tighter.
Actually, it follows from (5) that the upper bound given by (6) improves the upper bound given by (1).
More specifically, we present a simple example to show that the upper bound given by Theorem 3.1 not only can be tighter than the upper bound given by (1) but also a sharp bound.
By Theorem 3.1 and Lemma 2.1, we get This means that the upper bound given by (3.1) is tighter than the upper bound given by (1). Furthermore, by a simple computation, we get A 2 = 1. Then it follows from the definition of the nuclear norm that Thus, it holds then, according to the sufficient and necessary conditions of the nuclear norm decomposition obtained in [6], we get that (10) is a nuclear decomposition of A. This also means that the upper bound given by Theorem 3.1 is a sharp upper bound of the nuclear norm.

Upper bounds given by nuclear norms of the unfolding matrices of a tensor
In this subsection, we present a new way to establish the upper bounds on the nuclear norm of a tensor through the nuclear norms of the unfolding matrices of this tensor. Then Proof Let the singular value decomposition of the matrix X (1) be where u s ∈ R n 1 , v s ∈ R n 2 ×···×n D , u s 2 = 1, v s 2 = 1, σ s > 0, and 1 ≤ s ≤ S.
Then equality (12) can be expressed as the following form: where V s ∈ R n 2 ×···×n D are obtained by reordering the vector v s into (D -1)th order tensor with a certain order, 1 ≤ s ≤ S. Suppose that the orthogonal rank decomposition of V s is Then, by taking the expression of V s into the right-hand side of (13), we get Noting that where P s ≤R, 1 ≤ s ≤ S, then it follows from the definition of the nuclear norm and (14) that ≤ σ 1 P 1 + · · · + σ S P S by (15) and Cauchy-Schwarz inequality Remark 3.2 Comparing the upper bound given by (11) with the upper bound given by (2), which is obtained in [8], the new upper bound given by (11) is smaller.
Actually, it follows from inequality (5) that the upper bound given by (11) improves the upper bound given by (2).
Similar to the discussion of Hu [8], the upper bounds can also be obtained by other unfolding ways and further improved by considering the multi-linear ranks of a tensor (ranks of the unfolding matrices). Corollary 3.1 Let X ∈ R n 1 ×n 2 ×n 3 and r d = rank(X (d) ), 1 ≤ d ≤ 3. Then Proof According to the conditions of the corollary and the higher order singular value decomposition of the tensor [3], the tensor X can be expressed as By the definition of the tensor nuclear norm (Definition 2.3), one can easily verify the following conclusions: and It follows from Theorem 3.2 and (17) that Noting (16), we get X * ≤ min{r 2 , r 3 } X (1) * .
Thus, the conclusion is obtained.

Factors affecting the upper bounds on the nuclear norm and further results
In this subsection, we discuss the factors affecting the nuclear norm of a tensor. Especially, we focus on the structure analysis of a tensor. Based on the discussion, some new upper bounds on the nuclear norms of tensors are presented. Firstly, we give a simple example to illustrate that the nuclear norm of a tensor is closely related to the structure of this tensor.

Example 3.2 Let
It is well known that the nuclear norm of a tensor is closely related to the number field [6]. Actually, the tensor A can be expressed as the following form: Let Then it holds For the sake of convenience, letÂ = (W (1) , W (2) , W (3) ) · A. Then, using the same method as above, we have Noting the structures of the tensors given by (18) and (19), the above derivation process implies that the nuclear norm of a tensor is closely related to the structure of this tensor. In what follows, we discuss the block diagonal tensor, which can be illustrated by Fig. 1. Furthermore, the block diagonal tensor can be expressed by using the direct sum operation "⊕" [4], which is defined as follows: Figure 1 The block diagonal tensor with three diagonal blocks Let A = (a i 1 ···i D ) ∈ R n 1 ×···×n D and B = (b j 1 ···j D ) ∈ R n 1 ×···×n D , then the direct sum of A and B is an order-D tensor C = (c i 1 ···i D ) = A ⊕ B ∈ R (n 1 +n 1 )×···×(n D +n D ) defined by Based on the discussion above, we present some properties of the spectral norm and nuclear norm of the tensor. Then Proof According to the definition of the spectral norm of a tensor (Definition 2.3), it is easy to get Thus, the rest of the proof just needs to show Firstly, we consider the case of the third order tensors.
Suppose that X (l) ∈ R n (l) 1 ×n Then the following matrix is a block diagonal matrix. It follows Hence, inequality (21) is proved. This also implies that equality (20) is true for the third order tensors. Secondly, for the case of higher order tensors with order larger than or equal to four, the same result can be established by the recursive method.
In all, the conclusion is true.
Then, based on Lemma 3.1, the following two results related to the nuclear norms of tensors can be established.
D be a zero tensor, and X ∈ R n (2) 1 ×···×n Proof Suppose that where for all 1 ≤ p ≤ P, where all 0s denote zero vectors with suitable dimensions. This implies Furthermore, assume that D and Y 2 = 1.
Then, by Lemma 3.1, we have It follows from Definition 2.5 that Thus it holds O ⊕ X * = X * . Using the same method, the equality X ⊕ O * = X * can be proved. .
Proof We just need to prove the case of L = 2. For the general case, the conclusion can be obtained in a recursive way.
D are both zero tensors. Then, by using Lemma 3.2, we get Suppose that Then, by Lemma 3.1, we get Thus, according to Definition 2.5, we have Combined (22) with (23), the results can be obtained.
Based on the fact that the nuclear norm of a tensor is also kept invariant under the multi-linear orthogonal transformation, we get the following result. Corollary 3.2 Let X ∈ R n 1 ×···×n D . If the tensor X admits a diagonal structure under the multi-linear orthogonal transformations, then where σ p (1 ≤ p ≤ P) are the diagonal elements and P ≤ n 1 .
This case presented by Corollary 3.2 is consistent with the definition of the nuclear norm of the matrix case, and in this case, the nuclear norm of the tensor can be accurately calculated.
Taking into account the structure information of the tensor, some new results of the upper bounds on the nuclear norms can be obtained. For the convenience of comparison, we just present the upper bounds on the nuclear norms of tensors through the dimensions of the tensors, without considering the orthogonal rank of the tensors. Theorem 3.3 Let X ∈ R n 1 ×···×n D and L be the maximum number of diagonal blocks that the tensor X can attain under the multi-linear orthogonal transformations. Suppose that the size of each diagonal block is n (l) 1 × · · · × n (l) D and Then it holds Proof Assume that Then it follows from the invariance of the Frobenius norm of a tensor under the multilinear orthogonal transformation that Furthermore, since the nuclear norm of a tensor is also kept invariant under the multilinear orthogonal transformation, we get Hence, by Lemma 3.3 and (25), we get Thus, the upper bound given by (24) improves (1). Theorem 3.3 also shows that the upper bound on the nuclear norm can be improved by using the structural information.
Similarly, the following upper bound can also be obtained.  Then, by Theorem 1.2, we get It follows from Theorem 3.4 that There has been a marked improvement in the upper bounds on the nuclear norm.

Conclusions
In this paper, we provide a new estimation method for the upper bounds on the nuclear norms and obtain some new upper bounds related to the nuclear norms. Meanwhile, it is found that the upper bounds on the nuclear norms are not only related to the dimensions of the tensor but also to the structure of the tensor. Taking into consideration the structure information of the tensor, the upper bounds on the nuclear norms can be improved.