Robust Low-Rank Graph Multi-View Clustering via Cauchy Norm Minimization

Abstract

Graph-based multi-view clustering methods aim to explore the partition patterns by utilizing a similarity graph. However, many existing methods construct a consensus similarity graph based on the original multi-view space, which may result in the lack of information on the underlying low-dimensional space. Additionally, these methods often fail to effectively handle the noise present in the graph. To address these issues, a novel graph-based multi-view clustering method which combines spectral embedding, non-convex low-rank approximation and noise processing into a unit framework is proposed. In detail, the proposed method constructs a tensor by stacking the inner product of normalized spectral embedding matrices obtained from each similarity matrix. Then, the obtained tensor is decomposed into a low-rank tensor and a noise tensor. The low-rank tensor is constrained via nonconvex low-rank tensor approximation and a novel Cauchy norm with an upper bound is proposed to handle the noise. Finally, we derive the consensus similarity graph from the denoised low-rank tensor. The experiments on five datasets demonstrate that the proposed method outperforms other state-of-the-art methods on five datasets.

1. Introduction

Multi-view learning is a significant task in machine learning. Recently, a battery of multi-view learning methods have emerged, such as cross-view domain learning [1], multi-view classification [2,3], multi-view outlier detection [4] and multi-view clustering [5,6,7]. Multi-view data are ubiquitous in the real world, for example, images consist of texture, lighting, and color, while videos are made up of audio and frames. Multiple views contain complementary and principle information, which can provide multi-view learning with more knowledge. Therefore, utilizing multi-view data for clustering tasks can yield more precise results compared to using only a single view.

The clustering method aims to explore the latent categorical information of multi-view data in an unsupervised manner. The performance of most clustering methods heavily depends on the learned similarity matrix. To improve the quality of the similarity matrix, numerous clustering approaches have been proposed. Subspace learning and matrix decomposition [8,9,10,11] and graphs are widely used for multi-view clustering.

A graph is a powerful data structure that indicates the correlations among individuals and the structure of groups [12,13]. In this structure, nodes represent individuals, and the values of the edges indicate the strength of connections among individuals. The graph is usually represented as a similarity matrix, where elements determined by rows and columns denote the edges between individuals. Typically, the graph is constructed using the Euclidean distance with each element denoting the similarity between two samples.

Graph-based clustering methods typically obtain a consensus similarity matrix from multiple views. Subsequently, an additional clustering step, such as spectral clustering, is performed to obtain the clustering results. Although graph-based methods have achieved impressive performance, they still have some drawbacks. Most of the methods learn the similarity graph in the original data space, resulting in the learned graph missing structural information in the low-dimensional space. To address this limitation, the work in [14] constructs a similarity graph in the spectral embedding space, which captures both the low-dimensional information and local structure. However, [14] only uses the spectral embedding to handle the noise, and does not specifically process the noise in the spectral embedding accordingly. Furthermore, [14] uses a weighted tensor nuclear norm (assign weights to each singular value) to impose a low-rank constraint on the tensor constructed from the inner product of the normalized spectral embedding matrices. Assigning optimal weights becomes a challenge when dealing with a large number of data views. This is because the weight assignment relies on a priori knowledge of the data, which is difficult to obtain directly.

To address these limitations, the proposed approach is as follows: (1) A tensor nuclear norm with a nonconvex approximation function [15,16,17] is utilized to constrain the low-rank tensor. This approach assigns weights to each singular value using a parameter while constraining the contribution of each singular value to the nuclear norm to be close to 1. (2) Inspired by [18,19,20], a novel norm named the Cauchy norm with an upper bound is proposed to handle the noise in the tensor. Compared with the widely used $l_{2,1}$ norm, the Cauchy norm is more stable, due to its asymptotic upper bound, which alleviates the influence of noise, especially in the case of high-noise values. By using the Cauchy norm, we prevent large noise values dominating the objective function. By combining these two strategies, we introduce a novel multi-view clustering method called Robust Low-rank Graph Multi-view Clustering (RLGMC) via Cauchy Norm Minimization. The proposed method can be optimized with heuristic algorithms such as particle swarm optimization [21], collaborative neurodynamic optimization [22] or the alternating direction method of multipliers (ADMM) [23]. The flowchart of the proposed method is illustrated in Figure 1. We unify spectral embedding, low-rank constraint and noise constraint into a unified framework. The low-rank tensor constructed from the spectral embedding space is decomposed into a robust low-rank tensor and a noise term. Subsequently, the consensus graph S is obtained from a robust low-rank tensor. Finally, the clustering results are obtained by performing spectral clustering on S. The main contributions of our work are as follows:

• A novel multi-view clustering method referred to as RLGMC is proposed. In this method, we combine spectral embedding, low-rank tensor learning and noise constraints into a unified framework. By learning a robust tensor, the underlying structure implied in multiple views is effectively captured.
• To enhance the tensor nuclear norm, a nonconvex low-rank approximation is employed. This approach assigns weights to each singular value via a nonconvex function, improving the performance of low-rank approximation while only requiring one hyperparameter.
• We propose a norm called the Cauchy norm with an upper bound to handle the noise in the spectral embedding space. This norm suppresses the noise value, preventing large noise values from dominating the objective function.
• An alternating optimization algorithm is designed to solve the proposed method. Additionally, experiments conducted on six real-world datasets demonstrate that the proposed method outperforms state-of-the-art methods.

The remainder of this paper is organized as follows. Section 2 gives a brief review of the related works. Section 3 presents the notations and preliminaries used throughout this paper. The details and the proposed method are presented in Section 4. In Section 5, an efficient iterative optimization algorithm is designed to optimize the proposed method. Section 6 verifies the effectiveness of our approach on six datasets. Finally, the conclusion is provided in Section 7.

2. Related Works

In this section, we introduce several popular clustering methods.

2.1. Deep Multi-View Clustering Methods

Deep multi-view clustering methods discover the clustering patterns via deep neural network architecture. The work in [24] mimics the attributes of ‘self-expression’ via the self-expressive layer. Wang et al. unified two schools of graph representation learning strategies via adversarial training in a minimax game [25]. To adapt graph neural networks to multi-view clustering tasks, Li et al. proposed a graph auto-encoder for clustering and constructs a weighted graph for GAE via a generative graph representation model [26].

2.2. Subspace Clustering Methods

Subspace clustering methods learn a similarity matrix and then obtains clustering patterns via an additional clustering method. Sparse subspace clustering (SSC) and low-rank representation (LRR) are two representative methods, which explore sparse representation and low-rank representation, respectively [27,28]. However, both SSC and LRR are single-view methods limited by the inability to use complementary information from multiple views. To address this limitation, Zhang et al. explore the complementary information via a low-rank constraint [29]. To learn the higher-order information among multiple views, Xie et al. proposed a tensor nuclear norm based on tensor singular value decomposition to achieve a low-rank constraint on the rotated tensor [30]. The work in [31] learns an essential tensor for Markov chain-based spectral clustering.

2.3. Graph-Based Clustering Method

Graph-based methods typically try to find a consensus graph among multiple views. The work in [32] learns the similarity matrix by assigning optimal neighbors for each sample and a rank constant of the Laplacian matrix. Nie et al. proposed a parameter-free method to learn optimal weights for each view [33]. A step further, to address the drawback in [33], Nie et al. further proposed a novel self-weighted method to impose the graph as the center of a multi-view by a Laplacian rank constraint [34]. In [14], a framework for simultaneously learning spectral embeddings and low-rank tensor representations is proposed, achieving excellent performance.

3. Notations and Preliminaries

In this section, the notations, preliminaries and nomenclature declaration are given.

For convenience, the basic notations and preliminaries used in this paper are provided in this section. A summary of the primary notations is shown in

Table 1. Notations and nomenclature declaration.

Notations	Meaning
$z_i$	The i-th element of vector z
$Z_i$	The i-th column of matrix Z
$Z^T$	The Transposition of matrix Z
${\parallel Z\parallel }_F$	${\parallel Z\parallel }_F=\sqrt{{\sum }_{ij}{Z}_{ij}^2}$
${\parallel Z\parallel }_{2,1}$	${\parallel Z\parallel }_{2,1}={\sum }_j{\parallel Z(:,j)\parallel }_2$
${\parallel Z\parallel }_{\ast }$	Sum of singular values
${\mathcal{Z}}_{ijk}$	The ($i,j,k$)-th entry of $\mathcal{Z}$
$\mathcal{Z}(i,:,:)$	The i-th horizontal slice of $\mathcal{Z}$
$\mathcal{Z}(:,j,:)$	The j-th lateral slice of $\mathcal{Z}$
$\mathcal{Z}(:,:,k)$	The k-th frontal slice of $\mathcal{Z}$
${\mathcal{Z}}^T$	The Transposition of tensor $\mathcal{Z}$
${\mathcal{Z}}^{(i)}$	${\mathcal{Z}}^{(i)}=\mathcal{Z}(:,:,i)$
${\mathcal{Z}}_{(i)}$	The i-th matricization of $\mathcal{Z}$
${\parallel \mathcal{Z}\parallel }_F$	${\parallel \mathcal{Z}\parallel }_F=\sqrt{{\sum }_{ijk}{Z}_{ijk}^2}$
${\parallel \mathcal{Z}\parallel }_{\ast }$	Tensor nuclear norm
$l_{2,1}$ norm	${\parallel A\parallel }_{2,1}$, where A is matrix or tensor
$l_2$ norm	${\parallel A\parallel }_2$, for measuring Euclidean distance
$tr(·)$	The traces of matrix
Tensor $\mathcal{Z}$	A data structure made by stacking multiple matrices

. The calligraphy letters (e.g., $\mathcal{T}$), capital letters (e.g., T), bold lower case letters (e.g., $\mathit{t}$) and lower case letters (e.g., t) denote the tensor, matrices, vectors and scalar, respectively. The discrete fast Fourier transformation (DFFT) and the inverse operation of a tensor $\mathcal{T}$ along the third dimension are defined as $\overline{\mathcal{T}}=fft(\mathcal{T},\left[\right],3)$ and $\mathcal{T}=ifft(\overline{\mathcal{T}},\left[\right],3)$, respectively.

3.1. Tensor Construction and Rotation Operations

Tensor construction operation is represented by $\mathsf{\Psi }(·)$. $\mathsf{\Psi }(·)$ can construct the tensor by stacking multiple matrices.

The tensor rotation operation is represented by $\mathsf{\Phi }(·)$. Given a tensor $\mathcal{Z}\in {\mathbb{R}}^{N\times M\times V}$, $\mathsf{\Phi }(·)$ can rotate the size of $\mathcal{Z}$ to $N\times V\times M$.

3.2. Tensor Nuclear Norm and Nonconvex Approximation

The tensor singular value decomposition (t-SVD) [35] of a tensor $\mathcal{A}\in {\mathbb{R}}^{n_1\times n_2\times n_3}$ can be formulated as follows.

$$\mathcal{A}=\mathcal{U}\ast \mathcal{S}\ast {\mathcal{V}}^T$$

(1)

where $\mathcal{U}\in R^{n_1\times n_1\times n_3}$ and $\mathcal{V}\in {\mathbb{R}}^{n_2\times n_2\times n_3}$ are orthogonal tensors (a tensor satisfies $\mathcal{U}\ast {\mathcal{U}}^T={\mathcal{U}}^T\ast \mathcal{U}=\mathcal{I}$), $\mathcal{S}\in R^{n_1\times n_2\times n_3}$ is f-diagonal tensor (each of its front slices is a diagonal matrix). Furthermore, ∗ is the t-Product defined in [35].

The tensor nuclear norm (TNN) proposed in [36] of a tensor $\mathcal{A}$ is as follows.

$${\parallel \mathcal{A}\parallel }_{\ast }=\sum _{v=1}^V\sum _{j=1}^r\overline{S}(j,j,v)$$

(2)

where $r=min\{N,M\}$, $\overline{S}$ is denoted as the DFFT on S which is the singular value of $\mathcal{A}$.

The nonconvex low-rank tensor approximation TNN has a tighter approximation of the original sparsity regularized rank function than TNN [15], which is represented as follows.

$${\parallel \mathcal{Z}\parallel }_{\ast ,\theta }=\sum _{v=1}^V\sum _{j=1}^r\varphi ({\sigma }_j(\overline{S^{(v)}}),\theta )=\sum _{v=1}^V\sum _{j=1}^r\frac{(1+\theta ){\sigma }_j(\overline{S^{(v)}})}{\theta +{\sigma }_j(\overline{S^{(v)}})}$$

(3)

where $\varphi$, ${\sigma }_i$ and $\theta$ denote the nonconvex approximation function and the i-th largest singular value and function parameters, respectively.

3.3. Cauchy Norm with Upper Bound

The Cauchy norm with an upper bound of a tensor $\mathcal{E}\in {\mathbb{R}}^{N\times M\times V}$ can be formulated as follows:

$${\parallel \mathcal{E}\parallel }_{Cauchy}=\sum _{i=1}^{M}ln(1+\frac{{\parallel E(:,i)\parallel }_2}{\gamma })$$

(4)

where E obtained by $E={\mathcal{E}}_{(3)}$ is the third matricization of $\mathcal{E}$ and $\gamma$ is the scaling parameters. The value of $\gamma$ is positive.

Theorem 1.

Given a non-negative tensor $\mathcal{E}\in {\mathbb{R}}^{N\times M\times V}$, it is obvious that the $Cauchy$ norm with an upper bound is less sensitive to large noise values compared to the $l_{2,1}$ norm and the following inequality holds:

$${\parallel \mathcal{E}\parallel }_{Cauchy}\le {\parallel \mathcal{E}\parallel }_{2,1}$$

(5)

Proof. 

Clearly, x is greater than $ln(1+\frac{x}{\gamma })$; if x is larger than $1-\frac{1}{\gamma }$. Since ${\parallel \mathcal{E}\parallel }_{Cauchy}$ equals ${\sum }_{i=1}^{N}ln(1+\frac{{\parallel E(:,i)\parallel }_2}{\gamma })$ and ${\parallel \mathcal{E}\parallel }_{2,1}$ equals ${\sum }_{i=1}^N{\parallel E(:,i)\parallel }_2$, where $E={\mathcal{E}}_{(3)}$, we can construct the following inequality.

$$ln(1+\frac{{\parallel E(:,i)\parallel }_2}{\gamma })\le {\parallel E(:,i)\parallel }_2$$

(6)

The property shown in (6) helps prevent the noise values from dominating the objective function during optimization. Additionally, it is worth noting that each ${\parallel E(:,i)\parallel }_2$ must satisfy the condition ${\parallel E(:,i)\parallel }_2\ge 1-\frac{1}{\gamma }$. As a result, the value of $\gamma$ is confined within the range of 0–1 to ensure the constancy of Ineq. (6).    □

3.4. Similarity Graph Construction and Spectral Embedding

There is a set of data matrices ${\{X^{(v)}\}}_{v=1}^V$, $X\in {\mathbb{R}}^{d_v\times N}$, where N is the number of samples and $d_v$ is the dimension of the data in v-th view. The work in [37] constructs the similarity graph by the adaptive allocation of weights according to the Euclidean distance between two points as follows.

$$\begin{array}{cc}\hfill \underset{s^v}{min}& \sum _{v=1}^V{w}_v\sum _{i,j=1}^N\parallel x_i^v-x_j^v{\parallel }_2^2{S}_{i,j}+\lambda \ast {\parallel S\parallel }_F^2\hfill \\ \hfill & s.t.S_{i,j}\ge 0,S_i{\mathbf{1}}^T=1,rank(L_s)=n-c\hfill \end{array}$$

(7)

where $x_i^v$ and $x_j^v$ are i-th and j-th data points in the v-th view. n and c denote the number of samples and number of clusters, respectively. $s_{i,j}$ denotes the consensus similarity graph. $\lambda$ is the balance parameter. The Laplacian matrix $L_s$ is calculated from $L_s=D-S$, D is the diagonal matrix with $d_{i,i}={\sum }_{j=1}^n{S}_{i,j}$. The weight $w_v$ is equal to

$$\frac{1}{2\sqrt{{\sum }_{i=1}^n{\sum }_{j=1}^n}{\parallel x_i^v-x_j^v\parallel }_2^2{S}_{i,j}}.$$

(8)

Moreover, the work in [14] learns a low dimension embedding similarity graph to address the limit that the graph learned from Equation (7), which is inaccurate because of the presence of noise and redundancies. Low-dimensional spectral embedding graph can be obtained as follows.

$$\begin{array}{cc}\hfill & \underset{S}{min}\sum _{v=1}^V\sum _{i=1}^n\sum _{j=1}^n\parallel h_i^{\ast (v)}-h_j^{\ast (v)}{\parallel }_2^2{S}_{i,j}+\gamma {\parallel S\parallel }_F^2\hfill \\ \hfill & s.t.S_{i,j}\ge 0,S_i{\mathbf{1}}^T=1\hfill \end{array}$$

(9)

where $h_i^{\ast (v)}$ and $h_j^{\ast (v)}$ is the i-th and j-th rows of normalized spectral embedding matrix $H^{\ast }$. The matrix $H^{\ast (v)}$ can be obtained by the following formula.

$$h_i^{\ast (v)}=\frac{h_i^{(v)}}{\sqrt{h_i^{(v)T}{h}_i^{(v)}}}$$

(10)

$H^{(v)}\in {\mathbb{R}}^{N\times c}$ is the spectral embedding matrix of the v-th view and can be obtained by the spectral clustering of the similarity graph of the v-th view.

$$\begin{array}{cc}\hfill & \underset{H^{(v)}}{max}tr(H^{(v)T}{G}^{(v)}{H}^{(v)})\hfill \\ \hfill & s.t.H^{(v)}{H}^{(v)T}=I\hfill \end{array}$$

(11)

where $G^{(v)}=D^{(v)-0.5}{W}^{(v)}{D}^{(v)-0.5}$, $D^{(v)}$ is a diagonal matrix calculated by $D_{i,i}^{(v)}={\sum }_{i=1}^n{W}_{i,j}^{(v)}$. The similarity graph ${\{W^{(v)}\}}_{v=1}^V$ is computed as

$$W_{i,j}^{(v)}=\frac{A_{i,k+1}^{\ast (v)}-A_{i,j}^{\ast (v)}}{k{A}_{i,k+1}^{\ast (v)}-{\sum }_{m=1}^k{A}_{i,m}^{\ast (v)}}$$

(12)

where $A_{i,j}$ is computed as $\parallel x_i^{(v)}-x_j^{(v)}{\parallel }_2^2$, then, $A^{(v)}$ is normalized to $A^{\ast (V)}$ and k denotes the near-neighbor parameters.

4. The Proposed Method

According to Equation (9), the quality of the similarity graphs S depends on $D_{i,j}^{h(v)}$, where $D_{i,j}^{h(v)}={\parallel h_i^{\ast (v)}-h_j^{\ast (v)}\parallel }_2^2=2-h_i^{\ast (v)}{h}_j^{\ast (v)T}$. However, Equation (11) neglects the high-order information embedded in multi-view data and principle information shared among multiple views. To overcome this limitation, a third-order tensor $\mathcal{B}\in {\mathbb{R}}^{n\times n\times V}$ is constructed by stacking each $H^{\ast (v)}{H}^{\ast (v)T}$. To better capture the correlations between different views, the dimension of $\mathcal{B}$ is rotated to $n\times V\times n$ [30,31,38]. It is expected that the third-order tensor $\mathcal{B}$ has a low-rank structure since the sample correlations are consistent across views. Based upon this, the optimization problem can be formulated as follows:

$$\begin{array}{cc}\hfill & \underset{H^{(v)},\mathcal{B}}{min}-\lambda \sum _{v=1}^{V}tr(H^{(v)T}{G}^{(v)}{H}^{(v)})+\tau {\parallel \mathcal{B}\parallel }_{\ast }\hfill \\ \hfill & s.t.H^{(v)T}{H}^{(v)}=I\hfill \end{array}$$

(13)

where $\lambda$ is the balance parameter, ${\parallel ·\parallel }_{\ast }$ denotes the tensor nuclear norm, I indicates the identity matrix and $\tau$ denotes the singular values threshold.

Despite the utilization of tensor nuclear norm (TNN) in Equation (13) to capture low-rank information, it overlooks the presence of errors and noise in the spectral embedding space, leading to the learning of an inaccurate consensus graph. Additionally, solely relying on TNN may not achieve perfect tensor low-rank approximation. One potential solution is to employ the weighted TNN [39], but this introduces the challenge of determining appropriate weight values. To address these issues, inspired by [15,16], a nonconvex approximation TNN (see Section 3.2), which assigns different thresholds to different singular values, is used to constrain $\mathcal{B}$. Furthermore, in order to mitigate the influence of noise in the spectral embedding space and prevent large noise values dominating the objective function, we proposed a novel norm called the $Cauchy$ norm (see Section 3.3) with an upper bound. The overall formulation of the proposed method is as follows:

$$\begin{array}{cc}\hfill \underset{H^{(v)},\mathcal{Z},\mathcal{E}}{min}& -\lambda \sum _{v=1}^{V}tr(H^{(v)T}{G}^{(v)}{H}^{(v)})+{\parallel \mathcal{Z}\parallel }_{\ast ,\theta }+\alpha {\parallel \mathcal{E}\parallel }_{Cauchy}\hfill \\ \hfill s.t.& H^{(v)T}{H}^{(v)}=I\hfill \\ \hfill & H^{\ast (v)}{H}^{\ast (v)T}=Z^{(v)}+E^{(v)}\hfill \\ \hfill & \mathcal{Z}=\mathsf{\Phi }(\mathsf{\Psi }({\{Z^{(v)}\}}_{v=1}^V))\hfill \\ \hfill & \mathcal{E}=\mathsf{\Psi }({\{Z^{(v)}\}}_{v=1}^V)\hfill \end{array}$$

(14)

where $G^{(v)}$ can be obtained by first calculating Equation (12) and then $D_{i,i}^{(v)}={\sum }_{i=1}^n{W}_{i,j}^{(v)}$. $\mathcal{Z}$, $\mathcal{E}$ and I denote the low-rank tensor, noise tensor and identity matrix, respectively. The $\mathsf{\Phi }(·)$ operation and $\mathsf{\Psi }(·)$ operation are tensor construction and rotation operations (see Section 3.1). $\lambda$ and $\alpha$ are balance parameters, $\theta$ is the hyperparameters of nonconvex approximation TNN.

5. Optimization Algorithm

5.1. Optimization Steps

An efficient alternating optimization algorithm is designed to solve problem (14). The optimization problem is equivalent to the following subproblems.

5.1.1. $H^{(v)}$ Subproblem

With other variables fixed, $H^{(v)}$ can be obtained by solving the following problem.

$$\begin{array}{cc}\hfill \underset{H^{(v)}}{argmin}& -\lambda \sum _{v=1}^{V}tr(H^{(v)T}{G}^{(v)}{H}^{(v)})\hfill \\ \hfill s.t.& H^{(v)T}{H}^{(v)}=I\hfill \\ \hfill & H^{\ast (v)}{H}^{\ast (v)T}=Z^{(v)}+E^{(v)}\hfill \end{array}$$

(15)

Optimizing the above problem is equivalent to optimizing each of the following subproblems.

$$\begin{array}{cc}\hfill \underset{H^{(v)}}{argmax}& \lambda tr(H^{(v)T}{G}^{(v)}{H}^{(v)})\hfill \\ \hfill s.t.& H^{(v)T}{H}^{(v)}=I\hfill \\ \hfill & H^{\ast (v)}{H}^{\ast (v)T}=Z^{(v)}+E^{(v)}\hfill \end{array}$$

(16)

With reference to the augmented Lagrangian function, problem (16) can be rewritten as

$$\begin{array}{cc}\hfill \underset{H^{(v)}}{argmin}& -\lambda tr(H^{(v)T}{G}^{(v)}{H}^{(v)})\hfill \\ \hfill & +\frac{\mu }{2}{\parallel H^{\ast (v)}{H}^{\ast (v)T}-(Z^{(v)}+E^{(v)}-\frac{Y^{(v)}}{\mu })\parallel }_F^2\hfill \\ \hfill s.t.& H^{(v)T}{H}^{(v)}=I\hfill \end{array}$$

(17)

where $Y^{(v)}$ is the multiplier and $\mu$ indicates the penalty parameter. Then, the problem (17) is rewritten as

$$\begin{array}{cc}\hfill \underset{H^{(v)}}{argmin}& -\lambda tr(H^{(v)T}{G}^{(v)}{H}^{(v)})\hfill \\ \hfill & +\frac{\mu }{2}tr(H^{\ast (v)}{H}^{\ast (v)T}{H}^{\ast (v)}{H}^{\ast (v)T})\hfill \\ \hfill & -\frac{\mu }{2}tr(H^{\ast (v)}{H}^{\ast (v)T}(T^{(v)}+T^{(v)T}))\hfill \\ \hfill s.t.& H^{(v)T}{H}^{(v)}=I\hfill \end{array}$$

(18)

where $T^{(v)}=Z^{(v)}+E^{(v)}-\frac{Y^{(v)}}{\mu }$. Similarly to [14], let $P^{(v)}\in {\mathbb{R}}^{n\times n}$ be a diagonal matrix. Its value on the diagonal can be calculated as follows.

$$P_{i,i}^{(v)}=\frac{1}{\sqrt{h_i^{(v)}{h}_i^{(v)T}}}$$

(19)

It is obvious that we have $H^{\ast (v)}=P^{(v)}{H}^{(v)}$. Integrating Equation (19) into problem (18) yields

$$\begin{array}{cc}\hfill \underset{H^{(v)}}{argmax}& tr(H^{(v)T}{Q}^{(v)}{H}^{(v)})\hfill \\ \hfill s.t.& H^{(v)T}{H}^{(v)}=I\hfill \end{array}$$

(20)

where $Q^{(v)}=\lambda G^{(v)}+\frac{\mu }{2}{P}^{(v)}(T^{(v)}+T^{(v)T}-H^{\ast (v)}{H}^{\ast (v)T})P^{(v)}$. According to [40], the optimal solution of (20) is to take the eigenvector corresponding to the c largest eigenvalues of $Q^{(v)}$.

5.1.2. $\mathcal{Z}$ Subproblem

With other variables fixed, $\mathcal{Z}$ is obtained by solving the following augmented Lagrangian function.

$$\begin{array}{cc}\hfill \underset{\mathcal{Z}}{argmin}& {\parallel \mathcal{Z}\parallel }_{\ast ,\theta }+\frac{\mu }{2}{\parallel \mathcal{Z}-\mathcal{T}\parallel }_F^2\hfill \end{array}$$

(21)

where ${\mathcal{T}}^{(v)}=H^{\ast (v)}{H}^{\ast (v)T}-E^{(v)}+\frac{Y^{(v)}}{\mu }$.

According to the proof in [16], the optimal solution of (21) is as follows.

$$\begin{array}{c}\hfill \mathcal{Z}=\mathcal{U}\ast ifft({\mathcal{S}}_{\frac{\nabla \varphi }{\rho }}(\overline{\mathsf{\Gamma }}))\ast {\mathcal{V}}^T\end{array}$$

(22)

where $\mathcal{T}=\mathcal{U}\ast \mathsf{\Gamma }\ast {\mathcal{V}}^T$ is the t-SVD of $\mathcal{T}$, $\overline{\mathsf{\Gamma }}=fft(\mathsf{\Gamma })$, ∗ denotes the t-product [35], ${\mathcal{S}}_{\frac{\nabla \varphi }{\rho }}{(\overline{\mathsf{\Gamma }})}^{(v)}=\mathrm{diag}\{max\{({\overline{\mathsf{\Gamma }}}_{i,i}^{(v)}-\frac{1}{\rho }\nabla \varphi ({\sigma }_i(\overline{\mathsf{\Gamma }})))\}\}$ and $\nabla \varphi$ indicates the first-order derivative of the nonconvex approximate function $\varphi$.

5.1.3. $\mathcal{E}$ Subproblem

With other variables fixed, the $\mathcal{E}$ subproblem can be formulated as follows.

$$\begin{array}{cc}\hfill \underset{\mathcal{E}}{argmin}& {\alpha \parallel \mathcal{E}\parallel }_{Cauchy}+\frac{\mu }{2}{\parallel \mathcal{E}-\mathcal{A}\parallel }_F^2\hfill \end{array}$$

(23)

where ${\mathcal{A}}^{(v)}=H^{\ast (v)}{H}^{\ast (v)T}-Z^{(v)}+\frac{Y^{(v)}}{\mu }$. We introduce a theorem to solve the above optimization problem.

Theorem 2.

Given a tensor $\mathcal{B}\in {\mathbb{R}}^{N\times M\times V}$, we give the optimization problem as

$$\underset{\mathcal{W}}{argmin}{\lambda \parallel \mathcal{W}\parallel }_{Cauchy}+\frac{\rho }{2}{\parallel \mathcal{W}-\mathcal{B}\parallel }_F^2$$

(24)

and the optimal solution of problem (24) is

$$\begin{array}{cc}\hfill & W_i=\left\{\begin{array}{cc}\frac{\parallel D_i{\parallel }_2-\frac{\gamma \lambda }{ϵ}}{\parallel D_i{\parallel }_2}{D}_i,\hfill & \parallel D_i{\parallel }_2>\frac{\gamma \lambda }{ϵ}\hfill \\ 0,\hfill & \parallel D_i{\parallel }_2\le \frac{\gamma \lambda }{ϵ}\hfill \end{array}\right.\hfill \\ \hfill & ϵ=\rho -\gamma \lambda ,D=\frac{\mu B}{ϵ}\hfill \end{array}$$

(25)

where $W={\mathcal{W}}_{(3)}$ and $B={\mathcal{B}}_{(3)}$ are the third matricization of $\mathcal{W}$ and $\mathcal{B}$, respectively. $W_i$ and $D_i$ denote the i-th column of W and D. γ is the scaling parameter of the $Cauchy$ norm.

Proof. 

Since we have the second-order Taylor expansions as $ln(1+x)=x-\frac{x^2}{2}+\mathsf{\Theta }({(x)}^2)$, where $\mathsf{\Theta }({(x)}^2)$ is the remainder term of Taylor’s expansion, ${\parallel \mathcal{W}\parallel }_{Cauchy}$ can then be expanded as follows.

$${\parallel \mathcal{W}\parallel }_{Cauchy}=\sum _{i=1}^n(\gamma \parallel W_i{\parallel }_2-\frac{(\gamma \parallel W_i{{\parallel }_2)}^2}{2}+\mathsf{\Theta }((\gamma \parallel W_i{\parallel }_2{)}^2))$$

(26)

There is a drawback to using the second-order Taylor expansion when x is large, as it can result in inaccuracies. To eliminate the effects of this drawback, $\gamma$ is set to a small value, such as 0.3. As shown in Figure 2, it can be observed that choosing appropriate scaling parameters (in order to stop them being too small to prevent the complete loss of information about the noise) satisfies the constant condition stated in Equation (26). Assuming an ideal value of $\gamma$ (which is often challenging to achieve in practice), taking Equation (26) into problem (24) gives

$$\underset{\mathcal{W}}{argmin}\sum _{i=1}^n(\gamma \lambda \parallel W_i{\parallel }_2-\frac{\lambda (\gamma \parallel W_i{{\parallel }_2)}^2}{2}+\frac{\rho }{2}\parallel W_i-B_i{\parallel }_2^2)$$

(27)

Let $ϵ=\mu -\gamma \lambda$, $D=\frac{\mu B}{ϵ}$, and Equation (27) can be simplified as follows.

$$\underset{\mathcal{W}}{argmin}\gamma \lambda \parallel W_i{\parallel }_{2,1}+\frac{ϵ}{2}{\parallel W-D\parallel }_F^2$$

(28)

According to [28], the optimal solution is

$$W_i=\left\{\begin{array}{cc}\frac{\parallel D_i{\parallel }_2-\frac{\gamma \lambda }{ϵ}}{\parallel D_i{\parallel }_2}{D}_i,\hfill & \parallel D_i{\parallel }_2>\frac{\gamma \lambda }{ϵ}\hfill \\ 0,\hfill & \parallel D_i{\parallel }_2\le \frac{\gamma \lambda }{ϵ}\hfill \end{array}\right.$$

(29)

   □

Based on the above theorem, the optimal solution of problem (23) is as follows.

$$\begin{array}{cc}\hfill & E_i=\left\{\begin{array}{cc}\frac{\parallel D_i{\parallel }_2-\frac{\gamma \alpha }{ϵ}}{\parallel D_i{\parallel }_2}{D}_i,\hfill & \parallel D_i{\parallel }_2>\frac{\gamma \alpha }{ϵ}\hfill \\ 0,\hfill & \parallel D_i{\parallel }_2\le \frac{\gamma \alpha }{ϵ}\hfill \end{array}\right.\hfill \\ \hfill & ϵ=\mu -\gamma \alpha ,D=\frac{\mu A}{ϵ}\hfill \end{array}$$

(30)

5.1.4. Remaining Steps

The Lagrange multipliers are updated as follows.

$$\begin{array}{cc}\hfill & Y^{(v)}=Y^{(v)}+\mu (H^{\ast (v)}{H}^{\ast (v)T}-Z^{(v)}-E^{(v)})\hfill \end{array}$$

(31)

The entire process is summarized in Algorithm 1. Once the similarity matrix S is obtained, the final clustering results are calculated by clustering S with spectral clustering method.

Algorithm 1: RLGMC for multi-view clustering

Require:  Multi-view data: $X^{(v)}\in {\mathbb{R}}^{n\times d_i}$, $v=1,2,\cdots ,V$, parameters $\lambda$, $\alpha$, $\theta$ and $\gamma$; 1: First, compute $W^{(v)}$ by Equation (12), second, calculate $D^{(v)}$ by ${\sum }_{i=1}^n{W}_{i,j}^{(v)}$, and finally, calculate $G^{(v)}=D^{(v)-0.5}{W}^{(v)}{D}^{(v)-0.5}$. 2: Set $k=0$, ${\mu }^0={10}^{-3}$, $\beta =2$, ${\mathcal{Z}}^0={\mathcal{E}}^0={\mathcal{Y}}^0=H^{(v)}=0$, ${\mu }^{max}={10}^{10}$, and $\epsilon ={10}^{-6}$; 3: while not converged do 4:     for each $v\in [1,V]$ do 5:         Update $H^{(v),k+1}$ by solving problem (20); 6:     end for 7:     Obtained $\mathcal{Z}=\mathsf{\Phi }(\psi (Z^1,Z^2,\cdots ,Z^V))$; 8:     Update ${\mathcal{Z}}^{k+1}$ by Equation (22); 9:     Update ${\mathcal{E}}^{k+1}$ by Equation (30); 10:     for each $v\in [1,V]$ do 11:         Update $Y^{(v)k+1}$ by Equation (31); 12:     end for 13:     Update ${\mu }^{k+1}$ by ${\mu }^{k+1}=\min (\beta {\mu }^k,{\mu }_{max})$; 14:     Check the convergence conditions: $$\begin{array}{c}\hfill \parallel H^{\ast (v),k+1}{H}^{\ast (v)T,k+1}-Z^{(v),k+1}-E^{(v),k+1}{\parallel }_{\infty }\le \epsilon \end{array}$$ 15:     $k=k+1$; 16: end while 17: Construct consensus graph S by Equation (9); Ensure:  Robust consensus graph S;

5.2. Computational Complexity

The computational complexity of Algorithm 1 is analyzed as follows: updating $H^v$ needs to calculate the eigenvectors of $G^{(v)}$ which takes $\mathcal{O}(c{n}^2)$ on each view. Since FFT operation and IFFT operation totally cost $\mathcal{O}(V{n}^{2}log(n))$, and computing the SVD of each frontal slice of $\overline{\mathsf{\Gamma }}$ takes $\mathcal{O}(V^2{n}^2)$, the total complexity of updating $\mathcal{Z}$ is $\mathcal{O}(V{n}^{2}log(n)+V^2{n}^2)$. Furthermore, then, updating $\mathcal{E}$ takes $\mathcal{O}(n^{2}V)$ per iteration. The spectral clustering method cost $\mathcal{O}(n^3)$. To this end, the computational complexity of RLGMC for multi-view clustering is $\mathcal{O}(K(Vc{n}^2+V{n}^{2}log(n)+V^2{n}^2+n^{2}V)+n^3)\approx \mathcal{O}(KV{n}^{2}log(n)+n^3)$, where K denotes the number of iterations.

6. Experiments

6.1. Datasets

In this section, six multi-view datasets are used to evaluate the performance of the proposed model. The information from all the datasets is summarized in

Table 2. Information of Six Datasets.

Dataset	Objective	Instances	Clusters	View
ORL	Face	400	40	3
20newsgroups	Document	500	5	3
COIL-20	Object	1440	20	3
100leaves	Plant species	1600	100	3
UCI digits	Digit	2000	10	3
Handwritten	Digit	2000	10	6

. Moreover, we briefly introduce the six datasets as follows:

• ORL (http://www.uk.research.att.com/facedatabase.html, 15 January 2023) consists of 400 face images from 40 people. Three kinds of features including Intensity, LBP and Gabor are extracted.
• 20newsgroups (http://lig-membres.imag.fr/grimal/data.html, 15 January 2023) consists of five classes and 500 documents, with three views which are extracted.
• COIL-20 (https://www.cs.columbia.edu/CAVE/software/softlib/, 15 January 2023) includes 1440 images from 20 objects. Intensity, LBP and Gabor are extracted.
• 100leaves (https://archive.ics.uci.edu/ml/datasets/One-hundred+plant+species+leaves+data+set, 15 February 2023) contains 100 plant species with a total of 1600 samples. Three views consisting of the shape descriptor, texture histogram and fine scale margin are extracted.
• UCI digits (http://mlg.ucd.ie/datasets, 15 February 2023) consists of 2000 digital images with 10 classes. Similarly to [41], three types of features are extracted.
• Handwritten (https://archive.ics.uci.edu/ml/machine-learning-databases/optdigits/, 15 February 2023) consists of 2000 handwritten digital images. There are six types of features (FOU, FAC, KAR, Pix, ZER, MOR) being extracted.

6.2. Comparison Methods

The method proposed in this paper is compared with the following methods.

• SSC [27]: Sparse subspace clustering explores the sparse representation. The parameters of the SSC are set to the default parameters given in the program. Since it is a single-view method, the result is obtained from the first view.
• LRR [28]: Low-rank representation explores the low-rank subspace representation by nuclear norm. For LRR, the parameter is selected from [0.1, 0.5, ⋯, 4.5, 4.9]. Since it is a single-view method, the result is obtained from the first view.
• LT-MSC [29]: LT-MSC explores the high-order correlation of the multi-view using a low-rank constraint. For LT-MSC, the parameter is selected from [${10}^{-3}$, ${10}^{-2}$, ⋯, ${10}^1$, ${10}^2$].
• t-SVD-MSC [30]: tSVDMSC learns low-rank subspace representation by TNN and tensor rotation operation. For t-SVD-MSC, the parameter is selected from [${10}^{-2}$, ${10}^{-1}$, ⋯, ${10}^1$, ${10}^2$].
• ETLMSC [31]: ETLMSC learns the essential tensor constrained by TNN from the transition probability matrices. Then, the final clustering results are obtained through the Markov chain-based spectral clustering method. The parameter of ETLMSC is tuned from 0.001 to 5.
• MCGC [42]: MCGC reduces the disagreements among multiple views by a co-regularization term. The parameters of MCGC are set to 0.6.
• GMC [43] (https://github.com/cshaowang/gmc, 15 January 2023): GMC proposes a joint framework consisting of the learning of the similarity-induced graph, the learning of the unified graph and clustering tasks. For GMC, the default parameter is 1.
• DGF [44] (https://github.com/youweiliang/ConsistentGraphLearning, 15 January 2023): DGF learns the consistency and inconsistency among multiple views in a unified optimization model. For DGF, two parameters are selected from [${10}^{-6}$, ${10}^{-4}$] and [${10}^4$, ${10}^6$], respectively.
• CGL [14] (https://github.com/guanyuezhen/CGL, 15 January 2023): CGL simultaneously learns spectral embedding and explores the low-rank tensor representation. The parameters of CGL are selected from [1, 5, 10, 50, 100, 500, 1000, 5000].

6.3. Evaluation Metrics

To evaluate the performance of all methods, the following metrics are adopted.

Normalized mutual information (NMI) measures the similarity between predicted partition and the truth partition. The NMI is calculated as:

$$NMI(C,C^{\prime })=\frac{{\sum }_{i=1}^K{\sum }_{j=1}^S|C_i\cap C_j^{\prime }|log\frac{N|C_i\cap C_j^{\prime }|}{|C_i||C_j^{\prime }|}}{\sqrt{({\sum }_{i=1}^K|C_i|log\frac{C_i}{N})({\sum }_{j=1}^S|C_j^{\prime }|log\frac{C_j^T}{N})}}$$

(32)

where C and $C^{\prime }$ represent the predicted partition and the truth partition, respectively. Accuracy (ACC) measures the percentage of samples that are correctly predicted. It can be defined as follows:

$$ACC=\frac{{\sum }_{i=1}^N\delta (t_i,map(p_i))}{N}$$

(33)

where $t_i$ and $p_i$ denote the true label and predicted label, respectively. Function $map(·)$ denotes the best permutation mapping function [45]. $\delta$ is formulated as follows:

$$\delta (t,p)=\left\{\begin{array}{c}1,t=p\hfill \\ 0,t\ne p\hfill \end{array}\right.$$

(34)

The remaining metrics including precision, recall, F-score and AR, view clustering as a series of decisions, calculated from the different perspectives of decisions. The details about their definitions can be found in [46].

The higher values of the above six metrics indicate better performance. All methods including the proposed method were performed ten times for clustering, and then the average and standard deviation of the metrics are obtained.

6.4. Parameter Settings

For the proposed method, $\lambda$ is tuned in the range of [1, 5, 10, 15, 20, 50, 100, 500, 1000, 2000, 3000, 5000], $\alpha$ is tuned in the range of [1, 5, 10, 50, 100, 500, 1000], $\theta$ is tuned in the range of [0.5, 1, 1.2, 1.5, 2, 2.5, 3, 3.5, 4] and the scaling parameter $\gamma$ is fixed as $0.2$.

6.5. Analysis of Clustering Results

As shown in

Table 3. The comparison results on six datasets.

Datasets	Methods	ACC	NMI	Precision	Recall	F-Score	AR
ORL	SSC	$0.5803\pm 0.0172$	$0.7658\pm 0.0099$	$0.6442\pm 0.0159$	$0.3861\pm 0.0210$	$0.5323\pm 0.0229$	$0.4474\pm 0.0203$
	LRR	$0.6407\pm 0.0198$	$0.7935\pm 0.0104$	$0.5518\pm 0.0274$	$0.5012\pm 0.0181$	$0.5252\pm 0.0219$	$0.5137\pm 0.0223$
	LT-MSC	$0.7718\pm 0.0312$	$0.8992\pm 0.0090$	$0.6474\pm 0.0482$	$0.7722\pm 0.0193$	$0.7039\pm 0.0350$	$0.6964\pm 0.0361$
	t-SVD-MSC	$0.7998\pm 0.0207$	$0.8967\pm 0.0098$	$0.7052\pm 0.0253$	$0.7712\pm 0.0205$	$0.7367\pm 0.0224$	$0.7303\pm 0.0230$
	ETLMSC	$0.7230\pm 0.0433$	$0.8702\pm 0.0173$	$0.5984\pm 0.0509$	$0.7033\pm 0.0401$	$0.6463\pm 0.0456$	$0.6374\pm 0.0469$
	MCGC	$0.6625\pm 0.0000$	$0.7888\pm 0.0000$	$0.2290\pm 0.0000$	$0.7428\pm 0.0000$	$0.3501\pm 0.0000$	$0.3269\pm 0.0000$
	GMC	$0.6325\pm 0.0000$	$0.8035\pm 0.0000$	$0.2321\pm 0.0000$	$0.8011\pm 0.0000$	$0.3599\pm 0.0000$	$0.3367\pm 0.0000$
	CGL	$\underline{0.8279\pm 0.0180}$	$\underline{0.9159\pm 0.0046}$	$\underline{0.7343\pm 0.0195}$	$\mathbf{0.8110}\pm \mathbf{0.0115}$	$\underline{0.7707\pm 0.0149}$	$\underline{0.7651\pm 0.0153}$
	DGF	$0.6165\pm 0.0168$	$0.7976\pm 0.0056$	$0.4387\pm 0.0133$	$0.5927\pm 0.0109$	$0.5041\pm 0.0115$	$0.4909\pm 0.0118$
	Ours	$\mathbf{0.8533}\pm \mathbf{0.0049}$	$\mathbf{0.9185}\pm \mathbf{0.0024}$	$\mathbf{0.8222}\pm \mathbf{0.0057}$	$\underline{0.7657\pm 0.0056}$	$\mathbf{0.7929}\pm \mathbf{0.0066}$	$\mathbf{0.7880}\pm \mathbf{0.0068}$
20newsgroups	SSC	$0.2248\pm 0.0010$	$0.0211\pm 0.0007$	$0.2268\pm 0.0010$	$0.1986\pm 0.0000$	$0.9221\pm 0.0028$	$0.3268\pm 0.0001$
	LRR	$0.2080\pm 0.0000$	$0.0080\pm 0.0000$	$0.9840\pm 0.0000$	$0.1984\pm 0.0000$	$0.3302\pm 0.0000$	$0.0000\pm 0.0000$
	LT-MSC	$0.2040\pm 0.0000$	$0.0155\pm 0.0000$	$0.1984\pm 0.0000$	$0.9841\pm 0.0000$	$0.3302\pm 0.0000$	$0.0000\pm 0.0000$
	t-SVD-MSC	$\mathbf{0.9820}\pm \mathbf{0.0000}$	$0.9391\pm 0.0000$	$\mathbf{0.9642}\pm \mathbf{0.0000}$	$\mathbf{0.9643}\pm \mathbf{0.0000}$	$\mathbf{0.9643}\pm \mathbf{0.0000}$	$\mathbf{0.9554}\pm \mathbf{0.0000}$
	ETLMSC	$0.3830\pm 0.0327$	$0.2090\pm 0.0314$	$0.2896\pm 0.0196$	$0.3772\pm 0.0415$	$0.3273\pm 0.0264$	$0.1330\pm 0.0305$
	MCGC	$0.2880\pm 0.0000$	$0.0998\pm 0.0000$	$0.2049\pm 0.0000$	$0.7595\pm 0.0000$	$0.3228\pm 0.0000$	$0.0150\pm 0.0000$
	GMC	$\mathbf{0.9820}\pm \mathbf{0.0000}$	$\mathbf{0.9392}\pm \mathbf{0.0000}$	$\mathbf{0.9642}\pm \mathbf{0.0000}$	$\mathbf{0.9643}\pm \mathbf{0.0000}$	$\mathbf{0.9643}\pm \mathbf{0.0000}$	$\mathbf{0.9554}\pm \mathbf{0.0000}$
	CGL	$\underline{0.9800\pm 0.0000}$	$\underline{0.9363\pm 0.0000}$	$\underline{0.9600\pm 0.0000}$	$\underline{0.9608\pm 0.0000}$	$\underline{0.9604\pm 0.0000}$	$\underline{0.9506\pm 0.0000}$
	DGF	$0.2924\pm 0.0013$	$0.0984\pm 0.0015$	$0.2219\pm 0.0000$	$0.6520\pm 0.0001$	$0.3312\pm 0.0001$	$0.0499\pm 0.0001$
	Ours	$0.9624\pm 0.0008$	$0.8844\pm 0.0016$	$0.9265\pm 0.0015$	$0.9255\pm 0.0017$	$0.9260\pm 0.0016$	$0.9077\pm 0.0020$
COIL-20	SSC	$0.7726\pm 0.0004$	$0.9015\pm 0.0000$	$0.8431\pm 0.0000$	$0.6770\pm 0.0000$	$\mathbf{0.8832}\pm \mathbf{0.0001}$	$0.7665\pm 0.0000$
	LRR	$0.7088\pm 0.0153$	$0.8007\pm 0.0129$	$0.6933\pm 0.0194$	$0.6143\pm 0.0239$	$0.6513\pm 0.0202$	$0.6321\pm 0.0215$
	LT-MSC	$0.7751\pm 0.0202$	$0.8709\pm 0.0098$	$0.7277\pm 0.0228$	$0.7853\pm 0.0194$	$0.7552\pm 0.0177$	$0.7420\pm 0.0187$
	t-SVD-MSC	$0.7433\pm 0.0100$	$0.8163\pm 0.0076$	$0.6869\pm 0.0153$	$0.7048\pm 0.0136$	$0.6957\pm 0.0144$	$0.6797\pm 0.0152$
	ETLMSC	$0.7353\pm 0.0227$	$0.8272\pm 0.0081$	$0.6862\pm 0.0258$	$0.7209\pm 0.0145$	$0.7029\pm 0.0166$	$0.6871\pm 0.0177$
	MCGC	$0.7986\pm 0.0000$	$0.8839\pm 0.0000$	$0.7196\pm 0.0000$	$0.8393\pm 0.0000$	$0.7748\pm 0.0000$	$0.7622\pm 0.0000$
	GMC	$0.7910\pm 0.0000$	$0.9189\pm 0.0000$	$0.6938\pm 0.0000$	$\underline{0.9287\pm 0.0000}$	$0.7943\pm 0.0000$	$0.7819\pm 0.0000$
	CGL	$0.8404\pm 0.0065$	$0.9209\pm 0.0000$	$\underline{0.7958\pm 0.0059}$	$0.8868\pm 0.0000$	$0.8235\pm 0.0000$	$0.8137\pm 0.0000$
	DGF	$\underline{0.8542\pm 0.0000}$	$\mathbf{0.9450}\pm \mathbf{0.0000}$	$0.7574\pm 0.0000$	$\underline{0.9497\pm 0.0000}$	$0.8427\pm 0.0000$	$\underline{0.8336\pm 0.0000}$
	Ours	$\mathbf{0.8811}\pm \mathbf{0.0004}$	$\mathbf{0.9452}\pm \mathbf{0.0000}$	$\mathbf{0.8992}\pm \mathbf{0.0001}$	$0.8557\pm 0.0001$	$\underline{0.8769\pm 0.0001}$	$\mathbf{0.8704}\pm \mathbf{0.0001}$
100leaves	SSC	$0.5748\pm 0.0090$	$0.7704\pm 0.0032$	$0.6089\pm 0.0054$	$0.4087\pm 0.0093$	$0.4693\pm 0.0109$	$0.4369\pm 0.0095$
	LRR	$0.4933\pm 0.0182$	$0.7204\pm 0.0073$	$0.3664\pm 0.0150$	$0.3322\pm 0.0151$	$0.3484\pm 0.0150$	$0.3420\pm 0.0151$
	LT-MSC	$0.7167\pm 0.0152$	$0.8646\pm 0.0068$	$0.5987\pm 0.0182$	$0.6606\pm 0.0179$	$0.6281\pm 0.0176$	$0.6244\pm 0.0178$
	t-SVD-MSC	$0.7479\pm 0.0173$	$0.8744\pm 0.0079$	$0.6277\pm 0.0196$	$0.6995\pm 0.0214$	$0.6616\pm 0.0200$	$0.6582\pm 0.0202$
	ETLMSC	$0.7274\pm 0.0145$	$0.8923\pm 0.0069$	$0.6191\pm 0.0166$	$0.7393\pm 0.0174$	$0.6738\pm 0.0158$	$0.6704\pm 0.0159$
	MCGC	$0.7381\pm 0.0000$	$0.8353\pm 0.0000$	$0.2707\pm 0.0000$	$0.7391\pm 0.0000$	$0.3963\pm 0.0000$	$0.3879\pm 0.0000$
	GMC	$0.8238\pm 0.0000$	$0.9025\pm 0.0000$	$0.3521\pm 0.0000$	$0.8874\pm 0.0000$	$0.5042\pm 0.0000$	$0.4974\pm 0.0000$
	CGL	$\underline{0.9609\pm 0.0074}$	$\underline{0.9805\pm 0.0016}$	$\underline{0.9247\pm 0.0113}$	$\mathbf{0.9567}\pm \mathbf{0.0037}$	$\underline{0.9404\pm 0.0072}$	$\underline{0.9398\pm 0.0073}$
	DGF	$0.7294\pm 0.0154$	$0.8823\pm 0.0023$	$0.6044\pm 0.0098$	$0.7479\pm 0.0060$	$0.6685\pm 0.0068$	$0.6650\pm 0.0069$
	Ours	$\mathbf{0.9696}\pm \mathbf{0.0091}$	$\mathbf{0.9831}\pm \mathbf{0.0024}$	$\mathbf{0.9636}\pm \mathbf{0.0032}$	$0.9396\pm 0.0163$	$\mathbf{0.9514}\pm \mathbf{0.0089}$	$\mathbf{0.9509}\pm \mathbf{0.0090}$
UCI digits	SSC	$0.6697\pm 0.0007$	$0.7711\pm 0.0007$	$0.7594\pm 0.0006$	$0.6376\pm 0.0005$	$0.7954\pm 0.0009$	$0.7078\pm 0.0005$
	LRR	$0.7794\pm 0.0002$	$0.7619\pm 0.0001$	$0.7585\pm 0.0001$	$0.7017\pm 0.0001$	$0.7290\pm 0.0001$	$0.6977\pm 0.0002$
	LT-MSC	$0.8649\pm 0.0186$	$0.8223\pm 0.0018$	$0.7983\pm 0.0125$	$0.8198\pm 0.0021$	$0.8089\pm 0.0074$	$0.7874\pm 0.0084$
	t-SVD-MSC	$0.9669\pm 0.0005$	$0.9343\pm 0.0007$	$\underline{0.9345\pm 0.0010}$	$0.9377\pm 0.0008$	$\underline{0.9361\pm 0.0009}$	$\underline{0.9290\pm 0.0010}$
	ETLMSC	$0.9088\pm 0.0758$	$\underline{0.9447\pm 0.0399}$	$0.8825\pm 0.0949$	$0.9413\pm 0.0459$	$0.9102\pm 0.0712$	$0.8996\pm 0.0796$
	MCGC	$0.8570\pm 0.0000$	$0.8382\pm 0.0000$	$0.7613\pm 0.0000$	$0.9076\pm 0.0000$	$0.8280\pm 0.0000$	$0.8071\pm 0.0000$
	GMC	$0.7505\pm 0.0000$	$0.8096\pm 0.0000$	$0.6814\pm 0.0000$	$0.8179\pm 0.0000$	$0.7434\pm 0.0000$	$0.7122\pm 0.0000$
	CGL	$\underline{0.8796\pm 0.0011}$	$0.9154\pm 0.0077$	$0.8008\pm 0.0050$	$\underline{0.9490\pm 0.0126}$	$0.8686\pm 0.0082$	$0.8527\pm 0.0091$
	DGF	$0.4647\pm 0.0004$	$0.4906\pm 0.0002$	$0.3806\pm 0.0003$	$0.3883\pm 0.0005$	$0.3844\pm 0.0004$	$0.3156\pm 0.0004$
	Ours	$\mathbf{0.9775}\pm \mathbf{0.0000}$	$\mathbf{0.9464}\pm \mathbf{0.0000}$	$\mathbf{0.9558}\pm \mathbf{0.0000}$	$\mathbf{0.9551}\pm \mathbf{0.0000}$	$\mathbf{0.9554}\pm \mathbf{0.0000}$	$\mathbf{0.9505}\pm \mathbf{0.0000}$
Handwritten	SSC	$0.7958\pm 0.0009$	$0.8117\pm 0.0012$	$0.8384\pm 0.0009$	$0.7146\pm 0.0016$	$0.8263\pm 0.0015$	$0.7664\pm 0.0015$
	LRR	$0.7088\pm 0.0153$	$0.8007\pm 0.0129$	$0.6933\pm 0.0194$	$0.6143\pm 0.0239$	$0.6513\pm 0.0202$	$0.6321\pm 0.0215$
	LT-MSC	$0.9380\pm 0.0002$	$0.8854\pm 0.0001$	$0.8832\pm 0.0002$	$0.8857\pm 0.0002$	$0.8844\pm 0.0002$	$0.8716\pm 0.0003$
	t-SVD-MSC	$\mathbf{0.9995}\pm \mathbf{0.0000}$	$\mathbf{0.9986}\pm \mathbf{0.0000}$	$\mathbf{0.9990}\pm \mathbf{0.0000}$	$\mathbf{0.9990}\pm \mathbf{0.0000}$	$\mathbf{0.9990}\pm \mathbf{0.0000}$	$\mathbf{0.9989}\pm \mathbf{0.0000}$
	ETLMSC	$0.9306\pm 0.0966$	$\underline{0.9749\pm 0.0346}$	$0.9162\pm 0.1144$	$0.9756\pm 0.0325$	$0.9430\pm 0.0779$	$0.9361\pm 0.0876$
	MCGC	$0.9710\pm 0.0000$	$0.9330\pm 0.0000$	$0.9420\pm 0.0000$	$0.9435\pm 0.0000$	$0.9427\pm 0.0000$	$0.9364\pm 0.0000$
	GMC	$0.8820\pm 0.0000$	$0.8932\pm 0.0000$	$0.8260\pm 0.0000$	$0.9085\pm 0.0000$	$0.8653\pm 0.0000$	$0.8496\pm 0.0000$
	CGL	$0.9750\pm 0.0000$	$0.9455\pm 0.0000$	$0.9500\pm 0.0000$	$0.9510\pm 0.0000$	$0.9507\pm 0.0000$	$0.9452\pm 0.0000$
	DGF	$0.8110\pm 0.0000$	$0.7832\pm 0.0000$	$0.7116\pm 0.0000$	$0.7490\pm 0.0000$	$0.7298\pm 0.0000$	$0.6991\pm 0.0000$
	Ours	$\underline{0.9755\pm 0.0000}$	$0.9467\pm 0.0000$	$\underline{0.9521\pm 0.0000}$	$\underline{0.9511\pm 0.0000}$	$\underline{0.9516\pm 0.0000}$	$\underline{0.9463\pm 0.0000}$

The best clustering results are highlighted in bold and the second best results underlined.

, the superiority of the multi-view models compared to the single-view models indicates that complementary information in multiple views is necessary for clustering tasks.

Compared with the three subspace learning methods (LT-MSC, t-SVD-MSC, ETLMSC), the proposed method outperforms LT-MSC, t-SVD-MSC, and ETLMSC on the ORL, COIL-20, 100leaves, and UCI digits datasets. For example, the proposed method improves accuracy by 13.03%, 5.35% and 8.15% over ETLMSC, t-SVD-MSC and LT-MSC. On the Handwritten and 20newsgroups datasets, the proposed method is very close to t-SVD-MSC, with a slightly lower accuracy, which is 1.96% and 2.4% less than t-SVD-MSC in ACC on the two datasets, respectively. These results highlight the effectiveness of the proposed method in learning accurate similarity by using spectral embedding and low-rank tensor approximation.

Compared with other graph-based methods, the proposed method has obvious advantages. The proposed method improves the COIL-20 accuracy by 2.69%, 4.07%, 9.01% and 8.25% over DGF, CGL, GMC and MCGC. On the UCI digits dataset, the proposed method achieves 9.79%, 1.7%, 2.13%, 0.61%, 1.93% and 2.15% improvement over the second method in terms of ACC, NMI, precision, recall, F-score and AR. On the 20newsgroups dataset, although there is a slight disadvantage in ACC compared to CGL and GMC, the proposed method still provides improvements compared to the inspired method (CGL). On the ORL dataset, the proposed method has 2.54% and 0.26% improvement in terms of ACC and NMI compared to CGL. The results illustrate that utilizing a nonconvex function to achieve the low-rank tensor learning and Cauchy norm to handle noise can refine the clustering performance.

The visualizations of the learned similarity graphs on the ORL and Handwritten datasets are provided in Figure 3 and Figure 4. Compared with other methods, the block structure of the similarity graph learned by the proposed method is the clearest among all the methods, indicating its suitability for clustering tasks.

6.6. Parameter Sensitivity and Time Complexity Analysis

There are four hyperparameters in our method, i.e., two balance parameters $\lambda$ and $\alpha$, a nonconvex function parameter $\theta$ and a scale parameter $\gamma$. The parameters are analyzed on six datasets, $\lambda$ is tuned from [5, 10, 15, 20, 50, 100, 500, 1000, 5000], $\alpha$ is tuned from [1, 5, 10, 50, 100, 500, 1000, 5000], the range of $\theta$ is [0.5, 1, 2, 3, 4] and $\gamma$ is chosen from [0.001, 0.01, 0.1, 0.2, 0.25]. We only display the ACC values for different combinations of parameters.

First, we analyze $\lambda$ and $\alpha$ with fixed $\theta =2.8$ and $\gamma =0.2$, and the results are shown in Figure 5. Then, $\theta$ and $\gamma$ are analyzed with the optimal $\lambda$ and $\alpha$, and the results are shown in Figure 6. In Figure 5, the x and y axes denote the values of $\lambda$ and $\alpha$. In Figure 6, the x and y axes denote the values of $\theta$ and $\gamma$. In conclusion, the proposed method is more stable in the other datasets than in the 20newsgroups and 100leaves datasets.

We show the running time of different methods on each dataset in

Table 4. The running time comparisons on six datasets.

Time (s)	Methods	ORL	20newsgroups	COIL-20	100leaves	UCI Digits	Handwritten
Datasets
SSC		7.51	7.03	29.85	22.30	62.12	71.74
LRR		18.61	42.22	242.61	172.55	341.30	342.05
LT-MSC		51.70	51.85	633.59	420.87	924.88	1083.70
t-SVD-MSC		34.38	31.88	406.90	273.93	198.71	627.44
ETLMSC		0.90	3.69	16.12	48.69	66.32	66.47
MCGC		0.46	0.68	5.57	13.89	15.77	14.49
GMC		0.49	0.56	7.19	5.01	28.22	12.72
CGL		10.50	9.17	76.13	239.89	152.11	305.81
DGF		0.30	0.28	3.65	0.62	1.65	2.14
Ours		2.51	1.63	25.14	33.07	42.53	82.61

. Graph-based methods are more time-saving than subspace learning methods. Compared with CGL, the proposed method runs faster, indicating that the designed optimization algorithm is more efficient.

7. Conclusions

In this paper, we propose a novel multi-view graph-based method named RLGMC. Firstly, the embedding matrices are constructed using spectral embedding. Secondly, the gram matrices obtained from the inner product of normalized embedding matrices are reorganized into a tensor. Subsequently, we employ a nonconvex low-rank tensor approximation to capture the high-order principle information among multiple views. To handle the noise in the spectral embedding space, a novel Cauchy norm with an upper bound is introduced, which suppresses noise by the Cauchy function and the scaling parameter. By integrating spectral embedding, the nonconvex low-rank tensor approximation and the Cauchy norm into an unified framework, the robust low-rank tensor is obtained. Finally, the consensus similarity graph is obtained from the robust low-rank tensor. Experimental results on six datasets demonstrate that the proposed method achieves superior performance and represents an effective improvement over the inspired method.

Although the proposed method achieves excellent performance, it has two main drawbacks. Firstly, determining the value of nonconvex function parameter $\theta$ in practical applications is challenging. Secondly, the optimization step of nonconvex approximation TNN, which involves singular value decomposition, can be time-consuming. We aim to address these deficiencies in future research.