Federated Multi-View Spectral Clustering

Multi-view spectral clustering (MVSC) has become a popular approach to harvest knowledge about group information from multiple views of data, owned by different parties. A high quality MVSC approach usually requires collecting massive amount of data from each view party, in order to perform MVSC algorithm in a centralized manner. However, such centralized MVSC approach raises serious privacy concerns, not only in terms of the sensitivity property of many real-world data such as medical or financial records, but also in terms of the regulations from authorities to preclude centralized operations. Hence it is crucial to design new paradigm for training spectral clustering model on multi-view data in industrial scenarios. In this article, we propose a distributed and secure framework named Federated Multi-view Spectral Clustering (FMSC), in which a group of view parties collaboratively perform a MVSC model, but couldn’t learn the data of other participants. FMSC is inspired by the concept of federated learning, and utilize Homomorphic Encryption (HE) and Differential Privacy (DP) to achieve secure and private clustering. We conduct a series of extensive experiments to verify the effectiveness of FMSC on both synthetic and real-world datasets. Evaluations show that FMSC achieves respectable clustering results over conventional centralized approaches.


I. INTRODUCTION
Multi-view clustering has been receiving considerable attentions in recent years. The reason comes from the fact that objects are usually presented in different views. These views, held by same or different parties, often contain partial information of different domains of features associated with the same objects. For example, a webpage can be organized by images, texts and audios. In the real world, human activity may be captured by street cameras, GPS sensors, transportation card records, tweets on tweeter, etc. On the scenario of Internet of Things (IoT), multi-view data is more likely observed, collected and even stored by local edge devices.
Performing clustering algorithms on multi-view data usually requires centralizing these data and exploiting all features of diverse views. Unlike traditional single-view clustering (SVC) approaches to concatenate all views of features The associate editor coordinating the review of this manuscript and approving it for publication was Lefei Zhang . together, multi-view clustering (MVC) aims to cluster data directly from diverse views. Compared to SVC, MVC can leverage more structural and complementary information hidden in multi-view data to improve the clustering performance [1]. Existing MVC approaches can be categorized into two classes, namely, projection-based approaches and similarity-based approaches. Projection-based approaches [2]- [4] project each view with a compatible matrix into a common low-dimensional feature subspace, and then apply a standard clustering method (e.g., k-means) in the projected subspace. More extensive studies have focused on similaritybased approaches [5]- [7], which learn a clustering structure for different views by considering both within-view and between-view similarities. Among these approaches, multiview spectral clustering (MVSC) [7]- [11] has earned a lot of interests, and has shown encouraging results in comparison to others.
These pioneering approaches demonstrate the promise of multi-view clustering task, but suffer from a major limitation: VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ multi-view data should be collected to a central server to perform the clustering algorithms, without considering the possible leakage of privacy from sensitive data owned by each view party. In fact, in many applications multi-view data are collected on distributed parties, each with one view data [12], [13]. For example, a user may have credit accounts in a number of banks. User's financial activities in different banks are very sensitive to be collected into a server to get a more robust clustering model. In recent years, several regulations have been proposed to prohibit privacy leakage in commercial scenarios, such as European Union General Data Protection Regulation (GDPR) [14]. Thus, these approaches raise a critical question: how to derive a global multi-view spectral clustering structure in each view-data party if the view specific data are located on diverse distributed parties? To mitigate this issue, we should keep the following regulations which are very challenging: 1) No party should share its private view data to others, including, but not limited to other multi-view data owners and the central server. 2) To minimize information leakage, sensitive local parameters should also be impeded to directly share to others without any privacy preserving mechanism.
3) The performance of model proposed shouldn't degenerate too much to the model trained in a centralized manner. Inspired by the growing popularity of Federated Learning [15], which investigates a distributed architecture that a group of view data parties jointly train a machine learning model and nobody can learn the data of other participants, we propose a new approach named Federated Multi-view Spectral Clustering (FMSC) that addresses the aforementioned challenges in a principled way. Our approach has several contributions as follows.
• FMSC exploits a central server only to compute aggregated statistics of sensitive view specific parameters but in a secure manner: each party can encrypt individual parameters, namely, the discriminative eigenvectors of each view data, with Homomorphic Encryption. Every party receives and decrypts the encrypted aggregation of global eigenvectors, and then computes its local clustering structure at each iteration until convergence.
• We also utilize a Differential Privacy (DP) mechanism to prevent the information leakage between view parties when there are few parties involved. Under FMSC, we demonstrate that no private information has leaked to the central server and little view specific information has leaked to other view parties with theoretical assurance.
• We perform extensive experiments on both synthetic and real world benchmark datasets. Experiment results show that predictive accuracy of FMSC is almost approaching to the traditional centralized methods. The remainder of this article is organized as follows. Section II gives a brief review of related work on multi-view clustering and federated learning. In section III, we propose the architecture of our FMSC approach in details and address the privacy challenges. To evaluate the proposed approach, we conduct extensive experiments on a variety of datasets and demonstrate the effectiveness of our approach in Section IV. The conclusion is drawn in Section 5.

II. RELATED WORK
The present work is mainly based on a wide range of literature. In this section, we outline the most related works including multi-view spectral clustering and federated learning respectively.
A. MULTI-VIEW SPECTRAL CLUSTERING Spectral clustering [16], [17] is a popular clustering technique that has shown good performance on arbitrary shaped data. It is extensively studied as a significant framework by many literatures on multi-view datasets. Reference [18] proposed a random walk based solution to derive a global graph cut over the single graphs in other views. In [7], the authors applied the idea of co-training and proposed a Co-Training Multi-view Spectral Clustering (CTMSC), where the new graph similarity of one view is limited by solving the eigenvectors of the Laplacian of other views. Reference [8] added a co-regularization term to enforce the eigenvectors of diverse views having high pairwise similarities, and proposed a method called Co-Regularized Multi-view Spectral Clustering (CRMSC). Reference [10] presented a novel method named by MVSC-CEV, to computing the common eigenvectors of the Laplacian matrices derived from the similarity matrices of the input views. To address the possible dependencies among views, [11] adopts the brainstorming process to compensate the biases caused by information sharing between multiple views with dependent opinions, and finally a compromise opinion is merged.
Recently, deep learning based methods [9], [19] have been dedicated to incorporating the local invariance within every single view and the consistency across different views. However, all the aforementioned approaches assume the multiview data can be collected on a central server, which will expose their sensitive data to other parties. In this article, we consider a secure and privacy preserving framework to perform the multi-view spectral clustering.

B. FEDERATED LEARNING
With the prevalence of complex machine learning technique, most newly proposed models are data-hungry. There is a contradiction among many data owners or parties. On one hand, the accessible data for each party may be very limited, making the model un-trustable when learning only on its own data. As a result, an increasing number of data parties have desired to collaboratively learn a model together. On the other hand, they are not willing to share their data directly to other parties since the data may be sensitive and private. Some organizations or governments have formulated regulations and laws on privacy protection especially in the big data era (e.g., GDPR [14]).
To address this contradiction between the demand of big data for learning and the needs to protect the data privacy and security, the idea of federated learning (FL) was first proposed by Google [20] to learn a centered model among android users without revealing user's raw data. Research efforts then shed light on this area to apply FL to many scenarios and tasks, including, but not limited to, wireless networks [21], edge computing [22], urban environment sensing [23], topic modeling [24], latent dirichlet allocation [25], matrix factorization [26], neuro network [27], to name a few. To protect the sensitive information, differentially privacy (DP) [28] and homomorphic encryption (HE) [29] are widely used in federated learning. A comprehensive definition, taxonomy and review for FL has been reported on [15].
While a plethora of research works have been done on FL, there is as yet no efforts on spectral clustering, a fundamental and popular unsupervised machine learning technique. Since multi-view data always collected on distributed parties (each with one view data), we highly believe that studying multiview spectral clustering under federated learning is naturally required and fascinating.

III. FMSC FRAMEWORK
In this section, we first introduce the preliminaries of centralized MSC approach. Next we present the proposed Federated Multi-view Spectral Clustering framework, namely, FMSC for short. We give the details of FMSC including the architecture, learning algorithm and finally efficiency analysis.

A. MSC PRELIMINARIES
Given a training dataset X = {x i |i = 1, · · · , n, x i ∈ R d }, the clustering task aims to partition X into k clusters. Spectral clustering (SC) [16] is one of the most widely used techniques in unsupervised learning and often outperforms the traditional approaches such as k-means. The first step of spectral clustering is to define a similarity matrix W ∈ R n×n , indicating a measure of the similarity between data instances. The objective of standard SC is defined by: where L = D −1/2 WD −1/2 denotes the normalized laplacian matrix, D denotes the degree matrix defined as a diagonal matrix with degrees D ii = j W ij , and tr(·) denotes the trace of a matrix. The solution matrix U ∈ R n×k in Eq.(1) can be derived by computing the top k eigenvectors of L and integrating them as a R n×k matrix. Then it employs a traditional clustering method (e.g., k-means) on the row vectors of normalized matrix U.
For the multi-view scenarios, we assume there are m(m 2) views, each having diverse features of data. Let } denotes the data instances in view v. W (v) , D (v) , L (v) and U (v) denote similarity matrix, degree matrix, laplacian matrix and eigenvector matrix for the view v respectively. To encourage the similarities of cluster structure on different views, a regularization term [8] is introduced to force each view-specific eigenvector matrix U (v) towards a common centroid eigenvector matrix U * . Then the objective of MSC can be defined as follows: where λ (v) > 0 denotes the normalized weight for the v-th view. If we believe all views are equally contributed to the objective function, we can set With the optimal solution U * , we can conduct k-means algorithm on it to derive the clustering assignment. More detail can be seen in [7], [8].

B. FMSC ARCHITECTURE
Obtaining the solution of Eq.(2) centralized scenarios is straightforward since the central server owns data of all views. Now we consider a distributed scenario that each party has a sensitive view specific data X (v) . Following the typical assumption of federated learning [15], we assume all the parties (including the central server and all view parties) are honest-but-curious, which means they will be honest in operations and faithfully obey the collaborative learning protocal, but be curious to find out the sensitive data from others parties.
To solve Eq.(2) in a distributed way, we can split the objective into two sub-objectives and alternatively update the solutions of sub-objectives by each party. The first subobjective is to update view specific eigenvector matrix U (v) when fixing U * . Optimizing Eq.(2) equals to: The second sub-objective is to update U * by fixing eigenvector matrix U (v) , v = 1, · · · , m. Eq.(2) can be simplified as: By alternatively updating U (v) and U * according to Eq.(3) and Eq.(4), we can derive the final solution of Eq. (2). Note that to derive the common centroid eigenvector matrix U * , a party needs to obtain m v=1 λ (v) U (v) U (v) T denoted by S according to Eq.(4). We design a secure architecture to compute this term via a central server and run the rest optimizations at each party independently. Let S (v) = VOLUME 8, 2020 denote the information that party v needs to share. The key idea of keeping S (v) private to central server is to encrypt it via Additive Homomorphic Encryption (AHE) technique. Then central server can aggregate all shared information, since the additive homomorphic property enables the aggregation over every encrypted entry of matrices. After deriving the encrypted aggregation S, the central server transmits it to all view parties. The architecture is illustrated in Fig.1.
From Fig.1, two types of participants are involved in FMSC, the central server and the view parties. Taking view party v as example, there are three components: a cryptor, an optimizer, and a perturbation module. The optimizer includes two modules: a CEM Optimization (Opt in Fig.1 for short) module and a VEM Optimization module. The CEM Optimization module can solve the common eigenvector matrix U * by Eq.(4), and the VEM Optimization module can solve the view specific eigenvector matrix U (v) by Eq. (3).
The most important part of FMSC is the encryption, decryption modules in view parties, and aggregation module in the central server. The encryption module encrypts and sends the sensitive parameters, namely, S (v) , to the central server. The aggregation module receives all the encrypted parameters, and computes the aggregation Enc(S) via AHE. The decryption module decrypts the aggregation and derives S to feed into the CEM Opt module for the next iteration. In this article, we use Paillier Homomorphic Cryptosystem to endow central server the ability of aggregation on the cypher text, see subsection IV-A.
The role of perturbation module is to prevent differential inference attack to the information of eigenvector matrix U (v) as well as similarity matrix S (v) , especially when there are few parties. For example, we suppose the number of view parties are two (denoted by index 1 and 2). When receiving the aggregation S, party 1 can derive the similarity matrix S (2) of party 2 using S (1) = S − S (2) , which violates the second regulation in section I. We will introduce the details of this module in subsection IV-B.

Algorithm 1 FMSC Protocol
Input: Training datasets X (1) , X (2) , · · · , X (m) , parameters λ, Output: Cluster assignments Y 1: Initialize similarity matrices W (1) , W (2) , · · · , W (m) for each view party 2: for view party v = 1, · · · , m do 3: initialize the laplacian matrix L (v) . 4: solve the initial eigenvector matrix U (v) by Eq.(1). 5: perturb U (v) and derive U (v) . 6: encrypt S (v) and send it to central server. 7: end for 8: while not converged in each iteration do 9: Central Server: 10: compute the aggregation S on the cipher text. 11: perturb S and derive a noisy S. 12: send S back to all parties. 13: View Party: 14: for view party v = 1, · · · , m do 15: decrypt Dec(S) and derive S. 16: solve U * by Eq.(4). 17: solve U (v) by Eq.(3). 18: encrypt S (v) and send it to central server. 19: end for 20: end while 21: all view parties perform k-means algorithm on U * and 22: return cluster assignments Y The above modules will be executed iteratively until the eigenvector matrix satisfies the convergence criterion. After obtaining the final common eigenvector matrix U ( * ) , all view parties can perform traditional clustering algorithm (e.g., k-means) on U ( * ) to derive the final cluster assignments. We now present the clustering procedure of FMSC in Alg.(1) as a protocol that all parties should obey.

IV. SECURITY MECHANISMS
In this section, we will introduce the details of two security mechanisms. The first one is how to implement the secure aggregation, and the second one is to elucidate the perturbation scheme. Finally, we will give some privacy and efficiency analysis to the proposed FMSC approach.

A. SECURE AGGREGATION
To overcome the possible information leakage to central server, the shared weighted similarity matrices S (v) should be encrypted such that the server cannot inverse. Addictively Homomorphic encryption (AHE) is an efficient scheme that allows a third party to aggregate on the encrypted data without decrypting it in advance. In this article, we use Paillier Homomorphic Cryptosystem [30], an AHE cryptosystem which supports homomorphic addition operation on the cipher texts as follows: Enc pk (m 1 ) + Enc pk (m 2 ) = Enc pk (m 1 + m 2 ), where m 1 , m 2 are the cipher text of the messages, Enc(·) is an encryption algorithm, and pk denotes the public key used by the encryption process.
Note that a view party need to encrypt matrix before send to central server. Thus, S (v) should be encrypted on each entry: where S (v) ij denotes the i-th row and j-th column of S (v) , and n is the number of data instances. Here we omit pk for simplicity.
When central server has received all shared view-specific similarity matrices, it can implement the add operation on those cypher text and derive the encrypted aggregation: Enc(S (v) ).
Next the central server sends Enc(S) back to all parties. The view party v decrypts Enc(S) using the private key sk: Key Generation: Before running the FMSC protocol, the public key and secret key should be generated in advance and distributed to all parties. The key generation process can be carried out on an independent Key Distributor (KD), or one of the view parties. After the keys are generated, the public key can be distributed through an untrustworthy channel to all the parties including the central server, but the distribution of secret key should be shared by establishing different TLS/SSL secure channels among view parties.

B. PERTURBATION MECHANISM
In the proposed FMSC architecture, the perturbation module add some noises before sending the aggregated similarity matrix S back to view parties. Since we assume each view party is honest-but-curious, the aim of perturbation module is to prevent the possible leakage of sensitive information from other parties. If not perturbed, a view party v can easily derive the aggregations of other parties by subtracting S (v) from S. In addition, one party can directly obtain another parties' sensitive S (v) if the number of parties is small (e.g., 2).
To overcome this issue, we introduce differential privacy (DP) as a perturbation technique to add small noise on S. Differential privacy is one of the state-of-the-art privacy models against differential attack, which is first introduced in [28]. DP guarantees that the adversary couldn't infer sensitive information about any specific instance with high confidence (parameterized by a privacy budget ) from the queries(e.g., querying aggregations in this article), even if all the remaining instances of the data are known by the adversary. Similar to this concept, we want to guarantee that any view party v couldn't differentiate S − S (v) with high confidence from S, even if the similarity matrix S (v) is known in advance. Note that a smaller privacy budget value will induce a stronger privacy guarantee but a higher noise.
We firstly consider preserving the privacy of one elements S ij in the i-th row and j-th colomn of S.
ij } denotes the database of all received data, and x denotes a neighboring database that differs in at most one entry, e.g., x = {S (1) ij , · · · , S (m−1) ij }. One of the commonly used DP models is defined as follows: Definition 1 ( -Differential Privacy [31]): A randomized mechanism M : X → T is -differentially private, if for any neighboring databases x, x ∈ X , and for any possible output O ∈ T , we have where Pr[·] denotes the probability.
To satisfy the definition of DP, traditional approach is to employ the laplace mechanism and define the randomized mechanism M as: where f (·) denotes the aggregation function, and Y denotes the noise variable following a laplacian distribution. According to the theorem 3.6 in [32], we have the following results: Lemma 2: the laplace mechanism M preservesdifferential privacy if we define Y ∼ L(0, f ), where L denotes the laplacian distribution, and f = max x,x f (x) − f (x ) 1 denotes the l 1 -sensitivity of function f . While this is acceptable when applying all elements in S satisfying differential privacy, it is too restrictive in practice because the added noises are so large that the clustering algorithm couldn't easily converge. To address this issue we propose a method called Random Laplacian Response (RLR) mechanism, which composes a random response and a laplacian mechanism. We formulate the RLR perturbation mechanism as an algorithm in Alg. (2), where some parameters should be initialized as inputs. We denote η as parameter of a bernoulli distribution to control how many elements are Enc(S ij ) ← Enc(S ij ) + Enc(Y ) 9: end for 10: return Enc(S) selected to be perturbed. If b sampled from the bernoulli distribution equals to zero, then the noise would be set to zero (line 4). Otherwise we produce a noise sampled from a laplacian distribution and add it to the current elements. Note that the addition is implemented on the cyphertext via paillier AHE system (line 8).
In deed, parameter f controls the l 1 -sensitivity of function f , but suffers from a major limitation: it couldn't directly be computed in central server because {S (1) , · · · , S (m) } are encoded in cyphertext. Since f is bounded by the maximum of {S (1) , · · · , S (m) }, we enlarge f to fit all elements: where S (v) max demotes the maximum value in matrix S (v) . To this end, the central server can collect all the maximum values in plaintext for every iteration, without revealing the sensitive S (v) . Although the relaxed f would introduce larger noise, we can control the amount of overall noise by adjusting parameters and p.

C. PRIVACY ANALYSIS
In this subsection we give analysis about the privacy of our proposed FMSC approach.

1) SECURITY AGAINST CENTRAL SERVER
As demonstrated in Fig.1, only ciphertext from view parties are sent to the central server in FMSC. So no bit of information will be leaked to the central server because the paillier homomorphic encryption cryptosystem guarantees ciphertext indistinguishability against any possible plaintext attacks [33].

2) PRIVACY AGAINST VIEW PARTIES
From Alg. (2) we can see that the random laplacian response perturbation mechanism M is composed by two sub mechanisms: a random response M 1 and a laplacian mechanism M 2 . M 1 flips a coin and the result determines whether M 2 will be activated. Define the new privacy budget as: Then we give the following theorem: Theorem 3: The Random Laplacian Response (RLR) perturbation mechanism preserves -differential privacy.
The proof of Theorem 3 is demonstrated in Appendix.

V. EXPERIMENTS
In this section, we conduct experiments on both synthetic and real-world multi-view datasets. First, we introduce the experimental settings as well as baseline methods. Then we evaluate the effectiveness and performance of the proposed FMSC approach against the baseline methods. Finally, we analyze different parameters η and to demonstrate how perturbation effect the results.

A. EXPERIMENT SETTINGS
To evaluate the performance of the proposed FMSC approach, we adopt five multi-view datasets including both synthetic and real-world datasets. Synthetic data is generated from two views to four views with two clusters. Similar to the settings in [7] and [8], we randomly sample 1000 instances (n = 1000) for each view via 2-dimentional gaussian mixture model. The means and covariances of all clusters in different views are defined in Table 1.
We also adopt five popular real-world datasets:UCI digits, Reuters, BBC, BBCSports and Yale datasets. The datasets are introduced as follows: • UCI digits. This dataset is created from handwritten numerals from 0 to 9, and each instance forms an image with 15 × 16 pixels. It is available at the UCI repository.
• Reuters. Reuters multilingual corpus is a set of news articles written in five languages: English, French, German, Italian and Spanish. We sample 10, 000 instances into our experiments from the original corpus that contains 18, 758 articles. For each view, we use a word dictionary as features and compute the term frequencies as feature values.
• BBC and BBCSport datasets. Both of the two datasets are multi-view news articles from the BBC. BBC contains five possible topics (business, entertainment, politics, sport and technology), and BBCSport focuses on five sports topics (athletics, cricket, football, rugby and tennis). We also adopt term frequencies as feature values.
• Yale. Yale is an image dataset that consists of 165 face images of 15 classes in pixel. Each class has 11 images.
We summarize some characters of the above datasets in Table2. Note that the numbers in column '#Feature' are the sum of features in all views.
To validate the effectiveness of our FMSC approach, we run our approach on the abovemetioned datasets and compare the results against three baseline clustering methods: Concatenation, Co-training, and Co-reg. Concatenation is a straightforward approach that features of each view are concatenated to perform a standard spectral clustering. Cotraining [7] alternatively performs spectral clustering on each view by incorporating other view's eigenvector matrices as constraints. Co-reg refers to Co-Regularized multi-view spectral clustering approach proposed by [8]. Our work is based on Co-reg and extends it to the federated scenario.
We measure the quality of clustering results on several metrics: Normalized Mutual Information(NMI), Adjusted mutual information(AMI), Adjusted Rand index(ARI), Completeness(COM) and Homogeneity(HOM). NMI and AMI are two different normalized versions of Mutual Information (MI), which quantifies how much the estimated clustering is informative about the true clustering [34]. The ARI metric is similar to the clustering accuracy and measures the degree of agreement between the estimated clustering and the true clustering. COM and HOM are two important conditional entropy based metrics [35]. COM measures the ratio of the member of a given class that is assigned to the same cluster, while HOM measures the ratio of instances of a single class pertaining to a single cluster.
In order to evaluate the execution time of the proposed approach, we also introduce Running Time in seconds as a metric. For all the above metrics, a higher value means the corresponding method has a better performance. We repeat 10 times and report the mean values for each experiment. All approaches are run on an Intel i9-9900K 3.6 GHz with 64GB of RAM, using single-threaded processes.

B. RESULTS
Firstly, we test our FMSC approach without adding any noises. The aim is to verify whether it can receive as considerable clustering performance as centralized approaches. We  omit the Perturbation module in Fig.1 and report the results as follows. Table 3 demonstrates the clustering results on synthetic datasets. We can easily see that the overall performance of four-views dataset is better than that of three-views, and the overall performance of three-views dataset is better than that of two-views. The results suggest that more views can bring improvements on the clustering performance. For different approaches, the performance of Co-reg is superior than Cotraing and Concatenation. Our FMSC approach has almost as the same performance as Co-reg since FMSC is a distributed and secure variant of Co-reg. Table 4 shows the spectral clustering resutls on five realworld datasets. For UCI digits and Reuters, our FMSC approach has the best performance than other baselines. Our performance is slightly worse than Co-training or Co-reg on BBCSports and BBC datasets. But the declined performance of FMSC is acceptable as it protects the privacy of sensitive data. For Yale dataset, the performance of our approach is as almost the same as Co-training and Co-reg methods on all metrics. So, we can safely say that FMSC is a secure distributed spectral clustering approach with small performance drawbacks.
Finally, Table 5 shows the average running time over 10 executions of each approaches. Note that we omit the  communication time over parties for our FMSC since it is trivial compared with computation time. As expected, all three centralized baselines run faster than FMSC. Specifically, it only needs less than one second for Concatenation on all datasets. Our FMSC costs larger execution time because on each iteration it needs lots of encryption and decryption operations which are very time consuming. We can also see that the running times of FMSC are different for different datasets. The reason is these datasets have different number of views with different number of features. More views and features will cost more running times.

C. PARAMETER ANALYSIS
In this subsection, we report the effect of two parameters used in FMSC approach. The first parameter η controls how many elements in aggregated matrix S are sampled to add noises.
The second parameter determines the original privacy budget for each element.
We first investigate the influence of parameter η by fixing λ = 0.1 and = 0.001. We varying η from 0 to 1 with step 0.1. Fig.2 shows the performance of FMSC on five real-word datasets. We can find two interesting points: (1) for UCI digits and Reuters datasets, our approach has one clear minimum value on all five metrics. (2) But for BBCSports and BBC datasets, there are two obvious minimum values. Three minimum values could be find out for Yale dataset. In summary, if we get the privacy budget in advance, η can be adjusted easily to get a better clustering performance.
Then we conduct experiments on two representative datasets to observe the effect of different parameters . We fix λ = 0.1 and η = 0.5. Thus from Eq.(12) we derive the system privacy budget = . From Fig.3, we can see that when is small, the measurement values on all metrics are small, and the values tend to increase with the increase of . With ≥ 1, the values are stable and don't increase any more. This result conforms our idea that controls the privacy budget. Given a very large privacy budget, the clustering performance will not be influenced by the added negligible noises.    4 shows similar results of how effect clustering performance on yale dataset. However, the curves is flatter for all metrics. If becomes small, the clustering performance will slightly decrease. When ≥ 10, the clustering performance will not change. This reflects that yale is not sensitive to noise and we can distribute more privacy budgets for it.

VI. CONCLUSION
In this article, we propose a novel approach named Federated Multi-view Spectral Clustering (FMSC) to address the possible privacy leakage problem in centralized Multiview Spectral Clustering (MVSC) approaches. Specifically, we design an architecture for all view parties to collaboratively learning the underlying cluster structure, without revealing their own data to other participants. On one hand, we borrow Homomorphic Encryption (HE) to compute the aggregations on cypher text of sensitive parameters to protect privacy against central server. On the other hand, we also introduce a Differential Privacy (DP) mechanism to prevent the information leakage between view parties when there are few parties involved. We theoretically prove that our approach can achieve secure computation and differential privacy on sensitive shared parameters. Our quantitative experiments show that FMSC has considerable good performance compared with centralized approaches.
In the future work, we try to improve the efficiency of FMSC from the following direction: drop the time costing encryption-decryption mechanism and instead adopt a locally differential-privacy mechanism to get a more effi-cient distributed model. However, the performance of this kind of model may reduce compared with the proposed FMSC approach. In another way, there exists newly published approaches that may have good performance on multi-view data clustering task, e.g., low-rank approximation [36], subspace matrix factorization [37], probabilistic model [38] and deep learning [9]. However, they are central based methods and it is not trivial to transfer them on the distributed and federated scenarios. In the future, we will shed lights on the deep methods that transform them into more secure and private approaches.
3) If z = f (x) and z = f (x ), we have: By defining in Eq.(12), we can finally have: p x (z) ≤ exp( ) p x (z), which proves the result.