Personalized Recommendations Based on Sentimental Interest Community Detection

Communities have become a popular platform of mining interests for recommender systems.The semantics of topics reflect users’ implicit interests. Sentiments on topics imply users’ sentimental tendency. People with common sentiments can form resonant communities of interest. In this paper, a resonant sentimental interest community-based recommendation model is proposed to improve the accuracy performance of recommender systems. First, we learn the weighted semantics vector and sentiment vector to model semantic and sentimental user profiles. Then, by combining semantic and sentimental factors, resonance relationship is computed to evaluate the resonance relationship of users. Finally, based on resonance relationships, resonant community is detected to discover a resonance group tomake personalized recommendations. Experimental results show that the proposedmodel is more effective in finding semantics-related sentimental interests than traditional methods.


Introduction
Recently, socialized recommendation has become one of the most popular means of recommendations in various recommender systems that have been applied in the fields of E-commerce, social media platforms, web search engines, and so on [1].In socialized recommender systems, mining socialized relationships is critical for pushing or sharing interesting people and things with target users.Additionally, discovering similar users with common interests is important for determining recommendations.To address these problems, user profile techniques are used to reflect users' interests by describing socialized relationships and representing the history of browsing contents [2].Accurate user profiles can depict the preference of users, which is helpful for accurate socialized recommendations.
Semantics analysis involves identifying relevant information from a serious of views, products and sources [3].As a vector representation method, word embeddings are useful for identifying semantic relations from the context of documents or subjects [4].Word embeddings use neural network architectures to implement distributed representations of words [4,5].The distributed representations in vector allow computation of the semantic similarity of words, which can efficiently discover high quality semantics-relevant subjects [6].Therefore, we can model user profiles and analyze their semantic relationships regarding word embeddings to improve the quality of socialized recommendations.
Sentiment analysis, or opinion mining, has been applied to many fields, such as marketing, opinion monitoring, and information retrieval [7].Sentiment analysis includes positive, negative, or neutral attitudes.Through analyzing semantic information of strong emotional and subjective texts, sentiment analysis captures users' emotional behavior characteristics and sentimental attitudes [8].Sentiment user profiles can be searched for target users according to the presence and frequency of terms and opinionated words in documents [9].A sentiment user profile depicts a user's sentiment degree for a subject or a topic [10].Thus, users with similar sentiments or opinions may have a resonant interest tendency.Therefore, detecting a group of users with common sentiments provides a beneficial method for personalized resonant recommendations.
Considering the semantic and sentimental relationships between user profiles, we can divide users with similar sentimental interests into a cluster, forming a community.The 2 Scientific Programming target of community detection is to identify a series of clusters so that the users in the same cluster have high similarity but are very dissimilar with users in other clusters [11].Based on this assumption, we can capture the interests of similar users in the same community for target users to improve accuracy of recommendation systems and user's satisfaction.However, very little research has been conducted merging semantics and sentiments for community detection.In fact, regarding content resources, although users browse the same topics, view the same products, and comment on the same topics, they often have different opinions and emotions for these topics.Some users support the topic while some users are against it.The results of community detection considering content resource do not reflect detailed interest opinions.Users in the same community are not emotionally congruent [12].Therefore, identifying similar emotional interest users based on a community for socialized recommendations is a problem.By combining the semantic interests and sentimental interests, we can use resonant community to discover a group of similar users with common interest and sentiments, which can efficiently supplement a user's related interests to promote the accuracy of recommender systems.
In this paper, we present a resonant sentimental interest community-(RSIC-) based recommendation model to improve the accuracy performance of recommender systems.The proposed method considers the integration of semantics and sentiment.As a popular tool, word2vec proposed by Mikolov is used to learn distributed word representation to model the semantics vector and sentiment vector [10].Then, considering semantics and sentiment factors, resonance relationship is computed to evaluate the correlation of users.Additionally, based on resonance relationships among users, a RSIC is detected to discover resonance group, which includes users with common interests and sentiments.Finally, based on resonance users, a collaborative strategy is adopted to select semantics-related subjects for updating the interest subjects of target users.
The remainder of this paper is organized as follows.Section 2 discusses state of the art about RSIC-based recommendation model.In Section 3, we give a RSIC-based recommendation overview.Section 4 introduces the notations and methods to build a semantic user profile and sentiment user profile.In Section 5, we apply the RSIC detection and collaborative filtering recommendation.Section 6 shows the experimental results and discussions.Section 7 contains the conclusions of the paper.

Related Works
Research on socialized recommendations and sentiment user profiles is relevant to the proposed model.In this section, we discuss related works involved in user profiles, community detection and recommendation approaches.

User Profiles.
For most recommender systems, user profile modeling mines users' preferences for personalized search from users' histories or similar users' contents [1,2,13].By extracting a vector representation of the words, Boratto [14] proposed a novel method to model user profile and detected segments of users.Through integrating annotated tags and ratings, Du [15] proposed a multilevel user profiling model to make personalized search.By analyzing the feedback interactions between users, Liu [16] presented an unsupervised approach to automatically update the profiles.Kumar [17] constructed clustered user-interest profile in terms of use singular value decomposition (SVD), which includes clusters of semantically or syntactically related tags, to identify topics of users' interests.By considering the entities and aspects in the user's comments, Meguebli [18] focused on building a method of user profiles and article profiles, and then matching these profiles for the purpose of personalized recommendation.By constructing user profiles from folksonomy systems, Xie [19] computed user similarities within a random walking distance and identified user communities.Based on the behaviors of community neighbors, enriching user profiles was proposed to solve the data sparsity problems of conventional single user profiling.Considering the contextual information sources, White [20] evaluated the utility of social, historic, task, collection, and user interaction sources to model user profiles for predicting users' future interests.Accounting for the sentiment factor, Xie [21] incorporated sentiment information and proposed a SenticRank framework to create a personalized search.In the framework, the authors utilized the content-based method and collaborative strategy to obtain personalized ranking recommendations.
As emotions are important in daily life, some studies have made emotion prediction for individuals.Considering a probabilistic graphical model, Cui [22] combined userinterest and social influence factors to make an emotion prediction framework for individuals.As bursty sentimentaware topics can reveal sentiment-aware events, Qi [23] proposed a Time-User Sentiment/Topic Latent Dirichlet Allocation (TUS-LDA) to detect bursty sentiment-aware topics, which solved the problem of context sparsity problem in conventional LDA-based models.Although existing studies have got the remarkable achievements for personalized user profile modeling and individual emotion prediction, these works have not fully integrated the interest user profile and sentiment user profile to describe users' interests.

Community Detection.
The existing community detection methods identify communities by utilizing the network topology information.Some researches focused on the graph clustering approaches to detect nonoverlapping or disjointed communities, such as block model approximation, label propagation, and modularity maximization.Considering the local importance of a node in a community and the representability of the community, Bai [24] proposed a fast graph clustering description model to discover communities for a large-scale network.By constructing a sparse graph regarding the similarity matrix, Xiao [25] aggregated multiple clustering results from different clustering algorithms to obtain the final clustering of different shapes and sizes.Chen [26] designed a Bayesian mixture network (BMN) model to make overlapping communities detection for weighted networks, which presented soft partition and soft memberships solutions to solve the problems of detecting weighted networks and measuring the membership degree of a node belonging to a community.As users' interests are changing over time, Feng [27] developed a time-weighted overlapping community detection method in terms of association rule mining in order to model dynamic user interests for personalized recommendations.Lancichinetti et al. presented a local fitness maximization method to make overlapping and hierarchical community detection [28].Newman proposed the Fast-Newman algorithm to improve the quality of community detection [29].Considering community detection as a matrix blocking problem, Chen [30] recognized matrix column similarities and computed a partial clustering of the vertices in a dense subgraph to analyze graph structures and complex networks.
To our knowledge, social emotions are contagious and resonant.Some researches have focused on exploiting social emotion mining from the latent semantics and sentiments of individual words.Rao [31][32][33] detected social emotion and investigated social emotion classification for short texts in terms of topic models, such as affective topic model and topic-level maximum entropy (TME) models.Rao et al. [33] also concentrated on generating sentiment topic model by merging latent topics with social emotions, which can be used in the social emotion classification and social emotion lexicons modeling.Lee [34] identified individual user sentiments embedded in the messages, task-oriented content, and proactiveness to analyze collective sentiments, which can affect collective cocreation thinking, especially for the innovation process of cocreation communities.Zou [35] utilized community detection methods to investigate how to exploit weak dependency connections in communities as an aspect of social contexts for microblog sentiment analysis, including sentiment consistency and emotional contagion.In our paper, we detect a resonant sentimental community and identify the most similar users with concordant sentiments for personalized recommendations.

Recommendation Approaches.
Most recommender systems use three approaches to make recommendations, including content-based, collaborative filtering (CF), and hybrid approaches.Based on a personal ontology user profile, Cantador [36] investigated the tastes and preferences of users and computed their social relationships to identify communities of interest.Because short-form messages often express users' interests and opinions, Esparza [37] investigated users and products profiles from associated reviews and computed their relevance to make product recommendations.Taking into account the similarities among user profiles, cooccurrence of user names, and interaction behaviors, Xiong [38] presented a probabilistic graphical model to accurately measure the social relationships in online social networks for recommendations.By exploiting the high-order relational information of tag data and customizing different types of relations' influences, Zhu [39] developed a heterogeneous hypergraph embedding framework for document recommendation.Based on users' historical web search behaviors, Bai [40] utilized external usage information to the news service for news personalization.By considering structured and unstructured data with different semantics, Zhang [41] proposed an integrated framework, called collaborative knowledge base embedding (CKE), to learn the implicit representations and semantic representations in terms of collaborative filtering strategy.Based on the semantics of tags, Li [42] categorized tags into emotional types and developed an emotion ontology called UniEmotion for music recommendation.Although the content information has been successfully used for various recommender systems, sentimental information is also worthy of attention for capturing a personal user profile and improving the accuracy of document recommendations.

Recommendation Framework Based on the RSIC
We design a recommendation framework based on the resonant community.The resonant community is comprised of users with similar interests and common sentiments.
Figure 1 shows the process of RSIC detection and resonant interest selection.
In Figure 1, the recommendation framework involves three steps: weighted user profile modeling, resonance relationship calculation, and RSIC-based recommendation.First, by applying the term frequency-inverse document frequency (TF-IDF) mechanism and ontology structure, we compute the interest degree of subject for each user.Considering word embeddings and sentiment dictionary, a weighted semantic user profile and weighted sentiment user profile are modeled.Then, based on two kinds of weighted user profiles, we compute the resonance relationship between users, which differentiates the interests and emotions among users.Finally, we conduct RSIC detection by considering resonance relationships between users.According to the belonged community, resonance group is selected for target users to implement an interest update.By ranking updated interests, we push topk subjects and their relevant microblogs to target users.

Resonance Relationship
4.1.Content Interest.Messages posted or reposted by users contain many noun entities which reflect preferences of users.In our paper, we adopt the TF-IDF mechanism to measure the weight of a subject in messages.First, given a corpus, by removing stop words and splitting words, we identify significant noun entities in messages and calculate their TF-IDF weight.For a message , it can be represented as  = {( 1 ,  1 ), ( 2 ,  2 ), . . ., (  ,   )}.
Here,   is the relative importance of term  in , which can be computed using the TF-IDF scheme as follows: where   is the frequency number of term  in microblog  and max  (  ) is the frequency number of term  which has the maximum frequency in .  is the total amount of microblogs and   is the quantity of microblogs that contain term .The weight   depicts Then, for a given user , considering all the user's microblogs, we infer content interest degree of subject  as where   is the set of microblogs for user . is a subject set over knowledge base .The knowledge base  is involved in society, sports, economics, culture, and IT topics, which are from the category classifications of the Baidu Wikipedia.Figure 2 shows an example of a classification structure for the five topics.(, ) = 1 if  ∈ ; otherwise (, ) = 0.

Weighted Semantic User
Profile.However, the interest degree does not reflect close relationships between subjects from semantics, which is unsuitable for discovering similar users with latent semantic interests.Therefore, we introduce word embeddings to represent the characteristic of subjects and improve descriptions of user profiles.
The neural word embeddings proposed by Google's word2vec include the CBOW and n-gram models [6].Word2vec adopts || dimensional vector autoencoders to train a large quantity of text for representing the characteristic of words or contexts of words [9].Figures 3-4 show an illustration of the CBOW and n-gram models, respectively.
The CBOW model is a three-layer neural network to predict a word as the output of a vector regarding the context as input, while the n-gram model learns the vector representation of the context through the center word [9].
In our work, we try to learn the word vector using the n-gram model, which extracts the multidimensional vector representation of a word to represent the characteristic of a subject.Given a sequence of training noun entities  1 ,  2 , . . .,   , for a word   , its context includes the previous  words and following  words as (  ) = { − ,  −+1 , . . .,  +−1 ,  + }.Then, the n-gram model maximizes the conditional probability ((  ) |   ) to learn the word vector representation of   .By maximizing the words' average log probability, the n-gram model learns the objective function as follows [10]: where  is the size of the training context.For the n-gram model, we use a hierarchical softmax function to speed up training.Finally, we adopt a weighted path from the root to a leaf node to represent the vector of word as = { 1 :  1 ,  2 :  2 , . . .,   :   }.
Then, for an interest subject   , assigning its content interest weight to the vector representation, we utilize an integrated weighted vector to differentiate semantics and model the semantic user profile for a user, which is formalized below: The weighted semantic user profile can be efficiently used to depict the closeness of users from aspects of latent integrated semantics.For two users,   ,   , we can measure their semantic similarity by cosine metric as  users with fragmented information.One topic can trigger multiple messages, which includes positive comments, negative judgements, and neutral points.The subjects in each message express a kind of sentiment or emotion.Different subjects imply different sentiment tendencies.In our paper, we use the emotional vocabulary ontology [43] to detect the sentiment degree of each subject.Considering occurrence frequency of a subject in one's microblogs, we define the sentiment degree weight as Here,  is a sentiment degree of a subject, which involves several grades, such as scores 1, 3, 5, 7, and 9. (, ) = 1 if  ∈ ; otherwise (, ) = 0.The sentiment degree weight describes the sentimental importance of a subject for a user's sentiments.
Additionally, for a sentiment subject , we learn its sentiment vector representation as V   .Considering all the sentimental entities of user , we utilize a weighted sentiment vector to model sentiment user profile as The weighted sentiment user profile describes a user's sentiment preference from aspects of sentimental semantics, which shows the emotional subjects the user prefers.Based on weighted sentimental user profiles, we infer users' sentimental similarity as For users   and   , based on semantic similarity and sentimental similarity, we compute their weighted sum to model users' resonance relationship, shown in The coefficient weight  evaluates the relative importance of the semantic similarity and sentimental similarity on the measurement of the resonance relationship.

RSIC-Based Recommendation
Based on the resonance relationship between users, we determine close connected edges and implement community detection.The generated communities include users with common interests and similar emotions, named as resonant sentimental interest community.Considering the RSIC, we implement more accurate personalized recommendations.For all users, we first construct the resonance graph (, ) based on their resonance relationships, where  is the set of user vertices,  is the set of resonance edges which represent two users having a higher similarity.Equation (10) defines the resonance edges using the corresponding resonance relationship between users: where cutoff  ≥ 0 controls the number of resonance edges in the graph.Different thresholds generate different numbers of connection edges for a resonance relationship graph.In (10), we ensure that those connective nodes using the resonance edges in the graph are users with similar interests and common sentiments.The resonance edges are used to cluster resonant users into a community.
Based on the resonance graph (, ), we implement RSIC detection.As stated in [28], the fitness function measures the contribution of internal edges of nodes in the graph and external edges with other nodes in the remainder of the graph, which is shown as follows: where    and    are the total internal and external degrees of the nodes of graph .Parameter  > 0 determines the scale of communities.In our paper, we compute the weighted internal and external degrees of graph  as    = ∑ ∈ ∑ ∈   ⋅(  ,   ) and    = ∑ ∈ ∑ ∈   ⋅(  ,   ).Then, we set  = 1 to have iterative operations to detect the overlapping communities.The weighted internal and external degrees reflect the close relationship between users from aspects of semantics and sentiments.For a resonance graph  and a node , we utilize the fitness contribution   () =  +{} −  −{} to determine whether the node belongs to the community . Based on the fitness, the detailed steps of RSIC detection are given in Algorithm 1.
Algorithm 1 describes the detection process of a community for a user node .For all the vertex nodes in the graph, we detect communities until each node is contained in at least one community.Algorithm 2 presents the process of community detection.
Algorithm 2 shows the process of RSIC detection for all vertexes in the initialized resonance graph .At each iteration, steps (3)-( 4) perform the community detection for a new node, which is selected from the last subgraph .By implementing Algorithms 1 and 2, the RSIC of each vertex is discovered, and each vertex belongs to one community.
In each RSIC, the nodes in the community are similar regarding the subject semantics and sentiment attitudes.Based on this view, for each user, we utilize the RSIC to discover one's resonance group, as () = {  | (,   ) = 1,   ∈   }.The resonance group includes users who have close linkages with the target user.Then, considering the subjects deriving from the resonance group, we collaboratively predict the interest degree for the target users.Given a subject , the interest degree of the subject based on the resonance group is computed as By ranking the interest degree, the top- subjects are selected for helping to provide related microblogs to target users.

Experiments and Discussions
6.1.Experiment Strategies.In this section, we evaluate performance of the proposed RSIC-based method by introducing some other recommendation strategies to make comparisons, including LDA, CF, and TF-IDF methods.
For a given subject , we can get content interest degree of a user by TF-IDF mechanism in (2).Considering all the subjects, we can rank their content interest degree and provide top- subjects for target users to make recommendations.
Based on the semantic similarity in (5), we can compute the similarity of two users to measure their interest closeness.For a given user   , considering one's similar users set    , the collaborative interest degree is defined as in By ranking the collaborative interest degree of subjects, we can select top- subjects and push to target users.
LDA topic method is a generative probabilistic graphical model for personalized topic models [23].The model generates documents of latent topics in terms of two assumptions.Each document can be represented as a multinomial distribution over a set of T topics, and the topic is a multinomial distribution related to the set of vocabulary words, which are, respectively, defined as ( | ), ( | ), where , , and  denote the latent topic, the word, and the document, respectively.The topic   =  can be denoted as (  = ) = (  (  ) ).A multinomial distribution related to the set of vocabulary words can be denoted as (  |   = ) = (   () ), which depicts the meaning of the topic.Then, the document distribution and word distribution are Dirichlet distributions, which can be defined as   = ℎ() and   = ℎ().
In experiments, we adopt Gibbs sampling and set the hyperparameters  =  = 0.01 to model latent topic distributions.According to top- topic distribution, we select their maximal related subjects for each topic and give recommendation to the user.

Experiment Datasets.
To examine the quality of the proposed RSIC-based recommendation method, we used the  dataset and  dataset to verify the efficacy of the system.The two datasets contained subjective emotional Weibo microblogs or users' emotional comments, involved in lots of words in the emotional vocabulary ontology proposed in [43].For both of the datasets, we rely on various recommendation methods, which were calculated as follows: where   = { | () > 0} is the set of real subjects which were involved in the test microblogs and   is the set of subjects from the recommendation list.

Experiment Results.
In the experiments, we first model weighted semantic user profile and sentimental user profile.Then, by computing the resonance relationship of users, we detect the RSIC community and select the resonance group of users from the community.Based on the interest degree in (12), we can rank the subjects in community and make top- recommendations.
In the process of calculating the resonance relationships, we set the dimension of the word vector as 100 to depict the semantic user profile and sentimental user profile.After computing the interest degree of the subject in terms of resonance group, we changed the value of recommendation list size  to make personalized recommendations.As the resonance relationship between users was determined by both semantic similarity and sentiment similarity, the results of the RSIC were affected by the relative weight of the semantic similarity and sentiment similarity.By changing the value of the relative weight coefficient , we considered the resonance relationship to observe the variations in the recommendation results.Meanwhile, the cutoff  also controlled the scale of resonance edges among users, which formed a different resonance relationship graph and affected the results of the communities.Figures 5-6 show the results of precision, recall, and F1 under different recommendation list sizes by setting the fixed coefficient  and cutoff .As shown in the figures, when  increases, the performance of the RSIC recommendation method is clearly superior to LDA, CF, and TF-IDF methods.For example, in the  dataset, we can see that the precision of the RSIC method with a recommendation list size of 5 can get an approximately value of 0.67 while the values of LDA, CF, and TF-IDF are 0.53, 0.58, and 0.51.That is, in the RSIC method, more than 381 of 570 recommended subjects are matched with the target users' interests.In the microblog scenario, for a topic, different users often describe different semantics regarding different diverse text contents.However, their similar topic sentiment can assist in identifying the same interests for users.By considering the sentiment effects, more accurate subjects can be selected for the target users.
Additionally, for all methods in Figures 5-6, we can also observe that their precision values decrease and recall values increase with the recommendation list size increasing from 5 to 25. Especially for a larger recommendation list size, the recall of the RSIC method is superior to other methods, which shows that the introduction of sentiment can efficiently enhance the relevance of subjects.
By adjusting the values of coefficient , we can analyze the influence of semantic similarity and sentiment similarity factors for the recommendation results, which are shown in Figures 7-8.By setting the values of the cutoff as  = 0.5 and  = 0.6 for two datasets, Figures 7-8 show the trend of precision, recall, and F1 results at different recommendation list sizes under different coefficient .In the figures, we can see that both the precision and recall curves first increase and then decrease for different recommendation lists.On two datasets, the performance can get a better result at  = 0.7 and  = 0.5 for different values of .For the  dataset, the few microblogs can trigger a small semantic similarity between users; and the sentiment factor can obviously affect the resonance relationship of users, which effectively improve the performance of recommendation results.For the i dataset, the adequate microblog contents can reflect the users' semantic similarity, which is beneficial for selecting interest subjects.Thus, in the second dataset, the values of precision and recall are maximally at weighted coefficient  = 0.5, which shows that both the semantics factor and sentiment factor have the same significance in the process of discovering resonance users for resonant community detection.The phenomenon shows that an appropriate weight coefficient can achieve good results for RSIC-based recommendations.
As the cutoff  can affect the scale of the detected community, we set different cutoffs to identify different resonant communities.Based on different resonant communities, we obtain different resonance groups to make the RSIC recommendations.For two datasets, Figures 9-10 show the precision, recall, and F1 results at different recommendation lists under different cutoff  values with  = 0.7 and  = 0.5.
In the figures, we can see that the best performances are achieved at  = 0.5 and  = 0.6, respectively.For example, in the  dataset, the values of precision and recall first rise and then decline with the cutoff  changing.With the recommendation list size increasing from 5 to 25, the maximal precision and recall become 0.67, 0.48, 0.37, 0.31, and 0.26 and 0.44, 0.62, 0.73, 0.79, and 0.84 at  = 0.5, respectively.As we expected, in (10), the large cutoff is inappropriate for acquiring resonance graph with similar users, which creates a small community and few resonance users.A small number of resonance users cannot provide rich interest subjects for the target user, which affects the performance of the RSIC method.However, a small cutoff helps to model many similar resonance relationships; and many users in the RSIC generate a large resonance group.Although the sentiment of the users in the resonance group is close and consistent, their semantic similarity is small.The subjects from the resonance group are not accurate and relevant, which leads to a poor precision and recall.Therefore, it is suitable to set an appropriate threshold for detecting community and filtering out the best number of resonance users to make recommendations.

Conclusions
This paper proposed a method to merge the sentiment factor into user's semantic interests for computing resonance relationships between users.Then, considering the resonance relationship, a resonant sentimental interest community was detected for personalized recommendations.From evaluation of the method, some insights were found, as described below.
In our experiment, the RSIC recommendation method outperformed semantics-based LDA, CF, and TF-IDF methods, both in indexes of accuracy and recall.We contributed to designing a weighted RSIC-based recommendation method by taking into account the sentiment and semantics factors for community detection.In addition, both sentiment and semantics were beneficial for mining resonance users with common interests in a community.Especially for uses with high implicit semantic similarity, the sentiment can efficiently select their accurate and relevant subjects.
Interestingly, users' interests are diverse and multigranular.How to model the sentiment user profile in different grain subjects and discover multigrain RSIC is a promising problem.In most cases, the resonance similarity of users is large for the coarse-grain subject, while, in the fine-grain subject, the resonance similarity between users is generally small.In different grain subjects, we want to investigate the mechanism of community detection in terms of resonance similarity of users, which can generate different granular communities.According to communities of different granularity, we can consider subjects in coarse-grain community and in fine-grain community to get diverse combination results.Hence, how to design optimized combination recommendation results in communities of different granularity is important for improving users' satisfaction.In the future, it is expected that multigranularity RSIC can make more accurate recommendation services.

Figure 2 :Figure 3 :
Figure 2: A fragment of a knowledge base for the five topics.
+{} and   −{} define the fitness of the new graph G with  inside and outside.