Analysis of Recommendation Systems Based on Neural Networks

As the amount of online information explodes, the recommendation system has evolved into an effective strategy to overcome the problem of information overload, which has become the focus of academic and industrial circles, and has led to other related research results. Through the recommendation methods, the author mines the items including information, service, and goods that the user is interested in, and recommends the results to the user in the form of a personalized list. In this paper, the author mainly examines and summarizes the progress of research on neural network based recommendation systems in recent years, as well as giving conclusions about the differences and benefits between that and traditional algorithms. In the end, the future trend of the recommendation systems based on neural networks is prospected.


Introduction
The recommendation system, as a tool of information filtering, is based on massive data mining, which can effectively solve the problem of information overload and provide content that can meet the users' needs in a personalized way. In addition, recommendation systems act like a platform between the user and the information producer, not only helping the user find the information which is needed, but also allowing the producer to present the information to the user who is interested in that. Therefore, the author analyzes the recommendation systems in this paper. Also, recommendation systems have a wide range of applications, such as e-commerce, social networking, video and music recommendation, which are closely related to our lives.

Traditional Recommendation System
In the 1990s, recommendation systems developed into an independent discipline. The core of the recommender systems is recommender algorithms, which could help the users find the items that they are interested in, based on the relationship between them.

Content-based recommendation
Content-based recommendation is based primarily on what the user has already selected or scored and find other similar items, linking projects to projects, which falls under the SCHAFFER category [1]. It first establishes a model of user-interested features, which can be obtained through explicit or implicit feedback, including online browsing, rating and purchasing. Then by calculating the match degree between the user's attributes and the items characterized by their features, one can finally rank all the items to be predicted according to the match degree.
The advantage of this approach is that it merely relies on feature information about user preferences and items, and does not require a large number of scoring records, so there is no problem of cold start. The disadvantage is that it often meets the problem of feature extraction. In addition, this method can not measure the quality of the item recommended, and the probability of recommendation failure is usually high.

Collaborative filtering recommendation
The collaborative filtering recommendation method was put forward in 1992 by Goldberg et al., which is also known as social filtering [2]. It is widely adopted in the market and studied by many scholars, as one of the most successful algorithm at present. This method is playing a role in promoting the development of e-commerce, divided into two main categories, user-based federated filtering and itembased collaborative filtering. The main algorithm is to calculate the similarity among the clients based on the history and interests of the users and other data feedback, then to sort the target clients of the similarity and introduce them.
The collaborative filtering recommendation method is simple and easy to use, and the similarity between users can be calculated only according to the users' historical score data. But in many cases, the problems of sparse matrix caused by insufficient scoring data and the cold start of new users are often encountered.

Hybrid recommendation
The effectiveness of the recommendation systems can be improved by combining several recommendation methods to achieve hybrid recommendation. Most of the recommendation platforms used in real-world scenarios are a mixture of the above recommendations. However, in the face of multimodal auxiliary information, hybrid recommendation methods still face severe challenges.

Recommendation System Based on Neural Networks
At present, deep learning is also favored by many researchers, and it has become a new direction in the field of recommendation. The neural network in deep learning technology can learn not only the latent feature representation of users or projects, but also the complex nonlinear inter-features between users and projects, and deeply analyze user preferences, to solve some problems in the traditional recommendation method and to realize recommendation better.

Convolutional neural networks
In the 1960s, the convolutional neural network was proposed by Hubel and Wiesel, for performing simulations on locally sensitive neurons in the cat cerebral cortex [3]. The new cognition proposed by Fukushima is based on the local connections and hierarchies among neurons, and it is the first time to transform the image network. The structure of neural network can greatly reduce the complexity of neural network model and improve the generalization ability of neural network. CNN is a feed-forward neural network that is widely used in voice, image, and other fields.

Attention based convolutional neural networks. Gong et al. proposed a method to perform
hashtag recommendation in microblog using attention based convolutional neural networks [4]. The authors treated recommendation of hashtag as a multi-label classification problem, while the CNN was used as a extraction tool to capture the features of posts. The model proposed consisted of a global channel and a local attention channel. The global channel was composed by a convolutional layer and a pooling layer, and the local attention channel was composed by an attention layer and a pooling layer.  Figure 1. Attention based CNN model After that, Zhang et al. used co-attention networks to make hashtag recommendation for multimodal microblog [5]. Their work considered text and image, extracting features from images and text using CNN and RNN respectively, and then recommendation is made by combining two features. At the same time, they made a point that the tags are only related to part of the information in the image and text, and the work adopted attention networks to model this local correlation.
Zhou et al. proposed another attention based CNN algorithm, which constructed the user's personalized microblog library by extracting the temporal and regional situational data of users' attention [6]. They introduced the scenario model, which has been of great help in capturing users' preference, and also the classification model, which can improve the performance of recommendations. However, this paper only extracts temporal and regional scenarios model, the extracted scenarios model is relatively simple, how to refine the scenarios model, how to build more representative scenarios model, at the same time, how to make use of the advantage of convolutional neural network in abstract feature extraction to mine the semantic features of microblog words will be the focus of our next research.

Comparative deep learning. Lei et al. studied the problem of image recommendation based on
deep learning methods [7]. Their study pointed out that the most important thing in image recommendation is to build a bridge between the semantic understanding of the image and the user's preference or intention for the image. Thus, the image learned needs not only high expressiveness and classifiability, but also the reflection of the user's preference. To solve this problem, they proposed a comparative deep learning method, whose main idea is to use MLP and CNN to learn the implicit representation of the user and image from the user's multi-source heterogeneous data (including user portrait, label Information, etc.) and the visual information of the image, and respectively, the user and the image are mapped to the same hidden space.
In the training process of the model, the idea of comparative learning is used, in other words, the positive feedback image and the negative feedback image are used at the same time. Then the distance between the image and the user is compared, and ideally, the distance between the positive feedback image and the user should be smaller than the distance between the negative feedback image and the user. Finally, the distance between the user and the image is calculated to produce the image recommendation.

Music recommendation.
Van Den Oord et al. have studied how to use the deep learning model to solve the cold start problem in music recommendation systems [8]. In music recommendation, collaborative filtering usually faces the cold start problem, in other words, for certain music without enough data of playing, it cannot be recommend to users. The authors first use the user's historical listening data and the audio signal data of the songs, by combining weighted matrix factorization and convolutional neural network, to project the user and music into a shared hidden space to learn the implicit representation of the user and the song. Then the CNN trained is able to calculate the similarity between the user and the new music in a shared hidden space to recommend music.
He [9] used CNN to analyze the music in the data set by Log-Mel frequency spectrum, obtained the low-dimensional vector representation of the music features, and finally produces proper TopN recommendations for target users. The quality of the music recommendation that He received was higher than traditional methods such as Frunk-SVD, User-CF and CB.

Recurrent neural networks
In 1986, Wiliams et al. put forward the concept of recurrent neural networks [10]. Compared with normal fully connected network or CNN, RNN is able to process sequence data, which solves the limitation of traditional neural network in this respect. The application of RNN in recommendation system is mainly used to model the sequential effect between data, thus helping to obtain more effective user and item implicit information.

Attention based recurrent neural networks and LSTM. Similar to the attention based CNN model
we discussed earlier, the attention based mechanism is also used in recurrent neural networks to recommendation. Li et al.[11] proposed an attention based LSTM for hashtag recommendation in mircoblog, which is an advanced method combined attention with RNN to capture the sequential character of the context and identify the most informative words from the posts. Firstly, the model uses LSTM to learn the hidden states(h 1 , h 2 ,…, h n ) of the posts; meanwhile, they adopt a topic model to study the topic distribution of them. The hidden attention power value a j was calculated from the words near the first position of the post and the subject distribution of it. The output of the attention layer is vⅇc = ∑ a j n j=1 h j .

Context-aware sequential recommendation.
A new model based on RNN was proposed by Liu et al. [12], namely context-aware recurrent neural network. Instead of using constant input matrix and transition matrix in the conventional RNN model, they introduced context-aware input matrix and context-aware transition, letting the parameters of each layer in the matrix vary with the context input or transited. The adaptive context-aware input matrix can capture the external context of users' behavior, such as time, location and weather. The adaptive context-aware transformation matrix can capture the time intervals between adjacent actions in a historical sequence, having the effect of transforming global sequence features. The CA-RNN model can enrich the context and sequential information information, thus providing improved recommendation.

Future work
With the rapid development of big data era, the research of recommender system based on neural networks has become a hot spot. However, on basis of the discussion above, it can be seen that the application of neural networks in recommendation systems is still at the developing stage, and hopefully there will be more and more extensive attempts in the future. Therefore, we would like to elaborate on several possible research directions, which is believed to be relevant to the current neural network field.

Combination of attention based methods and neural networks
Currently, attention based mechanisms have been used in MLP, CNN, RNN and other deep learning models. Attention based RNN can do well in modeling long-term memory in sequential data, while attention based CNN can identify the most informative part from the context. Applying attention based methods to the recommendation system can help it to grasp the most informative features of the item, recommending the most representative projects while enhancing the interpretability of the model. However, in general, the current studies which combine both of them are still relatively scarce, so it is expected that there is more in-depth and more extensive research in the future.

Cross-domain recommendation based on neural networks
As the capability of data acquisition enhances, the history information of users in different domains can be obtained. By using deep learning technology, we can construct a prediction model based on neural networks to effectively fuse various types of cross-platform data for recommendation. As is applied in actual systems of companies like Google and Mircosoft, it will be a promising arena for scholars to explore more on cross-domain information fusion to construct deep learning recommender models.

Conclusion
Compared with traditional ones, deep learning based recommendation systems can automatically study the hidden features of users and items through modelling the sequential patterns in user behavior, reflect the users' preferences more effectively, and therefore improve the accuracy of recommendation. Apart from analysis of the problems of traditional recommendation algorithms, the author introduces and analyzes the research status and progress of the recommendation system based on deep neural networks, as well as discusses their directions forward, hoping to provide useful help for researchers and engineers in related fields this paper focuses on. However, the methods in recommender systems are not comprehensive enough, and more in-depth study and discussion will be carried out in the future.