Product recommender system using neural collaborative filtering for marketplace in indonesia

Marketplace has the potential growth in Indonesia indicated by the continued increase in the number of customers. However, the marketplace has some limitations to deliver personalized purchasing experience. Recommender system can support marketplace to overcome that limitations so that customer can find items or services based on their preferences. This study propose to develop product recommender system based on Neural Collaborative Filtering (NCF) algorithm. The product recommender system to be built is using implicit feedback data in the form of customer purchase data. Implicit feedback is reliable data for building recommendation system. The results have shown that NCF achieve the best performance and outperforms over the other collaborative filtering methods.


Introduction
Marketplace is the one of business sectors that has the great potential growth in Indonesia. According to definition, marketplace is an online site that contains many retailers to sell their products or services that can be similar to other retailers [1]. Tokopedia, one of the top marketplaces in Indonesia, has had a monthly user of 6.790.000 at Q4 2019 and increased by 45.91% from Q1 2017 [2]. However, that number is still around 2.5% of the total population of Indonesia that is 271.066.400 [3]. In fact, Google predicts that 52% of Southeast Asian e-commerce activities will be dominated by Indonesia in 2025 [4] and we also know that competition among the marketplace in Indonesia is getting tougher. There are Top 10 marketplaces in Indonesia including Shopee Indonesia, Tokopedia, Bukalapak, Lazada Indonesia, Orami, Blibli, JD.id, Sociolla, Zalora Indonesia, and Bhinneka [5].
However, some Indonesian society is still often shop offline. Based on survey data on Indonesian people who shop for groceries, 63% still choose to shop offline and 36% state that convenience is the most important factor regardless of online and offline [6]. Marketplace also has limitations which do not have staff who can help customers like in offline stores. Customers can get complete information and get convenience at purchasing experience through assistance from staff. The marketplace also has difficulty managing large number of products in a product catalogue. These limits can be overcome using features such as search and directory, but that not efficient because requires extra effort from the customer. Currently, marketplace have innovative solutions to overcome that problem using the recommendation system [7].
The recommendation system can recognize customer preferences and give suggestions of products that customers are interested in and tend to buy through the utilization of customer behavior data and  [8]. Research in the field of recommendation system is continuously improved. One of marketplace that has success in implementing a recommendation system is Amazon. Amazon utilize sales data that contains millions of products to identify potential products [9].
The recommender system algorithms are divide into collaborative filtering (CF), demographic filtering, content-based filtering, and hybrid filtering. In this study, we developed recommender system based on CF algorithm. CF is recommendation algorithm based on similarity of user past behaviour [10].
We used CF based on deep learning approach namely Neural Collaborative Filtering (NCF). NCF has been developed and showed the best performance compared to other collaborative filtering methods [11]. Some research has been done using NCF algorithm and shows the best performance such as commercial site recommendation [12] and e-learning recommendation [13]. Therefore, this paper proposed the implementation of NCF for product recommendation using implicit feedback data. We compare NCF with the conventional collaborative filtering method such as weighted matrix factorization (WMF) and bayesian personalized ranking (BPR) to measure the performance of the algorithm, and use some commonly adopted rank evaluation metrics consists of normalized discounted cumulative gain (NDCG), precision@k, and mean average precision (MAP).

Related Work
Recommender system generated by user feedback data. The user feedback can be collected either explicitly or implicitly. There are many research of product recommendation system. Research has been done in [14], [15], [16], [17], and [18]. However, That research mostly used explicit feedback based on users preference patterns from customer rating.
User interaction indirectly becomes input of implicit feedback data, so that the user is not disturbed or burdened, in contrast to explicit feedback data. Based on previous research, prediction algorithm that uses implicit feedback generate better prediction quality of user preferences than prediction algorithm that uses explicit ratings [19]. This paper uses implicit data in form of customer purchase data to generate recommender system model.
Next, we compare it with some recommender system algorithms such as WMF and BPR. In the study of Corso and Romani, there are some metrics that uses to evaluate the recommender system algorithm. In this study, the recommender system that will be built is using implicit feedback data, so the appropriate metrics are ranking metrics [20]. In this study we propose to use NDCG, precision@k, and MAP to compare the performance between algorithms.

Recommender System
The rapid growth of data causes users difficult to find products or services according to their needs and preferences. Recommender system can overcome that problems by suggesting products based on past behaviour of user [21]. There are 3 phases required to build a recommendation model [22] that depicted in Figure 1. 1. Information collection phase is part where personal users information in form of user's attribute, behaviours or content is collected. There are two different input of recommender system, implicit feedback data and explicit feedback data. Explicit feedback data is user feedback that requires effort from user. Implicit feedback data is user feedback that automatically generated by system. 2. Learning phase is when the algorithm apply to learn the user feedback obtained in the previous phase. 3. Prediction/recommendation phase which is where the algorithm give recommendation or prediction result about user preferences.

Neural Collaborative Filtering
NCF is collaborative filtering algorithm that ensemble Matrix Factorization (MF) and Multi-Layer Perceptron (MLP) that collaborate non-linearity of MLP and linearity of MF and for modelling interaction of user and item. NCF can overcome the non-linearity problem by removing the inner product and replacing it with neural architecture. Figure 2 shows an overview of the NCF architecture [11].

Figure 2. NCF architecture
The input layer consist of item feature vectors and user feature vectors which contains one hot encoded of items i and users u data. The next step is to get the latent vectors of item i and user u by and into dense vector p u and q i by the embedding layer. User-item interactions are modeled separately between linear and nonlinear perspectives at the Neural CF layer, then the two previous models are combined and used as the final result. The calculations can be defined as Equation 1-3 [11,12]: Item embedding of the MLP and GMF parts are indicated in and . User embedding of the MLP and GMF are indicated in and . The output and hidden activation functions described as σ and . The x-th layer's perceptron weight matrix and bias vector described as W x and b x . Cross entropy loss for implicit feedback is the NCF model optimization function, which is defined as Equation 4 [11,12].

Evaluation Metrics
There are some popular ranking-based evaluation metrics to measure the performance for each recommender algorithm consists of precision@k, MAP, and NDCG [23], [24]. Precision@K is extension of a conventional precision metric. Precision@K is the top K list prediction items that actually consumed by the user divided by the K list generated by the model for each user in the data. The higher precision@k value indicates that the model generate more accurate predictions. The average of all user precision values indicates the final of precision value. The Precision@K calculation is calculated as shown as follows [23].
MAP is very suitable for measuring performance for a ranking recommendation engine. The MAP value calculates how accurate the recommendations are in ranking items. The system will provide relevant advice among the first few recommended products if they have a high MAP score. The system will offer appropriate suggestions among the first few recommended products if they have a high MAP score. The first step in determining the MAP value is to calculate 'precision at the cutoff k' (P(k)) where precision is calculated by considering the list of recommendation subsets in order from first rank to k-th rank. The Average Precision (AP@k) can then be obtained using Equation 6. The MAP@k is calculated as follows [24].
A larger MAP indicates better recommendation performance. The weighted rank of the recommended item is considered by NDCG metrics. Items that have a high ranking have a greater weight on the final result. NDCG is calculated by adding up the weighted rankings of items for users. The ranking list on the proposed model shows many items that are suitable if they have high NDCG values. NDCG can be obtained by calculating DCG(u) and Z(u) first. The value Z(u) indicates the user ideal value of DCG(u) which is calculated as the best item rating for the user. Furthermore NDCG calculations can be obtained using Equations 8. and 9.

Data preparation
The data was collected at one of the marketplaces in Indonesia in 2019. The data consists of 11.918 transaction data of implicit feedback in form of customer purchase data. The data has been used consists of customer data, product data, and customer purchase data.

Implementation
At the data processing stage, the data is divided into 2 parts, 0.70 is used as training data and 0.30 is used as testing data. Then the training data is trained using a neural collaborative filtering algorithm. First, we should set the parameter model on the NCF algorithm including learning rate, batch size, and the number of predictive factors with a value of k = 10 (recommended number of top items). The last hidden layer of NCF is as a predictive factors because it determines the model capabilities. We experimented the batch size of 256, learning rate of 0.001 and 4 trials of predictive factors of [8, 16, 32, and 64]. For WMF and BPR methods, the number of predictive factors is obtain from the number of latent factors. The model that is formed from the previous data processing is measured using the NDCG, Precision@10, and MAP metrics.

Result and Discussion
The result of evaluation shown in Table 1 below. The results show that the greater the predictive factor, the better the performance of the model. Based on the evaluation results of the comparison in Table 1, NCF algorithm has shown the best performance on predictive factor of 64, this is indicated by the highest values of precision@10, MAP, and NDCG.

Conclusion
This paper studies the product recommender system for marketplace based on neural collaborative filtering. First, we generate the prediction model using NCF algorithm and then evaluate the performance of NCF method using ranking metrics consist of precision@k, MAP, and NDCG. Finally we compare NCF with BPR and WMF methods based on previous metrics. The results indicate that NCF shows best performance over the other methods.