Abstract
Click-Through Rate (CTR) prediction is a core task in nowadays commercial recommender systems. Feature crossing, as the mainline of research on CTR prediction, has shown a promising way to enhance predictive performance. Even though various models are able to learn feature interactions without manual feature engineering, they rarely attempt to individually learn representations for different feature structures. In particular, they mainly focus on the modeling of cross sparse features but neglect to specifically represent cross dense features. Motivated by this, we propose a novel Extreme Cross Network, abbreviated XCrossNet, which aims at learning dense and sparse feature interactions in an explicit manner. XCrossNet as a feature structure-oriented model leads to a more expressive representation and a more precise CTR prediction, which is not only explicit and interpretable, but also time-efficient and easy to implement. Experimental studies on Criteo Kaggle dataset show significant improvement of XCrossNet over state-of-the-art models on both effectiveness and efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The statistics for the dense and sparse features and proportion are based on the survey outcome conducted in December 2019 with the MLPerf Advisory Board.
- 3.
We release the source code at https://github.com/bigdata-ustc/XCrossNet/.
References
Blondel, M., et al.: Higher-order factorization machines. In: NeurIPS, pp. 3351–3359 (2016)
Chang, Y., et al.: Training and testing low-degree polynomial data mappings via linear SVM. J. Mach. Learn. Res. (JMLR) 11, 1471–1490 (2010)
Cheng, H.T., et al.: Wide & deep learning for recommender systems. In: 1st Workshop on Deep Learning for Recommender Systems, pp. 7–10 (2016)
Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Annals of Statistics, pp. 1189–1232 (2001)
Guo, H., et al.: DeepFM: a factorization-machine based neural network for CTR prediction. In: IJCAI, pp. 1725–1731 (2017)
He, X., Chua, T.S.: Neural factorization machines for sparse predictive analytics. In: SIGIR, pp. 355–364 (2017)
Huang, Z., et al.: An ad CTR prediction method based on feature learning of deep and shallow layers. In: CIKM, pp. 2119–2122 (2017)
Juan, Y., et al.: Field-aware factorization machines for CTR prediction. In: RecSys, pp. 43–50 (2016)
Lee, K., et al.: Estimating conversion rate in display advertising from past performance data. In: SIGKDD pp. 768–776. ACM (2012)
Lian, J., et al.: XDeepFM: combining explicit and implicit feature interactions for recommender systems. In: SIGKDD, pp. 1754–1763 (2018)
Liu, B., et al.: Feature generation by convolutional neural network for click-through rate prediction. In: WWW, pp. 1119–1129 (2019)
Liu, B., et al.: Autogroup: automatic feature grouping for modelling explicit high-order feature interactions in CTR prediction. In: SIGIR, pp. 199–208. ACM (2020)
Liu, Q., et al.: Personalized travel package recommendation. In: ICDM, pp. 407–416. IEEE (2011)
Liu, Q., et al.: A convolutional click prediction model. In: CIKM, pp. 1743–1746 (2015)
Luo, Y., et al.: Autocross: automatic feature crossing for tabular data in real-world applications. In: SIGKDD, pp. 1936–1945 (2019)
McMahan, H.B., et al.: Ad click prediction: a view from the trenches. In: SIGKDD, pp. 1222–1230. ACM (2013)
Pan, Z., et al.: Sparse factorization machines for click-through rate prediction. In: ICDM, pp. 400–409. IEEE (2016)
Qu, Y., et al.: Product-based neural networks for user response prediction. In: ICDM pp. 1149–1154. IEEE (2016)
Qu, Y., et al.: Product-based neural networks for user response prediction over multi-field categorical data. ACM Trans. Inf. Syst. (ACM TOIS) 37(1), 1–35 (2018)
Rendle, S.: Factorization machines. In: ICDM pp. 995–1000. IEEE (2010)
Wang, R., et al.: Deep & cross network for ad click predictions. In: ADKDD, pp. 1–7 (2017)
Wu, C.J., et al.: Developing a recommendation benchmark for MLPerf training and inference. arXiv preprint arXiv:2003.07336 (2020)
Xiao, J., et al.: Attentional factorization machines: learning the weight of feature interactions via attention networks. In: IJCAI, pp. 3119–3125 (2017)
Yan, L., et al.: Coupled group lasso for web-scale CTR prediction in display advertising. ICML 32, 802–810 (2014)
Yu, R., et al.: Collaborative list-and-pairwise filtering from implicit feedback. IEEE Trans. Know. Data Eng. (IEEE TKDE). https://doi.org/10.1109/TKDE.2020.3016732
Yu, R., et al.: Multiple pairwise ranking with implicit feedback. In: CIKM, pp. 1727–1730. ACM (2018)
Zhang, W., Du, T., Wang, J.: Deep learning over multi-field categorical data. In: Ferro, N., Crestani, F., Moens, M.-F., Mothe, J., Silvestri, F., Di Nunzio, G.M., Hauff, C., Silvello, G. (eds.) ECIR 2016. LNCS, vol. 9626, pp. 45–57. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1_4
Acknowledgements
This research was partially supported by grants from the National Key Research and Development Program of China (No. 2018YFC0832101), and the National Natural Science Foundation of China (Grants No. 61922073 and U20A20229). Qi Liu acknowledges the support of the Youth Innovation Promotion Association of CAS (No. 2014299).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, R. et al. (2021). XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12713. Springer, Cham. https://doi.org/10.1007/978-3-030-75765-6_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-75765-6_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75764-9
Online ISBN: 978-3-030-75765-6
eBook Packages: Computer ScienceComputer Science (R0)