Abstract
The enormous variety of products and their diverse attributes result in a large amount of product data that needs to be managed by retail companies. A single product can have several hundred attributes which are often entered manually. In addition, products have to be classified by hand in many cases by grouping them into categories based on their properties and their relationships to other products. This is a very labor-intensive, time-consuming and error-prone task.
In this paper, we present a hybrid approach for automated product classification, which assigns products automatically to the corresponding product category based on the information on their product images. For this purpose, graphical and textual information is extracted from the product images and matched with already classified data using machine learning methods. Our hybrid approach for automated product classification is based on the Global Product Classification (GPC) standard. Our experiments show that the combination of text-based and image-based classification leads to better results and is a promising approach to reduce the manual effort for product classification in retail.
\(^*\)This scientific work is an extension of the paper “A Hybrid Approach for Product Classification based on Image and Text Matching” by Bast et al. [1].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bast, S., Brosch, C., Krieger, R.: A hybrid approach for product classification based on image and text matching. In: Proceedings of the 11th International Conference on Data Science, Technology and Applications - DATA, pp. 293–300. SciTePress, Lisbon, Portugal (2022)
How Global Product Classification (GPC) works, GS1. https://www.gs1.org/standards/gpc/how-gpc-works. Accessed 20 Oct 2022
Karami, E., Prasad, S., Shehata, M.: Image matching using SIFT, SURF, BRIEF and ORB: performance comparison for distorted images. In: Proceedings of the 2015 Newfoundland Electrical and Computer Engineering Conference, St. Johns, Canada (2015)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)
Wei, Y., Tran, S., Xu, S., Kang, B., Springer, M.: Deep learning for retail product recognition: challenges and techniques. Comput. Intell. Neurosci. 2020, 1–23 (2020)
Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, CJam (2022). https://doi.org/10.1007/978-1-84882-935-0
Ma, J., Jiang X., Fan A., Jiang J., Yan J.: Image matching from handcrafted to deep features: a survey. Int. J. Comput. Vis. 129, 23–79. Springer International Publishing (2021)
Chavaltada, C., Pasupa, K., Hardoon, D.R.: A comparative study of machine learning techniques for automatic product categorisation. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10261, pp. 10–17. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59072-1_2
Allweyer, O., Schorr, C., Krieger, R.: Classification of products in retail using partially abbreviated product names only. In: Proceedings of the 9th International Conference on Data Science, Technology and Applications - DATA, pp. 67–77. SciTePress, Paris, France (2010)
Kannan, A., Talukdar, P. P., Rasiwasia, N., Ke, Q.: Improving product classification using images. In: IEEE 11th International Conference on Data Mining, pp. 310–319 (2011)
GS1 Product Images Application Guideline for the Retail Grocery & Foodservice Industries, GS1, https://www.gs1us.org/content/dam/gs1us/documents/industries-insights/by-industry/food/guideline-toolkit/GS1-US-Product-Images-Application-Guideline-for-the-Retail-Grocery-And-Foodservice-Industries.pdf. Accessed 14 Oct 2022
Basic Operations on Images, OpenCV. https://docs.opencv.org/4.x/d3/df2/tutorial_py_basic_ops.html. Accessed 16 Dec 2022
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition - CVPR, pp. 770–778. IEEE, Las Vegas, USA (2016)
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C.: A survey on deep transfer learning. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 270–279. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_27
Deng, J., Dong, W., Socher, R., Li, L., Kai, L., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition - CVPR, pp. 248–255, IEEE, Miami, USA (2009)
Lever, J., Krzywinski, M., Altman, N.: Principal component analysis. Nat. Methods 14, 641–642 (2017)
Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Proceedings of Advances in Neural Information Processing Systems - NIPS, pp. 513–520. NeurIPS Proceedings, Vancouver, Canada (2004)
Mitchell, T.: Machine Learning. McGraw-Hill Education Ltd, New York City (1997)
Machine Learning in Python, scikit-learn, https://scikit-learn.org. Accessed 14 Oct 2022
Tensorflow software library for machine learning and artificial intelligence, Google Brain. https://www.tensorflow.org. Accessed 02 Nov 2022
Google Vision API. https://cloud.google.com/vision. Accessed 14 Oct 2022
Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45, 427–437 (2009)
Metrics and scoring: quantifying the quality of predictions, section 3.3.2.9. Precision, recall and F-measures, scikit-learn, https://scikit-learn.org/stable/modules/model_evaluation.html. Accessed 14 Oct 2022
Nearest Neighbors, section 1.6.2. Nearest Neighbors Classification, scikit-learn, https://scikit-learn.org/stable/modules/neighbors.html. Accessed 02 Nov 2022
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: IEEE Conference on Computer Vision and Pattern Recognition - CVPR, pp. 2818–2826. IEEE, Las Vegas, USA (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations - ICLR, San Diego, USA (2015)
Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648. University of Wisconsin-Madison, Wisconsin, USA (2009)
Acknowledgments
This work was funded by the German Federal Ministry of Education and Research as part of the research program KMU-innovativ: IKT (FKZ 01IS20085).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bast, S., Brosch, C., Krieger, R. (2023). Combining Image and Text Matching for Product Classification in Retail\(^*\). In: Cuzzocrea, A., Gusikhin, O., Hammoudi, S., Quix, C. (eds) Data Management Technologies and Applications. DATA DATA 2022 2021. Communications in Computer and Information Science, vol 1860. Springer, Cham. https://doi.org/10.1007/978-3-031-37890-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-37890-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37889-8
Online ISBN: 978-3-031-37890-4
eBook Packages: Computer ScienceComputer Science (R0)