Abstract
Hierarchical text classification (HTC) is a challenging task that classifies textual descriptions with a taxonomic hierarchy. Existing methods have difficulties in modeling the hierarchical label structure. They focus on using the graph embedding methods to encode the hierarchical structure, ignoring that the HTC labels are based on a tree structure. There is a difference between tree and graph structure: in the graph structure, message passing is undirected, which will lead to the imbalance of message transmission between nodes when applied to HTC. As the nodes in different layers have inheritance relationships, the information transmission between nodes should be directional and hierarchical in the HTC task. In this paper, we propose a Top-Down Tree Structure Awareness Model to extract the hierarchical structure features, called ToSA. We regard HTC as a sequence generation task and introduce a priori hierarchical information in the decoding process. We block the information flow in one direction to ensure the graph embedding method is more suitable for the HTC task, then get the enhanced tree structure representation. Experiment results show that our model can achieve the best results on both the public WOS dataset and a collected E-commerce user intent classification dataset\(^3\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This E-commerce user intent classification dataset is collected from an intelligent service robot designed for creating an innovative online shopping experience in an E-commerce website.
References
Abburi, H., Parikh, P., Chhaya, N., Varma, V.: Fine-grained multi-label sexism classification using a semi-supervised multi-level neural approach. Data Sci. Eng. 6(4), 359–379 (2021)
Banerjee, S., Akkaya, C., Perez-Sorrosal, F., Tsioutsiouliklis, K.: Hierarchical transfer learning for multi-label text classification. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (2019)
Borges, H.B., Nievola, J.C.: Multi-label hierarchical classification using a competitive neural network for protein function prediction. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2012)
Cerri, R., Barros, R.C., PLF de Carvalho, A.C., Jin, Y.: Reduction strategies for hierarchical multi-label classification in protein function prediction. BMC Bioinform. 17(1), 1–24 (2016)
Cesa-Bianchi, N., Gentile, C., Tironi, A., Zaniboni, L.: Incremental algorithms for hierarchical classification. In: Advances in Neural Information Processing Systems, pp. 233–240 (2004)
Du, G., Zhou, L., Yang, Y., Lü, K., Wang, L.: Deep multiple auto-encoder-based multi-view clustering. Data Sci. Eng. 6(3), 323–338 (2021)
Fall, C.J., Törcsvári, A., Benzineb, K., Karetka, G.: Automated categorization in the international patent classification. In: ACM Sigir Forum, vol. 37, pp. 10–25. ACM, New York (2003)
Gopal, S., Yang, Y.: Recursive regularization for large-scale classification with hierarchical and graphical dependencies. In: Proceedings of the 19th ACM SIGKDD International Conference On Knowledge Discovery and Data Mining (2013)
Huang, W., et al.: Hierarchical multi-label text classification: An attention-based recurrent network approach. In: Proceedings of the 28th Acm International Conference On Information And Knowledge Management, pp. 1051–1060 (2019)
Kenton, J.D.M.W.C., Toutanova, L.K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT (2019)
Kim, Y.: Convolutional neural networks for sentence classification. In: Moschitti, A., Pang, B., Daelemans, W. (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, 25–29 October 2014, Doha, Qatar, pp. 1746–1751. ACL (2014)
Li, B.H., Liu, Y., Zhang, A.M., Wang, W.H., Wan, S.: A survey on blocking technology of entity resolution. J. Comput. Sci. Technol. 35(4), 769–793 (2020)
Li, B., et al.: Short text classification model combining knowledge aware and dual attention. J. Softw. 33(10), 3565–3581 (2022)
Mao, Y., Tian, J., Han, J., Ren, X.: Hierarchical text classification with reinforced label assignment. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 445–455 (2019)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27 (2014)
Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 1556–1566 (2015)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)
Wang, W., Feng, D., Li, B., Tian, J.: ATextCNN model: a new multi-classification method for police situation. In: Yang, X., Wang, C.-D., Islam, M.S., Zhang, Z. (eds.) ADMA 2020. LNCS (LNAI), vol. 12447, pp. 135–147. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65390-3_11
Wu, J., Xiong, W., Wang, W.Y.: Learning to learn and predict: A meta-learning approach for multi-label classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019)
Zhang, Y., Qi, P., Manning, C.D.: Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2205–2215 (2018)
Zhou, J., et al.: Hierarchy-aware global model for hierarchical text classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, D. et al. (2023). ToSA: A Top-Down Tree Structure Awareness Model for Hierarchical Text Classification. In: Li, B., Yue, L., Tao, C., Han, X., Calvanese, D., Amagasa, T. (eds) Web and Big Data. APWeb-WAIM 2022. Lecture Notes in Computer Science, vol 13422. Springer, Cham. https://doi.org/10.1007/978-3-031-25198-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-25198-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25197-9
Online ISBN: 978-3-031-25198-6
eBook Packages: Computer ScienceComputer Science (R0)