Abstract
Automated task-oriented dialog based system, generally stated as Chatbot, is widely used nowadays by service-oriented platforms such as banking, mobile service providers and travel management firms. The most imperative part of the task-oriented dialog system is to distinguish the intent of the queries asked. If the system erroneously identifies the intent of the query, then the given answer is either incorrect or not related to the query asked. This raises the risk of deteriorating the reliability of the entire system and the organization. Such kind of systems struggle when a user asks queries that contain words for which training classes are not available. These classes may be termed as unseen classes. Our aim is to find the unseen classes in an automated task-oriented dialog system. This paper focuses on open-world learning. Specifically, we propose a deep learning-based Intent Identification framework, OWI, to identify unseen classes for an automated dialog-based system. The OWI framework is based on convolutional neural network with a 1-vs-rest output layer to identify the unseen classes. The proposed model is evaluated on various performance matrices. In addition, we compare OWI with an existing state-of-the-art model. The experimental results show that the OWI outperforms the existing model with respect to identifying unseen classes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Thomas, N.T.: An e-business chatbot using AIML and LSA. In: International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 2740–2742 (2016)
Naveen Kumar, M., Linga Chandar, P.C., Venkatesh Prasad, A., Sumangali, K.: Android based educational chatbot for visually impaired people. In: IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), pp. 1–4 (2016)
Braun, D., Hernandez Mendez, A., Matthes, F., Langen, M.: Evaluating natural language understanding services for conversational question answering systems. In: Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pp. 174–185 (2017)
Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R.: A new chatbot for customer service on social media. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 3506–3510 (2017)
Paweł Budzianowski, T.-H.W., et al.: Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 5016–5026 (2018)
Purohit, H., Dong, G., Shalin, V., Thirunarayan, K., Sheth, A.: Intent classification of short-text on social media. In: Proceedings of the IEEE International Conference on Smart City/Socialcom/Sustaincom (Smartcity), pp. 222–228 (2015)
Helmi Setyawan, M.Y., Awangga, R.M., Efendi, S.R.: Comparison of multinomial naive bayes algorithm and logistic regression for intent classification in chatbot. In: Proceedings of the International Conference on Applied Engineering (ICAE), pp. 1–5 (2018)
Nigam, A., Sahare, P., Pandya, K.: Intent detection and slots prompt in a closed-domain chatbot. In: Proceedings of the IEEE 13th International Conference on Semantic Computing (ICSC), pp. 340–343 (2019)
Shridhar, K., et al.: Subword semantic hashing for intent classification on small datasets. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2019)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Wu, C.-S., Hoi, S., Socher, R., Xiong, C.: TOD-BERT: pre-trained natural language understanding for task-oriented dialogues. arXiv preprint arXiv:2004.06871 (2020)
Casanueva, I., Temčinas, T., Gerz, D., Henderson, M., Vulić, I.: Efficient intent detection with dual sentence encoders. arXiv preprint arXiv:2003.04807 (2020)
Lin, T.-E., Hua, X.: A post-processing method for detecting unknown intent of dialogue system via pre-trained deep neural network classifier. Knowl. Based Syst. 186, 104979 (2019)
Yan, G., et al.: Unknown intent detection using Gaussian mixture model with an application to zero-shot intent classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1050–1060 (2020)
Rifkin, R., Klautau, A.: In defense of One-Vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)
Larson, S., et al.: An evaluation dataset for intent classification and out-of-scope prediction. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1311–1316 (2019)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751 (2014)
Schuurmans, J., Frasincar, F.: Intent classification for dialogue utterances. IEEE Intell. Syst. 35(1), 82–88 (2019)
Bhardwaj, A., Rudnicky, A.: User intent classification using memory networks: a comparative analysis for a limited data scenario. arXiv preprint arXiv:1706.06160 (2017)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Subramani, S., Vu, H.Q., Wang, H.: Intent classification using feature sets for domestic violence discourse on social media. In: Proceedings of the 4th Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), pp. 129–136 (2017)
Zhou, J., Xu, W.: End-to-end learning of semantic role labeling using recurrent neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Volume 1: Long Papers, pp. 1127–1137 (2015)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
Gupta, A., Hewitt, J., Kirchhoff, K.: Simple, fast, accurate intent classification and slot labeling for goal-oriented dialogue systems. In: Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pp. 46–55 (2019)
Jayarao, P., Srivastava, A.: Intent detection for code-mix utterances in task oriented dialogue systems. In: Proceedings of the International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT), pp. 583–587 (2018)
Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Patt. Anal. Mach. Intell. 35(7), 1757–1772 (2012)
Jain, L.P., Scheirer, W.J., Boult, T.E.: Multi-class open set recognition using probability of inclusion. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 393–409. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_26
Shu, L., Xu, H., Liu, B.: DOC: deep open classification of text documents. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 2911–2916 (2017)
Li, Z., Hoiem, D.: Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 2935–2947 (2017)
Shu, L., Xu, H., Liu, B.: Unseen class discovery in open-world classification. arXiv preprint arXiv:1801.05609 (2018)
Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)
Vedula, N., Lipka, N., Maneriker, P., Parthasarathy, S.: Open intent extraction from natural language interactions. In: Proceedings of The Web Conference, pp. 2009–2020 (2020)
Vedula, N., Gupta, R., Alok, A., Sridhar, M.: Automatic discovery of novel intents & domains from text utterances. arXiv preprint arXiv:2006.01208 (2020)
Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Parmar, J., Soni, S., Chouhan, S.S. (2020). OWI: Open-World Intent Identification Framework for Dialog Based System. In: Bellatreche, L., Goyal, V., Fujita, H., Mondal, A., Reddy, P.K. (eds) Big Data Analytics. BDA 2020. Lecture Notes in Computer Science(), vol 12581. Springer, Cham. https://doi.org/10.1007/978-3-030-66665-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-66665-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66664-4
Online ISBN: 978-3-030-66665-1
eBook Packages: Computer ScienceComputer Science (R0)