Abstract
The objective of the paper is to analyze the algorithms and methods used in dialogue chatbot systems in terms of usability and quality of their functioning. The research used two knowledge corpuses differing in subject matter, number of specific intents, availability of entities, and number of training and test data. The research was performed on three platforms - RASA, Dialogflow, and IBM Watson. The influence of the number of intents and the presence of entities on the quality of the dialogue system using predefined metrics was examined. Additionally, the experiments concerned the analysis of chatbot operation for different ratios of training to test data. Moreover, the influence of the platform and corpus selection on the quality of operation and acceptance level of intent was examined.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abu Shawar, B., Atwell, E.: Different measurement metrics to evaluate a chatbot system. In: Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies, Rochester, NY, pp. 89–96. Association for Computational Linguistics, April 2007
Chandna, U., Iyer, M.R.: System for semi-automated chatbots query classification training corpus generation (2018)
Hristidis, V.: Chatbot technologies and challenges (2018)
Hussain, S., Ameri Sianaki, O., Ababneh, N.: A survey on conversational agents/chatbots classification and design techniques. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds.) WAINA 2019. AISC, vol. 927. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15035-8_93
Cahn, J.: CHATBOT: architecture, design & development, April 2017
Weizenbaum, J.: ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 1, 36–45 (1966)
Jain, M., Kumar, P., Kota, R., Patel, S.N.: Evaluating and informing the design of chatbots. In: Proceedings of the 2018 Designing Interactive Systems Conference, New York, NY, USA, DIS 2018, pp. 895–906. Association for Computing Machinery (2018)
Larson, S., et al.: An evaluation dataset for intent classification and out-of-scope prediction. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 1311–1316. Association for Computational Linguistics (2019)
Meteer, M., Hickey, M., Rothberg, C., Nahamoo, D., Eide, K.: Are the tools up to the task? An evaluation of commercial dialog tools in developing conversational enterprise-grade dialog systems. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers), Minneapolis, Minnesota, pp. 106–113. Association for Computational Linguistics (2019)
Lee, I., Shin, Y.J.: Machine learning for enterprises: applications, algorithm selection, and challenges. Bus. Horiz. 63(2), 157–170 (2020)
Turing, A.M.: Computing machinery and intelligence. Mind LIX(236), 433–460 (1950)
Minh, P.Q.N.: Natural language processing problems in chatbot system development (2020). https://techinsight.com.vn/language/en/natural-language-processing-problems-chatbot-system-development/. Accessed 26 Mar 2020
Radziwill, N.M., Benton, M.C.: Evaluating quality of chatbots and intelligent conversational agents (2017)
Rahman, A.M., Al Mamun, A., Islam, A.: Programming challenges of chatbot: Current and future prospective (2017)
Wallace, R.S.: The anatomy of A.L.I.C.E.. In: Epstein, R., Roberts, G., Beber, G. (eds.) Parsing the Turing Test, pp. 181–210. Springer, Dordrecht (2009). https://doi.org/10.1007/978-1-4020-6710-5_13
Shum, H.-Y., He, X.-D., Li, D.: From Eliza to Xiaoice: challenges and opportunities with social chatbots. Front. Inf. Technol. Electron. Eng. 19(1), 10–26 (2018)
Liu, X., Eshghi, A., Swietojanski, P., Rieser, V.: Benchmarking natural language understanding services for building conversational agents (2019)
WWW. What is a good value for the ML classification threshold in dialogflow? (2020). https://miningbusinessdata.com/good-value-ml-classification-threshold-dialogflow/. Accessed 30 Apr 2020
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pogoda, A., Lyko, E., Kedziora, M., Jozwiak, I., Pietraszko, J. (2022). Multiplatform Comparative Analysis of Intelligent Robots for Communication Efficiency in Smart Dialogs. In: Barolli, L., Hussain, F., Enokido, T. (eds) Advanced Information Networking and Applications. AINA 2022. Lecture Notes in Networks and Systems, vol 450. Springer, Cham. https://doi.org/10.1007/978-3-030-99587-4_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-99587-4_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-99586-7
Online ISBN: 978-3-030-99587-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)