Abstract
In multi-turn dialogue generation, the generated response should consider the content before the current turn of dialogue. Due to multiple turns, it is difficult to maintain the context consistency by using only a few previous turns of the dialogue indiscriminately. Except for the context information, we can retrieve additional candidates from historical contexts, according to semantic similarity. Therefore, in this paper, we integrate the historical information into the generative model called HRG. The HRG model can generate a response by using both context information and retrieved historical candidates, which contain richer information such as theme and latent information. We encode contexts, current turn and historical information separately to find the most important turns and give the current turn a higher level of attention. Then we propose a hierarchical fusion encoder to integrate the retrieval information through a KL divergence gate dynamically. Finally, we conduct experiments on the Ubuntu large-scale English multi-turn dialogue community dataset and Daily dialogue dataset. The results show that our hybrid model performs well on both automatic evaluation and human evaluation compared with the existing baseline models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Azzalini, F., Jin, S., Renzi, M., Tanca, L.: Blocking techniques for entity linkage: a semantics-based approach. Data Sci. Eng. 6(1), 20–38 (2021)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186 (2019)
Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 8–13 December 13 2014, Montreal, Quebec, Canada, pp. 2042–2050 (2014)
Ke, P., Guan, J., Huang, M., Zhu, X.: Generating informative responses with controlled sentence function. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 1499–1508 (2018)
Kong, Y., Zhang, L., Ma, C., Cao, C.: Hsan: A hierarchical self-attention network for multi-turn dialogue generation. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7433–7437. IEEE (2021)
Li, C., Yang, C., Liu, B., Yuan, Y., Wang, G.: LRSC: learning representations for subspace clustering. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 8340–8348 (2021)
Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 110–119 (2016)
Li, Y., Su, H., Shen, X., Li, W., Cao, Z., Niu, S.: DailyDialog: a manually labelled multi-turn dialogue dataset. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27– December 1, 2017 - Volume 1: Long Papers, pp. 986–995 (2017)
Liang, Y., Meng, F., Zhang, Y., Chen, Y., Xu, J., Zhou, J.: Infusing multi-source knowledge with heterogeneous graph neural network for emotional conversation generation. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, pp. 13343–13352 (2021)
Liu, Y., et al.: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Lowe, R., Pow, N., Serban, I., Pineau, J.: The ubuntu dialogue corpus: a large dataset for research in unstructured multi-turn dialogue systems. In: Proceedings of the SIGDIAL 2015 Conference, The 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 285–294 (2015)
Oluwatobi, O., Mueller, E.: DLGNet,: a transformer-based model for dialogue response generation. In: Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI (2020)
Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social media. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, pp. 583–593 (2011)
Serban, I., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Serban, I., et al.: A hierarchical latent variable encoder-decoder model for generating dialogues. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, ACL 2015, pp. 1577–1586 (2015)
Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., Nie, J.Y.: A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 553–562 (2015)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112. The MIT Press, London (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems NIPS 2017, pp. 5998–6008 (2017)
Wang, H., Lu, Z., Li, H., Chen, E.: A dataset for research on short-text conversations. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 935–945 (2013)
Wang, S., Jiang, J.: Learning natural language inference with LSTM. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, June 12–17, 2016, pp. 1442–1451 (2016)
Xing, C., et al.: Topic aware neural response generation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Yan, R., Song, Y., Wu, H.: Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 55–64 (2016)
Zhang, H., Lan, Y., Pang, L., Chen, H., Ding, Z., Yin, D.: Modeling topical relevance for multi-turn dialogue generation. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI (2020)
Zhang, H., Lan, Y., Pang, L., Guo, J., Cheng, X.: ReCoSa: detecting the relevant contexts with self-attention for multi-turn dialogue generation. In: Proceedings of ACL 2019, vol. 1: Long Papers, pp. 3721–3730 (2019)
Zhang, W., et al.: Multi-turn dialogue generation in e-commerce platform with the context of historical dialogue. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 1981–1990 (2020)
Zhao, X., Wu, W., Xu, C., Tao, C., Zhao, D., Yan, R.: Knowledge-grounded dialogue generation with pre-trained language models. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, 16–20 November 2020, pp. 3377–3390 (2020)
Zhu, Q., Cui, L., Zhang, W., Wei, F., Liu, T.: Retrieval-enhanced adversarial training for neural response generation. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL, pp. 3763–3773 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, D., Liu, X., Ning, B., Liu, C. (2022). HRG: A Hybrid Retrieval and Generation Model in Multi-turn Dialogue. In: Bhattacharya, A., et al. Database Systems for Advanced Applications. DASFAA 2022. Lecture Notes in Computer Science, vol 13247. Springer, Cham. https://doi.org/10.1007/978-3-031-00129-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-00129-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-00128-4
Online ISBN: 978-3-031-00129-1
eBook Packages: Computer ScienceComputer Science (R0)