Abstract
Text matching is a core problem in Natural Language Processing, and one popular approach is the Siamese model. To enhance the performance of the Siamese matching model, researchers have proposed diverse network blocks, which can be divided into Information Interaction Blocks and Feature Extraction Blocks based on their functions. Traditional approaches, employing a pipeline-based arrangement for these blocks, encounter limitations when it comes to capturing focused semantics via local reasoning. To better systematize these two types of blocks, we propose a nested Siamese text matching framework, with the introduction of the Interaction Extraction Module (IEM) and the Function Amplification Module (FAM). Specifically, the IEM acts as a Feature Extraction or Information Interaction Block, nested within both the FAM and the model structure itself. IEM assumes the responsibility for global matching within the model while FAM leverages the Late Attention and nested IEMs to sharpen local matching and reasoning capabilities. Through our experiments, the performance of this nested Siamese model has proven to be superior to that of other pipeline-based Siamese matching models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Code and appendix available: https://github.com/hggzjx/nie_match.
- 2.
In the robust experimental setup, we referred to the work of [28].
- 3.
References
Bowman, S., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632–642 (2015)
Cao, Q., Trivedi, H., Balasubramanian, A., Balasubramanian, N.: DeFormer: decomposing pre-trained transformers for faster question answering. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4487–4497 (2020)
Chen, Q., Zhu, X., Ling, Z.H., Wei, S., Jiang, H., Inkpen, D.: Enhanced LSTM for natural language inference. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1657–1668 (2017)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Dolan, B., Brockett, C.: Automatically constructing a corpus of sentential paraphrases. In: Third International Workshop on Paraphrasing (IWP2005). Asia Federation of Natural Language Processing (2005)
Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., Heck, L.: Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 2333–2338 (2013)
Humeau, S., Shuster, K., Lachaux, M.A., Weston, J.: Poly-encoders: architectures and pre-training strategies for fast and accurate multi-sentence scoring. In: Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (2020)
Iyer, S., Dandekar, N., Csernai, K., et al.: First Quora dataset release: question pairs. Data (2017). quora.com
Khattab, O., Zaharia, M.: ColBERT: efficient and effective passage search via contextualized late interaction over BERT. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 39–48 (2020)
Kim, S., Kang, I., Kwak, N.: Semantic sentence matching with densely-connected recurrent and co-attentive information. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6586–6593 (2019)
Li, D., et al.: VIRT: improving representation-based models for text matching through virtual interaction. CoRR abs/2112.04195 (2021). https://arxiv.org/abs/2112.04195
Liu, H., Dai, Z., So, D., Le, Q.V.: Pay attention to MLPS. In: Advances in Neural Information Processing Systems, vol. 34, pp. 9204–9215 (2021)
Ni, J., et al.: Sentence-T5: scalable sentence encoders from pre-trained text-to-text models. In: Findings of the Association for Computational Linguistics: ACL 2022, pp. 1864–1874 (2022)
Nogueira, R.F., Cho, K.: Passage re-ranking with BERT. CoRR abs/1901.04085 (2019). https://arxiv.org/abs/1901.04085
Palangi, H., et al.: Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval. IEEE/ACM Trans. Audio Speech Lang. Process. 24(4), 694–707 (2016)
Pang, L., Lan, Y., Guo, J., Xu, J., Wan, S., Cheng, X.: Text matching as image recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 1–67 (2020)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992 (2019)
Rey, D., Neuhäuser, M.: Wilcoxon-signed-rank test. In: Lovric, M. (ed.) International Encyclopedia of Statistical Science, pp. 1658–1659. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-04898-2_616
Rücklé, A., Pfeiffer, J., Gurevych, I.: MultiCQA: zero-shot transfer of self-supervised text matching models on a massive scale. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2471–2486 (2020)
Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G.: A latent semantic model with convolutional-pooling structure for information retrieval. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 101–110 (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, Z., Hamza, W., Florian, R.: Bilateral multi-perspective matching for natural language sentences. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 4144–4150 (2017)
Wu, Y., Wu, W., Xing, C., Zhou, M., Li, Z.: Sequential matching network: a new architecture for multi-turn response selection in retrieval-based chatbots. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 496–505 (2017)
Xiong, L., et al.: Approximate nearest neighbor negative contrastive learning for dense text retrieval. In: Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (2020)
Yang, R., Zhang, J., Gao, X., Ji, F., Chen, H.: Simple and effective text matching with richer alignment features. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4699–4709 (2019)
Zou, Y., et al.: Divide and conquer: text semantic matching with disentangled keywords and intents. In: Findings of the Association for Computational Linguistics: ACL 2022, pp. 3622–3632 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zang, J., Liu, H. (2023). How to Extract and Interact? Nested Siamese Text Matching with Interaction and Extraction. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14258. Springer, Cham. https://doi.org/10.1007/978-3-031-44192-9_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-44192-9_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44191-2
Online ISBN: 978-3-031-44192-9
eBook Packages: Computer ScienceComputer Science (R0)