Abstract
Text-based logical reasoning requires the model to understand the semantics of input text, and then understand the complex logical relationships within the text. Previous works equip pre-trained language models with the logical reasoning ability by training these models on datasets obtained by logical-driven text extension. However, these methods only generate instances based on logical expressions entailed within the input text. And we argue that external commonsense knowledge is still necessary for restoring the complete reasoning chains for generating more reasonable and abundant instances. To address this issue, in this paper, we propose CSKE, a commonsense knowledge enhanced text extension framework. CSKE incorporates abundant commonsense from an external knowledge base to restore the potentially missing logical expressions and encodes more logical relationships to then extend them through logical equivalence laws. Experiments on the benchmark datasets show that our method can improve the performance of logical reasoning, especially on the instances containing complex logical relationships.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, J., et al.: LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
Radford, A., et al.: Improving language understanding by generative pre-training (2018)
Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI blog 1.8, p. 9 (2019)
Devlin, J., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Yang, Z., et al.: Xlnet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems 32 (2019)
Liu, Y., et al.: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Lan, Z., et al.: Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Yu, W., et al.: ReClor: a reading comprehension dataset requiring logical reasoning. In: International Conference on Learning Representations (2019)
Wang, S., et al.: Logic-driven context extension and data augmentation for logical reasoning of text. arXiv preprint arXiv:2105.03659 (2021)
Jiao, F., et al.: MERIt: meta-path guided contrastive learning for logical reasoning. In: Findings of the Association for Computational Linguistics: ACL 2022 (2022)
Bowman, S., et al.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (2015)
Williams, A., Nikita N., Samuel B.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (2018)
Wang, A., et al.: GLUE: a multi-task benchmark and analysis platform for natural language understanding. In: Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP (2018)
Khot, T., Ashish S., Peter C.: Scitail: a textual entailment dataset from science question answering. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Liu, H., et al.: Natural language inference in context-investigating contextual reasoning over long texts. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 15 (2021)
LSAT Homepage. https://www.lsac.org/lsat/taking-lsat/test-format/logical-reasoning/logical-reasoning-sample-questions
GMAT Homepage. https://www.mba.com/exams/gmat/about-the-gmat-exam/gmat-exam-structure/verbal
Ouyang, S., Zhuosheng, Z., Hai Z.: Fact-driven logical reasoning. arXiv preprint arXiv:2105.10334 (2021)
Huang, Y., et al.: DAGN: discourse-aware graph network for logical reasoning. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2021)
Betz, G., Christian, V., Kyle, R.: Critical thinking for language models. In: Proceedings of the 14th International Conference on Computational Semantics (IWCS) (2021)
Clark, P., Oyvind, T., Kyle R.: Transformers as soft reasoners over language.In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence (2021)
ConceptNet Homepage. https://conceptnet.io/
Russell, S.J.: Artificial intelligence a modern approach. Pearson Education, Inc. (2010)
Zhao, J-K., Rudnick, E.M., Patel, J.H.: Static logic implication with application to redundancy identification. In: Proceedings. 15th IEEE VLSI Test Symposium (Cat. No. 97TB100125). IEEE (1997)
Ye, M., et al.: Unsupervised embedding learning via invariant and spreading instance feature. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Chen, T., et al.: A simple framework for contrastive learning of visual representations. In: International conference on machine learning. PMLR (2020)
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (2020)
Acknowledgements
We thank the anonymous reviewers for their constructive comments, and gratefully acknowledge the support of the Technological Innovation “2030 Megaproject” - New Generation Artificial Intelligence of China (2018AAA0101901), and the National Natural Science Foundation of China (62176079, 61976073).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zeng, Y., Ding, X., Du, L., Liu, T., Qin, B. (2022). CSKE: Commonsense Knowledge Enhanced Text Extension Framework for Text-Based Logical Reasoning. In: Sun, M., et al. Knowledge Graph and Semantic Computing: Knowledge Graph Empowers the Digital Economy. CCKS 2022. Communications in Computer and Information Science, vol 1669. Springer, Singapore. https://doi.org/10.1007/978-981-19-7596-7_9
Download citation
DOI: https://doi.org/10.1007/978-981-19-7596-7_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-7595-0
Online ISBN: 978-981-19-7596-7
eBook Packages: Computer ScienceComputer Science (R0)