Skip to main content
SpringerLink
Log in

Meta-path reasoning of knowledge graph for commonsense question answering

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Commonsense question answering (CQA) requires understanding and reasoning over QA context and related commonsense knowledge, such as a structured Knowledge Graph (KG). Existing studies combine language models and graph neural networks to model inference. However, traditional knowledge graph are mostly concept-based, ignoring direct path evidence necessary for accurate reasoning. In this paper, we propose MRGNN (Meta-path Reasoning Graph Neural Network), a novel model that comprehensively captures sequential semantic information from concepts and paths. In MRGNN, meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously. We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets, showing the effectiveness of MRGNN. Also, we conduct further ablation experiments and explain the reasoning behavior through the case study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lin B Y, Chen X, Chen J, Ren X. KagNet: knowledge-aware graph networks for commonsense reasoning. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2829–2839

  2. Feng Y, Chen X, Lin B Y, Wang P, Yan J, Ren X. Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. 2020, 1295–1309

  3. Yasunaga M, Ren H, Bosselut A, Liang P, Leskovec J. QA-GNN: reasoning with language models and knowledge graphs for question answering. In: Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021, 535–546

  4. Talmor A, Herzig J, Lourie N, Berant J. CommonsenseQA: a question answering challenge targeting commonsense knowledge. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4149–4158

  5. Hao S, Zhang H. Performance analysis of PHY layer for RIS-assisted wireless communication systems with retransmission protocols. Journal of King Saud University-Computer and Information Sciences, 2022, 34(8): 5388–5404

    Article  Google Scholar 

  6. Devlin J, Chang M W, Lee K, Toutanova K. BERT: pretraining of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186

  7. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. ALBERT: a lite BERT for self-supervised learning of language representations. In: Proceedings of the 8th International Conference on Learning Representations. 2020

  8. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P J. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 2020, 21(1): 140

    MathSciNet  MATH  Google Scholar 

  9. Khashabi D, Min S, Khot T, Sabharwal A, Tafjord O, Clark P, Hajishirzi H. UNIFIEDQA: crossing format boundaries with a single QA system. In: Proceedings of Findings of the Association for Computational Linguistics: EMNLP 2020. 2020, 1896–1907

  10. Speer R, Chin J, Havasi C. ConceptNet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 4444–4451

  11. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: a robustly optimized BERT pretraining approach. 2016, arXiv preprint arXiv: 1907.11692

  12. Shen T, Geng X, Qin T, Guo D, Tang D, Duan N, Long G, Jiang D. Multi-task learning for conversational question answering over a large-scale knowledge base. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 2442–2451

  13. Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. 2017, arXiv preprint arXiv: 1609.02907

  14. Schlichtkrull M, Kipf T N, Bloem P, Van Den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on the Semantic Web. 2018, 593–607

  15. Veličković P, Casanova A, Liò P, Cucurull G, Romero A, Bengio Y. Graph attention networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018

  16. Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 2787–2795

  17. Sun Y, Han J. Mining Heterogeneous Information Networks: Principles and Methodologies. Synthesis Lectures on Data Mining and Knowledge Discovery. San Rafael: Morgan & Claypool Publishers, 2012

    Book  Google Scholar 

  18. Santoro A, Raposo D, Barrett D G T, Malinowski M, Pascanu R, Battaglia P W, Lillicrap T. A simple neural network module for relational reasoning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 4974–4983

  19. Heilman M, Smith N A. Good question! statistical ranking for question generation. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association for Computational Linguistics. 2010, 609–617

  20. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

    Article  Google Scholar 

  21. Mihaylov T, Clark P, Khot T, Sabharwal A. Can a suit of armor conduct electricity? A new dataset for open book question answering. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 2381–2391

  22. Mihaylov T, Frank A. Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 821–832

  23. Monea A M, Marginean A N. Medical question entailment based on textual inference and fine-tuned BioMed-RoBERTa. In: Proceedings of the 17th IEEE International Conference on Intelligent Computer Communication and Processing. 2021, 319–326

  24. Wang X, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and the 31st Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. 2019, 7208–7215

  25. Zhan X, Huang Y, Dong X, Cao Q, Liang X. PathReasoner: explainable reasoning paths for commonsense question answering. Knowledge-Based Systems, 2022, 235: 107612

    Article  Google Scholar 

  26. Yan J, Raman M, Chan A, Zhang T, Rossi R, Zhao H, Kim S, Lipka N, Ren X. Learning contextualized knowledge structures for commonsense reasoning. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021, 4038–4051

  27. Wang J, Li X, Tan Z, Zhao X, Xiao W. Relation-aware bidirectional path reasoning for commonsense question answering. In: Proceedings of the 25th Conference on Computational Natural Language Learning. 2021, 445–453

  28. Saxena A, Tripathi A, Talukdar P P. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 4498–4507

  29. Bao J, Duan N, Yan Z, Zhou M, Zhao T. Constraint-based question answering with knowledge graph. In: Proceedings of the COLING 2016, 26th International Conference on Computational Linguistics: Technical Papers. 2016, 2503–2514

  30. Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A. Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: Proceedings of the 6th International Conference on Learning Representations. 2018

  31. Chen D, Yang M, Zheng H T, Li Y, Shen Y. Answer-enhanced path-aware relation detection over knowledge base. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 1021–1024

  32. Ji S, Pan S, Cambria E, Marttinen P, Yu P S. A survey on knowledge graphs: representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494–514

    Article  MathSciNet  Google Scholar 

  33. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of 2008 ACM SIGMOD International Conference on Management of Data. 2008, 1247–1250

  34. Kundu S, Khot T, Sabharwal A, Clark P. Exploiting explicit paths for multi-hop reading comprehension. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 2737–2747

  35. Chen W, Xiong W, Yan X, Wang W Y. Variational knowledge graph reasoning. In: Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018, 1823–1832

  36. Yih W T, Chang M W, He X, Gao J. Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 1321–1331

  37. Lan Y, Jiang J. Query graph generation for answering multi-hop complex questions from knowledge bases. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 969–974

Download references

Acknowledgements

This work was partially supported by the Key Research and Development Program of Hubei Province (2020BAB017), and the Scientific Research Center Program of National Language Commission (ZDI135-135), and the Fundamental Research Funds for the Central Universities (KJ02502022-0155, CCNU22XJ037).

Author information

Authors and Affiliations

  1. National Engineering Research Center for E-Learning, Central China Normal University, Wuhan, 430079, China

    Miao Zhang

  2. Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, 430079, China

    Miao Zhang, Tingting He & Ming Dong

  3. National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan, 430079, China

    Miao Zhang, Tingting He & Ming Dong

  4. School of Computer, Central China Normal University, Wuhan, 430079, China

    Tingting He & Ming Dong

Authors
  1. Miao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tingting He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Tingting He or Ming Dong.

Additional information

Miao Zhang received his MSc degree in computer technology from Central China Normal University, China in 2018. He is currently working toward to the PhD degree in National Engineering Research Center for E-Learning, Central China Normal University, China. His research interests include QA system and knowledge graph.

Tingting He received the BE degree and in Software Engineering from Wuhan University, China in 1985, the MSc degree in Applied Software and Theory of Computer from Wuhan University, China in 1988 and the PhD degree in applied linguistics from Central China Normal University, China in 2003. She is a Full Professor in School of Computer, Central China Normal University, China. Her areas of research interests are natural language processing, computational intelligence and deep learning.

Ming Dong received his PhD degree from Huazhong University of Science and Technology (HUST), China in 2021. He received his bachelor degree in Information Management and Information System from Zhongnan University of Economics and Law (ZUEL), China in 2015. He is a lecturer in Central China Normal University (CCNU), China. His research interests include social networks, rumor detection, and natural language processing.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, M., He, T. & Dong, M. Meta-path reasoning of knowledge graph for commonsense question answering. Front. Comput. Sci. 18, 181303 (2024). https://doi.org/10.1007/s11704-022-2336-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-2336-6

Keywords

Search

Navigation