Abstract
Knowledge graphs describe concepts, entities in the objective world and relations in a structured form, thus providing a better way to manage and understand the infinite information on the Internet. Although there are various knowledge embedding models, most of them only focus on factual triples. In fact, there are usually concise descriptions for entities, which cannot be well employed by these existing models. For instance, a knowledge embedding model based on convolutional networks (ConvKB [9]), has shown remarkable results in the knowledge link prediction, which have not fully utilized the complementary texts of entities. Therefore, we propose a text-augmented embedding model based on ConvKB, which firstly uses bidirectional short and long term memory network with attention (A-BiLSTM) to encode the descriptions of the entities, then combines the structure of the symbol triples embeddings and text embeddings with novel gate mechanism (in the form of the LSTM gates). In this way, structural representations and textual representations can all be learned. The experiments have shown that our method is superior to the previous ConvKB in tasks like link prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We calculate the averaged number \(\eta _{s}\) of head entities per tail entity and the averaged number \(\eta _{o}\) of tail entities per head entity. If \(\eta _{s}<1.5\) and \(\eta _{o}<1.5\), r is categorized one-to-one (1-1). If \(\eta _{s}<1.5\) and \(\eta _{o}>1.5\), r is categorized one-to-many (1-M). If \(\eta _{s} > 1 .5\) and \(\eta _{o}<1.5\), r is categorized many-to-one (M-1). If \(\eta _{s}>1.5\) and \(\eta _{o}>1.5\), r is categorized many-to-many (M-M).
- 2.
References
Predicting rich drug-drug interactions via biomedical knowledge graphs and text jointly embedding. CoRR, abs/1712.08875 (2017). Withdrawn
An, B., Chen, B., Han, X., Sun, L.: Accurate text-enhanced knowledge graph representation learning. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long Papers) (2018)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. TACL 5, 135–146 (2017)
Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held 5–8 December 2013, Lake Tahoe, Nevada, United States, pp. 2787–2795 (2013)
Cai, L., Wang, W.Y.: KBGAN: adversarial learning for knowledge graph embeddings. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, 1–6 June 2018, vol. 1 (Long Papers), pp. 1470–1480 (2018)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, 2–7 February 2018, pp. 1811–1818 (2018)
García-Durán, A., Niepert, M.: Kblrn: end-to-end learning of knowledge base representations with latent, relational, and numerical features. In Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, USA, 6–10 August 2018, pp. 372–381 (2018)
Heck, J., Salem, F.M.: Simplified minimal gated unit variations for recurrent neural networks. CoRR, abs/1701.03452 (2017)
Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.Q.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, 1–6 June 2018, vol. 2 (Short Papers), pp. 327–333 (2018)
Nguyen, D.Q., Sirts, K., Qu, L., Johnson, M.: Stranse: a novel embedding model of entities and relationships in knowledge bases. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, 12–17 June 2016, pp. 460–466 (2016)
Nickel, M., Tresp, V., Kriegel, H.-P.: A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 – July 2, 2011, pp. 809–816 (2011)
Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Workshop on Continuous Vector Space Models & Their Compositionality (2015)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, 19–24 June 2016, pp. 2071–2080 (2016)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, 27–31 July 2014, Québec City, Québec, Canada, pp. 1112–1119 (2014)
Xie, R., Liu, Z., Jia, J., Luan, H., TransEun, M.: Representation learning of knowledge graphs with entity descriptions. In: AAAI, pp. 2659–2665 (2016)
Xiong, S., Huang, W., Duan, P.: Knowledge graph embedding via relation paths and dynamic mapping matrix. In: Woo, C., Lu, J., Li, Z., Ling, T.W., Li, G., Lee, M.L. (eds.) ER 2018. LNCS, vol. 11158, pp. 106–118. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01391-2_18
Xu, J., Qiu, X., Chen, K., Huang, X.: Knowledge graph representation with jointly structural and textual encoding. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, 19–25 August 2017, pp. 1318–1324 (2017)
Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)
Zhong, H., Zhang, J., Wang, Z., Wan, H., Chen, Z.: Aligning knowledge and text embeddings by entity descriptions. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, 17–21 September 2015, pp 267–272 (2015)
Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, 7–12 August 2016, Berlin, Germany, vol. 2: Short Papers (2016)
Acknowledgment
This work was partially funded by National Natural Science Foundation of China (No.61877043 and 61877044).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, C. et al. (2019). Text-Augmented Knowledge Representation Learning Based on Convolutional Network. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11953. Springer, Cham. https://doi.org/10.1007/978-3-030-36708-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-36708-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36707-7
Online ISBN: 978-3-030-36708-4
eBook Packages: Computer ScienceComputer Science (R0)