Abstract
Statistical relational learning (SRL) algorithms have succeeded in many real-world applications as real-world data is relational and consists of different entities characterized by different sets of attributes. Real-world data can also be noisy and have incomplete information. Like traditional machine learning models, SRL models also assume training and testing data are sampled from the same distribution. If distributions differ, a new model must be trained using newly collected data. Employing Transfer Learning to machine learning models has become a great asset in handling such issues. It aims to leverage the knowledge learned in a source domain to train a model in a target domain. Moreover, SRL models may suffer from insufficient high-quality data instances and a long training time. Recent work has shown that applying transfer learning is suitable for SRL models as it admits training and testing data sampled from different distributions. However, an essential challenge is how to transfer the learned structure, mapping the vocabulary across different domains. This work relies on a previous approach that uses the similarity between pre-trained word embeddings to guide the mapping and applies theory revision to improve its inferential capacities. However, choosing the most suitable similarity metrics for a specific pair of source and target datasets is not trivial. Thus, we propose to combine different similarity metrics to map predicates. Experimental results showed that combining distinct similarity metrics has improved or equated performance compared to previous methods. It also requires less training time for some experiments.
Supported by CAPES, FAPERJ, and CNPq.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The source code and experiments are publicly available at https://github.com/MeLL-UFF/TransBoostler.
References
Azevedo Santos, R., Paes, A., Zaverucha, G.: Transfer learning by mapping and revising boosted relational dependency networks. Mach. Learn. 109(7), 1435–1463 (2020). https://doi.org/10.1007/s10994-020-05871-x
Bilenko, M., Mooney, R.J.: Adaptive duplicate detection using learnable string similarity measures. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 39–48. KDD 2003, Association for Computing Machinery, New York, NY, USA (2003). https://doi.org/10.1145/956750.956759
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching Word Vectors with Subword Information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017). https://doi.org/10.1162/tacl_a_00051
Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E.R., Mitchell, T.M.: Toward an architecture for never-ending language learning. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, pp. 1306–1313. AAAI 2010, AAAI Press (2010)
Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240. ICML 2006, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1143844.1143874
Raedt, L.: Logical and relational learning. In: Zaverucha, G., da Costa, A.L. (eds.) SBIA 2008. LNCS (LNAI), vol. 5249, pp. 1–1. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88190-2_1
Emerson, P.: The original borda count and partial voting. Soc. Choice Welf. 40(2), 353–358 (2013)
de Figueiredo, L.F., Paes, A., Zaverucha, G.: Transfer learning for boosted relational dependency networks through genetic algorithm. In: Katzouris, N., Artikis, A. (eds.) Inductive Logic Programming, pp. 125–139. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-030-97454-1_9
Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, vol. 2, pp. 1300–1307. IJCAI 1999, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1999)
Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning). The MIT Press, Cambridge (2007)
Haaren, J.V., Kolobov, A., Davis, J.: Todtler: two-order-deep transfer learning. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 3007–3015. AAAI 2015, AAAI Press (2015)
Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)
Khosravi, H., Bina, B.: A survey on statistical relational learning. In: Farzindar, A., Kešelj, V. (eds.) AI 2010. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence), vol. 6085, pp. 256–268. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13059-5_25
Khosravi, H., Schulte, O., Hu, J., Gao, T.: Learning compact Markov logic networks with decision trees. In: Muggleton, S.H., Tamaddoni-Nezhad, A., Lisi, F.A. (eds.) Inductive Logic Programming, pp. 20–25. Springer, Heidelberg (2012). https://doi.org/10.1007/s10994-012-5307-6
Kumaraswamy, R., Odom, P., Kersting, K., Leake, D., Natarajan, S.: Transfer learning via relational type matching. In: 2015 IEEE International Conference on Data Mining, pp. 811–816 (2015). https://doi.org/10.1109/ICDM.2015.138
Kusner, M., Sun, Y., Kolkin, N., Weinberger, K.: From word embeddings to document distances. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning Research, vol. 37, pp. 957–966. PMLR, Lille, France, 07–09 July 2015
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Luca, T., Paes, A., Zaverucha, G.: Mapping across relational domains for transfer learning with word embeddings-based similarity. In: Katzouris, N., Artikis, A. (eds.) Inductive Logic Programming, pp. 167–182. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-030-97454-1_12
Mewes, H.W., et al.: MIPS: a database for genomes and protein sequences. Nucleic Acids Res. 28, 37–40 (2000). https://doi.org/10.1093/nar/28.1.37
Mihalkova, L., Huynh, T., Mooney, R.J.: Mapping and revising Markov logic networks for transfer learning. In: Proceedings of the 22nd National Conference on Artificial Intelligence - Volume 1, pp. 608–614. AAAI 2007, AAAI Press (2007)
Mihalkova, L., Mooney, R.J.: Bottom-up learning of Markov logic network structure. In: Proceedings of the 24th International Conference on Machine Learning, pp. 625–632. ICML 2007, Association for Computing Machinery, New York, NY, USA (2007). https://doi.org/10.1145/1273496.1273575
Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., Joulin, A.: Advances in pre-training distributed word representations. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan (May 2018). https://aclanthology.org/L18-1008
Natarajan, S., Khot, T., Kersting, K., Gutmann, B., Shavlik, J.: Gradient-based boosting for statistical relational learning: the relational dependency network case. Mach. Learn. 86(1), 25–56 (2012). https://doi.org/10.1007/s10994-011-5244-9
Neville, J., Jensen, D.: Relational dependency networks. J. Mach. Learn. Res. 8, 653–692 (2007)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010). https://doi.org/10.1109/TKDE.2009.191
Salton, G., Wong, A., Yang, C.S.: A vector space model for automatic indexing. Commun. ACM 18(11), 613–620 (1975)
Sidorov, G., Gelbukh, A., Gomez Adorno, H., Pinto, D.: Soft similarity and soft cosine measure: similarity of features in vector space model. Computación y Sistemas 18, 491–504 (2014). https://doi.org/10.13053/cys-18-3-2043
Torregrossa, F., Allesiardo, R., Claveau, V., Kooli, N., Gravier, G.: A survey on training and evaluation of word embeddings. Int. J. Data Sci. Anal. 11, 1–19 (2021)
Wrobel, S.: First order theory refinement. Adv. Induct. Logic Program. 32, 14–33 (1996)
Yang, Q., Zhang, Y., Dai, W., Pan, S.J.: Transfer Learning. Cambridge University Press, Cambridge (2020). https://doi.org/10.1017/9781139061773
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Luca, T., Paes, A., Zaverucha, G. (2024). Combining Word Embeddings-Based Similarity Measures for Transfer Learning Across Relational Domains. In: Muggleton, S.H., Tamaddoni-Nezhad, A. (eds) Inductive Logic Programming. ILP 2022. Lecture Notes in Computer Science(), vol 13779. Springer, Cham. https://doi.org/10.1007/978-3-031-55630-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-55630-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-55629-6
Online ISBN: 978-3-031-55630-2
eBook Packages: Computer ScienceComputer Science (R0)