Abstract
Numerous graph classifiers are readily available and frequently used in both research and industry. Ensuring their performance across multiple domains and applications is crucial. In this paper, we conduct a comprehensive assessment of three commonly used graph-based classifiers across 24 graph datasets (we employ classifiers based on graph matchings, graph kernels, and graph neural networks). Our goal is to find out what primarily affects the performance of these classifiers in different tasks. To this end, we compare each of the three classifiers in three different scenarios. In the first scenario, the classifier has access to the original graphs, in the second scenario, the same classifier has access only to the structure of the graph (without labels), and in the third scenario, we replace the graph-based classifiers with a corresponding related statistical classifier, which has access only to an aggregated feature vector of the graph labels. On the basis of this exhaustive evaluation, we are able to suggest whether or not certain graph datasets are suitable for specific benchmark comparisons.
Supported by the Swiss National Science Foundation (SNSF) under Grant Nr. 200021_188496.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hutson, M.: Artificial intelligence faces reproducibility crisis. Science 359(6377), 725–726 (2018). https://doi.org/10.1126/science.359.6377.725
Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: Friedler, S.A., Wilson, C. (eds.) Conference on Fairness, Accountability and Transparency, FAT 2018, 23–24 February 2018, New York, USA. Proceedings of Machine Learning Research, vol. 81, pp. 77–91. PMLR (2018)
Shmueli, G.: To explain or to predict? Stat. Sci. 25(3) (2010). https://doi.org/10.1214/10-STS330
Errica, F., Podda, M., Bacciu, D., Micheli, A.: A fair comparison of graph neural networks for graph classification. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)
Morris, C., Kriege, N.M., Bause, F., Kersting, K., Mutzel, P., Neumann, M.: TUDataset: a collection of benchmark datasets for learning with graphs (2020). arXiv: 2007.08663
Bunke, H., Allermann, G.: Inexact graph matching for structural pattern recognition. Pattern Recogn. Lett. 1(4), 245–253 (1983). https://doi.org/10.1016/0167-8655(83)90033-8. May
Kriege, N.M., Johansson, F.D., Morris, C.: A survey on graph kernels. Appl. Netw. Sci. 5(1), 6 (2020). https://doi.org/10.1007/s41109-019-0195-3
Yang, Z., et al.: A comprehensive survey of graph-level learning (2023). arXiv: 2301.05860
Schulz, T., Welke, P.: On the necessity of graph kernel baselines. In: Graph Embedding and Mining Workshop at ECML PKDD, p. 13 (2019). https://www.semanticscholar.org/paper/On-the-Necessity-of-Graph-Kernel-Baselines-Schulz-Welke/83c7fa53129c983e3124ad0b0d436d982a58c44f
Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., USA (1990)
Riesen, K., Bunke, H.: Approximate graph edit distance computation by means of bipartite graph matching. Image Vis. Comput. 27(7), 950–959 (2009). https://doi.org/10.1016/j.imavis.2008.04.004. Jun
Shervashidze, N., Schweitzer, P., Leeuwen, E.J.v., Mehlhorn, K., Borgwardt, K.M. J.: Weisfeiler-Lehman graph kernels. Mach. Learn. Res. 12, 2539–2561 (2011). https://doi.org/10.5555/1953048.2078187
Zhang, M., Cui, Z., Neumann, M., Chen, Y.: An end-to-end deep learning architecture for graph classification. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI 2018), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2018), New Orleans, Louisiana, USA, 2–7 February 2018, pp. 4438–4445. AAAI Press (2018)
Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accuracy and its posterior distribution. In: 20th International Conference on Pattern Recognition, ICPR 2010, Istanbul, Turkey, 23–26 August 2010, pp. 3121–3124. IEEE Computer Society (2010). https://doi.org/10.1109/ICPR.2010.764
Bouckaert, R.R.: Choosing between two learning algorithms based on calibrated tests. In: Fawcett, T., Mishra, N. (eds.) Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), 21–24 August 2003, Washington, DC, USA, pp. 51–58. AAAI Press (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gillioz, A., Riesen, K. (2023). Graph-Based vs. Vector-Based Classification: A Fair Comparison. In: Vento, M., Foggia, P., Conte, D., Carletti, V. (eds) Graph-Based Representations in Pattern Recognition. GbRPR 2023. Lecture Notes in Computer Science, vol 14121. Springer, Cham. https://doi.org/10.1007/978-3-031-42795-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-42795-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42794-7
Online ISBN: 978-3-031-42795-4
eBook Packages: Computer ScienceComputer Science (R0)
