Skip to main content
SpringerLink
Log in

Enhanced Graph Representations for Graph Convolutional Network Models

  • 1222: Intelligent Multimedia Data Analytics and Computing
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Graph Convolutional Network (GCN) is increasingly becoming popular among researchers for its capability of solving the task of classification of nodes, graphs or links. Graphs being a very useful representation for several application domains are increasingly grabbing the attention of researchers. Methods are being proposed to extract meaningful information in a form which can be used by machine learning tasks. Graph convolutional networks(GCN) fall among such methods. They propagate and transform node features information. Following the message passing strategy, a Graph neural network learns a node’s embeddings representations by aggregating representations of its neighbours and itself. In this research work we incorporate the concept of overlap to the graph data thus capturing the structural similarities in the node features. The intuition behind this proposal is that the class or label of a document represented by node \({v}_{i}\) is influenced by its own node features and the node features of its neighbourhood. It is proposed to enhance the graph representation to capture this neighbourhood. This enhanced graph is then input to the graph convolutional network model for the classification task. These measures are seen to improve the accuracy of node classification. Experiments on a number of datasets with different similarity measures demonstrate that enhancing graph representations produces better results in terms of classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

Public datasets have been used.

Code availability

Code can be made available on request.

References

  1. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res (JMLR) 7(Nov):2399–2434

    MathSciNet  MATH  Google Scholar 

  2. Berger-Wolf T, Taheri A, Gimpel K (2018) Learning graph representations with recurrent neural network autoencoders. In: KDD’18.

  3. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184. Epub 2017 Apr 27.

  4. Cho K, van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder–decoder approaches. Syntax, Semantics and Structure in Statistical Translation, p 103

  5. Clough JR, Gollings J, Loach TV, Evans TS (2015) Transitive reduction of citation networks. J Complex Netw 3(2):189–203

    Article  MathSciNet  Google Scholar 

  6. Cui Z, Henrickson K, Ke R (2018) Traffic graph convolutional recurrent neural network: a deep learning framework for network-scale traffic learning and forecasting. arXiv preprintarXiv:1802.07007

  7. Dai H, Dai B, Song L (2016) Discriminative embeddings of latent variable models for structured data. Proceedings of the 33 rd International Conference on Machine Learning, New York, NY, USA, 2016

  8. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain

  9. Deng J, Dong W, Socher R, Li L-J, Li K, Fei Fei L (2009) ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  10. Fang Y, Ronald R (2001) Lattices in citation networks: An investigation into the structure of citation graphs. Scientometrics 50(2):273–287

  11. Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. arXiv:1808.03965v1 [cs.LG]

  12. Gehring J, Auli M, Grangier D, Dauphin YN (2017) A convolutional encoder model for neural machine translation. Annual Meeting of the Association for Computational Linguistics

  13. Gong C, Tao D, Liu W, Liu L, Yang J (2017) Label propagation via teaching-to-learn and learning-to-teach. IEEE Trans Neural Netw Learn Syst 28 (2017):1452–1465

  14. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Neural Networks, 2005. IJCNN’05. Proceedings. 2005 IEEE International Joint Conference on, volume 2, pp 729–734

  15. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York

  16. Hammond DK, Vandergheynst P, Gribonval R (2011) Wavelets on graphs via spectral graph theory. Appl Comput Harmon Anal 30(2):129–150

  17. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in Neural Information processing Systems

  18. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. IEEE International Conference on Computer Vision

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778

  20. Joan B, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. CoRR abs/1312.6203

  21. Karasuyama M, Mamitsuka H (2013) Manifold-based similarity adaptation for label propagation. In: Advances in Neural Information Processing Systems

  22. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, 1097–1105

  23. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  24. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

  25. Leo E, Ronald R (1990) Introduction to Informetrics: quantitative methods in library, documentation and information science. Elsevier Science Publishers, Amsterdam, p 228. ISBN 0-444-88493-9

  26. Leow YY, Laurent T, Bresson X (2019) GraphTSNE: a visualization technique for graph-structured data. arXiv preprint arXiv:1904.06915

  27. Li Q, Han Z, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: The 32nd AAAI Conference on Artificial Intelligence

  28. Liao R, Zhao Z, Urtasun R, Zemel RS, Lanczosnet (2019) Multi-scale deep graph convolutional networks. In: Proceedings of the 7th International Conference on Learning Representations

  29. Liu Y, Lee J, Park M, Kim S, Yang E, Hwang SJ, Yang Y (2019a) Learning to propagate labels: Transductive propagation network for few-shot learning. In: Proceedings of the 7th International Conference on Learning Representations

  30. Luong M-T, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. Conference on Empirical Methods in Natural Language Processing

  31. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems (NIPS), pp 3111–3119

  32. Monti F, Bronstein M, Bresson X (2017) Geometric matrix completion with recurrent multi-graph neural networks. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, pp 3697–3707

  33. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 701–710. ACM, New York

  34. Qu M, Bengio Y, Tang J, Gmnn (2019) Graph markov neural networks. In: Proceedings of the 36th International Conference on Machine Learning

  35. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, 91–99

  36. Seo Y, Defferrard M, Vandergheynst P (2018) Structured sequence modeling with graph convolutional recurrent networks. International Conference on Neural Information Processing, 362-373

  37. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Networks 20(1):61–80

    Article  Google Scholar 

  38. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93

    Google Scholar 

  39. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representations

  40. Shi X, Lv F, Seng D, Zhang J, Chen J, Xing B (2021) Visualizing and understanding graph convolutional network. Multimed Tools Appl 80:8355–8375. https://doi.org/10.1007/s11042-020-09885-4

    Article  Google Scholar 

  41. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

  42. Sun Y, Liang J, Niu P (2021) Personalized recommendation of english learning based on knowledge graph and graph convolutional network. In: Sun X, Zhang X, Xia Z, Bertino E (eds) Artificial Intelligence and Security. ICAIS 2021, vol 12737. Springer, Cham. https://doi.org/10.1007/978-3-030-78612-0_13

  43. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9

  44. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) Line: Large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web. ACM, New York, pp 1067–1077

  45. Thomas K, Welling M (2017) Semi-Supervised Classification with Graph Convolutional Networks. In: Proceedings of the 5th International Conference on Learning representation

  46. Veličković P, Cucurull G, Casanova A et al (2018) Graph attention networks. In: Proceedings of the International Conference on Learning Representations

  47. Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, New York, pp 1225–1234

  48. Wang F, Zhang C (2008) Label propagation through linear neighborhoods. IEEE Trans Knowl Data Eng 20(1):55–67. https://doi.org/10.1109/TKDE.2007.190672

  49. Wang H, Leskovec J (2020) Unifying graph convolutional neural networks and label propagation. arXiv:2002.06755v1 [cs.LG]

  50. Wangzhong L, Janssen J, Milios E, Japkowic N, Yongzheng Z (2007) Node similarity in the citation graph. Knowl Inf Syst 11(1):105–129

    Google Scholar 

  51. Weston J, Ratle F, Mobahi H, Collobert R (2012) Deep learning via semi supervised embedding. Neural Networks: Tricks of the Trade. Springer, Berlin, pp 639–655

  52. Xiao G, Wang R, Zhang C et al (2021) Demand prediction for a public bike sharing program based on spatio-temporal graph convolutional networks. Multimed Tools Appl 80:22907–22925. https://doi.org/10.1007/s11042-020-08803-y

    Article  Google Scholar 

  53. Xiao L, Hu X, Chen Y et al (2020) Multi-head self-attention based gated graph convolutional networks for aspect-based sentiment classification. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-10107-0

    Article  Google Scholar 

  54. Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations

  55. Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi K-i, Jegelka S (2018) Representation learning on graphs with jumping knowledge networks. In: Proceedings of the 35th International Conference on Machine Learning

  56. Yi L, Su H, Guo X et al (2017) Syncspeccnn: synchronized spectral cnn for 3d shape segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2282-2290

  57. Ying R, He R, Chen K (2018) et.al., Graph convolutional networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 974-983

  58. Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 3634-3640

  59. Yuan A, Jeannette J, Evangelos EM (2004) Characterizing and mining the citation graph of the computer science literature. Knowl Inf Syst 6(6):664–678

    Article  Google Scholar 

  60. Zhang B, Liu M, Zhou B, Liu X (2021) Graph learning in low dimensional space for graph convolutional networks. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-11033-5

    Article  Google Scholar 

  61. Zhang Z, Wang J, Mlle (2007) Modified locally linear embedding using multiple weights. In: Adv Neural Inf Process Syst 19:1593–1600

  62. Zhao, Dangzhi Z, Andreas S (2015) Analysis and visualization of citation networks. Morgan & Claypool Publishers, San Rafael. ISBN 978-1-60845-939-1

  63. Zhou K, Song Q, Huang X, Hu X ( 2019) Auto-GNN: Neural architecture search of graph neural networks. arXiv:1909.03184v2 [cs.LG]

  64. Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: International Conference on Machine Learning (ICML), vol 3, pp 912–919

  65. Zhu X, Mao Z, Chen Z et al (2021) Object-difference drived graph convolutional networks for visual question answering. Multimed Tools Appl 80:16247–16265. https://doi.org/10.1007/s11042-020-08790-0

    Article  Google Scholar 

  66. Zhu X, Lafferty J, Rosenfeld R (2005) Semi-supervised learning with graphs. PhD thesis, Carnegie Mellon University, school of language technologies institute

  67. Zitnik M, Leskovec J (2017) Predicting multicellular function through multi-layer tissue networks. Bioinformatics 33(14):i190–i198

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the valuable suggestions provided by the anonymous reviewers which greatly helped in preparing the paper in its present form.

Funding

The research has been supported by Birla Institute of Technology, Mesra, Ranchi.

Author information

Authors and Affiliations

  1. Birla Institute of Technology, Mesra, Ranchi, India

    Vandana Bhattacharjee, Raj Sahu & Amit Dutta

Authors
  1. Vandana Bhattacharjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raj Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amit Dutta
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first author is responsible for conceptualization, problem definition, solution design and report writing. The second author is responsible for implementation and support in report writing. The third author is responsible for implementation and support in report writing.

Corresponding author

Correspondence to Vandana Bhattacharjee.

Ethics declarations

Conflicts of interest/Competing interests

None.

Ethics approval

Not applicable.

Consent to participate

All the authors are aware of this submission. They have reviewed and consented to participate in this journal submission.

Consent for publication

All the authors are aware of this submission. They have reviewed and consented to participate in this journal submission.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Figs. 5, 6, and 7 present the accuracy and loss curves for all the datasets.   

Fig. 5
figure 5

Accuracy and Loss curves of GCN Model for Cora Dataset (Sim1)

Full size image
Fig. 6
figure 6

Accuracy and Loss curves of GCN Model for Citeseer Dataset (Sim1)

Full size image
Fig. 7
figure 7

Accuracy and Loss curves of GCN Model for Pubmed Dataset (Sim1)

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhattacharjee, V., Sahu, R. & Dutta, A. Enhanced Graph Representations for Graph Convolutional Network Models. Multimed Tools Appl 82, 9649–9666 (2023). https://doi.org/10.1007/s11042-021-11843-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11843-7

Keywords

Search

Navigation