Skip to main content
SpringerLink
Log in

Generalized few-shot node classification: toward an uncertainty-based solution

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

For real-world graph data, the node class distribution is inherently imbalanced and long-tailed, which naturally leads to a few-shot learning scenario with limited nodes labeled for newly emerging classes. There are many carefully designed solutions for such a few-shot learning problem via methods such as data augmentation, learning transferable initialization, learning prototypes, and many more. However, most, if not all, of them are based on a strong assumption that all the test nodes exclusively come from novel classes, which is impractical in real-world applications. In this paper, we study a broader and more realistic problem named generalized few-shot node classification, where the test samples can be from both novel classes and base classes. Compared with the standard few-shot node classification, this new problem imposes several unique challenges, including asymmetric classification and inconsistent preference. To counter those challenges, we propose a shot-aware neural node classifier (Stager) equipped with an uncertainty-based weight assigner module for adaptive propagation. As the existing meta-learning solutions cannot handle this new problem, we propose a novel training paradigm named imbalanced episodic training to ensure the label distribution is consistent between the meta-training and meta-test scenarios. Comprehensive experiments on four real-world datasets demonstrate the effectiveness of our proposed model and training paradigm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Epistemic uncertainty is defined to measure how well the model fits the data and is reducible as the size of training data increases [2, 29].

  2. https://nijianmo.github.io/amazon/index.html.

  3. https://www.aminer.cn/data/?nav=openData.

  4. https://github.com/abojchevski/graph2gauss/tree/master/data.

  5. https://github.com/pricexu/STAGER.

  6. https://github.com/pricexu/STAGER.

References

  1. Aggarwal CC, Nan L (2011) On node classification in dynamic content-based networks. In: SDM

  2. Alaa A, Van Der Schaar M (2020) Discriminative jackknife: quantifying uncertainty in deep learning via higher-order influence functions. In: ICML

  3. Smriti B, Graham C, Muthukrishnan S (2011) Node classification in social networks. Social network data analytics. Springer, New York, pp 115–148

    Google Scholar 

  4. Bojchevski A, Günnemann S (2018) Deep gaussian embedding of graphs: unsupervised inductive learning via ranking. In: ICLR

  5. Bose AJ, Jain A, Molino P, and Hamilton WL (2019) Meta-graph: few shot link prediction via meta learning. arXiv preprint arXiv:1912.09867

  6. Buda M, Maki A, Mazurowski MA (2018) A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw 106:249–259

    Article  Google Scholar 

  7. Chao W-L, Changpinyo S, Gong B, and Sha F (2016) An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In: ECCV

  8. Chauhan J, Nathani D, and Kaul M (2020) Few-shot learning on graphs via super-classes based on graph spectral measures. In: ICLR

  9. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  10. Chen X, Dai H, Li Y, Gao X, and Song L (2020) Learning to stop while learning to predict. In: International conference on machine learning. PMLR, pp 1520–1530

  11. Chien E, Peng J, Li P, and Milenkovic O (2021) Adaptive universal generalized pagerank graph neural network. In: ICLR

  12. Ding K, Wang J, Caverlee J, and Liu H (2022) Meta propagation networks for graph few-shot semi-supervised learning. In: AAAI

  13. Ding K, Wang J, Li J, Shu K, Liu C, and Liu H (2020) Graph prototypical networks for few-shot learning on attributed networks. In: CIKM

  14. Ding K, Xu Z, Tong H, and Liu H (2022) Data augmentation for deep graph learning: a survey. arXiv preprint arXiv:2202.08235

  15. Ding K, Zhou Q, Tong H, and Liu H (2021) Few-shot network anomaly detection via cross-network meta-learning. In: TheWebConf

  16. Elkan C (2001) The foundations of cost-sensitive learning. In: IJCAI

  17. Fan Z, Ma Y, Li Z, and Sun J (2021) Generalized few-shot object detection without forgetting. In: CVPR

  18. Felix R, Reid I, Carneiro G, et al (2018) Multi-modal cycle-consistent generalized zero-shot learning. In: ECCV

  19. Finn C, Abbeel P, and Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML

  20. Franceschi L, Donini M, Frasconi P, Pontil M (2017) Forward and reverse gradient-based hyperparameter optimization. In: ICML. PMLR

  21. Franceschi L, Frasconi P, Salzo S, Grazzi R, Pontil M (2018) Bilevel programming for hyperparameter optimization and meta-learning. In: ICML. PMLR

  22. Gal Y, and Ghahramani Z (2015) Bayesian convolutional neural networks with bernoulli approximate variational inference. arXiv preprint arXiv:1506.02158

  23. Gidaris S, and Komodakis N (2018) Dynamic few-shot visual learning without forgetting. In: CVPR

  24. Guan J, Liu J, Sun J, Feng P, Shuai T, and Wang W (2020) Meta metric learning for highly imbalanced aerial scene classification. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 4047–4051

  25. He H, and Garcia EA (2009) Learning from imbalanced data. TKDE

  26. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

  27. Kexin H, Marinka Z (2020) Graph meta learning via local subgraphs. NeurIPS 33:5862–5874

    Google Scholar 

  28. Japkowicz N, Stephen S (2002) The class imbalance problem: a systematic study. Intell Data Anal 6(5):429–449

    Article  Google Scholar 

  29. Kendall A, and Gal Y (2017) What uncertainties do we need in bayesian deep learning for computer vision? In: NIPS

  30. Kingma DP, and Ba J (2015) Adam: a method for stochastic optimization. In: ICLR

  31. Thomas N (2017) Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In: ICLR

  32. Klicpera J, Bojchevski A, and Stephan G. (2018) Graph neural networks meet personalized pagerank. In: ICLR, Predict then propagate

  33. Lan L, Wang P, Du X, Song K, Tao J, Guan X (2020) Node classification on graphs with few-shot novel labels via meta transformed network embedding. Adv Neural Inf Process Syst 33:16520–16531

    Google Scholar 

  34. Lee H, Lee H, Na D, Kim S, Park M, Yang E, and Hwang SJ (2020) Learning to balance: bayesian meta-learning for imbalanced and out-of-distribution tasks. In: 8th international conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net

  35. Liu Z, Fang Y, Liu C, and Hoi SCH (2021) Relative and absolute location embedding for few-shot node classification on graph. In: AAAI

  36. Liu Z, Miao Z, Zhan X, Wang J, Gong B, and Yu SX (2019) Large-scale long-tailed recognition in an open world. In: CVPR

  37. McAuley J, Pandey R, and Leskovec J (2015) Inferring networks of substitutable and complementary products. In: SIGKDD

  38. Nichol A, Achiam J, and Schulman J (2018) On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999

  39. Perez-Rua J-M, Zhu X, Timothy MH, and Tao X (2020) Incremental few-shot object detection. In: CVPR

  40. Qu L, Zhu H, Zheng R, Shi Y, and Yin H (2021) Imgagn: imbalanced network embedding via generative adversarial graph networks. In: Feida Z, Beng CO, and Chunyan M (eds) SIGKDD

  41. Snell J, Swersky K, and Zemel R (2017) Prototypical networks for few-shot learning. In: NIPS

  42. Ni Srivastava, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  Google Scholar 

  43. Tan J, Wang C, Li B, Li Q, Ouyang W, Yin C, and Yan J (2020) Equalization loss for long-tailed object recognition. In: CVPR

  44. Tang J, Zhang J, Yao L, Li J, Zhang L, and Su Z (2008) Arnetminer: extraction and mining of academic social networks. In: SIGKDD

  45. Tang J, Aggarwal C, and Liu H (2016) Node classification in signed social networks. In: SDM

  46. Thrun S (1996) Is learning the n-th thing any easier than learning the first? In: NIPS

  47. Thrun S (1998) Lifelong learning algorithms. Learning to learn. Springer, New York, pp 181–209

    Chapter  Google Scholar 

  48. Ting K M (2002) An instance-weighting method to induce cost-sensitive trees. TKDE

  49. Triantafillou E, Zhu T, Dumoulin V, Lamblin P, Evci U, Xu K, Goroshin R, Gelada C, Swersky K, Manzagol P-A, and Larochelle H (2020) Meta-dataset: a dataset of datasets for learning to learn from few examples. In: 8th international conference on learning representations. ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net

  50. Petar V, Guillem C, Arantxa C, Adriana R, Pietro L, and Yoshua B (2018) Graph attention networks. In: ICLR

  51. Vinyals O, Blundell C, Lillicrap T, Wierstra D, et al (2016) Matching networks for one shot learning. In: NIPS

  52. Wang N, Luo M, Ding K, Zhang L, Li J, and Zheng Q (2020) Graph few-shot learning with attribute matching. In: CIKM

  53. Wang Y-X, Ross G, Martial H, and Bharath H (2018) Low-shot learning from imaginary data. In: CVPR

  54. Gary MW, Foster P (2003) Learning when training data are costly: the effect of class distribution on tree induction. J Artif Intell Res 19:315–354

    Article  Google Scholar 

  55. Felix W, Souza A, Tianyi Z, Tao Y, and Kilian W (2019) Simplifying graph convolutional networks. In: ICML. Christopher Fifty

  56. Zhe X, Boxin D, Tong H (2022) Graph sanitation with application to node classification. In: Proceedings of the ACM web conference. pp 1136–1147

  57. Yao H, Zhang C, Wei Y, Jiang M, Wang S, Huang J, Chawla N, and Li Z (2020) Graph few-shot learning via knowledge transfer. In: AAAI

  58. Ye H-J, Hu H, and Zhan D-C (2021) Learning adaptive classifiers synthesis for generalized few-shot learning. Int J Comput Vis 1–24

  59. Yu Y, Ji Z, Han J, and Zhang Z (2020) Episode-based prototype generating network for zero-shot learning. In: CVPR

  60. Zhao T, Zhang X, and Suhang W (2021) Imbalanced node classification on graphs with graph neural networks. In: WSDM. Graphsmote

  61. Zhou F, Cao C, Zhang K, Trajcevski G, Zhong T and Ji G (2019) On few-shot node classification in graph meta-learning. In: CIKM. Meta-gnn

Download references

Author information

Authors and Affiliations

  1. University of Illinois Urbana-Champaign, Champaign, USA

    Zhe Xu, Yu-Xiong Wang & Hanghang Tong

  2. Arizona State University, Tempe, USA

    Kaize Ding & Huan Liu

Authors
  1. Zhe Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaize Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu-Xiong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hanghang Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors discussed the methodology and algorithm. Zhe Xu and Dr. Hanghang Tong wrote the main manuscript. Zhe did the experiments.

Corresponding author

Correspondence to Zhe Xu.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Z., Ding, K., Wang, YX. et al. Generalized few-shot node classification: toward an uncertainty-based solution. Knowl Inf Syst 66, 1205–1229 (2024). https://doi.org/10.1007/s10115-023-01975-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01975-7

Keywords

Search

Navigation