Skip to main content
SpringerLink
Log in

Few-Shot Learning with Complex-Valued Neural Networks and Dependable Learning

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We present a flexible, general framework for few-shot learning where both inter-class differences and intra-class relationships are fully considered to improve recognition performance significantly. We introduce complex-valued convolutional neural networks (CNNs) to describe the subtle difference among inter-class samples and Dependable Learning to capture the intra-class relationship. Conventional CNNs use only real-valued CNNs and fail to extract more detailed information. Complex-valued CNNs, on the other hand, can provide amplitude and phase information to enhance the feature representation ability based on the proposed complex metric module (CMM). Building upon the recent episodic training mechanism, CMMs can improve the representation capacity by extracting robust complex-valued features to facilitate the modeling of subtle relationships among few-shot samples. Furthermore, we use Dependable Learning as a new learning paradigm, to promote a robust model against perturbation based on a new bilinear optimization to enhance the feature extraction capacity for very few available intra-class samples. Experiments on two benchmark datasets show that the proposed methods significantly improve the performance over other approaches and achieve state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Amar, D., Sinnott-Armstrong, N., Ashley, E. A., & Rivas, M. A. (2021). Graphical analysis for phenome-wide causal discovery in genotyped population-scale biobanks. Nature Communications, 12(1), 1–11.

    Article  Google Scholar 

  • Arjovsky, M., Shah, A., & Bengio, Y. (2016) Unitary evolution recurrent neural networks. In ICML (pp. 1120–1128).

  • Athalye, A., Carlini, N., & Wagner, D. (2018). Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML (pp. 274–283).

  • Bertinetto, L., Henriques, J. F., Torr, P. H. S., & Vedaldi, A. (2019). Meta-learning with differentiable closed-form solvers. In ICLR (pp. 1–11).

  • Cao, C., Liu, X., Yang, Y., Yu, Y., Wang, J., Wang, Z., Huang, Y., Wang, L., Huang, C., Xu, W., Ranmanan, D., Huang, T.(2015). Look and think twice: Capturing top-down visual attention with feedback convolutional neural networks. In ICCV (pp. 2956–2964).

  • Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (pp. 39–57).

  • Chen, M., Fang, Y., Wang, X., Luo, H., Geng, Y., Zhang, X., Huang, C., Liu, W., & Wang, B. (2020). Diversity transfer network for few-shot learning. In AAAI (pp. 10559–10566).

  • Cubuk, E. D., Zoph, B., Schoenholz, S. S., & Le, Q. V. (2017). Intriguing properties of adversarial examples. In ICLR (pp. 1–17).

  • Danihelka, I., Wayne, G., Uria, B., Kalchbrenner, N., & Graves, A. (2016). Associative long short-term memory. In ICML (pp. 1986–1994).

  • Das, N., Shanbhogue, M., Chen, S., Hohman, F., Chen, L., Kounavis, M. E., & Chau, D. H. (2017). Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression. arXiv (pp. 1–15).

  • Dhillon, G. S., Chaudhari, P., Ravichandran, A., & Soatto, S. (2020). A baseline for few-shot image classification. In ICLR (pp. 1–20).

  • Dong, X., Han, J., Chen, D., Liu, J., Bian, H., Ma, Z., Li, H., Wang, X., Zhang, W., & Yu, N. (2020). Robust superpixel-guided attentional adversarial attack. In CVPR (pp. 12895–12904).

  • Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018). Boosting adversarial attacks with momentum. In CVPR (pp. 9185–9193).

  • Fehervari, I., Ravichandran, A., & Appalaraju, S. (2019). Unbiased evaluation of deep metric learning algorithms. arXiv (pp. 1–9).

  • Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In ICML (pp. 1126–1135).

  • Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., & Cord, M. (2019). Boosting few-shot visual learning with self-supervision. In ICCV (pp. 8059–8068).

  • Gidaris, S., & Komodakis, N. (2018). Dynamic few-shot visual learning without forgetting. In CVPR (pp. 4367–4375).

  • Gidaris, S., & Komodakis, N. (2019). Generating classification weights with gnn denoising autoencoders for few-shot learning. In CVPR (pp. 21–30).

  • Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In ICLR (pp. 1–11).

  • Guo, Y., & Cheung, N.-M. (2020). Attentive weights generation for few shot learning via information maximization. In CVPR (pp. 13499–13508).

  • Gupta, P., & Rahtu, E. (2019). Ciidefence: Defeating adversarial attacks by fusing class-specific image inpainting and image denoising. In ICCV (pp. 6708–6717).

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In CVPR (pp. 770–778).

  • Heide, F., Heidrich, W., & Wetzstein, G. (2015). Fast and flexible convolutional sparse coding. In CVPR (pp. 5135–5143).

  • Hirose, A., & Yoshida, S. (2012). Generalization characteristics of complex-valued feedforward neural networks in relation to signal coherence. IEEE Transactions on Neural Networks, 23(4), 541–551.

    Article  Google Scholar 

  • Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2(3), 194–203.

    Article  Google Scholar 

  • Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on pattern analysis and machine intelligence, 20(11), 1254–1259.

    Article  Google Scholar 

  • Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K. (2015). Spatial transformer networks. In NeurIPS (pp. 2017–2025).

  • Kim, J., Kim, H., & Kim, G. (2020). Model-agnostic boundary-adversarial sampling for test-time generalization in few-shot learning. In ECCV (pp. 599–617).

  • Kingma, D. P., & Adam, J. B. (2014). A method for stochastic optimization. In ICLR.

  • Koch, G., Zemel, R., & Salakhutdinov, R. (2015). Siamese neural networks for one-shot image recognition. In ICML deep learning workshop (pp. 1–8).

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In NeurIPS (pp. 1097–1105).

  • Kurakin, A., Goodfellow, I., & Bengio, S. (2016). Adversarial examples in the physical world. In ICLR (pp. 1–13).

  • Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266), 1332–1338.

    Article  MathSciNet  MATH  Google Scholar 

  • Larochelle, H., & Hinton, G. E. (2010). Learning to combine foveal glimpses with a third-order Boltzmann machine. In NeurIPS (pp. 1243–1251).

  • Lee, K., Maji, S., Ravichandran, A., & Soatto, S. (2019). Meta-learning with differentiable convex optimization. In CVPR (pp. 10657–10665).

  • Li, K., Zhang, Y., Li, K., & Fu, Y. (2020). Adversarial feature hallucination networks for few-shot learning. In CVPR (pp. 13470–13479).

  • Liao, F., Liang, M., Dong, Y., Pang, T., Hu, X., & Zhu, J. (2018). Defense against adversarial attacks using high-level representation guided denoiser. In CVPR (pp. 1778–1787).

  • Li, F. F., & Fergus, R. (2006). One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(4), 594–611.

    Google Scholar 

  • Liu, Y., Lee, J., Park, M., Kim, S., Yang, E., Hwang, S., & Yang, Y. (2019). Learning to propagate labels: Transductive propagation network for few-shot learning. In ICLR (pp. 1–14).

  • Liu, Y., Chen, X., Liu, C., & Song, D. (2016). Delving into transferable adversarial examples and black-box attacks. In ICLR (pp. 1–24).

  • Liu, Z., Zhang, B., & Guo, G. (2020). Few-shot learning with complex-valued neural networks. In BMVC (pp. 541–552).

  • Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. ICLR (pp. 1–28).

  • Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. JMLR, 11(Jan), 19–60.

    MathSciNet  MATH  Google Scholar 

  • Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2018). A simple neural attentive meta-learner. In ICLR.

  • Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K. (2014). Recurrent models of visual attention. In NeurIPS (pp. 2204–2212).

  • Mönning, N., & Manandhar, S. (2018). Evaluation of complex-valued neural networks on real-valued classification tasks. In arXiv (pp. 1–18).

  • Munkhdalai, T., Yuan, X., Mehri, S., & Trischler, A. (2018). Rapid adaptation with conditionally shifted neurons. In International conference on machine learning (pp. 3664–3673).

  • Mustafa, A., Khan, S., Hayat, M., Goecke, R., Shen, J., & Shao, L. (2019). Adversarial defense by restricting the hidden space of deep neural networks. In ICCV (pp. 3385–3394).

  • Na, T., Ko, J. H., & Mukhopadhyay, S. (2017). Cascade adversarial machine learning regularized with a unified embedding. In ICLR (pp. 1–15).

  • Nichol, A., Achiam, J., & Schulman, J. (2018). On first-order meta-learning algorithms. In ICML (pp. 324–330).

  • Nitta, T. (2002). On the critical points of the complex-valued neural network. ICNIP, 3, 1099–1103.

    Google Scholar 

  • Olshausen, B. A., Anderson, C. H., Essen, V., & David, C. (1993). A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. Journal of Neuroscience, 13(11), 4700–4719.

    Article  Google Scholar 

  • Oreshkin, B. N., Rodriguez, P., & Lacoste, A. (2018). Tadam: Task dependent adaptive metric for improved few-shot learning. In NeurIPS (pp. 719–729).

  • Petersen, K. B., Pedersen, M. S. (2008). The matrix cookbook. Technical University of Denmark, 7(15), 510.

  • Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., & Tian, Y. (2019). Transductive episodic-wise adaptive metric for few-shot learning. In ICCV (pp. 3603–3612).

  • Rassadin, A. (2020) Deep residual 3d u-net for joint segmentation and texture classification of nodules in lung. In ICIAR (pp. 419–427).

  • Ravi, S., & Larochelle, H. (2017). Optimization as a model for few-shot learning. In ICLR (pp. 1–11).

  • Ravichandran, A., Bhotika, R., & Soatto, S. (2019). Few-shot learning with embedded class models and shot-free meta training. In ICCV (pp. 331–339).

  • Reichert, D. P., & Serre, T. (2014). Neuronal synchrony in complex-valued deep networks. In ICLR (pp. 1–14).

  • Rizve, M. N., Khan, S., Khan, F. S., & Shah, M. (2021). Exploring complementary strengths of invariant and equivariant representations for few-shot learning. In CVPR (pp. 10836–10846).

  • Rusu, A. A., Rao, D., Sygnowski, J., Vinyals, O., Pascanu, R., Osindero, S., & Hadsell, R. (2019). Meta-learning with latent embedding optimization. In ICLR.

  • Shafahi, A., Najibi, M., Ghiasi, A., Xu, Z., Dickerson, J., Studer, C., Davis, L. S., Taylor, G., & Goldstein, T. (2019). Adversarial training for free! In NeurIPS (pp. 3358–3369).

  • Simon, C., Koniusz, P., Nock, R., & Harandi, M. (2020). Adaptive subspaces for few-shot learning. In CVPR (pp. 4136–4145).

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. In ICLR (pp. 1–14).

  • Snell, J., Swersky, K., & Zemel, R. S. (2017). Prototypical networks for few-shot learning. In NeurIPS (pp. 4080–4090).

  • Sun, Q., Liu, Y., Chua, T.-S., & Schiele, B. (2019). Meta-transfer learning for few-shot learning. In CVPR (pp. 403–412).

  • Sung, F., Yang, Yo., Zhang, L., Xiang, T., Torr, P. H. S., & Hospedales, T. M. (2018). Learning to compare: Relation network for few-shot learning. In CVPR (pp. 1199–1208).

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In CVPR (pp. 1–9).

  • Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. In ICLR.

  • Tian, Y., Wang, Y., Krishnan, D., Tenenbaum, J. B., & Isola, P. (2020). Rethinking few-shot image classification: a good embedding is all you need? In ECCV (pp. 266–282).

  • Trabelsi, C., Bilaniuk, O., Zhang, Y., Serdyuk, D., Subramanian, S., Santos, J. F., Mehri, S., Rostamzadeh, N., Bengio, Y., & Pal, C. (2018). Deep complex networks. In ICLR (pp. 1–19).

  • Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017). Ensemble adversarial training: Attacks and defenses. In ICLR (pp. 1–22).

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In NeurIPS (pp. 5998–6008).

  • Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., & Wierstra, D. (2016). Matching networks for one shot learning. In NeurIPS (pp. 3630–3638).

  • Wang, J., & Zhang, H. (2019). Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In ICCV (pp. 6629–6638).

  • Wang, R., Bao, Y., Zhang, B., Liu, J., Zhu, W., & Guo, G. (2022). Anti-retroactive interference for lifelong learning. In arXiv preprintarXiv:2208.12967

  • Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. In ACM computing surveys (pp. 1–34).

  • Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In ECCV (pp. 3–19).

  • Xie, C., Zhang, Z., Zhou, Y., Bai, S., Wang, J., Ren, Z., & Yuille, A. L. (2019). Improving transferability of adversarial examples with input diversity. In CVPR (pp. 2730–2739).

  • Yang, L., Li, C., Han, J., Chen, C., Ye, Q., Zhang, B., Cao, X., Liu, W. (2017). Image reconstruction via manifold constrained convolutional sparse coding for image sets. JSTSP, 11(7), 1072–1081.

  • Ye, S., Xu, K., Liu, S., Cheng, H., Lambrechts, J., Zhang, H., Zhou, A., Ma, K., Wang, Y., & Lin, X. (2019). Adversarial robustness vs. model compression, or both? In ICCV (pp. 111–120).

  • Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In ECCV (pp. 818–833).

  • Zhang, B., Shan, S., Chen, X., & Gao, W. (2006). Histogram of Gabor phase patterns (HGPP): A novel object representation approach for face recognition. IEEE Transactions on Image Processing, 16(1), 57–68.

  • Zhang, R., Che, T., Ghahramani, Z., Bengio, Y., & Song, Y. (2018). Metagan: An adversarial approach to few-shot learning. In NeurIPS (pp. 1–8).

  • Zhang, Z., Wang, H., Xu, F., & Jin, Y. (2017). Complex-valued convolutional neural network and its application in polarimetric SAR image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(12), 7177–7188.

    Article  Google Scholar 

  • Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020). Random erasing data augmentation. In AAAI (pp. 1–8).

  • Zhou, B., Cui, Q., Wei, X.-S., & Chen, Z.-M. (2020). BBN: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In CVPR (pp. 9719–9728).

  • Zhuo, L., Zhang, B., Yang, L., Chen, H., Ye, Q., Doermann, D., Ji, R., & Guo, G. (2020). Cogradient descent for bilinear optimization. In CVPR (pp. 7959–7967).

Download references

Acknowledgements

This work was supported by “the Fundamental Research Funds for the Central Universities”, and the National Natural Science Foundation of China under Grant 62076016, Beijing Natural Science Foundation-Xiaomi Innovation Joint Fund L223024

Author information

Authors and Affiliations

  1. Beihang University, Beijing, China

    Runqi Wang, Zhen Liu & Baochang Zhang

  2. Ant Group, Beijing, China

    Guodong Guo

  3. Institute of Deep Learning, Baidu Research, Beijing, China

    Guodong Guo

  4. National Engineering Laboratory for Deep Learning Technology and Application, Beijing, China

    Guodong Guo

  5. University at Buffalo, Buffalo, NY, USA

    David Doermann

Authors
  1. Runqi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baochang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guodong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Doermann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baochang Zhang.

Additional information

Communicated by Oisin Mac Aodha.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, R., Liu, Z., Zhang, B. et al. Few-Shot Learning with Complex-Valued Neural Networks and Dependable Learning. Int J Comput Vis 131, 385–404 (2023). https://doi.org/10.1007/s11263-022-01700-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01700-x

Keywords

Search

Navigation