Skip to main content
SpringerLink
Log in

Hybrid neural network model for large-scale heterogeneous classification tasks in few-shot learning

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

How to generalize and unify different few-shot learning tasks using neural network model is a difficult problem in the field of machine learning research. Aiming at the problem that the parameters of existing few-shot learning models cannot adapt with heterogeneous classification tasks, inspired by the human being recognition process, a hybrid neural network (HNN) model for large-scale heterogeneous classification tasks in few-shot learning is proposed. First, a meta-learning model is constructed, which uses a siamese graph convolutional network (SGCN) structure as bone network. The SGCN is trained by semi-supervised way with a small amount of incomplete labeled data. Then, random task slicing by group is performed according to the task size and meta-learning dimensions to ensure that the segmented task size matches the meta-learning model. Combined with the meta-learning model, a task discrimination network and object recognition network are constructed, to perform heterogeneous classification tasks while keeping the scale of HNN network parameters unchanged. Experimental results show that the HNN performs well under different datasets, and is suitable for large-scale heterogeneous tasks in few-shot learning without retraining.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  2. Mohammadi, M., Al-Fuqaha, A., Sorour, S., et al.: Deep learning for IoT big data and streaming analytics: a survey. IEEE Commun. Surv. Tutor. 20(4), 2923–2960 (2018)

    Article  Google Scholar 

  3. Jallad, K.A., Aljnidi, M., Desouki, S., et al.: Big data analysis and distributed deep learning for next-generation intrusion detection system optimization. J. Big Data 6(1), 1–18 (2019)

    Article  Google Scholar 

  4. Konstantinos, M., Konstantinos, K., Anastasios, D., et al.: Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)

  5. Carneiro, G., Chan, A.B., Moreno, P.J., et al.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 394–410 (2007)

    Article  Google Scholar 

  6. Li, X., Shen, H., Zhang, L., et al.: Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning. IEEE Trans. Geosci. Remote Sens. 52(11), 7086–7098 (2014)

    Article  Google Scholar 

  7. Lake, B., Salakhutdinov, R., Gross, J, et al.: One shot learning of simple visual concepts. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 33(33) (2011)

  8. Ma, Y., Kan, M., Shan, S., et al.: Learning deep face representation with long-tail data: an aggregate-and-disperse approach. Pattern Recogn. Lett. 133, 48–54 (2020)

    Article  Google Scholar 

  9. Wang, D., Cheng, Y., Yu, M., et al.: A hybrid approach with optimization-based and metric-based meta-learner for few-shot learning. Neurocomputing 349, 202–211 (2019)

    Article  Google Scholar 

  10. Tian, S., Shen, S., Tian, G., et al.: End-to-end deep metric network for visual tracking. Vis. Comput. 36(6), 1219–1232 (2020)

    Article  Google Scholar 

  11. Blaes, S., Burwick, T.: Few-shot learning in deep networks through global prototyping. Neural Netw. 94, 159–172 (2017)

    Article  Google Scholar 

  12. Wu, H., Miao, Z., Wang, Y., et al.: Optimized recognition with few instances based on semantic distance. Vis. Comput. 31(4), 367–375 (2015)

    Article  Google Scholar 

  13. Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, vol. 2 (2015)

  14. Oriol, V., Charles, B., Tim, L., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)

  15. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, pp. 4077–4087 (2017)

  16. Sung, F., Yang, Y., Zhang, L., et al.: Learning to compare: relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208 (2018)

  17. Santoro, A., Bartunov, S., Botvinick, M., et al.: Meta-learning with memory-augmented neural networks. In: International Conference on Machine Learning, pp. 1842-1850 (2016)

  18. Sun, Q., Liu, Y., Chua, T.S., et al.: Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 403–412 (2019)

  19. Tang, H., Li, Z., Peng, Z., et al.: BlockMix: meta regularization and self-calibrated inference for metric-based meta-learning. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 610–618 (2020)

  20. Peng, Z., Li, Z., Zhang, J., et al.: Few-shot image recognition with knowledge transfer. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 441-449 (2019)

  21. Jiang, L.B., Zhou, X.L., Jiang, W., et al.: One shot learning based on improved matching network. Syst. Eng. Electron. 41(6), 1210–1217 (2019)

    Google Scholar 

  22. Yu, Y., Feng, L., Wang, G.G., et al.: A Few-Shot learning model based on semisupervised with pseudo label. Acta Electron. Sin. 47(11), 2284–2291 (2019)

    Google Scholar 

  23. Zhang, L., Chang, X., Liu, J., et al.: Few-shot activity recognition with cross-modal memory network. Pattern Recogn. 108, 107348 (2020)

    Article  Google Scholar 

  24. Ji, Z., Chai, X., Yu, Y., et al.: Improved prototypical networks for Few-Shot learning. Pattern Recogn. Lett. 140, 81–87 (2020)

    Article  Google Scholar 

  25. Garcia, V., Bruna, J.: Few-shot learning with graph neural networks. In: 6th International Conference on Learning Representations, ICLR, Vancouver, BC, Canada (2018)

  26. Shen, J., Tang, X., Dong, X., et al.: Visual object tracking by hierarchical attention siamese network. IEEE Trans. Cybern. 50(7), 3068–3080 (2019)

    Article  Google Scholar 

  27. Liang, Z., Shen, J.: Local semantic siamese networks for fast tracking. IEEE Trans. Image Process. 29, 3351–3364 (2019)

    Article  Google Scholar 

  28. Dong, X., Shen, J., Wu, D., et al.: Quadruplet network with one-shot learning for fast visual object tracking. IEEE Trans. Image Process. 28(7), 3516–3527 (2019)

    Article  MathSciNet  Google Scholar 

  29. Kim, J., Kim, T., Kim, S., et al.: Edge-labeling graph neural network for fewshot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11–20 (2019)

  30. Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 459–474 (2018)

  31. Qian, K., Song, A.G.: An improved bionic cognitive neural network for robot. Chin. J. Electron. 43(6), 1084–1089 (2015)

    Google Scholar 

  32. Scarselli, F., Gori, M., Tsoi, A.C., et al.: Computational capabilities of graph neural networks. IEEE Trans. Neural Netw. 20(1), 81–102 (2008)

    Article  Google Scholar 

  33. Wang, W., Lu, X., Shen, J., et al.: Zero-shot video object segmentation via attentive graph neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9236–9245 (2019)

  34. Qi, S., Wang, W., Jia, B., et al. Learning human-object interactions by graph parsing neural networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 401–417 (2018)

  35. Phan, A.V., Le Nguyen, M., Nguyen, Y.L.H., et al.: DGCNN: a convolutional neural network over large-scale labeled graphs. Neural Netw. 108, 533–543 (2018)

    Article  Google Scholar 

  36. Fan, H., Zhong, Y., Zeng, G., et al.: Attributed network representation learning via improved graph attention with robust negative sampling. Appl. Intell. 4, 1–11 (2020)

    Google Scholar 

  37. Yin, J., Shen, J., Guan, C., et al.: LiDAR-based online 3D video object detection with graph-based message passing and spatiotemporal transformer attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11495–11504 (2020)

  38. Rahman, S., Khan, S., Porikli, F.: A unified approach for conventional zeroshot, generalized zero-shot, and few-shot learning. IEEE Trans. Image Process. 27(11), 5652–5667 (2018)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This paper was supported by Nanjing Institute of Technology High-level Scientific Research Foundation for the introduction of talent (No. YKJ201918) and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (No. 20KJB510049), partially supported by the National Key R&D Program of China (No.2017YFB1002802) and National Natural Science Foundation of China (No.51675259).

Author information

Authors and Affiliations

  1. School of Automation, Nanjing Institute of Technology, Nanjing, China

    Kui Qian & Xiulan Wen

  2. School of Instrument Science and Engineering, Southeast University, Nanjing, China

    Aiguo Song

Authors
  1. Kui Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiulan Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aiguo Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kui Qian.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qian, K., Wen, X. & Song, A. Hybrid neural network model for large-scale heterogeneous classification tasks in few-shot learning. Vis Comput 38, 719–728 (2022). https://doi.org/10.1007/s00371-020-02046-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-02046-6

Keywords

Search

Navigation