Skip to main content
SpringerLink
Log in

A multi-view-group non-negative matrix factorization approach for automatic image annotation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In automatic image annotation (AIA) different features describe images from different aspects or views. Part of information embedded in some views is common for all views, while other parts are individual and specific. In this paper, we present the Mvg-NMF approach, a multi-view-group non-negative matrix factorization (NMF) method for an AIA system which considers both common and individual factors. The NMF framework discovers a latent space by decomposing data into a set of non-negative basis vectors and coefficients. The views divided into homogeneous groups and latent spaces are extracted for each group. After mapping the test images into these spaces, a unified distance matrix is computed from the distance between images in all spaces. Then a search-based method is used to propagate tags from the nearest neighbors to test images. The evaluation on three datasets commonly used for image annotation showed that the Mvg-NMF is highly competitive with the recent state-of-the-art works.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The features we used are available in http://lear.inrialpes.fr/people/guillaumin/data.php

References

  1. Ballan L, Uricchio T, Seidenari L, Del Bimbo A (2014) A cross-media model for automatic image annotation. In: Proceedings of International Conference on Multimedia Retrieval. ACM, p 73

  2. BenAbdallah J, Caicedo JC, Gonzalez FA, Nasraoui O (2010) Multimodal image annotation using non-negative matrix factorization. In: Web Intelligence and Intelligent Agent Technology (WI-IAT), 2010 IEEE/WIC/ACM International Conference on. IEEE, pp 128–135

  3. Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560

    Article  Google Scholar 

  4. Caicedo JC, González FA (2012) Multimodal fusion for image retrieval using matrix factorization. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. ACM, pp 1–8

  5. Caicedo JC, BenAbdallah J, González FA, Nasraoui O (2012) Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization. Neurocomputing 76(1):50–60

    Article  Google Scholar 

  6. Chen M, Zheng A, Weinberger K (2013) Fast Image Tagging. In: Proceedings of The 30th International Conference on Machine Learning. pp 1274–1282

  7. Ding C, Li T, Jordan MI (2010) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1):45–55

    Article  Google Scholar 

  8. Driesen J (2012) Discovering words in speech using matrix factorization. Ph. D. dissertation, Ph. D. dissertation, KU Leuven, ESAT

  9. Duygulu P, Barnard K, de Freitas JF, Forsyth DA (2002, May) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In European conference on computer vision. Springer, Berlin, Heidelberg, pp 97–112

  10. Eweiwi A, Cheema MS, Bauckhage C (2013, September) Discriminative joint non-negative matrix factorization for human action classification. In German Conference on Pattern Recognition. Springer, Berlin, Heidelberg, pp 61–70

  11. Grubinger M (2007) Analysis and evaluation of visual information systems performance. Victoria University, Toronto

    Google Scholar 

  12. Guan X, Wang W, Zhang X (2009) Fast intrusion detection based on a non-negative matrix factorization model. J Netw Comput Appl 32(1):31–44

    Article  Google Scholar 

  13. Guan N, Tao D, Luo Z, Yuan B (2011) Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent. IEEE Trans Image Process 20(7):2030–2048

    Article  MathSciNet  MATH  Google Scholar 

  14. Guillaumin M, Mensink T, Verbeek J, Schmid C (2009) Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: Computer Vision, 2009 I.E. 12th International Conference on. IEEE, pp 309–316

  15. Johnson J, Ballan L, Fei-Fei L (2015) Love thy neighbors: Image annotation by exploiting image metadata. In: Proceedings of the IEEE International Conference on Computer Vision. pp 4624–4632

  16. Kalayeh MM, Idrees H, Shah M (2014) NMF-KNN: Image Annotation using Weighted Multi-view Non-negative Matrix Factorization. In: Computer Vision and Pattern Recognition (CVPR), 2014 I.E. Conference on. IEEE, pp 184–191

  17. Ke X, Guo W (2016) Multi-scale salient region and relevant visual keywords based model for automatic image annotation. Multimed Tools Appl 75(20):12477–12498

    Article  Google Scholar 

  18. Kejun H, Sidiropoulos ND, Swami A (2014) Non-negative matrix factorization revisited: uniqueness and algorithm for symmetric decomposition. IEEE Trans Signal Process 62(1):211–224. https://doi.org/10.1109/TSP.2013.2285514

    Article  MathSciNet  Google Scholar 

  19. Kim H, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12):1495–1502

    Article  Google Scholar 

  20. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  MATH  Google Scholar 

  21. Lin C, Pang M (2015) Graph regularized nonnegative matrix factorization with sparse coding. Math Probl Eng 2015. https://doi.org/10.1155/2015/239589

  22. Lin Z, Ding G, Hu M (2015) Image auto-annotation via tag-dependent random search over range-constrained visual neighbours. Multimed Tools Appl 74(11):4091–4116

    Article  Google Scholar 

  23. Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  MATH  Google Scholar 

  24. Liu J, Wang C, Gao J, Han J (2013) Multi-view clustering via joint nonnegative matrix factorization. In: Proc. of SDM. SIAM, pp 252–260

  25. Long X, Lu H, Peng Y, Li W (2014) Graph regularized discriminative non-negative matrix factorization for face recognition. Multimed Tools Appl 72(3):2679–2699

    Article  Google Scholar 

  26. Lowe DG (1999) Object recognition from local scale-invariant features. In: Computer vision, 1999. The proceedings of the seventh IEEE international conference on. IEEE, pp 1150–1157

  27. Makadia A, Pavlovic V, Kumar S (2008). A new baseline for image annotation. Computer Vision–ECCV 2008, pp. 316–329

  28. Manning CD, Raghavan P, Schütze H (2008) Introduction to information retrieval, vol 1. Cambridge university press, Cambridge

    Book  MATH  Google Scholar 

  29. Mirzaei S, Norouzi Y (2015) Blind audio source counting and separation of anechoic mixtures using the multichannel complex NMF framework. Signal Process 115:27–37

    Article  Google Scholar 

  30. Moran S, Lavrenko V (2014) A sparse kernel relevance model for automatic image annotation. Int J Multimed Inf Retr 3(4):209–229

    Article  Google Scholar 

  31. Murthy VN, Can EF, Manmatha R (2014) A hybrid model for automatic image annotation. In: Proceedings of International Conference on Multimedia Retrieval. ACM, p 369

  32. Prajapati SJ, Jadhav KR (2015) Brain tumor detection by various image segmentation techniques with introduction to non negative matrix factorization. Brain 4(3):600–603

    Google Scholar 

  33. Rad R, Jamzad M (2015) Automatic image annotation by a loosely joint non-negative matrix factorisation. IET Comput Vis 9(6):806–813

    Article  Google Scholar 

  34. Rad R, Jamzad M (2017) Image annotation using multi-view non-negative matrix factorization with different number of basis vectors. J Vis Commun Image Represent 46:1–12

    Article  Google Scholar 

  35. Sun S (2013) A survey of multi-view machine learning. Neural Comput & Applic 23(7–8):2031–2038

    Article  Google Scholar 

  36. Verma Y, Jawahar CV (2012, October) Image annotation using metric learning in semantic neighbourhoods. In European Conference on Computer Vision. Springer, Berlin, Heidelberg, pp 836-849

  37. Virtanen T (2007) Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria. IEEE Trans Audio Speech Lang Process 15(3):1066–1074

    Article  Google Scholar 

  38. Von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, pp 319–326

  39. Wang D, Lu H (2013) On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Process 93(6):1608–1623

    Article  Google Scholar 

  40. Wang Y-X, Zhang Y-J (2013) Nonnegative matrix factorization: a comprehensive review. IEEE Trans Knowl Data Eng 25(6):1336–1353

    Article  Google Scholar 

  41. Xiang Y, Zhou X, Chua T-S (2009) Ngo C-W A revisit of generative model for automatic image annotation using markov random fields. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, pp 1153–1160

  42. Xu C, Tao D, Xu C (2013) A survey on multi-view learning. Neural Comput Appl 23(7–8):2031–2038

  43. Xu H, Pan P, Xu C, Lu Y, Chen D (2016) Image auto-annotation via concept interdependency network. Multimed Tools Appl 75(11):6237–6261

    Article  Google Scholar 

  44. Yang Y, Zhang W, Xie Y (2015) Image automatic annotation via multi-view deep representation. J Vis Commun Image Represent 33:368–377

    Article  Google Scholar 

  45. Zar JH (1998) Spearman rank correlation. In: Armitage P, Colton T (eds-inchief) Encyclopedia of biostatistics, vol 5. John Wiley and Sons, Chichester, pp 4191–4196

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Sharif University of Technology, Tehran, Iran

    Roya Rad & Mansour Jamzad

Authors
  1. Roya Rad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mansour Jamzad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roya Rad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rad, R., Jamzad, M. A multi-view-group non-negative matrix factorization approach for automatic image annotation. Multimed Tools Appl 77, 17109–17129 (2018). https://doi.org/10.1007/s11042-017-5279-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5279-4

Keywords

Search

Navigation