skip to main content
research-article

Image annotation by kNN-sparse graph-based label propagation over noisily tagged web images

Published:24 February 2011Publication History
Skip Abstract Section

Abstract

In this article, we exploit the problem of annotating a large-scale image corpus by label propagation over noisily tagged web images. To annotate the images more accurately, we propose a novel kNN-sparse graph-based semi-supervised learning approach for harnessing the labeled and unlabeled data simultaneously. The sparse graph constructed by datum-wise one-vs-kNN sparse reconstructions of all samples can remove most of the semantically unrelated links among the data, and thus it is more robust and discriminative than the conventional graphs. Meanwhile, we apply the approximate k nearest neighbors to accelerate the sparse graph construction without loosing its effectiveness. More importantly, we propose an effective training label refinement strategy within this graph-based learning framework to handle the noise in the training labels, by bringing in a dual regularization for both the quantity and sparsity of the noise. We conduct extensive experiments on a real-world image database consisting of 55,615 Flickr images and noisily tagged training labels. The results demonstrate both the effectiveness and efficiency of the proposed approach and its capability to deal with the noise in the training labels.

References

  1. Belkin, M. and Niyogi, P. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput.. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Chang, C.-C. and Lin, C.-J. 2001. LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/&ctilde;jlin/libsvm.Google ScholarGoogle Scholar
  3. Chapelle, O., Zien, A., and Scholkopf, B. 2006. Semi-Supervised Learning. MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., and Zheng, Y.-T. 2009. NUS-WIDE: A real-world web image database from national university of singapore. In Proceedings of the ACM Conference on Image and Video Retrieval. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Donoho, D. L. 2006. For most large underdetermined systems of linear equations the minimal ℓ1-norm solution is also the sparsest solution. Comm. Pure Appl. Math. 59, 6, 797--829.Google ScholarGoogle ScholarCross RefCross Ref
  6. Duda, R., Stork, D., and Hart, P. 2000. Pattern Classification. J. Wiley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Fergus, R., Fei-Fei, L., Perona, P., and Zisserman, A. 2005. Learning object categories from google's image search. In Proceedings of the IEEE International Conference on Computer Vision. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Goh, K.-S., Chang, E. Y., and Lai, W.-C. 2004. Multimodal concept-dependent active learning for image retrieval. In Proceedings of the 12th Annual ACM International Conference on Multimedia. 564--571. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. He, J., Li, M., Zhang, H.-J., Tong, H., and Zhang, C. 2004. Manifold-ranking based image retrieval. In Proceedings of the 12th Annual ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. ℓ1 MAGIC. http://www.acm.caltech.edu/l1magic/.Google ScholarGoogle Scholar
  11. Li, X., Chen, L., Zhang, L., Lin, F., and Ma, W.-Y. 2006. Image annotation by large-scale content-based image retrieval. In Proceedings of the 14th Annual ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Mount, D. and Arya, S. 1997. Ann: A library for approximate nearest neighbor searching. In Proceedings of the CGC 2nd Annual Fall Workship on Computational Geometry.Google ScholarGoogle Scholar
  13. Rao, R., Olshausen, B., and Lewicki, M. 2002. Probabilistic Models of the Brain: Perception and Neural Function. MIT Press.Google ScholarGoogle ScholarCross RefCross Ref
  14. Roweis, S. T. and Saul, L. K. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323--2326.Google ScholarGoogle ScholarCross RefCross Ref
  15. Saad, Y. 2003. Iterative Methods for Sparse Linear Systems 2nd Ed. Society for Industrial and Applied Mathematics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Saad, Y. and Schultz, M. 1986. GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comp. 7, 856--869. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Sun, Y., Shimada, S., Taniguchi, Y., and Kojima, A. 2008. A novel region-based approach to visual concept modeling using web images. In Proceedings of the 16th ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Tang, J., Hua, X.-S., Qi, G.-J., Wang, M., Mei, T., and Wu, X. 2007. Structure-sensitive manifold ranking for video concept detection. In Proceedings of the 15th ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Tang, J., Hua, X.-S., Song, Y., Qi, G.-J., and Wu, X. 2008. Video annotation based on kernel linear neighborhood propagation. IEEE Trans. Multimedia 10, 4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Tang, J., Yan, S., Hong, R., Qi, G.-J., and Chua, T.-S. 2009. Inferring semantic concepts from community-contributed images and noisy tags. In Proceedings of the 17th ACM International Conference on Multimedia. 223--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Torralba, A., Fergus, R., and Freeman, W. 2008. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Trans. Patt. Anal. Mach. Intell. 30, 11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. TREC. Trec-10 proceedings appendix on common evaluation measures. http://trec.nist.gov/pubs/trec10/ appendices/measures.pdf.Google ScholarGoogle Scholar
  23. Wang, C., Jing, F., Zhang, L., and Zhang, H.-J. 2006a. Image annotation refinement using random walk with restarts. In Proceedings of the 14th ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Wang, F. and Zhang, C. 2008. Label propagation through linear neighborhoods. IEEE Trans. Knowl. Data Engin. 20, 1, 55--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Wang, X.-J., Zhang, L., Jing, F., and Ma, W.-Y. 2006b. Annosearch: Image auto-annotation by search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Wang, X.-J., Zhang, L., Li, X., and Ma, W.-Y. 2008. Annotating images by mining image search results. IEEE Trans. Patt. Anal. Mach. Intell. 30, 11, 1919--1932. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Wright, J., Yang, A., Ganesh, A., Sastry, S., and Ma, Y. 2009. Robust face recognition via sparse representation. IEEE Trans. Patt. Anal. Mach. Intell. 31, 2 (Feb.), 210--227. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Zhou, D., Bousquet, O., Lal, T. N., Weston, J., and Scholkopf, B. 2003. Learning with local and global consistency. In Proceedings of the 17th Annual Conference on Neural Information Processing Systems.Google ScholarGoogle Scholar
  29. Zhu, X. 2005. Semi-Supervised Learning with Graphs. Ph.D. dissertation, Carnegie Mellon University. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Zhu, X., Ghahramani, Z., and Lafferty, J. 2003. Semi-supervised learning using gaussian fields and harmonic function. In Proceedings of the 20th International Conference on Machine Learning.Google ScholarGoogle Scholar

Index Terms

  1. Image annotation by kNN-sparse graph-based label propagation over noisily tagged web images

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Intelligent Systems and Technology
          ACM Transactions on Intelligent Systems and Technology  Volume 2, Issue 2
          February 2011
          175 pages
          ISSN:2157-6904
          EISSN:2157-6912
          DOI:10.1145/1899412
          Issue’s Table of Contents

          Copyright © 2011 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 24 February 2011
          • Accepted: 1 August 2010
          • Revised: 1 June 2010
          • Received: 1 February 2010
          Published in tist Volume 2, Issue 2

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader