Jinfeng Zhuang
ABSTRACT
Tags of social images play a central role for text-based social image retrieval and browsing tasks. However, the original tags annotated by web users could be noisy, irrelevant, and often incomplete for describing the image contents, which may severely deteriorate the performance of text-based image retrieval models. In this paper, we aim to overcome the challenge of social tag ranking for a corpus of social images with rich user-generated tags by proposing a novel two-view learning approach. It can effectively exploit both textual and visual contents of social images to discover the complicated relationship between tags and images. Unlike the conventional learning approaches that usually assume some parametric models, our method is completely data-driven and makes no assumption of the underlying models, making the proposed solution practically more effective. We formally formulate our method as an optimization task and present an efficient algorithm to solve it. To evaluate the efficacy of our method, we conducted an extensive set of experiments by applying our technique to both text-based social image retrieval and automatic image annotation tasks, in which encouraging results showed that the proposed method is more effective than the conventional approaches.
- R. A. Baeza-Yates and B. A. Ribeiro-Neto. Modern Information Retrieval. ACM Press / Addison-Wesley, 1999. Google Scholar
Digital Library
- S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004. Google ScholarDigital Library
- R. Cilibrasi and P. M. B. Vitányi. The google similarity distance. IEEE Trans. Knowl. Data Eng., 19(3):370--383, 2007. Google ScholarDigital Library
- R. Datta, W. Ge, J. Li, and J. Z. Wang. Toward bridging the annotation-retrieval gap in image search by a generative modeling approach. In ACM Multimedia, pages 977--986, 2006. Google ScholarDigital Library
- L. Y. X.-S. H. H. J. Z. Dong Liu, Meng Wang. Tag quality improvement for social images. In Multimedia and Expo, pages 350--353, 2009. Google ScholarDigital Library
- J. D. R. Farquhar, D. R. Hardoon, H. Meng, J. Shawe-Taylor, and S. Szedmák. Two view learning: Svm-2k, theory and practice. In NIPS, 2005.Google Scholar
- S. A. Golder and B. A. Huberman. The structure of collaborative tagging systems. CoRR, abs/cs/0508082, 2005.Google Scholar
- J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In Proceedings of ACM Special Interest Group on Information Retrieval, pages 119--126, 2003. Google ScholarDigital Library
- Y. Jin, L. Khan, L. Wang, and M. Awad. Image annotations by combining multiple evidence & wordnet. In ACM Multimedia, pages 706--715, 2005. Google ScholarDigital Library
- M. Lades, J. C. Vorbrüggen, J. M. Buhmann, J. Lange, C. von der Malsburg, R. P. Würtz, and W. Konen. Distortion invariant object recognition in the dynamic link architecture. IEEE Trans. Computers, 42(3):300--311, 1993. Google ScholarDigital Library
- X. Li, C. G. M. Snoek, and M. Worring. Learning tag relevance by neighbor voting for social image retrieval. In Multimedia Information Retrieval, pages 180--187, 2008. Google ScholarDigital Library
- D. Liu, X.-S. Hua, L. Yang, M. Wang, and H.-J. Zhang. Tag ranking. In WWW, pages 351--360, 2009. Google ScholarDigital Library
- X. Liu, R. Ji, H. Yao, P. Xu, X. Sun, and T. Liu. Cross-media manifold learning for image retrieval & annotation. In Multimedia Information Retrieval, pages 141--148, 2008. Google ScholarDigital Library
- D. G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91--110, 2004. Google ScholarDigital Library
- C. Marlow, M. Naaman, D. Boyd, and M. Davis. Ht06, tagging paper, taxonomy, flickr, academic article, to read. In HYPERTEXT '06: Proceedings of the seventeenth conference on Hypertext and hypermedia, pages 31--40, 2006. Google ScholarDigital Library
- T. Ojala, M. Pietikáinen, and D. Harwood. A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1):51--59, 1996.Google ScholarCross Ref
- B. Sigurbjörnsson and R. van Zwol. Flickr tag recommendation based on collective knowledge. In WWW, pages 327--336, 2008. Google ScholarDigital Library
- J. C. Spall. Introduction to Stochastic Search and Optimization. John Wiley & Sons, Inc., New York, NY, USA, 2003. Google ScholarDigital Library
- C. Wallraven, B. Caputo, and A. Graf. Recognition with local features: the kernel recipe. In ICCV '03: Proceedings of the Ninth IEEE International Conference on Computer Vision, page 257, Washington, DC, USA, 2003. IEEE Computer Society. Google ScholarDigital Library
- C. Wang, F. Jing, L. Zhang, and H. Zhang. Image annotation refinement using random walk with restarts. In ACM Multimedia, pages 647--650, 2006. Google ScholarDigital Library
- C. Wang, F. Jing, L. Zhang, and H.-J. Zhang. Content-based image annotation refinement. In CVPR, 2007.Google ScholarCross Ref
- C. Wang, L. Zhang, and H.-J. Zhang. Learning to reduce the semantic gap in web image retrieval and annotation. In Proceedings of ACM Special Interest Group on Information Retrieval, pages 355--362, 2008. Google ScholarDigital Library
- K. Q. Weinberger, M. Slaney, and R. van Zwol. Resolving tag ambiguity. In ACM Multimedia, pages 111--120, 2008. Google ScholarDigital Library
- L. Wu, S. C. Hoi, J. Zhu, R. Jin, and N. Yu. Distance metric learning from uncertain side information with application to automated photo tagging. In Proceedings of ACM International Conference on Multimedia (MM2009), Beijing, China, Oct. 19-24 2009. Google ScholarDigital Library
- L. Wu, L. Yang, N. Yu, and X.-S. Hua. Learning to tag. In WWW, pages 361--370, 2009. Google ScholarDigital Library
- T. Zhang. Sequential greedy approximation for certain convex optimization problems. IEEE Transactions on Information Theory, 49(3):682--691, 2003. Google ScholarDigital Library
- Y. Zhao and G. Karypis. Empirical and theoretical comparisons of selected criterion functions for document clustering. Machine Learning, 55(3):311--331, 2004. Google ScholarDigital Library
- J. Zhu, S. C. H. Hoi, M. R. Lyu, and S. Yan. Near-duplicate keyframe retrieval by nonrigid image matching. In ACM Multimedia, pages 41--50, 2008. Google ScholarDigital Library
Index Terms
- A two-view learning approach for image tag ranking
Recommendations
Tag ranking
WWW '09: Proceedings of the 18th international conference on World wide webSocial media sharing web sites like Flickr allow users to annotate images with free tags, which significantly facilitate Web image search and organization. However, the tags associated with an image generally are in a random order without any importance ...
Content-based tag processing for Internet social images
Online social media services such as Flickr and Zooomr allow users to share their images with the others for social interaction. An important feature of these services is that the users manually annotate their images with the freely-chosen tags, which ...
Social image tag enrichment based on textual similarity modeling
In social image sharing websites, users provide several descriptive tags to annotate their shared images. Usually, the user annotated tags are noisy, biased and incomplete. How to improve tag quality is very important for tag based applications. The ...
Comments