Changhu Wang
Lei Zhang
ABSTRACT
With the prevalence of digital cameras, more and more people have considerable digital images on their personal devices. As a result, there are increasing needs to effectively search these personal images. Automatic image annotation may serve the goal, for the annotated keywords could facilitate the search processes. Although many image annotation methods have been proposed in recent years, their effectiveness on arbitrary personal images is constrained by their limited scalability, i.e. limited lexicon of small-scale training set. To be scalable, we propose a search-based image annotation (SBIA) algorithm that is analogous to Web page search. First, content-based image retrieval (CBIR) technology is used to retrieve a set of visually similar images from a large-scale Web image set. Then, a text-based keyword search (TBKS) technique is used to obtain a ranked list of candidate annotations for each retrieved image. Finally, a fusion algorithm is used to combine the ranked lists into the final annotation list. The application of both efficient search technologies and Web-scale image set guarantees the scalability of the proposed algorithm. Experimental results on U. Washington dataset show not only the effectiveness and efficiency of the proposed algorithm but also the advantage of image retrieval using annotation results over that using visual features.
- http://www.cs.washington.edu/research/imagedatabase/groundtruth/Google Scholar
- http://www.photosig.comGoogle Scholar
- Baeza-Yates, R. A. and Ribeiro-Neto, B. 1999 Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc. Google ScholarDigital Library
- Blei, D. M. and Jordan, M. I. 2003. Modeling annotated data. In Proceedings of the 26th Annual international ACM SIGIR Conference on Research and Development in informaion Retrieval. New York, NY, 127--134. Google ScholarDigital Library
- Brown, P. F., deSouza, P. V., Mercer, R. L., Pietra, V. J., and Lai, J. C. 1992. Class-based n-gram models of natural language. Comput. Linguist. 18, 4 (Dec. 1992), 467--479. Google ScholarDigital Library
- Chang, E., Kingshy, G., Sychay, G., and Wu, G. CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Trans. on CSVT, 13(1):26--38, Jan. 2003. Google ScholarDigital Library
- Cusano, C., Ciocca, G., and Schettini, R. Image annotation using SVM. In Proc. Of Internet imaging IV, Vol. SPIE, 2004.Google Scholar
- Duygulu, P. and Barnard, K. Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In Seventh European Conference on Computer Vision, 4:97--112, 2002. Google ScholarDigital Library
- Fan, X., Xie, X., Li, Z., Li, M., and Ma, W. Y. Photo-to- Search: Using Multimodal Queries to Search the Web from Mobile Devices. 7th ACM SIGMM Workshop on MIR, 2005. Google ScholarDigital Library
- Feng, S. L., Manmatha, R., and Lavrenko, V. Multiple bernoulli relevance models for image and video annotation. In The International Conference on Computer Vision and Pattern Recognition, Washington, DC, June, 2004. Google ScholarDigital Library
- Ferhatosmanoglu, H., Tuncel, E., Agrawal, D., and Abbadi, A. E. Approximate nearest neighbor searching in multimedia databases. In Proceedings of the 17th IEEE Int'l. Conference on Data Engineering, Heidelberg, Germany, April, 2001, 2-6, pp. 503--511. Google ScholarDigital Library
- Jeon, J., Lavrenko, V., and Manmatha, R. Automatic Image Annotation and Retrieval Using Cross-media Relevance Models. In Proc. of ACM SIGIR conference on Research and development in information retrieval, pp. 119--126, July, 2003. Google ScholarDigital Library
- Lavrenko, V., Manmatha, R., and Jeon, J. A Model for Learning the Semantics of Pictures. In Proc. of the 17th Annual Conf. on Neural Information Processing Systems, 2003.Google Scholar
- Li, J. and Wang, J. Z. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. on PAMI, 25(10), Oct. 2003. Google ScholarDigital Library
- Miller, G. A. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (Nov. 1995), 39--41. Google ScholarDigital Library
- Mori, Y., Takahashi, H., and Oka, R. Image-to-word transformation based on dividing and vector quantizing images with words. In MISRM'99 First International Workshop on Multimedia Intelligent Storage and Retrieval Management, 1999.Google Scholar
- Page, L., Brin, S., Motwani, R., and Winograd, T. The Pagerank Citation Ranking: Bringing Order to the web, technical report, Stanford University, Stanford, CA, 1998.Google Scholar
- Robertson, S. E. and Walker, S. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 345--354. Springer-Verlag, 1994. Google ScholarDigital Library
- Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., and Jain, R. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000. Google ScholarDigital Library
- Wang, X. J., Zhang, L., Jing, F., and Ma, W. Y. AnnoSearch: Image Auto-Annotation by Search. In The International Conference on Computer Vision and Pattern Recognition, New York, June, 2006. Google ScholarDigital Library
- Yanai, K. and Barnard, K. 2005. Image region entropy: a measure of "visualness" of web images associated with one concept. In Proceedings of the 13th Annual ACM international Conference on Multimedia. New York, NY, 419--422. Google ScholarDigital Library
- Yeh, T., Tollmar, K., and Darrell, T. Searching the Web with Mobile Images for Location Recognition. In The International Conference on Computer Vision and Pattern Recognition, 2004, pp. 76--81. Google ScholarDigital Library
- Zeng, H., He, Q., Chen, Z., Ma, W., and Ma, J. 2004. Learning to cluster web search results. SIGIR, 2004. New York, NY, 210--217. Google ScholarDigital Library
- Zhang, L., Hu, Y., Li, M., Ma, W., and Zhang, H. 2004. Efficient propagation for face annotation in family albums. In Proceedings of the 12th Annual ACM international Conference on Multimedia. New York, NY, 716--723. Google ScholarDigital Library
Index Terms
- Scalable search-based image annotation of personal images
Recommendations
ConceptRank for search-based image annotation
Multimedia information is becoming an ubiquitous part of our lives, which brings an equally ubiquitous need for efficient multimedia retrieval. One of the possible solutions to this problem is to attach text descriptions to multimedia data objects, thus ...
Image annotation by large-scale content-based image retrieval
MM '06: Proceedings of the 14th ACM international conference on MultimediaImage annotation has been an active research topic in recent years due to its potentially large impact on both image understanding and Web image search. In this paper, we target at solving the automatic image annotation problem in a novel search and ...
Learning to reduce the semantic gap in web image retrieval and annotation
SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrievalWe study in this paper the problem of bridging the semantic gap between low-level image features and high-level semantic concepts, which is the key hindrance in content-based image retrieval. Piloted by the rich textual information of Web images, the ...
Comments