Liqiang Nie
Meng Wang
Skip Abstract Section
Abstract
This article studies a novel problem in image search. Given a text query and the image ranking list returned by an image search system, we propose an approach to automatically predict the search performance. We demonstrate that, in order to estimate the mathematical expectations of Average Precision (AP) and Normalized Discounted Cumulative Gain (NDCG), we only need to predict the relevance probability of each image. We accomplish the task with a query-adaptive graph-based learning based on the images’ ranking order and visual content. We validate our approach with a large-scale dataset that contains the image search results of 1,165 queries from 4 popular image search engines. Empirical studies demonstrate that our approach is able to generate predictions that are highly correlated with the real search performance. Based on the proposed image search performance prediction scheme, we introduce three applications: image metasearch, multilingual image search, and Boolean image search. Comprehensive experiments are conducted to validate our approach.
- Abusalah, M., Tait, J., and Oakes, M. P. 2005. Literature review of cross language information retrieval. In Proceedings of World Academy of Science, Engineering and Technology Conference.Google Scholar
- Ahonen, T., Hadid, A., and Pietikainen, M. 2004. Face recognition with local binary patterns. In Proceedings of the European Conference on Computer Vision.Google Scholar
- Amati, G., Carpineto, C., Romano, G., and Bordoni, F. U. 2004. Query difficulty, robustness and selective application of query expansion. In Proceedings of the European Conference on IR Research.Google Scholar
- Balasubramanian, N., Kumaran, G., and Carvalho, V. R. 2010. Predicting query performance on the web. In Proceedings of the ACM International SIGIR Conference. Google ScholarDigital Library
- Banerjee, S. and Pedersen, T. 2003. Extended gloss overlaps as a measure of semantic relatedness. In Proceedings of the Joint Conference on Artificial Intelligence. Google ScholarDigital Library
- Benitez, A., Beigi, M., Beigi, I., and Chang, S. F. 1998. A content-based image meta-search engine using relevance feedback. IEEE Internet Comput. 2, 59--69. Google ScholarDigital Library
- Carmel, D., Yom-Tov, E., Darlow, A., and Pelleg, D. 2006. What makes a query difficult? In Proceedings of the ACM International SIGIR Conference. Google ScholarDigital Library
- Cronen-Townsend, S., Zhou, Y., and Croft, W. B. 2002. Predicting query performance. In Proceedings of the ACM International SIGIR Conference. Google ScholarDigital Library
- Hauff, C., Hiemstra, D., and de Jong, F. 2008a. A survey of pre-retrieval query performance predictors. In Proceedings of the Conference on Information and Knowledge Management. Google ScholarDigital Library
- Hauff, C., Murdock, V., and Baeza-Yates, R. 2008b. Improved query difficulty prediction for the web. In Proceedings of the Conference on Information and Knowledge Management. Google ScholarDigital Library
- Hauff, C., Azzopardi, L., and Hiemstra, D. 2009. The combination and evaluation of query performance prediction methods. In Proceedings of the European Conference on IR Research. Google ScholarDigital Library
- He, B. and Ounis, I. 2004. Inferring query performance using pre-retrieval predictors. In Proceedings of the Symposium on String Processing and Information Retrieval.Google Scholar
- Hsu, W., Kennedy, L., and Chang, S. F. 2007. Video search reranking through random walk over document-level context graph. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
- Jing, Y. and Baluja, S. 2008. Visualrank: Applying pagerank to large-scale image search. IEEE Trans. Pattern. Anal. Mach. Intell. 30, 11, 1877--1890. Google ScholarDigital Library
- Lew, M. S. 2006. Content-Based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Comm. Appl. 2, 1. Google ScholarDigital Library
- Li, H., Wang, M., and Hua, X. S. 2009. Msra-mm 2.0: A large-scale web multimedia dataset. In Proceedings of the IEEE ICDM International Conference on Data Mining Workshop. Google ScholarDigital Library
- Li, J. and Wang, J. Z. 2003. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Pattern. Anal. Mach. Intell. 25, 9, 1075--1088. Google ScholarDigital Library
- Li, J., Allinson, N., Tao, D., and Li, X. 2006. Multi-Training support vector machine for image retrieval. IEEE Trans. Image Process. 15, 11, 3597--3601. Google ScholarDigital Library
- Li, Y., Luo, Y., Tao, D., and Xu, C. 2011. Query difficulty guided image retrieval system. In Proceedings of the International Conference on Advances in Multimedia Modeling. Google ScholarDigital Library
- Liu, Y. and Mei, T. 2011. Optimizing visual search reranking via pairwise learning. IEEE Trans. Multimedia 13, 2, 280--291. Google ScholarDigital Library
- Manning, C. D., Raghavan, P., and Schtze, H. 2008. Introduction to Information Retrieval. Cambridge University Press. Google ScholarDigital Library
- Manoj, M. and Elizabeth, J. 2008. Information retrieval on internet using meta-search engines: A review. J. Sci. Industr. Res. 67, 739--746.Google Scholar
- Monay, F. and Gatica-Perez, D. 2003. On image auto-annotation with latent space models. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
- Mothe, J. and Tanguy, L. 2005. Linguistic features to predict query difficulty. In Proceedings of the ACM International SIGIR Workshop.Google Scholar
- Natsev, A., Naphade, M. R., and TešiĆ, J. 2005. Learning the semantics of multimedia queries and concepts from a small number of examples. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
- Patwardhan, S. 2006. Using wordnet-based context vectors to estimate the semantic relatedness of concepts. In Proceedings of the European Association of Chinese Linguistics.Google Scholar
- Popescu, A. 2007. Image retrieval using a multilingual ontology. In Large Scale Semantic Access to Content (Text, Image, Video, and Sound). Google ScholarDigital Library
- Powell, A. L., French, J. C., Callan, J., Connell, M., and Viles, C. L. 2000. The impact of database selection on distributed searching. In Proceedings of the ACM International SIGIR Conference. Google ScholarDigital Library
- Quénot, G., Tan, T. P., Le, V. B., Ayache, S., Besacier, L., and Mulhem, P. 2010. Content-based search in multilingual audiovisual documents using the international phonetic alphabet. Multimedia Tools Appl. 48, 1, 123--140. Google ScholarDigital Library
- Selberg, E. and Etzioni, O. 1995. Multi-Service search and comparison using the metacrawler. In Proceedings of the International World Wide Web Conference.Google Scholar
- Si, L. and Callan, J. 2005. Modeling search engine effectiveness for federated search. In Proceedings of the ACM International SIGIR Conference. Google ScholarDigital Library
- Smeulders, A., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern. Anal. Mach. Intell. 22, 12. Google ScholarDigital Library
- Smith, L. S. and Hurson, A. R. 2003. A search engine selection methodology. In Proceedings of the International Conference on Information Technology: Computers and Communications. Google ScholarDigital Library
- Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., and Hua, X. S. 2008. Bayesian video search reranking. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
- Wang, M., Hua, X. S., Hong, R., Tang, J., Qi, G. J., and Song, Y. 2009a. Unified video annotation via multi-graph learning. IEEE Trans. Circ. Syst. Video Technol. 19, 5. Google ScholarDigital Library
- Wang, M., Hua, X. S., Tang, J., and Hong, R. 2009b. Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans. Multimedia 11, 3. Google ScholarDigital Library
- Wang, M., Yang, K., Hua, X. S., and Zhang, H. J. 2010. Towards a relevant and diverse search of social images. IEEE Trans. Multimedia 12, 8, 829--842. Google ScholarDigital Library
- Xing, X., Zhang, Y., and Han, M. 2010. Query difficulty prediction for contextual image retrieval. In Proceedings of the European Conference on IR Research. Google ScholarDigital Library
- Yan, R., Hauptmann, A., and Jin, R. 2003. Multimedia search with pseudo-relevance feedback. In Proceedings of the International Conference on Image and Video Retrieval. Google ScholarDigital Library
- Yang, L. and Hanjalic, A. 2010. Supervised reranking for web image search. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
- Zhao, Y., Scholer, F., and Tsegay, Y. 2008. Effective pre-retrieval query performance prediction using similarity and variability evidence. In Proceedings of the European Conference on IR Research. Google ScholarDigital Library
- Zhou, Y. and Croft, W. B. 2007. Query performance prediction in web search environments. In Proceedings of the ACM International SIGIR Conference. Google ScholarDigital Library
Index Terms
- Oracle in Image Search: A Content-Based Approach to Performance Prediction
Recommendations
Social-Sensed Image Search
Although Web search techniques have greatly facilitate users’ information seeking, there are still quite a lot of search sessions that cannot provide satisfactory results, which are more serious in Web image search scenarios. How to understand user ...
Bing, the fastest growing image search engine
MM '14: Proceedings of the 22nd ACM international conference on MultimediaSince the launch of Bing (www.bing.com) in June 2009, we have seen Bing web search market share in the US more than doubled and Bing image search query share quadrupled. In this talk, I will share our experience building Bing image search as the fastest ...
Why People Search for Images using Web Search Engines
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data MiningWhat are the intents or goals behind human interactions with image search engines? Knowing why people search for images is of major concern to Web image search engines because user satisfaction may vary as intent varies. Previous analyses of image ...
Comments