Abstract
We present an approach for image retrieval using a very large number of highly selective features and efficient learning of queries. Our approach is predicated on the assumption that each image is generated by a sparse set of visual “causes” and that images which are visually similar share causes. We propose a mechanism for computing a very large number of highly selective features which capture some aspects of this causal structure (in our implementation there are over 46,000 highly selective features). At query time a user selects a few example images, and the AdaBoost algorithm is used to learn a classification function which depends on a small number of the most appropriate features. This yields a highly efficient classification function. In addition we show that the AdaBoost framework provides a natural mechanism for the incorporation of relevance feedback. Finally we show results on a wide variety of image queries.
Similar content being viewed by others
References
Abdel-Mottaleb, M., Dimitrova, N., Desai, R., and Martino, J. 1996. CONIVAS: Content-based image and video access system. In ACM Multimedia, pp. 427–428.
Amit, Y. and Geman, D. 1997. Shape quantization and recognition with randomized trees. Neural Computation, 9(7):1545–1588.
Aslandogan, Y.A., Thier, C., and Yu, C. 1996. A system for effective content based image retrieval. In ACM Multimedia, pp. 429–430.
Bach, J.R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Jain, R., and Shu, C.F. 1996. The Virage image search engine: An open framework for image management. In International Society for Optical Engineering, vol. 4, pp. 76–87.
Belongie, S., Carson, C., Greenspan, H., and Malik, J. 1998. Colorand texture-based image segmentation using EM and its application to content-based image retrieval. In International Conference on Computer Vision, pp. 675–682.
Burt, P.J. and Adelson, E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4):532–540.
Cohn, D., Atlas, L., and Ladner, R. 1994. Improving generalization with active learning. Machine Learning, 15(2):201–221.
Cortes, C. and Vapnik, V. 1995. Support vector networks. Machine Learning, 20(3):273–297.
Cox, I., Miller, M., Minka, T., and Yianilos, P.N. 1998. An optimized interaction strategy for bayesian relevance feedback. In Computer Vision and Pattern Recognition, pp. 553–558.
DeBonet, J.S. and Viola, P. 1998. Structure driven image database retrieval. In Neural Information Processing Systems, vol. 10, pp. 866–872.
Diaconis, P. and Freedman, D. 1984. Asymptotics of graphical projection pursuit. Annals of Statistics, 12:793–815.
Flickner, M., Sawhney, H., Niblack W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Computer, 28(9):23–32.
Freund, Y. and Schapire, R.E. 1997. A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and Systems Science, 55(1):119–139.
Guo, G., Jain, A.K., Ma W., and Zhang, H. 2002. Learning similarity measure for natural image retrieval with relevance feedback. IEEE Transactions on Neural Networks, 13(4):811–820.
Huang, J., Kumar, S.R., Mitra, M., Zhu, W.J., and Zabih, R. 1997. Image indexing using color correlograms. In Computer Vision and Pattern Recognition, pp. 762–768.
Jacobs, C.E., Finkelstein, A., and Salesin, D.H. 1995. Fast multiresolution image querying. In ACM SIGGRAPH, pp. 277–286.
Kelly, M. and Cannon, T.M. 1995. Query by image example: The CANDID approach. In International Society for Optical Engineering, vol. 2420, pp. 238–248.
La Cascia, M. and Ardizzone, E. 1996. JACOB: Just a content-based query system for video databases. In International Conference on Acoustics Speech and Signal Processing, vol. 2, pp. 1216–1219.
LeCun, Y., Boser, B., Denker, J.S., Henderson, B., Howard, R.E., Hubbard, W., and Jackel, L.D. 1989. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551.
Lipson, P., Grimson, E., and Sinha, P. 1997. Configuration based scene classification and image indexing. In Computer Vision and Pattern Recognition, pp. 1007–1013.
Nastar, C., Mitschke, M., and Meilhac, C. 1998. Efficient query refinement for image retrieval. In Computer Vision and Pattern Recognition, pp. 547–552.
Ogle, V.E. and Stonebraker, M. 1995. Chabot: Retrieval from a relational database of images. IEEE Computer, 28(9):40–48.
Papageorgiou, C., Oren, M., and Poggio, T. 1998. A general framework for object detection. In International Conference on Computer Vision, pp. 555–562.
Pentland, A., Picard, R.W., and Sclaroff, S. 1996. Photobook: Content-based manipulation of image databases. International Journal of Computer Vision, 18(3):233–254.
Ratan, A.L. and Maron, O. 1998. Multiple instance learning for natural scene classification. In International Conference on Machine Learning, pp. 341–349.
Rowley, H.A., Baluja, S., and Kanade, T. 1998. Neural networkbased face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38.
Rui, Y. and Huang, T. 2000. Optimizing learning in image retrieval. In Computer Vision and Pattern Recognition, vol. 1, pp. 236–243.
Rui, Y., Huang, T., and Chang, S. 1999. Image retrieval: Current techniques, promising directions and open issues. Journal of Visual Communication and Image Representation, 10:39–62.
Rui, Y., Huang, T., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in mars. In IEEE International Conference on Image Processing, vol. 2, pp. 815–815.
Schapire, R.E., Freund, Y., Bartlett, P., and Lee, W.S. 1998. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651–1686.
Simoncelli, E.P. and Adelson, E.H. 1996. Noise removal via bayesian wavelet coring. In IEEE International Conference on Image Processing, pp. 379–382.
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349–1380.
Smith, J.R. and Chang, S.-F. 1996. VisualSEEk: A fully automated content-based image query system. In ACM Multimedia, pp. 87–98.
Swain, M.J. and Ballard, D.H. 1991. Color indexing. International Journal of Computer Vision, 7(1).
Vasconcelos, N. and Lippman, A. 2000. A probabilistic architecture for content-based image retrieval. In Computer Vision and Pattern Recognition, vol. 1, pp. 216–221.
Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, vol. 1, pp. 511–518.
Wu, Y., Tian, Q., and Huang, T.S. 2000. Discriminant-EM algorithm with application to image retrieval. In ComputerVision andPattern Recognition, vol. 1, pp. 222–227.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Tieu, K., Viola, P. Boosting Image Retrieval. International Journal of Computer Vision 56, 17–36 (2004). https://doi.org/10.1023/B:VISI.0000004830.93820.78
Issue Date:
DOI: https://doi.org/10.1023/B:VISI.0000004830.93820.78