Skip to main content
SpringerLink
Log in

Selective Search for Object Recognition

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper addresses the problem of generating possible object locations for use in object recognition. We introduce selective search which combines the strength of both an exhaustive search and segmentation. Like segmentation, we use the image structure to guide our sampling process. Like exhaustive search, we aim to capture all possible object locations. Instead of a single technique to generate possible object locations, we diversify our search and use a variety of complementary image partitionings to deal with as many image conditions as possible. Our selective search results in a small set of data-driven, class-independent, high quality locations, yielding 99 % recall and a Mean Average Best Overlap of 0.879 at 10,097 locations. The reduced number of locations compared to an exhaustive search enables the use of stronger machine learning techniques and stronger appearance models for object recognition. In this paper we show that our selective search enables the use of the powerful Bag-of-Words model for recognition. The selective search software is made publicly available (Software: http://disi.unitn.it/~uijlings/SelectiveSearch.html).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. We found no difference in recognition accuracy when using the Random Forest assignment of Uijlings et al. (2010) or kmeans nearest neighbour assignment in van de Sande et al. (2010) on the Pascal dataset.

References

  • Alexe, B., Deselaers, T., Ferrari, V. (2010). What is an object? In CVPR.

  • Alexe, B., Deselaers, T., & Ferrari, V. (2012). Measuring the objectness of image windows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), 2189–2202.

    Article  Google Scholar 

  • Arbeláez, P., Maire, M., Fowlkes, C., & Malik, J. (2011). Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), 898–916.

    Article  Google Scholar 

  • Carreira, J., Sminchisescu, C. (2010). Constrained parametric min-cuts for automatic object segmentation. In CVPR.

  • Chum, O., Zisserman, A. (2007). An exemplar model for learning object classes. In CVPR.

  • Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603–619.

    Article  Google Scholar 

  • Csurka, G., Dance, C. R., Fan, L., Willamowski, J., & Bray, C. (2004). In ECCV statistical learning in computer vision: Visual categorization with bags of keypoints.

  • Dalal, N., Triggs, B. (2005). Histograms of oriented gradients for human detection. In CVPR.

  • Endres, I., Hoiem, D. (2010). Category independent object proposals. In ECCV.

  • Everingham, M., Gool, L. V., Williams, C., Winn, J., & Zisserman, A. (2011). The Pascal visual object classes challenge workshop: Overview and results of the detection challenge.

  • Everingham, M., van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010). The Pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88, 303–338.

    Google Scholar 

  • Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1627–1645.

    Article  Google Scholar 

  • Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision, 59, 167–181.

    Article  Google Scholar 

  • Geusebroek, J. M., van den Boomgaard, R., Smeulders, A. W. M., & Geerts, H. (2001). Color invariance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 1338–1350.

    Article  Google Scholar 

  • Gu, C., Lim, J. J., Arbeláez, P., & Malik, J. (2009). In CVPR: Recognition using regions.

  • Harzallah, H., Jurie, F., & Schmid, C. (2009). In ICCV: Combining efficient object localization and image classification.

  • Lampert, C. H., Blaschko, M. B., & Hofmann, T. (2009). Efficient subwindow search: A branch and bound framework for object localization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 2129–2142.

    Google Scholar 

  • Lazebnik, S., Schmid, C., Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR.

  • Li, F., & Carreira, J., Sminchisescu, C. (2010). In CVPR: Object recognition as ranking holistic figure-ground hypotheses.

  • Liu, C., Sharan, L., Adelson, E.H., Rosenholtz, R. (2010). Exploring features in a bayesian framework for material recognition. In Computer vision and pattern recognition 2010. IEEE.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110.

    Article  Google Scholar 

  • Maji, S., Berg, A. C., & Malik, J. (2008). In CVPR: Classification using intersection kernel support vector machines is efficient.

  • Maji, S., & Malik, J. (2009). Object detection using a max-margin hough transform. In CVPR.

  • Ojala, T., Pietikainen, M., & Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.

    Article  Google Scholar 

  • Perronnin, F., Sánchez, J., & Thomas M. (2010). In ECCV: Improving the Fisher Kernel for large-scale image classification.

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 888–905.

    Article  Google Scholar 

  • Sivic, J., Zisserman, A.(2003). Video google: A text retrieval approach to object matching in videos. In ICCV.

  • Sonnenburg, S., Raetsch, G., Henschel, S., Widmer, C., Behr, J., Zien, A., et al. (2010). The shogun machine learning toolbox. Journal of Machine Learning Research, 11, 1799–1802.

    MATH  Google Scholar 

  • Tu, Z., Chen, X., Yuille, A. L., & Zhu, S. (2005). Image parsing: Unifying segmentation, detection and recognition. Marr Prize Issue. International Journal of Computer Vision.

  • Uijlings, J. R. R., Smeulders, A. W. M., & Scha, R. J. H. (2010). Real-time visual concept classification. IEEE Transactions on Multimedia, 12(7), 665–681.

    Article  Google Scholar 

  • van de Sande, K. E. A., & Gevers, T. (2012). Illumination-invariant descriptors for discriminative visual object categorization. Technical report : University of Amsterdam.

  • van de Sande, K. E. A., Gevers, T., & Snoek, C. G. M. (2010). Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1582–1596.

    Article  Google Scholar 

  • van de Sande, K. E. A., Gevers, T., & Snoek, C. G. M. (2011). Empowering visual categorization with the GPU. IEEE Transactions on Multimedia, 13(1), 60–70.

    Article  Google Scholar 

  • Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009). In ICCV: Multiple kernels for object detection.

  • Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In CVPR, Volume 1, 511–518.

  • Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57, 137–154.

    Article  Google Scholar 

  • Zhou, X., Kai, Y., Zhang, T., & Huang, T. S. (2010). In ECCV: Image classification using super-vector coding of local image descriptors.

  • Zhu, L., Chen, Y., Yuille, A., & Freeman, W. (2010). In CVPR: Latent hierarchical structural learning for object detection.

Download references

Author information

Authors and Affiliations

  1. University of Trento, Trento, Italy

    J. R. R. Uijlings

  2. University of Amsterdam, Amsterdam, The Netherlands

    K. E. A. van de Sande, T. Gevers & A. W. M. Smeulders

Authors
  1. J. R. R. Uijlings
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. E. A. van de Sande
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. T. Gevers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. W. M. Smeulders
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. R. R. Uijlings.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T. et al. Selective Search for Object Recognition. Int J Comput Vis 104, 154–171 (2013). https://doi.org/10.1007/s11263-013-0620-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0620-5

Keywords

Search

Navigation