Skip to main content
SpringerLink
Log in

Collaborative image retrieval via regularized metric learning

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users' feedback that has been collected by CBIR systems in history, which is also called “collaborative image retrieval.” In this paper, we present a novel metric learning approach, named “regularized metric learning,” for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users' relevance judgments. Compared to the previous research, a regularization mechanism is used in our algorithm to effectively prevent overfitting. Meanwhile, we formulate the proposed learning algorithm into a semidefinite programming problem, which can be solved very efficiently by existing software packages and is scalable to the size of log data. An extensive set of experiments has been conducted to show that the new algorithm can substantially improve the retrieval accuracy of a baseline CBIR system using Euclidean distance metric, even with a modest amount of log data. The experiment also indicates that the new algorithm is more effective and more efficient than two alternative algorithms, which exploit log data for image retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ashwin, T.V., Navendu, J., Ghosal, S.: Improving image retrieval performance with negative relevance feedback. In: Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (ICASSP'01), UT (2001)

  2. Belkin, M., Niyog, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from examples. Technical Report, Computer Science Technical Report TR-2004-06, University of Chicago (2004)

  3. Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems, vol. 14 (2002)

  4. Blei, D., Jordan, M.I.: Modeling annotated data. In: Proceedings of the 26th International ACM SIGIR Conference (SIGIR′03), pp. 127–134 (2003)

  5. Burges, C.J.C.: A tutorial on support vector machine for pattern recognition. Knowl. Discov. Data Min. 2(2), 121–167 (1998)

    Article  Google Scholar 

  6. Cox, I.J., Miller, M., Minka, T., Yianilos, P.: An optimized interaction strategy for Bayesian relevance feedback. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR′98), pp. 553–558, Santa Barbara, CA (1998)

  7. Duygulu, P., Barnard, K., de Freitas, J., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Proceedings of the 7th European Conference on Computer Vision, pp. 97–112 (2002)

  8. Gill, P.E., Murray, W., Wright, M.H.: Practical optimization. Academic, London (1981)

    MATH  Google Scholar 

  9. Gong, Y., Chua Zhang, H.Z., Sakauchi, H.C.M.: An image database system with content capturing and fast image indexing abilities. In: IEEE International Conference on Multimedia Computing and Systems (1994)

  10. He, J., Li, M., Zhang, H.J., Tong, H., Zhang, C.: Manifold ranking based image retrieval. In: Proceedings of ACM Multimedia 2004 (2004)

  11. He, X., Ma, W.-Y., Zhang, H.-J.: Learning an image manifold for retrieval. In: Proceedings of ACM MM 2004 (2004)

  12. He, X., King, O., Ma, W.-Y., Li, M., Zhang, H.J.: Learning a semantic space from user's relevance feedback for image retrieval. IEEE Trans. Circ. Syst. Video Technol. 13(1), 39–48 (2003)

    Article  Google Scholar 

  13. Heesch, D., Yavlinsky, A., Rüuger, S.: Performance comparison between different similarity models for cbir with relevance feedback. In: Proceedings of International Conference on Image and Video Retrieval (CIVR′03), LNCS 2728, pp. 456–466. Springer, Berlin Heidelberg New York (2003)

    Google Scholar 

  14. Hoi, C.H., Lyu, M.R.: Group-based relevance feeedback with support vector machine ensembles. In: Proceedings 17th International Conference on Pattern Recognition (ICPR′04), Cambridge, UK (2004)

  15. Hoi, C.-H., Lyu, M.R.: A novel log-based relevance feedback technique in content-based image retrieval. In: Proceedings of ACM Multimedia 2004 (2004)

  16. Hoi, C.H., Lyu, M.R., Jin, R.: Integrating user feedback log into relevance feedback by coupled svm for content-based image retrieval. In: Proceedings of the 1st IEEE International Workshop on Managing Data for Emerging Multimedia Applications (EMMA 2005) (2005)

  17. Hoi, C.-H., Lyu, M.R.: Web image learning for searching semantic concepts in image databases. In: Poster Proceedings of the 13th International World Wide Web Conference (WWW′2004), New York (2004)

  18. Huang, T.S., Zhou, X.S.: Image retrieval by relevance feedback: from heuristic weight adjustment to optimal learning methods. In:Proceedings of IEEE International Conference on Image Processing (ICIP′01), Thessaloniki, Greece (2001)

  19. Jain, A.K., Vailaya, A.: Shape-based retrieval: a case study with trademark image database. Pattern Recognit. 9, 1369–1390 (1998)

    Article  Google Scholar 

  20. Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th International ACM SIGIR Conference (SIGIR′03), pp. 119–126 (2003)

  21. King, I., Zhong, J.: Integrated probability function and its application to content-based image retrieval by relevance feedback. Pattern Recognit. 36(9), 2177–2186 (2003)

    Article  MATH  Google Scholar 

  22. Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: Advances in Neural Information Processing Systems (NIPS′03) (2003)

  23. Lebanon, G.: Learning Riemannian metrics. In: Proceedings of the 19th Conference on Uncertainty in Articial Intelligence (2003)

  24. Lu, Y., Hu, C., Zhu, X., Zhang, H.J., Yang, Q.: A unified framework for semantics and feature based relevance feedback in image retrieval systems. In: MULTIMEDIA ′00: Proceedings of the Eighth ACM International Conference on Multimedia, pp. 31–37, ACM, New York (2000)

  25. Muller, H., Pun, T., Squire, D.: Learning from user behavior in image retrieval: Application of market basket analysis. Int. J. Comput. Vis. 56(1–2), 65–77 (2004)

    Article  Google Scholar 

  26. Porkaew, K., Chakrabarti, K., Mehrotra, S.: Query refinement for multimedia retrieval and its evaluation techniques in mars. In:Proceedings of ACM Multimedia (MM′99), Orlando, FL (1999)

  27. Rui, Y., Huang, T.S.: A novel relevance feedback technique in image retrieval. In: Proceedings of the ACM Multimedia (MM′99), pp. 67–70, Orlando, FL (1999)

  28. Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance feedback: a power tool in interactive content-based image retrieval. IEEE Trans. Circ. Syst. Video Technol. 8(5), 644–655 (1998)

    Article  Google Scholar 

  29. Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Inf. Proc. Manag.: Int. J. 24(5), 513–523 (1988)

    Article  Google Scholar 

  30. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.:Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)

    Article  Google Scholar 

  31. Smith, J., Chang, S.-F.: Automated image retrieval using color and texture. IEEE Trans. Pattern Anal. Mach. Intell. (1996)

  32. Sturm, J.F.: Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimization Methods and Software, pp.~11–12, 625–653. Special issue on Interior Point Methods (CD supplement with software) (1999)

  33. Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: Proceedings of The Ninth ACM International Conference on Multimedia, pp. 107–118. ACM, New York (2001)

  34. Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38(1), 49–95 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  35. Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance metric learning with application to clustering with side-information. In:Thrun, S., Becker, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 505–512. MIT Press, Cambridge, MA (2003)

  36. Zhou, X.-D., Zhang, L., Liu, L., Zhang, Q., Shi,B.-L.: A relevance feedback method in image retrieval by analyzing feedback log file. In: Proceedings of the International Conference on Machine Learning and Cybernetics, vol. 3, pp. 1641–1646, Beijing (2002)

Download references

Author information

Authors and Affiliations

  1. Language Technology Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Luo Si

  2. Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, 48824, USA

    Rong Jin

  3. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong

    Steven C. H. Hoi & Michael R. Lyu

Authors
  1. Luo Si
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven C. H. Hoi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael R. Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luo Si.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Si, L., Jin, R., Hoi, S.C.H. et al. Collaborative image retrieval via regularized metric learning. Multimedia Systems 12, 34–44 (2006). https://doi.org/10.1007/s00530-006-0033-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-006-0033-1

Keywords

Search

Navigation