Yuanlong Shao
ABSTRACT
We propose a novel technique for semi-supervised image annotation which introduces a harmonic regularizer based on the graph Laplacian of the data into the probabilistic semantic model for learning latent topics of the images. By using a probabilistic semantic model, we connect visual features and textual annotations of images by their latent topics. Meanwhile, we incorporate the manifold assumption into the model to say that the probabilities of latent topics of images are drawn from a manifold, so that for images sharing similar visual features or the same annotations, their probability distribution of latent topics should also be similar. We create a nearest neighbor graph to model the manifold and propose a regularized EM algorithm to simultaneously learn a generative model and assign probability density of latent topics to images discriminatively. In this way, databases with very few labeled images can be annotated better than previous works.
- K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107--1135, 2003. Google Scholar
Digital Library
- M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399--2434, 2006. Google ScholarDigital Library
- D. M. Blei and M. I. Jordan. Modeling annotated data. In Proc. ACM Int. Conf. on Research and Development in Informaion Retrieval(ACM SIGIR), pages 127--134, 2003. Google ScholarDigital Library
- D. Cai, Q. Mei, J. Han, and C. Zhai. Modeling hidden topics on document manifold. In Proc. ACM Conf. on Information and knowledge management(CIKM'08), pages 911--920, 2008. Google ScholarDigital Library
- G. Csurka, C. R. Dance, L. Fan, J. Willamowski, and C. Bray. Visual categorization with bags of keypoints. In Workshop on Statistical Learning in Computer Vision, ECCV, pages 1--22, 2004.Google Scholar
- X. He, D. Cai, Y. Shao, H. Bao, and J. Han. Laplacian regularized gaussian mixture model for data clustering. Preprint.Google Scholar
- Q. Mei, D. Cai, D. Zhang, and C. Zhai. Topic modeling with network regularization. In Proc. ACM Int. Conf. on World Wide Web (WWW'08), pages 101--110, 2008. Google ScholarDigital Library
- F. Monay and D. Gatica-Perez. On image auto-annotation with latent space models. In Proc. ACM Int. Conf. on Multimedia (SIGMM'03), pages 275--278, 2003. Google ScholarDigital Library
- F. Monay and D. Gatica-Perez. Plsa-based image auto-annotation: constraining the latent space. In Proc. ACM Int. Conf. on Multimedia (SIGMM'04), pages 348--351, 2004. Google ScholarDigital Library
- R. M. Neal and G. E. Hinton. A view of the em algorithm that justifies incremental, sparse, and other variants. In Learning in graphical models, pages 355--368. 1999. Google ScholarDigital Library
- R. Zhang, Z. M. Zhang, M. Li, W.-Y. Ma, and H.-J. Zhang. A probabilistic semantic model for image annotation and multi-modal image retrieval. In Proc. IEEE Int. Conf. on Computer Vision (ICCV'05), pages 846--851, 2005. Google ScholarDigital Library
- X. Zhu, J. Lafferty, and Z. Ghahramani. Semi-supervised learning using gaussian fields and harmonic functions. In Proc. Int. Conf. Machine Learning(ICML'05), 2005.Google Scholar
Index Terms
- Semi-supervised topic modeling for image annotation
Recommendations
Opinion integration through semi-supervised topic modeling
WWW '08: Proceedings of the 17th international conference on World Wide WebWeb 2.0 technology has enabled more and more people to freely express their opinions on the Web, making the Web an extremely valuable source for mining user opinions about all kinds of topics. In this paper we study how to automatically integrate ...
Automatic image annotation using semi-supervised generative modeling
Image annotation approaches need an annotated dataset to learn a model for the relation between images and words. Unfortunately, preparing a labeled dataset is highly time consuming and expensive. In this work, we describe the development of an ...
A Novel Region-based Image Annotation Using Multi-instance Learning
WKDD '09: Proceedings of the 2009 Second International Workshop on Knowledge Discovery and Data MiningIn this paper, we formulate image annotation as a semi-supervised learning problem under multi-instance learning framework. A novel graph based semi-supervised learning approach to image annotation using multiple instances is presented, which extends ...
Comments