Peter V. Gehler
ABSTRACT
Probabilistic modelling of text data in the bag-of-words representation has been dominated by directed graphical models such as pLSI, LDA, NMF, and discrete PCA. Recently, state of the art performance on visual object recognition has also been reported using variants of these models. We introduce an alternative undirected graphical model suitable for modelling count data. This "Rate Adapting Poisson" (RAP) model is shown to generate superior dimensionally reduced representations for subsequent retrieval or classification. Models are trained using contrastive divergence while inference of latent topical representations is efficiently achieved through a simple matrix multiplication.
- Airoldi, E., Cohen, W., & Fienberg, S. (2005). Bayesian methods for frequent terms in text. Proc. of the CSNA & INTERFACE Annual Meetings.]]Google Scholar
- Blei, D. M., & Jordan, M. I. (2004). Variational inference for dirichlet process mixtures. Bayesian Analysis, 1, 121--144.]]Google ScholarCross Ref
- Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993--1022.]] Google ScholarDigital Library
- Buntine, W. (Ed.). (2002). Variational extensions to em and multinomial pca, vol. 2430 of Lecture Notes in Computer Science. Helsinki, Finland: Springer.]] Google ScholarDigital Library
- Buntine, W., & Jakulin, A. (2004). Applying discrete pca in data analysis. Proceedings of the 20th conference on Uncertainty in artificial intelligence (pp. 59--66). Banff, Canada.]] Google ScholarDigital Library
- Carreira-Perpinan, M., & Hinton, G. (2005). On contrastive divergence learning. Tenth International Workshop on Artificial Intelligence and Statistics. Barbados.]]Google Scholar
- Casella, G., & Robert, C. (1996). Rao-blackwellisation of sampling schemes. Biometrika, 83(1), 81--94.]]Google ScholarCross Ref
- Deerwester, S., Dumais, S., Landauer, T., Furnas, G., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41, 391--407.]]Google ScholarCross Ref
- Fergus, R., Fei-Fei, L., Perona, P., & Zisserman, A. (2005). Learning object categories from google's image search. Proceedings of the International Conference on Computer Vision.]] Google ScholarDigital Library
- Girolami, M., & Kaban, A. (2003). On an equivalence between PLSI and LDA. Proceedings of SIGIR 2003.]] Google ScholarDigital Library
- Griffiths, T., & Steyvers, M. (2002). A probabilistic approach to semantic representation. Proceedings of the 24th Annual Conference of the Cognitive Science Society.]]Google Scholar
- Hinton, G. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14, 1771--1800.]] Google ScholarDigital Library
- Hinton, G., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief networks. Neural Computation. to appear.]] Google ScholarDigital Library
- Hofmann, T. (1999). Probabilistic latent semantic analysis. Proc. of Uncertainty in Artificial Intelligence, UAI '99. Stockholm.]]Google Scholar
- Kadir, T., & Brady, M. (2001). Saliency, scale and image description. Int. J. Comput. Vision, 45, 83--105.]] Google ScholarDigital Library
- Lee, D., & Seung, H. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788--791.]]Google ScholarCross Ref
- Li, F., & Perona, P. (2005). A bayesian hierarchical model for learning natural scene categories. Proceedings of the Conference on Computer Vision and Pattern Recognition.]] Google ScholarDigital Library
- McCallum, A. (1996). Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering. http://www.cs.cmu.edu/~mccallum/bow.]]Google Scholar
- Minka, T., & Lafferty, J. (2002). Expectation-propogation for the generative aspect model. Proc. of the 18th Annual Conference on Uncertainty in Artificial Intelligence (pp. 352--359).]]Google Scholar
- Olshausen, A., & Field, D. (1997). Sparse coding with over-complete basis set: A strategy employed by v1? Vision Research, 37, 3311--3325.]]Google ScholarCross Ref
- Roweis, S. (1997). Em algorithms for pca and spca. Neural Information Processing Systems (pp. 626--632).]] Google ScholarDigital Library
- Welling, M., & Hinton, G. (2001). A new learning algorithm for mean field Boltzmann machines. Proc. of the Int'l Conf. on Artificial Neural Networks. Madrid, Spain.]] Google ScholarDigital Library
- Welling, M., Hinton, G., & Osindero, S. (2002). Learning sparse topographic representations with products of student-t distributions. Neural Information Processing Systems.]]Google Scholar
- Welling, M., Rosen-Zvi, M., & Hinton, G. (2004). Exponential family harmoniums with an application to information retrieval. Neural Information Processing Systems.]]Google Scholar
- Xing, E., Yan, R., & Hauptman, A. (2005). Mining associated text and images with dual-wing harmoniums. Proc. of the Conf. on Uncertainty in Artificial Intelligence.]]Google Scholar
Index Terms
- The rate adapting poisson model for information retrieval and object recognition
Recommendations
A study of Poisson query generation model for information retrieval
SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrievalMany variants of language models have been proposed for information retrieval. Most existing models are based on multinomial distribution and would score documents based on query likelihood computed based on a query generation probabilistic model. In ...
Affine invariant topic model for generic object recognition
ISNN'10: Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part IIThis paper presents a novel topic model named Affine Invariant Topic Model(AITM) for generic object recognition Abandoning the “bag of words” assumption in traditional topic models, AITM incorporates spatial structure into traditional LDA AITM extends ...
Discriminative information preservation for face recognition
It is usually difficult to find the optimal low dimensional subspace for face recognition. Patch alignment framework (PAF) is an important systematic framework that can be applied to understand the common thought and essential differences of a numerous ...
Comments