Abstract
We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to minimizing the divergence between the data distribution and the equilibrium distribution, we maximize the divergence between one-step reconstructions of the data and the equilibrium distribution. This eliminates the need to estimate equilibrium statistics, so we do not need to approximate the multimodal probability distribution of the free network with the unimodal mean field distribution. We test the learning algorithm on the classification of digits.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
G.E. Hinton. Training products of experts by minimizing contrastive divergence. Technical Report GCNU TR 2000-004, Gatsby Computational Neuroscience Unit, University College London, 2000.
G.E. Hinton and T.J. Sejnowski. Learning and relearning in Boltzmann machines, volume Volume 1: Foundations. MIT Press, 1986.
G.E. Hinton, M. Welling, Y.W. Teh, and K. Osindero. A new view of ICA. In Int. Conf. on Independent Component Analysis and Blind Source Separation, 2001.
J.J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. In Proceedings of the National Academy of Sciences, volume 79, pages 2554–2558, 1982.
J.R. Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connectionist Models, Proceedings of the 1990 Summer School, pages 10–17, 1991.
C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995–1019, 1987.
I. Stoianov, M. Zorzi, S. Becker, and C. Umilta. Associative arithmetic with Boltz-mann machines: the role of number representations. In International Conference on Artificial Neural Networks, 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Welling, M., Hinton, G.E. (2002). A New Learning Algorithm for Mean Field Boltzmann Machines. In: Dorronsoro, J.R. (eds) Artificial Neural Networks — ICANN 2002. ICANN 2002. Lecture Notes in Computer Science, vol 2415. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46084-5_57
Download citation
DOI: https://doi.org/10.1007/3-540-46084-5_57
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44074-1
Online ISBN: 978-3-540-46084-8
eBook Packages: Springer Book Archive