Conference publications


XXIX conference

Statistical mechanics application for deep-belief networks

Rudamenko R., Perepelkin E.1, Polyakova R.2, Inozemtseva N., Sadovnikov B.

Faculty of Physics, Lomonosov Moscow State University



1 pp. (accepted)

Humans and other animals can understand concepts from only a few examples. while standard machine learning algorithms require a large number of examples to extract hidden features. Unsupervised learning is procedure of revealing hidden features from unlabeled data.In deep neural network training, unsupervised data pre-training increases the final accuracy of the algorithm by decreasing an initial parameter space from which fine-tuning begins. However, there are few theoretical papers devoted to detail unsupervised learning description. Crucial reason is that unsupervised learning process in deep neural network is usually complicated. That’s why understanding the mechanism of it in elementary models plays an important role.Boltzmann machines are the basic unit for developing deep-belief networks. Due to their ability to reveal hidden internal representations and solve complex combinatorial problems, they are used in machine learning and statistical patterns infer. In this work we consider a Restricted Boltzmann machine (RBM), with links between neurons of different layers, but without internal.To solve computational problems, the machine firstly pass training,according to the selected algorithms. After that, the visible layer is initialized with a given state, and the system evolves to a stationary distribution. Finally, the output layer represents the solution of the problem.Dealing with deep networks often issues the loss of interpretation of the obtained features, i.e., the loss of physical essence.Despite success in practical applications, the rigorous mathematical description of Boltzmann machines remains a challenge. The symmetry property of the weights matrix and the equality of the main diagonal determine the similarity of the Boltzmann machine with the physical model of spin glasses. RBM with binary bonds is equivalent to a bipartite spin glass with layer variables of different nature. The purpose of this work is to physically describe the RBM and study its process modes by analytical and numerical methods.


© 2004 Designed by Lyceum of Informational Technologies №1533