The Restricted Boltzmann Machine: from the statistical physics of disordered systems to a practical and interpretative generative machine learning.

In this talk I will present our recent work on the Restricted Boltzmann Machine (RBM). RBMs were introduced decades ago by Hinton as a variant of the Boltzmann Machine (BM), but with hidden variables and a characteristic bipartite architecture. RBMs, introduced at the time as a "product of experts", were successfully trained as a generative model using the so-called contrastive-divergence method and, despite their shallow and simple architecture, were able to generate convincing new samples for complex real-world datasets. They later became popular as building blocks for pre-trained deep neural networks before the advent of more sophisticated methods.

The increasing interest of physicists and statistical physicists in RBMs in recent years is driven to several factors. First, RBMs can be seen as a generalization of BM that can be used to study interesting emerging phenomena, such as the phase diagram of the learned machine, how the learned free energy landscape is related to the properties of the dataset, or how the features of the dataset are encoded during the learning dynamics. Secondly, its practicality and simplicity make it an accessible model for physicists, providing a more understandable alternative to large, opaque neural networks.

I will present our understanding of the phase diagram and the learning dynamics of this model at both analytical and numerical levels. I will then show how we can construct equivalences between RBMs and generalized BMs where the weights of the RBM can be mapped into effective K-body interactions so that we are able to infer interacting components for a given dataset.