Building effective shallow generative models through side information

The remarkable progress in machine learning over the past decade has been driven largely by the adoption of deep, highly nonlinear architectures. Modern networks now contain up to a trillion parameters, demanding enormous computational and energy resources. Although these models achieve exceptional performance, their depth and nonlinearity often render them uninterpretable. This lack of interpretability poses challenges for sensitive applications where understanding why a model makes a prediction is as important as the prediction itself. I present evidence that much of this apparent nonlinearity arises from neglecting relevant side information, and that simple, shallow generative models become viable when such information is included. By analogy with physics, the Ising model is a simple pairwise description of a ferromagnet, but integrating out spins under a renormalization group flow produces an effective Hamiltonian with increasingly complex, higher-order interactions. These kinds of effects are endemic in biological datasets, where systems are often only partially observed and other relavant features are not considered. I develop an inference approach for low-rank Ising models with side information to create effective, simple, and shallow descriptions of real datasets. Beyond being an effective tool for modelling high-dimensional complex datasets, this approach is useful tool for determining what data features are important for understanding the macroscopic behavior of the system.