Minimax entropy: The statistical physics of optimal constraints

In the study of high-dimensional systems, one would like to uncover a small set of features that explain a wide range of behaviors. The maximum entropy principle provides the unique mapping from fine-scale features to large-scale behaviors, yet it does not tell us which features to include in a model. Here we show that one should select the features that minimize the entropy of the maximum entropy model, resulting in a “minimax entropy” principle. While applying this principle is generally difficult, we discuss several versions of the problem that admit tractable solutions. As experiments advance to larger and larger systems -- from genetic networks and chromatin structure to neural activity and animal behavior -- we present a unified framework for identifying the most important features in high-dimensional data.