The Other Side of Entropy

Following its inception in the mid-19th century, our understanding of thermodynamic entropy has undergone many revisions, most notably through the development of microscopic descriptions by Boltzmann and Gibbs, which led to a deep understanding of equilibrium thermodynamics. The role of entropy has since moved beyond the traditional boundaries of equilibrium thermodynamics, towards problems for which the development of a statistical mechanical theory seems plausible but the a-priori probabilities of states are not known, making the definition and calculation of entropy-like quantities challenging. In this talk, I will discuss information theoretic ideas and methods that enable these computations. First, we will explore why universal data compression (Lempel Ziv coding) provides a good starting point for estimating entropy in and out of equilibrium. Then I will show through a simple argument how from the classical LZ bound we can derive a pattern matching estimator that readily generalizes to higher dimensions and that provides a tight bound on the entropy, overcoming the limitations of previous approaches. Finally, starting again from the simple LZ bound, I will show how we can obtain a new KL divergence estimator that outperforms existing methods, and how we used it to estimate local entropy production and to explore its relation to extractable work in active matter. I will illustrate these ideas by considering their applications in a variety of contexts: from colloidal systems, to absorbing-state models, to active matter, in simulations and in experiments. Throughout the talk, I will highlight challenges and promising future directions for these measurements.