An Information-Theoretic Order Parameter for Non-Equilibrium Systems

In non-equilibrium statistical physics, symmetry and free energy are difficult to define, preventing application of classical machinery for analysis of phase transitions. Can one define a ``universal'' order parameter that would be measurable from experimental data, would allow identification of an onset of a phase transition, and would be meaningful independently of the underlying systems dynamics? We suggest that predictive information, which is the mutual information between the sequence of the observed past states of a system and its future states, introduced by us in 2001,\footnote{W Bialek, I Nemenman, N Tishby. Neural Computation (2001),13,2409} may serve as such order parameter. We study this suggestion in the context of a model non-stationary Langevin process. We show analytically that the predictive information attains its maximum value at the phase transition, diverging logarithmically with the length of the observed past. We demonstrate that the speed of divergence is related to traditional critical exponents. Finally, we show how the onset of a phase transition can be found empirically from data, independently of its parameterization.