Toward a Physics of Deep Learning and Brains

Deep neural networks and brains may seem fundamentally different, yet they both learn through adaptive interactions among interconnected processing units. How far this microscopic resemblance extends to emergent computational principles remains largely unknown.

Here, we show that mathematical laws originally developed to describe cascades of neuronal activity in the cortex also govern avalanches in deep networks. Rooted in non-equilibrium statistical physics, our framework reveals that deep networks can approach a genuine critical phase transition. Although finite-size effects and strong input drive prevent them from reaching an exact critical point, they can be initialized in a quasicritical regime that still exhibits hallmark predictions like powerlaw statistics, scaling exponent relations, susceptibilty peaks and self-similar patterns of activity.

By training networks with different initial conditions, we demonstrate that heightened susceptibility serves as a robust predictor of learning performance, confirming a long-standing theoretical prediction that has been difficult to verify in biological systems. Finally, using finite-size scaling, we identify the universality classes governing deep networks, including Barkhausen noise and directed percolation.

Overall, our results show that both brains and deep networks obey shared macroscopic principles of collective computation, paving the way for the design of new architectures guided by susceptibility and universality.