Local Diffusion Models and Phases of Data Distributions

Diffusion models are powerful generative frameworks but typically learn global score functions that are costly to train and sample. We introduce a “phases of data distributions” perspective that clarifies when local denoisers are valid. We define two distributions to be in the same phase if they are connected by spatially local operations, and show that the reverse denoising trajectory has an early trivial phase and a late data phase, separated by a narrow transition where local denoisers necessarily fail. We prove an information-theoretic bound—via conditional mutual information—diagnosing when local denoising is possible, and we validate this with experiments on MNIST that locate the phase transition and track the breakdown of small receptive-field denoisers. These results imply more efficient diffusion architectures: use small local networks away from the transition and reserve global networks for the short critical interval. The phase picture offers new tools for understanding generative AI and for designing physics-inspired neural networks.