Eigenvalue Distribution of Weight Matrices and Convolutional Network Performance

In recent studies, convolutional neural networks have became a ubiquitous tool for computer vision this has lead to an interest in maximizing its performance. We analyzed eigenvalue distribution of weight matrices using Random Matrix Theory, comparing them to the Marchenko–Pastur distribution. We showed that in lazy learning regimes, where parameter updates remain close to initialization, weight spectra exhibit minimal change and coincide with RMT predictions, while in rich learning regimes, where parameters evolve significantly, larger deviations appear and coincide with higher accuracy. Our result suggests that spectral deviations from Random Matrix Theory prediction is a quantitative indicator of learning regime and model expressivity.