Statistical analysis of neural network structures for the Iris dataset

At the core of Artificial Intelligence, artificial neural networks possess many interesting and intriguing features that remain poorly understood. For their detailed investigation, we analyze a feedforward neural network trained on the Iris dataset through the lens of statistical physics. Neural networks often exhibit phenomena reminiscent of phase transitions, and here we probe this connection by studying the learning dynamics, the geometry of the loss landscape, and the density of states in hyperparameter space. The Iris dataset is selected as a model system: it is small enough to allow systematic exploration, yet sufficiently complex to form regions in phase space separated by phase boundaries. This controlled setting enables us to examine how neural network training can be compared to complex optimization dynamics in physical systems. By mapping the structure of the network onto statistical observables such as the density of states and the topology of the loss landscape, we identify signatures of phase transitions by means of microcanonical statistical analysis. This approach provides new insights into how concepts from physics can deepen our understanding of the emergent properties of learning in artificial neural networks.