Mitigating heavy tails in error distributions of MLIPs by optimizing training strategies

Machine-learning interatomic potentials (MLIPs) have become ubiquitous in materials simulations owing to their excellent scalability with system size and ability to achieve chemical accuracy. Despite these successes, instabilities in molecular dynamics (MD) simulations have been reported, including unphysical abrupt melting [1], dissociation [2], or segregation [3]. These failures are triggered by large errors residing in the heavy tails of error distributions [4].

To identify conditions that mitigate these heavy tails, we trained ensembles of NequIP [5] MLIPs for silicon at 300 K, systematically varying model complexity, training dataset size, and the broadness of potential energy surface (PES) sampling for training. The joint distributions of estimated errors (computed via ensemble variance) and actual prediction errors on the test set not only reveal the strategy to realize the optimal training conditions but also deepen the understanding of the origin of the heavy-tailed error distributions. Models with insufficient complexity exhibited underfitting, characterized by weak correlations between estimated and actual errors, indicating overconfidence in predictions regardless of their actual accuracy. In contrast, optimally fitted models displayed strong correlations between estimated and actual errors. Models with excessive complexity and small training datasets often showed overfitting, marked by error distributions with extremely heavy tails. Furthermore, sampling training data from a broader PES (training on 1200 K, testing on 300 K) halved the extent of the heavy tails compared to the tight sampling of training data (training and testing on 300 K), even under optimal model complexity and dataset size.