Deep Learning embedding layers for better prediction of atomic forces in solids

The evaluation of atomic forces and total energy is a key challenge for large-scale atomistic simulations of materials. In recent years, machine learning techniques are successfully used to predict potential energies and to derive the atomic forces through the energy gradient. The training data is usually produced by quantum calculations, typically Density Functional Theory (DFT). The direct prediction of atomic forces by deep learning (DL) models was recently demonstrated by us and other groups, it has the advantage of being local and slightly faster while still maintaining state of the art mean absolute error (MAE). A disadvantage is that the predicted forces might be non-conserving. Like models which predict the energy, direct force models should behave well under symmetry operations and permutation of atoms. Here, we show how the use of self-learned embedding layers help to achieve better models for direct prediction of atomic forces. We also examine some sophisticated loss models to assure that the forces are smooth and close to conserving. We demonstrate this by the calculation of phonons in several solids and by the analysis of force derivatives in systems where we move single atoms and compare the DL predicted force derivatives to the DFT results.