Scaling the Force Calculations of the Real Space Pseudopotential DFT solver PARSEC on Haswell and KNL systems

The ability to compute atomic forces through quantum contributions rather than through simple pairwise potentials is one of the most compelling reasons materials scientists use Kohn-Sham pseudopotential density functional theory (DFT). PARSEC is an actively developed real space pseudopotential DFT solver that uses Fortran MPI+OpenMP parallelization. PARSEC provides atomic forces by self-consistently solving for the electronic structure and then summing local and nonlocal contributions. Through experimentation with PARSEC, we present why increasingly bulk synchronous processing and vectorization of the contributions is not enough to fully utilize current HPC hardware. We address this limitation through a demonstration of multithreaded communication approaches for local and nonlocal force computations on Intel Knights Landing supercomputers that yield feasible calculation times for systems of over 20,000 atoms.