Asynchrony-tolerant finite difference method for partial differential equations at extreme scales

Computer simulations have been an important tool in understanding a wide variety of multiscale problems in physics: from molecular to geophysical to astrophysical phenomena. Many of these phenomena are modeled with partial differential equations that are often complex and nonlinear in nature, and thus demand massive computations. With increasing degree of parallelism in today's supercomputers, simulations are routinely performed on hundreds of thousands of processing elements (PEs) on Petascale machines. At this scale, communication between PEs take substantial amount of time during which PEs remain idle, leading to unused compute cycles and poor scalability. In this work, we propose a novel asynchronous method based on commonly used finite-difference schemes, where computations are carried out independent of the status of communication between PEs. We show that, while current schemes are stable and consistent under asynchrony, their accuracy is significantly affected. We derive new schemes that are tolerant to asynchrony due to slow communications relative to computations and maintain accuracy. We will also show results from numerical experiments to demonstrate the scalability of the method. These numerical schemes may provide a viable path towards true Exascale simulations.