Unsupervised Reinforcement Learning of ALE Mesh Management Strategies for Hohlraum Simulations in HYDRA

We report on our implementation of reinforcement learning (RL) algorithms in PyTorch to learn and automate mesh management strategies of hohlraum simulations in HYDRA. We define regions of the simulation mesh and extract a set of features for each that contain information about the quantity and degree of distortion and irregularity of nodes within the region. This feature set (and its time history) comprise the state of the system at a given time step. Based on a reward function defined to favor minimal intervention but enable the simulation to continue (i.e. not crash), the RL algorithm predicts an action on that region, e.g. relax the mesh, freeze it, or do nothing. The simulation is advanced another time step with this action implemented, and the process is repeated. In this way training episodes are run and recorded for later replay and training. The trained net can then be used as an inference engine to make mesh relaxation decisions in the regions as a simulation runs.