Dynamical Implications of Hyperparameters in Reinforcement Learning

Reinforcement learning has been gaining popularity within the physics research community over past several years. Many of the algorithms require selection of hyperparameters, and it is unclear how these choices affect the learned policy. Often in practice, researchers choose hyperparameters for their simulations using brute force methods such as exhaustive grid search or heuristics without fully understanding how they may alter the effective reward structure and value function landscape. Here, we investigate the possibility of using a nonlinear dynamics approach to guide our selection and adaptive optimization of appropriate hyperparameters that are better aligned to achieve particular aims of the agent. We then extend our investigation to the setting of multi-agent reinforcement learning and game theory to see how the choice of agents hyperparameters affect the interplay between each other in competitive, cooperative, and mixed settings.