Deep Reinforcement Learning for Autonomous Navigation in Complex Flows

In turbulent environments, navigating accurately and efficiently poses significant challenges.

This is especially the case if the only information that can be accessed are the local velocity

gradients. This is the sort of challenging task that a plankton is faced with while drifting in the

oceans, or that of a GPS-denied autonomous underwater vehicle (AUV).

Reinforcement learning has become a popular tool to solve navigation problems

with local flow information. So far, these studies are mostly proofs-of-concept. They

demonstrate that better-than-naive strategies can be learnt purely from experience.

But the next challenge for reinforcement learning is to discover truly smart strategies

that exploit the complex features of the underlying turbulent flow. This requires the

use of state-of-the-art reinforcement learning methods.

Here we explore this path to address the problem of directional navigation, we demonstrate

that DRL consistently matches or beat the performance of approximately optimal strategies

derived analytically. We also discuss the importance of using deep neural networks as opposed

to tabular policies. Overall, this study sheds light on the promising applications of reinforcement

learning in tackling directional navigation problems in turbulent environments.