Reinforcement Learning and the Scientific Method

Model-based reinforcement learning allows an agent to learn how their environment behaves around them, allowing for an optimal solution to be planned preemptively. In past reports, model-based reinforcement learning has been found to sample an environment more efficiently than model-free reinforcement learning, in which the agent is learning the best action to take given any one state.
In this work, we present an alternative to model-based reinforcement learning in which exploring and updating the model are separate actions. As a result, the agent must act in an environment when there is uncertainty in the state and learn to take detailed observations accordingly to update the model. To achieve success, the agent needs to extrapolate from the known model, conduct a test in order to obtain new data, and then base future decisions on those results. This is a well-known process: the scientific method. We then show how our newly developed method can be used to learn the dynamics of physics-based models and exploit the knowledge gained to achieve a given objective with a measurable confidence.