Maximizing thermal efficiency of heat engines using neuroevolutionary strategies for reinforcement learning

Classic control problems such as Mountain Car [1] and Acrobot [2] are based on simple Newtonian physics. Both have been solved previously with reinforcement learning algorithms. Here, we show that reinforcement learning can also be used to solve classical problems in thermodynamics. Using a reinforcement learning method based on genetic algorithms, our software agent can learn to reproduce thermodynamic cycles without prior knowledge of physical laws. We have created a simulated learning environment which models a simple piston, where an agent can activate thermodynamic processes. With this method, we were able to optimize an artificial neural network based policy to maximize the thermal efficiency for several different cases. Depending on the actions available to the agent, different known cycles emerged, including the Carnot, Stirling, and Otto cycles. Importantly, we show an example of how reinforcement learning can be used to aid scientists in finding solutions to problems that have yet to be fully explored. In one of the heat engine environments, we introduced a non-adiabatic process which caused the engine to lose energy. In this case, the agent produced, what is to the best our knowledge, the best solution for the problem.
[1] A. W. Moore, 1990
[2] S. A. Bortoff, 1992