Free Energy–Based Reinforcement Learning Using Quantum Monte Carlo and Quantum Annealing

We investigate whether quantum or numerical sampling from select many-body systems can be used to improve upon classical methods in reinforcement learning. We introduce free energy–based reinforcement learning (FERL) as an application of such sampling methods. In our experiments, we focus on transverse field Ising spin Hamiltonians with layouts of qubits similar to that of deep Boltzmann machines (DBM), and use simulated quantum annealing (SQA) as a subroutine of a reinforcement learning framework. In the absence of a transverse field, the DBMs train more effectively than restricted Boltzmann machines with the same number of weights. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field. This further improves the DBM-based reinforcement learning method. To perform physical experiments, we propose a method for processing a quantum annealer’s measured qubit spin configurations in approximating the free energy of a QBM. We then apply this method to perform reinforcement learning on the grid-world problem using the D-Wave 2000Q quantum annealer. The experimental results show that our technique is a promising method for harnessing the power of quantum sampling in reinforcement learning tasks.