Scaling the Search for Fault Tolerant State-Preparation Circuits with Reinforcement Learning

Quantum computing faces a critical challenge: balancing the need for shallow circuit depth—essential for fault tolerance—with the high accuracy demanded by complex tasks such as quantum chemistry and error correction, which often require deeper circuits. We address this trade-off by introducing a novel reinforcement learning framework with an incremental training procedure. This approach begins with an arbitrary, deep state preparation circuit and incrementally transforms it into a logically equivalent circuit with significantly reduced depth and gate count, drawing inspiration from quantum architecture search techniques. Preliminary results demonstrate, for example, significant reductions in two-qubit gate counts for surface code state preparation circuits. Our method scales beyond existing techniques by improving learnability and performance on large, previously unseen circuit instances.