Benchmarking Transformer Neural Quantum States of Lattice Models

Neural quantum states have emerged as a powerful ansatz for variational quantum Monte Carlo studies of strongly-correlated systems. Here, we utilize autoregressive transformer neural network models for representing the ground state of small transverse-field Ising and Fermi-Hubbard [1] systems and measure their success using exact methods. We leverage the attention mechanisms and positional encodings, as well as other physically-inspired techniques, such as ramping of the Hamiltonian parameters during the training, to improve the performance of the transformer and benchmark our results against those from state-of-the-art variational methods.

[1] Ibarra-García-Padilla et al., "Autoregressive neural quantum states of Fermi Hubbard models", Phys. Rev. Research 7, 013122 (2025).