First-order phase transition in atom-molecule quantum degenerate mixtures with coherent three-body recombination

We investigate the role of coherent atom-molecule interconversion, at quantum degeneracy, with competing Feshbach coupling and the recently observed coherent three-body recombination (cTBR) process. Based on the two-mode approximation, we map out the ground-state phase diagram featuring a structural modification from a conventional second-order (when Feshbach coupling dominates) to a first-order quantum phase transition (when cTBR prevails) for varying molecular detuning. This behavior originates from the emergence of a double-well structure in the effective free-energy landscape, giving rise to metastable molecular states. In the vicinity of the first-order transition, the system exhibits macroscopic atom-molecule superposition states associated with substantial entanglement. Our results establish cTBR as a powerful knob for engineering quantum phase transitions, their entanglement and associated ultracold chemical reactions in atom-molecule quantum gases.