Tuning the Extended Heisenberg-Kitaev Model to α-RuCl$_3$: Stabilizing Zigzag Order and Quantifying Proximity to the Fractionalized Quantum Spin Liquid

Quantum Spin Liquids (QSLs) are exotic, magnetically disordered states with fractionalized excitations. While the idealized Kitaev model predicts a QSL, the candidate material α-RuCl$_3$ exhibits zigzag magnetic order, demanding an extended theoretical framework. We study the extended Heisenberg-Kitaev (HK) model, incorporating J2, third-neighbor J3, and the Γ exchange terms. Parameters are specifically tuned to α-RuCl$_3$ to derive a material-specific phase diagram. We use numerical exact diagonalization (ED) to determine the stability of the Kitaev QSL against competing magnetic phases. Our results show that J3 is a critical control parameter, rapidly destabilizing the QSL and stabilizing the observed zigzag order. The Γ term is essential for quantitative agreement with measured anisotropy and the low-energy continuum observed in inelastic neutron scattering. The resultant HKΓJ3 framework quantifies α-RuCl$_3$'s proximity to the QSL state. This guides experimental strategies (e.g., pressure or doping) needed to suppress magnetic order and realize the elusive fractionalized phase.