Exotic Spin Liquid phases in Generalized Kitaev JKΓΓ' model on honeycomb lattice

Employing a gauge invariant Majorana fermion decomposition of spin, we derive a renormalized mean-field theory (RMFT) for the Generalized Kitaev JKΓΓ' model under magnetic fields on a honeycomb lattice. We allow for the most general mean field parameters permitted by the projected symmetry group. Our RMFT approach provides good results compared with other numerical methods such as DMRG and VMC on the same model, e.g., reproducing the 8-,14-,20-cone quantum spin-liquid (QSL) states in the KΓmodel and under fields. Therefore, we are confident that this RMFT approach provides an economic way to investigate the mean-field spinon band structure properties (Chern number etc.) and complex vortices structure in JKΓΓ' model under field in arbitrary directions.

By using our RMFT, we find several new stable QSL states which are not found by early VMC works. In particular, a new exotic 2-cone state with negative Wilson loop values and Chern number ν=±1 is found. In the presence of magnetic fields along ab plane, this 2-cone state holds opposite Chern number compared to the Kitaev spin liquid, and is thus a new state that is distinct from the Kitaev spin liquid. Furthermore, it displays non-Abelian statistics in weak magnetic fields when gap is opened, which provides a new perspective to examine experimental results