Quantum phase transition in a strongly interacting 2D model of Majorana fermions

Abstract: We study a 2D strongly-interacting time-reversal-invariant system consisting of Majorana fermions on the square lattice. This model may be realized as an Abrikosov vortex lattice in the superconducting surface state of a strong topological insulator. From the mean-field calculations, we show that for strong interactions a time-reversal-symmetry breaking phase occurs with chiral edge states. We further investigate the quantum critical behavior at the phase transition point by renormalization group analysis.