Topological edge states in correlated honeycomb materials with strong spin-orbit coupling

We study the topological nature of single layers of correlated honeycomb materials $\alpha$-RuCl$_3$ and A$_2$IrO$_3$ (A=Li, Na) with strong spin-orbit coupling. An effective tight-binding model based on first principles band structure calculations including Hubbard interaction and spin-orbit coupling is derived. Two pairs of propagating edge modes centered at the zone center and zone boundary are found when their one-dimensional boundary forms a zig-zag shape, while the bulk has a gap with trivial time-reversal Z$_2$ invariants. The effects of strong electronic interactions and doping on the edge modes in these Mott insulators are discussed. We further suggest a heterostructure of $\alpha$-RuCl$_3$/IrCl$_3$ to search for the proposed topological Mott phase.