The indirect exchange interaction between magnetic impurities located in the bulk of a two-dimensional topological insulator decays exponentially with the distance. The indirect exchange interaction for magnetic impurities mediated by the helical states at the edge of the topological insulator demonstrates behavior which is typical for the Ruderman-Kittel-Kasuya-Yosida interaction in a one-dimensional metal. We have shown that interference between the bulk and the edge states in the two-dimensional topological insulator results in existence of an unusual contribution to the indirect exchange interaction which, on the one hand, decays exponentially with a distance at the length scale controlled by the Fermi energy of the edge states and, on the other hand, oscillates with distance along the helical edge with the period determined by the Fermi wavelength. We found that this interference contribution to the indirect exchange interaction becomes dominant for such configurations of two magnetic impurities that one of them is situated close to the helical edge whereas the other one is located far away in the bulk.

• Received 26 December 2016
• Revised 3 March 2017

DOI:https://doi.org/10.1103/PhysRevB.95.115430