Towards microscopic control of the magnetic exchange coupling at the surface of a topological insulator

Magnetically doped topological insulators may produce novel states of electronic matter, where for instance the quantum anomalous Hall effect state can be realized. Pivotal to this goal is a microscopic control over the magnetic state, defined by the local electronic structure of the dopants and their interactions. We report on the magnetic coupling among Mn or Co atoms adsorbed on the surface of the topological insulator Bi2Te3. Our findings uncover the mechanisms of the exchange coupling between magnetic atoms coupled to the topological surface state in strong topological insulators. The combination of x-ray magnetic circular dichroism and ab initio calculations reveals that the sign of the magnetic coupling at short adatom–adatom distances is opposite for Mn with respect to Co. For both elements, the magnetic exchange reverses its sign at a critical distance between magnetic adatoms, as a result of the interplay between superexchange, double exchange and Ruderman–Kittel–Kasuya–Yoshida interactions.


Introduction
Topological insulators (TIs) define a novel state of electronic matter, characterized by the coexistence of a bulk insulating gap and a non-degenerate metallic surface band. The interplay between TIs and magnetism is of great interest for future spintronics and low power devices [1][2][3][4]. In particular the discovery of the quantum anomalous Hall effect (QAHE) [5] led to intense research activities in magnetically doped TIs. To date, the observation of the QAHE was only successful at low temperatures with bulk-doped samples [6]. However, the mechanism for a robust realization of the QAHE phase in magnetically doped TIs is still unclear, despite numerous research efforts on magnetically doped topological insulators .
In this context, surface doping of TIs with magnetic atoms is especially interesting as a playground for the control of the magnetic state. Surface magnetic atoms enable the microscopic control over the impurities and are strongly coupled to the topological surface state, which potentially enhances long-range coupling due to the Rudermann-Kittel-Kasuya-Yoshida (RKKY) interactions [32,33]. In fact, the strict surface localization of the topological surface state causes the interaction to be longer ranged than in situations where the RKKY interaction is concurrently mediated by three-dimensional states. The topological insulator Bi 2 Te 3 shows furthermore a particularly strong hexagonal warping of its topological surface state [34]. This causes a focusing effect [35] which can be exploited to enhance the interaction strength at large distances in certain crystallographic directions. Moreover, avoiding doping of the bulk potentially facilitates the realization of bulk insulating samples, which is one of the major challenges TI materials face on their way towards technological application. So far, studies of surface doped TIs with transition metals mainly focused on the geometry, the magnetic moments and magnetocrystalline anisotropy of individual magnetic adatoms [8,[22][23][24][25][29][30][31]. The present paper describes a combined experimental and theoretical study of the concentration-dependent exchange coupling mechanism towards a better understanding of the physical phenomena combining magnetism and topologically nontrivial matter potentially enabling the microscopic control over these phenomena.
We present evidence of significant magnetic exchange coupling between isolated Mn or Co atoms adsorbed on the surface of Bi 2 Te 3 . Our investigations, based on x-ray magnetic circular dichroism (XMCD) and density functional theory (DFT), show that the surface Dirac electrons of the topological insulator mediate magnetic exchange interactions, mainly of the RKKY type, of similar strength but of opposite sign for Mn and Co. This results, at dilute coverages in the range of » n 0.01 ML, in a ferromagnetic or antiferromagnetic coupling, respectively. Our analysis reveals the fundamental role of back-scattering of spin-momentum-locked electrons off magnetic impurities in the amplitude of the long-range interaction. On the other hand, at larger coverages and thus shorter 3d-3d distances, the magnetic exchange on the surface results from the competition between antiferromagnetic superexchange, ferromagnetic double-exchange and RKKY-type interactions, resulting in ferromagnetic coupling for Co atoms and antiferromagnetic coupling for Mn atoms.
Our combination of experimental and theoretical findings reveals that microscopic control over competing contributions to the exchange interactions can be achieved through appropriate band structure and impurity engineering. In particular we show that control over the Fermi level, the concentration and type of the magnetic impurities deposited on the surface and their spatial arrangement is a suitable way to control the magnetic coupling strength in magnetically doped topological insulators.

Results and discussion
2.1. Strength and distance dependence of the magnetic exchange We start the analysis by discussing our ab initio calculations of the Mn and Co magnetic moments and exchange coupling. We employed the local spin density approximation to density functional theory as implemented in the Korringa-Kohn-Rostoker (KKR) Green function method for embedded impurities in surfaces. Details on the calculations can be found in the Methods section and in supplementary note 1, which is available in the supporting information online at stacks.iop.org/JPMATER/1/015002/mmedia.
As we elaborate below, at short distances there is a competition between ferromagnetic and antiferromagnetic contributions while, at long distances, a RKKY type of interaction takes over, that for higher Fermi levels is highly directional because of the anisotropic form of the Fermi surface. Figures 1(a)-(c) depict the distance dependent and spatially anisotropic exchange interaction energy for Mn and Co, respectively. The interaction energy is expressed as the difference ΔE=E AFM −E FM between the antiferromagnetic (AFM) and ferromagnetic (FM) configuration (ΔE>0 implies ferromagnetic and ΔE<0 antiferromagnetic coupling), with the single-ion moments taken in the z-direction (out-of-plane). The latter assumption is justified, because the single-ion anisotropy is out-of-plane and sizeable (see figure S2 of the supporting information), exceeding the antisymmetric exchange after the 2nd nearest neighbor, and fixing the spins in the z-direction. The exchange interaction energy is shown in figure 1(a) from the 2nd nearest neighbor on as a function of distance in the GK direction (x-direction and equivalent ones by 60°rotations). In other directions, and in particular along GM (y-direction), the interaction strength is found to decay faster with distance after the 3rd nearest neighbor, which is seen in the maps of the exchange interaction energies shown in figures 1(b), (c) for Mn and Co, respectively.
We find that the short-distance Mn interaction is AFM, while Co is FM, in line with conventional wisdom expected from bulk materials. This can be understood from the density of states, shown in figure 1(e). Evidently, Mn possesses a half-filled d-shell while the Co minority-spin resonances are bisected by the Fermi level. The former property is known to lead to a level repulsion between occupied and unoccupied d resonances of nearby adatoms, promoting AFM coupling; in the latter case (Co), the AFM mechanism is still present, however, it is weaker than the FM double-exchange mechanism promoted by a broadening of the resonance. This analysis is in the spirit of the Alexander-Anderson model [37] and along the lines of previous density-functional-based studies of transition element defects in metals [38,39].
At larger distances, relevant for low surface transition-metal coverages, an RKKY oscillatory behavior sets in (see inset of figure 1(a) and figure S4 of the supporting information). We find that the oscillation periods in the GM and GK directions are compatible with the back-scattering vectors q 1 and q 2 , respectively, on the Fermi surface, shown in figure 1(d). A third vector q 3 in the GM-direction, connecting the Fermi surface snowflake tip-to-tip, is expected to play only a minor role because of the large curvature of the tip (flat segments of the Fermi surface produce longer-reaching coherent waves and thus longer-reaching interactions [35]).
The short-range mechanism persists up to intermediate distances of x≈22 Å (5th neighbor in the x-direction) and competes with the RKKY mechanism, as we find especially for Co. This is reckoned by the fact that the oscillations of the z-component of the magnetization m z induced on the surface state by a single adatom (shown in figure 1(f) for Mn and Co) change sign at different values of x as compared to the exchange interaction strength.
An additional effect that we must account for is an upward shift of the Fermi level by doping. This is a result of the competition between the tendency towards local charge neutrality, whence the resonance tends to be pinned with respect to E F , and the tendency of the resonance to be pinned at its original position because of the form of the atomic potential. The extra charge accompanying the Fermi level shift comes of course from the additional dopants. This has been experimentally observed, for example in [40], and has consequences on the sign and strength of the interaction, as we find in our calculations. Reference [40] reports an upward Fermi level shift of ΔE F =100 meV (200 meV) upon surface transition-metal doping of n=0.01 ML (0.02 ML). Here, we used the Fermi level as a parameter and performed self-consistent calculations for a range −20 meVΔE F 425 meV, that is used to model the coverage dependent shift of the Fermi level (note that for ΔE F =100 meV, the Fermi level has already entered the bulk conduction band). The implications that we find in the adatoms electronic structure and in the exchange are shown in figure 2. The minority-spin resonance is only partially (i.e., by less than ΔE F ) dragged to higher energies along with the Fermi level. The resonance of Mn becomes partly filled, while that of Co becomes progressively more filled (see figures 2(a), (b)). Consequently, the short-range Mn-Mn interaction obtains a sizeable FM component, which competes with the existing AFM component. It also strongly reduces the magnitude of the 2nd neighbor exchange interaction strength (at 9 Å), and eventually it changes the sign of the 3rd neighbor interaction (at 13.5 Å, see figure 2(c)). For Co, the ferromagnetic interaction strength at short distance drops as the Fermi level cuts the resonance only through its tail. As exemplified in figure 2(b) for Co, at a Fermi level shift of +425 meV, the lower peak of the minority-spin resonance is fully occupied, while the higher peak is bisected by the Fermi energy. This higher peak is of d z 2 character (see figure S5 of the supporting information), oriented out-of-plane, and thus contributes only little to the double-exchange interaction. Additionally the change of the Fermi level influences the Fermi surface (see figure S6 of the supporting information) and this has an impact on the dominant wavelength dictating the oscillatory behavior of the RKKY interaction. To summarize, higher doping causes the short range Mn-Mn interaction to become less antiferromagnetic, and the short range Co-Co interaction to become less ferromagnetic. Figures 3(a), (b) display the coverage dependence of the L 3 x-ray absorption (XAS) spectra and the corresponding XMCD, respectively, for Mn deposited at temperatures T 10 K on the (111) surface of Bi 2 Te 3 (see figure S7(a) and S8 of the supporting information for the as-measured and background-subtracted full L 2,3 -edges XAS and XMCD, respectively). The data shown refer to measurements at temperatures T4.3 K and magnetic field B=6 T and are recorded at either normal (θ=0°) or grazing (θ=70°) incidence (see Methods for details about the XMCD experiments). The main absorption peak at the L 3 edge of the Mn XAS spectra is at a lower energy in Mn/Bi 2 Te 3 than in MnO (a reference for a d 5 configuration), as shown in figure 3(c). Thus a higher occupation of the d-shell is expected for Mn/Bi 2 Te 3 . Indeed, our DFT calculations predict that the Mn hybridization with Bi and Te leads to a configuration close to d 6 . The spectral shape is independent of coverage, suggesting that the electronic configuration is stable for all studied Mn concentrations. Unlike the case of Mn doped into the insulating bulk of Bi 2 Te 3 [11], atomic multiplet structures are barely visible in the present case, consistent with the metallic environment of the surface of the topological insulator. On the other hand, a slight anisotropy is observed in the XAS spectral shape between normal and grazing incidence, with a low energy shoulder visible below the L 3 edge main peak only at grazing incidence, as shown by the arrow in figure 3(a). This anisotropy likely results from the trigonal crystal field experienced by the Mn atoms in their hollow adsorption site within the surface Te layer [40].

Electronic and magnetic configuration of Mn and Co on Bi 2 Te 3
The XMCD is larger, at all coverages, when the magnetic field is applied perpendicular to the sample surface 13 , indicating a slight out-of-plane magnetic anisotropy. Moreover, as shown in figure 3(d), the magnitude of the XMCD along the easy-axis decreases with increasing Mn coverage, especially for coverages n>0.03 ML. Figures 4(a), (b) display the coverage dependence of the L 3 XAS and the corresponding XMCD, respectively, for Co deposited on the (111) surface of Bi 2 Te 3 , under identical conditions to those of the data described in figure 3 (see figure S7(b) and S9 of the supporting information for the as-measured and background-subtracted full L 2,3 -edges XAS and XMCD, respectively). At the lowest coverage, as shown in figure 4(c), the XAS main feature at the L 3 edge is located at an energy of about 1.5 eV lower than the center of mass of the XAS spectrum of CoO, which is a reference for the d 7 configuration. This suggests a higher occupation of the d-shell in the case of Co/Bi 2 Te 3 . Indeed, our DFT calculations predict a configuration close to d 8 . The main L 3 -edge feature is composed of two peaks (see arrows in figure 4(a)), of which the higher energy one (at 777.0 eV) is more intense at normal incidence while the lower energy one (at 776.7 eV) is more pronounced at grazing incidence. This anisotropy is expected due to the trigonal crystal field experienced by the Co atoms which adsorb in hollow sites of the Te layer [40].
Contrary to the case of Mn, the spectral shape depends on coverage: the broad shoulder at an energy around 778 eV gains relative intensity with respect to the main L 3 -edge features, starting at coverages n≈0.01 ML,  13 Note that the magnetic field is always applied in the x-ray propagation direction, and thus it is applied perpendicular to the sample surface when the XAS is recorded at normal incidence.
where dimer formation starts. This suggests that, in the case of Co on Bi 2 Te 3 , single Co atoms are characterized by XAS spectral features which are well separated from those of dimers and larger clusters.
At coverages n0.014 ML the XMCD is much larger at normal incidence, indicating a strong out-of-plane magnetic anisotropy. At higher coverage, the magnetic anisotropy is reduced. In parallel, as shown in figure 4(d), the magnitude of the XMCD along the easy-axis decreases with increasing Co coverage with, moreover, a progressive transfer of spectral weight towards higher energies (see inset to figure 4(d)).
Quantitative information about the magnetic moments carried by Mn and Co adatoms on the Bi 2 Te 3 (111) surface can be extracted by applying the magneto-optical sum rules [41,42]  S ,ef f for Mn and Co assuming occupations of the d-shell n d =5.9 and 8.2, respectively, as determined by DFT. In order to include the jjmixing between the p 2 3 2 and p 2 1 2 core levels (particularly large for Mn), a correction factor has been applied to the spin sum-rule (1.47 for Mn and 1.1 for Co), as previously discussed [43,44]. Moreover, the magnetic moments obtained from the sum rules have been rescaled to take into account the incomplete magnetization even at the highest magnetic field (which is especially important for Co), as determined from the magnetization cycles which will be discussed in the following section.
At the most dilute coverage, Mn has a spin moment close to 4.4(3) μ B , slightly above the maximum expected for a d 6 configuration. Atomic multiplet calculations performed in C 3v symmetry (not shown) indicate that m D may be of the order of up to 20% of m S , so that we expect a spin moment close to the atomic value, m S ≈3.8-4.0 μ B . The orbital moment m L =0.29(5) μ B is small and parallel to the spin moment, thus confirming that the d shell is more than half filled. As expected from the coverage dependence of the XMCD shown in figure 3(d), the total magnetic moment m S +m D +m L decreases slowly with increasing coverage for n0.03 ML. In this range, the decrease of the spin moment m S +m D is partially compensated by an increase of the orbital moment m L . Thus the ratio between orbital and effective spin moment increases by about a factor of two for 0.005n0.03 ML. At higher coverages, on the other hand, this ratio decreases rapidly as a consequence of the strong decrease of the orbital moment.
The case of Co is similar to that of Mn. The spin plus dipolar moment at the lowest coverage has a value close to m S +m D =2.3(4) μ B , higher than expected for the pure spin moment m S of the d 8 configuration, suggesting an important contribution of the spin dipolar moment. Again we calculate that m D may be of the order of up to 20% of m S , and thus we expect a spin moment close to the atomic value (as for Mn), m S ≈1.8-2.0 μ B . The orbital moment is large, with a value of m L =1.0(2) μ B . Despite the fact that both the spin and the orbital magnetic moments decrease with increasing coverage, their ratio is independent of coverage and at the lowest coverage has a value of m L /(m S +m D )=0.4(3), consistent with previous results [45].
The experimental findings of figures 5(a), (b) can be interpreted by comparison to our density-functional calculations of the doping dependence (i.e., of the dependence on ΔE F ) of the spin and orbital moments shown in figure 6. The measured adatom spin moment showed a drop for increasing coverage for both Mn and Co (top panels of figure 5). Two effects may be at play: (i) the progressive formation of larger antiferromagnetic clusters, and (ii) the upward shift of the Fermi level. For Mn, the upward shift of the Fermi level with increasing coverage leads to a calculated spin moment reduction by only about 10% in the ΔE F range studied (see figure 6(a)). Moreover, the drop of the spin moment is counterbalanced by an increase of the orbital moment, leading to an increased orbital-to-spin moment ratio, as observed experimentally. However, we observe a much faster decrease of the measured spin moment than of the calculated one. Thus we conclude that for Mn the observed decrease of the spin moment is mainly related to the formation of small AFM clusters. The dominant cluster type in the studied coverage range are dimers, and in a triangular lattice we expect their number to increase from about 6% for n=0.01 ML to about 17% for n=0.04 ML [46]. For Co the spin moment drops much more severely than for Mn, as shown in figure 5(b), top panel. Here, importantly, the upward shift of the Fermi level with increasing coverage leads to a significant reduction of the spin moment from 1.2 μ B to 0.7 μ B at 425 meV shift (see figure 6(b)), at a similar rate as the decrease of the experimental spin moment. It is thus the cooperation of this effect with the formation of AFM clusters that causes the enhanced drop shown in the top panel of figure 5(b) for the case of Co.

Coverage dependence of the effective saturation magnetization
Complementary information about the magnetization of Mn and Co adatoms on the surface of Bi 2 Te 3 , as well as on their magnetic anisotropy, was obtained experimentally from the magnetization cycles recorded by following the magnetic field dependence of the main XMCD peak at the L 3 edge of each transition-metal. Contrary to the sum-rules, which give a magnetic moment averaged over all 3d atoms contributing to the XAS, the magnetization cycles carry information about the magnetization of the 3d sites contributing to a specific spectral feature of the XMCD, and thus are not influenced, for example, by antiferromagnetic clusters, as long as these do not contribute to the XMCD. We therefore consider that the magnetization cycles reflect mostly the magnetic behavior of Mn (Co) monomers at all coverages considered in this study, i.e. , up to n=0.08 (0.04) ML. Moreover, while the sum-rules give the average magnetic moment of the atoms at a given magnetic field value, the field dependence of the magnetization (i.e., the magnetic susceptibility) is sensitive to the collective behavior of an ensemble of magnetically interacting spin centers, and thus contains information not only about the magnetic polarization of the nearest neighbors, but also and most importantly about ferromagnetic or antiferromagnetic correlations among 3d atoms. Figure 7 shows the coverage dependence of the magnetization cycles recorded for both Mn and Co on the (111) surface of Bi 2 Te 3 . The out-of-plane anisotropy is confirmed for both Mn and Co by the steeper field dependence of the low-field magnetization in the normal incidence direction. We did not observe remanent magnetization for any of the measured coverages, thus we can rule out the presence of long range ferromagnetic order. This justifies analyzing the magnetization cycles within a semi-classical super-paramagnetic model as previously used, for example, for Co adatoms and clusters on the Pt(111) surface [47]. Above the blocking temperature, this model qualitatively captures the magnetization of a collection of magnetically coupled atoms that are within each other's reach in terms of the correlation length. The effective magnetization can be expressed in the form:  M 0 eff is the modulus of the effective magnetic moment, K is the uniaxial magnetic anisotropy constant, B is the magnetic field, Θ and Φ are the polar and azimuthal coordinate of the magnetic moment, and Θ 0 is the polar angle of the easy-axis of the magnetization. Here, we should note that M 0 eff can be larger (smaller) than the single-ion moment because of short-range ferromagnetic (antiferromagnetic) correlations. This is in analogy with the effective magnetization of the Curie-Weiss law, that can well exceed the ground-state value in case of short range ferromagnetic order above the Curie temperature [48].
By fitting simultaneously the magnetization cycles at normal and grazing incidence for each 3d element we can thus extract the coverage dependence of both the effective magnetization M 0 eff , shown in figure 8 (as well as in figure S3 of the supporting information), and the magnetic anisotropy constant K (see supplementary note 2). For comparison, green horizontal lines indicate the total magnetic moment per atom, as extracted by applying the magneto-optical sum rules to the XAS and XMCD spectra recorded at the lowest coverage. The top axis in each panel shows the average distance between adatoms, corresponding to the measured coverage range within a statistical distribution [49] of 3d adatoms on the Bi 2 Te 3 (111) surface. In the case of Mn/Bi 2 M Te , 3 0 eff is already larger than the total magnetic moment at the lowest coverage, but it increases considerably in the range 0.01n0.03 ML, where it reaches values far beyond what can be expected for a single Mn atom, with a peak at n≈0.016 ML where M 0 eff reaches a value of 8.95 μ B . This is in stark contrast with the decrease of the magnitude of the XMCD and the consequent decrease of the total magnetic moment (see figure 5(a)) observed in the same coverage range. On the one hand, as discussed above, we ascribe the decrease of the total moment mainly to the progressive, statistically unavoidable with increasing coverage, formation of close pairs (dimers), that are strongly antiferromagnetic and neutralize the contribution of a fraction of the magnetic ions to the total moment itself. On the other hand, we ascribe the large values of M 0 eff to a collective ferromagnetic behavior of groups of lone-standing Mn atoms, that interact ferromagnetically (at the average inter-impurity distance) in the metallic environment of the surface state of the topological insulator. We note that at coverages up to about 0.02 ML, i.e., at distances larger than 20 Å we expect the interaction to be mediated by the surface state (the bulk conduction band is above the Fermi level in this coverage range [40]) and to be of the RKKY type. The ferromagnetic exchange strength and the magnetic anisotropy cannot overcome the effect of the thermal energy, Figure 7. Coverage dependence of the magnetization cycles for Mn (a) an Co (b) on the Bi 2 Te 3 (111) surface. Normal (grazing) incidence data are shown as darker (brighter) dots, while lines refer to the corresponding fits with the model outlined in the main text. The grazing incidence data corresponding to the coverages of 0.005 and 0.009 ML are taken with an incidence angle θ=55°, while all other grazing incidence data refer to θ=70°. thus we do not observe ferromagnetic order but rather superparamagnetism with no remanent magnetization. At larger coverages, the total magnetization decreases, suggesting a competition between ferromagnetic and antiferromagnetic correlations.
The case of Co is opposite to that of Mn. At low coverages (n0.03 ML), M 0 eff lies just below the total magnetic moment extracted by applying the magneto-optical sum rules to the XAS and XMCD of the lowest coverage (n=0.004 ML), suggesting weak antiferromagnetic correlations, whereas at higher coverages a tendency towards ferromagnetic correlations is observed.

Comparison of the collective magnetic state in experiment and simulation
Concerning the behavior of the effective magnetization as a function of increasing transition-metal coverage, figure 8(a) shows a peaked trend of M 0 eff for Mn, with a maximum at n≈0.016 ML, while figure 8(b) shows an initial hump for Co, with a local maximum at n≈0.009 ML, then a local minimum at n≈0.014 ML, and finally a rise at higher coverages. The peak of the effective Mn magnetization in figure 8(a) at the coverage n=0.016 ML is explained as follows. At coverages below and up to approximately n=0.016 ML, the average interaction is FM, because the average inter-adatom distance is beyond the first oscillatory node of the interaction, located at x≈20 Å (see figure 2(c)). Thus, FM correlations cause an enhancement of the effective magnetization. For n>0.016 ML, however, the average distance is reduced below the first oscillatory node (i.e., x<20 Å), and AFM correlations prevail, so that the effective magnetization drops.
The effects of distance dependence and Fermi level shift, occurring simultaneously in the experiment, are combined in the theoretical coverage dependence of the magnetic exchange energies shown in figure 9(a) for Mn-doping. Two models of the Fermi level shift with coverage (shown in figure 9(c)) are compared. Model A assumes a linear shift of the Fermi level with increasing coverage up to a maximum shift of 200 meV, after which a saturation is assumed. This is chosen to account for the fact that the density of states of the surface state is small within the bulk band gap, so that the response of the system to additional charge coming from the dopants is far stronger than for the case where the Fermi level has already entered the conduction band, where the additional charge with increasing doping only has a small effect on the resulting Fermi level. Model B, on the other hand, shows the coverage dependence of the exchange energy for a linearly increasing shift of the Fermi level up to 425 meV. We find that the peak of the exchange energy compares very well with the experimental data in the case of model A (see figures 8(a) and 9(a)). Around a coverage n=0.025 ML we find strong ferromagnetic coupling, which fits the observation of a strongly increased measured effective magnetization.
In the case of Co, both the strength and oscillations of the interaction change with increasing Fermi level shift. Figure 2(d) shows that the short-range interactions drop by 80%, while the first node of the oscillation (i.e., the turning point to AFM coupling) is first pulled towards shorter distances and then pushed outwards again. Thus we interpret the measured effective magnetization of figure 8(b) as follows. Starting from low coverages, as the concentration grows up to n=0.01 ML (i.e., for distances decreasing from about 40 to about 28.5 Å), the rise of the effective Co magnetization is caused by the increased tendency towards FM correlations arising from the subtle balance of the opposite effects of Fermi level shift ΔE F , favoring a FM alignment, and reduced interadatom distance x, favoring AFM alignment. Then, for 0.01<n<0.015 ML, both the increase of ΔE F and the decrease of x cooperate in strengthening the AFM interactions, with a consequent decrease of the effective magnetization. Finally, for n>0.015 ML (i.e., for x<20 Å), the AFM correlations are rapidly lost (see figure 2(d) at ΔE F >200 meV). Even though the short-range FM interaction strength is ever decreasing as the coverage rises, the denser adatom population counteracts this decrease, causing stronger FM correlations and a rise of the effective magnetization again.
The comparison with the theoretical coverage dependence of the magnetic exchange energies (figure 9(b)) further confirms that our theory qualitatively reproduces well the experimental data. In particular the decrease of the effective magnetization (seen for n≈0.015 ML in figure 8(b)) is well accounted for by the antiferromagnetic coupling shown in figure 9(b) for coverages 0.01n0.03 ML, while the sign change towards ferromagnetic coupling occurs at higher coverages.
The long-range interactions have a larger energy scale in Mn than in Co, as we see by comparing figures 9(a) and (b). It is known that, at long distances, the RKKY interaction follows the profile of the energy-integrated spin-density oscillations and, on topological insulator surfaces, takes approximately the form where R is the distance between the impurities, Φ is a phase shift, k is the crystal momentum at the Fermi level, and -

Conclusion
Our findings finally allow us to explain the crucial difference between Mn and Co, which has led to the observation of magnetically-induced backscattering only when Mn, but not Co, dopes the surface of Bi 2 Te 3 [40]. This happens because the synergistic effect of electron-doping by the magnetic adatoms and exchange interaction among them enhances, only in a restricted Mn coverage range, the ferromagnetic exchange mediated by the topological state, thus stabilizing a large effective magnetization. This, in turn, is one of the necessary conditions, together with the characteristic hexagonal warping of the constant energy contours of topological insulators, to achieve the transmission of spin information over long distances, as previously observed [35]. In the case of Co, on the other hand, the AFM exchange mechanisms dominating in the coverage range of predominantly individual adatoms lead to low effective moments, unable to open intense backscattering channels [40].
Our results show that magnetically doped topological insulator surfaces offer a rich playground for magnetic phenomena. Indeed, the magnetic moments and interaction sign strength can be tuned by changing the adatom type and concentration, but also by shifting the Fermi level by doping. Even though the Fermi level shift that we considered here was assumed to be in one-to-one correspondence with the transition-metal concentration, it can be further tuned by n-or p-doping with non-magnetic defects. In this respect, it should be possible to adjust the interactions to values that will allow wished-for magnetic states. Ferromagnetism, antiferromagnetism, or more complex configurations like skyrmionic states are pertinent examples of what tailored interactions may induce. If combined with the strong spatial anisotropy of the exchange interactions that comes along with the focusing effect in Bi 2 Te 3 , these effects may open new ways of engineering novel technology or designing strong ferromagnetic order which is a prerequisite for a robust observation of the QAHE towards its technological application.

Density functional theory calculations
For the density-functional calculations we employed the full-potential relativistic KKR Green function method for embedded defects on crystal surfaces [51]. Exchange and correlation effects were treated within the localspin-density approximation [52]. An angular momentum cutoff of ℓ max =3 was used. The surface of Bi 2 Te 3 was simulated taking a 6 quintuple layer thick film with the experimental lattice constant [53]. The charge density and potential perturbation around the defects were treated self-consistently up to the first shell of nearest neighbors, which is sufficient for our purposes, as tests have shown. The energy difference between the ferromagnetic and antiferromagnetic configuration was calculated within the approximation of the force theorem. The adatom positions were assumed to be in the threefold fcc hollow site in the Te atomic layer, in accordance with previous studies [29] without further structural relaxations. Although slight quantitative differences could be expected for the hcp hollow site in comparison with the fcc site, we expect the uncovered mechanisms and observed trends to not depend on the specific adatom position as long as the here assumed single-impurity approximation is fulfilled.

Sample preparation
The Bi 2 Te 3 single crystals were cleaved at room temperature at a pressure p<2 × 10 −9 mbar and then immediately cooled to liquid helium temperature. Mn and Co were evaporated in situ onto Bi 2 Te 3 (111) at T<10 K and p≈1 × 10 −10 mbar.

XMCD and XAS experiments
The major part of the XMCD experiments were carried out at the BOREAS beamline [54] of the ALBA synchrotron radiation facility. Additional experiments (Co coverages n=0.005 and 0.009 ML) were performed at the ID32 beamline [55] of the European Synchrotron Radiation Facility. The measurements were carried out in the total-electron-yield mode at temperatures in the range 2.5-6 K, and magnetic fields up to 6 T (ALBA) or 8 T (ESRF), applied parallel to the x-ray beam propagation direction. The average background subtracted Mn/Co L 2,3 -edge XAS [( + + -I I )/2, where I + and I − are the XAS spectra recorded with right and left circularly polarized x-rays, respectively], as shown in figures 3 and 4 of the manuscript, and S8 and S9 of the supporting information, is obtained by subtracting the XAS spectra of the bare Bi 2 Te 3 (111) crystals, taken prior to Mn/Co evaporation, from those of Mn/Bi 2 Te 3 (111) and Co/Bi 2 Te 3 (111) recorded under identical conditions (see figure S7(a), (b) of the supporting information), and then subtracting step functions at the two edges. The XMCD is then calculated as --+ I I . In order to compare XAS and XMCD of different coverages, these have been normalized to the corresponding integral of the XAS spectrum as recorded in the normal incidence geometry.
Atomic multiplet calculations were performed with the Xclaim code [56]. In order to simulate the trigonal crystal field expected at the 3d atom adsorption site on the (111) surface of Bi 2 Te 3 , we used the three Wybourne parameters B 20 , B 40 and B 43 .