Surface alloy engineering in 2D trigonal lattice: giant Rashba spin splitting and two large topological gaps

We demonstrate that sp2 based trigonal lattice can exhibit giant Rashba splitting and two large topological gaps simultaneously. First, an effective tight binding model is developed to describe the Rashba spin–orbit coupling (SOC) on a real surface and give a topological phase diagram based on two independent SOC parameters. Second, based on density functional theory calculations, it is proposed that Au/Si(111)- 3 × 3 surface with 1/3 monolayer Bi coverage is a good material candidate to realize both giant Rashba splitting and two large topological gaps. These results would inspire great research interests for searching two-dimensional topological insulator and manipulating Rashba spin splitting through surface alloy engineering.


Introduction
Two-dimensional (2D) topological insulator (TI) [1], also known as quantum spin Hall effect [2,3], are characterized with insulating bulk states and gapless Dirac edge states. The robust Dirac edge states are protected against backscattering induced by non-magnetic impurities, enabling non-dissipative transport. Among all the 2D lattices, there are two Bravias systems which have attracted much attention. One is the square lattice where two quantum well materials that have been experimentally confirmed to be 2D TI in the past decade [3,4]. The other is the hexagonal lattice which derives three kinds of lattices: honeycomb, trigonal and Kagome lattice. All of these three lattices can host TI in the free-standing limit [5][6][7][8][9][10][11][12][13]. However, free-standing materials do not exist in nature. They must be either grown on the substrate or exfoliated from the bulk material, greatly hindering the experimental realization. Besides, they need to be placed or transferred onto a suitable substrate for measurement. Therefore, interaction between TIs and substrate should be taken into consideration.
Rashba spin-orbit coupling (SOC) [14] is an important effect for inversion symmetry breaking systems, such as non-centrosymmetric semiconductors and semiconductor heterojunctions. It can inter-converse a charge current to spin current without magnetism on the surface [15,16]. The parameter to measure the strength of Rashba SOC is the Rashba coefficient α R [14], and its typical value is 10 −2 eV Å on InGaAs/InAlAs inter-surface [17] and 10 −1 eV Å on Au(111) surface [18]. Since a large α R implies a higher charge-to-spin inter-conversion efficiency, the search for materials with large α R is an attractive topic in spintronics [19][20][21]. One promising candidate material is the semiconductor surface covered by metal adatom with large SOC, such as Bi/Si(111) [22], Tl/Si(111) [23] and Pb-Tl/Si(111) [24], with the typical α R of 1 eV Å.
In this work, we study the topological states in trigonal lattice with Rashba SOC. In general, Rashba SOC has a negative effect for the TIs in a honeycomb lattice grown on the substrate [5,25]. However, if the Rashba SOC is not so strong compared to the intrinsic SOC, the nontrivial TI phase can still be kept on the substrate, such as the Original content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence.
Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Bi-Cl/Si(111) [26], Bi/SiC(0001) [27] and Bi/SiC(111) [28]. How about the TIs in a trigonal lattice? To answer this question, a sp 2 tight binding model is constructed and a p z orbital mediated Rashba SOC is included to mimic the real surface. We found a special TI phase with two nontrivial gaps that are fairly robust to the Rashba SOC. Furthermore, based on density functional theory (DFT) calculations of an experimental sample of Au/Si (111)-3 3surface, we demonstrate that it will be a large gap 2D TI at 1/3 ML adsorption of Bi adatom. The stable trigonal lattice made of Bi-Au alloy realizes a band structure with a giant Rashba spin splitting α R of 1.58 eV Å and two nontrivial SOC gaps of 0.28 and 0.07 eV. Our results extend the 2D TIs to trigonal lattice and pave a new way to search it in surface alloy materials. The coexistence of 2D TI and giant Rashba spin splitting also provides an ideal platform to study the competition between intrinsic and Rashba SOC for effectively manipulating the topological states.

Tight binding model
As shown in figure 1(a), the (s, p x , p y ) basis is considered to be the 'minimal basis' to construct TIs and is also assumed to be the low-energy effective orbitals for semiconductor surface adsorbed by metal adatoms [12,28] where m, n are cell index, α, β are orbital index, ε α is onsite energy for orbital α and t mα,nβ is the hopping energy between α orbital in cell m and β orbital in cell n with 〈m, n〉 labeled the nearest-neighbor.  † c and  c are creation and annihilation operators. Written the Hamiltonian in reciprocal space and the hopping strength in the Slater-Koster form [29], we have:  where σ ss , σ pp and σ sp are σ-type hopping between s-s, p-p and s-p respectively, π pp is the π-type hopping between p-p.
In the (s, p x , p y ) basis, the onsite intrinsic SOC [30] can be written as: where λ I is the intrinsic SOC strength and τ j ( j=x, y, z) is the Pauli matrix for the spin degree of freedom. The Hamiltonian H 0 +H I conserves the inversion symmetry, so its spin-up and spin-down components are decoupled. However, a real surface will break such a symmetry and couple spin-up and spin-down together. In the (s, p x , p y ) basis (an ideal sp 2 basis), the onsite Rashba SOC between different orbitals is zero [31]. In order to include the Rashba SOC in our model, an additional p z orbital is used as [32]: where σ, σ′ and σ″ are spin index, λ′ and λ are coefficient for standard Rashba SOC and intrinsic SOC. Here, the including of p z orbital is physically reasonable, because the (s, p x , p y ) basis is not in an plane on the surface, as shown in figure 1(b). Therefore, the onsite Rashba SOC can be written as: where λ R is the Rashba SOC strength. Based on the above Hamiltonian, we found that it can realize a band structure similar to the Kagome lattice with two nontrivial gaps. The spinless band structure is shown in figure 1(c) with label 1. One can see two Dirac cones, located at Γ and K point, respectively.
These two Dirac cones are not protected by the SOC and band structures with three different sets of (λ I , λ R ) labeled as 2, 3 and 4 are shown in figure 1(c). One can see two SOC gaps (Δ 1 and Δ 2 ) are opened up and bands of 3 and 4 are splitting into Kramer's pair by Rashba SOC. Using (λ I , λ R ) as two independent parameters, two topological phase diagrams are illustrated in figures 1(d) and (e) for Δ 1 and Δ 2 respectively, reflecting the competition between the intrinsic and Rashba SOC. The two topological gaps can persist for a large Rashba SOC as long as the nontrivial gap does not close down [33], i.e. the TI phase can survive in a large Rashba SOC, which is very different to the honeycomb lattice studied before. Based on this finding, in the following part, we will use the DFT to design a surface system that can exhibit large spin splitting and two topological gaps simultaneously.

DFT calculation
Our DFT calculations are performed using Quantum Espresso code [34] with ONCV [35] norm-conserving pseudopotential based on Perdew et al [36] exchange-correlation functional and a plane-wave cutoff of 50 Ry. SOC is included through the relativistic approximation. The optimized bulk Si lattice constant is 5.45 Å. Si(111) surface is simulated by a slab structure with four Si bilayers and one reconstructed layer. A vacuum space of ∼20 Å is used to eliminate the inter-slab interaction. Hydrogen atoms are used to passivate the bottom layer Si dangling bonds. During structural relaxation, bottom Si bilayer and hydrogen atoms are fixed, and the other atoms are relaxed until the forces are smaller than 0.01 eV Å -1 .
The Au/Si(111)-3 3surface has been studied extensively. There are two well established models for it: twist trimer (TT) model and conjugate honeycomb-chained-trimer (CHCT) model [37], as shown in figures 2(a) and (b), respectively. According to Lee's calculation [37], TT model is slightly more stable by 0.01 eV than that of CHCT model and there is an energy barrier of 0.05 eV between these two models, making it unsuitable for practical applications. Later, Gruznev [38] found that a sub-monolayer adsorption of In adatom on Au-Si(111) surface can greatly eliminate the domains and yield a well-ordered homogeneous3 3 structure. Therefore, the sub-monolayer metal adsorption provides a feasible way to treat Au/Si(111)-3 3surface as a uniform substrate. Recently, using such a surface as the substrate, Chuang has shown that 2/3 monolayer (ML) Bi adatom can be grown on it [39]. Here we reduce the adsorption concentration to 1/ 3 ML (one Bi adatom per unit cell) to form the trigonal lattice.

Results and discussions
To find the lowest-energy structure with 1/3 ML adsorption of Bi adatoms, nine possible initial configurations (labeled as 1-9) are optimized based on the symmetry of TT and CHCT models, as shown in figures 2(a) and (b). After structure relaxation, five stable structures (labeled as I-V) are obtained, and more than half of the initial configurations are relaxed to the same lowest-energy structure (I), as shown in figure 2(c). The lowest-energy structure (I) has an adsorption energy of 3.40 eV/cell, which is 0.20 eV and 0.24 eV larger than the second (II) and third (III) lowest-energy structure respectively. Additionally, through the nudged elastic band method, we found that the energy barrier between I and II (III) structure is 0.38 eV (0.84 eV), as represented in figure 2(d). Such a huge energy barrier indicates the lowest-energy structure is very stable and the migration of Bi adatoms is hard. To further understand the stability of the lowest-energy structure, first-principles molecular dynamics calculations are performed for a 2×2 surpercell with NVT ensemble at 100 K, 300 K and 500 K, respectively, as shown in figures 3(a)-(c). The snap shots taken at 1000, 3000, 5000 ps show that the lowest-energy structure is still stable at 300 K. However, the Bi adatoms start to diffuse at 500 K. From the above analysis, one can see that the proposed 1/3 ML Bi can form ordered trigonal lattice on Au/Si(111)-3 3surface and the surface alloy structure is dynamically stable even to the room temperature, making it more feasible to be grown experimentally.
The lowest-energy structure of 1/3 ML Bi adsorbed Au/Si(111)-3 3surface is shown in figures 4(a) and (b). The Bi-Au alloy has a trigonal lattice pattern (space group P3) with the lattice constant of 6.67 Å. Bi-Au alloy is bonded strongly with the top Si atoms, forming a three-layer structure in the consequence of Bi, Au and Si. The vertical distance between Bi-Au and Au-Si is 1.37 Å and 0.86 Å respectively, which is within the characterized chemical bonding. Starting from this structure, we first calculate its band structures without SOC. As shown in figure 4(c), it is a metal without magnetism. Checking carefully around the Fermi-level, one can see there is a Kagome-shaped band structure with one Dirac (labeled as B1, B2) and one flat (labeled as B3) band. Such a three-band structure results in two degenerate points sitting at K (labeled as D1) and Γ (labeled as D2) point, reflecting the three-fold rotation symmetry respectively. The orbital-component analysis further shows that these three bands are mainly constructed from p orbital of Bi, and s and p orbital of surface Si-trimer, as shown in figure 5. The Dirac cones are not robust with respect to SOC. To check this point, the band structures with SOC are further calculated. 3surface at 100 K, 300 K and 500 K, respectively. Insets are snap shots taken at 1000, 3000 and 5000 ps.
As shown in figure 4(d), two significant features are observed by including SOC. First, the band degeneracies at K and Γ point are lifted, and two large indirect band gaps are opened. In addition, due to the strong SOC in Bi and Au atoms, the band structures are dramatically changed. The bottom gap (0.16 eV below Fermi-level) and top gap (0.29 eV above Fermi level) is Δ 1 =0.07 eV and Δ 2 =0.28 eV, respectively. Second, the top branch of the Dirac band (B2), sitting between the two SOC gaps, shows a large spin splitting, while the Kramer's degeneracy is conserved at Γ and M points. Such band splitting can be attributed to the Rashba effect induced by inversion asymmetry surface. Here, the B2 band is across the Fermi-level and totally separated from the other bands, so it provides an ideal Rashba material for designing spintronic devices. Following the Rashba-Bychkov model [14], we can estimate the Rashba spin splitting parameters as: momentum offset k 0 =0.14 Å −1 , Rashba energy E R =111 meV and Rashba coefficient α R =1.58 eV Å. These values are comparable to the largest ones that have been reported before [40][41][42], demonstrating a giant Rashba spin splitting effect. Given the giant Rashba spin splitting here, the topology of these two SOC gaps especially interests us, because our TB model implies that the two gaps are quite robust to Rashba SOC.
To check the topology of these two gaps, their Z 2 numbers are calculated directly by Z2Pack [43], which is used to track the evolution of Wannier charge centers for all the occupied bands. By setting the Fermi-level into these two SOC gaps, we obtain Z 2 =1 for both of them, signifying a 2D TI phase. The corresponding evolution of WCCs for Δ 1 and Δ 2 is shown in figures 6(a) and (b), respectively. To further support Z 2 number calculations, the topological edge states for the surface alloy are also calculated. A 3×3 Wannier Hamiltonian is constructed by using the maximally localized Wannier functions (MLWFs) in Wannier90 package [44]. As shown in figure 6(c), without SOC, the MLWF bands show excellent agreement with the DFT bands. The three effective MLWFs are shown in figure 6(f), which are equivalent to each other with respect to the three-fold rotational symmetry around center Bi adatom. Each MLWF has mixed components from p orbital of Bi, and s and p orbitals of Si, demonstrating an effective sp 2 basis in 2D trigonal lattice. This is consistent with the orbitalcomponent analysis shown in figure 5. Furthermore, with SOC, a comparison between the MLWF and DFT bands is shown in figure 6(d), showing excellent agreement again. Based on the Wannier Hamiltonian with SOC, the semi-infinite edge states are calculated by using the surface Green's function method [45]. The one dimensional spectral function is shown in figure 6(e). One can see there is one pair of gapless Dirac edge states within the energy window of two SOC gaps. The Dirac edge states connect the upper and lower bulk band edge and form a 1D Dirac cone at the Brillouin zone center, demonstrating the typical characters of topological edge states.
We note that the nontrivial topology for the same 1/3 ML Bi adsorbed Au/Si(111) structure has also been studied in a recent work [45], but it is only used as a comparison for the 2/3 ML Bi adsorption, in which the Δ 1 gap is found to be nontrivial. However, in our work, we construct a general TB model in trigonal lattice to study the band topology between the competition of intrinsic and Rashba SOC, and apply it to the 1/3 ML Bi adsorbed Au/Si(111) structure. Our results provide a deeper physical understanding about the topological states under giant Rashba SOC. Additionally, a much larger nontrivial gap of Δ 2 is found, which is neglected in previous work [45].

Conclusions
In conclusion, a surface system with two topological gaps as well as giant Rashba spin splitting is proposed by both tight binding model and DFT calculations. We demonstrate two nontrivial gaps in a trigonal lattice against the Rashba SOC, and confirm it in the 1/3 ML Bi covered Au/Si(111)-3 3surface. The structure stability and the large SOC gap further indicate that our proposal is a good material candidate for realizing the room temperature TI. Our results provide a new platform to search large gap 2D TI and giant Rashba spin splitting in surface alloy materials, which can also be extended to other metal adatom and semiconductor surfaces for future experimental exploration.