Model Hamiltonian for topological Kondo insulator SmB6

Starting from the kp method in combination with first-principles calculations, we systematically derive the effective Hamiltonians that capture the low energy band structures of recently discovered topological Kondo insulator SmB6. Using these effective Hamiltonians we can obtain both the energy dispersion and the spin texture of the topological surface states, which can be detected by further experiments.


I. INTRODUCTION
Searching for new topological insulators (TI) has become an active research field in condensed matter physics 1,2 . A topological insulator has insulating and topologically non-trivial bulk band structure giving rise to robust Dirac like surface states, which are protected by time reversal symmetry and have the spin-momentum locking feature. Such topological surface states have several remarkable properties. For example, the suppression of back scattering and localization on the TI surface [3][4][5][6][7] . Furthermore, if superconductivity is induced on the surface of TIs via proximity effects, the Majorana bound states can be induced [8][9][10] . These novel properties make topological insulator a promising platform for the design of spintronics devices and future quantum computing applications 11 .
Recently the mixed valence compound SmB 6 has been proposed to be topological insulator and attracts lots of research interests [12][13][14][15][16][17] . Unlike the other well studied topological insulator materials, i.e. the Bi 2 Se 3 family, the strong correlation effects in mixed valence TIs are crucial to understand the electronic structure due to the partially filled 4f bands [17][18][19] . There are two main effects induced by the on-site Coulomb interaction among the f-electrons, one is the strong modification of the 4f band width, the other is the correction to the effective spinorbital coupling and crystal field [20][21][22] . As a consequence, the band inversion in the modified band structure happens between the 5d and 4f band (with total angular momentum j=5/2) around the three X points in the Brillouin Zone. Unlike the situation in Bi 2 Se 3 family, where the band inversion happens between two bands both with the p character and similar band width, the band inversion in SmB 6 happens between two bands with the band widths differing by orders, which leads to very unique low energy electronic structure. Since the band inversion happens at the Zone boundary (X points), which project to three different points in surface Brillouin Zone leading to three different Dirac points on generic surfaces.
Experimentally, the first evidence of topological surface states has been found by transport measurements, and then by angle resolved photo emission spectroscopy (ARPES), Scanning Tunneling Spectroscopy (STS), quantum oscillation magneto-resistance measurements [23][24][25][26][27][28][29][30][31][32][33][34][35] . Unlike the electronic structures in large energy scale, which is mainly determined by the local atomic physics, the topological nature of the electronic structure can be fully described by the quasiparticle structure only, whose form can be determined from the symmetry principles. In the present paper, we will construct a k·p model capturing the full topological and symmetry features of the low energy quasi-particle structure, which leads to topological surface states with the renormalized Fermi velocities. All the symmetry allowed terms in the above k·p model have been obtained by fitting with the band structure obtained by the LDA+Gutzwiller calculation introduced in a previous paper 17 . Such a analytical model gives a clear theoretical description for the quasi-particle structure of SmB 6 , which can be widely used in the further studies. The organization of the present paper is as follows. In Sec.II, we present the crystal structure and band structure of SmB 6 . Then we construct the effective models to describe the bulk band structure for this material from the symmetry considerations in Sec.III. Furthermore we calculate surface states and the spin texture on the the (001) surface based on our model Hamiltonian and show that it is consistent with the tight-binding calculation results. Conclusions are given in the end of this paper.

II. CRYSTAL STRUCTURE AND BAND STRUCTURE
In this section we first describe the crystal structure of the SmB 6 and then discuss the nontrivial topological bulk band structure of it.
crystal structure: SmB 6 has the CsCl-type crystal structure with P m3m space group. The Sm ions are located at the corner and B 6 octahedron are located at the body center of the cubic lattice as shown in Fig-FIG. 1. (Color online) (a) The CsCl-type structure of SmB6with P m3m space group. Sm ions and B6 octahedron are located at the corner and center of the cubic lattice respectively. (b) The bulk Brillouin zone of SmB6 (black cubic) and its projection onto the (001) (blue square). The X points (black dots) in the bulk BZ are projected toΓ andX points (blue square points ) in the (001) surface BZ. ure 1(a). The corresponding bulk and projected Brillouin zone (BZ) of (001) surface for SmB 6 are shown in Fig.1 electronic structure: Previous electronic structure studies find that in SmB 6 the band inversion happens between Sm 4f bands with total angular momentum j = 5/2 and one of the 5d bands, which leads to fractional occupation in 4f j = 5/2 orbitals or "non-integer chemical valence" [36][37][38][39] . Since the band inversion happens at three X points in the BZ, where the 4f and 5d states have opposite parities, the Z 2 topological nontrivial band structure is formed 40,41 . In order to include the strong Coulomb interaction among the f electrons, we implement the local density approximation in density functional theory with the Gutzwiller variational method (LDA+Gutzwiller) and apply it to calculate the renormalized band structure of SmB 6 .
Here we briefly introduce the LDA+Gutzwiller method, for detailed descriptions please refer to Ref. [17 and 42]. The LDA+Gutzwiller method combines the LDA with Gutzwiller variational method, which takes care of the strong atomic feature of the f-orbitals in the ground state wave function. In this method, we implement the single particle Hamiltonian obtained by LDA with on site interaction terms describing the atomic multiplet features, which can be written as, where H LDA , H int and H DC represent the LDA Hamiltonian, on-site interaction and the double counting terms respectively 43 . The LDA Hamiltonian can be expressed in a tight binding form by constructing the projected Wannier functions for both 5d and 4f bands [44][45][46] .The onsite interactions can be described in terms of Slater integrals as introduced in detail in the previous paper 17 . The double counting term H DC subtracts the correlation energy already included in LDA calculation. Within the Gutzwiller approximation, an effective Hamiltonian H ef f describing the quasi-particle band structure can be obtained, which describes the low energy dynamics including the topological surface states 17 . To further study the low energy physics of SmB 6 , i.e. the behavior of the surface states, a simple k·p model Hamiltonian will be very useful. In the next section, we will construct such a model by expanding the H ef f near the three X points.

III. MODEL HAMILTONIAN FOR SMB6
In this section, we will systematically derive the effective Hamiltonian near X points based on k·p theory combined with the results of first-principle calculations. We only give the effective Hamiltonian at X 1 = (0, 0, 1 2 ) point. The effective Hamiltonian at the other two X points can be obtained by acting C 4x or C 4y rotation operations on the Hamiltonian at X 1 . Using the symmetry group at the X point we can construct the effective k·p model near this point and all the parameters used in such a model can be obtained by fitting to the renormalized band structure obtained by LDA+Gutzwiller.
The k·p Hamiltonian is obtained from our one-partial effective Hamiltonian where ψ n,k = e ik·r u n,k (r) are the Bloch wave functions and the effective Hamiltonian only consists of the kineticenergy operator, a local periodic crystal potential, and the spin-orbit interaction term: In terms of the cellular functions u n,k (r), Eq. (2) becomes where n,k ≡ E n,k −h 2 k 2 2m . Expanding the above Hamiltonian at given high symmetry point k 0 , the eigen-equation at k 0 + k can be obtained by where we have ignored the k-dependent spin-orbit term, which is usually much small. Once E n,k0 and u n,k0 are known, the function u n,k0+k (r) can be obtained by treating the term H kp =h m k · p in Eq.(5) as a perturbation. It is more convenient to rewrite the perturbation as where the operator p ± = p x ± ip y and k ± = k x ± ik y .
In the SmB 6 system, we expand the H ef f near the three X points. We chose the k·p basis function at X point as which can well describe the orbital characters for the eigenstates near the Fermi energy. We can then project the k·p Hamiltonian into above basis and the matrix elements of H kp are constrained by the crystal symmetries at X point. The little group at X point is D 4h , which contains the following symmetry operations: (1) fourfold rotation along the z directionĈ 4z =e −i 2π 4Ĵ z , whereĴ α (α = x, y, z) is the operator for the α component of the total angular momentum.
(4) twofold rotation along y directionĈ 2y = e −iπĴy . The symmetry operation can help us to reduce the independent parameters that appear in the k · p Hamiltonian. For example, considering the rotation C 4z around the z direction, we have  3 2 |p + | 5 2 , 5 2 f get a minus sign under C 4z rotation, which means it must vanish. Following the same procedge, we can get c 1 ≡ h 2m d 3 2 , − 3 2 |p + | 5 2 , − 5 2 f is finite. When considering the 2-fold rotation along the y direction C 2y , we get the relation between c 1 and c 1 as where d = D+D xy (k 2 x +k 2 y )+D z k 2 z , fi = F i +F i,xy (k 2 x + k 2 y ) + F i,z k 2 z (i = 5, 3, 1) and k ± = k x ± ik y . The parameters are listed in Table.I. The fitted energy dispersion for SmB 6 is plotted in Fig.2. It shows that our model Hamiltonian with eight bands captures the main features of the band dispersion near X point. An important physical consequence of the non-trivial topological band structure is the existence of Dirac like surface states with chiral spin texture. The X points in the bulk BZ are projected toΓ andX points in the (001) surface BZ as shown in Fig.1(b). To study the surface state and the spin texture nearΓ point, we consider a thick slab limited in z ∈ [−d/2, should be replaced by −i∂ z . The eigenwave function will be given by ψ(k x , k y , z), which can be expanded using basis {ϕ n (z) = 2/d sin[nπ(z + d 2 )/d]} (n = 0, 1, 2, 3, ...). The Hamiltonian for the slab structure is written as H slab Γ,mn (k x , k y ) = ϕ m (z)|H s (k x , k y , −i∂ z )|ϕ n (z) . The surface states nearΓ point can be calculated directly from H slab Γ (k x , k y ). For the surface states and spin texture nearX point, we can use the same method but change the z direction to x direction. The calculated surface states nearΓ andX points are shown in Fig.3(b) and compared with the results from tight-binding calculation Fig.3(a). There are three Dirac cone like surface states. One located atΓ points, the other two located at twō X points, which is different with most of the known 3D topological insulators, such as Bi 2 Se 3 .
From H slab Γ,X we can further get the spin texture of the surface states nearΓ andX points. The spin texture at the energy of 6meV is shown in Fig.4, which shows a strong spin-moment locking on the surface states. We will discuss the spin texture from the symmetry consideration below.
To understand the spin orientation on the surface states, we can construct the surface effective Hamiltonian atΓ andX points on the 2D projected surface BZ.
On the 2D BZ, the little group is C 4v atΓ point and C 2v atX point. The surface states at these two points are transformed as the states with j z = ±3/2 under the symmetry operation in the little group. Form the symmetry consideration, we can write down the effective Hamiltonian for the surface states atΓ andX points respectively: 1) AtΓ point: The Hamiltonian satisfy the C 4v symmetry and time-reversal symmetry must having the following form up to the third order of k: in the basis of {|j z = 3 2 , |j z = − 3 2 }. Whereσ ± = σ x ±iσ y , k ± = k x ±ik y and k 2 = k 2 x +k 2 y . The parameters are listed in table.II Here we should determine the spin operatorsŝ x,y,z for the surface model Hamiltonian atΓ point . We start from the tight-binding Hamiltonian H ef f , which is written in the tight-binding basis space expanded by d and f orbitals located on Sm atoms. Based on this tightbinding model we can construct a thick slab terminated in the (001) direction and calculate the surface states |Ψ α , α = 1, 2. The spin operatorŜ for this slab system can be easily written in the tight-binding basis. Then, we project the spin operatorŜ onto the surface states subspace. Finally, we find that the spin operators for surface states are The total angular momentum operators in the surface states subspace can also be calculated asĵ x = −0.6648σ x , j y = 0.6648σ yĵz = −0.5405σ z . In order to predict or understand properties of the surface states under the external magnetic field, we give the Zeeman coupling terms for surface states, which takes the following form which is obtained by projecting the H Zeeman = µ B h (L + 2Ŝ) · B term into the surface states subspace. HereL and S are the orbital angular momentum and spin operator of the slab system. The the non-zero matrix elements of the g factor matrix for surface states atΓ point are listed in table.II 2) AtX point: As shown in Fig.(3), the Dirac cone has a good linear dispersion. The surface effective Hamiltonian which satisfying the C 2v symmetry and timereversal symmetry must have the following form up to the second order of k in the basis of {|j z = 3 2 , |j z = − 3 2 }, H ss X (k) = ( 0 +a 0 k 2 )σ 0 + i(a 1 k + +a 2 k − )σ + +h.c. .  The spin operators for surface states arê s x = 0.0687σ x ;ŝ y = −0.1223σ y ;ŝ z = −0.1484σ z (15) The total angular momentum operators in the surface states subspace can also be calculated asĵ x = −0.4201σ x , j y = 0.7870σ yĵz = 0.9309σ z , The Zeeman term for surface states atX point is the same as Eq.(13) and the g factors are listed in table.II.

CONCLUSIONS
To summarize, we have derived the model Hamiltonians around the three X points for the 3D TI SmB 6 based on the first principles results and the symmetry considerations. The bulk band structure, the surface states on (001) surface and the spin texture of the surface states can be well described by our model Hamiltonians. These effective Hamiltonians could facilitate further investigations of similar intriguing materials.