Strong magneto-optical effects due to surface states in three-dimensional topological insulators

We show that a thin film of a three-dimensional topological insulator such as Bi$_2$Se$_3$ or Bi$_2$Te$_3$ can exhibit strong linear and nonlinear magneto-optical effects in a transverse magnetic field. In particular, one can achieve an almost complete circular polarization of an incident terahertz or mid-infrared radiation and an efficient four-wave mixing.


Introduction
Three-dimensional topological insulators (TIs) such as Bi 2 Se 3 or Bi 2 Te 3 are ordinary gapped insulators in the bulk with gapless chiral Dirac fermion states on the surface that show a number of fascinating properties [1,2] and promise applications in electronic devices as well as terahertz and infrared optics [3]. The Fermi velocity for low-energy excitations near the Dirac point is close to that in graphene, which gives rise to similar matrix elements of the optical transitions between surface states. Unlike the Dirac fermions in monolayer graphene, surface states are not spin-degenerate. The spin direction is locked to the direction of momentum forming a chiral structure. Momentum scattering of carriers in surface states requires a spin flip and is suppressed due to the time-reversal symmetry. This leads to a long scattering time in highquality samples without magnetic impurities, which enhances both linear and nonlinear optical response.
Applying a magnetic field breaks the time-reversal symmetry but the momentum scattering is still partially suppressed. Recent experiments in a strong transverse magnetic field have revealed the Landau levels of massless Dirac fermions states with energies E n that scale as B|n|, and demonstrated their longer decoherence time as compared to the Dirac fermion states in graphene [4,5,6]. This leads to sharper transition lines and stronger light-matter coupling, giving rise to strong magneto-optical effects such as a giant Kerr rotation θ K ∼ 1 rad at THz wavelengths [7]. Furthermore, some of these materials have a relatively large bulk band gap (0.3 eV for Bi 2 Se 3 and 0.2 eV for Bi 2 Te 3 [2,8]) and a tunable Fermi level which can be put within the bulk gap close to the Dirac point. Therefore, long-wavelength mid-infrared and THz radiation is not affected by interband absorption between the bulk states and can be selectively coupled to surface states.
In this paper we concentrate on the polarization and nonlinear optical effects in TI films. We show that high-quality thin films can provide a nearly complete circular polarization of the incoming radiation and lead to strong four-wave mixing effects in the mid-infrared and THz range. Their thickness is constrained from below by electron tunneling which leads to the opening of a band gap. This means that the films can be as thin as several nanometers. In Section II we describe the electron states and linear optical properties of the surface states in Bi 2 Se 3 , both with and without magnetic field. We calculate the polarization coefficient for TI films showing the possibility of a near-complete circular polarization of an incoming radiation. In Section III we present the results on the four-wave mixing and stimulated Raman scattering of TI films.

Linear optical properties of the surface states
We start with the effective Hamiltonian for the topological insulator Bi 2 Se 3 without a magnetic field. It has been considered a number of times; here we give a brief summary of the main results relevant for the optical properties. We will use the parameters from [11]. Following the approach in Ref. [10,9], we use the hybridized states of Se and Bi orbitals {|p1 The Hamiltonian is given by where x + k 2 y . The parameters are [11]: M = 0.28 eV, A 1 = 2.2 eVÅ, A 2 = 4.1 eVÅ, B 1 = 10 eVÅ 2 , B 2 = 56.6 eVÅ 2 , C = −0.0068 eV, D 1 = 1.3 eVÅ 2 , D 2 = 19.6 eV A 2 . By expanding the Hamiltonian with the solutions of the surface states at the Γ point, the Hamiltonian can be expressed in a block-diagonal form with Here E + and E − are the energies of surface states at the Dirac point. The explicit expressions for E 0 , D, ∆ and B are thickness-dependent and can be found in Ref. [10,9]. When the film thickness is greater than about 6 quintuple layers (QLs), i.e. greater than 6 nm, the electron coupling between top and bottom surface states becomes weak and the above parameters become constant with E 0 = 0.0337 eV, D = −12.25 eVÅ 2 , hυ F = 4.07 eVÅ, and ∆ and B almost zero. As a result, the surface states becomes gapless in the vicinity of the Dirac point, and h + (k) = h − (k). We then use only one block as the effective Hamiltonian and add a surface degeneracy factor of g s = 2 when calculating the optical response for the radiation interacting with both surfaces. The energy bands are shown in Fig. 1. It is clear that for low-energy excitations up to ∼ 100 meV the k 2 term can be neglected. If a TI film is placed in a uniform perpendicular magnetic field, its effect on the orbital motion is included by the Perierls substitution π = k + ē hc A. The new Hamiltonian is introduced using annihilation and creation operators a = l c √ 2 π − and a † = l c √ 2 π + . Here l c = hc eB is the magnetic length and π ± = π x ± iπ y .
Here C 0 = 1/ √ 2, C n =0 = 1, and ψ |n| is orthogonal Hermite polynomials. The magnetic field condenses the continuous k-dependent states into discrete surface Landau states, and brings a k-degeneracy of 1 For low-energy excitations in the terahertz or long-wavelength mid-infrared region below 100 meV, it is safe to only keep linear terms in the Hamiltonian as the second-order correction is small; see Fig. 1(a). When the terms with D are neglected, the eigenenergies in Eq. (5) have the same universal behavior as in graphene. Therefore, in the following discussion we will use a simplified linear Hamiltonian After finding energies and wave functions for the surface states of topological insulators in a strong magnetic field one can calculate the optical response. For a normally incident optical field, the interaction Hamiltonian describing the coupling between the surface Landau states and light is where˜ µ is defined as ieυ F ω σ . The matrix element of the dipole moment corresponding to the optical transition between states n and m can be calculated as It has the same magnitude and linear scaling with λ as the one in monolayer graphene. The selection rules are similar to that for graphene. Namely, ∆|n| = ±1, andê RHS photons are absorbed when |n f | = |n i | − 1, whileê LHS photons are absorbed when |n f | = |n i | + 1. The transition frequencies between LLs are in the mid/far-infrared region when the magnetic field strength is of the order of a few Tesla, as shown in Fig. 1(b).
High-frequency absorbance in a TI film in the absence of a magnetic field has a constant value π 2 · e 2 hc [3], where we included a degeneracy factor of 2 stemming from two surfaces. In a quantizing magnetic field transitions between the Landau levels (LLs) give rise to multiple cyclotron resonances with peak absorbance values scaling as ω γ · e 2 hc , where γ is a line broadening.
Here we use the density matrix formalism to calculate the linear susceptibility and magnetooptical effects due to the surface states in a TI film. The 2D optical polarization is defined as P( r,t) = N · tr(ρ · µ).
where N is the surface density of states per each surface LL. The density matrix elements ρ nm are calculated from the master equation with phenomenological relaxation rates γ nm : For a given incident field E(ω) = E(ω)e −iωtê , using the 1st-order perturbation solution for a density matrix, the corresponding resonant part of the polarization per one surface layer of a TI is

Polarization effects for a thin film
If the thickness of a TI film is much smaller than the wavelength of incidence and in the limit ω c γ e 2 hc 1, one can directly apply standard formulas for a weak absorption and Faraday rotation of a linearly polarized light [12]. When ω c γ e 2 hc > 1, a TI film becomes optically thick in the centers of the cyclotron lines. We should no longer use the standard formulas, but instead solve Maxwell's equations together with an induced surface current j ⊥ = −iωχ ω E ⊥ . Here E ⊥ is the in-plane electric field andχ ω is the surface (2D) susceptibility tensor. Consider an incident field to be close to resonance with one particular transition between states n and m (ω ≈ ω nm ). Then the linear optical response is dominated by this particular transition: where ε i jk is the Levi-Civita symbol and there is a summation with respect to index i . In the geometry of Fig. 2(a) the resulting linear susceptibility tensor is in a gyrotropic form: where χ ⊥ and g can be calculated from Eq. (12) as with For the radiation normally incident on a TI film or graphene on a substrate (see Fig. 2(a)), with an in-plane electric field given byẼ ⊥ = (E y , E x ) T , the transmitted field is: where α ≡ i 2πω c . If we define the in-plane polarization coefficient as K ≡ E y /E x , the corresponding polarization coefficient for the transmitted wave is The real and imaginary parts of K are shown in Fig. 3(c). As shown in the figure (see also Fig. 4(a)), in the limit ω c γ e 2 hc > 1 K → −i, i.e. the field component with resonant polarization will be almost completely reflected and only the non-resonant circular polarization will go through, thus resulting in a nearly complete circular polarization of the transmitted radiation. Although Figs. 2(c) and 4(a) are plotted for the film thickness d = 0.01λ , the same result is obtained for smaller film thicknesses as long as d > 6 nm.
Although we considered only normal incidence, for a thin layer Eq. (16) remains valid for an obliquely incident light. In this case it describes the projection of the electric field vector on the plane of the layer. By varying the angle of incidence α one can change the eccentricity of the polarization ellipse. For example, if the transmitted polarization at normal incidence is a circle, the ratio of axes of an ellipse scales as a/b = cos α with increasing α.

Polarization effects in a slab geometry
A slab geometry with two 2D massless fermion layers on opposite surfaces emerges naturally for a thick TI film and can also be implemented by putting graphene layers or two thin TI films on two sides of a dielectric substrate; see Fig. 3(a). For a slab thickness d comparable to the wavelength one has to take into account multiple optical reflections between the two surface layers. For an optical field normally incident from z < 0 onto the slab, one can similarly derive the transmission matrices for the lower and upper surface: where n is a high-frequency refractive index of a bulk material between the surfaces. The transmitted in-plane electric field is As can be seen in Fig. 3(b,c), the standard Fabry-Perot transmission peaks at 2d/λ = N where N = 1, 2, ... correspond to the points of near-complete circular polarization when K → −i, similarly to the case of a thin film d λ . In addition, surface layers give rise to extra peaks at 2d/λ ≈ 1/2, 3/2, ... where the transmitted field has a circular polarization with an opposite sense. Therefore, the same material can produce a beam circularly polarized in both directions by changing d or resonant wavelength λ . This is illustrated in Fig. 4(b).

Nonlinear optical properties
Although there are clear differences between 2D electron states in the topological insulators and in graphene that one can see e.g. in the chiral structure of the Hamiltonian (4) and its eigenenergies/eigenstates (5), they share the same ± |n|B sequence of the LLs and the same structure of the dipole matrix elements. This results in similar nonlinear optical properties. Here we consider just one example: a resonant four-wave mixing process based on resonant transitions between the LLs of surface states in Bi 2 Se 3 (Fig. 5) which is similar to the one considered in [14] and shows similar nonlinear conversion efficiency. Another four-wave mixing process of an efficient two-photon parametric decay in a four-level scheme of LLs was considered in [15]. In Fig. 5 all transitions connected by arrows are allowed and the dipole moment matrix of the 4-level-system is given by Consider a strong bichromatic normally incident field E = E 1 exp(−iω 1 t) + E 2 exp(−iω 2 t) + c.c. with ω 1 resonant with transition from n = −1 to n = 2 and ω 2 resonant with transition between n = 0 and n = ±1, where E 1 has left circular polarization and E 2 has linear polarization. As a result, the field E 2 is coupled to both transitions 0 → ±1. The partially degenerate 4wave-mixing interaction generates a right-circularly polarized signal field E 3 with frequency ω 3 = ω 1 − 2ω 2 . The signal frequency is in the THz range in a magnetic field of order 1 T. The third-order nonlinear susceptibility corresponding to this process can be calculated using the density matrix approach similarly to [14]: Here˜ µ mn is defined as ieυ F ω m| σ |n , which coincides with dipole moment µ mn at resonance; Γ mn is the complex dephasing factor between surface LLs m and n [14]. At resonance, the dephasing factors become real numbers, and we further assume all the detuning rates are the same Γ i j ∼ γ = 10 12 s −1 in the plot below. When all fields are below saturation intensity and the Fermi level is between states 0 and -1, the equivalent 2D third order nonlinear susceptibility for the thin film is 10 −5 (1/B(T )) esu. When incident fields increase in intensity, the population differences on the transitions coupled by the pump fields decrease. As a result, the third order susceptibility drops after the 4 level system gets saturated and the signal intensity decays as well.
The electric field of the generated signal can be calculated by solving the density matrix equations together with Maxwell's equations. Neglecting the depletion of the pump fields, the relation between the signal field and the nonlinear optical polarization is The resulting signal field is then given by The intensity of the nonlinear signal is plotted in Fig. 6(a) as a function of the pump intensity. The maximum signal is reached when the pump fields are of the order of their saturation values ∼ 10 4 − 10 5 W/cm 2 for the scattering rate ∼ 10 12 − 10 13 s −1 . The maximum signal intensity decreases as 1/γ 3 with increasing scattering rate. It increases roughly as B 2 with the magnetic field, if we ignore the magnetic field dependence of the scattering rate.  Intensity of the four wave mixing signal E 3 as a function of the normalized intensity of the pump field E 1 , x = I 1 /I sat where the saturation intensity I sat ≈ 10 4 W/cm 2 in the magnetic field of 1T and for a scattering rate γ = 10 12 s −1 . The normalized intensity of the second pump field E 2 is taken as 0.6x. Bottom panel: the Raman gain G for the field E 3 under the same conditions and in the absence of the second field E 2 .
Another way to generate a coherent THz signal within the level scheme of Fig. 5 is through stimulated Raman Stokes scattering of the pump field E 1 into the field E 3 . In this case one does not need the second pump field E 2 and the signal amplification develops exponentially, as E 0 exp G, where G is the gain per crossing of the TI film by a normally incident field, which has the form similar to that in graphene [16]: where Ω 41 =μ 41 E 1 /h is the Rabi frequency of the pump field. The gain dependence from the pump intensity is plotted in Fig. 6(b). Similarly to the four-wave mixing case, the gain reaches maximum when the pump field is of the order of the saturation value.
These results show remarkably high values of the four wave mixing efficiency and Raman gain per monolayer of massless 2D fermions. The nonlinear signal could be enhanced even further by placing the TI film in a cavity which increases the effective interaction length of the fields with 2D layers. If the Raman gain is greater than the round-trip losses in a high-Q cavity, one could realize a Raman laser.

Conclusion
We have shown that high-quality thin films of topological insulators can be used as basic building blocks for the polarization optics. A nearly complete circular polarization of the incoming radiation can be achieved for a cyclotron resonance with a quality factor ω/γ ∼ 100. The film thickness can be as small as several nm, and is bounded from below by electron tunneling between the two surfaces which opens the gap. The films also exhibit high third-order optical nonlinearity resulting in a strong four-wave mixing signal and stimulated Raman scattering in the mid/far-infrared range.
This work has been supported by NSF Grants OISE-0968405 and EEC-0540832.