Enhanced third-order and fifth-order Kerr nonlinearities in a cold atomic system via Rydberg-Rydberg interaction

We investigate the optical Kerr nonlinearities of an ensemble of cold Rydberg atoms under the condition of electromagnetically induced transparency (EIT). By using an approach beyond mean-field theory, we show that the system possesses not only enhanced third-order nonlinear optical susceptibility, but also giant fifth-order nonlinear optical susceptibility, which has a cubic dependence on atomic density. Our results demonstrate that both the third-order and the fifth-order nonlinear optical susceptibilities consist of two parts, contributed respectively by photon-atom interaction and Rydberg-Rydberg interaction. The Kerr nonlinearity induced by the Rydberg-Rydberg interaction plays a leading role at high atomic density. We find that the fifth-order nonlinear optical susceptibility in the Rydberg-EIT system may be five orders of magnitude larger than that obtained in traditional EIT systems. The results obtained may have promising applications in light and quantum information processing and transmission at weak-light level.

showed that if the atomic density N a is large than 4 × 10 8 cm −3 the mean-field approximation is broken down. . 57. The third-order nonlinear optical susceptibilities in Rydberg atomic gases can arrive at the order of magnitude of 10 −7 m 2 V −2 and has a cubic dependence on atomic density, which have been reported in experment (see Refs. [15,[23][24][25]) and theory (see Refs. [30,32,35]). 58. Since these high-order nonlinear optical susceptibilities are much smaller than the third-and fifth-order ones, the discussion on them is omitted here.

Intruduction
The study of optical Kerr effect, i.e. nonlinear response of optical materials to applied light field, is one of main topics in nonlinear optics because it is essential for the realization of most nonlinear optical processes [1]. Optical Kerr effect has also found many new applications, including nonlinear and quantum controls of light fields, quantum nondemolition measurement, all-optical deterministic quantum logic, single-photonic switches and transistors, and so on [2,3]. However, Kerr effect is usually produced in passive optical media such as glass-based optical fibers, in which far-off resonance excitation schemes are employed to avoid serious optical absorption. As a result, the Kerr nonlinearity in passive optical media is weak and hence to obtain a significant Kerr effect a long propagation distance or a high light intensity is required. In recent years, many efforts have focused on the study of electromagnetically induced transparency (EIT) [4,5]. Light propagation in EIT media possesses many striking features, including the suppression of optical absorption, the reduction of group velocity, and an enhancement of Kerr nonlinearity [4][5][6], by which many important applications (e.g. quantum mem-ory, highly efficient four-wave mixing, optical clocks, and slow-light solitons, etc.) are possible [4][5][6][7][8][9][10][11][12][13]. However, the largest Kerr nonlinearity, obtained in conventional EIT media [14], is still too small for nonlinear optics at single-photon level [15].
Recently, much attention has been paid to the investigation of cold Rydberg gases [15][16][17], i.e. highly excited atoms with very large principal quantum number. Due to their long lifetime and large electric dipole moment, Rydberg atoms have many practical applications [18]. Especially, the strong and controllable atom-atom interaction in Rydberg gases (called Rydberg-Rydberg interaction for short [17][18][19]) brings many intriguing aspects that can be used to design quantum gates and simulate strongly correlated quantum many-body systems, etc [15][16][17][18][19][20].
However, all studies up to now on the Kerr nonlinearity in Rydberg-EIT systems are limited to the third-order one. Because of the requirement of many applications in quantum and nonlinear optics, such as highly efficient six-wave mixing [39,40], three-photon phase gates [41, 42], multi-photon entangled states and Schrödinger cat states of light [43,44], and stabilization of spatial optical solitons [6,45], it is necessary to find a giant high-order Kerr nonlinearity that can be realized at very weak light level [46].
In this article, we make a systematic theoretical investigation on the optical Kerr effect in a cold Rydberg atomic system via EIT. By using an approach beyond mean-field theory [30,35,47] on the correlators of one-body, two-body, and three-body based on a secondorder ladder approximation, we show that the system possesses not only an enhanced thirdorder nonlinear optical susceptibility, but also a giant fifth-order nonlinear optical susceptibility, which has a cubic dependence on atomic density and can be arrived at the order of magnitude 10 −11 m 4 V −4 . Our results demonstrate that both the third-order and the fifth-order nonlinear optical susceptibilities consist of two parts. One part is contributed by photon-atom interaction and another part comes from the Rydberg-Rydberg interaction. The Kerr nonlinearity induced by the Rydberg-Rydberg interaction plays a leading role at high atomic density. We find that the fifth-order nonlinear optical susceptibility in the Rydberg-EIT system may be five orders of magnitude larger than that obtained in traditional EIT systems, which may have promising applications in light and quantum information processing and transmission at weak-light level.
Before preceding, we note that third-order Kerr nonlinearity was considered in [30,35,47] where nonlinearity is estimated by using the approach beyond mean-field theory, and in [47] where a second-order ladder approximation is adopted to investigate coherent population trapping in Rydberg atoms. Furthermore, the interaction between Rydberg atoms via EIT was also suggested in [31,32]. However, our work is different from [30-32, 35, 47]. First, no fifth-order Kerr nonlinearity was considered in [30-32, 35, 47] (see also recent review [15]). Second, our study (see below) shows that both the photon-atom interaction and the Rydberg-Rydberg interaction have significant contributions to the Kerr nonlinearities (including thirdorder and fifth-order ones), but the contribution of the photon-atom interaction was overlooked in [30-32, 35, 47].
The remainder of the article is arranged as follows. In Sec. 2, the physical model of the Rydberg-EIT system under study is described. In Sec. 3, a perturbation expansion is used to solve the equations of motion of many-body correlators. In Sec. 4, explicit expressions of the frequency ω p and half Rabi frequency Ω p couples the levels |1 and |2 , and the control field with angular frequency ω c and half Rabi frequency Ω c couples the levels |2 and |3 . ∆ 2 and ∆ 3 are oneand two-photon detunings, respectively; Γ 12 (Γ 23 ) is the spontaneous emission decay rate from |2 to |1 (|3 to |2 ). (b) The long-range interaction potential of 87 Rb atoms V (r i j ) = −C 6 /r 6 i j (red solid line) as a function of r i j = |r i − r j |, describing the interaction between the atom at r i and the atom at r j (represented by yellow spheres), both of which are at the Rydberg state |3 = |60 S 1/2 . (c) Schematic of Rydberg blockade. The long-range interaction between Rydberg atoms blocks the excitation of the atoms within blockade spheres (i.e. the ones with the boundary indicated by the yellow dashed lines) of radius R b . In each blocked sphere only one Rydberg atom (small yellow sphere) is excited and other atoms (small blue spheres) are prevented to be excited. The orange (blue) arrow indicates the propagating direction of the probe (control) field. nonlinear optical susceptibilities are presented. Finally, the last section contains a summary of the main results of our work.

Model
We consider an ensemble of lifetime-broadened three-level atomic gas with a ladder-type level configuration, shown schematically in Fig. 1(a). We assume the atomic gas are loaded into a magneto-optical trap and works at a ultracold temperature so that their center-of-mass motion is negligible. A weak probe field of angular frequency ω p (half Rabi frequency Ω p ) couples to the transition between |1 and |2 , and a strong control field of angular frequency ω c (half Rabi frequency Ω c ) couples to the transition between |2 and |3 . The electric field of the system can be written as E(r,t) = E p (r,t) + E c (r,t) with E p (r,t) = e p E p exp[i(k p · r − ω p t)] + c.c. and E c (r,t) = e c E c exp[i(k c · r − ω c t)] + c.c., where c.c. represents complex conjugate and k p , e p and E p ( k c , e c and E c ) are respectively the wavevector, polarization unit vector and amplitude of the probe field (control) field. The upper state |3 is chosen as a Rydberg state, which is assumed to be |3 = |60 S 1/2 for simplicity. The atoms in Rydberg states (i.e. Rydberg atoms) have many exaggerated properties, including long radiative lifetime, large electric dipole moment, strong Rydberg-Rydberg interaction, and so on [16].
Under electric-dipole and rotating-wave approximations, in the Heisenberg picture the Hamiltonian of the atomic gas including the Rydberg-Rydberg interaction is given byĤ H = N a d 3 rĤ H (r,t), with N a the atomic density andĤ H (r,t) the Hamiltonian of the atom at position r of the form wherehω α the eignenergy of the state |α , Ω p = (e p · p 21 )E p /h and Ω c = (e c · p 32 )E c /h are respectively the half Rabi frequencies of the probe and control fields with p αβ the electric dipole matrix element associated with the transition from |β to |α , where δ αβ is Kronecker symbol. The last term on the right side of Eq. (1) is the contribution of the Rydberg-Rydberg interaction, i.e. the Rydberg atom at position r interacts with the Rydberg atom at position r ′ described by the long-range interaction potential V (r ′ − r). Because the Rydberg-Rydberg interaction results in a Rydberg blockade [17][18][19], the integration region of the radial coordinate r ′ in the integral in the last line of Eq.
From the above equations we see that there are two evident nonlinear characters in the system: (i) There is a photon-atom interaction due to the resonant coupling between the probe field and the atoms even when the Rydberg-Rydberg interaction is absent. (ii) There is an atomatom interaction reflected by the last terms on the left hand side of Eq. 3(e) and Eq. 3(f), i.e. the two-body density matrix elements (or the two-body correlators) ρρ 33,3α (r ′ , r,t) ≡ Ŝ 33 (r ′ ,t)Ŝ 3α (r,t) (α = 1, 2) contributed from the Rydberg-Rydberg interaction. It is just these two different nonlinear characters that make the Rydberg-EIT system possess very interesting nonlinear optical properties. Especially, two different types of Kerr nonlinearities (one is resulted from the photon-atom interaction and another one is resulted from the Rydberg-Rydberg interaction) occur in the system, as will be illustrated below. Our model can be easily realized by experiment. One of candidates is the laser-cooled 87 Rb atomic gas with the atomic states shown in Fig. 1(a) assigned as [23,49] with n principle quantum number and other parameters taken as Ω c = 2π × 32 MHz, ∆ 2 = 2π × 160 MHz. All calculations given below will be based on these realistic physical parameters. The long-range interaction potential between two Rydberg atoms has the form as V (r i j ) = −C 6 /r 6 i j [23], where r i j = |r i − r j | is the distance between the ith and jth Rydberg atoms, represented by the yellow spheres in Fig. 1(b). The red solid line in Fig. 1(b) is the curve of the long-range interaction potential −C 6 /r 6 i j as a function of r i j for n = 60, with the dispersion parameter C 6 ≃ −2π × 140 GHz µm 6 , adopted from [23].
Due to the Rydberg-Rydberg interaction, an atom in the state |3 would induce an energyshift V (R) of the state |3 of another atom separated by distance R, which translates into an effective two-photon detuning. Then the long-range interaction energy-shift will block the excitation of all the atoms for which V (R) ≥ δ EIT , where δ EIT is the linewidth of EIT transmission spectrum (i.e. the width of EIT transparency window), defined by Ω 2 c /γ 12 for ∆ 2 = 0 and . Thus the blockade sphere has the radius [23,31,50]. Comparing this to the average interatomic separation obtained byR = (5/9)N −1/3 a ≃ 2.5µm for N a = 10 10 cm −3 , the blockade effect can be obviously observed, as shown in Fig. 1(c). The system can be divided into many blockade spheres (represented by the spheres with the boundary indicated by yellow dashed line in Fig. 1(c) ) and each blockade sphere contains only one Rydberg atom (represented by the small yellow sphere in Fig. 1(c)). Hence, the spatial coarse-graining distance between two nearest Rydberg atoms has size 2R b [32].
We are interested in the optical Kerr effects, especially the third-order and fifth-order nonlinear optical susceptibilities of the system. To this aim, we need the relation between the optical susceptibility of the probe field and the density matrix elements. Since the total electric polarization intensity of the system is given by , the electric polarization intensity of the probe field reads P p = N a {p 12 ρ 21 exp[i(k p · r − ω p t)] + c.c.}, by which one can obtain the optical susceptibility χ p of the probe field by using the formula P p = ε 0 χ p e p E p exp[i(k p · r − ω p t)] + c.c., which yields To obtain the explicit expression of ρ 21 , we must solve Eqs. (3a)-(3f). However, due to the Rydberg-Rydberg interaction, we must also solve the motion of equations for the two-body correlators Ŝ 33Ŝ31 and Ŝ 33Ŝ32 simultaneously where r ′ = r ′′ andŜ αβŜµν in the terms without integration meansŜ αβ (r ′ ,t)Ŝ µν (r,t).
Eqs. (5a) and (5b) have the following features. (i) The equations for two-body correlators Ŝ 33Ŝ3α (α = 1, 2) involve many other two-body correlators (e.g. Ŝ 23Ŝ31 , etc.). Thus one also have to solve additional equations of other two-body correlators. An explicit list of the equations of motion for two-body correlators Ŝ αβŜµν of the system is too long and omitted here. (ii) The equations for the two-body correlators involve three-body correlators (e.g. Ŝ 33Ŝ33Ŝ31 , etc.), which obey the equations of motion for three-body correlators (which are lengthy and not listed here) and also have to be solved. Similarly, the equations of motion of the three-body correlators involve four-body correlators. Finally, one obtains an infinite hierarchy of equations of motion for the correlators of one-body, two-bodies, three-bodies, and so on. Obviously, to make the problem tractable one must truncate the hierarchy of the equations for many-body correlators by using an appropriate method. Here we adopt a second-order ladder approximation, such that for moderate atomic density the three-body correlation terms in the two-body correlator equations are factorized in the following way [30,35,47, 51] As a special case, when the atomic density is low and the interaction between atoms is weak so that the correlation between atoms are negligible, one has S αβ (r ′′ )S µν (r ′ )S α ′ β ′ (r) → S αβ (r ′′ ) S µν (r ′ ) S α ′ β ′ (r) , corresponding to a mean-field approximation. We stress that the mean-field approximation is not valid for the Rydberg gases even at lower atomic density because of the strong Rydberg-Rydberg interaction. On the other hand, to acquire a giant nonlinear optical effect, a higher atomic density is usually needed and hence one must adopt a method beyond the mean-field approximation [52]. The factorization method stated above is an effective approach for dealing with interacting multi-body problems and has been widely adopted in nonlinear laser spectroscopy [51] and Bose-condensed gases [53], by which the equations of motion for one-body and two-body correlators are closed and hence can be solved by using some suitable techniques.

Solutions based on perturbation expansion
Although by using the factorization method stated above the equations of motion for one-body and two-body correlations can be made to be closed and their number becomes finite, they are still nonlinear due to the coupling with the applied laser field. Fortunately, since the probe field in the EIT-based experiments [21][22][23][24][25][26][27] is weak we hence can make a perturbation expansion of the correlators in the powers of the Rabi frequency of probe field Ω p [35] to solve these nonlinear equations in a systematic way. In fact, when EIT systems are weakly driven, the half Rabi frequency Ω p is a natural expansion parameter for investigating many weak nonlinear phenomena, including ultraslow and weak-light solitons in EIT-based systems [11][12][13].
To investigate the optical Kerr effects in the present Rydberg-EIT system, we make the perturbation expansion [35] β β = δ β 1 δ β 1 (α = 2, 3; β = 1, 2, 3). Substituting this expansion into the Eq. (3) for the one-body density matrix elements and comparing the expansion parameter of each power Ω p , we obtain a set of approximated equations for ρ (l) αβ , which are listed in Appendix A. The approximated equations for the two-body density matrix (correlator) elements ρρ (l) αβ ,µν after using the factorization formula (6) can also be obtained, which are listed in Appendix B. In order to acquire third-order and fifth-order nonlinear optical susceptibilities, we must solve the expansion equations from the first order to the fifth order. Although these expansion equations are lengthy and complicated, they become linear after the above expansion and thus can be solved analytically order by order in a systematical and clear way. Notice that in this work we are interested in static (or instantaneous) nonlinear optical susceptibilities, both the probe and control fields are assumed to be continuous waves. Thus the operator ∂ /∂t in all equations of the correlators can be put into zero.
At the first order (l = 1), we obtain the solution ρ where a general expression of ρρ 33,31 is given in Appendix B [see Eq. (18) ]. Because we have assumed the probe field to be weak, the Rydberg-Rydberg interaction gives the contribution to the solution starting only from the third order approximation.
At the fourth order (l = 4), the solution is given by ρ With the above solutions, we go to the fifth order (l = 5). The solution at this order reads ρ where the expressions of ρρ

Giant third-order and fifth-order nonlinear optical susceptibilities
Collecting the first-order to the fifth-order solutions of ρ 21 obtained in the last section, we obtain ρ 21 ≃ ρ (1) 21 ( j = 1, 3, 5, ...) are independent of Ω p and their explicit expressions have been given in the previous section. Using the formula (4) and the definition Ω p = (e p · p 21 )E p /h, we have where χ p , and χ (5) p are respectively the first-order (linear), the third-order and the fifthorder (nonlinear) optical susceptibilities of the probe field, defined by where χ (3) p1 and χ (5) p1 are the third-order and the fifth-order nonlinear optical susceptibilities arising from the interaction between the probe field and the atoms (i.e. by the photon-atom interaction), χ (3) p2 and χ (5) p2 are third-order and fifth-order nonlinear optical susceptibilities arising from the Rydberg-Rydberg interaction (i.e. by the atom-atom interaction). From the expressions (13a) and (13c), we see that χ (3) p1 and χ (5) p1 have a linear dependence on the atomic density N a . Differently, from the expressions (13b) and (13d) we observe that χ (3) p2 has a quadratic dependence on the atomic density (i.e. on N 2 a ) and χ (5) p2 has not only quadratic but also cubic dependence on the atomic density (i.e. on N 2 a and N 3 a ), which implies that the nonlinear optical susceptibilities in the Rydberg-EIT system are very sensitive to the change of the atomic density N a .

Rydberg-Rydberg interaction
We now calculate the numerical values of the third-order and the fifth-order nonlinear optical susceptibilities in the system based on the experimental parameters as given above. The other system parameters are selected as n = 60, N a = 3 × 10 10 cm −3 and ∆ 3 = 2π × 0.8 MHz. By using the solutions presented in Sec. 3 and the susceptibility formulas given in Eq. (13), we obtain the third-order nonlinear optical susceptibilities χ pα ) (α = 1, 2) and the fifth-order nonlinear optical susceptibilities χ (5) pα = Re(χ (5) pα ) + iIm(χ (5) pα ) (α = 1, 2) of the system, which are listed in Table I. Note that the value at the second row and the second column in the table is the real part of χ Lines from 1 to 3 correspond to N a = 4 × 10 10 cm −3 , 2 × 10 10 cm −3 and 1 × 10 10 cm −3 , respectively. 10 −13 m 2 V −2 ), etc. In the last column of Table I, the physical origins of various nonlinear optical susceptibilities of the system are given.
From Table I, we can obtain the following conclusions: (i) The nonlinear optical susceptibilities in the present Rydberg-EIT system are much larger than those obtained with conventional optical media such as optical fibers. They are also larger than that obtained by using conventional EIT [14]. (ii) For the given atomic density (i.e. N a = 3 × 10 10 cm −3 ), the nonlinear optical susceptibilities contributed by the Rydberg-Rydberg interaction (i.e. χ ). Thus at this atomic density the Rydberg-Rydberg interaction plays a leading role for the contribution of the nonlinear optical susceptibilities in the system. In particular, the fifth-order nonlinear optical susceptibility originating from the Rydberg-Rydberg interaction can reach the order of magnitude of 10 −12 m 4 V −4 . (iii) The imaginary parts of the all nonlinear optical susceptibilities are much smaller than their corresponding real parts, which means that the nonlinear absorption can be suppressed in the nonlinear optical processes of the system. The physical reason for such suppression of the nonlinear absorption is the quantum interference effect induced by the control field (i.e. EIT effect) and also the introduction of the larger one-photon detuning ∆ 2 , which makes the system have the giant optical nonlinearity of dispersive type [36].
The most interesting property of the nonlinear optical susceptibilities in the system is their dependence on the atomic density and the probe-field intensity when the other physical parameters are fixed [23]. Fig. 2(a) shows the real part of the third-order nonlinear optical susceptibilities Re(χ   Table I except for N a , which is now taken as a variable. From the figure we see that: (i) The Kerr nonlinearities contributed by the photon-atom interaction and the Rydberg-Rydberg interaction are comparable with atomic density N a around 10 8 cm −3 , and both of them are increasing functions of N a . (ii) For N a less than 0.9 × 10 8 cm −3 , the Kerr nonlinearity by the Rydberg-Rydberg interaction is smaller than that by the photon-atom interaction. However, the both Kerr nonlinearities arrive at the same value Re(χ at N a = 0.9 × 10 8 cm −3 . (iii) When N a is larger than 0.9 × 10 8 cm −3 the Kerr nonlinearity by the Rydberg-Rydberg interaction surpasses that by the photon-atom interaction and grows rapidly as N a increases. Shown in Fig. 2(b) are the real parts of the fifth-order nonlinear optical susceptibilities Re(χ (5) p1 ) (black dashed-dotted line) and Re(χ (5) p2 ) (red solid line) as functions of N a , which originate respectively from the photon-atom and the Rydberg-Rydberg interactions. We observe that both Re(χ (5) p1 ) and Re(χ (5) p2 ) are comparable, and they are decreasing functions of N a . Furthermore, for N a less than 1.86 × 10 8 cm −3 , the fifth-order nonlinear optical susceptibility by the Rydberg-Rydberg interaction is larger than that by the photon-atom interaction. At N a = 1.86 × 10 8 cm −3 , the both optical susceptibilities become equal to have the value Re(χ When N a is larger than 1.86 × 10 8 cm −3 Re(χ (5) p2 ) becomes to be smaller than Re(χ (5) p1 ). From the above results, we see that there exist various, synergetic optical nonlinearities in the Rydberg-EIT system. Due to the N 2 a -and N 3 a -dependence, the nonlinear optical susceptibilities contributed by the Rydberg-Rydberg interaction are sensitive to the atomic density and hence they can exceed the optical nonlinearities contributed by the photon-atom interaction for large N a . From Fig. 2(a) and Fig. 2(b) we also see that the sign of χ (3) pα is opposite to that of χ (5) pα (α = 1, 2) and χ (5) pα grows faster than χ (3) pα , which means that there exists a competition between χ (3) pα and χ (5) pα when N a becomes larger. Different from Fig. 2(a) and Fig. 2(b), where the dependence of the third-order and the fifthorder nonlinear optical susceptibilities on the atomic density N a are illustrated, in Fig. 2 i.e. the real part of the total nonlinear optical susceptibility of the probe field, as a function of |E p | (E p is the envelope of the probe field) for several different atomic density. Lines from 1 to 3 in the figure are for N a taken to be 6 × 10 10 cm −3 , 4 × 10 10 cm −3 , and 3 × 10 10 cm −3 , respectively. We observe that, for all N a , Re(χ N ) grows fast initially, then arrives a peak value, and finally decreases as |E p | increases. We also observe that the higher the atomic density, the faster Re(χ N ) arrives to its peak value, which is due to the effect coming from the fifth-order nonlinear susceptibilities. We stress that the maximum value of the probe field used in Fig. 2(c) is |E pmax | = 120 V/m, which is within the validity domain of the perturbation theory used above because |Ω pmax /Ω c | ≃ 0.1.
For comparison, in Fig. 3(a) (Fig. 3(b) ) we show the third-order (fifth-order) nonlinear optical susceptibility obtained for several typical physical systems, including optical fibers [54], conventional EIT system [14], active Raman gain (ARG) system [55,56], and the present Rydberg EIT system, with the value of χ (3) p (χ (5) p ) indicated by the yellow solid circle, green solid circle, blue solid circle, and red solid circle, respectively. The black vertical line at each solid circle indicates the range of the nonlinear optical susceptibility for the atomic density varying from 10 9 cm −3 to 6 × 10 10 cm −3 . The Grey shaded area in the lower part of the figure symbolizes the range of the nonlinear optical susceptibilities for optical fibers. We see that the third-order and the fifth-order nonlinear optical susceptibilities obtained by using the present Rydberg EIT system have the highest values in comparison with the other systems [14,[54][55][56]. Especially, the third-order nonlinear optical susceptibilities in the present Rydberg EIT system can reach the order of magnitude of 10 −7 m 2 V −2 for atomic density N a = 6 × 10 10 cm −3 , which agrees fairly with reported experimental and theoretical results [57]. If the atomic density increases to N a = 5.0 × 10 12 cm −3 (used in [14]), we obtain χ Thus the third-order nonlinear optical susceptibility in the present Rydberg-EIT system is five orders of magnitude larger than that obtained in the conventional EIT system, where χ [14]. Furthermore, The fifth-order nonlinear optical susceptibilities in the present Rydberg EIT system can reach the order of magnitude of 10 −11 m 4 V −4 , which is five orders of magnitude larger than that of the conventional EIT system and the ARG systems for the peak  atomic density N a = 6 × 10 10 cm −3 . The physical reasons for such giant third-order and fifthorder optical Kerr effects obtained in the present Rydberg-EIT system are the cooperative response of a large number of atoms, the quantum interference contribution from the EIT, and the Rydberg-Rydberg interaction in the system. Such giant third-order and the fifth-order optical nonlinearities are very promising for the investigation of many nonlinear optical processes not possible by using conventional optical media up to now.
Note that when taking ∆ 2 ≪ Γ 12 , we can also gain another type of optical nonlinearity of the system. For instance, if we choose N a = 3 × 10 10 cm −3 , Ω c = 2π × 16 MHz, ∆ 2 = 1 kHz and ∆ 3 = 0 (the blockade sphere radius R b = 3.26 µm) and the other parameters the same as those given above we obtain χ p and χ (5) p have the same orders of magnitude as their corresponding real parts, i.e. the system displays an optical nonlinearity of dissipative type. The large imaginary part in χ (3) p and χ (5) p will result inevitably in high photon loss for the nonlinear behavior in the system. Notice that the dissipative-type third-order nonlinear optical susceptibility was considered in [30], in which the photon-atom interaction has negligible contribution to optical susceptibilities. The reasons are the following. (i) The Rydberg state has a long lifetime (i.e. Γ 23 is very small); (ii) The two-photon detuning ∆ 3 was taken to be zero. As a result, in [30] only the Rydberg-Rydberg interaction contributes to the nonlinear optical susceptibilities. In our work, both the third-order and the fifth-order nonlinear optical susceptibilities have been considered by taking a non-zero ∆ 3 , and hence both the photon-atom interaction and the Rydberg-Rydberg interaction play significant roles for the optical nonlinearity of the system.

Summary
In this article, we have investigated the optical Kerr effects in an ensemble of cold Rydberg atoms via EIT. By using an approach beyond mean-field approximation, we have proved that the system can possess not only an enhanced third-order nonlinear optical susceptibility, which has a N 2 a -dependence, but also a giant fifth-order nonlinear optical susceptibility, which has N 2 a -and N 3 a -dependence. We have demonstrated that both the third-order and the fifth-order nonlinear optical susceptibilities consist of two parts, which are contributed respectively by the photon-atom interaction and the strong Rydberg-Rydberg interaction. The Kerr nonlinearity induced by the Rydberg-Rydberg interaction plays a leading role for high atomic density. We have found that the fifth-order nonlinear susceptibility in the present Rydberg-EIT system may be five orders of magnitude larger than that obtained in traditional EIT systems, which may have promising applications in light and quantum information processing and transmission at few photon level. The theoretical method proposed here is systematic and can be used to calculate other high-order (e.g. the seventh-order, ninth-order, etc.) nonlinear optical susceptibilities [58], and for other Rydberg states (e.g. nD states for which the dipole-dipole interaction is angular dependent [59,60] where M 31 = iΓ 12  33,31 = P 0 + P 1 V (r ′ − r) + P 2 V (r ′ − r) 2 Q 0 + Q 1 V (r ′ − r) + Q 2 V (r ′ − r) 2 where M 43 = iΓ 12 + iΓ 23 where K n and J n are functions of the spontaneous emission decay rate γ µν , detunings ∆ µ and half Rabi frequency Ω c . The fourth order of atomic population ρ 33 Ω c d * The solution of the Eqs. (21) and (22) is given by ρρ  αα,β 1 = aa (5) αα,β 1 + N a bb (5) αα,β 1 , ρρ (5) 32,α1 = aa (5) 32,α1 + N a bb (5) 32,α1 and ρρ (5) α1,23 = aa (5) α1,23 + N a bb (5) α1,23 (α, β = 2, 3). General expressions for the radial dependence of aa (5) 33,31 and bb (5) 33,31 are given as: aa (5) 33,31 = ∑ 8 n=0 W n V (r ′ − r) n ∑ 9 n=0 Y n V (r ′ − r) n , bb (5) 33,31 = were W n , Y n , X n and Z n are functions of the spontaneous emission decay rate γ µν , detunings ∆ µ and half Rabi frequency Ω c . The fifth-order nonlinear susceptibility χ (5) p2 can be obtained by integrating these general forms analytically. The explicit expressions of aa (5) αα,β 1 , aa (5) 32,α1 , aa (5) α1,23 , bb (5) αα,β 1 , bb (5) 32,α1 and bb (5) α1,23 are omitted here.