Tuning the order of the nonequilibrium quantum phase transition in a hybrid atom-optomechanical system

We show that a hybrid atom-optomechanical quantum many-body system with two internal atom states undergoes both first- and second-order nonequilibrium quantum phase transitions (NQPTs). A nanomembrane is placed in a pumped optical cavity, whose outcoupled light forms a lattice for an ultracold Bose gas. By changing the pump strength, the effective membrane-atom coupling can be tuned. Above a critical intensity, a symmetry-broken phase emerges which is characterized by a sizeable occupation of the high-energy internal states and a displaced membrane. The order of this NQPT can be changed by tuning the transition frequency. For a symmetric coupling, the transition is continuous below a certain transition frequency and discontinuous above. For an asymmetric coupling, a first-order phase transition occurs.


Introduction
Using the concept of phase transitions, a great variety of different physical systems can be classified in terms of their emergent collective behavior [1][2][3]. While phase transitions of both classical and quantum systems in equilibrium are by now quite well understood, the extension to nonequilibrium is a relatively new field. In particular, it is of interest to understand which equilibrium properties survive at nonequilibrium, involving both external driving and dissipation. On the other hand, novel properties may emerge in driven dissipative systems, where energy is not conserved, and the detailed balance condition and the fluctuation-dissipation theorem are no longer valid. Yet, from an experimental point of view, it is not easy to realize and control systems with a nonequilibrium phase transition, in particular when quantum fluctuations dominate over thermal effects from the environment. Currently discussed systems, which show nonequilibrium quantum phase transitions (NQPTs), are ultracold atoms in a lattice inside an optical cavity [4][5][6][7][8] and microcavity-polariton systems [9][10][11]. Laser-driving offers the unique possibility to address and switch between different phases of quantum many-body systems by tuning the pump strength.
Recently, also for hybrid atom-optomechanical quantum systems [12][13][14][15][16], a NQPT of second-order has been predicted [17] and a rich phase diagram has been obtained [18]. Such hybrids combine optomechanics with atom optics, as theoretically proposed [12] and later experimentally realized [13][14][15][16]. The vibrational motion of a nanomembrane in an optical cavity is coupled to the spatial motion of a distant cloud of cold 87 Rb atoms that reside in the optical lattice of the outcoupled light field. By combining different cooling mechanisms such as optical feedback cooling [15] and sympathetic cooling by utilizing the atom gas as a coolant [12][13][14][15], the nanooscillator can be cooled close to its quantum mechanical ground state. Quantum many-body effects lead to collective atomic motion with an instability [16] and a second-order NQPT [17,18] to a state with a spatially shifted optical lattice. Besides, indirect quantum measurement, atom-membrane entanglement and coherent state transfer are in the focus of interest [19][20][21][22][23].
A significant drawback in the motional coupling scheme [12] is the strong frequency mismatch between the nanooscillator and the atomic motion in the optical trap which hinders resonant coupling. A decisive advance is the use of internal atomic states instead of their spatial degree of freedom, such that this internal state coupling scheme [24] enables resonant coupling. Here, the motion of the mechanical membrane is indirectly coupled to transitions between internal states of the atoms via translating the phase shift of the light, caused by the membrane displacement, into a polarization rotation using a polarizing beam splitter (PBS). By this scheme, membrane cooling [24,25], or a displacement-squeezed membrane [26] can be realized. The atoms can implement an effective harmonic oscillator with negative mass [27], that, in turn, can be utilized for quantum back-action evading measurements [28], enabling a high displacement sensitivity. Moreover, the collective nature of the hybrid system mediates long-range interactions in the atom gas, similar to those in a spinor dipolar Bose-Einstein condensate [29,30].
In this work, we show that the internal state coupling scheme also allows for a NQPT, whose order can be readily tuned by changing the atomic transition frequency. Thus, both a firstand a second-order NQPT can be realized in the same physical set-up by only changing a directly accessible control parameter. We show this for the 'membrane-in-the-middle-setup' [24], where the adiabatic elimination of the light field yields an effective coupling between the membrane and the transition between two states in the atom gas, see figure 1. In a meanfield description, the atomic part is reduced to a single-site problem with a Gaussian ansatz for the condensate profile. Tuning the atom-membrane coupling by changing the laser intensity, the system undergoes a NQPT. We provide simple analytical expressions for the resulting critical point. Moreover, by tuning the atomic transition frequency, even the order of the phase transition can be changed from second-to first-order and vice versa. In case of a discontinuous phase transition, the system exhibits a characteristic hysteresis which can be detected by measuring the occupation of the internal states of the atom gas. Throughout this work, we assume natural units and consequently set ÿ=c=1.

Model and adiabatic elimination of the light field
We consider an ensemble of N ultracold 87 Rb atoms placed in an external optical lattice. The atoms exhibit three relevant internal states , , e t Î + -{ }that are arranged in a Λ-type level scheme. The two low-energy states are energetically separated by the atomic transition frequency a W , which can be tuned by an external magnetic field.
The transition between the states +ñ | and eñ | is driven by an applied σ − circularly polarized laser with frequency L w . The passing beam is directed to a PBS, which divides the circularly polarized light into linearly polarized π x and π y light beams on two perpendicular arms, which are of equal length measured for an undisplaced membrane, see figure 1. The vertical path involves a fixed mirror which reflects light with conserved polarization π x . In the horizontal path, a nanomembrane with resonance frequency m W is placed inside a low-finesse cavity, which reflects y y p p  light when undisplaced. The light of both arms is directed back onto the atoms mediating the effective atom-membrane coupling.
In a quasi-static picture, a finite displacement of the membrane induces a finite phase shift on the propagating horizontal π y beam leading to a rotated polarization after the light has passed the PBS backwards. The emergent σ + photon now impinges on an atom and may induce a two-photon transition between the states -ñ | ↔ +ñ | , when the resonance condition m a W W  is met. The back-action of the atoms on the membrane is induced by a transition of the atoms between the states -ñ | and +ñ | . The emitted σ + photons pass the PBS with 50% probability horizontally and hit the membrane. This changes the radiation pressure on the membrane.

Linearized coupling of the membrane and the light field
The external pumping laser has a σ − polarization, such that the coherent drive is included by the linear replacement at the laser frequency In the following, we assume 1 L a  | | such that the interaction between the light field and the membrane (atoms) can be linearized in the operators b w-and b w+ . In a reference frame that rotates with the laser frequency L w , the linearized membrane-light field interaction takes the form with the coupling strength m l . In the 'membrane-in-the-middle' setup, the membrane-light field coupling r 2 scales with the cavity finesse  and the light field amplitude L a . Moreover, r m | | is the membrane reflectivity and M m m 1 2 = Wℓ ( ) denotes the amplitude of the zero-point motion of the membrane, where M stands for the mass of the membrane [12]. Here, we have neglected the quadratic term in L a , which leads to a constant linear force on the membrane and, thus, only alters its equilibrium position. This can be accounted for by a simple redefinition of the zero-point position.
The dipolar interaction of the atoms with the light field induces an AC-Stark shift of the electronic levels of the atoms. After the elimination of the auxiliary excited state eñ | and the linearization in the light field operators, the atom-light field coupling is given by which includes two essentially different processes. On the one hand, the first line couples atoms in the internal state +ñ | to the photon field quadrature in a fashion similar to the motional coupling scheme [12]. This interaction emerges due to the driving of the atomic transition e +ñ « ñ | | and scales according to , where μ − is now the atomic transition dipole moment between the two states e -ñ « ñ | | .
In order to simplify the linearized atom-light field coupling, the external potential V τ (z) can be chosen such that the atoms are positioned around the lattice sites z j , defined by the relation z sin 2 1 ) . An additional potential for the atoms in the state +ñ | has to be provided in order to cancel the lattice potential generated by the coherent drive. This leads to a constant term that redefines the atomic transition frequency. Overall, we choose the potential according to , where V characterizes the lattice depth.

Adiabatic elimination of the light field and effective equations of motion
In the following, we consider an optical cavity in the bad cavity regime in order to adiabatically eliminate the light field modes. We assume that the cavity linewidth is large compared to both the atomic and the membrane frequency such that both sideband photons are well accommodated in the response profile of the cavity. To do so, we start with the linearized Hamiltonian of equation (1) in the interaction picture with respect to the light field Hamiltonian H l . Hence, the light mode operators transform via b t b t exp i where the index I labels the interaction picture.
The formal solution of the Schrödinger equation for any arbitrary state yñ | in the interaction picture reads t s Hs with the time-ordering operator  and H H H tot l ¢ = -ˆˆ. Next, we expand the equation on the right-hand side for small time steps δt. Up to second order, the relevant terms read Moreover, we assume that the initial state is a product state 0 v a c a m l denotes the vacuum state of the light field and a m yñ + | stands for an arbitrary state in the atom-membrane subspace. Under these assumptions, the photon mode operators fulfill b 0 0 y ñ = wm | ( ) and we may define the noise-increment operators In addition, we make the assumptions that, first, the coupling between a single field mode and the membrane or a single atom is usually very small and, second, the time scale on which the dynamics occurs for the compound of atoms and membrane is very long compared to that of the photons. In combination with a strong photon loss in the cavity, the light field rapidly approaches its steady state and a Markov approximation is justified. Hence, we take the limit t 0 d  and exploit that the noise-increment operators after a time step δt do not depend on their form at an earlier time. With this, we can derive a quantum stochastic Schrödinger equation (QSSE) in the Ito form [24,31] with where we have scaled and shifted the atom position variable z z 2 L w p  + , such that the lattice minima for V>0 are located at the position z j =jπ with j  Î . Here, we have defined the atom-membrane coupling constant 2 m l l l =  , which corresponds to the process that induces transitions between the internal states under the creation (annihilation) of a phonon, and m 2 is the atomic recoil frequency. Moreover, the coupling of the membrane to the number of atoms in the internal state +ñ | is given by 2 m a l l l ¢ = . The latter, in fact, is not independent of the internal state coupling constant λ as l l c m m ¢ = = + -, such that we can choose the parametrization l lc ¢ = . In addition, we have neglected terms introduced by the light field that lead to long-range interactions in the atom gas. This assumption is justified if the laser frequency L w is far detuned from the transition frequency between the states +ñ | and eñ | . Finally, fluctuations introduced by the light field have been neglected. A phenomenological damping of the membrane mode has been introduced with rate m G together with the corresponding bosonic noise operator m x that is characterized by the autocorrelation functions Here, N m is the environment occupation number which determines the steady-state occupation of the phonon mode in the isolated limit 0 l = .

Regime of realistic experimental parameters
The currently realized motional coupling schemes utilize membranes in the kHz-regime [14,15] . Besides, mechanical oscillators in the GHz-regime [33] are also applicable, which roughly corresponds to vibrational frequencies in the region 1000

Tuning the order of the quantum phase transition
Assuming that the atoms are prepared at low temperature in combination with a weak atom-membrane coupling such that a large fraction of atoms occupies the ground state and a condensate is formed, the combined system dynamics is subject to the set of coupled mean-field equations of motion Here, the first equation describes the motion of the membrane, and the second and third equation describe the dynamics of the atomic condensate in the internal state -ñ | and +ñ | , respectively. The complex amplitude a N a = á ñ is the scaled mean value of the ladder operator â and N y = áY ñ   is the condensate wave function of the corresponding internal atom state, where N denotes the total number of atoms. Throughout this work, we refer to the case with χ=0 as the symmetric coupling, because the equations of motion are symmetric under the exchange a a W  -W and y y « -+ . Consequently, 0 c ¹ defines the case of asymmetric coupling, when this symmetry no longer holds.

Single mode approximation and cumulant expansion
For a sufficiently deep lattice V R w  , the condensate profile is well described by a sum of Gaussians residing in the individual lattice wells. When the wave function overlap between neighboring sites is small, the problem reduces to an effective single-site problem. It is then reasonable to make the ansatz with a constant number of atoms, i.e. the occupation amplitudes t g  ( ) fulfill the condition t t 1 2 2 , the individual condensate widths t s t ( ) and the corresponding phases η τ (t) which are used to induce the dynamics for t s t ( ). In order to reduce the number of parameters, we restrict ourselves to the special case g g = tt¢ . Moreover, we note that the atoms will always be symmetrically distributed around each lattice site which is determined by the form of the lattice and the atom-membrane coupling potential. This is in contrast to the motional coupling scheme [17,34], where the membrane displacement is linearly coupled to the center-of-mass displacement of the atomic condensate. Finally, a mixing of the condensate profiles for different internal states is the energetically preferred state. Already from equation (19) it can be concluded that a maximally mixed condensate maximizes the effective coupling between the atoms and the membrane. This will eventually lead to a minimization of an effective nonequilibrium potential, which we will derive in the following.
In order to justify the ansatz, we numerically determine the steady state of the extended Gross-Pitaevskii equation (GPE) (19)-(21) by using an imaginary time evolution with the Crank-Nicolson scheme. Due to the periodicity of the potential, we use periodic boundary conditions and evaluate the steady state within the interval from −π/2 to π/2. The condensate profile around a single potential well is shown for the symmetric coupling case χ=0 in figures 2(a) and (b) and the asymmetric case χ=1 in figures 2(c) and (d). Here, different coupling constants λ have been chosen according to the color coding in figure 2(b). The panels (a), (c) and (b), (d) show the condensate profile of the atoms in the internal states -ñ | and +ñ | , respectively, as a function of the position coordinate z. The well minimum is located at z=0. Moreover, the insets compare the individual widths σand σ + obtained from a Gaussian fit to the condensate profile according to equation (22). The deviations between the individual widths are negligible in most cases and slightly increase only in the vicinity of a certain critical point. Consequently, we can approximate the condensate profiles by a unified condensate width s s s º = -+ and an equal phase h h h º = -+ . Next, we perform a cumulant expansion [17,35] of the equations of motion in order to determine the dynamics of the respective variational parameters. Thus, we calculate the (i) zeroth and (ii) second cumulants by multiplying equations (20) and (21) the remaining equations of motion are given in a compact form as We note that the last term in equation (23) reflects the argument that a maximally mixed atomic condensate minimizes the effective potential energy.

Nonequilibrium potential and steady-state configuration
In the presence of damping, the system will eventually relax to a steady nonthermal state. Thus, each of the parameters can be split into its steady-state value and deviations from the steady state. In this section, we are mainly interested in the steady-state properties of the combined hybrid system. That is, we make the ansatz t For a qualitative understanding of the role of both the atom-membrane coupling λ and the asymmetry χ, we study the potential surface of equation (28). Now, any local minimum of the effective nonequilibrium potential E , g that minimize the effective potential (28). In addition, this feature is also present in the respective effective potential. Thus, we note that the effective potential E E , 0 s g s s = ( ) [ ( ) ] always exhibits a single stable steady-state configuration σ 0 . On the other hand, the effective potential E g ( ) exhibits either a single, two or three local minima in the relevant parameter regime 1  g | | . Rather than minimizing the potential with respect to all three parameters, we minimize it with respect to 0 a , σ 0 for a given occupation amplitude g , which is taken as an order parameter, and study the resulting potential energy surface E E , 0 g g s g = ( ) [ ( )]. The global symmetry properties of the hybrid system are then determined by g via the influence of the occupations of the condensate species.
In figure 3, we show the steady-state occupation amplitude 0 g (dashed curves) as a function of the atommembrane coupling strength l, which is determined by the equations (24)-(26) in the steady-state limit. In addition, the background illustrates the resulting normalized energy surface , respectively. In figure 3(a), the NQPT is of second order with a critical behavior 0 ) . Interestingly, by tuning the atomic transition frequency a W , the NQPT becomes a symmetric first-order phase transition where the order parameter shows a jump at the critical point, see figure 3(b). Here, the bistable phase corresponds to the two states . For a nonvanishing asymmetry χ>0, even an asymmetric first-order phase transition occurs at a critical coupling, where the left branch 2 -ñ -+ñ (| | ) is energetically raised, such that the right branch 2 -ñ + +ñ (| | ) represents the minimum, see figure 3(c).
In the case of a second-order NQPT, we label the critical coupling by s2 l . An implicit definition of s2 l is found by inserting the steady-state solution of the condensate width log Yet, in the event of a first-order NQPT, such an implicit definition of the corresponding critical coupling rate can not be found on the basis of a set of steady-state equations. However, a procedure to find the critical points can be defined by performing a Landau expansion of the effective nonequilibrium potential E g ( ). Moreover, Landau theory allows us to classify the order of the phase transition by evaluating the expansion coefficients.

Landau expansion of the nonequilibrium potential
In order to verify whether figure 3(b) indeed shows a first-order phase transition, we expand the nonequilibrium potential E g ( ) in the order parameter around 0 0 g = . Due to the asymmetry in the coupling, the Taylor expansion takes in general the form E a a n n n 0 2  g g = + å ( ) , allowing also odd orders in n. In order to fix the condensate width to its value 0 s g ( ), we define the auxiliary function As the second derivative of the width 0 s is always smaller than zero, the sixth-order expansion coefficient consequently fulfills a 6 >0 for the set of parameters considered in the following other than 0 l = .
In order to describe a first-order NQPT in the symmetric coupling regime (χ=0), it is sufficient to consider the Landau coefficients up to sixth order. In this regime, the odd Landau coefficients vanish, since a n In figure 4(a) is shown that by tuning either the potential depth V or the atomic transition frequency a W , the order of the phase transition can be changed from second-(blue region below dashed curve) to first-order (orange region). Alternatively, by changing the atomic interaction strength Ng, the order may also be tuned, which is shown in figure 4(b). In that sense, one might also consider to define a critical optical lattice depth V c or critical interaction strength g c (or atom number N c ) rather than a critical atomic transition frequency. The critical coupling rate s2 l in the second-order regime has been derived by simple arguments on the basis of the steady-state equations, which is equivalent to evaluating the lowest-order Landau coefficient a 2 . On the other hand, in order to determine the critical coupling rate s1 l in the symmetric first-order regime, one has to know at least the expansion coefficients up to sixth order. For a 4 <0, the effective nonequilibrium potential E(γ) exhibits three minima on the real axis, which are located at We recover the result s1 s2 l l = in the limit a c W = W and find that the first-order phase transition occurs in general at smaller coupling rates than the second-order NQPT, i.e. s1 s2 l l < for a c W > W .
For asymmetric coupling ( 0 c ¹ ), also odd orders in the Landau expansion take a finite value. This breaks the symmetry in the nonequilibrium potential E g ( ) with respect to 0 g = . In order to estimate the critical coupling rate a1 l in the first-order regime, we consider atomic transition frequencies that satisfy ) such that a 4 >0 is always satisfied. In this case, it is sufficient to take the Landau expansion up to fourth order. With the same arguments as in the symmetric first-order regime, we derive an expression for the critical coupling rate from the condition E a which is independent of the sign of χ. It is straightforward to show that also a1 s2 l l < and a1 s2 l l = is recovered in the limit 0 c = .
3.4. Hysteresis in the first-order regime A characteristic feature of a first-order phase transition is the occurrence of a hysteresis when the atommembrane coupling l is adiabatically tuned. In terms of the nonequilibrium potential, this is included by the existence of two or more local minima. At a certain coupling rate, these local minima become dynamically ustable and eventually turn into a maximum. At this point, the system jumps to the neighboring local minimum and remains there until this minimum becomes unstable. In the following, we consider the two generic cases of a symmetric and an asymmetric coupling to discuss this effect.
In order to describe the hysteretic behavior, we take the equations of motion(24)-(26) with t t g g = + ( ) ( ) and adiabatically alter the atom-membrane coupling strength. Thus, we obtain for each value of l a long-time solution t lim t g g = ¥ ¥ ( ) which becomes time independent. In figures 5(a) and (c), we show the hysteresis for the symmetric ( 0 c = ) and asymmetric ( 0.25 c = ) first-order phase transition, respectively. On the forward path, the coupling strength l is adiabatically increased and the system is initially prepared in the minimum with the occupation amplitude 0 g = .
The system stays there until it becomes unstable at 3.5. Advantages over motional coupling and tuning of the asymmetry parameter Current experimental set-ups use the motional coupling scheme [12,13]. In order to understand whether a firstorder phase transition is possible in this set-up or not, one has to compare two different energy scales, one which corresponds to the breathing mode frequency of the condensate (here w s ) and the other which is of the order of the energy difference between the normal and the symmetry-broken phase (here E a D~W ). In the motional coupling scheme, the symmetry-broken phase is a configuration where the atomic center-of-mass position is displaced with respect to the minima of the uncoupled lattice defined by the optical potential with depth V. Hence, the relevant energy difference scales with the frequency of the displacement mode that is given by - [17]. Accordingly, both w s and w z do not scale independently and a first-order phase transition cannot be observed.
The internal state coupling scheme overcomes this limitation. The direct observation is possible by either measuring the membrane eigenfrequency or the condensate width σ [17]. In the case of a first-order NQPT, these quantities exhibit a jump at the critical coupling rate, rather than a continuous behavior as in the case of a second-order NQPT, see section 4. Moreover, a direct measurement of the condensate occupation amplitude 0 g (g ¥ ) can detect the NQPT in a straightforward way. Furthermore, from a quantum information perspective, the internal state coupling scheme is superior to the motional coupling scheme, because the information can be stored in discrete atomic states rather than continuous, motional states. The former are commonly less susceptible to external fluctuations.
Though the asymmetry parameter χ is fixed by the ratio between the transition dipole elements m  , it can be tuned effectively. This is achieved by applying an additional perpendicular laser field that drives the transition between the states -ñ | and +ñ | . This gives rise to a term * * d g g g g + + --+ ( ) in the auxiliary potential energy (23). In turn, by tuning the parameter δ, it is possible to reach a point where the coupling between each atom species to