Quantum dynamics, particle delocalization and instability of Mott states: the effect of fermion-boson conversion on Mott states

We study the quantum dynamics of superfluids of bosons hybridized with Cooper pairs near Feshbach resonances and the influence of fermion-boson conversion on Mott states. We derive a set of equations of motion which describe novel low energy dynamics in superfluids and obtain a new distinct branch of {\em gapped} collective modes in superfluids which involve anti-symmetric phase oscillations in fermionic and bosonic channels. We also find that Mott states in general are unstable with respect to fermion-boson conversion; particles become delocalized and the off-diagonal long-range order of superfluids can be developed when a finite conversion is present. We further point out a possible hidden order in Mott states. It is shown that the quantum dynamics of Fermi-Bose states can be characterized by either an effective coupled $U(1)\otimes U(1)$ quantum rotor Hamiltonian in a large-N limit or a coupled XXZ $\otimes$ XXZ spin Hamiltonian in a single-orbit limit.


I. INTRODUCTION
The phenomenon of Feshbach resonances in ultra cold atomic gases has attracted much attention. The Zeemanfield-driven two-body resonances between fermion pairs in open channels and bound molecules provide a fascinating way to tune the scattering length between atoms in open channels. Remarkably, this sort of simple twobody physics results in extremely rich quantum manybody states in atomic vapors which have not been explicitly observed in conventional solid state systems. Indeed, by varying the two-body scattering length near Feshbach resonances, several groups have successfully achieved fermionic superfluids in a strongly interacting regime 1,2,3,4 .
The superfluids near Feshbach resonances are related to the BCS-BEC crossover studied a while ago 5,6,7,8 ; this was pointed out by a few groups 9,10,11,12,14,15 . Various efforts have been made to incorporate the twobody resonance between Cooper pairs (open channel) and molecules (close channel) explicitly in the many-body Hamiltonian and many interesting results were obtained. Relations between the multiple-channel model and the previous single-channel model have also been studied and clarified.
After all these interesting efforts, a very reasonable understanding has been achieved. Three general features of Feshbach resonances deserve emphasizing here. The first one is that near a Feshbach resonance the usual Cooper pairing amplitude and molecule condensate wavefunction are proportional to each other. Particularly, phases of two components (fermionic and bosonic) in the manybody wave functions are completely locked. In most cases, it has been shown that molecules mediate an effective interaction between fermions. It also has been emphasized in various occasions that molecules can be integrated out and at low energies one only needs to deal with an effective theory of fermions with attractive interactions. Indeed, in the mean field approximation the Feshbach resonance introduces an effective interaction between fermions, the interaction constant of which is γ 2 F B /(2µ − v). Here γ F B is the coupling strength (see Eq. (1)), v is the detuning energy of molecules and µ is the chemical potential of fermions.
The second feature is the behavior of many-body states at Feshbach resonances. It turns out that the properties of states at resonances very much depend on the underlying two-body parameters. If the resonance width is very larger compared with the Fermi energy, then at resonances the energy per particle in unit of the Fermi energy of free particles is universal, independent of particle densities, or background scattering length, or other microscopic properties of two-body resonances. It is also in this limit one can establish an explicit connection between the two-channel model currently employed to study the physics near Feshbach resonances and the one-channel model studied long time ago. However, if the resonance width is very narrow, then the properties of many-body states further depend on microscopic parameters of twobody physics.
The distinction between these two limits is even more severe if one zooms in and looks into the molecule fraction or the chemical potential at resonances. This is the third general feature we would like to turn to. At wide resonances, the chemical potential of fermions is still of the order of the free particle Fermi energy, while the molecule fraction is actually inversely proportional to the width and is very small. At narrow resonances, the chemical potential is depleted to almost zero and the molecule fraction is substantial.
However, in these previous approaches, three important aspects of this phenomenon have been overlooked, and sometimes, miscomprehended. One is the issue of quantum phase dynamics of bosons and fermions. If we treat molecules and atoms as independent bosons and fermions respectively, there are no particular reasons why there has to be only one condensate phase for two-component superfluids. In fact, it is natural to assume that the bosonic or fermionic superfluid has its own quantum dynamics.
In fact, a critical examination of the problem suggests there should be two phases. Though in the mean field approximation employed in most of previous works on this subject, the two phases are usually locked, dynamically these two phases do have their distinct features and are never truly identical. The extra phase degree of freedom indicates an extra branch of collective modes which can have rather low energies in the limit of narrow resonances 27 . These new excitations are an analogue of small fluctuations of a relative phase between two condensates discussed in Ref. 16. It remains to be studied in details and to be observed experimentally.
The second issue is related to the possibility of having a boson-fermion mixture but with decoupled low energy dynamics. Though this possibility hardly exists in high dimensions, in 1D there can be a phase transition between the usual phase locked superfluid and more exotic phase unlocked states. The critical point is determined by a Sine-Gordon type theory. This was discussed in a recent preprint 17 and has received further critical examination in a unpublished work 18 .
The third aspect is the new quantum dynamics due to the conversion between fermions and bosons. The conversion actually violates the particle number conservation of fermions and bosons respectively and only conserves their total number. Although this violation plays little role in BEC-BCS crossover superfluids because of the large local density fluctuations, it can have a vital impact on other many-body states close to Feshbach resonances. One example is the instability of certain Mott states when the number conservation is violated. The other example is the development of a certain hidden order in Mott states. The purpose of this article is to further investigate these issues and explore the consequences of these observations. A brief discussion on some of these issues was previously presented 27 .
In section II, we introduce a model to study the stability/instability of Mott states of fermion-Boson systems. We discuss the validity of the model and the relevance to the physics near Feshbach resonances. In section III, we exam the stability of certain Mott states of bosons when Fermion-Boson conversion is present. We show that the conversion leads to delocalization of particles in a Mott regime and destabilizes the insulating phase. A Mott state appears to develop a finite density of states at energies well below the Mott gap.
In section IV, we further derive an effective Hamiltonian of fermion-boson systems in a large-N limit. We also obtain the equations of motion and investigate the novel quantum dynamics of fermions and bosons in this limit. General structures of collective modes are studied in a semiclassical approximation. In section V, we demonstrate that the delocalization leads to superfluidity by explicitly showing the development of the off-diagonal long ranger order (ODLO). These calculations also indicate that strong repulsive interactions between bosons or Cooper pairs do not renormalize the superfluid density to zero in some limit. In section VI, we examine the hidden order in certain Mott states and point out various topological excitations in Mott states. These remain to be explored experimentally.

II. HAMILTONIAN FOR LATTICE FESHBACH RESONANCES
The model we employ to study this subject is an Morbit Fermi-Bose Hubbard Model (FBHM). Consider the following general form of FBHM Here k, η and σ label lattice sites, on-site orbits and spins; η = 1, 2, ...M , σ =↑, ↓. f † kησ (f kησ ) is the creation (annihilation) operator of a fermion at site k, with on-site orbital energy ǫ η and spin σ. b † k (b k ) is the creation (annihilation) operator of a boson at site k. For simplicity, we assume that there is only one bosonic orbital degree of freedom at each site. The fermion and boson number operators are, respectively,n bk = b † k b k ,n f k = η,σ f † kησ f kησ . t f and t b are hopping integrals of fermions and bosons respectively, and hopping occurs over neighboring sites labeled as kl . µ is the chemical potential of fermions and v is the binding energy of bosons which are made up of bound states of fermions. λ is the attractive coupling constant in the Cooper channel which we assume to be much larger than the rest of couplings. Finally, V f , V b and V bf are the strength of repulsive interactions between fermions and bosons in the density-density channel 19 (One further assumes V b V f > V 2 f b to ensure that homogeneous states are stable). We only include conversion between a molecule and two fermions in timereversal doublets, and in Eq. 1, we choose to work with doublets of (η ↑, η ↓). FBHMs similar to Eq.(1) were previously applied to study Bose-Fermi mixtures in optical lattices 20 ; most recently an FBHM with fermionboson conversion was generalized to study the BCS-BEC crossover in lattices 21 . In the absence of the conversion term, FBHM consists of decoupled (attractive) Fermi-Hubbard model and Bose-Hubbard model; main properties of latter are known 22 .
In the FBHM, the conversion is between a molecule and two fermions in the same orbit η. This approximation correctly describes the physics near Feshbach resonances at least in the following three limits.
a) The high density limit where fermions mostly occupy high energy on-site orbits. Generally speaking, other conversion terms are allowed and the Hamiltonian should be if we assume the conversion is local. Here Φ 0 is the wavefunction for bosonic molecules, and ψ η,η ′ are wavefunctions of η, η ′ orbits at a given lattice site. If Φ 0 is approximated as a constant, thus the selection rule yields η ′ = η and molecules are only converted into two fermions in same orbits. In a harmonic trap where Φ 0 is a Gaussian wavepacket, one then needs to take into account fermions in different orbits as implied and demonstrated previously 25,26 . However, if orbit η and η ′ correspond to highly excited states, the conversion between molecules and timereversal doublets is considerably larger than other terms. This yields dominating contributions in the large-N limit. In this case, the form of the on-site conversion term approaches the form in the bulk limit; up to a finite size effect, the conversion is between a bosonic molecule and two fermionic atoms in the same orbit because of the wavefunction orthogonality. The Hamiltonian with the conversion between a molecule and two fermions in same orbits thus describes the physics in this limit if M takes a large value and the number of fermions per site is big. b) The low density limit near narrow resonances when non-interacting fermions mostly occupy the lowest orbit. In this case, one can argue that as far as the resonance width is small compared to the spacing between the lowest orbit and higher orbits, the hybridization of molecules and atoms occurs in the lowest energy state; so fermions remain in the lowest orbit. One only needs to take into account resonances between molecular states and fermions in the lowest orbit. And in this case, the fermionic sector of the Hamiltonian is equivalent to a negative-U Hubbard model if one sets M to be one. (see more discussions in section VI). c) The low density limit with magnetic fields not too close to wide Feshbach resonances. The validity in this limit is justified by the following observations. Not too close to wide resonances, again the lowest eigenstate of two interacting fermions in a lattice site mostly involves two fermions in the lowest orbit and a molecular bound state. This implies that the fermion-boson conversion should again be described by terms such as b † k f kη↓ f kη↑ , η = 1.
However, right at wide resonances, the molecule state is effectively hybridized not only with two fermions in the lowest orbit but also with two fermions in different orbits; this has been correctly pointed out and appreciated 25,26 . Even in the low density limit where free fermions occupy the lowest on-site orbit, the above FBHM Hamiltonian when applied to Feshbach resonances is indeed no longer valid from this microscopic point of view. It remains to be understood how many-body physics will be affected by this complication.
Without losing generality, in this article we study the effective low energy theory in a limit where the fermionboson conversion strength is weak (i.e., narrow resonance) and discuss the issue of Mott states' instability. However, we would like to argue that physics discussed in this article would not be affected by the presence of additional conversion terms when the conversion strength is strong (i.e., wide resonance). The main reason is that the form of the long wave length effective Hamiltonian described below is subject to severe constraints from the symmetries and hydrodynamics in our problem and has little dependence on microscopics. As far as these extra terms only renormalize coefficients in equations of motion but do not alter the general form of hydrodynamics discussed in the article, most of conclusions arrived here remain valid even in this delicate limit. This is evident from a general renormalization point of view but has been unfortunately overlooked in the last two references of Ref. 26. We believe that the significance of extra conversion terms on the long wave length physics has been overstated previously.

III. DELOCALIZATION OF PARTICLES UNDER THE INFLUENCE OF FESHBACH RESONANCES
In this section, we are going to study a Mott state under the influence of Feshbach resonances, especially the effect of fermion-boson conversion. A Mott state of bosons or Cooper pairs appears whenever bosons or Cooper pairs in lattices are strongly repulsively interacting and if the corresponding filling factors are integers. One of important properties of a Mott state is its incompressibility, or a finite energy gap in its excitation spectra, thus a Mott state is believed to be robust. When hopping is renormalized to zero due to repulsive interactions, the number of particles at each site can be strictly quantized and discrete; particles are locally conserved.
Below we are going to show that in general Mott states are unstable with respect to Feshbach resonances. The primary reason is that particle numbers of fermions or bosons involved in resonating conversion are not conserved separately. So the conversion not only mediates an attractive interaction between fermions as realized before, but also, more importantly violates the local con- servation law. This introduces new low energy degrees of freedom and results in a novel mechanism to transport particles. It leads to delocalization of particles in the limit of large repulsive interactions.
To address the issue of localization of particles, we first introduce the following time-ordered Green's functions 13 Now we assume t b /V b ≪ 1 and the number of bosons per lattice site n b is an integer so that the ground state of bosons is a Mott state. Without losing generality, we also assume that the chemical potential µ is precisely in the middle of the Mott gap so that the system is particlehole symmetric and V bf = 0. Meanwhile, the number of fermion pairs per lattice site is either a non-integer or an integer but t f /V f ≫ 1 so that the ground state of fermions is a superfluid. We are interested in the effect of fermion-boson conversion on the Mott state.
When there is no fermion-boson conversion, one can evaluate the boson Green's function by an expansion in terms of the parameter t b /V b . For instance, the zeroth order Green's function is reflecting the zero bandwidth in this limit. All low energy excitations are localized and gapped with a single energy V b . Here n b is the number of bosons per lattice site. The small finite hopping amplitude leads to corrections to this form of the Green's function; following the diagram in Fig. 1c, one finds that where R k is the distance between two lattice sites k and 0. To obtain this result, we have assumed that n b is much larger than one. It is obvious that at low energy ǫ ≪ V b , the two-point Green's function decays exponentially as a function of distance R k . Eq.(5) also implies that the localization length at small finite t b should scale as in the unit of the lattice constant. The localization of particles in a conventional Mott state is largely due to the absence of available low energy states below the energy scale set by V b . So to remove a particle at the point 0 and for the particle to travel to the point k, one has to confront a sequence of energy barriers of height V b . This blockade results in the localization.
When a fermionic superfluid is present and the fermion-boson conversion occurs, there is an additional channel for a particle to travel from site 0 to k. The mechanism is schematically shown in Fig. 2. Instead of removing a bosonic particle from site 0 and adding to site k, one can remove a Cooper pair at site 0 and transport it to site k. For this, a Cooper pair experiences no energy barrier imposed by repulsive interactions because there are sufficient low energy degrees of freedom available for particle-hole excitations in a superfluid. At a latter stage, one then turns on Feshbach resonances to remove the boson at site 0 by converting it into a Cooper pair to fill up the hole left behind by the transported Cooper pair; similarly, the Cooper pair transported can be converted to a boson as an additional particle at site k. The net effect is that a bosonic hole is created at site 0 and bosonic particle excitation is now at site k. Since the fermionic channel has a long range order, this process therefore yields a long range particle-hole excitation.
At a formal level, one can study this contribution by introducing a vertex for the fermion-boson conversion. Furthermore, in the weakly coupling limit, the long range component of the fermion Green's function reflects the usual off-diagonal-long range order. In the mean field approximation, one obtains Here ... represents other contributions which decay over large distance. Let us emphasize that F (R k ) is a constant and is independent of the distance R k between k and 0. Following the diagrams in Fig. 1d, one obtains the contribution of the boson Green's function Eq. (8) illustrates two important properties of the state under consideration, which are intimately connected.
Firstly, the non-exponentially decay component of the boson Green's function is proportional to the off-diagonallong-range order in the fermionic channel. The long range component in Eq. (8) shows that at least a fraction of all bosons actually become delocalized. The fermion-boson conversion effectively leads to the delocalization of bosons as argued above (also see Fig.2 for more explicit discussions); it is the delocalization of bosons which in fact induces superfluidity in the bosonic channel. Secondly, the zero energy peak (at ǫ = 0) in the Green's function in the mean field approximation suggests that some bosons should now condense at the zero energy; the condensation fraction is (γ F B /V b ) 2 . Notice that now the resultant bosonic state is compressible. It is thus implied that there should be additional low energy states well below the original Mott gap. These extra states are one of the consequences of the unusual hydrodynamics in the problem. We also anticipate that the low energy structure of δG b such as the peak height is to be modified when various fluctuations are taken into account.
In the next two sections, we are going to analyze the long range order in details. To understand the induced superfluidity, it is most convenient to first obtain an effective theory where the typical issues of broken symmetries can be easily addressed. So in section III, we derive an effective coupled U (1) ⊗ U (1) quantum rotor model for the FBHM. In section IV, we employ the effective model to examine the long range order.

IV. AN EFFECTIVE HAMILTONIAN AND THE EQUATION OF MOTION
A. a U (1) ⊗ U (1) coupled quantum rotor model We first study the large-N limit where n b and n f (< M ), the average numbers of fermions and bosons are both much bigger than unity. Because λ is much larger than other coupling constants, the ground state of fermions for the on-site part of the Hamiltonian H f naturally should be a BCS state. For simplicity, we also assume that the Fermi energy as well as the BCS gap are larger than the Fermi-Boson coupling strength γ F B so that we can neglect the Fermi degrees of freedom at low energies and obtain the effective Hamiltonian written in terms of various collective coordinates (see below).
This suggests that it should be convenient to work with the following coherent state representation, Here u η , v η are the coherence factors in the BCS wavefunction which minimize the total on-site energy; g 0 (n bk ) is a unity for n max + n b > n bk > n b − n max , n max is much larger than one. These states form a low energy Note in e), a particle effectively experiences an energy barrier with height V b ; the amplitude of finding a particle-hole pair separated with a large distance is therefore exponentially small.
Hilbert subspace and are orthogonal in the limit which interests us, or At last, in the coherent-state representation one shows thatn f k /2 = i∂/∂φ f k , andn bk = i∂/∂φ bk ; or So in the subspace of coherent states, we find the effective Hamiltonian is The exchange couplings J f , J b and Γ F B can be estimated as E η = (ǫ η − µ) 2 + ∆ 2 0 is the quasi-particle energy and ∆ 0 is the BCS energy gap. Furthermore, E k is the on-site energy of n f k particles and its second derivative is inversely proportional to the compressibility of a BCS state. In a recent work of one of the authors 27 , it was assumed that V f is much bigger than the second term in the above equation and V ′ f ≈ V f . However, when Fermions do not have repulsive interactions in the density-density channel (V f = 0), V ′ f is equal to ∂µ BCS /∂n f k , µ BCS is the chemical potential of n f fermions in a BCS state, which is typically of order of the one-particle level spacing at each lattice site. From now on, we will assume V f is much larger than the level spacing and omit prime in V ′ f n f and n b are functions of µ, v and V f,b,f b : .
Here µ 0 = µ − µ BCS . Obviously, the detuning energy v has to be sufficiently small in order for n b to be positive. Eq. (10) and Eq. (11) define the low energy quantum dynamics of fermions and bosons under the influence of fermion-boson conversion in Feshbach resonances. In the absence of Feshbach resonances (Γ F B = 0) and V bf , the effective Hamiltonian describes two decoupled sets of quantum U (1) rotors in a lattice, the behaviors of which are well known. If n f /2 or n b is a positive integer, the effective model can be used to study superfluid-Mott state transitions. A Mott phase corresponds to U (1) symmetry restored states and U (1)-symmetry breaking solutions represent a superfluid phase. For the bosonic (Cooper pair) sector, the phase transition takes place when > 1) is the coordination number). The critical values which are of order of unity are usually calculated numerically.
In the presence of Γ F B , the Hamiltonian Eq.(11) describes a coupled U (1) ⊗ U (1) quantum rotor model in a lattice. U (1) ⊗ U (1) symmetry breaking solutions when both r f,b are much larger than unity correspond to a superfluid phase.
In general, the wavefunctions for the many-body ground state and excitations Ψ n ({φ bk }; {φ f k })(n = 0, 1, 2, ...) are the eigenstates of the Hamiltonian in Eq. (11). The boundary conditions are periodical along the directions of φ f k,bk with a period 2π, so the wavefunctions are effectively defined on an S 1 ⊗ S 1 torus with radius of each S 1 equal to one. If the average number n f /2 and n b are integers, one introduces a gauge transformation the shifted number operators become The effective Hamiltonian and eigenstates in the shifted basis are given by the following equation It is evident, following the above equations that when both fermions and bosons are in superfluid phases, quantum phases of two superfluids are locked to minimize the potential energy −Γ F B cos(φ bk − φ f k ). That is So in this new basis a spontaneous symmetry breaking solution with the wavefunction represents a typical superfluid. A symmetry-unbroken solution with the wavefunction In a superfluid phase, the Hamiltonian in Eq.(11) further leads to the following semiclassical equation of motion in the long wave length limit Here δn f k,bk =n f k,bk − n f,b . we have taken a continuum limit and k labels the coordinate of phases of bosons and fermion pairs (φ f k,bk ) in this equation; ∆ is a Laplacian operator. The lattice constant has been set to be one. The above set of equations were previously derived 27 .
In the absence of Fermion-boson conversion, the third and fourth formulae in Eq.(21) are the conservation laws for fermions and bosons respectively.
where supercurrents are defined as J f k,bk = −J f,b ∇φ f k,bk (the definition of phases differs from the conventional one by a minus sign). Obviously, the fermion-boson conversion violates the conservation law and introduces a source term which is proportional to Γ F B . It is this new quantum dynamics which yields the delocalization in the previous section. Below we show that in addition to the usual gapless Goldstone mode, the quantum dynamics in this case also leads to a new branch of collective modes which are fully gapped.
Let us introduce the plane wave representation for φ f k,bt (t) and study the eigenmodes. The above semiclassical equation suggests spectra of collective excitations. The equation for eigen modes φ f,b (ω, Q) reads as Here the matrix elements M α,β (α,β = b, f ) are defined as The eigenfrequencies of modes are the solutions of the following equation By solving the equation for eigen frequencies, one obtains the collective mode spectrum. The above equation shows that there should be two branches of collective modes the dispersion relations of which are given below: φ f,b (ω, Q) are the Fourier components of phase fields φ f k,bk (t). It is worth emphasizing that in the long wave length limit, mode a) is fully symmetric in phase oscillations of fermions and bosons, independent of various parameters; mode b) represents out-of-phase oscillations in fermionic and bosonic channels and becomes fully antisymmetry when V b = V f . In the absence of conversion (Γ F B = 0), these two modes correspond to two gapless Goldstone modes associated with breaking two decoupled U (1) symmetries. However, in the presence of Feshbach resonances only the symmetric mode a) remains gapless corresponding to the usual Goldstone mode of superfluid while the antisymmetric mode b) is fully gapped because of the phase-locking effect of Feshbach resonances. In general, Ω 0 , α and β depend on various parameters in the Hamiltonian; Ω 0 is always proportional to Γ F B , and α on the other hand is independent of Γ F B . When Consequently, the dispersion relations are given by Eq. (26) with Ω 2 0 = 4Γ F B V 0 , α = β = 2J 0 V 0 . A more interesting and realistic limit is when V bf is small and set to be zero but V b,f are not equal. In this case, Eq.(25) becomes One then obtains the dispersion with coefficients given below Note that in the noninteracting limit, V f is equal to ∂µ/∂n f and is finite. The above equations show that the velocity of mode a) (the symmetric Goldstone mode), √ α, decreases when interactions between bosons V b become smaller. This is because as bosons become weakly interacting, the density fluctuations in the symmetric mode are dominated by those of bosons and the fermion density fluctuations become insignificant. So although the sound velocity of fermion superfluids is finite, the fermionic contribution to the symmetric mode is negligible and the Goldstone mode becomes softer and softer as V b goes to zero.
On the other hand, the gap in the antisymmetric mode b) remains finite in the limit when V b = 0; that is Ω 2 0 = 2Γ F B V f . As discussed in the previous subsection, if repulsive interactions between fermions are zero, V f approaches the value of ∂µ/∂n bk . One also notice that when V bf is zero, the total density fluctuations in mode b) at Q = 0 are zero, that is following the last equation in Eq.(26). The above semiclassical approach to collective modes is valid when V f,b,f b are small so that various renormalization effects can be neglected. Collective modes in a large-V limit can be more conveniently studied using a saddle point expansion. This alternative approach to study the collective modes is explored in a unpublished work 28 .

A. Molecule mean field approximation
We first introduce order-parameters to classify various states. The order parameters which can be used to classify states arẽ  to be ground states if there were no Feshbach resonances. To understand the influence of Feshbach resonances on Mott states, we first consider a situation where again both n b and n f /2 are integers and r b is much less than r bc , so that bosons are in a Mott state in the absence of Feshbach resonances. On the other hand r f is much bigger than the critical value r f c so that Cooper pairs are condensed. For simplicity, we have also assumed that V bf is much smaller than V b so that it can be treated as a perturbation. We are interested in the responses of bosonic Mott states to fermion-boson conversion and carry out the rest of discussions in a mean field approximation (MFA).
In this MFA, φ f k = φ f , φ bk = φ b for any lattice site k. The ground state Ψ 0 (φ b , φ f ) (again defined on an S 1 ⊗ S 1 torus with radius 2π) is the lowest energy state of the following MFA Hamiltonian Here again z is the coordination number; we have also introduced two self-consistent order parameters Here stands for an average taken in the ground state, and Ψ 0 is the ground state wavefunction. Notice that the order parameters defined above are nonzero only when the U(1) symmetries are broken; particularly, ∆ f is proportional to the usual BCS pairing amplitude. Following Eq.(9) and discussions above one indeed shows that So ∆ f,b vanish in Mott states and are nonzero in superfluids.
As zJ f is much larger than V f , φ f has very slow dynamics; and the corresponding ground state for φ f can be approximated as a symmetry breaking solution. In the linear order of J b and Γ, one obtains the following solution Here ∆ f should be approximately equal to one in this limit; and in the zeroth order of V −1 b , Ψ 0b does not break the U (1) symmetry and stands for a Mott-state solution. Finally taking into account Eq.(33) and (35), one finds that the self-consistent solution to ∆ b is In the absence of Γ F B , ∆ b vanishes as expected for a Mott state. However, the Mott state solution is unstable in the presence of any Feshbach conversion and the molecular condensation order parameter ∆ b is always nonzero in this limit. We want to emphasize that the average number of bosons per site is not affected by the fermion-boson conversion and remains to be an integer (I); rather, closely connected with the instability is the breakdown of particle number quantization. Indeed, one obtains in the MFA the following results forn bk , (37) This illustrates that the resonance between states with different numbers of bosons at a lattice site eventually leads to a nonzero molecular condensation order parameter ∆ b .
Alternatively, one can consider the renormalization of the condensate amplitude due to enhanced quantum fluctuations when repulsive interactions are introduced. When repulsive interactions are weak, one can carried out usual perturbative calculations.
The results above on the other hand provide information about what happens when interactions are dominating and the conventional perturbation expansion fails. One of the most important consequences of fermionboson conversion is that the suppression of condensate amplitude is never complete if one increases V b only while maintaining small value of V f . The renormalization of the condensate amplitude as a function of V b is plotted schematically in Fig.4.
At last, let us briefly consider the case that both bosons and fermion pairs are in Mott states, i.e., r b < r bc and r f < r f c respectively. Then the mass gaps in two channels behave like m b ∝ r bc − r b and m c ∝ r f c − r f in the absence of the boson-fermion conversion term Γ F B . The conversion term leads to a hybridization between two components, and we can diagonalize their mass matrices to find the new mass gaps. When Γ 2 > m f m b ∝ (r bc −r b )(r f c −r c ), one eigenvalue becomes negative which suggests that the Mott state should be unstable, i.e., a superfluid state should be formed.

B. Saddle point approximation
In this subsection, we are going to provide an alternative approach to ODLO based on a saddle point approximation. We will show again that any finite fermion-boson conversion leads to a finite condensation of bosons disregarding the strength of repulsive interactions between bosons.
For this purpose we first introduce the following partition function where f σ ( r), f † σ ( r) are fermion field variables defined in the continuum, b k are the field variables for bosonic molecules in the closed channel defined at lattice site k, f σ (k) is a coarse average of f σ ( r) within the k-th unit cell as where Ω is the volume of one unit cell. For the reasons outlined in Appendix A, it is more convenient to introduce a φ-field variable to describe the dynamics of bosons; the φ-field can be interpreted as the condensate wavefunction. Following discussions there, in the long wave length limit we obtain the following φ 4theory description of bosons According to Appendix A, when the system has a particle-hole symmetry, one can set r ′ to be zero. This also corresponds to a system where n b is an integer, a situation we have discussed in the previous session. Moreover, when α > 0(< 0), the system is in a Mott phase with an energy gap (in the superfluid phase). Using the standard Hubbard-Stratonovich transformation, we decouple the 4-fermion interaction term by introducing the pairing field ∆. After integrating out the fermions in S ′ bf , S f (see Appendix A, B and especially Eq. B1, B2), we arrive at the following effective action where τ 1,2,3 are the Pauli's matrices in the Nambu's representation, and τ +,− = (τ 1 ± iτ 2 )/2. Consider the standard mean field ansatz ∆( r, τ ) =∆ + δ∆ φ( r, τ ) =φ + δφ, Taking into account the contribution from molecules in Eq. 44, we then obtain the standard self-consistent equa-tion for∆ andφ The dispersion relation for quasi-particles is Eq.(48) is of the same form as the equation for ∆,φ derived in a zero-V limit where bosons and cooper pairs are non-interacting. However, in our case, the equation is valid even when various repulsive interactions are strong and bosons are in a Mott state. We derive the relation betweenφ and∆ from Eq. 48 as It is clear the saddle point values ofφ and∆ are locked with the same phase. For convenience, we assume both of them to be real in the following. First consider a situation when molecules are deeply in the Mott state, i.e. α ≫ Γ, ∆ > 0. Eq. 50 can be approximated up to Γ's second order as This equations shows that condensation amplitude of bosons is finite for any finite coupling Γ, disregarding the value of α. When Γ is nonzero, this equation indicates that the minimum for the total energy should be located at a finiteφ instead of zero.
On the other hand, if molecules are deeply in the superfluid state, i.e., −α ≫ Γ > 0, the solution to Eq. 50 can be approximated asφ where φ 0 is the saddle point value without the bosonfermion conversion, and φ ′ is the correction, It is worth remarking again that in this limit the phase ofφ is precisely locked with the phase of∆, following Eq.(50). This is consistent with the Hamiltonian-based discussion in section IV A.

VI. ODLO IN THE SINGLE BAND LIMIT
The main conclusions arrived so far do not depend on the large-N approximation introduced above. One can consider the opposite limit by assuming M = 1 and there is only one orbital degree of freedom at each lattice site. In the single-orbit limit, the two interaction terms (with two interaction constants λ and V f ) in H f (see Eq. (1)) can be rewritten in one term: In the limit where λ is much larger than V f , fermions are paired at each lattice site. Furthermore, I assume bosons have hard core interactions (V b = ∞) such that there can be only zero or one boson at each site.
So the low energy Hilbert subspace S k at each lattice site k consists of four states: 1) no Cooper pair, no boson; 2) no Cooper pair, one boson; 3) one Cooper pair, no boson; 4) one Cooper pair, one boson. They also correspond to a product of two pseudo spin S = 1/2 subspaces: S k is the on-site Hilbert space, and S f k,bk are the on-site pseudo spin spaces for fermions and bosons respectively. More explicitly, these four states are |vac f,b are the vacuum of fermion and bosons respectively. Finally, in this truncated subspace, the following identities hold So to have superfluidity, either σ bk or σ f k , or both of them need to have a finite expectation value in the XY plane. For instance to have fermionic superfluids, the expectation values of σ ± f k need to be nonzero. The effective Hamiltonian can then be written as (see also Ref.27) Here The Hamiltonian is invariant under a rotation around the z-axis or has an XY symmetry. The z-direction fully polarized phase of pseudo spins σ bk (σ f k ) represents the Mott phase of bosons (fermions), and the XY symmetry breaking states of pseudo spins σ bk (σ f k ) stand for the superfluid phase of bosons (fermions). The fermionic sector of this Hamiltonian was previously obtained and studied 29 ; it was also used to study BEC-BCS crossover in lattices 21 .
When Γ 1 F B = 0, the Mott phase for bosons with filling factor equal to one occurs when h z b is much larger than J b . Assume that in this case h z f is much less than J f so that σ f k are ordered in the xy plane; then fermions form Cooper pairs. Taking into account a finite amplitude of Γ 1 F B , one considers a solution where the pseudo spin symmetry of σ f k is spontaneously broken along a direction in the XY plane specified by σ f k (the expectation value is taken in the ground state).
In the molecular mean field approximation, the effective external field acting on pseudo spins σ bk is σ bk is calculated self-consistently in the ground state when h b,ef f is applied; the effective MFA Hamiltonian is One then arrives at the following self-consistent solution where σ bk has been projected along the direction of σ f k which lies in the XY plane. As mentioned before, development of such a component signifies superfluidity, or molecular condensation. To summarize, we have shown that certain Mott states are unstable with respect to the resonating fermion-boson conversion; in general superfluidity invades Mott phases because of the fermion-boson conversion.

VII. HIDDEN ORDER AND VORTICES IN MOTT STATES
In addition to introducing superfluidity to Mott states in some limit, the fermion-Boson conversion also results in a hidden order in Mott states. In the presence of fermion-boson conversion, one finds it is more convenient to introduce trilinear order parameters to characterize a Mott state A superfluid phase would have a nonzero order parameter of ∆ + bf type, but ∆ − bf can be either zero or nonzero. For a usual superfluid near Feshbach resonances, ∆ − bf is nonzero. However, there might be more exotic superfluids where ∆ − bf is zero; when this occurs, the superfluid will have two decoupled components with unlocked phases. On the other hand, a Mott state has vanishing ∆ + bf but always has non-vanishing ∆ − bf as hidden order as far as the fermion-boson conversion is present(see below). Here the appearance of ∆ − bf order is due to the fermion-boson conversion. Effectively, it can be viewed as an order parameter of a boson and a Cooper-pair hole pairing, which bears resemblance of the electron-hole exciton formation in semiconductors 13 and in quantum Hall bilayer systems 30 .
To understand this issue, we first consider an extreme situation when V f = V b = V bf = V 0 and all of them are much larger than t b,f and γ F B . Minimizing the potential energy leads to the following constraint on n f k,bk in the ground state Here I is an integer, Int[I + ǫ] is equal to I if 0 ≤ ǫ < 1/2, and to I + 1 if 1/2 < ǫ ≤ 1; at ǫ = 1/2, Int takes either I or I + 1. Let us assume that the chemical potentials and interactions are such that N f b is equal to an integer I. All states satisfy the constraint are degenerate when γ F R is zero, and thus the degeneracy is proportional to N f b .
In this limit, we can truncate the Hilbert space and consider the effect of Fermion-boson conversion in the degenerate subspace only. We study the following ground state trial wavefunctions constructed out of these degenerate states, where n kη = 0 or 1 satisfying η n kη = n kf b , w(n kη ) = u η at n kη = 0 and v η at n kη = 1, respectively (u η , v η are coherence factors). One can easily verify that states with different {φ kf b } are approximately orthogonal when N f b is much larger than unity. But any finite conversion leads to a lift of degeneracy. The energy associated with the conversion is Minimization takes place when φ kf b = 0 for any lattice site k. The symmetry here is broken not spontaneously as in superfluids but actually broken explicitly by the fermion-Boson conversion. The ground state is non-degenerate and does not have the usual U (1) vacuum manifold. This state is characterized by the following expectation values The existence of the trilinear order in Mott states is very unique and defines a hidden order. There are a few consequences. One is the collective excitations. In addition to excitations which have an energy gap V 0 , there are another branch of excitations involved the creation of a bosonic particle and annihilation of a cooper pair, q1,q2 b † q1+Q f q2η↑ f −q2+q1η↓ ; these excitations are gapped by the energy of order Γ F B instead of the Mott gap.
Furthermore, a hidden order also implies new classes of topological excitations. The wavefunction of a topological excitation centered at the origin is given by Eq.(64) where φ kf b is defined by the following equation Φ(R k ) is the azimuthal angle of R k . The vortex is orientated along the z-direction. The energy per unit length of this excitation unfortunately scales as the area of the system in the xy plane; i.e.
The situation discussed here is not generic and requires fine tuning. Let us now turn to a more general situation where V b = V f . If r b,f are much smaller than r bc,f c , then both Cooper pairs and bosons are in Mott states. Up to the first order approximation of Γ F B , the corresponding wavefunction is where the distribution n ′ kη satisfies η n ′ kη = n f /2 ± 1, and n kη satisfies η n kη = n f /2. φ kf b has to be uniform and zero for the ground state.
Similar calculations lead to self-consistent solutions ∆ f = ∆ b = 0 and more importantly, the following correlations for ∆ ± bf , The second equality above simply shows the absence of superfluidity. But the first one indicates a subtle hidden order in the Mott states under consideration. Notice that ∆ ± bf represent tri-linear order and are proportional to cos(φ kb ± φ kf ) .
One can easily show that the vortex wavefunction is given by the same expression but with φ kf b = Φ(R k ); the energy per unit length in this case is much smaller In this article, we examine the stability of bosonic Mott states under the influence of fermion-boson conversion and study various aspects of Mott states and superfluids when repulsive interactions among bosons are very strong. We have found that when bosonic Mott states are coupled to fermionic superfluids via the fermion-boson conversion, there appears to be a finite condensation fraction of bosons in the ground state. There are extra low energy states below the Mott gap representing gapless extended excitations in the Mott-superfluid mixture, due to delocalization of bosons. We also show the existence of off-diagonal long-range order in the bosonic channel due to fermion-boson conversion.
The second issue we look into here is the novel collective excitations in superfluids. This branch of excitations involves oscillations of the difference between the boson and fermion density. Unlike the usual Goldstone modes in superfluids, it is fully gapped. The gap energy is proportional to Γ F B when bosons are weakly interacting, or in a superfluid state.
At last, we study the Mott states of boson-fermion mixture and find hidden trilinear order in Mott states. We analyze the order in a few limits and briefly study the novel topological excitations in Mott states.
FZ is in part supported by a Discovery grant from NSERC, Canada and a research grant from the Dean of Science's office, UBC; FZ is also an A. P. Sloan fellow. CW is supported by the NSF Grant No. Phy99-07949. FZ wants to thank CASTU, Tsinghua University, Beijing and INT, University of Washington, Seattle for hospitalities. Finally, both authors want to thank ASPEN center for physics for its hospitality during the workshop "Quantum gases 2005".

APPENDIX A: THE EFFECTIVE ACTION
In this section, we present the effective action for the boson-fermion conversion in the optical lattices near Feshbach resonances. We start with the microscopic actions of Eq. 38, 39, 41, 42 in Sec. VB. Eq. 39 describes the attraction in the open channel for the formation of BCS Cooper pairs; Eq. 41 describes the Bose-Hubbard (BH) model of the boson molecules with t the hopping integral and U the on-site repulsion; Eq. 42 describes the conversion between the Cooper pairs and molecules.
The BH model of Eq. 41 exhibits a superfluid (SF)-Mott insulating (MI) phase transition 22 . The MI phase only exists in the strong coupling regime with commensurate fillings, i.e., small values of t/U and integer values of n b . The SF-MI transition can be obtained by two different ways. First, the boson filling is kept commensurate while t/U is tuned larger than the corresponding critical value. This transition belongs to the XY universal class, and the resulting SF is particle-hole symmetric. Second, we can also add or remove particles to the commensurate MI background, i.e., dope the MI with extra particles or holes. Because the particle-hole symmetry is broken, this transition is not XY-like. The resulting SF are either particle-like or hole-like. Consequently, although only one connected SF phase exists in the phase diagram, it actually exhibits rich structures, including the particlelike, hole-like, or even relativistic (particle-hole symmetric) SF, which are connected by smooth cross-overs.
The bare boson operators b k , b † k in Eq. 41 are for nonrelativistic particles. However, near the SF-MI transition, it is not convenient to use them to describe above rich structures in the SF phase. For example, b k means both an annihilation of a particle and a creation of a hole in the MI background. On the other hand, the SF-MI transition is in the strong coupling regime by using the bare operators of b k , b † k , and it is hard to do perturbation theory for the Hubbard U term. Thus we follow Fisher et al. 22 to introduce another complex bose field φ( r) to describe the molecular superfluidity. This can be formally done by keeping the on-site Hubbard term in Eq. 41 as the leading term, and decoupling the inter-site hopping term as perturbations. Basically, this transformation turns the original strongly interacting non-relativistic systems into weakly interacting quasirelativistic systems. It is shown in the equation of motion that φ plays the role of the expectation value of b in the ground state, i.e., φ is the superfluid order parameter.
Phenomenologically, the symmetry allows an effective action for the φ-field upto the quartic level as S φ = β 0 dτ d r φ † ( r) r ′ ∂ ∂τ + r ∂ 2 ∂τ 2 − κ∇ 2 + α φ( r) which includes the r, κ, α, λ terms as in the standard relativistic complex φ 4 theory, and also an additional first order time derivative term of r ′ . Whether the mass α > 0 or α < 0 determines the system either in the MI phase with a charge gap or in the SF phase, respectively. All these coefficients in Eq. A1 can be determined by the values of t, U, µ in the original BH model perturbatively 22 . However, for simplicity we treat them as phenomenological parameters. It is proved through gauge invariance that r ′ is related with α through 22 Near the SF-MI transition, as the filling n b changes from one integer to another integer, the superfluidity is enhanced and suppressed alternatively. As a result, α oscillates, and then r ′ can be negative, positive, or even zero. Roughly speaking, when n is larger (smaller) than an integer number, r ′ > 0 (r ′ < 0), and then the system is particle-like (hole-like). As long as r ′ = 0, the first order time derivative term dominates over the second order one below a certain energy scale in the sense of