We discuss the theory of mixtures of bosonic and fermionic atoms in periodic potentials at zero temperature. We derive a general Bose-Fermi Hubbard Hamiltonian in a one-dimensional optical lattice with a superimposed harmonic trapping potential. We study the conditions for linear stability of the mixture and derive a mean-field criterion for the onset of a bosonic superfluid transition. We investigate the ground-state properties of the mixture in the Gutzwiller formulation of mean-field theory, and present numerical studies of finite systems. The bosonic and fermionic density distributions and the onset of quantum phase transitions to demixing and to a bosonic Mott-insulator are studied as a function of the lattice potential strength. The existence is predicted of a disordered phase for mixtures loaded in very deep lattices. Such a disordered phase possessing many degenerate or quasidegenerate ground states is related to a breaking of the mirror symmetry in the lattice.

• Received 11 April 2003

DOI:https://doi.org/10.1103/PhysRevA.68.023606