We describe the localization transition of superfluids on two-dimensional lattices into commensurate Mott insulators with average particle density $p∕q$ ($p,q$ relatively prime integers) per lattice site. For bosons on the square lattice, we argue that the superfluid has at least $q$ degenerate species of vortices which transform under a projective representation of the square-lattice space group (a PSG). The formation of a single-vortex condensate produces the Mott insulator, which is required by the PSG to have density wave order at wavelengths of $q∕n$ lattice sites ($n$ integer) along the principle axes; such a second-order transition is forbidden in the Landau-Ginzburg-Wilson frame-work. We also discuss the superfluid-insulator transition in the direct boson representation and find that an interpretation of the quantum criticality in terms of deconfined fractionalized bosons is only permitted at special values of $q$ for which a permutative representation of the PSG exists. We argue [and demonstrate in detail in a companion paper: L. Balents et al., following paper, Phys. Rev. B 71, 144509 (2005)] that our results apply essentially unchanged to electronic systems with short-range pairing, with the PSG determined by the particle density of Cooper pairs. We also describe the effect of static impurities in the superfluid: the impurities locally break the degeneracy between the $q$ vortex species, and this induces density-wave order near each vortex. We suggest that such a theory offers an appealing rationale for the local density-of-states modulations observed by Hoffman et al. [Science 295, 466 (2002)], in scanning tunneling microscopy (STM) studies of the vortex lattice of ${\mathrm{Bi}}_2{\mathrm{Sr}}_2{\mathrm{CaCu}}_2{\mathrm{O}}_{8+\delta }$ and allows a unified description of the nucleation of density-wave order in zero and finite magnetic fields. We note signatures of our theory that may be tested by future STM experiments.

• Received 6 September 2004

DOI:https://doi.org/10.1103/PhysRevB.71.144508