The physics of interacting particles confined to a lattice defines a vast field of research, for both experimentalists and theoreticians interested in quantum phenomena. Ultracold atoms trapped by interfering laser beams provide the opportunity for generating such lattice-confined many-body systems under nearly ideal experimental conditions. The great purity and exquisite experimental control at nanokelvin temperatures, together with a wide range of design choices — such as the ability to tune interactions and the option of choosing a desired dimensionality — offer a route to testing theoretical models and exploring new types of quantum matter. The past decade has seen a surge in the number of studies using cold matter in optical lattices, and among the most outstanding examples are arguably the realizations of Mott-insulating phases for both bosons1 and fermions2,3. These achievements notwithstanding, the field certainly hasn't reached its maturity yet, and more ambitious studies of quantum phases seem possible, among them fermionic anti-ferromagnetism and d-wave superfluidity.

Now, reporting in Nature Physics, Seiji Sugawa and colleagues4 redeem such promise in a study of quantum phases of a two-component mixture of bosons and fermions in an optical lattice. Their most striking observation is that of a dual Mott-insulating state, where none of the individual components is a Mott insulator, but the mixture behaves as one, owing to the interactions between the components. Moreover — and importantly — by comparing their experimental findings to theoretical models, Sugawa et al. reveal that the observation of the fragile new state depends on a novel cooling mechanism, which relies on a change of entropy in the gas. As ever lower temperatures are needed to reach more complex quantum phases, such a new cooling method is warmly welcomed.

The transition from a conducting to an insulating state, as first proposed by Mott5, can be described by the so-called Hubbard model6. For a gas of particles in a lattice potential the behaviour is governed, at sufficiently high filling and sufficiently low temperature, by two main parameters: first, by the probability of the atoms to tunnel between lattice sites, which can be controlled by varying the depth of the optical-lattice potential and hence by the intensity of the lasers; second, by the interaction between atoms occupying the same lattice site, which depends on the nature of the atomic species trapped. As the strength of the lattice potential is increased, tunnelling between lattice sites becomes less likely and therefore sustaining phase correlation in the gas more difficult. At high-enough (but finite) lattice depth, tunnelling is suppressed altogether and phase ordering is lost. The gas then becomes an insulator owing to repulsive interaction, with typically exactly one particle at each lattice site.

In their extensive work, Sugawa et al.4 have loaded ytterbium atoms at a temperature of about 40 nK into a cubic optical lattice. Several isotopes of ytterbium can be trapped, and by choosing specific isotope combinations, two-component Bose–Fermi mixtures with repulsive or attractive interactions can be created. To enter the insulating regime, Sugawa and colleagues start with a single-component bosonic gas in the insulating state and increases the admixture of the fermionic isotope. For the case of repulsive interaction, they observe a dual-insulator phase, where the probabilities of a lattice site being occupied by a boson or a fermion are 50% each, hence adding to unity occupation; double and pair occupancies are suppressed. On further increasing the admixture of fermions, repulsive interactions start to dominate, and as the external harmonic-trapping potential offsets the lattice potential, phase separation into an inner core of bosonic and an outer shell of fermionic Mott phases occurs. For the case of attractive interactions, composite particles comprising one or more bosons and fermions appear localized simultaneously at the same lattice site.

Lacking direct spatially resolving probes for the different quantum phases, Sugawa et al.4 rely on modelling the trapped gas to support their findings. From the solutions of a simplified Hubbard model, different site-occupancy numbers can be inferred and integrated over the spatial profile of the gas. Most notable in the measurements are the absence of double occupancies of bosons and fermions and of Bose–Fermi pairs, which would be indicative of the respective insulating states. The fraction of pair occupancies is measured by photo-association of pairs into molecules, which leads to their loss from the trap. Sugawa et al. further support their findings by verifying strong nearest-neighbour correlations between the components, which is necessarily occurring in the dual Mott state and absent in the case of phase separation. In their model, the dual Mott phase occurs only at temperatures significantly lower than the experimental temperature of the single-component gas of bosons after adiabatic loading into the optical lattice. Sugawa et al. interpret this remarkable finding as adiabatic cooling of the gas, attributed to a change in entropy arising from the increase in the number of states available to the system as the fermionic admixture is increased. Measurements on the attractive mixture are consistent with heating due to a decrease in the number of available states arising from the formation of localized particles.

The work of Sugawa and colleagues4 highlights the opportunities that atomic mixtures, in particular, offer for exploring novel types of quantum matter. As straightforward temperature measurements seem unlikely in this field, thermometry will need to rely on accurate comparisons with underlying theoretical models. Finally, it would be most exciting if these experiments could be combined with recent advances involving spatially resolved detection of atoms in optical lattices7,8,9.