Monte Carlo simulations of the frustrated two-dimensional (2D) XY model were carried out at small commensurate values of the frustration f. For f=1/30 a single transition was observed at which phase coherence (finite helicity modulus) and vortex lattice orientational order vanish together. For f=1/56 a new phase in which phase coherence is absent but orientational order persists was observed. Where compariosn is possible, the results are in detailed agreement with the behavior of the lattice Coulomb gas model of vortices. It is argued that the helicity modulus of the frustrated 2D XY model vanishes for any finite temperature in the limit of weak frustration f.

• Received 4 January 1995

DOI:https://doi.org/10.1103/PhysRevB.51.11951