To understand the 2D triangular Heisenberg antiferromagnetic system, we investigated the magnetic structures and the dynamics of ${\mathrm{Lu}}_y{\mathrm{Y}}_{1-y}{\mathrm{MnO}}_3$ in detail. The substitutions are adjusted to the Mn atomic position close to $x_{\text{Mn}}=\frac{1}{3}$. The neutron powder diffraction data claims that the magnetic structure of ${\mathrm{Lu}}_y{\mathrm{Y}}_{1-y}{\mathrm{MnO}}_3$ is described as a mixture of ${\mathrm{\Gamma }}_3$ ($P{6}_3^{\prime }c{m}^{\prime }$) and ${\mathrm{\Gamma }}_4$ ($P{6}_3^{\prime }{c}^{\prime }m$) at the $x_{\text{Mn}}$ position for $y=0.15$, 0.30, and 0.45. The ratio of ${\mathrm{\Gamma }}_3$ and ${\mathrm{\Gamma }}_4$ depends on temperature and composition and the fraction of ${\mathrm{\Gamma }}_3$ increases upon cooling, while no clear trimerization was observed at the $x_{\text{Mn}}$ position. We estimated exchange parameters from the analysis of the low-energy part of the spin waves. The results showed a weak trimerization effect on cooling because the nearest-neighbor exchange interaction is slightly enhanced. The temperature dependence of the spin-wave dispersion around the $\mathrm{\Gamma }$ point shows that the spin gap closes with increasing temperature because the exchange interactions in the nearest Mn-Mn neighbor become smaller. Gapless diffusive magnetic excitation from a Mn triangular lattice has been observed in a wide range in $Q$ and $E$ space of ${\mathrm{Lu}}_y{\mathrm{Y}}_{1-y}{\mathrm{MnO}}_3$. We found that ${\mathrm{Lu}}_{0.3}{\mathrm{Y}}_{0.7}{\mathrm{MnO}}_3$ could be an ideal case to investigate the trimerization, frustrated magnetism, and magnetoelastic coupling often observed in two-dimensional triangular lattice Heisenberg antiferromagnet systems.

• Received 28 August 2022
• Revised 8 March 2023
• Accepted 12 May 2023
• Corrected 14 June 2023

DOI:https://doi.org/10.1103/PhysRevB.107.214407