P-type elemental Te has exhibited a high thermoelectric (TE) figure of merit zT of ∼1.0 around 600 K, but its performance near room temperature is much deteriorated due to grain boundary scattering. In this work, the average zT of Te_0.98As_0.02 alloys is remarkably enhanced via increasing the grain size. Through increasing the grain size, the room temperature power factor is boosted from 0.44 × 10⁻³ W m⁻¹ K⁻² to 1.2 × 10⁻³ W m⁻¹ K⁻² due to the suppression of grain boundary scattering, while the lattice thermal conductivity remains unchanged, resulting in an ∼32% enhancement of device zT. The TE performance is further enhanced via Se alloying to reduce the lattice thermal conductivity. In the end, a high device zT of ∼0.64 at 600 K with a calculated conversion efficiency ∼8% is realized in the grain size-optimized Te_0.94Se_0.04As_0.02 alloys, about 60% and 45% increments over that of the initial Te_0.98As_0.02 sample, respectively. The peak zT is maintained at ∼1.0 at 600 K. The strategy used here should be instructive for optimizing the TE properties of other materials with remarkable grain boundary scattering.