The multiple-reference-frame motion estimation (ME) is one of the features to improve the compression efficiency. However, the computational complexity for prediction increases in proportion to the number of reference frames. This paper proposes the reference frame selection algorithm for a hardware-based HEVC encoder. The integer-ME explores multiple reference frames to find the best one and the fractional ME is then performed for the best reference frame which is determined by the IME. Simulation results show that a significant time saving of 74% is achieved with a negligible drop in compression efficiency.