Ultrafine-grained specimens with the average grain size of less than 2μm and medium-grained specimens were prepared from steel plates produced by an advanced thermo-mechanical control process. The smooth specimens were fatigued under cyclic axial tension compression at room temperature and the crack initiation and early propagation behaviors were investigated by atomic force microscopy. In the medium-grained specimens, the fatigue cracks initiated from the simple slip lines. On the other hand, in the ultrafine-grained specimens, complex slip deformation was formed in the vicinity of grain boundaries prior to the initiation of fatigue cracks. The fatigue cracks were nucleated at the grain boundary between the grains with the complex slip deformation. The crack propagation rate in the early stage decreases because of the grain-boundary blocking and crack deflection in the ultrafine-grained specimens.