Non-combustible magnesium alloy garnered attention in lightweight automobiles. Nonetheless, its practical application necessitates the support of joining and welding techniques. Self-pierce riveting (SPR) has widespread adoption in aluminum alloys. However, there are few reports concerning non-combustible magnesium alloy SPR joints. This study investigated the fatigue property of Mg-4%Al-1%Ca-0.2%Mn (hereinafter referred to as AX41) alloy similar and dissimilar material SPR joints based on failure mechanism analysis. AA6061 (Al-1%Mg-0.05%Si) and SPCC (Steel Plate Cold Commercial) were used for the dissimilar material joints. The influencing factors on strength were discussed based on the result of the fatigue test. According to fatigue test results, except for the joint with SPCC as the upper sheet, the fatigue life of dissimilar material joints was longer than that of similar material joints. According to the observation of the fracture surface, the processing cracks near the rivet foot act as a secondary influencing factor of fatigue strength properties. The processing cracks near the rivet foot may induce rotation of a rivet, which makes the lower sheet easy to bend. To investigate the influence of bending stiffness on fatigue strength, different thicknesses of an upper and a lower sheet were prepared to test. The test results affirmed that bending stiffness is the primary influencing factor of fatigue strength. The bending stiffness ratio of the upper/lower sheets directly affects the failure mode of SPR joints.