The crystallography of the low-temperature phases (β′′) for shape memory alloys (NbRu and TaRu) has been debated for decades. Though a P2/m monoclinic structure has been proposed for the β′′ phase, the proposed structure is not able to completely represent the measured diffraction data. In this work, the crystallography of the β′′ phase was investigated by first-principles calculations. We showed that the previously reported P2/m monoclinic structure was lattice unstable due to the presence of the soft phonon mode. A P2₁/m monoclinic structure was derived from the P2/m monoclinic structure by displacing its atoms according to the eigenvector of the soft phonon mode at the Γ point. The P2₁/m and the P2/m monoclinic structures are structurally similar, but the former one is energetically and structurally more favorable than the latter one. We concluded that the β′′ phase preferred to crystallize in the P2₁/m monoclinic structure rather than the previously reported P2/m monoclinic structure. Our results offer guidance for the experimental determination of the crystallography of the β′′ phase for NbRu and TaRu.