Hydrogels are often employed for tissue engineering and moistening applications. However, they are rarely used for load-bearing purposes because of their limited stiffness and the stiffness–toughness compromise inherent to them. By contrast, nature uses hydrogel-based materials as scaffolds for load-bearing and protecting materials by mineralizing them. Inspired by nature, the stiffness or toughness of synthetic hydrogels has been increased by forming minerals, such as CaCO₃, within them. However, the degree of hydrogel reinforcement achieved with CaCO₃ remains limited. To address this limitation, we form CaCO₃ biominerals in situ within a model hydrogel, poly(acrylamide) (PAM), and systematically investigate the influence of the size, structure, and morphology of the reinforcing CaCO₃ on the mechanical properties of the resulting hydrogels. We demonstrate that especially the structure of CaCO₃ and its affinity to the hydrogel matrix strongly influence the mechanical properties of mineralized hydrogels. For example, while the fracture energy of PAM hydrogels is increased 3-fold if reinforced with individual micro-sized CaCO₃ crystals, it increases by a factor of 13 if reinforced with a percolating amorphous calcium carbonate (ACC) nano-structure that forms in the presence of a sufficient quantity of Mg²⁺. If PAM is further functionalized with acrylic acid (AA) that possesses a high affinity towards ACC, the stiffness of the hydrogel increases by a factor 50. These fundamental insights on the structure-mechanical property relationship of hydrogels that have been functionalized with in situ formed minerals has the potential to enable tuning the mechanical properties of mineralized hydrogels over a much wider range than what is currently possible.