Implementing the hierarchical structures of non-noble-metal-based electrocatalysts and modulating their composition can help accelerate surface reactions and fulfill the promise of renewable energy devices via water splitting. Herein, molybdenum-based compounds are constructed on activated nickel foam (act-NF) by a one-step hydrothermal growth. The product generated on the act-NF is Ni_xCo_1−xMoO₄@CoMoO₄, with a novel 3D hierarchical heterostructure, wherein the one-dimensional CoMoO₄ nanorods are hierarchically integrated with the two-dimensional Ni_xCo_1−xMoO₄ nanosheets (NCMO@CMO/act-NF). The formation of Ni_xCo_1−xMoO₄@CoMoO₄ attributes to the release and diffusion of Ni²⁺ from act-NF. Heterogeneous Ni_xCo_1−xMoO₄@CoMoO₄ has compositional differences, and synergistic interaction between cobalt and nickel results in the modulated electronic states. Meanwhile, the hierarchical structure facilitates the exposure of active sites. Combining these two advantages, NCMO@CMO/act-NF presents a low η₁₀ value of 61 and 180 mV in 1.0 M KOH for the HER and OER, respectively, and it shows a low cell voltage of 1.46 V for overall water splitting with robust stability. DFT calculations reveal that Ni doping leads to the charge depletion of Co, which further optimizes the d-band center of metal sites and tunes the adsorption of adsorbates to facilitate the water splitting reaction. Thus, a promising strategy of incorporating the nanostructure design with compositional modulation is presented to develop functional materials for energy conversion.