Direct numerical simulation of turbulent heat transfer over hemispherical roughness is carried out by the double-distribution function lattice Boltzmann method (LBM). The D3Q27 multiple-relaxation-time and D3Q19 regularized single-relaxation-time LBMs are applied to the flow and thermal fields, respectively. Three types of roughness pattern are considered at the friction Reynolds number of 660 and the Prandtl number of 0.71. For the wall boundary condition of the thermal fields, the constant heat flux condition is applied. The double (space and time) averaged turbulent heat transfer statistics are discussed. From the analysis of the budget terms in the double averaged stress and heat flux equations, it is found that although each of the Reynolds and dispersion stresses is significantly affected by the roughness patterns, the total stress is rather insensitive to the roughness patterns. For the thermal field, the same trend is observed in the behaviours of the turbulent and dispersion heat fluxes. It is then found that the hemispherical roughness affects more the friction than the heat transfer because the hemispheres significantly increase the pressure drag which is the main factor for friction but not for heat transfer.