We studied the scattering process of He molecules on a single wall carbon nanotube (SWNT) using molecular dynamics simulations and evaluated the collisional energy transfer between SWNT and He molecule. The decrease of the incident angle enhances the energy exchange between SWNT and He molecule because of the surface structure which deforms easily in radial direction. In contrast, the deflection angle does not substantially affect the amount of the energy transfer. In order to analyze the effect of the anisotropic surface structure on the scattering process in detail, we decomposed the amount of the energy transfer into the normal and the tangential component and found that the temperature dependence of the post-collisional energy is mainly due to the change of the normal component while the tangential one is almost independent of the temperature.