Relationship between the residual stress generated by cold-rolling and the absorbed energy in Charpy impact test of LS type specimens of a 5083 aluminum alloy were investigated. The residual stress by cold-rolling of about 30% was measured using X-ray technique. Absorbed energy was obtained at 77K using an instrumented Charpy impact test equipment. The residual stress was accurately measured by assuming the plane stress condition for the specimen. The macroscopic residual stress was different depending on the chemical composition and microstructure of specimens. The microscopic residual stress estimated from the half-value breadth (FWHM) change showed good correlation with the micro Vickers hardness. The absorbed energy decreased with the decrease in the microscopic residual stress. Further investigation was required to understand the relation between the macroscopic residual stress and the fibrous structure formed by cold-rolling.