It is important to estimate the vibration characteristic of structures for the wind resistant design or the seismic design. Especially, the influence of the damping characteristic has on the vibration is large. In general, the Fourier analysis is used for the evaluation of the vibration characteristics. However, these methods are due to the statistical method based on the stationary stochastic process, and attention is required to apply to nonstationary phenomenon. Then, this paper proposes the technique for evaluating the work of external force, the increment of vibration energy and the dissipation of the structural energy from the generalization wind force and the building response by using the wavelet transform that is effective to handle nonstationary phenomenon. Moreover, this technique was applied to the field measurement result of using the square pillar elastic model, and the relation between the work of the wind force and the vibration energy was shown.