We proposed a system for reproducing a soccer game from a video sequence, using animation. The system firstly calibrates camera parameters with white lines drawn on the field, then recovers position of the players and ball. Due to the calibration for each image, free camera panning is allowed. Action of the players is estimated from a change in time of the position of the players and ball. For example, the system interprets that a player kicked the ball if the player is close to the bending point of the ball's locus. The system can judge 9 types of action with 42 variation. We chose this approach because size of players in video images is too small to recognize their pose. Finally, the system synthesizes motion of CG actors by modifying the standard sequence of human action. Parameters of the sequence are adjusted in order to fit moving speed of the CG actors to the measured speed from the video. If a player changes his action in a scene, selected actions are smoothly jointed. The synthesized scene can be viewed from any position on demands.