In this paper, we propose an estimation method of a starting point of a rally, and a score transition from a table tennis match video. First, a rally scene is estimated from a table tennis video. The rally scene is estimated with utilizing an appearance frequency, and a hue similarity of the rally scene. Next, a server is estimated from the rally scene to analyze a player's posture. In addition, the score transition is estimated from a scoreboard in the rally scene. As a result of an evaluation experiment, the proposed method is used by a table tennis coach, and its contribution is confirmed.