Hand-pose is one of the most important communication tools in human’s daily life. To enhance the human-computer interaction, the objective of this research is to develop a system that automatically traces the hand-pose trajectory using video sequences. Here, the hand-pose trajectory is defined as the path of position changes of fingertips, with the assumption that the hand-pose remains invariant. Our system includes: preprocessing, segmentation of palm & fingers, feature extraction, and trajectory tracking. The core techniques involve a rule-based approach for the segmentation of palm and fingers, and methods to hand-pose feature extraction and fingertips’ trajectory tracking. The experimental results demonstrated that our system is able to successfully segment fingers from a human hand, to trace the trajectories of single or multiple fingertips given a specific hand-pose, and to re-trace the trajectories upon hand-pose changes. In conclusion, our system could ultimately be incorporated in a computer user-interface that uses a video camera as an input device, and interprets the hand-pose in motion. In addition, our system may offer a cost-effective solution to other motion capturing systems that often require special motion sensors or markers.