Jinxiang Chai
Skip Abstract Section
Skip Supplemental Material SectionAbstract
This paper introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals from the user's performance are supplemented by a database of pre-recorded human motion. At run time, the system automatically learns a series of local models from a set of motion capture examples that are a close match to the marker locations captured by the cameras. These local models are then used to reconstruct the motion of the user as a full-body animation. We demonstrate the power of this approach with real-time control of six different behaviors using two video cameras and a small set of retro-reflective markers. We compare the resulting animation to animation from commercial motion capture equipment with a full set of markers.
Supplemental Material
- Aha, D. 1997. Editorial, special issue on lazy learning. In Artificial Intelligence Review. 11(1--5):1--6. Google Scholar
Digital Library
- Arikan, O., and Forsyth, D. A. 2002. Interactive motion generation from examples. In ACM Transactions on Graphics. 21(3):483--490. Google ScholarDigital Library
- Arikan, O., Forsyth, D. A., and O'Brien, J. F. 2003. Motion synthesis from annotations. In ACM Transactions on Graphics. 22(3):402--408. Google ScholarDigital Library
- Atkeson, C. G., Moore, A. W., and Schaal, S. 1997a. Locally weighted learning. In Artificial Intelligence Review. 11(1--5):11--73. Google ScholarDigital Library
- Atkeson, C. G., Moore, A. W., and Schaal, S. 1997b. Locally weighted learning for control. In Artificial Intelligence Review. 11(1--5):75--113. Google ScholarDigital Library
- Badler, N. I., Hollick, M., and Granieri, J. 1993. Real-time control of a virtual human using minimal sensors. In Presence. 2(1):82--86.Google ScholarDigital Library
- Bazaraa, M. S., Sherali, H. D., and Shetty, C. 1993. Nonlinear Programming: Theory and Algorithms. John Wiley and Sons Ltd. 2nd Edition.Google Scholar
- Bishop, C. 1996. Neural Network for Pattern Recognition. Cambridge University Press. Google ScholarDigital Library
- Brand, M., and Hertzmann, A. 2000. Style machines. In Proceedings of ACM SIGGRAPH 2000. 183--192. Google ScholarDigital Library
- Brand, M. 1999. Shadow puppetry. In Proceedings of IEEE International Conference on Computer Vision. 1237--1244. Google ScholarDigital Library
- Bregler, C., and Omohundro, S. 1995. Nonlinear image interpolation using manifold learning. In Advances in Neural Information Processing Systems 7. 973--980.Google Scholar
- Cheung, G., Baker, S., and Kanade, T. 2003. Shape-from-silhouette of articulated object and its use for human body kinematics estimation and motion capture. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 77--84. Google ScholarDigital Library
- Daelemans, W., and Van De Bosch. A. 2001. Treetalk: Memory-based word phonemisation. In Data-Driven Techniques in Speech Synthesis, Kluwer. 149--172.Google Scholar
- Dontcheva, M., Yngve, G., and Popovic, Z. 2003. Layered acting for character animation. In ACM Transactions on Graphics. 22(3):409--416. Google ScholarDigital Library
- Freeman, W. T., Anderson, D., Beardsley, P., Dodge, C., Kage, H., Kyuma, K., Miyake, Y., Roth, M., Tanaka, K., Weissman, C., and Yerazunis, W. 1998. Computer vision for interactive computer graphics. In IEEE Computer Graphics and Applications. 18(3):42--53. Google ScholarDigital Library
- Fukunaga, K., and Olsen. D. 1971. An algorithm for finding intrinsic dimensionality of data. In IEEE Transactions on Computers. C-20:176--183.Google Scholar
- Grochow. K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Style-based inverse kinematics. In ACM Transactions on Graphics. 23(3):522--531. Google ScholarDigital Library
- Guo. S., and Roberge, J. 1996. A high level control mechanism for human locomotion based on parametric frame space interpolation. In Eurographics Workshop on Computer Animation and Simulation '96. 95--107. Google ScholarDigital Library
- Hinton. G., Revow, M., and Dayan, P. 1995. Recognizing handwritten digits using mixtures of linear models. In Advances in Neural Information Processing Systems 7. 1015--1022.Google Scholar
- Howe. N., Leventon, M., and Freeman, W. 1999. Bayesian reconstruction of 3d human motion from single-camera video. In Advances in Neural Information Processing Systems 12. 820--826.Google Scholar
- Konami Boxing and Police 911 GAME, 2001. http://www.konami.com.Google Scholar
- Kovar. L., and Gleicher, M. 2004. Automated extraction and parameterization of motions in large data sets. In ACM Transactions on Graphics. 23(3):559--568. Google ScholarDigital Library
- Kovar. L., Gleicher, M., and Pighin, F. 2002. Motion graphs. In ACM Transactions on Graphics. 21(3):473--482. Google ScholarDigital Library
- Lawrence, N. D. 2004. Gaussian process latent variable models for visualization of high dimensional data. In Advances in Neural Information Processing Systems 16. 329--336.Google Scholar
- Lee, J., Chai, J., Reitsma, P., Hodgins, J., and Pollard, N. 2002. Interactive control of avatars animated with human motion data. In ACM Transactions on Graphics. 21(3):491--500. Google ScholarDigital Library
- Li, Y., Wang, T., and Shum, H.-Y. 2002. Motion texture: A two-level statistical model for character synthesis. In ACM Transactions on Graphics. 21(3):465--472. Google ScholarDigital Library
- Mardia, K., Kent, J., and Bibby, M. 1979. Multivariate Analysis. Academy Press.Google Scholar
- MicroStrain 3DM-G, 2004. http://www.microstrain.com.Google Scholar
- Oore, S., Terzopoulos, D., and Hinton, G. 2002. A desktop input device and interface for interactive 3d character. In Proceedings of Graphics Interface 2002. 133--140.Google Scholar
- Pullen, K., and Bregler, C. 2002. Motion capture assisted animation: Texturing and synthesis. In ACM Transactions on Graphics. 21(3):501--508. Google ScholarDigital Library
- Ren, L., Shakhnarovich, G., Hodgins, J. K., Pfister, H., and Viola, P. A. 2004. Learning silhouette features for control of human motion. In Computer Science Technical Reports 2004, Carnegie Mellon University. CMU-CS-04-165.Google Scholar
- Rose, C., Cohen, M. F., and Bodenheimer, B. 1998. Verbs and adverbs: Multidimensional motion interpolation. In IEEE Computer Graphics and Applications. 18(5):32--40. Google ScholarDigital Library
- Roweis, S., and Saul, L. 2000. Nonlinear dimensionality reduction by locally linear embedding. In Science. 290(5500):2323--2326.Google ScholarCross Ref
- Safonova, A., Hodgins, J., and Pollard, N. 2004. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. In ACM Transactions on Graphics. 23(3):514--521. Google ScholarDigital Library
- Scholkopf, B., Smola, A., and Muller, K.-R. 1999. Kernel principal component analysis. In Advances in Kernel Methods-SV Learning, MIT Press. 327--352. Google ScholarDigital Library
- Semwal, S., Hightower, R., and Stansfield, S. 1998. Mapping algorithms for real-time control of an avatar using eight sensors. In Presence. 7(1):1--21. Google ScholarDigital Library
- Shin, H. J., Lee, J., Gleicher, M., and Shin, S. Y. 2001. Computer puppetry: An importance-based approach. In ACM Transactions on Graphics. 20(2):67--94. Google ScholarDigital Library
- Sidenbladh, H., Black, M. J., and Sigal, L. 2002. Implicit probabilistic models of human motion for synthesis and tracking. In European Conference on Computer Vision. 784--800. Google ScholarDigital Library
- Sony Eye Toy Systems, 2003. http://www.eyetoy.com.Google Scholar
- Stone, M. 1974. Cross-validatory choice and assessment of statistical predictions. In Journal of the Royal Statistical Society. 36:111--147.Google Scholar
- Tenenbaum, J., Silva, V., and Langford, J. 2000. A global geometric framework for nonlinear dimensionality reduction. In Science. 290(5500):2319--2323.Google ScholarCross Ref
- Vicon Systems, 2004. http://www.vicon.com.Google Scholar
- Wiley, D. J., and Hahn, J. K. 1997. Interpolation synthesis of articulated figure motion. In IEEE Computer Graphics and Applications. 17(6):39--45. Google ScholarDigital Library
- Xsens Mt-9, 2004. http://www.xsens.com.Google Scholar
- Xu, G., and Zhang, Z. 1996. Epipolar Geometry in Stereo, Motion, and Object Recognition: A Unified Approach. Kluwer. Google ScholarDigital Library
- Yamane, K., and Nakamura, Y. 2003. Natural motion animation through constraining and deconstraining at will. In IEEE Transactions on Visualization and Computer Graphics. 9(3):352--360. Google ScholarDigital Library
- Yamane, K., Kuffner, J. J., and Hodgins, J. K. 2004. Synthesizing animations of human manipulation tasks. In ACM Transactions on Graphics. 23(3):532--539. Google ScholarDigital Library
- Yin, K., and Pai, D. K. 2003. Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 329--338. Google ScholarDigital Library
- Zhang, Z. 1999. Flexible camera calibration by viewing a plane from unknown orientations. In Proceedings of the International Conference on Computer Vision. 666--673.Google ScholarCross Ref
Index Terms
- Performance animation from low-dimensional control signals
Recommendations
Performance animation from low-dimensional control signals
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersThis paper introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals ...
VideoMocap: modeling physically realistic human motion from monocular video sequences
SIGGRAPH '10: ACM SIGGRAPH 2010 papersThis paper presents a video-based motion modeling technique for capturing physically realistic human motion from monocular video sequences. We formulate the video-based motion modeling process in an image-based keyframe animation framework. The system ...
VideoMocap: modeling physically realistic human motion from monocular video sequences
This paper presents a video-based motion modeling technique for capturing physically realistic human motion from monocular video sequences. We formulate the video-based motion modeling process in an image-based keyframe animation framework. The system ...
Comments