Skip to main content
SpringerLink
Log in

New algorithm for 3D facial model reconstruction and its application in virtual reality

  • Pattern Recognition and Image Processing
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

3D human face model reconstruction is essential to the generation of facial animations that is widely used in the field of virtual reality (VR). The main issues of 3D facial model reconstruction based on images by vision technologies are in twofold: one is to select and match the corresponding features of face from two images with minimal interaction and the other is to generate the realistic-looking human face model. In this paper, a new algorithm for realistic-looking face reconstruction is presented based on stereo vision. Firstly, a pattern is printed and attached to a planar surface for camera calibration, and corners generation and corners matching between two images are performed by integrating modified image pyramid Lucas-Kanade (PLK) algorithm and local adjustment algorithm, and then 3D coordinates of corners are obtained by 3D reconstruction. Individual face model is generated by the deformation of general 3D model and interpolation of the features. Finally, realisticlooking human face model is obtained after texture mapping and eyes modeling. In addition, some application examples in the field of VR are given. Experimental result shows that the proposed algorithm is robust and the 3D model is photo-realistic.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lee Y C, Terzopoulos D, Waters K. Realistic modeling for facial animation. InProc. ACM SIGGRAPH 95, Los Angeles, USA, 1995, pp.56–62.

  2. DeCarlo D, Metaxas D, Stone M. An anthropometric face model using variational techniques. InProc. ACM SIGGRAPH 98, Orlando, USA, 1998, pp.67–74.

  3. Parke F I, Waters K. Computer Facial Animation. AK Peters, 1996.

  4. Hong P. An integrated framework for face modeling, facial motion analysis and synthesis [Dissertation]. University of Illinois at Urbana Champaign, 2001.

  5. Fua P, Leclerc Y G. Object-centered surface reconstruction: Combining multi-image stereo and shading.Int. J. Computer Vision, 1995, 16(1): 35–56.

    Article  Google Scholar 

  6. Samaras D, Metaxas D. Incorporating illumination constraints in deformable models. InProc. the IEEE 1998 Conf. Computer Vision and Pattern Recognition (CVPR), 1998, Santa Barbara, USA, pp. 322–329.

  7. Tang L, Huang T S. Analysis-based facial expression synthesis. In1996 IEEE Int. Conf. Image Processing (ICIP), 1996, Lausanne, Switzerland, ICIP-III (94): 98–102.

  8. Wong K, Cipolla R. Structure and motion from Silhouettes. InProc. 8th IEEE Int. Conf. Computer Vision (ICCV), 2001, Vancouver, Canada, pp. 217–222.

  9. Pighin F, Hecker J, Lischinski Det al. Synthesizing realistic facial expressions from photographs.Computer Graphics, Annual Conference Series, SIGGRAPH, 1998, pp.75–84.

  10. Horace H S, Yin L. Constructing a 3D individual head model from two orthogonal views.The Visual Computer, 1996, 12(5): 254–266.

    Google Scholar 

  11. Lee W S, Thalmann N M. Head modeling from pictures and morphing in 3D with image metamorphosis based on triangulation. Inthe Int. Workshop on Modelling and Motion Capture Techniques for Virtual Environments (CAPTECH), Geneva, Switzerland,LNAI 1537, Springer-Verlag, 1998, pp.254–267.

  12. Waters K. A muscle model for animating threedimensional facial expressions.Computer Graphics, 1987, 21(4): 17–24.

    Article  MathSciNet  Google Scholar 

  13. Lavagetto F, Pockaj R. The facial animation engine: Toward a high-level interface for the design of MPEG-4 compliant animated faces.IEEE Trans. Circuits Syst. Video Technol., 1999, 9(2): 277–289.

    Article  Google Scholar 

  14. Fua P. Face models from uncalibrated video sequences. Inthe Int. Workshop on Modelling and Motion Capture Techniques for Virtual Environments (CAPTECH), 1998, Geneva, Switzerland,LNAI 1537, Springer-Verlag, pp. 214–222.

  15. Fua P. Using model-driven bundle-adjustment to model heads from raw video sequences. InProc. 7th IEEE Int. Conf. Computer Vision (ICCV), 1999, Kerkyra, Corfu, Greece, pp.46–53.

  16. Blanz V, Vetter T. A morphable model for the synthesis of 3D faces. InComputer Graphics, Annual Conference Series, SIGGRPAH, 1999, pp.187–194.

  17. Liu Z, Zhang Z, Jacobs Cet al. Rapid Modeling of Animated Faces from Video. 2000. http://research.microsoft.com/research/pubs/trpub.aspx.

  18. Zhang Z. Flexible camera calibration by viewing a plane from unknown orientations. InProc. 7th IEEE Int. Conf. Computer Vision (ICCV), Kerkyra, Corfu, Greece, 1999, pp.666–673.

  19. Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision, InProc. Seventh Int. Joint Conference on Artificial Intelligence (IJCAI), 1981, Vancouver, Canada, pp.674–679.

  20. Shi J, Tomasi C. Good feature to track. InProc. the IEEE 1994 Conf. Computer Vision and Pattern Recognition (CVPR) 1994, Seattle. USA, pp.593–600.

  21. Jia B, Zhang Y, Lin X. General and fast algorithm for disparity error detection and correction.J. Tsinghua University, 2000, 40(1): 28–31 (in Chinese)

    Google Scholar 

  22. Zhang Z, Deriche R, Faugeras Oet al. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry.Artificial Intelligence, 1995, 78(1–2): 87–119.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science, Zhejiang University, 310027, Hangzhou, P. R. China

    Rong-Hua Liang & Chun Chen

  2. College of Information and Engineering, Zhejiang University of Technology, 310014, Hangzhou, P.R. China

    Rong-Hua Liang

  3. State Key Laboratory of CAD and CG, Zhejiang University, 310027, Hangzhou, P.R. China

    Zhi-Geng Pan

Authors
  1. Rong-Hua Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi-Geng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong-Hua Liang.

Additional information

This research is supported by the National Grand Fundamental Research 973 Program of China under Grant No.2002CB312100, Excellent Youth NSF of Zhejiang Province under Grant No.RC40008, and Zhejiang Provincial Natural Science Foundation under Grant Nos.602134 and Z603262.

Rong-Hua Liang received his Ph.D. degree from Zhejiang University in 2003. He is now a faculty member in in College of Information and Engineering, Zhejiang University of Technology. His current research interests include computer vision, computer graphics, and artificial intelligence.

Zhi-Geng Pan received his Ph.D. degree from Zhejiang University in 1993. He is now a professor of State Key Lab of CAD&CG, Zhejiang University. He is also a Ph.D. supervisor of College of Computer Science, Zhejiang University. His current research interests include computer vision, computer graphics, and VR.

Chun Chen received his Ph.D. degree from Zhejiang University in 1990. He is now the dean and a professor of College of Computer Science, Zhejiang University. He is also a Ph.D. supervisor of College of Computer Science, Zhejiang University. His current research interests include computer vision, image processing, and CSCW.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liang, RH., Pan, ZG. & Chen, C. New algorithm for 3D facial model reconstruction and its application in virtual reality. J. Compt. Sci. & Technol. 19, 501–509 (2004). https://doi.org/10.1007/BF02944751

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02944751

Keywords

Search

Navigation