Skip to main content
Log in

Building highly realistic facial modeling and animation: a survey

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper provides a comprehensive survey on the techniques for human facial modeling and animation. The survey is carried out from two different perspectives: facial modeling, which concerns how to produce 3D face models, and facial animation, which regards how to synthesize dynamic facial expressions. To generate an individual face model, we can either perform individualization of a generic model or combine face models from an existing face collection. With respect to facial animation, we have further categorized the techniques into simulation-based, performance-driven and shape blend-based approaches. The strength and weakness of these techniques within each category are discussed, alongside with the applications of these techniques to various exploitations. In addition, a brief historical review of the technique evolution is provided. Limitations and future trend are discussed. Conclusions are drawn at the end of the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abrantes, G.A., Pereira, F.: MPEG-4 facial animation technology: survey, implementation, and results. IEEE Trans. Circuits Syst. Video Technol. 9(2), 290–305 (1999)

    Article  Google Scholar 

  2. Allen, B., Curless, B., Popović, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003)

    Article  Google Scholar 

  3. Badler, N., Platt, S.: Animating facial expressions. ACM SIGGRAPH Comput. Graph. 15(3), 245–252 (1981)

    Article  Google Scholar 

  4. Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM SIGGRAPH, New York (1999) (URL: http://www.kyb.tuebingen.mpg.de/bu/people/volker/)

    Chapter  Google Scholar 

  5. Blanz, V., Scherbaum, K., Vetter, T., Seidel, H.-P.: Exchanging faces in images. Comput. Graph. Forum 23(3), 669–676 (2004)

    Article  Google Scholar 

  6. Chai, J.-X., Xiao, J., Hodgins, J.: Vision-based control of 3D facial animation. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 193–206. EUROGRAPHICS, San Diego (2003) (URL: http://faculty.cs.tamu.edu/jchai/projects/face-animation/)

    Google Scholar 

  7. Debevec, P.E.: Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 189–198. ACM SIGGRAPH, New York (1998)

    Chapter  Google Scholar 

  8. DeCarlo, D., Metaxas, D., Stone, M.: An anthropometric face model using variational techniques. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 67–74. ACM SIGGRAPH, New York (1998)

    Chapter  Google Scholar 

  9. Du, Y., Lin, X.: Emotional facial expression model building. Pattern Recognit. Lett. 24, 2923–2934 (2003)

    Article  Google Scholar 

  10. Farkas, L.: Anthropometry of the Head and Face, 2nd edn. Raven Press, New York (1994)

    Google Scholar 

  11. Gelder, A.V.: Approximate simulation of elastic membranes by triangulated spring meshes. J. Graph. Tools 3(2), 21–42 (1998)

    Google Scholar 

  12. Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F.: Making faces. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 55–66. ACM SIGGRAPH, Boston, MA (1998)

    Chapter  Google Scholar 

  13. Hiwada, K., Maki, A., Nakashima, A.: Mimicking video: real-time morphable 3D model fitting. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 132–139. ACM SIGGRAPH, Osaka (2003)

    Google Scholar 

  14. Joshi, P., Tien, W.C., Desbrun, M., Pighin, F.: Learning controls for blend shape based realistic facial animation. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 187–192. EUROGRAPHICS, San Diego (2003)

    Google Scholar 

  15. Kähler, K., Haber, J., Yamauchi, H., Seidel, H.-P.: Head shop: generating animated head models with anatomical structure. In: Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 55–63. EUROGRAPHICS, San Antonio, TX (2002) (URL http://www.mpi-inf.mpg.de/∼kaehler/slides/sca02-headshop_files/ v3_document.htm)

    Chapter  Google Scholar 

  16. Kähler, K., Haber, J., Seidel, H.-P.: Reanimating the dead: reconstruction of expressive faces from skull data. ACM Trans. Graph. 22(3), 554–561 (2003)

    Article  Google Scholar 

  17. Kalra, P., Garchery, S., Kshirsagar, S.: Facial deformation models. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Handbook of Virtual Humans, chap. 6. John Wiley & Sons, West Sussex, England (2004)

  18. Koch, A.: Structured design implementation – a strategy for implementing regular data paths on FPGAs. In: Proceedings of the 1996 ACM 4th International Symposium on Field Programmable Gate Arrays, pp. 151–157. ACM SIGDA, Monterey, CA (1996)

    Chapter  Google Scholar 

  19. Kshirsagar, S., Egges, A., Garchery, S.: Expressive speech animation and facial communication. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Handbook of Virtual Humans, chap. 10. John Wiley & Sons, West Sussex, England (2004)

  20. Lee, Y., Terzopoulos, D., Waters, K.: Realistic modeling for facial animation. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 55–62. ACM SIGGRAPH, New York (1995)

    Book  Google Scholar 

  21. Lee, W., Goto, T., Kshirsagar, S., Molet, T.: Face cloning and face motion capture. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Handbook of Virtual Humans, chap. 2. John Wiley & Sons, West Sussex, England (2004)

  22. Litwinowicz, P., Williams, L.: Animating images with drawings. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, pp. 409–412. ACM SIGGRAPH, New York (1994)

    Chapter  Google Scholar 

  23. Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 271–276. ACM SIGGRAPH, New York (2001)

    Chapter  Google Scholar 

  24. Magnenat-Thalmann, N., Thalmann, D.: Handbook of Virtual Humans. John Wiley & Sons, West Sussex, England (2004)

    Google Scholar 

  25. Marschner, S.R., Greenberg, D.P.: Inverse lighting for photography. In: IST/SID 5th Colort Imaging Conference, pp. 262–265. IS&T, Scottsdale (1997) (URL: http://www.graphics.cornell.edu/pubs/1997/MG97.html)

    Google Scholar 

  26. Noh, J., Neumann, U.: A Survey of Facial Modeling and Animation Techniques. USC Technical Report 99-705, Integrated Media Systems Center, University of Southern California (1998)

  27. Parke, F.: Computer generated animation of faces. In: Proceedings of the ACM Annual Conference, pp. 451–457. ACM, Boston, MA (1972)

    Chapter  Google Scholar 

  28. Parke, F.I.: A Parametric Model for Human Faces. PhD Thesis, University of Utah, Salt Lake City, UTEC-CSc-75-047, USA (1974)

  29. Parke, F.I., Waters, K.: Computer Facial Animation. AK Peters, Wellesley, MA (1996)

    Google Scholar 

  30. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23(3), 664–672 (2004)

    Article  Google Scholar 

  31. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 75–84. ACM SIGGRAPH, New York (1998)

    Chapter  Google Scholar 

  32. Pighin, F., Szeliski, R., Salesin, D.: Resynthesizing facial animation through 3D model-based tracking. In: Proceedings of the 7th IEEE International Conference on Computer Vision, vol. 1, pp. 143–150. IEEE Computer Society, Los Alamitos, CA, USA (1999)

    Google Scholar 

  33. Pratscher, M., Coleman, P., Laszlo, J., Singh, K.: Outside-in anatomy based character rigging. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 329–338. Eurographics, Los Angeles (2005)

    Chapter  Google Scholar 

  34. Seo, H., Magnenat-Thalmann, N.: An automatic modeling of human bodies from sizing parameters. In: Proceedings of the 2003 Symposium on Interactive 3D Graphics, pp. 19–26. ACM SIGGRAPH, Monterey, CA (2003)

    Chapter  Google Scholar 

  35. Seo, H., Cordier, F., Magnenat-Thalmann, N.: Synthesizing animatable body models with parameterized shape modifications. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 120–125. Eurographics, San Diego (2003)

    Google Scholar 

  36. Shashua, A., Riklin-Raviv, T.: The quotient image: class-based re-rendering and recognition with varying illuminations. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 129–139 (2001)

    Article  Google Scholar 

  37. Sifakis, E., Neverov, I., Fedkiw, R.: Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24(3), 417–425 (2005)

    Article  Google Scholar 

  38. Terzopoulos, D., Waters, K.: Physically-based facial modeling, analysis, and animation. Vis. Comput. Animation 1, 73–80 (1990)

    Google Scholar 

  39. Tu, P.-H., Lin, I.-C., Yeh, J.-S., Liang, R.-H., Ouhyoung, M.: Surface detail capturing for realistic facial animation. J. Comput. Sci. Technol. 19(5), 618–625 (2004)

    Article  Google Scholar 

  40. Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. ACM Trans. Graph. 24(3), 426–433 (2005)

    Article  Google Scholar 

  41. Ward, K., Bertails, F., Kim, T.Y., Marschner, S.R., Cani, M.P., Lin, M.C.: A survey on hair modeling: styling, simulation, and rendering. IEEE Trans. Vis. Comput. Graph. 13(2), 213–234 (2007) (URL: http://www.cs.unc.edu/∼wardk/research.html)

    Article  Google Scholar 

  42. Waters, K.: A muscle model for animating three-dimensional facial expression. Comput. Graph. 22(4), 17–24 (1987)

    Google Scholar 

  43. Wilhelms, J., Gelder, A.V.: Anatomically based modeling. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 173–180. ACM Press/Addison-Wesley Publishing Co., New York (1997)

    Chapter  Google Scholar 

  44. Williams, L.: Performance driven facial animation. Comput. Graph. 24(4), 235–242 (1990)

    Article  Google Scholar 

  45. Yin, L., Weiss, K.: Generating 3D views of facial expressions from frontal face video based on topographic analysis. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 360–363. ACM SIGMULTIMEDIA, New York (2004)

    Chapter  Google Scholar 

  46. Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-driven photorealistic facial expression synthesis. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 177–186. Eurographics, San Diego (2003)

    Google Scholar 

  47. Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., Shum, H.: Geometry-driven photorealistic facial expression synthesis. IEEE Trans. Vis. Comput. Graph. 12(1), 48–60 (2006)

    Article  Google Scholar 

  48. Zhang, Y., Sim, T., Tan, C.L.: Rapid modeling of 3D faces for animation using an efficient adaptation algorithm. In: Proceedings of the 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, pp. 173–181. ACM SIGGRAPH, Singapore (2004)

    Chapter  Google Scholar 

  49. Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23(3), 548–558 (2004) (URL: http://grail.cs.washington.edu/projects/stfaces)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nikolaos Ersotelos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ersotelos, N., Dong, F. Building highly realistic facial modeling and animation: a survey. Visual Comput 24, 13–30 (2008). https://doi.org/10.1007/s00371-007-0175-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-007-0175-y

Keywords

Navigation