Skip to main content
SpringerLink
Log in

Frontal and Semi-Frontal Facial Caricature Synthesis Using Non-Negative Matrix Factorization

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, we present a novel approach to synthesizing frontal and semi-frontal cartoon-like facial caricatures from an image. The caricature is generated by warping the input face from the original feature points to the corresponding exaggerated feature points. A 3D mean face model is incorporated to facilitate face to caricatures by inferring the depth of 3D feature points and the spatial transformation. Then the 3D face is deformed by using non-negative matrix factorization and projected back to image plane for future warping. To efficiently solve the nonlinear spatial transformation, we propose a novel initialization scheme to set up Levenberg-Marquardt optimization. According to the spatial transformation, exaggeration is applied to the most salient features by exaggerating their normalized difference from the mean. Non-photorealistic rendering (NPR) based stylization completes the cartoon caricature. Experiments demonstrate that our method outperforms existing methods in terms of view angles and aesthetic visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lee E J, Kwon J, Lee I K. Caricature video. Computer Animation and Virtual Worlds, 2007, 18(4-5): 279.

    Article  Google Scholar 

  2. Mo Z, Lewis J P, Neumann U. Improved automatic caricature by feature normalization and exaggeration. In Proc. ACM SIGGRAPH 2004 Sketches, Los Angeles, USA, Aug. 8-12, 2004, p.57.

  3. Lee D D, Seung H S. Learning the parts of objects by nonnegative matrix factorization. Nature, Oct. 1999, 401(6755): 788–791.

    Article  Google Scholar 

  4. Brennan S E. Caricature generator [Master’s Thesis]. Massachusetts Institute of Technology, September 1982.

  5. Tseng C, Lien J. Synthesis of exaggerative caricature with inter and intra correlations. In Proc. the 8th Asian Conf. Computer Vision, Tokyo, Japan, Nov. 18-22, 2007, pp.314–323.

  6. Chen H, Xu Y Q, Shum H Y, Zhu S C, Zheng N N. Examplebased facial sketch generation with non-parametric sampling. In Proc. IEEE Int. Conf. Computer Vision, Vancouver, Canada, Jul. 7-14, 2001, Vol2., pp. 433–438.

  7. Lai K H, Edirisinghe E A, Chung P W H. A facial component based hybrid approach to caricature generation using neural networks. In Proc. ACAT on Computational Intelligence, San Francisco, USA, Nov. 20-22, 2006, pp.174–179.

  8. Liang L, Chen H, Xu Y Q, Shum H Y. Example-based caricature generation with exaggeration. In Proc. the 10th Pacific Conference on Computer Graphics and Applications (PG 2002), Beijing, China, Oct. 9-11, 2002, p.386.

  9. Liu J, Chen Y, Gao W. Mapping learning in eigenspace for harmonious caricature generation. In Proc. the 14th Annual ACM International Conference on Multimedia, Santa Barbara, USA, Oct. 23-27, 2006, pp.683–686.

  10. Koshimizu H, Tominaga M, Fujiwara T, Murakami K. On KANSEI facial image processing for computerized facial caricaturing system PICASSO. In Proc. IEEE International Conference on Systems, Man, and Cybernetics, Tokyo, Japan, Oct. 12-15, 1999, Vol.6, pp.294–299.

  11. Chiang P Y, Liao W H, Li T Y. Automatic caricature generation by analyzing facial features. In Proc. Asian Conference on Computer Vision, Korea, 2004, pp.27–30.

  12. Xu GZ, Kaneko M, Kurematsu A. Synthesis of facial caricature using eigenspaces. IEICE Transactions on Information and Systems, 2001, 84(7): 1279–1288.

    Google Scholar 

  13. Blanz V, Vetter Thomas. A morphable model for the synthesis of 3D faces. In Proc. the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH1999), Los Angeles, USA, Aug. 8-13, 1999, pp.187–194.

  14. Chen Y L, Liao W H, Chiang P Y. Generation of 3D caricature by fusing caricature images. In Proc. IEEE Int. Conf. Systems, Man and Cybernetics 2006, Taipei, China, Oct. 8-11, 2006, pp.866–871.

  15. Gao X, Chen Y, Liu J, Zhou J. Personalized 3D caricature generation based on PCA subspace. In Proc. the 9th Pacific Rim Conference on Multimedia (PCM2008), Tainan, China, Dec. 9-13, 2008, pp.713–720.

  16. Li P, Chen Y, Liu J, Fu G. 3D caricature generation by manifold learning. In Proc. IEEE International Conference on Multimedia and Expo, Hannover, Germany, Jun. 23-26, 2008, pp.941–944.

  17. Liu J, Chen Y, Miao C, Xie J, Ling C X, Gao X, Gao W. Semi-supervised learning in reconstructed manifold space for 3D caricature generation. Computer Graphics Forum, 2009, 28(8): 2104–2116.

    Article  Google Scholar 

  18. Clarke L, Chen M, Mora B. Automatic generation of 3D caricatures based on artistic deformation styles. IEEE Transactions on Visualization and Computer Graphics, IEEE Computer Society Digital Libruary, May 20, 2010.

  19. Huang Y Z, Martin R R, Rosin P L, Meng X X, Yang C L. Expressive line drawings of human faces from range images. Sci. China Ser. F-Inf. Sci., 2009, 52(2): 295–307.

    Article  MATH  Google Scholar 

  20. Murphy-Chutorian E, Trivedi M M. Head pose estimation in computer vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, April 2009, 31(4): 607–626.

    Article  Google Scholar 

  21. Stiefelhagen R. Estimating head pose with neural networks — results on the Pointing04 ICPR workshop evaluation data. In Proc. ICPR Workshop Visual Observation of Deictic Gestures, Cambridge, UK, 2004, pp.17–25.

  22. Matthews I, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60(2): 135–164.

    Article  Google Scholar 

  23. Tech M. The BJUT-3D large-scale Chinese face database. Graphics Lab, Technical Report, Beijing University of Technology, 2005.

  24. Cootes T F, Taylor C J, Cooper D H, Graham J. Active shape models — Their training and application. Computer Vision and Image Understanding, 1995, 61(1): 38–59.

    Article  Google Scholar 

  25. Cootes T F, Edwards G J, Taylor C J. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6): 681–685.

    Article  Google Scholar 

  26. Gu L, Kanade T. 3D alignment of face in a single image. In Proc. the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), New York, USA, Jun. 17-22, 2006. pp.1305–1312.

  27. Spellucci P. An SQP method for general nonlinear programs using only equality constrained subproblems. Mathematical Programming, 1998, 82(3): 413–448.

    Article  MathSciNet  Google Scholar 

  28. Kim H, Park H. Non-negative matrix factorization based on alternating non-negativity constrained least squares and active set method. SIAM Journal in Matrix Analysis and Applications, 2008, 30(2): 713–730.

    Article  MATH  Google Scholar 

  29. Lee S, George W, Shin S Y. Scattered data interpolation with multilevel B-splines. IEEE Transactions on Visualization and Computer Graphics, 1997, 3(3): 228–244.

    Article  Google Scholar 

  30. Winnemöller H, Olsen S C, Gooch B. Real-time video abstraction. ACM Transactions on Graphics, 2006, 25(3): 1221–1226.

    Article  Google Scholar 

  31. Bousseau A, Neyret F, Thollot J, Salesin D. Video watercolorization using bidirectional texture advection. ACM Transactions on Graphics, 2007, 26(3): 104:1–104:7.

    Article  Google Scholar 

  32. Bousseau A, Kaplan M, Thollot J, Sillion F X. Interactive watercolor rendering with temporal coherence and abstraction. In Proc. the 4th International Symposium on Non-Photorealistic Animation and Rendering, Annecy, France, Jun. 5-7, 2006, pp.141–149.

  33. Hertzmann A. Fast paint texture. In Proc. the 2nd International Symposium on Non-Photorealistic Animation and Rendering, Annecy, France, Jun. 3-5, 2002, pp.91–96, 161.

  34. Zhang S H, Chen T, Zhang Y F, Hu S M, Martin R R. Vectorizing cartoon animations. IEEE Transactions on Visualization and Computer Graphics, 2009, 15(4): 618–629.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Xi’an Jiaotong University, Xi’an, 710049, China

    Hua Huang (Senior Member, CCF, Member, IEEE) & Xiang-Wang Ma

Authors
  1. Hua Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang-Wang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua Huang.

Additional information

This work is partly supported by the Program for New Century Excellent Talents in University under Grant No. NCET-09-0635, the Key Project of Chinese Ministry of Education under Grant No. 109142 and the MOE-Intel Joint Research Fund under Grant No. MOE-INTEL-09-01.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, H., Ma, XW. Frontal and Semi-Frontal Facial Caricature Synthesis Using Non-Negative Matrix Factorization. J. Comput. Sci. Technol. 25, 1282–1292 (2010). https://doi.org/10.1007/s11390-010-9405-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-010-9405-2

Keywords

Search

Navigation