Skip to main content
SpringerLink
Log in

A survey of sketch based modeling systems

  • Review Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

As 3D technology, including computer graphics, virtual reality and 3D printing, has been rapidly developed in the past years, 3D models are gaining an increasingly huge demand. Traditional 3D modeling platforms such as Maya and ZBrush, utilize “windows, icons, menus, pointers” (WIMP) interface paradigms for fine-grained control to construct detailed models. However, the modeling progress can be tedious and frustrating and thus too hard for a novice user or even a well trained artist. Therefore, a more intuitive interface is needed. Sketch, an intuitive communication and modeling tool for human beings, becomes the first choice of modeling community. So far, various sketch-based modeling systems have been created and studied. In this paper, we attempt to show how these systems work and give a comprehensive survey.We review and categorize the systems in four aspects: the input, the knowledge they use, the modeling approach and the output. We also discuss about inherent challenges and open problems for researchers in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Botsch M, Pauly M, Kobbelt L, Alliez P, Lévy B, Bischoff S, Röossl C. Geometric modeling based on polygonal meshes. In: Proceedings of the ACM SIGGRAPH Course Notes. 2007

    Google Scholar 

  2. Perry R N, Frisken S F. Kizamu: a system for sculpting digital characters. In: Proceedings of the 28th ACM Annual Conference on Computer Graphics and Interactive Techniques. 2001, 47–56

    Google Scholar 

  3. Zoran A, Shilkrot R, Nanyakkara S, Paradiso J. The hybrid artisans: a case study in smart tools. ACM Transactions on Computer-Human Interaction, 2014, 21(3): 15

    Article  Google Scholar 

  4. Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R. Real-time human pose recognition in parts from single depth images. Communications of the ACM, 2013, 56(1): 116–124

    Article  Google Scholar 

  5. Henry P, Krainin M, Herbst E, Ren X F, Fox D. RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research, 2012, 31(5): 647–663

    Article  Google Scholar 

  6. Smisek J, Jancosek M, Pajdla T. 3D with kinect. In: Fossati A, Gall J, Helmut Grabner H, et al. eds. Consumer Depth Cameras for Computer Vision: Research Topics and Applications. London: Springer-Verlag, 2013, 3–25

    Chapter  Google Scholar 

  7. Quan L. Image-based modeling. Springer Science & Business Media, 2010

    Book  MATH  Google Scholar 

  8. Fan L B, Wang R M, Xu L L, Deng J S, Liu L G. Modeling by drawing with shadow guidance. Computer Graphics Forum, 2013, 23(7): 157–166

    Article  Google Scholar 

  9. Yuan X R, Xu H, Nguyen M, Shesh A, Chen B Q. Sketch-based segmentation of scanned outdoor environment models. In: Proceedings of EG Workshop on Sketch-Based Interfaces and Modeling. 2005, 19–26

    Google Scholar 

  10. Wu H Y, Pan C H, Pan J, Yang Q, Ma S D. A sketch-based interactive framework for real-time mesh segmentation. In: Proceedings of Computer Graphics International. 2007

    Google Scholar 

  11. Schmidt R, Wyvill B, Sousa M C, Jorge J A. Shapeshop: sketch-based solid modeling with blobtrees. In: Proceedings of ACM SIGGRAPH 2007 courses. 2007

    Google Scholar 

  12. Turquin E, Wither J, Boissieux L, Cani M P, Hughes J F. A sketchbased interface for clothing virtual characters. IEEE Computer Graphics and Applications, 2007, 27(1): 72–81

    Article  Google Scholar 

  13. Igarashi T, Matsuoka S, Tanaka H. Teddy: a sketching interface for 3D freeform design. In: Proceedings of the 26th ACM Annual Conference on Computer Graphics and Interactive Techniques. 1999, 409–416

    Google Scholar 

  14. Olsen L, Samavati F F, Sousa M C, Jorge J A. Sketch-based modeling: a survey. Computers & Graphics, 2009, 33(1): 85–103

    Article  Google Scholar 

  15. Cook M T, Agah A. A survey of sketch-based 3-D modeling techniques. Interacting with Computers, 2009, 21(3): 201–211

    Article  Google Scholar 

  16. Kho Y, Garland M. Sketching mesh deformations. In: Proceedings of ACM SIGGRAPH 2007 Courses. 2007

    Google Scholar 

  17. Karpenko O A, Hughes J F. Smoothsketch: 3D free-form shapes from complex sketches. ACM Transactions on Graphics, 2006, 25(3): 589–598

    Article  Google Scholar 

  18. Xu K, Chen K, Fu H B, Sun W K L, Hu S M. Sketch2Scene: sketchbased co-retrieval and co-placement of 3D models. ACM Transactions on Graphics, 2013, 32(4): 123

    Article  Google Scholar 

  19. Iarussi E, Bommes D, Bousseau A. Bendfields: regularized curvature fields from rough concept sketches. ACM Transactions on Graphics, 2015, 34(3): 24

    Article  MATH  Google Scholar 

  20. Bae S H, Balakrishnan R, Singh K. ILoveSketch: as-natural-aspossible sketching system for creating 3D curve models. In: Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology. 2008, 151–160

    Chapter  Google Scholar 

  21. Lee S, Feng D, Grimm C, Gooch B. A sketch-based user interface for reconstructing architectural drawings. Computer Graphics Forum, 2008, 27(1): 81–90

    Article  Google Scholar 

  22. Bae S H, Balakrishnan R, Singh K. EverybodyLovesSketch: 3D sketching for a broader audience. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology. 2009, 59–68

    Chapter  Google Scholar 

  23. Zimmermann J, Nealen A, Alexa M. SilSketch: automated sketchbased editing of surface meshes. In: Proceedings of the 4th Eurographics Workshop on Sketch-based Interfaces and Modeling. 2007, 23–30

    Chapter  Google Scholar 

  24. Zimmermann J, Nealen A, Alexa M. Sketchbased interfaces: sketching contours. Computers Graphics, 2008, 32(5): 486–499

    Article  Google Scholar 

  25. Kraevoy V, Sheffer A, van de Panne M. Modeling from contour drawings. In: Proceedings of the 6th ACM Eurographics Symposium on Sketch-Based Interfaces and Modeling. 2009, 37–44

    Chapter  Google Scholar 

  26. Nealen A, Sorkine O, Alexa M, Cohen-Or D. A sketch-based interface for detail-preserving mesh editing. ACM Transactions on Graphics, 2005, 24(3): 1142–1147

    Article  Google Scholar 

  27. Nam S H, Chai Y H. SPACESKETCH: shape modeling with 3D meshes and control curves in stereoscopic environments. Computers Graphics, 2012, 36(5): 526–533

    Article  Google Scholar 

  28. Rivers A, Durand F, Igarashi T. 3D modeling with silhouettes. ACM Transactions on Graphics, 2010, 29(4): 109

    Google Scholar 

  29. Levi Z, Gotsman C. ArtiSketch: a system for articulated sketch modeling. Computer Graphics Forum, 2013, 32: 235–244

    Article  Google Scholar 

  30. Lee Y J, Zitnick C L, Cohen M F. Shadowdraw: real-time user guidance for freehand drawing. ACM Transactions on Graphics, 2011, 30(4): 27

    Article  Google Scholar 

  31. Andre A, Saito S. Single-view sketch based modeling. In: Proceedings of the 8th ACM Eurographics Symposium on Sketch-Based Interfaces and Modeling. 2011, 133–140

    Chapter  Google Scholar 

  32. Gingold Y, Igarashi T, Zorin D. Structured annotations for 2D-to-3D modeling. ACM Transactions on Graphics, 2009, 28(5): 148

    Article  Google Scholar 

  33. Hoffman D D. Visual intelligence: how we create what we see. New York: W.W. Norton & Company, 2000

    Google Scholar 

  34. Xu B X, Chang W, Sheffer A, Bousseau A, McCrae J, Singh K. True2form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics, 2014, 33(4): 131

    Google Scholar 

  35. Schmidt R, Khan A, Singh K, Kurtenbach G. Analytic drawing of 3D scaffolds. ACM Transactions on Graphics, 2009, 28(5): 149

    Article  Google Scholar 

  36. Zou M, Holloway M, Carr N, Ju T. Topology-constrained surface reconstruction from cross-sections. ACM Transactions on Graphics, 2015, 34(4): 128

    Article  MATH  Google Scholar 

  37. Shtof A, Agathos A, Gingold Y, Shamir A, Cohen-or D. Geosemantic snapping for sketch-based modeling. Computer Graphics Forum, 2013, 32: 245–253

    Article  Google Scholar 

  38. Chen T, Zhu Z, Shamir A„ Hu S M, Cohen-Or D. 3-sweep: extracting editable objects from a single photo. ACM Transactions on Graphics, 2013, 32(6): 195

    Google Scholar 

  39. Chen X J, Kang S B, Xu Y Q, Dorsey J, Shum H Y. Sketching reality: realistic interpretation of architectural designs. ACM Transactions on Graphics, 2008, 27(2): 11

    Article  Google Scholar 

  40. Jiang N J, Tan P, Cheong L F. Symmetric architecture modeling with a single image. ACM Transactions on Graphics, 2009, 28(5): 113

    Article  Google Scholar 

  41. Chen C J, Neubert B, Xu Y Q, Deussen O, Kang S B. Sketch-based tree modeling using Markov random field. ACM Transactions on Graphics, 2008, 27(5): 109

    Article  Google Scholar 

  42. Tan P, Fang T, Xiao J X, Zhao P, Quan L. Single image tree modeling. ACM Transactions on Graphics, 2008, 27(5): 108

    Article  Google Scholar 

  43. Davis J, Agrawala M, Chuang E, Popović Z, Salesin D. A sketching interface for articulated figure animation. In: Proceedings of the 2003 ACMSIGGRAPH/Eurographics Symposium on Computer Animation. 2003, 320–328

    Google Scholar 

  44. Kazmi I K, You L H, Yang X S, Jin X G, Zhang J J. Efficient sketchbased creation of detailed character models through data-driven mesh deformations. Computer Animation and Virtual Worlds, 2015, 26(3–4): 469–481

    Article  Google Scholar 

  45. Yan F L, Gong ML, Cohen-Or D, eussen O, Chen N Q. Flower reconstruction from a single photo. Computer Graphics Forum. 2014, 33(2): 439–447

    Article  Google Scholar 

  46. Pan Z R, Huang J, Tong Y Y, Zheng C X, Bao H J. Interactive localized liquid motion editing. ACM Transactions on Graphics, 2013, 32(6): 184

    Google Scholar 

  47. Xu K, Kim V G, Huang Q X, Kalogerakis E. Data-driven shape analysis and processing. 2015, arXiv preprint arXiv:1502.06686

    Google Scholar 

  48. Shin H J, Igarashi T. Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. In: Proceedings of Graphics Interface 2007. 2007, 63–70

    Chapter  Google Scholar 

  49. Xu K, Zheng H L, Zhang H, Cohen-Or D, Liu L G, Xiong Y S. Photo-inspired model-driven 3D object modeling. ACM Transactions on Graphics, 2011, 30(4): 80

    Google Scholar 

  50. Su H, Huang Q X, Mitra N J, Li Y Y, Guibas L. Estimating image depth using shape collections. ACM Transactions on Graphics, 2014, 33(4): 37

    Google Scholar 

  51. Xie X H, Xu K, Mitra N J, Cohen-Or D, Gong W Y, Su Q, Chen B Q. Sketch-to-design: context-based part assembly. Computer Graphics Forum, 2013, 32(8): 233–245

    Article  Google Scholar 

  52. Williams L R, Jacobs D W. Stochastic completion fields: a neural model of illusory contour shape and salience. Neural computation, 1997, 9(4): 837–858

    Article  Google Scholar 

  53. Nealen A, Igarashi T, Sorkine O, Alexa M. Fibermesh: designing freeform surfaces with 3D curves. ACM Transactions on Graphics, 2007, 26(3): 41

    Article  Google Scholar 

  54. Sorkine O, Cohen-Or D, Lipman Y, Alexa M, Rössl C, Seidel H P. Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ ACM SIGGRAPH Symposium on Geometry Processing. 2004, 175–184

    Chapter  Google Scholar 

  55. Pan H, Liu Y, Sheffer A, Vining N, Li C J, Wang W P. Flow aligned surfacing of curve networks. ACM Transactions on Graphics, 2015, 34(4): 127

    Article  MATH  Google Scholar 

  56. Alexa M. Differential coordinates for local mesh morphing and deformation. The Visual Computer, 2003, 19(2): 105–114

    MATH  Google Scholar 

  57. De Paoli C, Singh K. SecondSkin: Sketch-based construction of layered 3D models. ACM Transactions on Graphics, 2015, 34(4): 126

    Article  Google Scholar 

  58. Eitz M, Richter R, Boubekeur T, Hildebrand K, AlexaM. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 31

    Google Scholar 

  59. Funkhouser T, Kazhdan M, Shilane P, Min P, Kiefer W, Tal A, Rusinkiewicz S, Dobkin D. Modeling by example. ACM Transactions on Graphics, 2004, 23(3): 652–663

    Article  Google Scholar 

  60. Eitz M, Hildebrand K, Boubekeur T, Alexa M. Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Transactions on Visualization and Computer Graphics, 2011, 17(11): 1624–1636

    Article  Google Scholar 

  61. Eitz M, Hays J, Alexa M. How do humans sketch objects? ACM Transactions on Graphics, 2012, 31(4): 44

    Google Scholar 

  62. Schneider R G, Tuytelaars T. Sketch classification and classificationdriven analysis using fisher vectors. ACM Transactions on Graphics, 2014, 33(6): 174

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, University of Science and Technology of China, Hefei, 230026, China

    Chao Ding & Ligang Liu

Authors
  1. Chao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ligang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ligang Liu.

Additional information

Chao Ding is studying in the Department of Computational Mathematics from University of Science and Technology of China (USTC), China. He received a Bachelor degree in 2013, and will also receive a master degree in computational mathematics from USTC in 2016. During 2013 and 2016, he has been doing research in the Graphics and Geometric Computing Lab at USTC.

Ligang Liu is a professor at the School of Mathematical Sciences, University of Science and Technology of China, China. He received his BS (1996) and his PhD (2001) from Zhejiang University, China. Between 2001 and 2004, he worked at Microsoft Research Asia. Then he worked at Zhejiang University during 2004 and 2012. He paid an academic visit to Harvard University during 2009 and 2011. He serves as the associated editors for journals of Computer Aided Geometric Design and IEEE Computer Graphics and Applications. He is the Awardee of the NSFC Excellent Young Scholars Program in 2012. His research interests include digital geometric processing, computer graphics, and image processing.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, C., Liu, L. A survey of sketch based modeling systems. Front. Comput. Sci. 10, 985–999 (2016). https://doi.org/10.1007/s11704-016-5422-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-016-5422-9

Keywords

Search

Navigation