Abstract
Line drawings, as a concise form, can be recognized by infants and even chimpanzees. Recently, how the visual system processes line-drawings attracts more and more attention from psychology, cognitive science and computer science. The neuroscientific studies revealed that line drawings generate similar neural actions as color photographs, which give insights on how to efficiently process big media data. In this paper, we present a comprehensive survey on line drawing studies, including cognitive mechanism of visual perception, computational models in computer vision and intelligent process in diverse media applications. Major debates, challenges and solutions that have been addressed over the years are discussed. Finally some of the ensuing challenges in line drawing studies are outlined.
Similar content being viewed by others
References
Fischetti M. Computers vs. brains. Scientific American, 2011, 305(5): 104–104
Fu X L, Cai L H, Liu Y, Jia J, Chen WF, Zhang Y, Zhao GZ, Liu YJ, Wu C X. A computational cognition model of perception, memory and judgment. Science China Information Sciences, 2014, 57(3): 1–15
Ullman S. High-level vision: Object recognition and visual cognition. Cambridge: MIT press, 2000
Liu Y J, Fu Q F, Liu Y, Fu X L. A distributed computational cognitive model for object recognition. Science China Information Sciences, 2013, 56(9): 1–13
Riddoch M, Humphreys G. Object Recognition. In Rapp B, ed. Handbook of Cognitive Neuropsychology. Hove: Psychology Press
Canny J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986 (6): 679–698
Cole F, Golovinskiy A, Limpaecher A, Barros H S, Finkelstein A, Funkhouser T, Rusinkiewicz S. Where do people draw lines? ACM Transactions on Graphics, 2008, 27(3): 88
Sayim B, Cavanagh P. What Line Drawings Reveal About the Visual Brain. Frontiers in Human Neuroscience, 2011, 5
Yonas A, Arterberry M E. Infants perceive spatial structure specified by line junctions. Perception. 1994, 23(12): 1427–1435
Itakura S. Recognition of line-drawing representations by a chimpanzee (Pan troglodytes). The Journal of general psychology, 1994, 121(3): 189–197
Cole F, Sanik K, DeCarlo D, Finkelstein A, Funkhouser T, Rusinkiewicz S, Singh M. How well do line drawings depict shape? ACM Transactions on Graphics, 2009, 28(3): 28
Koenderink J J, Van Doorn A J, Kappers A M L. Surface perception in pictures. Perception & Psychophysics, 1992, 52(5): 487–496
Koenderink J J, van Doorn A J, Kappers A M L, Todd J T. Ambiguity and the “mental eye” in pictorial relief. Perception, 2001, 30(4): 431–448
Marr D, Vision A. A Computational Investigation into the Human Representation and Processing of Visual Information. Cambridge: MIT Press, 2010
Watt R J, Rogers B J. Human vision and cognitive science. Research Directions in Cognitive Science: A European Perspective, 1989, 1: 9–22
Sonka M, Hlavac V, Boyle R. Image processing, analysis, and machine vision. Cengage Learning, 2014
Biederman I. Recognition-by-components: a theory of human image understanding. Psychological review, 1987, 94(2): 115
Biederman I, Ju G. Surface versus edge-based determinants of visual recognition. Cognitive Psychology, 1988, 20(1): 38–64
Fu Q, Liu Y J, Chen W, Fu X. The time course of natural scene categorization in human brain: simple line-drawings vs. color photographs. Journal of Vision, 2013, 13(9): 1060
Del Viva M M, Punzi G, Benedetti D. Information and Perception of Meaningful Patterns. PLoS One, 2013, 8(7): e69154
Morgan M J. Features and the ‘primal sketch’. Vision Research, 2011, 51(7): 738–753
Delorme A, Richard G, Fabre-Thorpe M. Key visual features for rapid categorization of animals in natural scenes. Frontiers in Psychology, 2010, 1: 21
Naber M, Hilger M, Einhäuser W. Animal detection and identification in natural scenes: image statistics and emotional valence. Journal of Vision, 2012, 12(1): 25
Derrington A M, Krauskopf J, Lennie P. Chromatic mechanisms in lateral geniculate nucleus of macaque. The Journal of Physiology, 1984, 357(1): 241–265
Campbell F W, Robson J G. Application of Fourier analysis to the visibility of gratings. The Journal of physiology, 1968, 197(3): 551
Joubert O R, Rousselet G A, Fabre-Thorpe M, Fize D. Rapid visual categorization of natural scene contexts with equalized amplitude spectrum and increasing phase noise. Journal of Vision, 2009, 9(1): 2
Jolicoeur P. The time to name disoriented natural objects. Memory & Cognition, 1985, 13(4): 289–303
Farah M J, Hammond K M. Mental rotation and orientation-invariant object recognition: dissociable processes. Cognition, 1988, 29(1): 29–46
Westheimer G. The Fourier theory of vision. Perception, 2001, 30(5): 531–542
Oppenheim A V, Lim J S. The importance of phase in signals. In: Proceedings of the IEEE. 1981, 69(5): 529–541
Piotrowski L N, Campbell F W. A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception, 1982, 11(3): 337–346
Guyader N, Chauvin A, Peyrin C, Hérault J, Marendaz C. Image phase or amplitude? Rapid scene categorization is an amplitude-based process. Comptes Rendus Biologies, 2004, 327(4): 313–318
Chen W F, Liang J, Liu Y J, Fu Q F, Fu X L. Rapid natural scene categorization of line drawings is less influenced by amplitude spectra: evidence from a subliminal perception study. ASSC 8: Poster Session
Morrone M C, Burr D C. Feature detection in human vision: a phasedependent energy model. In: Proceedings of the Royal Society of London. 1988, 235(1280): 221–245
Devlin H, Tracey I, Johansen-Berg H, Clare S. What is Functional Magnetic Resonance Imaging (fMRI)? FMRIB Centre, Department of Clinical Neurology, University of Oxford
Walther D B, Caddigan E, L F F, Beck D M. Natural scene categories revealed in distributed patterns of activity in the human brain. The Journal of Neuroscience, 2009, 29(34): 10573–10581
Walther D B, Chai B, Caddigan E, Beck DM, Li F F. Simple line drawings suffice for functional MRI decoding of natural scene categories. In: Proceedings of the National Academy of Sciences, 2011, 108(23): 9661–9666
Kim S G, Richter W, Uǧurbil K. Limitations of temporal resolution in functional MRI. Magnetic Resonance in Medicine, 1997, 37(4): 631–636
Vanrullen R, Thorpe S J. The time course of visual processing: from early perception to decision-making. Journal of Cognitive Neuroscience, 2001, 13(4): 454–461
Johnson J S, Olshausen B A. Timecourse of neural signatures of object recognition. Journal of Vision, 2003, 3(7): 4
Fu Q F, Liu Y J, Dienes Z, Wu J H, Chen W F, Fu X L. Different time courses of natural scene categorization of color photographs and line-drawings: evidence from event-related potentials. Submitted for publication, 2014
Liu Y J, Ma C X, Fu Q, Fu X L, Qin S F, Xie L X. A Sketch-Based Approach for Interactive Organization of Video Clips. ACM Transactions on Multimedia Computing, Communications, and Applications, 2014, 11(1)
Snodgrass J G, Vanderwart M. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 1980, 6(2): 174
Magnié M N, Besson M, Poncet M, Dolisi C. The Snodgrass and Vanderwart set revisited: norms for object manipulability and for pictorial ambiguity of objects, chimeric objects, and nonobjects. Journal of Clinical and Experimental Neuropsychology, 2003, 25(4): 521–560
Rossion B, Pourtois G. Revisiting Snodgrass and Vanderwart’s object pictorial set: the role of surface detail in basic-level object recognition. Perception, 2004, 33(2): 217–236
Viggiano M P, Vannucci M, Righi S. A new standardized set of ecological pictures for experimental and clinical research on visual object processing. Cortex, 2004, 40(3): 491–509
Abel S, Weiller C, Huber W, Willmes K. Neural underpinnings for model-oriented therapy of aphasic word production. Neuropsychologia, 2014, 57: 154–165
Janssen N, Carreiras M, Barber H A. Electrophysiological effects of semantic context in picture and word naming. Neuroimage, 2011, 57(3): 1243–1250
Schnur T T. The persistence of cumulative semantic interference during naming. Journal of Memory and Language, 2014, 75: 27–44
Strijkers K, Holcomb P J, Costa A. Conscious intention to speak proactively facilitates lexical access during overt object naming. Journal of Memory and Language, 2011, 65(4): 345–362
Kang H, Lee S, Chui C. Coherent line drawing. In: Proceedings of 5th International Symposium on Non-photorealistic Animation and Rendering. 2007, 43–50
Arbel T, Ferrie F P. Viewpoint selection by navigation through entropy maps. In: Proceedings of the 7th IEEE International Conference on Computer Vision. 1999, 1: 248–254
Laporte C, Arbel T. Efficient discriminant viewpoint selection for active bayesian recognition. International Journal of Computer Vision, 2006, 68(3): 267–287
Secord A, Lu J, Finkelstein A, Sing M, Nealen A. Perceptual models of viewpoint preference. ACM Transactions on Graphics, 2011, 30(5): 109
Chen D Y, Tian X P, Shen Y T, Ouhyoung M. On visual similarity based 3D model retrieval. Computer graphics forum. Publishing, 2003, 22(3): 223–232
Cyr C M, Kimia B B. 3D object recognition using shape similiarity based aspect graph. In: Proceedings the 8th IEEE International Conference on Computer Vision. 2001, 1: 254–261
Liu Y J, Luo X, Joneja A, Ma C X, Fu X L, Song D W. User-adaptive sketch-based 3-D CAD model retrieval. IEEE Transactions on Automation Science and Engineering, 2013, 10(3): 783–795
Duda R O, Hart P E, Stork D G. Pattern Classification. John Wiley & Sons, 1999.
Chalechale A, Naghdy G, Mertins A. Sketch-based image matching using angular partitioning. IEEE Transactions on Systems, Man and Cybernetics, 2005, 35(1): 28–41
Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 31
Hung C C, Carlson E T, Connor C E. Medial axis shape coding in macaque inferotemporal cortex. Neuron, 2012, 74(6): 1099–1113
Lescroart M D, Biederman I. Cortical representation of medial axis structure. Cerebral Cortex, 2013, 23(3): 629–637
Li Z, Qin S, Jin X. Skeleton-enhanced line drawings for 3D models. Graphical Models, 2014, 76(6): 620–632
Lake B M, Salakhutdinov R, Tenenbaum J. One-shot learning by inverting a compositional causal process. Advances in neural information processing systems. 2013: 2526–2534
Ma C X, Liu Y J, Yang H Y, Teng D X, Wang H A, Dai G Z. KnitSketch: a sketch pad for conceptual design of 2D garment patterns. IEEE Transactions on Automation Science and Engineering, 2011, 8(2): 431–437
Sivic J, Zisserman A. Video Google: A text retrieval approach to object matching in videos
Baeza-Yates R, Ribeiro-Neto B. Modern Information Retrieval. New York: ACM Press, 1999
Sugihara K. Machine Interpretation of Line Drawings. Cambridge: MIT Press, 1986
Hoffman D D. Visual Intelligence: How We Create What We See. New York: W.W. Norton & Company, 2000
Chen T, Cheng M M, Tan P, Ariel S, Hu S M. Sketch2Photo: internet image montage.ACM Transactions on Graphics, 2009, 28(5): 124
Cao Y, Wang C H, Zhang L Q, Zhang L. Edgel index for large-scale sketch-based image search. IEEE Conference on Computer Vision and Pattern Recognition, 2011: 761–768
Truong B T, Venkatesh S. Video abstraction: a systematic review and classification. ACM Transactions on Multimedia Computing, Communications, and Applications, 2007, 3(1): 3
Ma C X, Liu Y J, Wang H A, Teng D X, Dai G Z. Sketch-based annotation and visualization in video authoring. IEEE Transactions on Multimedia, 2012, 14(4): 1153–1165
Collomosse J P, McNeill G, Qian Y. Storyboard sketches for content based video retrieval. In: Proceedings of the 12th IEEE International Conference on Computer Vision, 2009, 245–252
Igarashi T, Matsuoka S, Tanaka H. Teddy: a sketching interface for 3D freeform design. In: Proceedings of ACM SIGGRAPH Courses, 2007: 21
Nealen A, Igarashi T, Sorkine O, Alexa M. FiberMesh: designing freeform surfaces with 3D curves. ACM Transactions on Graphics, 2007, 26(3): 41
Liu Y J, Ma C X, Zhang D L. EasyToy: plush toy design using editable sketching curves. IEEE Computer Graphics and Applications, 2011, 31(2): 49–57
Zhu X, Jin X, Liu S, Zhoo H. Analytical solutions for sketch-based convolution surface modeling on the GPU. The Visual Computer, 2012, 28(11): 1115–1125
Yu C C, Liu Y J, Wu T F, Li K Y, Fu X L. A global energy optimization framework for 2.1D sketch extraction from monocular images. Graphical Models, 2014, 76(5): 507–521
Liu Y J, Zhang J B, Hou J C, Ren J C. Cylinder detection in large-scale point cloud of pipeline plant. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(10): 1700–1707
Author information
Authors and Affiliations
Corresponding author
Additional information
Yongjin Liu received his BE from Tianjin University, China in 1998, and his PhD from Hong Kong University of Science and Technology, China in 2004. He is now an associate professor with Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, China. His research interests include pattern recognition, computer graphics, computational geometry and computer-aided design. He is a member of IEEE, a member of IEEE Computer Society and IEEE Communications Society.
Minjing Yu received her BE from Wuhan University, China, and she is now a PhD student at Department of Computer Science and Technology, Tsinghua University, China. Her research interests include image processing, computer graphics and cognitive science.
Qiufang Fu received her PhD from Institute of Psychology, Chinese Academy of Sciences, in 2006. She is now an associate professor in Institute of Psychology, Chinese Academy of Sciences, China, with interests in implicit learning and unconscious knowledge. She intends to explore the neural and cognitive mechanisms responsible for the dissociation of implicit and explicit processes.
Wenfeng Chen received his BS from Peking University, China in 1999, and his PhD from Institute of Psychology, Chinese Academy of Sciences, China in 2005. He is now an associate professor of psychology in the Institute of Psychology, Chinese Academy of Sciences, and a guest Associate Editor of Frontiers in Psychology. He is interested in face learning and recognition, visual attention, emotion and cognitive modelling.
Ye Liu received her PhD from Institute of Psychology, Chinese Academy of Sciences, China in 2005. She is now an associate professor of psychology in the Institute of Psychology, Chinese Academy of Sciences. She is interested in semantic knowledge representation and semantic processing, metaphor comprehension, and affective computing.
Lexing Xie is Senior Lecturer in the Research School of Computer Science at the Australian National University, Australia. She was research staff member at IBM T.J. Watson Research Center in New York, USA from 2005 to 2010, and adjunct assistant professor at Columbia University, USA from 2007 to 2009. She received BS from Tsinghua University, China, and MS and PhD degrees from Columbia University, all in Electrical Engineering. Her research interests include applied machine learning, multimedia and social media. Her research has received five best student paper and best paper awards between 2002 and 2011. She currently serves as an associate editor of IEEE Transactions on Multimedia, and ACM Transactions on Multimedia Computing, Communications and Applications.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Liu, Y., Yu, M., Fu, Q. et al. Cognitive mechanism related to line drawings and its applications in intelligent process of visual media: a survey. Front. Comput. Sci. 10, 216–232 (2016). https://doi.org/10.1007/s11704-015-4450-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-015-4450-1