Comic Character Recognition (CCR): Extraction of Speech Balloon Context and Character of Interest in Comics

Abstract

Document analysis is an active field of research which can attain a complete understanding of semantics of a given document. Further, digitalization of physical forms of various data has become vital in today’s data extraction. Comic digitalization is becoming widespread as it is one of the easily understandable graphic content and is attention seeking from early education readers to middle aged groups. Extracting the frames, panels, and speech balloons from digital comic is crucial for techniques that felicitate comic reading. However, Automatic Panel Extraction for Digital Comic is challenging, largely because of its layout design, visual symbols, speech balloons attached to almost all the panels, throughout the page. In this proposed work, it is proposed to automatically extract panels from digital pages using contour analysis and watershed canny operator. The first method identifies the difference between frames associated with the panels, whereas the second method identifies the difference in the color between the panel and the gutters. Speech balloons are segmented by methods of K-means clustering and contour analysis. K-means clustering is used to identify the closest related components, and contour analysis is used to differentiate the speech balloons from the other components in the comic panel. Text Area Recognition is a subtle approach that is implemented by Optical Character Recognition (OCR). And finally, comic character extraction has three steps of annotating the object, training the model, and detection of the comic character. The annotation is performed by bounding box algorithm, followed by training of the custom model using a pre-trained YOLOV3 algorithm. Once the custom model is trained, it is provided with comic strips as input to detect the dominant characters with their probability of confidence, which is the correctness of comic character. The implementation considerably serves better in performance when compared to the previous traditional methods of comic component analysis and extraction. Evaluation on the same has also been performed for the trained as well as input dataset.