Chang-Jun Nan
ABSTRACT
TV drama is a kind of big data, containing enormous knowledge of modern human society. As the character-centered stories unfold, diverse knowledge, such as economics, politics and the culture, is displayed. However, unless we have efficient dynamic multi-modal data processing and picture processing methods, we cannot analyze drama data effectively. Here, we adopt the recently proposed deep concept hierarchies (DCH) and convolutional-recursive neural network (C-RNN) models to analyze the social network between the drama characters. DCH uses multi hierarchies structure to translate the vision-language concepts of drama characters into diversified abstract concepts, and utilizes Markov Chain Monte Carlo algorithm to improve the retrieval efficiency of organizing conceptual spaces. Adopting approximately 4400-minute data of TV drama - Friends, we process face recognition on the characters by using convolutional-recursive deep learning model. Then we establish the social network between the characters by deep concept hierarchies model and analyze their affinity and the change of social network while the stories unfold.
- B. J. Biddle, "Recent development in role theory", in Annual Review of Sociology, 1986, pp. 12:67-92.Google Scholar
Cross Ref
- J.-W. Ha, K.-M. Kim and B.-T. Zhang, "Automated construction of visual-linguistic knowledge via concept learning from cartoon videos", in Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015, pp. 522--528.Google ScholarDigital Library
- A. N. Meltzoff, "Toward a Developmental Cognitive Science: The Implications of Cross-modal Matching and Imitation for Development of Representation and Memory in Infancy", in Annual New York Academy Science, 1990, pp. 608:1-31.Google ScholarCross Ref
- A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-Scale Video Classification with Convolutional Neural Networks", in IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1725--1732. Google ScholarDigital Library
- I.-H. Jhuo and D.T. Lee, "Video Event Detection via Multi-modality Deep Learning", in International Conference on Pattern Recognition, 2014, pp. 666--671. Google ScholarDigital Library
- H.-W. Chen, J.-H. Kuo, W.-T. Chu and J.-L. Wu, "Action movies segmentation and summarization based on tempo analysis", in the 6th ACM SIGMM international workshop on Multimedia information retrieval, 2004, pp. 251--258. Google ScholarDigital Library
- C.-W. Wang, W.-H. Cheng, J.-C. Chen, S.-S. Yang, J.-L. Wu, "Film narrative exploration through the analysis of aesthetic elements", in the 13th international conference on Multimedia Modeling - Volume Part I, 2007, pp. 606--615. Google ScholarDigital Library
- D. Tran, L. Bourdev, R. Fergus, L. Torresani and M. Paluri, "C3D: Generic Features for Video Analysis", in IEEE Conference on Computer Vision and Pattern Recognition, 2014.Google Scholar
- V. Ramanathan, B. Yao, and L. Fei-Fei, "Social Role Discovery in Human Events", in IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 2475--2482. Google ScholarDigital Library
- Y.-F. Zhang, C.-S. Xu, H.-Q. Lu and Y-M Huang, "Character Identification in Feature-Length Films Using Global Face-Name Matching", in IEEE Transactions on Multimedia, 2009, pp. 1276--1288. Google ScholarDigital Library
- C.-Y. Weng, W.-T. Chu and J.-L. Wu, "RoleNet: Movie Analysis from the Perspective of Social Networks", in IEEE Transactions on Multimedia, 2009, pp. 256--271. Google ScholarDigital Library
- T. Lan, L. Sigal, and G. Mori, "Social roles in hierarchical models for human activity recognition", in IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 1354--1361. Google ScholarDigital Library
- T. Yu, S.-N. Lim, K. Patwardhan, and N. Krahnstoever, "Monitoring, recognizing and discovering social networks", in IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 1462--1469.Google ScholarCross Ref
- L. Ding, A. Yilmaz, "Learning Relations among Movie Characters: A Social Network Perspective", in European Conference on Computer Vision, 2010, pp 410-423. Google ScholarDigital Library
- L. Ding, A. Yilmaz, "Inferring social relations from visual concepts", in IEEE International Conference on Computer Vision, 2011, pp. 699--706. Google ScholarDigital Library
- G. Wang, A. Gallagher, J.-B. Luo, D. Forsyth, "Seeing People in Social Context: Recognizing People and Social Relationships", in European Conference on Computer Vision, 2010, pp 169-182. Google ScholarDigital Library
- M. Kiefer, E. J. Sim, B. Herrnberger, J. Grothe, and K. Hoenig, "The sound of concepts: Four markers for a link between auditory and conceptual brain systems", in Journal of Neuroscience, 2008, pp. 28:12224-12230,Google ScholarCross Ref
- B.-T. Zhang, J.-W. Ha and M. Kang. "Sparse population code models of word learning in concept drift". in Proceedings of Annual Meeting of the Cognitive Science Society, 2012, pp. 1221--1226.Google Scholar
- R. Socher, B. Huval, B. Bath, C. D. Manning and A. Y. Ng, "Convolutional-recursive Deep Learning for 3D Object Classification", in Advances in Neural Information Processing Systems, 2012, pp. 665--673.Google ScholarDigital Library
- R. Girshick, J. Donahue, T. Darrell, J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation", in Proceedings of International Conference on Pattern Recognition, 2014.Google Scholar
- T. Mikolov, I. Sutskever, K. Chen, G. Corrado, J. Dean, "Distributed Representations of Words and Phrases and their Compositionality", in Proceedings of Advances in Neural Information Processing Systems, 2013.Google Scholar
- Social Network Analysis of TV Drama Characters via Deep Concept Hierarchies
Recommendations
Multi-screen cloud social TV: transforming TV experience into 21st century
MM '13: Proceedings of the 21st ACM international conference on MultimediaNowadays, TV experience has been transformed from the traditional "laid-back" video watching experience to a "lean-forward" social and multi-screen experience. In this demo, we design and develop a multi-screen cloud social TV system in response to this ...
ShapeShifting TV: interactive screen media narratives
This paper presents a paradigm, called ShapeShifting TV, for the realisation of interactive TV narratives or, more generally, of interactive screen-media narratives. These are productions whose narrations respond on the fly (i.e. in real time) to ...
Enhancing Use of Social Media in TV Broadcasting
TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online VideoTraditional linear TV is decreasing in popularity and the broadcast industry has identified the need to change communication with their audience as a way to counteract on this. Especially younger generations are using social media twenty-four-seven and ...
Comments