Abstract
In this paper, we present an intuitive graphic framework introduced for the effective visualization of video content and associated audio-visual description, with the aim to facilitate a quick understanding and annotation of the semantic content of a video sequence. The basic idea consists in the visualization of a 2D feature space in which the shots of the considered video sequence are located. Moreover, the temporal position and the specific content of each shot can be displayed and analysed in more detail. The selected features are decided by the user, and can be updated during the navigation session. In the main window, shots of the considered video sequence are displayed in a Cartesian plane, and the proposed environment offers various functionalities for automatically and semi-automatically finding and annotating the shot clusters in such feature space. With this tool the user can therefore explore graphically how the basic segments of a video sequence are distributed in the feature space, and can recognize and annotate the significant clusters and their structure. The experimental results show that browsing and annotating documents with the aid of the proposed visualization paradigms is easy and quick, since the user has a fast and intuitive access to the audio-video content, even if he or she has not seen the document yet.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Chang, S.-F., Ma, W.-Y., Smeulders, A.: Recent advances and challenges of semantic image/video Search. In: Proceedings of ICASSP-2007. Hawaii, USA (2007)
Wang Y., Liu Z., Huang J.-C.: Multimedia content analysis using both audio and visual clues. IEEE Signal Process. Mag. 17(11), 12–36 (2000)
Izquierdo, E., et al.: State of the art in content-based analysis, indexing and retrieval. IST-2001-32795 SCHEMA Del. 2.1, Feb. 2005. http://www.iti.gr/SCHEMA
Manjunath B.S., Salembier P., Sikora T.: Introduction to MPEG-7: Multimedia Content Description Language. Wiley, London (2002)
Ngo C.-W., Ma Y.-F., Zhang H.-J.: Video summarization and scene detection by graph modeling. IEEE Trans. Circuits Syst. Video Technol. 15(2), 296–305 (2005)
Takahashi, Y., Nitta, N., Babaguchi, N.: Video summarization for large sports video archives. In: Proceeding of ICME-2005. Amsterdam, The Netherlands (2005)
Li Y., Lee S.-H., Yeh C.-H., Jay Kuo C.-C.: Techniques for movie content analysis and skimming. IEEE Signal Process. Mag. 23(2), 79–89 (2006)
Wang, T., Mei, T., Hua, X.-S., Liu, X.-L., Zhou, H.-Q.: Video collage: a novel presentation of video sequence. In: Proceedings of ICME-2007. Beijing, China, pp. 1479–1482 (2007)
Leonardi R., Migliorati P.: Semantic indexing of multimedia documents. IEEE Multimedia 9(2), 44–51 (2002)
Campanella, M., Leonardi, R., Migliorati, P.: Future-Viewer: an efficient framework for navigating and classifying audio-visual documents. In: Proceedings of WIAMIS-2005. Montreaux, Switzerland (2005)
Campanella, M., Leonardi R., Migliorati, P.: An intuitive graphic environment for navigation and classification of multimedia documents. In: Proceedings of ICME-2005. Amsterdam, The Netherlands (2005)
Campanella, M., Leonardi, R., Migliorati, P.: The future-viewer visual environment for semantic characterization of video sequences. In: Proceedings of ICIP-2005. Genoa, Italy (2005)
Olsen, K.A., Korfhage, R.R., Sochats, K.M., Spring, M.B., Williams, J.G.: Visualization of a document collection: the VIBE system (1992). http://Itl13.exp.sis.pitt.edu/Website/Webresume/ VIBEPaper/VIBE.htm
Cugini, J., Piatko, C., Laskowsky, S.: Interactive 3D visualization for document retrieval. In: Proceedings of ACM CIKM-1996. Rockville, USA (1996)
Carey, M., Heesch, D.C., Ruger, S.M.: Info navigator: a visualization tool for document searching and browsing. In: Proceedings of DMS-2003. Miami, USA (2003)
Moghaddam, B., Tian Q., Huang, T.S.: Spatial visualization for content-based image retrieval. In: Proceedings of ICME-2001. Tokyo, Japan (2001)
Meiers, T., Keller, S., Sikora, T.: Hierarchical image browsing system with embedded relevence feedback. In: Proceedings of WIAMIS-2003. London, UK (2003)
Lin, C.-Y., Tseng, B.L., Smith, J.R.: VideoAnnEx: IBM MPEG7 annotation tool for multimedia indexing and concept learning. In: Proceedings of CME-2003. Baltimore, USA (2003)
Djordjevic D., Izquierdo E.: An object- and user-driven system for semantic-based image annotation and retrieval. IEEE Trans. Circuits Syst. Video Technol. 17(3), 313–323 (2007)
Rehatschek, H., Bailer, W., Neuschmied, H., Ober, S., Bischof, H.: A tool supporting annotation and analysis of video. Booktitle: Reconfigurations. Interdisciplinary Perspectives on Religion in a Post-Secular Society, Vienna, pp. 253–268 (2007)
Worring, M., Snoek, C.G.M., de Rooij, O., Nguyen, G.P., Smeulders, A.W.M.: The mediamill semantic video search engine. In: Proceedings of ICASSP-2007. Hawaii, USA (2007)
Snoek C.G.M., Worring M., de Rooij O., van de Sande K.E.A., Rong Y., Hauptman A.G.: VideOlympics: real-time evaluation of multimedia retrieval systems. IEEE Multimedia 15(1), 86–91 (2008)
Jackson J.E.: A User’s Guide to Principal Components. Wiley, London (1991)
Barbieri, M., Mekenkamp, G., Ceccarelli, M., Nesvadba, J.: The color browser: a content driven linear video browsing tool. In: Proceedings of ICME-2001. Tokyo, Japan (2001)
Tou J.T., Gonzalez R.C.: Pattern Recognition Principles. Addison-Wesley, Reading (1974)
Vendrig J., Worring M.: Systematic evaluation of logical story unit segmentation. IEEE Trans. Multimedia. 4(4), 492–499 (2002)
Yeung M.M., Yeo B.-L.: Video visualization for compact presentation and fast browsing of pictorial content. IEEE Trans. Circuits Syst. Video Technol. 7(5), 771–785 (1997)
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution,and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Campanella, M., Leonardi, R. & Migliorati, P. Interactive visualization of video content and associated description for semantic annotation. SIViP 3, 183–196 (2009). https://doi.org/10.1007/s11760-008-0071-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-008-0071-6