Ying Zhang
Roger Zimmermann
ABSTRACT
User-generated videos have become increasingly popular in recent years. Due to advances in camera technology it is now very easy and convenient to record videos with mobile devices, such as smartphones. Here we consider an application where users collect and share a large set of videos that are related to a geographic area, say a city. Such a repository can be a great source of information for prospective tourists when they plan to visit a city and would like to get a preview of its main areas. The challenge that we address is how to automatically create a preview video summary from a large set of source videos.
The main features of our technique are that it is fully automatic and leverages meta-data sensor information which is acquired in conjunction with videos. The meta-data is collected from GPS and compass sensors and is used to describe the viewable scenes of the videos. Our method then proceeds in three steps through the analysis of the sensor data. First, we generate a single video summary. Shot boundaries are detected based on different motion types of camera movements and key frames are extracted related to motion patterns. Second, we build video skims for popular places (i.e., hotspots) aiming to provide maximal coverage of hotspot areas with minimal redundancy (per-spot multi-video summary). Finally, the individual hotspot skims are linked together to generate a pleasant video tour that visits all the popular places (multi-spot multi-video summary).
- Geovid. http://geovid.org/.Google Scholar
- K. Aizawa, K. Ishijima, and M. Shiina. Summarizing Wearable Video. In International Conference on Image Processing, volume 3, pages 398--401 vol.3, 2001.Google Scholar
- K. Aizawa, D. Tancharoen, S. Kawasaki, and T. Yamasaki. Efficient Retrieval of Life Log based on Context and Content. In ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, CARPE, pages 22--31, 2004. Google ScholarDigital Library
- Y. Arase, X. Xie, T. Hara, and S. Nishio. Mining People's Trips from Large Scale Geo-tagged Photos. In ACM Multimedia, MM '10, pages 133--142, 2010. Google ScholarDigital Library
- S. Arslan Ay, R. Zimmermann, and S. H. Kim. Viewable Scene Modeling for Geospatial Video Search. In ACM Multimedia, MM '08, pages 309--318, 2008. Google ScholarDigital Library
- Z. Botev, D. Kroese, and T. Taimre. Generalized Cross-entropy Methods with Applications to Rare-event Simulation and Optimization. Simulation, 83:785--806, 2007. Google ScholarDigital Library
- M. Buchin, A. Driemel, M. van Kreveld, and V. Sacristán. An Algorithmic Framework for Segmenting Trajectories based on Spatio-temporal Criteria. In SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS '10, pages 202--211, 2010. Google ScholarDigital Library
- A. Ekin, A. Tekalp, and R. Mehrotra. Automatic Soccer Video Analysis and Summarization. IEEE Transactions on Image Processing, 12(7):796--807, 2003. Google ScholarDigital Library
- C. Graham. Vision and Visual Perception. 1965.Google Scholar
- J. Kim, H. S. Chang, K. Kang, M. Kim, J. Kim, and H. Kim. Summarization of News Video and its Description for Content-based Access. International Journal of Imaging Systems and Technology, 13(5):267--274, 2003.Google ScholarCross Ref
- M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based Multimedia Information Retrieval: State of the Art and Challenges. ACM Transactions on Multimedia Computing, Communications and Applications, 2:1--19, 2006. Google ScholarDigital Library
- Y. Li and B. Merialdo. Multi-video Summarization based on AV-MMR. In Int'l Workshop on Content-Based Multimedia Indexing, CBMI '10, pages 1--6, 2010.Google Scholar
- Y. Li and B. Merialdo. Multi-video summarization based on OB-MMR. In Int'l Workshop on Content-Based Multimedia Indexing, CBMI '11, pages 163--168, 2011.Google Scholar
- X. Lu, C. Wang, J.-M. Yang, Y. Pang, and L. Zhang. Photo2Trip: Generating Travel Routes from Geo-tagged Photos for Trip Planning. In ACM Multimedia, MM '10, pages 143--152, 2010. Google ScholarDigital Library
- Y. Ma, L. Lu, H. Zhang, and M. Li. A User Attention Model for Video Summarization. In ACM Multimedia, MM '02, pages 533--542, 2002. Google ScholarDigital Library
- Microsoft. Image Composite Editor Panorama Stitcher. http://research.microsoft.com/en-us/um/redmond/groups/ivm/ICE/.Google Scholar
- A. G. Money and H. Agius. Video Summarisation: A Conceptual Framework and Survey of the State of the Art. Journal of Visual Communication and Image Represention, 19:121--143, 2008. Google ScholarDigital Library
- I. Otsuka, K. Nakane, A. Divakaran, K. Hatanaka, and M. Ogawa. A Highlight Scene Detection and Video Summarization System using Audio Feature for a Personal Video Recorder. IEEE Transactions on Consumer Electronics, 51(1):112--116, 2005. Google ScholarDigital Library
- J. Shao, D. Jiang, M. Wang, H. Chen, and L. Yao. Multi-video Summarization using Complex Graph Clustering and Mining. Computer Science and Information Systems, 7(1):85--98, 2010.Google ScholarCross Ref
- F. Shipman, A. Girgensohn, and L. Wilcox. Creating Navigable Multi-level Video Summaries. In IEEE International Conference on Multimedia and Expo, ICME '03, pages 753--756, 2003. Google ScholarDigital Library
- D. Tjondronegoro, Y.-P. P. Chen, and B. Pham. Highlights for more Complete Sports Video Summarization. IEEE Multimedia, 11(4):22--37, 2004. Google ScholarDigital Library
- B. T. Truong and S. Venkatesh. Video Abstraction: A Systematic Review and Classification. ACM Transactions on Multimedia Computing, Communications and Applications, 3, 2007. Google ScholarDigital Library
- F. Wang and B. Merialdo. Multi-document Video Summarization. In IEEE International Conference on Multimedia and Expo, ICME '09, pages 1326--1329, 2009. Google ScholarDigital Library
- C. Xu, J. Wang, K. Wan, Y. Li, and L. Duan. Live Sports Event Detection based on Broadcast Video and Web-casting Text. In ACM Multimedia, MM '06, pages 221--230, 2006. Google ScholarDigital Library
Index Terms
- Multi-video summary and skim generation of sensor-rich videos in geo-space
Recommendations
Points of Interest Detection from Multiple Sensor-Rich Videos in Geo-Space
MM '14: Proceedings of the 22nd ACM international conference on MultimediaRecently, the popularity of user generated videos has highlighted efficient video indexing and browsing as an urgent problem. Points of interest (POI) detection is a technique to address this issue by establishing the implicit relationship among ...
Brief and high-interest video summary generation: evaluating the AT&T labs rushes summarizations
TVS '08: Proceedings of the 2nd ACM TRECVid Video Summarization WorkshopVideo summarization is essential for the user to understand the main theme of video sequences in a short period, especially when the volume of the video is huge and the content is highly redundant. In this paper, we present a video summarization system, ...
DVS: a dynamic multi-video summarization system of sensor-rich videos in geo-space
MM '12: Proceedings of the 20th ACM international conference on MultimediaIt is now very easy to produce user generated videos (UGV) due to progress in camera and recording technologies on mobile devices, such as smartphones. Additionally, the ubiquitous, built-in sensors in digital devices can greatly enrich these videos ...
Comments