Skip to main content
SpringerLink
Log in

A novel compact yet rich key frame creation method for compressed video summarization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video summarization has great potential to enable rapid browsing and efficient video indexing in many applications. In this study, we propose a novel compact yet rich key frame creation method for compressed video summarization. First, we directly extract DC coefficients of I frame from a compressed video stream, and DC-based mutual information is computed to segment the long video into shots. Then, we select shots with static background and moving object according to the intensity and range of motion vector in the video stream. Detecting moving object outliers in each selected shot, the optimal object set is then selected by importance ranking and solving an optimum programming problem. Finally, we conduct an improved KNN matting approach on the optimal object outliers to automatically and seamlessly splice these outliers to the final key frame as video summarization. Previous video summarization methods typically select one or more frames from the original video as the video summarization. However, these existing key frame representation approaches for video summarization eliminate the time axis and lose the dynamic aspect of the video scene. The proposed video summarization preserves both compactness and considerably richer information than previous video summaries. Experimental results indicate that the proposed key frame representation not only includes abundant semantics but also is natural, which satisfies user preferences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. http://butlermovies.blogspot.com/2014/02/frozen-2013-movie-free-download.html (accessed 2016. 02.22)

  2. https://www.youtube.com/watch?v=RpXw8eV07_w (accessed 2016.02.24)

  3. ftp://ftp.pets.rdg.ac.uk/pub/PETS2000/ (accessed 2016.02.24)

  4. Acha AR, Pritch Y, Peleg S (2006) Making a long video short: Dynamic video synopsis Proceedings CVPR, pp 435–441

    Google Scholar 

  5. Aner-Wolf A, Kender JR (2004) Video summaries and cross-referencing through mosaic-based representation. Comput Vis Image Underst 95(2):201–237

    Article  Google Scholar 

  6. Avila SEFD, Lopes APB, Luz AD et al (2011) VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recogn Lett 32(1):56–68

    Article  Google Scholar 

  7. Baber J, Afzulpurkar N, Dailey MN et al (2012) Shot boundary detection from videos using entropy and local descriptor 17th International Conference on IEEE, pp 1–6

    Google Scholar 

  8. Cernekova Z, Pitas I, Nikou C (2006) Information theory-based shot cut/fade detection and video summarization. IEEE Trans Circuits Syst Video Technol 16(1):82–91

    Article  Google Scholar 

  9. Chen Q, Li D, Tang CK (2013) KNN Matting. IEEE Trans on PAMI 35 (9):2175–2188

    Article  Google Scholar 

  10. Cheng MM, Zhang GX, Mitra NJ et al (2015) Global contrast based salient region detection. IEEE Trans on PAMI 37(3):569–582

    Article  Google Scholar 

  11. Cong Y, Yuan JS, Luo JB (2012) Towards scalable summarization of consumer videos via sparse dictionary selection. IEEE Trans Multimedia 14(1):66–75

    Article  Google Scholar 

  12. Dhagdi MST, Deshmukh PR (2012) Keyframe based video summarization using automatic threshold and edge matching rate. International Journal of Scientific and Research Publication 2(7):1– 12

    Google Scholar 

  13. Ejaz N, Mehmood I, Baik SW (2014) Feature aggregation based visual attention model for video summarization. Comput Electr Eng 40(3):993–1005

    Article  Google Scholar 

  14. Furini M, Geraci F, Montangero M et al (2010) STIMO: STILl and MOving video storyboard for the web scenario. Multimedia Tools and Applications 46(1):47–69

    Article  Google Scholar 

  15. Gong B, Chao WL, Grauman K et al (2014) Diverse sequential subset selection for supervised video summarization. Advances in Neural Information Processing Systems :2069–2077

  16. Guan G, Wang Z, Lu S et al (2013) Keypoint based keyframe selection. IEEE Trans Circuits Syst Video Technol 23(4):729–734

    Article  Google Scholar 

  17. Gygli M, Grabner H, Gool LV (2015) Video summarization by learning submodular mixtures of objectives Proceedings CVPR, pp 3090–3098

    Google Scholar 

  18. Gygli M, Grabner H, Riemenschneider H et al (2014) Creating summaries from user videos Proceedings ECCV, pp 505–520

    Google Scholar 

  19. Ioannidis A, Chasanis V, Likas A (2016) Weighted multi-view key-frame extraction[J]. Pattern Recogn Lett 72:52–61

    Article  Google Scholar 

  20. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science, New Series 220(4598):671–680

    MathSciNet  MATH  Google Scholar 

  21. Khosla A, Hamid R, Lin CJ et al (2014) Large-scale video summarization using web-image priors Proceedings CVPR, pp 2698–2705

    Google Scholar 

  22. Kolmogorov V, Zabih R (2002) What energy functions can be minimized via graph cuts?[J]. IEEE Trans on PAMI 26(2):147–159

    Article  MATH  Google Scholar 

  23. Li L, Zhou K, Xue GR et al (2011) Video summarization via transferrable structured learning International conference on world wide web, WWW 2011, hyderabad, India, March 28 - April, pp 287–296

    Google Scholar 

  24. Lienhart R, Pfeiffer S, Effelsberg W (1997) Video abstracting. Commun ACM 40(12):54–62

    Article  Google Scholar 

  25. Liu YL, Xiao Y (2013) A robust image hashing algorithm resistant against geometrical attacks. Radio Eng 22(4):1072–1081

    MathSciNet  Google Scholar 

  26. Mei S, Guan G, Wang Z et al (2015) Video summarization via minimum sparse reconstruction. Pattern Recogn 48(2):522–533

    Article  Google Scholar 

  27. Mundur P, Rao Y, Yesha Y (2006) Keyframe-based video summarization using Delaunay clustering. Int J Digit Libr 6(2):219–232

    Article  Google Scholar 

  28. Ngo CW, Pong TC, Zhang HJ (2002) Motion-based video representation for scene change detection. Int J Comput Vis 50(2):127–142

    Article  MATH  Google Scholar 

  29. Nie Y, Xiao C, Sun H et al (2012) Compact video synopsis via global spatiotemporal optimization. In IEEE Trans Vis Comput Graph 19(10):1664–1676

    Article  Google Scholar 

  30. Panagiotakis C, Doulamis A, Tziritas G (2009) Equivalent key frames selection based on iso-content principles. IEEE Trans Circuits Syst Video Technol 19(3):447–451

    Article  Google Scholar 

  31. Panagiotakis C, Ovsepian N, Michael E (2013) Video synopsis based on a sequential distortion minimization method International Conference on Computer Analysis of Images and Patterns Springer Berlin Heidelberg

    Google Scholar 

  32. Potapov D, Douze M, Harchaoui Z et al (2014) Category-specific video summarization Proceedings ECCV, pp 540–555

    Google Scholar 

  33. Pritch Y, Rav-Acha A, Peleg S (2008) Nonchronological video synopsis and indexing. IEEE Trans on PAMI 30(11):1971–1984

    Article  Google Scholar 

  34. The Shawshank Redemption (1994) Full Movie. https://www.youtube.com/watch?v=lrxgVXpsmzY (accessed 2016.02.22)

  35. Vila M, Bardera A, Xu Q et al (2013) Tsallis entropy-based information measures for shot boundary detection and keyframe selection. SIViP 7(3):507–520

    Article  Google Scholar 

  36. Wang J, Bhat P, Colburn RA et al (2005) Interactive video cutout. ACM Transactions on Graphics (ToG). ACM 24(3):585–594

    Google Scholar 

  37. Wang M, Hong R, Li G (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimedia 14(4):975–985

    Article  Google Scholar 

  38. Wu J, Zhong SH, Jiang J et al (2016) A novel clustering method for static video summarization. Multimedia Tools and Applications :1–17

  39. Yarmohammadi H, Rahmati M, Khadivi M (2013) Content based video retrieval using information theory Proceedings Iran Conferernce Machine Vis and Image Processing

    Google Scholar 

  40. Zhang Q, Yu SP, Zhou DS et al (2011) An efficient method of key-frame extraction based on a cluster algorithm. Journal of Human Kinetics 39(1):5–13

    Google Scholar 

  41. Zhao B, Xing P (2014) Quasi real-time summarization for consumer videos Proceedings CVPR, pp 2513–2520

    Google Scholar 

  42. Zhao L, Qi W, Li SZ et al (2000) Key-frame extraction and shot retrieval using nearest feature line (NFL) Proceedings 2000 ACM workshops on Multimedia, pp 217–220

    Chapter  Google Scholar 

  43. Zhou X, Yang C, Yu W (2013) Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans on PAMI 35(3):597–610

    Article  Google Scholar 

  44. Zhu X, Loy CC, Gong S (2013) Video synopsis by heterogeneous multi-source correlation Proceedings ICCV, pp 81–88

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, 310027, China

    Mengjuan Fei, Wei Jiang & Weijie Mao

Authors
  1. Mengjuan Fei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weijie Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fei, M., Jiang, W. & Mao, W. A novel compact yet rich key frame creation method for compressed video summarization. Multimed Tools Appl 77, 11957–11977 (2018). https://doi.org/10.1007/s11042-017-4843-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4843-2

Keywords

Search

Navigation