Abstract
Video summarization deals with the generation of a condensed version of the original video by including meaningful frames or segments while eliminating redundant information. The main challenge in a video summarization task is to identify important frames or segments corresponding to human perception which varies from one genre to another. In the past two decades, several summarization techniques ranging from conventional non-learning to deep learning based mechanisms have been developed. This study provides a comprehensive survey focusing on the massive literature with scope ranging from general to domain specific methods, single view to multi-view processes, generic to user-interaction based mechanisms and conventional to deep learning-based approaches. The presented work provides general pipeline and broad classification of video summarization systems. The survey also presents genre-wise datasets description, various evaluation techniques and future recommendations. The key-points of presented work lie in its approach of analyzing literature in a systematic manner and its wide coverage by including some of the domains that have been overlooked over the time like aerial videos, medical videos and user-customization based approaches. The research work in each category is investigated, compared and analyzed on the basis of various intrinsic characteristics. The main objective of this manuscript is to guide future researchers about state-of-the-art work done in various domains of the video summarization field, so that the scope and performance of automatic video summarization systems can be enhanced further by designing new approaches or by improving different existing techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abd-Almageed, W. (2008). Online, simultaneous shot boundary detection and key frame extraction for sports videos using rank tracing. In: 2008 15th IEEE International Conference on Image Processing. IEEE, pp 3200–3203
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(1):1–8
Agyeman R, Muhammad R, Choi GS (2019) Soccer video summarization using deep learning. In: Proceedings—2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, pp 270–273. https://doi.org/10.1109/MIPR.2019.00055
Ai X, Song Y, Li Z (2018) Unsupervised video summarization based on consistent clip generation. In: 2018 IEEE 4th International Conference on Multimedia Big Data, BigMM 2018, pp 1–7. https://doi.org/10.1109/BigMM.2018.8499188
Aktar R, AliAkbarpour H, Bunyak F, Kazic T, Seetharaman G, Palaniappan K (2018) Geospatial content summarization of UAV aerial imagery using mosaicking. In: Proceedings of SPIE 10645, Geospatial Informatics, Motion Imagery, and Network Analytics VIII, 106450I, April, 18. https://doi.org/10.1117/12.2309417
Alam MS, Natesha BV, Ashwin TS, Guddeti RMR (2019) UAV based cost-effective real-time abnormal event detection using edge computing. Multimed Tools Appl 78(24):35119–35134. https://doi.org/10.1007/s11042-019-08067-1
Alcantarilla PF, Nuevo J, Bartoli A (2013) Fast explicit diffusion for accelerated features in nonlinear scale spaces. In: BMVC 2013—Electronic Proceedings of the British Machine Vision Conference 2013. https://doi.org/10.5244/C.27.13
Alexe B, Deselaers T, Ferrari V (2012) Measuring the objectness of image windows. IEEE Trans Pattern Anal Mach Intell 34(11):2189–2202. https://doi.org/10.1109/TPAMI.2012.28
Almeida J, Leite NJ, Torres RDS (2012) VISON: VIdeo Summarization for ONline applications. Pattern Recogn Lett 33(4):397–409. https://doi.org/10.1016/j.patrec.2011.08.007
Amel AM, Abdessalem BA, Abdellatif M (2010) Video shot boundary detection using motion activity descriptor. J Telecommun 2(1):54–59
Anirudh R, Masroor A, Turaga P (2016) Diversity promoting online sampling for streaming video summarization. In: 2016 IEEE International Conference on Image Processing (ICIP), pp 2–6
Apostolidis E (2021) Combining global and local attention with positional encoding for video summarization. In: IEEE International Symposium on Multimedia (ISM)
Apostolidis E, Metsai AI, Adamantidou E, Mezaris V, Patras I (2019) A stepwise, label-based approach for improving the adversarial training in unsupervised video summarization. In: AI4TV 2019—Proceedings of the 1st International Workshop on AI for Smart TV Content Production, Access and Delivery, Co-Located with MM 2019, pp 17–25. https://doi.org/10.1145/3347449.3357482
Apostolidis E, Adamantidou E, Metsai AI, Mezaris V, Patras I (2020a) AC-SUM-GAN: connecting actor-critic and generative adversarial networks for unsupervised video summarization. IEEE Trans Circuits Syst Video Technol 1–15
Apostolidis E, Adamantidou E, Metsai AI, Mezaris V, Patras I (2020b) Unsupervised video summarization via attention-driven adversarial learning. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11961 LNCS(Mmm), pp 492–504. https://doi.org/10.1007/978-3-030-37731-1_40
Apostolidis E, Adamantidou E, Metsai AI, Mezaris V, Patras I (2021) Video summarization using deep neural networks: a survey. Proc IEEE 109(11):1838–1863. https://doi.org/10.1109/JPROC.2021.3117472
Apostolidis E, Balaouras G, Mezaris V, Patras I (2022) Summarizing videos using concentrated attention and considering the uniqueness and diversity of the video frames. In: Proceedings of the 2022 International Conference on Multimedia Retrieval, pp 407–415. https://doi.org/10.1145/3512527.3531404
Archana N, Malmurugan N (2021) Multi-edge optimized LSTM RNN for video summarization. J Ambient Intell Humaniz Comput 12(5):5381–5395. https://doi.org/10.1007/s12652-020-02025-8
Asadi E, Charkari NM (2012) Video summarization using fuzzy C-means clustering. In: 20th Iranian Conference on Electrical Engineering (ICEE2012). IEEE, pp 690–694
Asha Paul MK, Kavitha J, Jansi Rani PA (2018) Key-frame extraction techniques: a review. Recent Patents Comput Sci 11(1):3–16. https://doi.org/10.2174/2213275911666180719111118
Avila S, Eliza S, De Avila F, Paula A, Lopes BL Jr, De Albuquerque A (2011) VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recogn Lett 32(1):56–68. https://doi.org/10.1016/j.patrec.2010.08.004
Avola D, Cinque L, Foresti GL, Martinel N, Pannone D, Piciarelli C (2020) A UAV video dataset for mosaicking and change detection from low-altitude flights. IEEE Trans Syst Man Cybern: Syst 50(6):2139–2149. https://doi.org/10.1109/TSMC.2018.2804766
Avola D, Foresti GL, Martinel N, Micheloni C, Pannone D, Piciarelli C (2017) Real-time incremental and geo-referenced mosaicking by small-scale uavs. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10484 LNCS, pp 694–705. https://doi.org/10.1007/978-3-319-68560-1_62
Banwaskar MR, Rajurkar AM (2020) Creating video summary using speeded up robust features. Appl Comput vis Image Process Proc ICCET 2020:01
Basavarajaiah M, Sharma P (2021) GVSUM: generic video summarization using deep visual features. Multimed Tools Appl 80(9):14459–14476. https://doi.org/10.1007/s11042-020-10460-0
Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24:509. https://doi.org/10.1109/TIP.2004.826126
Ben-Hur A, Weston J (2010) A user’s guide to support vector machines. Methods Mol Biol (clifton, N.J.) 609(January 2010):223–239. https://doi.org/10.1007/978-1-60327-241-4_13
Bi C, Yuan Y, Zhang J, Shi Y, Xiang Y, Wang Y, Zhang R (2018) Dynamic mode decomposition based video shot detection. IEEE Access 6(March 2019):21397–21407. https://doi.org/10.1109/ACCESS.2018.2825106
Blank M, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as space-time shapes. Proc IEEE Int Conf Comput vis II:1395–1402. https://doi.org/10.1109/ICCV.2005.28
Bleakley K, Vert J-P (2011) The group fused Lasso for multiple change-point detection. 1–25. https://arXiv.org/1106.4199
Boutsidis C, Mahoney MW, Drineas P (2009) An improved approximation algorithm for the column subset selection problem. In: Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 968–977. https://doi.org/10.1137/1.9781611973068.105
Breszcz M, Breckon TP, Cowling I (2011) Real-time mosaicing from unconstrained video imagery for UAV applications. In: Proceedings of the 26th International Unmanned Air Vehicle Systems Conference, pp 1–15. http://breckon.eu/toby/publications/papers/breszcz11uavmosaic.pdf. Accessed 25 March 2011
Broder AZ, Karlin AR, Raghavan P, Upfal E (1994) Trading space for time in undirected s-t connectivity. SIAM J Comput 23(2):324–334. https://doi.org/10.1137/S0097539790190144
Cai S, Zuo W, Davis LS, Zhang L (2018) Weakly-supervised Video Summarization using Variational Encoder-Decoder and Web Prior. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 184–200
Chamasemani FF, Khalid F (2017) Video abstraction using density-based clustering algorithm. Vis Comput. https://doi.org/10.1007/s00371-017-1432-3
Chen BW, Wang JC, Wang JF (2009) A novel video summarization based on mining the story-structure and semantic relations among concept entities. IEEE Trans Multimed 11(2):295–312. https://doi.org/10.1109/TMM.2008.2009703
Chen Y, Zhang B (2014) Surveillance video summarisation by jointly applying moving object detection and tracking. Int J Comput vis Robot 4(3):212–234. https://doi.org/10.1504/IJCVR.2014.062936
Chen J, Zou Y, Wang Y (2016) Wireless capsule endoscopy video summarization: a learning approach based on Siamese neural network and support vector machine. In: Proceedings—International Conference on Pattern Recognition, pp 1303–1308. https://doi.org/10.1109/ICPR.2016.7899817
Chen J, Wang Y, Chen Z, Zou Y (2017) Sequence-guided siamese neural network for video summarization of unmanned aerial vehicles. In: International Conference on Digital Signal Processing, DSP, 2017-Augus. https://doi.org/10.1109/ICDSP.2017.8096070
Chen Y, Tao L, Wang X, Yamasaki T (2019a) Weakly supervised video summarization by hierarchical reinforcement learning. In: 1st ACM International Conference on Multimedia in Asia, MMAsia 2019a. https://doi.org/10.1145/3338533.3366583
Choi J, Oh TH, Kweon IS (2018a) Contextually customized video summaries via natural language. In: Proceedings—2018a IEEE Winter Conference on Applications of Computer Vision, WACV 2018, 2018-Janua, pp 1718–1726. https://doi.org/10.1109/WACV.2018.00191
Choudary C, Liu T (2007) Summarization of visual content in instructional videos. IEEE Trans Multimed 9(7):1443–1455. https://doi.org/10.1109/TMM.2007.906602
Chu WS, Song Y, Jaimes A (2015) Video co-summarization: video summarization by visual co-occurrence. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12-June, pp 3584–3592. https://doi.org/10.1109/CVPR.2015.7298981
Cisco (2020) Cisco Annual Internet Report (2018–2023). Cisco, 1–41. http://grs.cisco.com/grsx/cust/grsCustomerSurvey.html?SurveyCode=4153&ad_id=US-BN-SEC-M-CISCOASECURITYRPT-ENT&KeyCode=000112137. Accessed 10 July 2021
Cizmeciler K, Erdem E, Erdem A (2022) Leveraging semantic saliency maps for query-specific video summarization. Multimed Tools Appl 81(12):17457–17482. https://doi.org/10.1007/s11042-022-12442-w
Cong Y, Yuan J, Luo J (2012) Towards scalable summarization of consumer videos via sparse dictionary selection. IEEE Trans Multimed 14(1):66–75. https://doi.org/10.1109/TMM.2011.2166951
Cong Y, Liu J, Sun G, You Q, Li Y, Luo J (2017) Adaptive greedy dictionary selection for web media summarization. IEEE Trans Image Process 26(1):185–195. https://doi.org/10.1109/TIP.2016.2619260
Crete F, Dolmiere T, Ladret P, Nicolas M (2007) The blur effect: perception and estimation with a new no-reference perceptual blur metric. Hum vis Electron Imaging XII 6492(March):64920I. https://doi.org/10.1117/12.702790
Cutler A (1994) Archetypal analysis. Technometrics 36(4):338–347
Davids DM, Christopher CS (2021) An efficient video summarization for surveillance system using normalized k-means and quick sort method. Microprocess Microsyst 83(September 2020):103960. https://doi.org/10.1016/j.micpro.2021.103960
Davila K, Zanibbi R (2017) Whiteboard video summarization via spatio-temporal conflict minimization. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). IEEE, p 1
Davila K, Xu F, Setlur S, Govindaraju V (2021) FCN-lecturenet: extractive summarization of whiteboard and chalkboard lecture videos. IEEE Access 9:104469–104484. https://doi.org/10.1109/ACCESS.2021.3099427
Dogra DP, Ahmed A, Bhaskar H (2016) Smart video summarization using mealy machine-based trajectory modelling for surveillance applications. Multimed Tools Appl 75(11):6373–6401. https://doi.org/10.1007/s11042-015-2576-7
Ejaz N, Tariq TB, Baik SW (2012) Adaptive key frame extraction for video summarization using an aggregation mechanism. J vis Commun Image Represent 23(7):1031–1040. https://doi.org/10.1016/j.jvcir.2012.06.013
Ejaz N, Mehmood I, Baik SW (2013a) MRT Letter: visual attention driven framework for hysteroscopy video abstraction. Microsc Res Tech 563(January):559–563. https://doi.org/10.1002/jemt.22205
Ejaz N, Mehmood I, Wook Baik S (2013b) Efficient visual attention based framework for extracting key frames from videos. Signal Process Image Commun 28(1):34–44. https://doi.org/10.1016/j.image.2012.10.002
Ejaz N, Baik SW, Majeed H, Chang H, Mehmood I (2018) Multi-scale contrast and relative motion-based key frame extraction. EURASIP J Image Video Process 2018:1
Elfeki M, Borji A (2019) Video summarization via actionness ranking. In: Proceedings—2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019, pp 754–763. https://doi.org/10.1109/WACV.2019.00085
Elfeki M, Wang L, Borji A (2022) Multi-stream dynamic video summarization. In: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp 185–195
Elhamifar E, Sapiro G, Vidal R (2012) See all by looking at a few: sparse modeling for finding representative objects. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1600–1607. https://doi.org/10.1109/CVPR.2012.6247852
Elhamifar E, De Paolis Kaluza MC (2017) Online summarization via submodular and convex optimization. In: Proceedings—30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-Janua, pp 1818–1826. https://doi.org/10.1109/CVPR.2017.197
Emam AZ, Ali YA, Ben Ismail MM (2015) Adaptive features extraction for Capsule Endoscopy (CE) video summarization. In: Proceedings—International Conference on Computer Vision and Image Analysis Applications, ICCVIA 2015, October. https://doi.org/10.1109/ICCVIA.2015.7351879
Etezadifar P, Farsi H (2017) Scalable video summarization via sparse dictionary learning and selection simultaneously. Multimed Tools Appl 76(6):7947–7971. https://doi.org/10.1007/s11042-016-3433-z
Evangelopoulos G, Rapantzikos K, Potamianos A, Maragos P, Zlatintsi A, Avrithis Y (2008) Movie summarization based on audiovisual saliency detection. In: Proceedings—International Conference on Image Processing, ICIP, May 2014, pp 2528–2531. https://doi.org/10.1109/ICIP.2008.4712308
Evangelopoulos G, Zlatintsi A, Potamianos A, Maragos P, Rapantzikos K, Skoumas G, Avrithis Y (2013) Multimodal saliency and fusion for movie summarization based on aural, visual, and textual attention. IEEE Trans Multimed 15(7):1553–1568. https://doi.org/10.1109/TMM.2013.2267205
Fajtl J, Sokeh HS, Argyriou V, Monekosso D, Remagnino P (2019) Summarizing Videos with Attention. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11367 LNCS, pp 39–54. https://doi.org/10.1007/978-3-030-21074-8_4
Fathi A, Li Y, Rehg JM (2012) Learning to recognize daily actions using gaze. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7572 LNCS(PART 1), pp 314–327. https://doi.org/10.1007/978-3-642-33718-5_23
Felzenszwalb P, McAllester D, Ramanan D (2014) A Discriminatively trained, multiscale, deformable part model. Proc IEEE Comput Soc Conf Comput vis Pattern Recogn 330(6):1299–1305
Feng L, Kuang Z, Li Z, Zhang W (2018) Extractive video summarizer with memory augmented neural networks. In: MM 2018—Proceedings of the 2018 ACM Multimedia Conference, pp 976–983. https://doi.org/10.1145/3240508.3240651
Fernandes P, Allamanis M, Brockschmidt M (2019) Structured neural summarization. In: 7th International Conference on Learning Representations, ICLR 2019, 2018, pp 1–18
Fu H, Wang H (2021) Self-attention binary neural tree for video summarization. Pattern Recogn Lett 143:19–26. https://doi.org/10.1016/j.patrec.2020.12.016
Fu Y, Guo Y, Zhu Y, Liu F, Song C, Zhou Z, Member S (2010) Multi-view video summarization. IEEE Trans Multimed 12(7):717–729. https://doi.org/10.1109/TMM.2010.2052025
Fu TJ, Tai SH, Chen HT (2019) Attentive and adversarial learning for video summarization. In: Proceedings—2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019, pp 1579–1587. https://doi.org/10.1109/WACV.2019.00173
Furini M, Geraci F, Montangero M, Pellegrini M (2010) STIMO: STIll and MOving video storyboard for the web scenario. Multimed Tools Appl 46(1):47–69. https://doi.org/10.1007/s11042-009-0307-7
Garcia A, Boix X, Lim J, Tan A (2017) Active video summarization: customized summaries via on-line interaction with the user. In: Thirty-First AAAI Conference on Artificial Intelligence, pp 4046–4052
Gavião W, Scharcanski J, Frahm JM, Pollefeys M (2012) Hysteroscopy video summarization and browsing by estimating the physician’s attention on video segments. Med Image Anal 16(1):160–176. https://doi.org/10.1016/j.media.2011.06.008
Ghauri JA, Hakimov S, Ewerth R (2020) Classification of important segments in educational videos using multimodal features. In: CEUR Workshop Proceedings, p 2699
Gianluigi C, Raimondo S (2006) An innovative algorithm for key frame extraction in video summarization. J Real-Time Image Proc 1(1):69–88. https://doi.org/10.1007/s11554-006-0001-1
Gong B, Chao WL, Grauman K, Sha F (2014) Diverse sequential subset selection for supervised video summarization. Adv Neural Inf Process Syst 3(January):2069–2077
Gonuguntla N, Mandal B, Puhan N (2019) Enhanced deep video summarization network. In: 30th British Machine Vision Conference, pp 1–9
Gupta D, Sharma A (2021) Attentive convolution network-based video summarization. Lecture Notes Electr Eng 778:333–346. https://doi.org/10.1007/978-981-16-3067-5_25
Gygli M, Van Gool L (2015) Video summarization by learning submodular mixtures of objectives. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3090–3098
Gygli M, Grabner H, Riemenschneider H, Van Gool L (2014) Creating summaries from user videos. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8695 LNCS(PART 7), pp 505–520. https://doi.org/10.1007/978-3-319-10584-0_33
Habeeb NJ, Mohammed RS, Abbass MK (2016) Surveillance video summarization based on histogram differencing and sum conditional variance. Int J Comput Inf Eng 10(9):1674–1679
Habib HA, Mufti M (2005) Gesture recognition based framework for video lecture handout generation: a video summarization application. WSEAS Trans Syst 4(11):2109–2114
Han B, Hamm J, Sim J (2011) Personalized video summarization with human in the loop. In: 2011 IEEE Workshop on Applications of Computer Vision, WACV 2011, pp 51–57. https://doi.org/10.1109/WACV.2011.5711483
Han J, Ji X, Hu X, Han J, Liu T (2014) Clustering and retrieval of video shots based on natural stimulus fMRI. Neurocomputing 144:128–137. https://doi.org/10.1016/j.neucom.2013.11.052
Han MX, Hu HM, Liu Y, Zhang C, Tian RP, Zheng J (2017) An auto-encoder-based summarization algorithm for unstructured videos. Multimed Tools Appl 76(23):25039–25056. https://doi.org/10.1007/s11042-017-4485-4
Haq IU, Ullah A, Muhammad K, Lee MY, Baik SW (2019) Personalized movie summarization using deep CNN-assisted facial expression recognition. Complexity 2019:1–10
Harel J, Radmann C, Perona P (1994) Graph-based visual saliency Medgg. Adv Neural Inf Process Syst 13:35–54
Hari R, Wilscy M (2015) Event detection in cricket videos using intensity projection profile of Umpire gestures. In: 11th IEEE India Conference: Emerging Trends and Innovation in Technology, INDICON 2014, pp 30–35. https://doi.org/10.1109/INDICON.2014.7030519
He X, Hua Y, Song T, Zhang Z, Xue Z, Ma R, Robertson N, Guan H (2019) Unsupervised video summarization with attentive conditional generative adversarial networks. In: MM 2019—Proceedings of the 27th ACM International Conference on Multimedia, pp 2296–2304. https://doi.org/10.1145/3343031.3351056
Hesham M, Hani B, Fouad N, Amer E (2018) Smart trailer: automatic generation of movie trailer using only subtitles. In: Proceedings of IWDRL 2018: 2018 1st International Workshop on Deep and Representation Learning, pp 26–30. https://doi.org/10.1109/IWDRL.2018.8358211
Ho HI, Chiu WC, Wang YCF (2018) Summarizing first-person videos from third persons’ points of views. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11219 LNCS, pp 72–89. https://doi.org/10.1007/978-3-030-01267-0_5
Hu T, Li Z (2018) Video summarization via exploring the global and local importance. Multimed Tools Appl 77(17):22083–22098. https://doi.org/10.1007/s11042-017-5479-y
Hu W, Xie N, Li L, Zeng X, Maybank S (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern Part C Appl Rev 41(6):797–819. https://doi.org/10.1109/TSMCC.2011.2109710
Huang D, Cai X, Wang CD (2019) Unsupervised feature selection with multi-subspace randomization and collaboration. Knowl-Based Syst. https://doi.org/10.1016/j.knosys.2019.07.027
Huang JH, Worring M (2020) Query-controllable video summarization. In: ICMR 2020—Proceedings of the 2020 International Conference on Multimedia Retrieval, pp 242–250. https://doi.org/10.1145/3372278.3390695
Hussain T, Muhammad K, Ser JD, Baik SW, De Albuquerque VHC (2020a) Intelligent embedded vision for summarization of multiview videos in IIoT. IEEE Trans Ind Inf 16(4):2592–2602. https://doi.org/10.1109/TII.2019.2937905
Hussain T, Muhammad K, Ullah A, Cao Z, Baik SW, De Albuquerque VHC (2020b) Cloud-assisted multiview video summarization using CNN and bidirectional LSTM. IEEE Trans Ind Inf 16(1):77–86. https://doi.org/10.1109/TII.2019.2929228
Hussain T, Muhammad K, Ding W, Lloret J, Wook S, Hugo V, Albuquerque CD (2021a) A comprehensive survey of multi-view video summarization. Pattern Recogn J. https://doi.org/10.1016/j.patcog.2020.107567
Hussain T, Muhammad K, Ullah A, Ser JD, Gandomi AH, Sajjad M, Baik SW, De Albuquerque VHC (2021b) Multiview summarization and activity recognition meet edge computing in IoT environments. IEEE Internet Things J 8(12):9634–9644. https://doi.org/10.1109/JIOT.2020.3027483
Ide I, Zhang Y, Tanishige R, Doman K, Kawanishi Y, Deguchi D, Murase H (2017) Summarization of News Videos Considering the Consistency of Auditory and Visual Contents. In: Proceedings—2017 IEEE International Symposium on Multimedia, ISM 2017, 2017-Janua, pp 193–199. https://doi.org/10.1109/ISM.2017.33
Ioannis K, Tsevas S, Maglogiannis I, Iakovidis DK (2010) Enabling distributed summarization of wireless capsule endoscopy video. In: 2010 IEEE International Conference on Imaging Systems and Techniques, IST 2010—Proceedings, pp 17–21. https://doi.org/10.1109/IST.2010.5548478
Iparraguirre J, Nacional UT, Delrieux CA (2014) Online video summarization based on local features. Int J Multimed Data Eng Manage. https://doi.org/10.4018/ijmdem.2014040103
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. In: 1254 IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, issue no. 11, pp 295–297. https://doi.org/10.1111/j.1463-1318.2005.00780.x
Jacob H, Pádua FLC, Lacerda A, Pereira ACM (2017) A video summarization approach based on the emulation of bottom-up mechanisms of visual attention. J Intell Inf Syst 49(2):193–211. https://doi.org/10.1007/s10844-016-0441-4
Jappie Z, Torpey D, Celik T (2020) SummaryNet: a multi-stage deep learning model for automatic video summarisation. https://arXiv.org/2002.09424
Javed A, Bajwa KB, Malik H, Irtaza A, Malunood MT (2017) A hybrid approach for summarization of cricket videos. In: 2016 IEEE International Conference on Consumer Electronics-Asia, ICCE-Asia 2016, May 2019. https://doi.org/10.1109/ICCE-Asia.2016.7804835
Javed A, Irtaza A, Khaliq Y, Malik H, Tariq M (2019a) Replay and key-events detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine. Appl Intell 49(8):2899–2917
Javed A, Irtaza A, Malik H, Mahmood MT, Adnan S (2019b) Multimodal framework based on audio-visual features for summarisation of cricket videos. IET Image Proc 13(4):615–622. https://doi.org/10.1049/iet-ipr.2018.5589
Jeong D, Yoo HJ, Cho NI (2017) Open Access A static video summarization method based on the sparse coding of features and representativeness of frames. EURASIP J Image Video Process. https://doi.org/10.1186/s13640-016-0122-9
Jesorsky O, Kirchberg KJ, Frischholz RW (2001) Robust face detection using the Hausdorff Distance. Gesture 90–95.
Ji Z, Zhang Y, Pang Y, Li X (2018) Hypergraph dominant set based multi-video summarization. Signal Process 148:114–123. https://doi.org/10.1016/j.sigpro.2018.01.028
Ji Z, Ma Y, Pang Y, Li X (2019a) Query-aware sparse coding for web multi-video summarization. Inf Sci 478:152–166. https://doi.org/10.1016/j.ins.2018.09.050
Ji Z, Zhang Y, Pang Y, Li X, Pan J (2019b) Multi-video summarization with query-dependent weighted archetypal analysis. Neurocomputing 332:406–416. https://doi.org/10.1016/j.neucom.2018.12.038
Ji Z, Jiao F, Pang Y, Shao L (2020a) Deep attentive and semantic preserving video summarization. Neurocomputing 405:200–207. https://doi.org/10.1016/j.neucom.2020.04.132
Ji Z, Xiong K, Pang Y, Li X (2020b) Video summarization with attention-based encoder-decoder networks. IEEE Trans Circuits Syst Video Technol 30(6):1709–1717. https://doi.org/10.1109/TCSVT.2019.2904996
Ji Z, Zhao Y, Pang Y, Li X (2020c) Cross-modal guidance based auto-encoder for multi-video summarization. Pattern Recogn Lett 135:131–137. https://doi.org/10.1016/j.patrec.2020.04.011
Ji Z, Zhao Y, Pang Y, Li X, Han J (2021) Deep attentive video summarization with distribution consistency learning. IEEE Trans Neural Netw Learn Syst 32(4):1765–1775. https://doi.org/10.1109/TNNLS.2020.2991083
Jiang X (2009) Feature extraction for image recognition and computer vision. In: Proceedings—2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009, pp. 1–15. https://doi.org/10.1109/ICCSIT.2009.5235014
Jiang P, Han Y (2019) Hierarchical variational network for user-diversified & query-focused video summarization. In: ICMR 2019—Proceedings of the 2019 ACM International Conference on Multimedia Retrieval, pp 202–206. https://doi.org/10.1145/3323873.3325040
Jin H, Song Y, Yatani K (2017) ElasticPlay: interactive video summarization with dynamic time budgets. In: MM 2017—Proceedings of the 2017 ACM Multimedia Conference, pp 1164–1172. https://doi.org/10.1145/3123266.3123393
Jodoin JP, Bilodeau GA, Saunier N (2014) Urban tracker: multiple object tracking in urban mixed traffic. In: 2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014, pp 885–892. https://doi.org/10.1109/WACV.2014.6836010
Joho H, Jose J (2009) Exploiting facial expressions for affective video summarisation. In: Proceedings of the ACM International Conference on Image and Video Retrieval, 2009(Civr)
Jung Y, Cho D, Kim D, Woo S, Kweon IS (2019) Discriminative feature learning for unsupervised video summarization. In: 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, pp 8537–8544. https://doi.org/10.1609/aaai.v33i01.33018537
Jung Y, Cho D, Woo S, Kweon IS (2020) Global-and-Local Relative Position Embedding for Unsupervised Video Summarization. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12370 LNCS, pp 167–183. https://doi.org/10.1007/978-3-030-58595-2_11
Kanehira A, Van Gool L, Ushiku Y, Harada T (2018) Viewpoint-Aware Video Summarization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 7435–7444. https://doi.org/10.1109/CVPR.2018.00776
Kannan R, Ghinea G, Swaminathan S (2015) What do you wish to see? A summarization system for movies based on user preferences. Inf Process Manage 51(3):286–305. https://doi.org/10.1016/j.ipm.2014.12.001
Kato K, Ide I, Deguchi D, Murase H (2014) Estimation of the representative story transition in a chronological semantic structure of news topics. In: ICMR 2014—Proceedings of the ACM International Conference on Multimedia Retrieval 2014, pp 487–490. https://doi.org/10.1145/2578726.2578800
Kato K, Ide I, Deguchi D, Murase H (2015) Generation of a video summary on a news topic based on SNS responses to news stories. In: CrowdMM 2015 - Proceedings of the 4th International Workshop on Crowdsourcing for Multimedia, Co-Located with MM 2015, pp 21–26. https://doi.org/10.1145/2810188.2810189
Katti H, Yadati K, Kankanhalli M, Tat-Seng C (2011) Affective video summarization and story board generation using pupillary dilation and eye gaze. In: Proceedings—2011 IEEE InternationalSymposium on Multimedia, ISM 2011, pp 319–326. https://doi.org/10.1109/ISM.2011.57
Kaushal V, Kothawade S, Iyer R, Ramakrishnan G (2020) Realistic video summarization through VISIOCITY: a new benchmark and evaluation framework. In: AI4TV 2020—Proceedings of the 2nd International Workshop on AI for Smart TV Content Production, Access and Delivery, pp 37–44. https://doi.org/10.1145/3422839.3423064
Khan YS (2015) Video summarization: survey on event detection and summarization in soccer videos. Int J Adv Comput Sci Appl (IJACSA) 6(11):256–259
Khan AA, Shao J, Ali W, Tumrani S (2020) Content-aware summarization of broadcast sports videos: an audio-visual feature extraction approach. Neural Process Lett 52(3):1945–1968. https://doi.org/10.1007/s11063-020-10200-3
Khosla A, Hamid R, Lin CJ, Sundaresan N (2013) Large-scale video summarization using web-image priors. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 2698–2705. https://doi.org/10.1109/CVPR.2013.348
Kim JG, Chang HS, Kim YT, Kang K, Kim M, Kim J, Kim HM (2002) Multimodal approach for summarizing and indexing news video. ETRI J 24(1):1–11. https://doi.org/10.4218/etrij.02.0102.0101
Kolekar MH (2011) Bayesian belief network based broadcast sports video indexing. Multimed Tools Appl 54(1):27–54. https://doi.org/10.1007/s11042-010-0544-9
Koutras P, Maragos P (2019) SUSiNet: see, understand and summarize it. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2019-June, pp 809–819. https://doi.org/10.1109/CVPRW.2019.00109
Koutras P, Zlatintsi A., Iosif E, Katsamanis A, Maragos P, Potamianos A (2015) Predicting audio-visual salient events based on visual, audio and text modalities for movie summarization. In: Proceedings—International Conference on Image Processing, ICIP, 2015-Decem, pp 4361–4365. https://doi.org/10.1109/ICIP.2015.7351630
Kuanar SK, Ranga KB, Chowdhury AS (2015) Multi-view video summarization using bipartite matching constrained optimum-path forest clustering. IEEE Trans Multimed 17(8):1166–1173
Kumar M, Loui AC (2011) Key frame extraction from consumer videos using sparse representation. Proc Int Conf Image Process ICIP 1:2437–2440. https://doi.org/10.1109/ICIP.2011.6116136
Kumar K, Shrimankar DD, Singh N (2017) Event BAGGING: a novel event summarization approach in multiview surveillance videos. In: Proceedings of 2017 International Conference on Innovations in Electronics, Signal Processing and Communication, IESC 2017, April, pp 106–111. https://doi.org/10.1109/IESPC.2017.8071874
Kumar K, Shrimankar DD, Singh N (2018a) Eratosthenes sieve based key-frame extraction technique for event summarization in videos. Multimed Tools Appl. https://doi.org/10.1007/s11042-017-4642-9
Kumar K, Shrimankar DD, Singh N (2018b) SOMES : an efficient SOM technique for event summarization in multi-view surveillance videos. In: Sa PK, Bakshi S (eds) Recent findings in intelligent computing techniques. Springer, Singapore, pp 383–389. https://doi.org/10.1007/978-981-10-8633-5
Lai PK, Decombas M, Moutet K, Laganiere R (2016) Video summarization of surveillance cameras. In: 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2016, August, pp 286–294. https://doi.org/10.1109/AVSS.2016.7738018
Lal S, Duggal S, Sreedevi I (2019) Online video summarization: predicting future to better summarize present. In: Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019, pp 471–480. https://doi.org/10.1109/WACV.2019.00056
Lan L, Ye C (2021) Recurrent generative adversarial networks for unsupervised WCE video summarization. Knowl Based Syst 222:106971. https://doi.org/10.1016/j.knosys.2021.106971
Lee YJ, Ghosh J, Grauman K (2012) Discovering important people and objects for egocentric video summarization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1346–1353. https://doi.org/10.1109/CVPR.2012.6247820
Lei J, Luan Q, Song X, Liu X, Tao D, Song M (2019) Action parsing-driven video summarization based on reinforcement learning. IEEE Trans Circuits Syst Video Technol 29(7):2126–2137. https://doi.org/10.1109/TCSVT.2018.2860797
Leszczuk M, Grega M, Koźbiał A, Gliwski J, Wasieczko K, Smaïli K (2017) Video summarization framework for newscasts and reports—work in progress. Commun Comput Inf Sci 785(October):86–97. https://doi.org/10.1007/978-3-319-69911-0_7
Li Y, Merialdo B (2016) Multimedia maximal marginal relevance for multi-video summarization. Multimed Tools Appl 75(1):199–220. https://doi.org/10.1007/s11042-014-2287-5
Li Z, Yang L (2021) Weakly supervised deep reinforcement learning for video summarization with semantically meaningful reward. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp 3239–3247. https://doi.org/10.1109/WACV48630.2021.00328.
Li P, Guo Y, Sun H (2011) Multi-keyframe abstraction from videos. In 2011 18th IEEE International Conference on Image Processing, IEEE, November 2016, pp 2473–2476. https://doi.org/10.1109/ICIP.2011.6116162
Li J, Yang T, Yu J, Lu Z, Lu P, Jia X, Chen W (2014) Fast aerial video stitching. Int J Adv Robot Syst. https://doi.org/10.5772/59029
Li X, Zhao B, Lu X, Member S (2017) A general framework for edited video and raw video summarization. IEEE Trans Image Process 26(8):3652–3664
Li P, Tang C, Xu X (2021a) Video summarization with a graph convolutional attention network. Front Inf Technol Electron Eng 22(6):902–913. https://doi.org/10.1631/FITEE.2000429
Li P, Ye Q, Zhang L, Yuan L, Xu X, Shao L (2021b) Exploring global diverse attention via pairwise temporal relation for video summarization. Pattern Recogn 111:107677. https://doi.org/10.1016/j.patcog.2020.107677
Li W, Pan G, Wang C, Xing Z, Han Z (2022) From coarse to fine: hierarchical structure-aware video summarization. ACM Trans Multimed Comput Commun Appl. https://doi.org/10.1145/3485472
Liang B, Li N, He Z, Wang Z, Fu Y, Lu T (2021) News video summarization combining surf and color histogram features. Entropy. https://doi.org/10.3390/e23080982
Liang G, Lv Y, Li S, Zhang S, Zhang Y (2022) Video summarization with a convolutional attentive adversarial network. Pattern Recogn. https://doi.org/10.1016/j.patcog.2022.108840
Liao M, Shi B, Bai X, Wang X, Liu W (2017) TextBoxes: a fast text detector with a single deep neural network. In: 31st AAAI Conference on Artificial Intelligence, AAAI 2017, pp 4161–4167
Lin CC, Pankanti S, Smith J (2015) Accurate coverage summarization of UAV videos. In: Proceedings—Applied Imagery Pattern Recognition Workshop, 2015-Febru(February). https://doi.org/10.1109/AIPR.2014.7041923
Lin R, Xiao J, Fan J (2019) NeXtVLAD: an efficient neural network to aggregate frame-level features for large-scale video classification. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11132 LNCS, pp 206–218. https://doi.org/10.1007/978-3-030-11018-5_19
Lin T, Zhao X, Su H, Wang C, Yang M (2018). BSN: boundary sensitive network for temporal action proposal generation. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11208 LNCS(Cmic), pp 3–21. https://doi.org/10.1007/978-3-030-01225-0_1
Liu T (2020) Compare and select: video summarization with multi-agent reinforcement learning. http://arXiv.org/2007.14552
Liu H, Fang B, Sun F, Zhang X (2019a) Interactive video summarization with human intentions. Multimed Tools Appl 78(2):1737–1755
Liu YT, Li YJ, Yang FE, Chen SF, Wang YCF (2019b) Learning hierarchical self-attention for video summarization. In: Proceedings—International Conference on Image Processing, ICIP, 2019b-Septe, pp 3377–3381. https://doi.org/10.1109/ICIP.2019.8803639
Liu T, Meng Q, Vlontzos A, Tan J, Rueckert D, Kainz B (2020) Ultrasound video summarization using deep reinforcement learning. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12263 LNCS, pp 483–492. https://doi.org/10.1007/978-3-030-59716-0_46
Liu T, Meng Q, Huang JJ, Vlontzos A, Rueckert D, Kainz B (2022) Video summarization through reinforcement learning with a 3D spatio-temporal U-net. IEEE Trans Image Process 31:1573–1586. https://doi.org/10.1109/TIP.2022.3143699
Loukas C, Varytimidis C, Rapantzikos K, Kanakis MA (2018) Keyframe extraction from laparoscopic videos based on visual saliency detection. Comput Methods Programs Biomed 165:13–23. https://doi.org/10.1016/j.cmpb.2018.07.004
Lu Z, Grauman K (2013) Story-driven summarization for egocentric video. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 2714–2721. https://doi.org/10.1109/CVPR.2013.350
Lu G, Zhou Y, Li X, Yan P (2017) Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos. Multimed Tools Appl 76(5):6309–6331. https://doi.org/10.1007/s11042-016-3263-z
Lucas BD, Kanade T (1981) Iterative image registration technique with an application to stereo vision. In: Proceedings of Imaging Understanding Workshop, 2(April 1981), pp 121–130.
Luo J, Papin C, Costello K (2009) Key frames from personal video clips: from humans to computers. IEEE Trans Circuits Syst Video Technol 19(2):289–301
Lux M, Marques O, Schöffmann K, Böszörmenyi L, Lajtai G (2010) A novel tool for summarization of arthroscopic videos. Multimed Tools Appl 46(2–3):521–544. https://doi.org/10.1007/s11042-009-0353-1
Ma YF, Lu L, Zhang HJ, Li M (2002) A user attention model for video summarization. In: Proceedings of the ACM International Multimedia Conference and Exhibition, pp 533–542. https://doi.org/10.1145/641113.641116
Ma M, Mei S, Wan S, Wang Z, Feng D (2019) Video summarization via nonlinear sparse dictionary selection. IEEE Access 7(c):11763–11774. https://doi.org/10.1109/ACCESS.2019.2891834
Ma M, Mei S, Wan S, Hou J, Wang Z, Feng DD (2020) Video summarization via block sparse dictionary selection. Neurocomputing 378:197–209. https://doi.org/10.1016/j.neucom.2019.07.108
Ma M, Mei S, Wan S, Wang Z, Ge Z, Lam V, Feng D (2021) Keyframe extraction from laparoscopic videos via diverse and weighted dictionary selection. IEEE J Biomed Health Inform 25(5):1686–1698. https://doi.org/10.1109/JBHI.2020.3019198
Mademlis I, Tefas A, Nikolaidis N, Pitas I (2016) Multimodal stereoscopic movie summarization conforming to narrative characteristics. IEEE Trans Image Process 25(12):5828–5840. https://doi.org/10.1109/TIP.2016.2615289
Mahasseni B, Lam M, Todorovic S (2017) Unsupervised Video Summarization with Adversarial LSTM Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017-Janua, pp 202–211. https://doi.org/10.1109/CVPR.2017.318
Mademlis I, Tefas A, Pitas I (2018a) A salient dictionary learning framework for activity video summarization via key-frame extraction. Inf Sci 432:319–331. https://doi.org/10.1016/j.ins.2017.12.020
Mademlis I, Tefas A, Pitas I (2018b) Summarization of human activity videos using a salient dictionary. In: Proceedings—International Conference on Image Processing, ICIP, 2017-Septe, pp 625–629. https://doi.org/10.1109/ICIP.2017.8296356
Mahmoud KM, Ghanem NM, Ismail MA (2013a) VGRAPH: an effective approach for generating static video summaries. In: 2013a IEEE International Conference on Computer Vision Workshops, 2013, pp 811–818. https://doi.org/10.1109/ICCVW.2013.111
Mahmoud KM, Ismail MA, Ghanem NM (2013b) VSCAN: an enhanced video summarization using density-based spatial clustering. In: International Conference on Image Analysis and Processing. Springer, Berlin, Heidelberg, pp 733–742
Marszałek M, Laptev I, Schmid C (2009) Actions in context. In: Computer Vision and Pattern Recognition, IEEE, i, pp 2929–2936.
Mathe J (2017) Automated shot detection software (p. https://github.com/johmathe/Shotdetect. Accessed 1 March 2022
Mathews RP, Panicker MR, Hareendranathan AR, Chen YT, Jaremko JL, Buchanan B, Narayan KV, Mathews G (2021) Unsupervised multi-latent space reinforcement learning framework for video summarization in ultrasound imaging. https://arXiv.org/2109.01309v1
Mehmood I, Sajjad M, Rho S, Baik SW (2016) Divide-and-conquer based summarization framework for extracting affective video content. Neurocomputing 174:393–403. https://doi.org/10.1016/j.neucom.2015.05.126
Mei S, Guan G, Wang Z, He M, Hua XS, Dagan Feng D (2014) L2,0 constrained sparse dictionary selection for video summarization. In: Proceedings - IEEE International Conference on Multimedia and Expo, 2014-Septe(Septmber). https://doi.org/10.1109/ICME.2014.6890179
Mei S, Guan G, Wang Z, Wan S, He M, Dagan Feng D (2015) Video summarization via minimum sparse reconstruction. Pattern Recogn 48(2):522–533. https://doi.org/10.1016/j.patcog.2014.08.002
Mei S, Ma M, Wan S, Hou J, Wang Z, Feng DD (2021) Patch based video summarization with block sparse representation. IEEE Trans Multimed 23(c):732–747. https://doi.org/10.1109/TMM.2020.2987683
Mendi E, Clemente HB, Bayrak C (2013) Sports video summarization based on motion analysis. Comput Electr Eng 39(3):790–796. https://doi.org/10.1016/j.compeleceng.2012.11.020
Messaoud S, Lourentzou I, Boughoula A, Zehni M, Zhao Z, Zhai C, Schwing AG (2021) DeepQAMVS: query-Aware Hierarchical Pointer Networks for Multi-Video Summarization. In: SIGIR 2021—Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Vol. 1, Issue 1). Association for Computing Machinery. https://doi.org/10.1145/3404835.3462959
Mohan J, Nair MS (2018) Dynamic summarization of videos based on descriptors in space-time video volumes and sparse autoencoder. IEEE Access 6:59768–59778. https://doi.org/10.1109/ACCESS.2018.2872685
Money AG, Agius H (2008) Video summarisation: a conceptual framework and survey of the state of the art. J vis Commun Image Represent 19(2):121–143. https://doi.org/10.1016/j.jvcir.2007.04.002
Money AG, Agius H (2010) ELVIS: entertainment-led video summaries. ACM Trans Multimed Comput Commun Appl. https://doi.org/10.1145/1823746.1823751
Muhammad K, Ahmad J, Sajjad M, Baik SW (2016) Visual saliency models for summarization of diagnostic hysteroscopy videos in healthcare systems. Springerplus. https://doi.org/10.1186/s40064-016-3171-8
Muhammad K, Sajjad M, Young M, Wook S (2017) Efficient visual attention driven framework for key frames extraction from hysteroscopy videos. Biomed Signal Process Control 33:161–168. https://doi.org/10.1016/j.bspc.2016.11.011
Muhammad K, Hussain T, Baik SW (2020) Efficient CNN based summarization of surveillance videos for resource-constrained devices. Pattern Recogn Lett 130:370–375. https://doi.org/10.1016/j.patrec.2018.08.003
Mundur P, Rao Y, Yesha Y (2006) Keyframe-based video summarization using Delaunay clustering. Int J Digit Libr 6:219–232. https://doi.org/10.1007/s00799-005-0129-9
Murugan AS, Devi KS, Sivaranjani A, Srinivasan P (2018) A study on various methods used for video summarization and moving object detection for video surveillance applications. Multimed Tools Appl 77:23273
Nair MS, Mohan J (2022) VSMCNN-dynamic summarization of videos using salient features from multi-CNN model. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-022-04112-4
Narasimhan H, Satheesh S, Sriram D (2010) Automatic summarization of cricket video events using genetic algorithm. In: Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO ’10—Companion Publication, pp 2051–2054. https://doi.org/10.1145/1830761.1830858
Nasir MH, Javed A, Irtaza A, Malik H, Mahmood MT (2018) Event detection and summarization of cricket videos. J Image Gr 6(1):27–32. https://doi.org/10.18178/joig.6.1.27-32
Natsev A, Smith JR, Tešić J, Xie L, Yan R (2008) IBM multimedia analysis and retrieval system. In: CIVR 2008—Proceedings of the International Conference on Content-Based Image and Video Retrieval, pp 553–554. https://doi.org/10.1145/1386352.1386427
Nesterov Y (2013) Gradient methods for minimizing composite functions. Math Program 140(1):125–161. https://doi.org/10.1007/s10107-012-0629-5
Ng JYH, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R, Toderici G (2015) Beyond short snippets: deep networks for video classification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12-June, pp 4694–4702. https://doi.org/10.1109/CVPR.2015.7299101
Nie L, Hong R, Zhang L, Xia Y, Tao D, Sebe N (2016) Perceptual attributes optimization for multivideo summarization. IEEE Trans Cybern 46(12):2991–3003. https://doi.org/10.1109/TCYB.2015.2493558
Oh S, Hoogs A, Perera A, Cuntoor N, Chen C, Lee JT, Mukherjee S, Aggarwal JK, Lee H, Davis L, Swears E, Wang X, Ji Q, Reddy K, Shah M, Vondrick C, Pirsiavash H, Ramanan D, Yuen J, et al (2011) A large-scale benchmark dataset for event recognition in surveillance video. In: CVPR 2011, IEEE, vol 2, pp 3153–3160
Oosterhuis H, Ravi S, Com SG, Bendersky M, Com BG (2016a) Semantic video trailers. ArXiv
Otani M, Nakashima Y, Rahtu E, Yokoya N (2016) Video summarization using deep semantic features. In: Asian Conference on Computer Vision. Springer, Cham, pp 1–16
Otani M, Nakashima Y, Rahtu E, Heikkila J (2019b) Rethinking the evaluation of video summaries. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019b-June, pp 7588–7596. https://doi.org/10.1109/CVPR.2019.00778
Ou S-H, Lee C-H, Somayazulu VS, Chen Y, Chien S (2015) On-line multi-view video summarization for wireless video sensor network. IEEE J Select Topics Signal Process 9(1):165–179
Pan G, Qu X, Lv L, Guo S, Sun D (2018) Video clip growth: a general algorithm for multi-view video summarization, vol 1. Springer, Berlin. https://doi.org/10.1007/978-3-030-00764-5
Pan G, Zheng Y, Zhang R, Han Z, Sun D, Qu X (2019) A bottom-up summarization algorithm for videos in the wild. EURASIP J Adv Signal Process 2019:1–11
Pan Y, Huang O, Ye Q, Li Z, Wang W, Li G, Chen Y (2022) Exploring global diversity and local context for video summarization. IEEE Access 10:43611–43622. https://doi.org/10.1109/ACCESS.2022.3163414
Panda R, Roy-Chowdhury AK (2017a) Collaborative summarization of topic-related videos. In: Proceedings—30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017a, 2017a-Janua, pp 4274–4283. https://doi.org/10.1109/CVPR.2017.455
Panda R, Roy-Chowdhury AK (2017b) Multi-view surveillance video summarization via joint embedding and sparse optimization. IEEE Trans Multimed 19(9):2010–2021. https://doi.org/10.1109/TMM.2017.2708981
Panda R, Das A, Roy-chowdhury AK (2016) Video summarization in a multi-view camera network. In: 2016 23rd International Conference on Pattern Recognition (ICPR), IEEE, vol 3, pp 2971–2976
Panda R, Mithun NC, Roy-Chowdhury AK (2017) Diversity-aware multi-video summarization. IEEE Trans Image Process 26(10):4712–4724. https://doi.org/10.1109/TIP.2017.2708902
Park J, Lee J, Kim IJ, Sohn K (2020) SumGraph: video summarization via recursive graph modeling. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12370 LNCS, pp 647–663. https://doi.org/10.1007/978-3-030-58595-2_39
Paul M, Musfequs Salehin M (2019) Spatial and motion saliency prediction method using eye tracker data for video summarization. IEEE Trans Circuits Syst Video Technol 29(6):1856–1867. https://doi.org/10.1109/TCSVT.2018.2844780
Peng J, Xiao-Lin Q (2010) Keyframe-based video summary using visual attention clues. IEEE Multimed 17(2):64–73. https://doi.org/10.1109/MMUL.2009.65
Peng WT, Chu WT, Chang CH, Chou CN, Huang WJ, Chang WY, Hung YP (2011) Editing by viewing: automatic home video summarization by viewing behavior analysis. IEEE Trans Multimed 13(3):539–550. https://doi.org/10.1109/TMM.2011.2131638
Phaphuangwittayakul A, Guo Y, Ying F, Xu W, Zheng Z (2021) Self-attention recurrent summarization network with reinforcement learning for video summarization task Department of Computer Science and Engineering , East China University of Science and Technology , China National Engineering Laboratory for Big Data D. In: IEEE International Conference on Multimedia and Expo (ICME), Icdi
Pirsiavash H, Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 2847–2854. https://doi.org/10.1109/CVPR.2012.6248010
Potapov D, Douze M, Harchaoui Z, Schmid C, Potapov D, Douze M, Harchaoui Z, Category-specific CS, Fleet D, Pajdla T, Schiele B, Tuytelaars T, European E, Potapov D, Douze M, Harchaoui Z, Schmid C (2014) Category-specific video summarization. In: European Conference on Computer Vision. Springer, Cham, pp 540–555
Qayyum H, Majid M, Haq EU, Anwar SM (2019) Generation of personalized video summaries by detecting viewer’s emotion using electroencephalography. J vis Commun Image Represent 65:102672. https://doi.org/10.1016/j.jvcir.2019.102672
Rafiq M, Rafiq G, Agyeman R, Jin SI, Choi GS (2020) Scene classification for sports video summarization using transfer learning. Sensors (switzerland) 20(6):1–18. https://doi.org/10.3390/s20061702
Rahman MR, Shah S, Subhlok J (2020) Visual summarization of lecture video segments for enhanced navigation. In: Proceedings—2020 IEEE International Symposium on Multimedia, ISM 2020, pp 154–157. https://doi.org/10.1109/ISM.2020.00033
Raikwar SC, Bhatnagar C, Jalal AS (2015) A framework for key frame extraction from surveillance video. In: Proceedings—5th IEEE International Conference on Computer and Communication Technology, ICCCT 2014, pp 297–300. https://doi.org/10.1109/ICCCT.2014.7001508
Rasheed Z, Shah M (2005) Detection and representation of scenes in videos. IEEE Trans Multimed 7(6):1097–1105. https://doi.org/10.1109/TMM.2005.858392
Ravi A, Venugopal H, Paul S, Tizhoosh HR (2019) A dataset and preliminary results for umpire pose detection using SVM classification of deep features. In: Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018, pp 1396–1402. https://doi.org/10.1109/SSCI.2018.8628877
Ren J, Jiang J, Eckes C (2008) Hierarchical modeling and adaptive clustering for real-time summarization of rush videos in trecvid’08. In: MM’08 - Proceedings of the 2008 ACM International Conference on Multimedia, with Co-Located Symposium and Workshops, pp 26–30. https://doi.org/10.1145/1463563.1463566
Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Rochan M, Wang Y (2019) Learning video summarization using unpaired data. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June, pp 7894–7903.
Rochan M, Ye L, Wang Y (2018b) Video summarization using fully convolutional sequence networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 347–363. https://doi.org/10.1007/978-3-030-01258-8_22
Roy-chowdhury AK (2017) Weakly supervised summarization of web videos (supplementary material). In: The IEEE International Conference on Computer Vision (ICCV), vol. 3, issue no. c. http://cse.iitkgp.ac.in/~adas/papers/ICCV_2017_Summarization.pdf
Sahrawat D, Agarwal M, Sinha S, Adhikary A, Agarwal M, Shah RR, Zimmermann R (2019) Video summarization using global attention with memory network and LSTM. In: Proceedings - 2019 IEEE 5th International Conference on Multimedia Big Data, BigMM 2019, pp 231–236. https://doi.org/10.1109/BigMM.2019.00-20
Salehin M, Paul M (2015) Summarizing surveillance video by saliency transition and moving object information. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE.
Salehin M, Paul M, Kabir MA (2017) Video summarization using line segments, angles and conic parts. PLoS ONE 12(11):1–22
Saquil Y, Chen D, He Y, Li C, Yang Y-L (2021) Multiple pairwise ranking networks for personalized video summarization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1718–1727
Scharcanski J, Gavião W (2006) Hierarchical summarization of diagnostic hysteroscopy videos. In: Proceedings—International Conference on Image Processing, ICIP, pp 129–132. https://doi.org/10.1109/ICIP.2006.312376
Scott GL, Longuet-Higgins HC (1991) An algorithm for associating the features of two images. Proc R Soc b: Biol Sci 244(1309):21–26. https://doi.org/10.1098/rspb.1991.0045
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, 2017-Octob, pp 618–626. https://doi.org/10.1109/ICCV.2017.74
Shao J, Jiang D, Wang M, Chen H, Yao L (2010) Multi-video summarization using complex graph clustering and mining. Comput Sci Inf Syst 7(1):85–97. https://doi.org/10.2298/CSIS1001085S
Shao D, Zhao Y, Dai B, Lin D (2020) FineGym: a hierarchical video dataset for fine-grained action understanding. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 2613–2622. https://doi.org/10.1109/CVPR42600.2020.00269
Sharghi A, Laurel JS (2017). Query-focused video summarization : dataset , evaluation , and a memory network based approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4788–4797.
Sharghi A, Gong B, Shah M (2016) Query-focused extractive video summarization. In: European Conference on Computer Vision. Springer, Cham, pp 1–18
Shaw P, Uszkoreit J, Vaswani A (2018) Self-attention with relative position representations. In: NAACL HLT 2018–2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies—Proceedings of the Conference, vol. 2, pp 464–468. https://doi.org/10.18653/v1/n18-2074
Shin T, Kim J, Kim J, An B-H (2000) Statistical approach to shot-boundary detection in an MPEG-2-compressed video sequence. Vis Commun Image Process 2000(4067):143. https://doi.org/10.1117/12.386587
Shin HV, Berthouzoz F, Li W, Durand F (2015) Visual transcripts. ACM Trans Gr 34(6):1–10. https://doi.org/10.1145/2816795.2818123
Shingrakhia H, Patel H (2020) Emperor Penguin optimized event recognition and summarization for cricket highlight generation. Multimed Syst 26(6):745–759. https://doi.org/10.1007/s00530-020-00684-3
Shingrakhia H, Patel H (2021) SGRNN-AM and HRF-DBN: a hybrid machine learning model for cricket video summarization. Vis Comput. https://doi.org/10.1007/s00371-021-02111-8
Shroff N, Turaga P, Chellappa R (2010) Video prcis: highlighting diverse aspects of videos. IEEE Trans Multimed 12(8):853–868. https://doi.org/10.1109/TMM.2010.2058795
Shukla P, Sadana H, Bansal A, Verma D, Elmadjian C, Raman B, Turk M (2018) Automatic cricket highlight generation using event-driven and excitement-based features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2018-June, pp 1881–1889. https://doi.org/10.1109/CVPRW.2018.00233
Singh Parihar A, Pal J, Sharma I (2021) Multiview video summarization using video partitioning and clustering. J vis Commun Image Represent 74(November 2020):102991. https://doi.org/10.1016/j.jvcir.2020.102991
Smeaton AF, Over P, Doherty AR (2010) Video shot boundary detection: seven years of TRECVid activity. Comput vis Image Underst 114(4):411–418. https://doi.org/10.1016/j.cviu.2009.03.011
Song Y, Vallmitjana J, Stent A, Jaimes A (2015) TVSum: summarizing web videos using titles. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12-June, pp 5179–5187. https://doi.org/10.1109/CVPR.2015.7299154
Song X, Sun L, Lei J, Tao D, Yuan G, Song M (2016) Event-based large scale surveillance video summarization. Neurocomputing 187:66–74. https://doi.org/10.1016/j.neucom.2015.07.131
Soomro K, Zamir AR, Shah M (2012) UCF101: a dataset of 101 human actions classes from videos in the wild. November. http://arXiv.org/1212.0402
Spachos D, Zlatintsi A (2008) MUSCLE movie database: a multimodal corpus with rich annotation for dialogue and saliency detection. In: Programme of the Workshop on Multimodal Corpora, vol. 16. http://users.uoi.gr/cs01702/MargaritaKotti/MypublicationsPDFs/Musclemovie.pdf
Sreeja MU, Kovoor BC (2019) Towards genre-specific frameworks for video summarisation: a survey. J vis Commun Image Represent 62:340–358. https://doi.org/10.1016/j.jvcir.2019.06.004
Srinivas M, Pai MMM, Pai RM (2016) An improved algorithm for video summarization—a rank based approach. Procedia Procedia Comput Sci 89:812–819. https://doi.org/10.1016/j.procs.2016.06.065
Subudhi BN, Veerakumar T, Esakkirajan S, Chaudhury S (2020) Automatic lecture video skimming using shot categorization and contrast based features. Expert Syst Appl 149:113341. https://doi.org/10.1016/j.eswa.2020.113341
Sukhwani, M., & Kothari, R. (2017). A parameterized approach to personalized variable length summarization of soccer matches. 1–6. http://arXiv.org/1706.09193
Sun K, Zhu J, Lei Z, Hou X, Zhang Q, Duan J, Qiu G, Hou X, Lei Z, Zhang Q, Qiu G (2017) Learning deep semantic attributes for user video summarization. In: IEEE International Conference on Multimedia and Expo (ICME), July
Sung YL, Hong CY, Hsu YC, Liu TL (2020) Video summarization with anchors and multi-head attention. In: IEEE International Conference on Image Processing (ICIP), pp 2396–2400
Sushma B, Aparna P (2021) Summarization of wireless capsule endoscopy video using deep feature matching and motion analysis. IEEE Access 9:13691–13703. https://doi.org/10.1109/ACCESS.2020.3044759
Vajda S, Rothacker L, Fink GA (2011) a camera-based interactive whiteboard reading system. In: 4th International Workshop on Camera-Based Document Analysis and Recognition, June, pp 91–96. https://doi.org/10.1007/978-3-642-29364-1
Tao H, Huang TS (1998) Connected vibrations: a modal analysis approach for non-rigid motion tracking. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 735–740. https://doi.org/10.1109/CVPR.1998.698685
Tejero-de-Pablos A, Nakashima Y, Sato T, Yokoya N (2016) Human action recognition-based video summarization for rgb-d personal sports video. In: Proceedings of the IEEE International Conference on Multimedia and Expo.
Tejero-de-pablos A, Nakashima Y, Sato T, Yokoya N, Linna M, Rahtu E (2018) Summarization of user-generated sports video by using deep action recognition features. IEEE Trans Multimed 20(8):2000–2011. https://doi.org/10.1109/TMM.2018.2794265
Thomas SS, Gupta S, Subramanian VK (2018) Event detection on roads using perceptual video summarization. IEEE Trans Intell Transp Syst 19(9):2944–2954. https://doi.org/10.1109/TITS.2017.2769719
Tiwari V, Bhatnagar C (2021) A survey of recent work on video summarization: approaches and techniques. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10977-yA
Trinh H, Li J, Miyazawa S, Moreno J, Pankanti S (2012) Efficient UAV video event summarization. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012) IEEE, pp 2226–2229
Truong BATU, Venkatesh S (2007) Video abstraction: a systematic review and classification. ACM Trans Multimed Comput Commun Appl 3(1):1–37. https://doi.org/10.1145/1198302.1198305
Tsai CM, Kang LW, Lin CW, Lin W (2013) Scene-based movie summarization via role-community networks. IEEE Trans Circuits Syst Video Technol 23(11):1927–1940. https://doi.org/10.1109/TCSVT.2013.2269186
Tseng CH, Hsieh CC, Jwo DJ, Wu JH, Sheu RK, Chen LC (2021) Person retrieval in video surveillance using deep learning-based instance segmentation. J Sens. https://doi.org/10.1155/2021/9566628
Urala Kota B, Davila K, Stone A, Setlur S, Govindaraju V (2018) Automated detection of handwritten whiteboard content in lecture videos for summarization. In: Proceedings of International Conference on Frontiers in Handwriting Recognition, ICFHR, 2018-Augus, pp 19–24. https://doi.org/10.1109/ICFHR-2018.2018.00013
Urala Kota B, Davila K, Stone A, Setlur S, Govindaraju V (2019) Generalized framework for summarization of fixed-camera lecture videos by detecting and binarizing handwritten content. Int J Doc Anal Recogn 22(3):221–233. https://doi.org/10.1007/s10032-019-00327-y
Valdés V, Martínez JM (2007) On-line video skimming based on histogram similarity. In: Proceedings of the International Workshop on TRECVID Video Summarization, TVS ’07, Co-Located with the ACM Multimedia 2007, MM’07, pp 94–98
Valdés V, Martínez JM (2008) Binary tree based on-line video summarization. In: MM’08—Proceedings of the 2008 ACM International Conference on Multimedia, with Co-Located Symposium and Workshops, pp 134–138. https://doi.org/10.1145/1463563.1463588
Valdés V, Martínez JM (2012) On-line video abstract generation of multimedia news. Multimed Tools Appl 59(3):795–832. https://doi.org/10.1007/s11042-011-0774-5
Varghese EB, Thampi SM (2021) Visual attention based cognitive informative frame extraction method for smart crowd surveillance. In: 2021 IEEE Conference on Norbert Wiener in the 21st Century: Being Human in a Global Village, 21CW 2021. https://doi.org/10.1109/21CW48944.2021.9532519
Vasudevan AB, Gygli M, Volokitin A, Van Gool L (2017) Query-adaptive video summarization via quality-aware relevance estimation. In: MM 2017 - Proceedings of the 2017 ACM Multimedia Conference, pp 582–590. https://doi.org/10.1145/3123266.3123297
Vermaak J, Perez P, Blake A, Gangnet M (2013) Rapid summarisation and browsing of video sequences. BMVC 40(1–40):10. https://doi.org/10.5244/c.16.40
Vezhnevets V, Degtiareva A (2003) Robust and accurate eye contour extraction. In: Proceeding of the Conference {GraphiCon}, pp 81–84
Vezzani R, Cucchiara R (2010) Video surveillance online repository (ViSOR): an integrated framework. Multimed Tools Appl 50(2):359–380. https://doi.org/10.1007/s11042-009-0402-9
Viguier R, Lin CC, Aliakbarpour H, Bunyak F, Pankanti S, Seetharaman G, Palaniappan K (2015) Automatic video content summarization using geospatial mosaics of aerial imagery. In: Proceedings—2015 IEEE International Symposium on Multimedia, ISM 2015, pp 249–253. https://doi.org/10.1109/ISM.2015.124
Viola P, Jones MJ (2004) Robust real-time face detection PAUL. Int J Comput vis 57(2–3):137–154. https://doi.org/10.1112/jlms/s2-30.3.419
Vivekraj VK, Sen D, Raman B (2019) Video skimming: taxonomy and comprehensive survey. ACM Comput Surv 52(5).
Vovk V, Nouretdinov I, Gammerman A (2003) Testing exchangeability on-line. In: Proceedings, Twentieth International Conference on Machine Learning, vol. 2, pp 768–775
Wang F, Ngo CW (2012) Summarizing rushes videos by motion, object, and event understanding. IEEE Trans Multimed 14(1):76–87. https://doi.org/10.1109/TMM.2011.2165531
Wang J, Wang Y, Zhang Z (2011) Visual saliency based aerial video summarization by online scene classification. In: Sixth International Conference on Image and Graphics Visual, pp 2–7. https://doi.org/10.1109/ICIG.2011.43
Wang M, Hong R, Li G, Zha ZJ, Yan S, Chua TS (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimed 14(4):975–985. https://doi.org/10.1109/TMM.2012.2185041
Wang L, Fang X, Guo Y, Fu Y (2016) Multi-view metric learning for multi-view video summarization. In: Proceedings—2016 International Conference on Cyberworlds, CW 2016, pp 179–182. https://doi.org/10.1109/CW.2016.38
Wang J, Wang W, Wang Z, Wang L, Feng D, Tan T (2019a) Stacked memory network for video summarization. In: MM 2019a—Proceedings of the 27th ACM International Conference on Multimedia, pp 836–844. https://doi.org/10.1145/3343031.3350992
Wang L, Zhu Y, Pan H (2019b) Unsupervised reinforcement learning for video summarization reward function. ACM Int Conf Proc Ser Part F 1477:40–44. https://doi.org/10.1145/3317640.3317658
Wang L, Liu D, Puri R, Metaxas DN (2020a) Learning trailer moments in full-length movies with co-contrastive attention. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12363 LNCS, pp 300–316. https://doi.org/10.1007/978-3-030-58523-5_18
Wang X, Nie X, Liu X, Wang B, Yin Y (2020b) Modality correlation-based video summarization. Multimed Tools Appl 79:33875
Wei H, Ni B, Yan Y, Yu H, Yang X (2018) video summarization via semantic attended networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, issue no. 1, pp 216–223
Wu B, Nevatia R (2007) Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. Int J Comput vis 75(2):247–266. https://doi.org/10.1007/s11263-006-0027-7
Wu J, Zhong S, Jiang J (2016) A novel clustering method for static video summarization. Multimed Tools Appl. https://doi.org/10.1007/s11042-016-3569-x
Wu J, Zhong S, Ma Z, Heinen SJ, Jiang J (2018) Foveated convolutional neural networks for video summarization. Multimed Tools Appl 77(22):29245–29267. https://doi.org/10.1007/s11042-018-5953-1
Wu J, Zhong SH, Liu Y (2019) MVSGCN: a novel graph convolutional network for multi-video summarization. In: MM 2019—Proceedings of the 27th ACM International Conference on Multimedia, pp 827–835. https://doi.org/10.1145/3343031.3350938
Wu J, Zhong S, Liu Y (2020) Dynamic graph convolutional network for multi-video summarization. Pattern Recogn. https://doi.org/10.1016/j.patcog.2020.107382
Wu G, Lin J, Silva CT (2022) IntentVizor: towards generic query guided interactive video summarization. https://arXiv.org/2109.14834v2
Xiang Y, Alahi A, Savarese S (2015) Learning to track: Online multi-object tracking by decision making. In: Proceedings of the IEEE International Conference on Computer Vision, 2015 Inter, pp 4705–4713. https://doi.org/10.1109/ICCV.2015.534
Xiao S, Zhao Z, Zhang Z, Guan Z, Cai D (2020a) Query-biased self-attentive network for query-focused video summarization. IEEE Trans Image Process 29:5889–5899
Xiao S, Zhao Z, Zhang Z, Yan X, Yang M (2020b) Convolutional hierarchical attention network for query-focused video summarization. In: AAAI 2020b - 34th AAAI Conference on Artificial Intelligence, pp 12426–12433. https://doi.org/10.1609/aaai.v34i07.6929
Xu F, Davila K, Setlur S, Govindaraju V (2019) Content extraction from lecture video via speaker action classification based on pose information. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, pp 1047–1054. https://doi.org/10.1109/ICDAR.2019.00171
Xu L, Neufeld J, Larson B, Schuurmans D (2005) Maximum margin clustering. Adv Neural Inf Process Syst
Xu J, Mukherjee L, Li Y, Warner J, Rehg JM, Singh V (2015) Gaze-enabled egocentric video summarization via constrained submodular maximization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12-June, pp 2235–2244. https://doi.org/10.1109/CVPR.2015.7298836
Yadav K, Gandhi A, Biswas A, Shrivastava K, Srivastava S, Deshmukh O (2016) ViZig: anchor points based navigation and summarization in educational videos. In: International Conference on Intelligent User Interfaces, Proceedings IUI, 07–10-Marc, pp 407–418. https://doi.org/10.1145/2856767.2856788
Yalınız G, Ikizler-Cinbis N (2019) Unsupervised Video Summarization with Independently Recurrent Neural Networks. In: 27th Signal Processing and Communications Applications Conference (SIU), pp 1–4
Yan X, Gilani SZ, Feng M, Zhang L, Qin H, Mian A (2020) Self-supervised learning to detect key frames in videos. Sensors (switzerland) 20(23):1–18. https://doi.org/10.3390/s20236941
Yang H, Wang B, Lin S, Wipf D, Guo M, Guo B (2015) Unsupervised extraction of video highlights via robust recurrent auto-encoders. In: Proceedings of the IEEE International Conference on Computer Vision, 2015 Inter, pp 4633–4641. https://doi.org/10.1109/ICCV.2015.526
Yao T, Mei T, Rui Y (2016) Highlight detection with pairwise deep ranking for first-person video summarization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, pp 982–990. https://doi.org/10.1109/CVPR.2016.112
Ye ZY, Fu W, Zhuang YT (2003) A robust fusion algorithm for shot boundary detection. J Comput Aided Des Comput Gr (In Chinese with English Abstract) 15:950–955
Yeh FH, Lee GC, Chen YJ, Liao CH (2014). Robust handwriting extraction and lecture video summarization. In: Proceedings—2014 10th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2014, pp 357–360. https://doi.org/10.1109/IIH-MSP.2014.95
Yeung S, Fathi A, Fei-fei L (2014) VideoSET: video summary evaluation through text. https://arXiv.org/1406.5824
Yoon UN, Hong MD, Jo GS (2021) Interp-sum: unsupervised video summarization with piecewise linear interpolation. Sensors. https://doi.org/10.3390/s21134562
Yuan Y, Meng MQH (2013) Hierarchical key frames extraction for WCE video. In: 2013 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2013, pp 225–229. https://doi.org/10.1109/ICMA.2013.6617922
Yuan Y, Zhang J (2022) Unsupervised video summarization via deep reinforcement learning with shot-level semantics. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2022.3197819
Yuan J, Wang H, Xiao L, Zheng W, Li J, Lin F, Zhang B (2007) A formal study of shot boundary detection. IEEE Trans Circuits Syst Video Technol 17(2):168–186. https://doi.org/10.1109/TCSVT.2006.888023
Yuan Y, Mei T, Cui P, Zhu W (2017) Video summarization by learning deep side semantic embedding. IEEE Trans Circuits Syst Video Technol 29:226
Yuan Y, Li H, Wang Q (2019) Spatiotemporal modeling for video summarization using convolutional recurrent neural network. IEEE Access 7:64676–64685. https://doi.org/10.1109/ACCESS.2019.2916989
Yuan L, Tay FEH, Li P, Feng J (2020) Unsupervised video summarization with cycle-consistent adversarial LSTM networks. IEEE Trans Multimed 22(10):2711–2722. https://doi.org/10.1109/TMM.2019.2959451
Yusoff Y, Christmas W, Kittler J (2000) Video shot cut detection using adaptive thresholding. In: BMVC, November, 37.1–37.10. https://doi.org/10.5244/c.14.37
Zeng KH, Chen TH, Niebles JC, Sun M (2016b) Title generation for user generated videos. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9906 LNCS(September), pp. 609–625. https://doi.org/10.1007/978-3-319-46475-6_38
Zhang Y, Zimmermann R (2016) Efficient summarization from multiple georeferenced user-generated videos. IEEE Trans Multimed 18(3):418–431. https://doi.org/10.1109/TMM.2016.2520827
Zhang Y, Wang G, Seo B, Zimmermann R (2012) Multi-video summary and skim generation of sensor-rich videos in geo-space. In: MMSys’12—Proceedings of the 3rd Multimedia Systems Conference, pp 53–64. https://doi.org/10.1145/2155555.2155565
Zhang K, Chao WL, Sha F, Grauman K (2016a) Summary transfer: exemplar-based subset selection for video summarization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016a-Decem, pp 1059–1067. https://doi.org/10.1109/CVPR.2016.120
Zhang K, Chao WL, Sha F, Grauman K (2016b) Video summarization with long short-term memory. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9911 LNCS. https://doi.org/10.1007/978-3-319-46478-7_47
Zhang S, Zhu Y, Roy-Chowdhury AK (2016c) Context-aware surveillance video summarization. IEEE Trans Image Process 25(11):5469–5478. https://doi.org/10.1109/TIP.2016.2601493
Zhang K, Grauman K, Sha F (2018a) Retrospective encoders for video summarization. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11212 LNCS, pp. 391–408. https://doi.org/10.1007/978-3-030-01237-3_24
Zhang Y, Kampffmeyer M, Liang X, Tan M, Xing EP (2018b) Query-conditioned three-player adversarial network for video summarization, pp 1–13. https://arXiv.org/1807.06677v1
Zhang Y, Liang X, Zhang D, Tan M, Xing EP (2018c) Unsupervised object-level video summarization with online motion auto-encoder. Pattern Recogn Lett 130:376
Zhang Y, Kampffmeyer M, Liang X, Zhang D, Tan M, Xing EP (2019a) DTR-GAN: dilated temporal relational adversarial network for video summarization. In: Proceedings of the ACM Turing Celebration Conference-China, pp 1–12
Zhang Y, Kampffmeyer M, Zhao X, Tan M (2019b) Deep reinforcement learning for query-conditioned video summarization. Appl Sci (switzerland) 9(4):12–15. https://doi.org/10.3390/app9040750
Zhao B, Xing EP (2014) Quasi real-time summarization for consumer videos. Proc IEEE Comput Soc Conf Comput vis Pattern Recogn. https://doi.org/10.1109/CVPR.2014.322
Zhao B, Li X, Lu X (2017) Hierarchical recurrent neural network for video summarization. In: MM 2017—Proceedings of the 2017 ACM Multimedia Conference, pp 863–871. https://doi.org/10.1145/3123266.3123328
Zhao B, Li X, Lu X (2018) HSA-RNN: hierarchical structure-adaptive RNN for video summarization. Proc IEEE Comput Soc Conf Comput vis Pattern Recogn. https://doi.org/10.1109/CVPR.2018.00773
Zhao Y, Guo Y, Sun R, Liu Z, Guo D (2019) Unsupervised video summarization via clustering validity index. Multimed Tools Appl 79:33417
Zhao B, Li X, Lu X (2020) Property-constrained dual learning for video summarization. IEEE Trans Neural Netw Learn Syst 31(10):3989–4000. https://doi.org/10.1109/TNNLS.2019.2951680
Zhao B, Li H, Lu X, Li X (2021) Reconstructive sequence-graph network for video summarization. IEEE Trans Pattern Anal Mach Intell 8828(c):1–10. https://doi.org/10.1109/TPAMI.2021.3072117
Zhao B, Gong M, Li X (2022) Hierarchical multimodal transformer to summarize videos. Neurocomputing 468:360–369. https://doi.org/10.1016/j.neucom.2021.10.039
Zhong S, Wu J, Jiang J (2019) Video summarization via spatio-temporal deep architecture. Neurocomputing 332:224–235. https://doi.org/10.1016/j.neucom.2018.12.040
Zhong R, Wang R, Zou Y, Hong Z, Hu M (2021) Graph attention networks adjusted Bi-LSTM for video summarization. IEEE Signal Process Lett 28:663–667. https://doi.org/10.1109/LSP.2021.3066349
Zhong S, Lin J, Lu J, Science C (2022) Deep semantic and attentive network for unsupervised video summarization. ACM Trans Multimed Comput Commun Appl 18(2):1–21
Zhou K, Qiao Y, Xiang T (2018) Deep reinforcement learning for unsupervised video summarization with diversity-representativeness reward. In: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pp 7582–7589
Zhu W, Lu J, Li J, Zhou J (2021) DSNet: a flexible detect-to-summarize network for video summarization. IEEE Trans Image Process 30:948–962. https://doi.org/10.1109/TIP.2020.3039886
Zhu W, Han Y, Lu J, Member S, Zhou J, Member S (2022a) Relational reasoning over spatial-temporal graphs for video summarization. IEEE Trans Image Process 31:3017–3031
Zhu W, Lu J, Han Y, Zhou J (2022b) Learning multiscale hierarchical attention for video summarization. Pattern Recogn 122:108312. https://doi.org/10.1016/j.patcog.2021.108312
Zlatintsi A, Koutras P, Evangelopoulos G, Malandrakis N, Efthymiou N, Pastra K, Potamianos A, Maragos P (2017) COGNIMUSE: a multimodal video database annotated with saliency, events, semantics and emotion with application to summarization. Eurasip J Image Video Process 2017(1):1–24. https://doi.org/10.1186/s13640-017-0194-1
Author information
Authors and Affiliations
Contributions
Deeksha Gupta and Akashdeep Sharma wrote the main manuscript text and Deeksha Gupta prepared figures and All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gupta, D., Sharma, A. A comprehensive study of automatic video summarization techniques. Artif Intell Rev 56, 11473–11633 (2023). https://doi.org/10.1007/s10462-023-10429-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-023-10429-z