Abstract
A well-produced video always creates a strong impression on the viewer. However, due to the limitations of the camera, the ambient conditions or the skills of the videographer, the quality of captured videos sometimes falls short of one's expectations. On the other hand, we have a vast amount of superbly captured videos available on the web and in digital libraries. In this paper, we propose the novel approach of video analogies that provides a powerful ability to improve the quality of a video by borrowing features from a higher quality video. We want to improve the given target video in order to obtain a higher quality output video. During the matching phase, we find the correspondence between the pair by using feature matching. Then for the target video, we utilize this correspondence to transfer some desired traits of the source video into the target video in order to obtain a new video. Thus, the new video will obtain the desired features from the source video while retaining the merits of the target video. The video analogies technique provides an intuitive mechanism for automatic editing of videos. We demonstrate the utility of the analogies method by considering three applications – colorizing videos, reducing video blurs, and video rhythm adjustment. We describe each application in detail and provide experimental results to establish the efficacy of the proposed approach.
Similar content being viewed by others
References
Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graphics Appl. 21(5), 34–41 (2001)
Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to grayscale images. In: Proceedings of ACM SIGGRAPH'02, vol.~20, pp. 277–280. San Antonio, USA (2002)
Yan, W.-Q., Kankanhalli, M.S.: Colorizing infrared home videos. In: Proceedings of IEEE ICME'03. Baltimore, USA (2003)
Yan, W.-Q., Kankanhalli, M.S.: Erasing video logos based on image inpainting. In: Proceedings of IEEE ICME'02, pp. 521–524. Lausanne, Switzerland (2002)
Mulhem, P., Kankanhalli, M.S., Hasan, H., Ji, Y.: Pivot vector space approach for audio-video mixing. IEEE Multimedia. 28–40 (2003)
Chang, S., Sundaram, H.: Structural and semantic analysis of video. In: Proceedings of IEEE ICME'00, vol.~2, pp. 687–690. New York, USA (2000)
Kender, J., Teo, B.: Video scene segmentation via continuous video coherence. In: Proceedings of IEEE CVPR'98, pp. 367–373. Santa Barbara, USA (1998)
Hertzmann, A., Jacobs, C., Oliver, N., Curless, B., Salesin, D.: Image analogies. In: Proceedings of ACM SIGGRAPH'01, pp. 327–340. Los Angels, USA (2001)
Evans, T.: A program for the solution of geometric analogy intelligence test questions. In: Semantic Information Processing. MIT Press, New York (1968)
Hertzmann, A., Oliver, N., Seitz, S., Curless, B.: Curves analogies. In: Proceedings of Eurographics Workshop on Rendering'02, pp.~233–246. Pisa, Italy (2002)
Gentner, D.: Structure mapping: a theoretical framework for analogy. Cogn. Sci. 7(2), 155–170 (1983)
Winston, P.: Learning and reasoning by analogy. Commun. ACM 23(12) (1980)
Casares, J., Myers, B.A., Long, A.C., Bhatnagar, R., Stevens, S.M., Dabbish, L., Corbett, A.: Simplifying video editing with intelligent interaction. In: Proceedings CHI'02: Human Factors in Computing Systems, pp.~672–673. Minneapolis, MN (2002)
Girgensohn, A., Bly, S., Shipman, F., Boreczky, J., Wilcox, L.: Home video editing made easy-balancing automation and user control. In: Proceedings of INTERACT'01, pp. 464–471. Tokyo, Japan (2001)
Girgensohn, A., Boreczky, J., Chiu, P., Doherty, J., Foote, J., Golovchinsky, G., Uchihashi, S., Wilcox, L.: A semi-automatic approach to home video editing. CHI Lett. 2, 81–89 (2000)
Davis, M.: Editing out video editing. IEEE Multimedia 10(2), 54–64 (2003)
Dorai, C., Venkatesh, S.: Computational media aesthetics: finding meaning beautiful. IEEE Multimedia 8(4), 10–12 (2001)
Dorai, C., Venkatesh, S.: Bridging the semantic gap with computational media aesthetics. IEEE Multimedia 10(2), 15–17 (2003)
Tekalp, M.: Digital Video Processing. Prentice-Hall, New York (1995)
Sand, P., Teller, S.: Video matching. ACM Trans. Graphics (2004)
Chiang, M., Boult, T.: Local blur estimation and super-resolution. In: Proceedings of IEEE CVPR'97, pp. 821–826. San Juan, Puerto Rico (1997)
Ma, W.-Y., Zhang, H.-J.: Blur determination in the compressed domain using DCT information. In: Proceedings of IEEE ICIP'99, vol.~2, pp. 386–390. Kobe, Japan (1999)
Gonzalez, R., Woods, R.: Digital Image Processing. Addison-Wesley, Reading (1992–1993)
Sun, X.-D., Divakaran, A., Manjunath, B.S.: A motion activity descriptor and its extraction in compressed domain. In: Proceedings of IEEE PCM'01, pp. 450–457. Beijing, China (2001)
Pattiand, A., Sezan, M., Tekalp, A.: Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6(8), 1067–1076 (1997)
Nguyen, N., Milanfar, P.: A computationally efficient super resolution image reconstruction algorithm. IEEE Trans. Image Process. 10(4), 573–583 (2001)
William, T., Touis, R., Egon, C.: Example-based super-resolution. IEEE Comput. Graphics Appl. 56–65 (2002)
Yan, W.-Q., Kankanhalli, M.S.: Detection and removal of lighting and shaking artifacts in home videos. In: Proceedings of ACM Multimedia'02, pp. 107–116. Jan-les-Pins, France (2002)
Farin, G.: Curves and Surfaces for Computer Aided Geometric Design: A Practical Guide, 3rd edn. Academic Press, Boston (1992)
Adams, B., Dorai, C., Venkatesh, S.: Automated film rhythm extraction for scene analysis. In: Proceedings of the IEEE ICME'01, pp. 1192–1195. Tokyo, Japan (2001)
Adams, B., Dorai, C., Venkatesh, S.: Formulating film tempo–-the computational media aesthetics methodology in practice. In: Media Computing: Computational Media Aesthetics, pp. 57–79. Kluwer Academic Publishers, Drodrecht (2002)
Adams, B., Venkatesh, S., Dorai, C.: Finding the beat: an analysis of the rhythmic elements of motion pictures. Int. J. Image Graphics 2(2), 215–245 (2002)
Herbert, Z.: Sight, Sound, Motion: Applied Media Aesthetics, 3rd edn. Wadsworth Publishing Company (1999)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yan, WQ., Kankanhalli, M.S. & Wang, J. Analogies based video editing. Multimedia Systems 11, 3–18 (2005). https://doi.org/10.1007/s00530-005-0186-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-005-0186-3