Abstract
Every user of multimedia technology expects good image and video visual quality independently of the particular characteristics of the receiver or the communication networks employed. Unfortunately, due to factors like processing power limitations and channel capabilities, images or video sequences are often downsampled and/or transmitted or stored at low bitrates, resulting in a degradation of their final visual quality. In this paper, we propose a region-based framework for intentionally introducing downsampling of the high resolution (HR) image sequences before compression and then utilizing super resolution (SR) techniques for generating an HR video sequence at the decoder. Segmentation is performed at the encoder on groups of images to classify their blocks into three different types according to their motion and texture. The obtained segmentation is used to define the downsampling process at the encoder and it is encoded and provided to the decoder as side information in order to guide the SR process. All the components of the proposed framework are analyzed in detail. A particular implementation of it is described and tested experimentally. The experimental results validate the usefulness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alam M.S., Bognar J.G., Hardie R.C., Yasuda B.J. (2000) Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames. IEEE Transactions Instrumentation Measurement 49: 915–923
Barreto, D., Alvarez, L., & Abad, J. (2006). Motion estimation techniques in super-resolution image reconstruction. A performance evaluation. In: Virtual observatory. Plate content digitalization, archive mining and image sequence processing, Sofia, Bulgary, Vol. I, pp.254–268.
Borman, S. (2004). Topics in multiframe superresolution restoration. Ph.D. dissertation, University of Notre Dame, Notre Dame, IN.
Brown J. (1981) Multi-channel sampling of low pass signals. IEEE Transactions on Circuits and Systems 28: 101–106
Callicó, G. M., Núnez, A., Llopis, R. P., & Sethuraman, R. (2003). Low-cost and real-time super-resolution over a video encoder ip. In: Fourth international symposium on quality electronic design (ISQED’03), Los Alamitos, CA, USA, pp.79–84.
Capel, D., & Zisserman, A. (2001). Super-resolution from multiple views using learnt image models. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, Kauai, Hawaii USA, Vol. 2, pp.627–634.
Chaudhuri S., Manjunath J. (2005) Motion-free super-resolution. Springer, Berlin
Erickson, K. J., & Schultz, R. R. (2000). Mpeg-1 super-resolution decoding for the analysis of video stills. In: Proceedings 4th IEEE southwest symposium image analysis, Austin, TX, pp.13–20.
Farsiu S., Robinson D., Elad M., Milanfar P. (2004) Fast and robust multi-frame super-resolution. IEEE Transactions on Image Processing 13: 1327–1344
Freeman W.T., Jones T.R., Pasztor E.C. (2002) Example based super-resolution. IEEE Computer Graphics and Applications 22: 56–65
Gunturk B., Batur A., Altunbasak Y., Hayes M., Mersereau R. (2003) Eigenface-domain super-resolution for face recognition. IEEE Transactions on Image Processing 12: 597–606
“Information technology—generic coding of moving pictures and associated audio information: Video,” ISO/IEC 13818-2:2000, Tech. Rep., 2000.
“Information technology—coding of audio-visual objects—part 2: Visual,” ISO/IEC 14496-2:2004, Tech. Rep., 2004.
“Information technology—coding of audio-visual objects—part 10: Advanced video coding,” ISO/IEC 14496-10:2005, Tech. Rep., 2005.
Karunaratne, P. V., Segall, C., & Katsaggelos, A. (2001). Rate distortion optimal video pre-processing algorithm. In: Proceedings of the IEEE international conference on image processing Thessaloniki, Greece, Vol. 1, pp. 481–484.
Mardia K., Kent J., Bibby J. (1979) Multivariate analysis. Academic Press, New York
Mateos, J., Katsaggelos, A. K., & Molina, R. (2000). Simultaneous motion estimation and resolution enhancement of compressed low-resolution video. In: Proceedings IEEE international conference on image processing, Vancouver, B.C. Canada, Vol. 2, pp.653–656.
Molina, R., Katsaggelos, A., Alvarez, L., & Mateos, J. (2006). Towards a new video compression scheme using super-resolution. In: Proceedings of the SPIE conference on visual communications and image processing, San Jose, CA, USA, Vol. 6077, pp.607706/1–607706/13.
Molina, R., Mateos, J., & Katsaggelos, A. K. (2006). Super resolution reconstruction of multispectral images. In: Virtual observatory. Plate content digitalization, archive mining and image sequence processing, Sofia, Bulgary, Vol. I, pp.211–220.
Nguyen, N., & Milanfar, P. (2000). Efficient wavelet-based algorithm for image superresolution. In: Proceedings of the IEEE international conference on image processing, Vancouver, B.C. Canada, Vol. 2, pp.351–354.
Papoulis A. (1977) Generalized sampling theorem. IEEE Transactions on Circuits and Systems 24: 652–654
Schultz R.R., Meng L., Stevenson R.L. (1998) Subpixel motion estimation for super-resolution image sequence enhancement. Journal of Visual Communication and Image Representation 9: 38–50
Segall, C. A., Elad, M., Milanfar, P., Webb, R., & Fogg, C. (2004). Improved high-definition video by encoding at an intermediate resolution. In: Proceedings of the SPIE conference on visual communications and image processing, San Jose, CA, USA, Vol. 5308, pp.1007–1018.
Ur H., Gross D. (1992) Improved resolution from sub-pixel shifted pictures. CVGIP: Graphical Models and Image Processing 54: 181–186
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Barreto, D., Alvarez, L.D., Molina, R. et al. Region-based super-resolution for compression. Multidim Syst Sign Process 18, 59–81 (2007). https://doi.org/10.1007/s11045-007-0019-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-007-0019-y