Skip to main content
SpringerLink
Log in

A content-based intra rate-distortion model for HEVC-SCC

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Screen content (SC) videos have different characteristics compared to camera-captured videos. New encoding modes such as Intra Block Copy (IBC) and Palette modes have been introduced to the SC coding (SCC) extension of the high-efficiency video coding (HEVC) standard. In SCC, a large percentage of bits is consumed by periodic intra frames and also by the abrupt-change frame changes in which all or most of the coding units are coded as intra modes (e.g. Intra, IBC, and Palette modes). Therefore, a proper intra rate-distortion (RD) model is required to support these encoding modes. The RD model must be content-based to be able to independently encode the intra frames and abrupt frames. The existing RD models are not content-based or do not support the new encoding modes. In this paper, a new content-based intra RD model for HEVC-SCC is proposed. The proposed model is adapted for different sizes and modes of the coding units and it consists of two parts. One part depends on the coding complexity of video content and the second part is a function of the encoder RD operating point. The experimental results show that the model parts are regular functions of the quantization parameter. However, these functions depend on the size and mode of the coding units. According to the results, the model performs with high degrees of accuracy and fitness in terms of the average estimation error and the coefficient of determination criteria, respectively. Also, it shows a very good performance while using it in two low delay rate control algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Andrews HC (1968) Entropy considerations in the frequency domain. Proc IEEE 56(1):113–114

    Article  Google Scholar 

  2. Chang CY, Lin T, Chan DY, Hung SH (2006) A low complexity rate-distortion source modeling framework. Int. Con. Acoustics, Speech, and Signal Processing (ICASSP ‘06), pp 929–932

  3. Chen Y, Kwong S, Zhou M, Wang S, Zhu G, Wang Y (2020) "Intra frame rate control for versatile video coding with quadratic rate-distortion modelling." ICASSP - 2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 4422-4426. https://doi.org/10.1109/ICASSP40776.2020.9054633.

  4. Chiang T, Zhang Y-Q (1997) A new rate control scheme using quadratic rate distortion model. IEEE Trans Circ Syst Video Technol 7(1):246–250

    Article  Google Scholar 

  5. Choi H, Nam J, Yoo J, Sim D, Bajic IV (2012) Rate control based on unified RQ model for HEVC. ITU-T SG16 contribution, JCTVC-H0213, San Jose

  6. Choi H, Yoo J, Nam J, Sim D, Bajic IV (2013) Pixel-wise unified rate-quantization model for multi-level rate control. IEEE J Select Top Signal Process 7(6):1112–1123

    Article  Google Scholar 

  7. Fani D, Rezaei M (2018) Novel PID-fuzzy video rate controller for high-delay applications of the HEVC standard. IEEE Trans Circ Syst Video Technol 28(6):1379–1389. https://doi.org/10.1109/TCSVT.2017.2669214

    Article  Google Scholar 

  8. Gao Y, Zhu C, Li S, Yang T (2017) Temporally dependent rate-distortion optimization for low-delay hierarchical video coding. IEEE Trans Image Process 26(9):4457–4470

    Article  MathSciNet  Google Scholar 

  9. Guo Y, Li B, Sun S, Xu J (2015) Rate control for screen content coding in HEVC. IEEE international symposium on circuits and systems (ISCAS), Lisbon, pp 1118–1121

  10. Guo Y, Li B, Sun S, Xu J (2015) Rate control for screen content coding based on picture classification. In 2015 Visual Communications and Image Processing (VCIP), pp 1–4

  11. He Z (2001) ρ -domain rate-distortion analysis and rate control for visual coding and communication. Ph.D. Dissertation, University of California, Santa Barbara

  12. Hu H-M, Zhou M, Liu Y, Yin N (2016) A region-based intra frame rate control scheme by jointing inter-frame dependency and inter-frame correlation. Multimed Tools Appl 76(10):12917–12940

    Article  Google Scholar 

  13. Hyun MH, Lee B, Kim M (2020) A frame-level constant bit-rate control using recursive Bayesian estimation for versatile video coding. IEEE Access 8:227255–227269. https://doi.org/10.1109/ACCESS.2020.3046043

    Article  Google Scholar 

  14. Kao J-Y, Azimi Hashemi M, Xiu X, Ye Y, He Y, Dong J (2015) "Improved transform skip mode for HEVC screen content coding," 2015 international conference on image processing theory, tools and applications (IPTA). Orleans 2015:504–509. https://doi.org/10.1109/IPTA.2015.7367197

    Article  Google Scholar 

  15. Kwon D-K, Shen M-Y, Kuo C-CJ (2007) Rate control for H.264 video with enhanced rate and distortion models. IEEE Trans Circ Syst Video Technol 17(5):517–529

    Article  Google Scholar 

  16. Li B, Li H, Li L, Zhang J (2012) Rate control by R-lambda model for HEVC. JCTVC-K0103, Shanghai, CN, 10–19

  17. Li B, Li H, Li L, Zhang J (2014) λ Domain Rate Control Algorithm for High Efficiency Video Coding. IEEE Trans Image Process 23(9):3841–3854

    Article  MathSciNet  Google Scholar 

  18. Li Y, Li B, Liu D, Chen Z (2017) A convolutional neural network-based approach to rate control in HEVC intra coding. 2017 IEEE visual communications and image processing (VCIP), St. Petersburg, FL, 2017, pp 1–4

  19. Li Y, Liu Z, Chen Z, Liu S (2020) Rate control for versatile video coding. 2020 IEEE international conference on image processing (ICIP), Abu Dhabi, United Arab Emirates, pp 1176–1180. https://doi.org/10.1109/ICIP40778.2020.9191125

  20. Li X, Oertel N, Hutter A, Kaup A (2009) Laplace distribution based Lagrangian rate distortion optimization for hybrid video coding. IEEE Trans Circ Syst Video Technol 19(2):193–205. https://doi.org/10.1109/TCSVT.2008.2009255

    Article  Google Scholar 

  21. Li W, Ren P, Zhang E, Zhao F (2019) Rate control for HEVC intra-coding with a CTU-dependent distortion model. SIViP 13:17–25. https://doi.org/10.1007/s11760-018-1323-8

    Article  Google Scholar 

  22. Li B, Xu J, Sullivan GJ, Zhou Y, Lin B (2014) Adaptive motion vector resolution for screen content, document JCTVC-S0085, Joint Collaborative Team Video Coding, Strasbourg, France

  23. Li B, Zhang D, Li H, Xu J (2012) QP determination by lambda value. Document ITU- T/ ISO/IEC, JCT-VC JCTVC-I0426, Geneva, CH, 27

  24. Li S, Zhu C, Gao Y, Zhou Y, Dufaux F, Sun M-T (2016) Lagrangian multiplier adaptation for rate-distortion optimization with inter-frame dependency. IEEE Trans Circ Syst Video Technol 26(1):117–129

    Article  Google Scholar 

  25. Lu Q, Cen F, Xu W (2013) Efficient rate control for intra-only coding in high efficiency video coding. Ninth international conference on natural computation (ICNC), Shenyang, pp 1329–1333

  26. Lu X, Zhou B, Jin X, Martin G (2020) A rate control scheme for HEVC intra coding using convolution neural network (CNN). 2020 data compression conference (DCC), Snowbird, UT, USA, 2020, pp 382–382

  27. Peng W-H, Walls FG, Cohen RA, Xu J, Ostermann J, MacInnis A, Lin T (2016) Overview of screen content video coding: technologies, standards, and beyond. IEEE J Emerg Select Top Circ Syst 6(4):393–408

    Article  Google Scholar 

  28. Pu W, Karczewicz M, Joshi R, Seregin V, Zou F, Sole J, Sun Y-C, Chuang T-D, Lai P, Liu S, Hsiang S-T, Ye J, Huang Y-W (2016) Palette mode coding in HEVC screen content coding extension. IEEE J Emerg Select Top Circ Syst 6(4):420–432

    Article  Google Scholar 

  29. Rec ITU-T H.265 v5 (02/2018), Series H: audiovisual and multimedia systems, Infrastructure of audiovisual services – Coding of moving video

  30. Rezaei M (2008) Advances on video coding algorithms for streaming applications. PhD thesis, Tampere University of Technology

  31. Rezaei M, Gabbouj M, Wenger S (2005) Analyzed rate distortion model in standard video codecs for rate control. IEEE Workshop on Signal Processing Systems Design and Implementation, Athens, Greece, pp 550–555

  32. Ribas-Corbera J, Shawmin L (1999) Rate control in DCT video coding for low-delay communications. IEEE Trans Circ Syst Video Technol 9(1):172–185

    Article  Google Scholar 

  33. Sanchez V (2020) Rate control for predictive transform screen content video coding based on RANSAC. IEEE Trans Circ Syst Video Technol. https://doi.org/10.1109/TCSVT.2020.3047296

  34. Sun L, Au OC, Dai W, Guo Y, Zou R (2012) An adaptive frame complexity based rate quantization model for intra-frame rate control of high efficiency video coding (HEVC). Proceedings of The Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hollywood, CA, pp 1–6

  35. Tang T, Li L (2019) A low delay rate control method for screen content coding. Multimed Tools Appl 78(19):28231–28256

    Article  Google Scholar 

  36. Tang T, Yang J, Du B, Tang L (2019) Down-sampling based rate control for Mobile screen video coding. IEEE Access 7:139560–139570

    Article  Google Scholar 

  37. Tian L, Zhou Y, Sun Y (2012) Novel rate control scheme for intra frame video coding with exponential rate–distortion model on H.264/AVC. J Vis Commun Image Represent 23(6):873–882

    Article  Google Scholar 

  38. Viterbi AJ, Omura JK (1979) Principles of digital communication and coding. McGraw-Hill Electrical Engineering Series, New York

    MATH  Google Scholar 

  39. Wang S, Li J, Wang S, Ma S, Gao W (2018) A frame level rate control algorithm for screen content coding. IEEE international symposium on circuits and systems (ISCAS), Florence, pp 1–4

  40. Wang M, Ngan KN, Li H (2015) An efficient frame-content based intra frame rate control for high efficiency video coding. IEEE Signal Process Lett 22(7):896–900

    Article  Google Scholar 

  41. Xiao J, Li B, Sun S, Xu J (2017) Rate control with delay constraint for screen content coding. IEEE visual communications and image processing (VCIP), St. Petersburg, FL, pp 1–4

  42. Xu J, Joshi R, Cohen RA (2016) Overview of the emerging HEVC screen content coding extension. IEEE Trans Circ Syst Video Technol 26(1):50–62

    Article  Google Scholar 

  43. Xu X, Liu S (2021) Overview of screen content coding in recently developed video coding standards. IEEE Trans Circ Syst Video Technol. https://doi.org/10.1109/TCSVT.2021.3064210

  44. Xu X, Liu S, Chuang T-D, Huang Y-W, Lei S-M, Rapaka K, Pang C, Seregin V, Wang Y-K, Karczewicz M (2016) Intra block copy in HEVC screen content coding extensions. IEEE J Emerg Select Top Circ Syst 6(4):409–419

    Article  Google Scholar 

  45. Yang Y, Shen L, Yang H, An P (2019) A content-based rate control algorithm for screen content video coding. J Vis Commun Image Represent 60:328–338

    Article  Google Scholar 

  46. Yang H, Shen L, Yang Y, Lin W (2020) A novel rate control scheme for video coding in HEVC-SCC. IEEE Trans Broadcast 66(2):333–345

    Article  Google Scholar 

  47. Yang K, Wan S, Gong Y, Wu HR, Feng Y (2017) An efficient Lagrangian multiplier selection method based on temporal dependency for rate–distortion optimization in H.265/HEVC. Signal Process Image Commun 57:68–75

    Article  Google Scholar 

  48. Yuan L et al (2018) Heading judgement for indoor position based on the gait pattern. In: Hung J, Yen N, Hui L (eds) Frontier Computing. FC 2017. Lecture notes in electrical engineering, vol 464. Springer, Singapore https://doi.org/10.1007/978-981-10-7398-4_21

  49. Zhang F, Bull DR (2015) An adaptive Lagrange multiplier determination method for rate-distortion optimization in hybrid video codecs. IEEE international conference on image processing (ICIP), Quebec City, QC, 2015, pp 671–675

  50. Zhang L, Xiu X, Chen J, Karczewicz M, He Y, Ye Y, Xu J, Sole J, Kim W-S (2016) Adaptive color-space transform in HEVC screen content coding. IEEE J Emerg Select Top Circ Syst 6(4):446–459

    Article  Google Scholar 

  51. Zhou Y, Sun Y, Feng Z, Sun S-X (2009) New rate–distortion modeling and efficient rate control for H.264/AVC video coding. Signal Process Image Commun 24(5):345–356

    Article  Google Scholar 

  52. Zhou Y, Tian L, Ning X (2013) Intra frame constant rate control scheme for high efficiency video coding. International Conference on Computing, Networking and Communications (ICNC), San Diego, CA, pp 648–652

Download references

Author information

Authors and Affiliations

  1. Information Technology Engineering Department, University of Sistan and Baluchestan, Zahedan, Iran

    Hadi Esmaeeli

  2. Department of Communications Engineering, University of Sistan and Baluchestan, Zahedan, Iran

    Mehdi Rezaei

Authors
  1. Hadi Esmaeeli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehdi Rezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Rezaei.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Esmaeeli, H., Rezaei, M. A content-based intra rate-distortion model for HEVC-SCC. Multimed Tools Appl 81, 16515–16536 (2022). https://doi.org/10.1007/s11042-022-12361-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12361-w

Keywords

Search

Navigation