Skip to main content
SpringerLink
Log in

Cost Sensitive Learning Based HEVC Screen Content Intra Coding for Mobile Devices

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

The distinguished features of screen contents lead to the development of the Screen Content Coding (SCC) extension of High Efficiency Video Coding (HEVC). Intra coding of HEVC-SCC has to evaluate traditional HEVC intra coding modes and additional modes for screen contents, which poses a challenge for coding such a content on mobile devices. Furthermore, the heterogeneous mobile devices have varying complexity requirements. In this paper, a flexible screen content intra coding scheme is proposed, which can trade between encoding complexity and rate-distortion performance degradation via cost sensitive learning. Taking the cost of different decisions into account, binary classifiers are designed to determine whether a particular kind of coding modes will be evaluated. By adopting the cost-proportionate weighted sampling, the proposed scheme can learn a flexible coding policy offline via reinforcement learning. Experimental results show the effectiveness of the proposed scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Bjøntegaard G (2001) Calculation of average PSNR differences between RD-curves. In: Proceedings of the ITU-t video coding experts group (VCEG) Thirteenth meeting

  2. Chung CH, Peng WH, Hu JH (2017) HEVC/H.265 coding unit split decision using deep reinforcement learning. In: Proceedings of IEEE international symposium on intelligent signal processing and communication systems (ISPACS), pp 570–575

  3. Correa G, Dall’Oglio P, Palomino D, Agostini L (2019) Proceedings of online machine learning for fast coding unit decisions in HEVC. In: Proceedings of 2019 data compression conference (DCC), pp 564–564

  4. Duanmu F, Ma Z, Wang Y (2015) Fast CU partition decision using machine learning for screen content compression. In: Proceedings of IEEE international conference on image processing (ICIP), pp 4972–4976

  5. Duanmu F, Ma Z, Wang Y (2016) Fast mode and partition decision using machine learning for intra-frame coding in hevc screen content coding extension. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 6(4):517–531

    Article  Google Scholar 

  6. Erabadda B, Mallikarachchi T, Kulupana G, Fernando A (2018) Machine learning approaches for intra-prediction in hevc. In: 2018 IEEE 7Th global conference on consumer electronics (GCCE), pp. 206–209

  7. Gao X, Li Z, Zhang Y (2018) A fast hevc inter cu size decision algorithm based on multi-class learning. In: Proceedings of 10th international conference on intelligent human-machine systems and cybernetics (IHMSC), vol 02, pp 64–68

  8. Grellert M, Zatt B, Bampi S, da Silva Cruz LA (2019) Fast coding unit partition decision for hevc using support vector machines. IEEE Transactions on Circuits and Systems for Video Technology 29(6):1741–1753

    Article  Google Scholar 

  9. Guo L, Pu W, Zou F, Sole J, Karczewicz M, Joshi R (2014) Color palette for screen content coding. In: Proceedings of IEEE international conference on image processing (ICIP), IEEE, pp 5556–5560

  10. Helle P, Schwarz H, Wiegand T, Müller K. R. (2017) Reinforcement learning for video encoder control in HEVC. In: Proceedings of IEEE international conference on systems, signals and image processing (IWSSIP), pp 1–5

  11. Hu JH, Peng WH, Chung CH (2018) Reinforcement learning for HEVC/h.265 intra-frame rate control. In: Proceedings of 2018 IEEE international symposium on circuits and systems (ISCAS), pp 1–5

  12. Kalva H, Christodoulou L (2007) Using machine learning for fast intra mb coding in h.264. visual communications and image processing 6508

  13. Kuang W, Chan Y, Tsang S, Siu W (2019) Machine learning based fast intra mode decision for HEVC screen content coding via decision trees. IEEE Transactions on Circuits and Systems for Video Technology, pp 1–1

  14. Lei J, Li D, Pan Z, Sun Z, Kwong S, Hou C (2017) Fast intra prediction based on content property analysis for low complexity HEVC-based screen content coding. IEEE Trans Broadcast 63(1):48–58

    Article  Google Scholar 

  15. Li H, Meng L, Zhang J, Tan Y, Ren Y, Zhang H (2019) Multiple description coding based on convolutional auto-encoder. IEEE Access 7:26,013–26,021

    Article  Google Scholar 

  16. Lu Y, Liu H, Lin Y, Shen L, Yin H (2018) Efficient coding mode and partition decision for screen content intra coding. Signal Processing: Image Communication 68:249 –257

    Google Scholar 

  17. Momcilovic S, Roma N, Sousa L, Milentijevic I (2015) Run-time machine learning for HEVC/h.265 fast partitioning decision. In: Proceedings of IEEE international symposium on multimedia (ISM), pp 347–350

  18. Pang C, Sole J, Chen Y, Seregin V, Karczewicz M (2015) Intra block copy for HEVC screen content coding. In: Proceedings of data compression conference, IEEE, pp 465–465

  19. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  20. Peng WH, Walls FG, Cohen RA, Xu J, Ostermann J, MacInnis A, Lin T (2016) Overview of screen content video coding: Technologies, standards, and beyond. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 6(4):393–408

    Article  Google Scholar 

  21. Pu W, Karczewicz M, Joshi R, Seregin V, Zou F, Sole J, Sun YC, Chuang TD, Lai P, Liu S et al (2016) Palette mode coding in HEVC screen content coding extension. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 6(4):420–432

    Article  Google Scholar 

  22. Qin J, Bai H, Zhang M, Zhao Y (2017) Fast intra coding algorithm for HEVC based on decision tree. IEICE Transactions 100-A(5):1274–1278

    Article  Google Scholar 

  23. Ruizcoll D, Adzic V, Fernandezescribano G, Kalva H, Martinez JL, Cuenca P (2014) Fast partitioning algorithm for HEVC intra frame coding using machine learning. pp 4112–4116

  24. Shan Y, Yang E (2017) Fast hevc intra coding algorithm based on machine learning and laplacian transparent composite model. In: 2017 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 2642–2646

  25. Sullivan GJ, Ohm J, Han W, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22(12):1649–1668

    Article  Google Scholar 

  26. Tang J, Alelyani S, Liu H (2014) Feature selection for classification: a review. Data Classification:, Algorithms and Applications, pp 37–64

  27. Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the h.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13(7):560–576

    Article  Google Scholar 

  28. Xu J, Joshi R, Cohen RA (2016) Overview of the emerging HEVC screen content coding extension. IEEE Transactions on Circuits and Systems for Video Technology 26(1):50–62

    Article  Google Scholar 

  29. Xu X, Liu S, Chuang TD, Huang YW, Lei SM, Rapaka K, Pang C, Seregin V, Wang YK, Karczewicz M (2016) Intra block copy in HEVC screen content coding extensions. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 6(4):409–419

    Article  Google Scholar 

  30. Yang H, Shen L, An P (2018) Efficient screen content intra coding based on statistical learning. Signal Processing: Image Communication 62:74–81

    Google Scholar 

  31. Zadrozny B, Langford J, Abe N (2003) Cost-sensitive learning by cost-proportionate example weighting. In: Proceedings of Third IEEE international conference on data mining, pp 435–442

  32. Zhang M, Guo Y, Bai H (2014) Fast intra partition algorithm for HEVC screen content coding. In: Proceedings of IEEE visual communications and image processing conference, VCIP, pp 390–393

  33. Zhang M, Zhang Y, Bai H (2015) Fast CU splitting in HEVC intra coding for screen content coding. IEICE Transactions 98-D(2):467–470

    Article  Google Scholar 

  34. Zhao L, Bai H, Wang A, Zhao Y (2019) Multiple description convolutional neural networks for image compression. IEEE Trans Circuits Syst Video Techn 29(8):2494–2508

    Article  Google Scholar 

  35. Zhu L, Zhang Y, Pan Z, Wang R, Kwong S, Peng Z (2017) Binary and multi-class learning based low complexity optimization for hevc encoding. IEEE Trans Broadcast 63(3):547–561

    Article  Google Scholar 

  36. Zong J, Meng L, Zhang H, Wan W (2017) JND-Based multiple description image coding. KSII Transactions on Internet and Information Systmes 11(8):3935–3949

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hohai University, Nanjing, 211100, China

    Yuanyuan Xu

  2. Nanjing University of Aeronautics and Astronautics, Nanjing, 211100, China

    Kun Zhu

Authors
  1. Yuanyuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kun Zhu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supported by National Natural Science Foundation of China under Grant No. 61801167, the Fundamental Research Funds for the Central Universities under Grant No. B200202189, and Natural Science Foundation of Jiangsu Province of China under Grant No. BK20170805 and BK20160874.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Y., Zhu, K. Cost Sensitive Learning Based HEVC Screen Content Intra Coding for Mobile Devices. Mobile Netw Appl 25, 2471–2481 (2020). https://doi.org/10.1007/s11036-020-01594-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-020-01594-4

Keywords

Search

Navigation