Tunable early CU size decision for depth map intra coding in 3D-HEVC using unsupervised learning

Abstract

To further improve coding gain for depth map in 3D extension of High Efficiency Video Coding (3D-HEVC), many new coding techniques are introduced, which drastically increase the coding complexity. In this work, an early intra coding unit (CU) size decision scheme is proposed for 3D-HEVC intra depth map coding based on unsupervised learning approach. First, we treat the early CU size decision as a clustering problem. Then, three clustering models are developed for those CUs with the sizes of 64×64, 32×32 and 16×16 to early determine whether they be further split or not. Among them, the center of the clustering model is obtained by using intra learning method. Finally, in order to meet the user's specific application preference, the similarity distance is introduced into the early CU size decision. By adjusting the similarity distance, a tunable early CU size decision is achieved to obtain different levels of coding complexity reduction. Experimental results show that the proposed scheme achieves encoding time reduction ranging from 54.0% to 68.4% on average for depth map intra coding, while the Bjontegaard delta bit rate (BDBR) of synthesized views only increased by 0.01% to 0.73%, it outperforms the state-of-the-art works in term of coding complexity reduction and BDBR increase.

Introduction

With the rapid development of three-dimensional (3D) video acquisition and display technologies, 3D videos are increasingly popular for viewers. Because 3D videos can provide more immersive and real-world visual experiences, they are widely anticipated in various video applications such as Free-viewpoint TV [1] and 3D movies. 3D videos are commonly presented as the multi-view video plus depth (MVD) format [2], which include multiple texture views and their corresponding depth maps. Meanwhile, vivid virtual views can be synthesized by using Depth-Image-Based-Rendering (DIBR) [3]. Thus, high efficient compression is extremely important for 3D video data to save storage space and transmission bandwidth. To efficiently encode single texture video, High Efficiency Video Compression (HEVC) standard [4]-[5] has been developed by the Joint Collaborative Team on Video Coding (JCT-VC), which achieves higher compression efficiency than earlier H.264/AVC standard. To encode 3D video data, an advanced 3D video coding standard, which is actually a 3D extension of HEVC (3D-HEVC) [6]-[7], is developed by the Joint Collaborative Team on 3D Video Coding (JCT-3V).

3D-HEVC inherits some coding tools such as quadtree Coding Tree Unit (CTU) partitioning of HEVC [8]-[9]. The coding frame is firstly divided into many CTUs with the same size of 64×64. Then, each CTU can be further recursively divided into smaller coding units (CUs). Fig. 1 (a) shows an example of the optimal CTU partitioning. The CU sizes support 64×64, 32×32, 16×16 and 8×8, and their corresponding CU depths are “0”, “1”, “2”, and “3”, respectively. The 3D video format includes multiple texture views and associated depth maps [10], especially for depth map, it is actually a grayscale image in which each pixel represents the geometrical information of the scene. Depth map and texture view are different in nature, since the former is composed of large smooth regions with sharp edges, whereas the latter contains complex content. The edge distortions of depth map coding might also lead to the degradation of synthesized views. To preserve the quality of sharp edges and further improve coding efficiency, in addition to supporting 35 intra prediction modes, as shown in Fig. 1 (b), several new coding techniques are adopted for depth map coding such as Depth Intra Skip (DIS) [11], Depth Modeling Modes (DMM) [12] and Segment-wise Direct Component Coding (SDC) [13]. Especially for the new intra prediction mode DMM, it divides a CU block into two non-rectangular regions, and each region is indicated by a constant value. DMM supports two partition types, wedgelet segmentation and arbitrary contour segmentation, as shown in Fig. 1 (c) and (d).

These advance coding techniques can achieve high compression efficiency for depth map coding in 3D-HEVC, but also lead to huge computational complexity. This brings enormous challenges for real-time applications such as ultra HD video on mobile devices with limited computational resources. Thus, it is necessary to investigate faster coding techniques to reduce the encoding complexity of 3D-HEVC while simultaneously keeping negligible encoding loss.

Though there exist many fast CU size decision works for depth map intra coding to reduce the encoding complexity of 3D-HEVC [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the different tradeoff between coding loss and coding time saving could be further investigated due to the following reason. For the user's specific application preference, some application scenarios require lower coding complexity, while other application scenarios may need to reduce coding complexity with coding quality is almost lossless simultaneously. In this work, to meet the user's specific application preference, we propose a tunable early CU size decision scheme for depth map intra coding in 3D-HEVC to achieve different tradeoff between coding loss and coding time saving. The novelties and the contributions of the proposed scheme are summarized as follows:

• A simple yet effective clustering based early CU size decision approach is proposed for 3D-HEVC depth map intra coding, in which only one valid feature is selected.

• It is an unsupervised learning method that adaptively updates the cluster center in each coding frame to adapt to the texture characteristics of different coding frames.

• To meet users' preference for specific application, tunable trade-off between coding time saving and coding loss is achieved by introducing the similarity distance.

The rest of this paper is organized as follows. Section 2 reviews the related works on coding time reduction for 3D-HEVC depth map intra coding. Section 3 describes the motivation and statistical analyses. Section 4 presents the early intra CU size decision scheme for 3D-HEVC depth map coding. Experimental results are provided in Section 5. Section 6 concludes this paper.