ERROR CO N CEALME N T ALGORITHMS APPLIED O N MULTI-VIEW VIDEO SEQUE N CES

Advances in display and camera technology enable new applications for three-dimensional (3D) scene communication, such as 3DTV. 3D Multi-view video contains a large amount of inter-view statistical dependencies, since all cameras capture the same scene from different viewpoints. Due to the vast raw bit rate, multi-view video data are expected to consume a large portion of available bandwidth. Therefore, an efficient compression techniques are the key for 3D scene communication. Error concealment techniques have the advantage of improving the received video quality without encoder modifications or transmission rate changes. To improve the quality of reconstructed 3D video, we propose different algorithms to conceal the erroneous and lost blocks of intra and inter coded frames exploiting the temporal, spatial and inter-view correlation between frames and views. There is a hybrid of space domain error concealment (SDEC) and time domain error concealment (TDEC) introduced for intra-frames losses. Three modes are discussed for inter-frames as well, which are time domain error concealment (TDEC), inter-view domain error concealment (IVDEC) and joint time and inter-view domain error concealment (TIVDEC).


INTRODUCTION
3D multi-view video has received wide attention lately and is expected to quickly replace traditional 2D video in many applications.In multi-view video coding (MVC), the original video content is a group of video sequences captured by multiple cameras at the same time for the same scene from different viewpoint angles.A multi-view video sequence exhibits high inter-view correlation, in addition to spatial-temporal correlations within each view.Error concealment exploits this advantage of inter-view and intra-view correlation to conceal lost packets or frames [1].
3D Video transmitted over wireless networks is always subject to packet losses including both random and burst errors.Due to the predictive coding structure of MVC compressed video, which utilizes intra and inter coded frames, errors could propagate to the subsequent frames and to the adjacent views and result in poor video quality [2,3].It is not possible to retransmit all erroneous or lost packets due to delay constraints on real-time video transmission [10].Therefore there is a need for postprocessing Error Concealment (EC) methods at decoder.EC algorithms are attractive since they have the advantage of reducing the visual artifacts caused by channel errors or erasures without increasing the bit rate or transmission delay or requiring any difficult modifications in the encoder.EC algorithms were proposed to protect monoview videos or stereo videos against transmission errors [5,7].These EC algorithms can be adopted to conceal erroneous frames in 3D multi-view video sequences.However, they are very efficient in concealing errors in multi-view video sequences as they take the advantage of the inter-view correlation [6].
In this paper, we focus on pre-compressed MVC sequences generated by Joint MVC (JMVC) reference software, based on H.264/AVC [4].We propose different hybrid EC algorithms which exploit the inter-view and intra-view spatial-temporal correlation to conceal lost blocks in intra-frames and inter-frames.Our goal is to reconstruct the 3D multi-view video sequences with high quality and low complexity.The rest of this paper is organized as follows: Section 2 presents the proposed error concealment algorithms for inter-frames and intraframes, Section 3 presents our experimental simulation results and Section 4 concludes the paper.

PROPOSED MVC ERROR CONCEALMENT ALGORITHMS
In this section, we present our proposed error concealment algorithms for 3D MVC intra-frames and inter-frames.In the following we detail our proposed EC algorithms: space domain EC (SDEC), time domain EC (TDEC), interview domain EC (IVDEC), joint space and time domain EC (STDEC) and joint time domain and interview EC (TIVDEC) algorithms.A high-level flow chart diagram of our proposed joint inter-frame and intraframe EC algorithm at the decoder showing when each algorithm is applicable is presented in Fig. 1.Intra-frame error concealment is not only essential for improving the video quality of reconstructed intra-frames but also for improving the video quality of reconstructed inter-frames in the subsequent frames and views.We present below the proposed error concealment algorithms for intra-frames to improve the subjective and objective quality of reconstructed 3D MVC video [6].
Consider an intra-frame with a missing macro-block (MB) as shown in Fig. 2. The vectors P U1 , P U2 , P R1 , P R2 , P B1 , P B2 , P L1 , and P L2 consist of the outside boundary pixels of the upper, right, bottom and left sides of the missing MB, respectively.The upper, right, bottom and left inner boundary pixels of the candidate MBs are represented by the vectors X U1 , X U2 , X R1 , X R2 , X B1 , X B2 , X L1 and X L2 respectively [5,7]. ) )

Proposed Hybrid Time and Inter-view Domain Error Concealment for 3D Multi-view Video Inter-frames
The assumed prediction structure for MVC is shown in Fig. 3 [7].Thus we propose that EC techniques for interview frames (P and B-frames) should be different across views: In even views (e.g.S2, S4 and S6), the first frame is concealed using interview frames as reference frames (IVEC) whereas other frames are concealed using the previous and following frames in the same view as reference frames (STDEC).In odd views (e.g.S1, S3 and S5), EC is done using the previous and following frames in the same view as well as the left and right frames in the adjacent views as the reference frames (TIVDEC).Thus, except the first frames, there are two candidate blocks for EC of lost MBs of inter-frames in even views, and four candidates for EC of lost MBs of inter-frames in odd views.
Fig. 3 Efficient prediction structure for MVC Decoder Motion Vector Estimation (DVME) techniques can estimate the motion vectors of the lost blocks by using a full search in the reference frames, as proposed in [8], and are useful in identifying the replacing MBs that minimize the boundary distortion error.Since the 3D video should often be decoded in real-time, the process of searching for MBs that replace erroneous ones using TDEC or IVDEC must be fast.Thus, we propose using the method of Outer Block Boundary Matching Algorithm (OBBMA) [1] instead, where the decoder determines the motion vector for error-free neighboring pixels in the reference frame which are adjacent to the lost MB.So, it only checks the highly correlated neighborhood motion vectors and the neighborhood MBs can be predicted using disparity vectors values.
Our proposed for inter-frames EC has three modes and is described below:

Even-view P Frame IVDEC algorithm
1. Find the locations of the lost MBs inside P frame.2. Apply Weighted Pixel Averaging algorithm [8] to find the matching pixels inside the left frame.3. Search for the most correlated candidates DVs to lost MB within left frame.4. Average DV values of the candidate MBs. 5. Replace the lost MB with the candidate MBs by using the averaged calculated value.

Even-view B Frame TDEC algorithm
1. Find the locations of the lost MBs inside B frame. 2. Apply OBBMA algorithm [1] to find the matching pixels inside the previous and subsequent frames.

SIMULATION RESULTS
In order to evaluate the performance of the proposed EC algorithms, we realized some experiments on the wellknown test sequences given by table (1).JMVC [4] reference software is employed as the platform for our proposed simulation work.All encoding parameters are set according to the JVT common test condition [9].For each sequence, the bit streams are produced and then transmitted over a communication channel with various random packet loss rates (3%, 5%, 10% and 20%) and then decoded by the proposed algorithms [6].Fig. 4, 5 show the subjective simulation results for the "Ballroom" sequence.We selected 241-th intra-coded frame with packet loss rate = 20%.We concealed the 241-th intraframe with the three proposed algorithms -SDEC, TDEC and STDEC.We can declare, that the proposed STDEC algorithm gives the highest subjective and objective results compared to TDEC and SDEC algorithms, where there is a lot of redundant information in space and time domains that can be exploited in EC.This is also demonstrated by the objectives results given in Fig. 6, 7.
Our simulation results show that the proposed STDEC algorithm is always better, in case of 3D MVC sequences containing significant movement [6], such as "Ballroom" sequence, as well as sequences with smooth motion (in the case of Exit sequence) and high loss rate.
To demonstrate the effectiveness of the proposed EC algorithms for P and B inter-frames, "Ballroom" and "Exit" sequences were used with low complexity parameters at the decoder which makes the EC process more realistic with small delay to be suitable for real-time transmission in wireless mobile systems.In the simulation tests, the inter-view frames in views S 1 and S 2 are corrupted, with random packet loss rates (3%, 5%, 10% and 20%).Fig. 8, 9 and Fig. 10, 11 present the objective results of the proposed EC algorithms for inter-frames in odd views and in even views, respectively.Fig. 12 and Fig. 13 show the subjective simulation results of the "Exit" sequence.In Fig. 12, it is noted that the 241-th inter B frame in view S 1 is concealed using proposed hybrid TIVDEC algorithm in (b) without taken into account intra-frame EC and in (c) with taking into account intraframe EC.In Fig. 13, it is clear that the 241-th inter P frame in view S 2 is concealed using proposed IVDEC algorithm in (b) without using intra-frame EC and in (c) with using intra-frame EC.We note that the performance of EC of inter-frames is improved by taking advantage of EC algorithms for both intra-frames and inter-frames and thus achieving a higher objective and subjective quality for the overall 3D video transmitted sequence.
This part includes the data on the measuring method and instruments as well as experimental results.

CONCLUSION
In this paper, we have proposed different error concealment algorithms with low complexity for intra and inter coded frames of 3D multi-view video sequences corrupted by errors due to transmission over communication channels.The major idea of our work is, that spatial, temporal and inter-view correlations are jointly utilized for EC of both intra and inter-frames.
Simulation results have proven, that the proposed hybrid algorithms of STDEC, TDEC, IVDEC and TIVDEC are significantly superior to conventional algorithms that exploit concealment only in space domain or only in time domain.Our simulation results show that the proposed EC algorithms can conceal lost blocks of intra-frames and inter-frames efficiently with low complexity and can recover 3D multi-view video sequences with high video quality.

Fig. 1
Fig. 1 Flowchart of the proposed MVC error concealment algorithm

Fig. 2
Fig. 2 8 sub-block adjacent to the lost MB and the corresponding sub-blocks in the previous frame with the candidate MB connected to them

3 .
Searching for the most matched candidates MVs to the lost MB within previous and subsequent frames.4. Averaging MV values of the candidate MBs. 5. Replace the lost MBs with the candidate MBs by using the averaged calculated value.

Fig. 8
Fig. 8 PSNR performance for "Ballroom" test sequence with different PLR for inter-frames within odd-views

Table 1
Properties of the test videosequences