Fuzzy Based Adaptive Deblocking Filters at Low-Bitrate HEVC Videos for Communication Networks

: In-loop filtering significantly helps detect and remove blocking artifacts across block boundaries in low bitrate coded High Efficiency Video Coding (HEVC) frames and improves its subjective visual quality in multimedia services over communication networks. However, on faster processing of the complex videos at a low bitrate, some visible artifacts considerably degrade the picture quality. In this paper, we proposed a four-step fuzzy based adaptive deblocking filter selection technique. The proposed method removes the quantization noise, blocking artifacts and corner outliers efficiently for HEVC coded videos even at low bit-rate. We have considered Y (luma), U (chroma-blue), and V (chroma-red) components parallelly. Finally, we have developed a fuzzy system to detect blocking artifacts and use adaptive filters as per requirement in all four quadrants, namely up 45 ◦ , down 45 ◦ , up 135 ◦ , and down 135 ◦ across horizontal and vertical block boundaries. In this context, experimentation is done on a wide variety of videos. An objective and subjective analysis is carried out with MATLAB software and Human Visual System (HVS). The proposed method substantially outperforms existing post-processing deblocking techniques in terms of YPSNR and BD_rate. In the proposed method, we achieved 0.32–0.97 dB values of YPSNR. Our method achieved a BD_rate of + 1.69% for the luma component, − 0.18% (U) and − 1.99% (V) for chroma components, respectively, with respect to the state-of-the-art methods. The proposed method proves to have low computational complexity and has better parallel processing, hence suitable for a real-time system in the near future.


Introduction
There is a requirement for large bandwidth in high-definition video content in the present era of multimedia applications [1][2][3][4][5]. Researchers have an incredibly challenging task to save bandwidth by performing adequate compression without affecting the visual contents over low bandwidth networks. Due to the large bandwidth capacity of video contents, performing a

Related Work
Many researchers have proposed different methods in the last decade to alleviate the corner outlier and blocking artifacts. The various methods include post-processing algorithms [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20] and in-loop filtering methods [10][11][12][13][14][15], which efficiently removes blocking artifacts. Numerous researches have studied variable intra-coding prediction properties of HEVC standard: Huang et al. [6] introduced a deblocking approach for improving the perceptual video quality of H.264 standard. Hannuksela et al. [7] presented different features of H.264/AVC. The author introduced the International Standardization Organization (ISO)/IEC (International Electro-technical Commission) along with ITU-T as standardized encoding and decoding techniques. The main disadvantage of such an approach is the lack of maximal freedom in implementing different applications without degrading image or video quality, implementation time, and cost. Dias et al. [8] proposed a rate distortion-based hypothesis to improve the quantization rate by HEVC. The authors investigated the Mean opinion score (MOS) and multimedia video quality assessment. Tang et al. [9] introduced HEVC video compression. Trzcianowski et al. [10] has reported a similar type of work. Chen et al. [11] demonstrated a vast survey of Ultra High Definition (UHD) document utilization and its implementation in the proposed systems. The authors achieved around 64% bitrate using the HEVC standard. He et al. [12] presented an estimation technique for UHD video data with minimum computational cost using VLSI design frameworks. Ahn et al. [13] proposed a parallel processing technique for the HEVC standard for quality improvement, whereas Blasi et al. [14] implemented the movement compensation technique to identify the complexity of coding performance using adaptive precision.
In the HEVC method, a frame is divided into a code tree of variable samples (16 × 16, 32 × 32, 64 × 64) and further divided into smaller blocks known as Coding Units (CU). The size of the Prediction Unit (PU), as well as the Transform Unit (TU), increases with the increase in the size of the Large Coding Unit (LCU), which results in some annoying artifacts even after applying in-loop filtering. Moreover, the in-loop filters cannot remove corner outliers due to its 1-D filtering properties [6][7][8][9][10][11][12][13][14]. On the other hand, post-processing techniques are more flexible and can be applied to any standards like MPEG 4/AVC and HEVC. The rectification of blocking artifacts can be achieved by using different post-processing approaches such as frequency domain analysis [15][16][17]19], Projection Onto Convex Sets (POCS) [16,17], Wavelet-based techniques [18], estimation theory [20,21], and filtering approach [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22]. The most common method is to apply a low-pass filter across the block boundaries to remove artifacts. The main disadvantage of the spatial filtering technique is over smoothing, attributed to its low pass properties. Kim et al. [17] presented a POCS based post-processing technique to remove blocking artifacts. POCS is more complex and requires high computations due to more iteration steps performed during Discrete Cosine Transform (DCT) as well as Inverse Discrete Cosine Transform (IDCT). Singh et al. [18] produced a DCT based filtration method for the smooth region. However, it has a poor performance. Hu et al. [20] proposed a Singular Valued Decomposition (SVD) technique. On the other hand, Yang et al. proposed an iteration-based Fields of Experts (FoE) technique. Due to its iterative approach, this method is not useful in the real-time image/video applications. The main drawback of FoE is associated with high computational complexity, as it works efficiently for an optimum (16 × 16) block size only. The techniques discussed in the literature can alleviate blocking artifacts to a maximum extent; however, the subjective performance at a low bitrate is far-flung from expectation.
The paper proposes a new four steps deblocking fuzzy filter selection technique to mitigate HEVC coding problems. Initially, we remove the quantization noise to avoid the wrong selection of the deblocking filter. Secondly, this work develops an efficient adaptive blocking artifact and corner outlier detection method. Finally, we introduce a novel fuzzy-based adaptive filter selection technique to simultaneously alleviate blocking artifacts and corner outliers. The complete approach works on HEVC luma and chroma components.
The rest of the paper is organized as follows: In Section 2, the proposed algorithm has been introduced, Section 3 gives results and discussion, and Section 4 concludes this research paper.

Materials and Methods
In this section, at the initial state, the quantization error is eliminated with the help of a preprocessing spatial filter, and an adaptive deblocking algorithm is implemented after that. Fuzzy filter selection for different regions is introduced along with simultaneously detecting and removing the corner outliers.
The proposed technique aims at removing an abrupt signal change in the consecutive frames. The subjective quality of the HEVC coded frames is significantly improved by removing all types of artifacts. The proposed technique is explained in Fig. 1. The details of each block are explained subsequently in the subsection of the proposed method.

Removal of Quantization Error using Spatial Filtering
Humans are always sensitive to abrupt changes in the signal in case of decoded frames. Fig. 2 depicts a pixel with an abrupt high or low contrast values than its neighboring pixels. The mean filter is applied to remove such kind of discontinuities. S defines a set of eight surrounding pixels, and the ninth pixel (p11) with an abrupt change is under consideration [9].   Let the pixel (p11) has a quantization signal; then it must satisfy the following conditions: Mathematically: where N is neighboring pixels, M is median, and λ is a threshold value. On the other hand, (Tp/f) is the threshold value to calculate the dissimilarity between two adjacent frames or pixels.
We consider (N = 8), (λ = 3) and (Tp/f = 6). If Eqs. (1)-(3) are satisfied by (p11), then it is observed that the pixel has a large signal value, and it will be replaced with the mean of all the eight neighboring pixels to remove undesired noise as mentioned in the below relation The subjective analysis of HEVC decoded image compressed at Quantization Parameter (QP) = 33 is shown in Fig. 3.

Blocking Artifacts Detection and Removal
Independent coding and decoding of blocks create discontinuities across block boundaries. The size of the prediction block varies from (4 × 4) to (64 × 64) for standard HEVC. LCU is used for the smooth part of the picture, whereas images with intricate details prefer small blocks to large blocks. In traditional HEVC, (8 × 8) block size was used in the deblocking filter to remove blocking artifacts. We process (4 × 4) grid to alleviate blocking artifacts while preventing spatial dependencies across the picture edges.  Although the HEVC method works on variable block sizes, to reduce its complexity, HEVC generally detects artifacts along (8 × 8) block only and enhances its overall processing time. However, blocking artifacts across some prediction units like (4 × 8), (4 × 16) is ignored and reduces its efficiency. This paper is processing every prediction unit across the block edges using a (4 × 4) grid. Each row of the grid is processed independently using a horizontal and vertical interlaced scan pattern, as shown in Fig. 6.
If there is an abrupt change in the pixel value across bilateral edges, then |p 3 − q 3 | should be greater than the threshold value (T ba ). The threshold (T ba ) value of the blocking artifact is set as T ba = (100 − QP) for the flat region and T ba = 2 (100 − QP) for the complex region. The region complexity of Fig. 4a is calculated by Eqs. (5) and (6) using (4 × 4) grid. From Fig. 4c, the average gradient or boundary activity function of left-side horizontal block edge pixels are represented by Ψ Left side . Similarly, the average gradient or boundary activity function of right-side horizontal block edge pixels are represented by Ψ Right side and is calculated as where T p/f = 6, the same threshold was used earlier to eliminate the quantization error. If the condition (5)-(8) are satisfied, then the region is considered as flat region, as shown in Fig. 7a.
For the intermediate regions, as shown in Figs. 7b and 7c, |Ψ Left side − Ψ Right side | should be small enough, such that it must satisfy at least one of the conditions stated in Eqs. (9) and (10).
where as  In a particular case, when a small object appears across the edges of a complex region, which significantly disturbs the frame's perceptual quality, as shown in Fig. 7(d), Eq. (12) is calculated. We considered pixels p2 and q2 instead of p3 and q3 to avoid disturbance to natural edges along horizontal, diagonal, and vertical block boundaries.
If the difference between p 3 and q 3 is enormous, an intermediate threshold (T i ) value will change adaptively, as shown in Eq. (12).
Such that For each (4 × 4) block, the absolute difference between S 1 and S 2 , as well as S 3 and S 4 , should be greater than T i as shown in Fig. 8. If Eq. (12) is satisfied for more than one row in a (4 × 4) grid, then the complete block of size (4 × 4) is considered as blocking edge.  Fig. 9 illustrates corner outlier across block boundaries with a block size of (4 × 4) for simplicity; a few pixels are considered [5][6][7][8][9][10]. Different corner outliers are determined (upper 45 • , upper 135 • , lower 135 • , and lower 45 • ). The amount of data lost during compression is detailed by (QP). The pixels (a4), (b4), (c4), and (d4) represent the center pixel, as shown in Fig. 9f.

Detection of Corner Outliers
Similarly, we can detect corner outliers in regions B, C, and D, as explained in Region A.

Removal of Corner Outliers
The pixels with high frequency compared to its neighboring pixels cause stair-shaped discontinuities and are updated using Eq. (14) For regions B, C, and D similar approach is being used while taking Fig. 9f into consideration.

Adaptive Filtering for Blocking Artifacts
The filters designed for various types of blocking artifacts are processed from top to bottom, starting from the pixels near the block boundaries on both the sides of block edges, as shown in Fig. 6. The filtration process must use the updated pixels as determined in Eq. (14). A strong filter is used in a flat region, a weak filter is sufficient for a complex region, and a medium filter gives outstanding results in the intermediate region. Tab. 1 represents the final selection of pixels and the corresponding filter type.

Fuzzy Filter Selection for Horizontal as well as Vertical Block Edges
This subsection proposes an efficient yet straight forward and fast fuzzy filter selection model. It will remove blocking artifacts in HEVC video sequences and reduce corner outliers. This will enhance the overall perceptual quality of the HEVC video frame processed from QCIF to FHD (1080p) with frame rates 30 and 60 fps. The given fuzzy approach also applies to real-time captured videos. It gives an efficient selection of filters with a low computational cost. Tab. 2 explains the detailed working of fuzzy filter selection. The proposed work uses the Mamdani fuzzy filter selection model with inputs as blocking artifacts across regions A, B, C, D, and corner outliers' representation for upper 45 • , upper 135 • , lower 135 • , lower 45 • for horizontal as well as vertical block boundaries simultaneously. The output of the proposed fuzzy system varies between 0-1, and the filters are selected accordingly.  Tab. 2 is explained by considering the row 12 of the table in which ABCD is 1011. As shown in Fig. 9a, for Upper 45 • , pixel B will be considered. The pixels adjacent s to B are A and D, as seen in Fig. 9e. The pixel combination across the horizontal block boundary is B (0) and D (1), the region is flat-complex, and hence, filter F 2 will be selected as shown in Tab. 1. The pixel combination across the vertical block boundary is B (0) and A (1), a flat-complex region, which means the fuzzy selector will select filter F2. Similarly, for the upper 135 • , the pixel to be considered is A with adjacent pixels as B and C, as depicted in Fig. 9b. Across the horizontal boundary, A and C are 1 (complex) and 1 (complex), respectively; hence, filter F4 comes into utilization. On the contrary, for vertical boundary pixel A (1) and B (0), the region is complexflat, so the filter F 3 is operational. In Lower 135 • , consider pixel D with neighboring pixels B and C. The horizontal block boundary pixels, D (1) and B (0) make the region as complexflat, and the filter F3 will work. In contrast, across vertical axis pixel D (1) and C( 1), the region becomes complex-complex, so F . 4 filter will be used. In the Lower 45 • , C pixel will be under consideration, having A on its horizontal boundary and D on its vertical boundary. In this case, both horizontal pixels C (1) and A (1) and vertical axis pixels C (1) and D (1) form a complex-complex combination that initiates the F 4 filter. The complete table operates similarly.
The proposed work also considers the chroma component, which is usually not considered in the HEVC deblocking filters across block edges of (8 × 8) block size. Block Edge Strength (BES) value depends on two significant factors, namely the difference between the motion vector of surrounding regions and the prediction modes. Block strength generally varies between (0-2). For the luma component (BES = 1) and (BES − 2), strong and light filters are used respectively.
On the other hand, when the video is coded at a low bitrate and contains complex motion data (i.e., BES = 2), most of the B and P frames are processed by I-frames. So, a regular deblocking filter is not advisable in this case. With an increase in the size of the transform unit and prediction unit of HEVC videos, blocking artifacts are more prominent across edges due to the chroma components than the luma, as shown in Fig. 10. Fig. 10 clearly shows that the blocking artifacts mostly occurred due to chroma components. The difference between the V component's value on both sides of the block edges is predominating the other components.

Results and Discussion
The following test sequences validate our results and compare them with various methods, as discussed in Standard HEVC decoded frame, Kim et al. [17], Chen et al. [22], Norkin et al. [23], Karimzadeh et al. [25], Wang et al. [28]. The details of test sequences are as shown in Tab. 3.
Our method uses the same HEVC coding, which Kim et al. [17], Chen et al. [22], Norkin et al. [23], Karimzadeh et al. [25], Wang et al. [28] used in the Standard HEVC frame decoding approach along with adaptive loop filtering. In fast videos with very little information (Blue_sky, Basketball_Drive), blocking artifacts are visible rarely when compressed at QP < 30. On the other hand, videos with intricate details (KristenAndSara, Duck_take_off) coded at a fast speed, when compressed at QP < 25, result in blocking artifacts' disappearance. Therefore, we have analyzed all the results above QP > 30.  . 11 depicts the luma component peak signal to noise ratio (YPSNR) vs. bitrate for standard definition video sequences, namely container and Vtc1nw. Both video sequences are encoded using HEVC at QP = 33 and analyzed using different post-processing techniques and the proposed method. The variation of average YPSNR for most of the methods, namely Kim et al. [17], Chen et al. [22], Norkin et al. [23], Karimzadeh et al. [25], Wang et al. [28] are almost linear in terms of bitrate for low-resolution videos. When the resolution of videos increases, the different methods' performance significantly varied. The proposed method is comparatively linear and is almost independent of the resolution of video sequence due to its adaptive nature. Moreover, the fuzzy system provides efficient filtering for blocking artifacts, quantization noise, corner outlier removal, and provides perceptual quality of reconstructed videos. The proposed method outperforms existing techniques in terms of objective metrics YPSNR. It is more enhanced by (0.483-0.97dB) for different bitrates than the standard HEVC encoding method. For the SD video sequence, the average improvement in YPSNR ranges between (0.5-0.86dB); for HD, it is lying between (0.32-0.47 dB) w.r.t existing techniques.  Fig. 13b, even for complex video sequences like ducks_take_off, the flying movement of ducks or ripples in water cause lots of artifacts when coded with HEVC standard at QP = 33. We have marked different areas A, B, C, and D, where we can easily observe artifacts and degradation in the image/frame's perceptual quality.  Fig. 13h also depicts the result of the quantization error removal stage of the proposed method in region A, corner outlier removal in region B, and blocking artifacts in Region C and D, respectively, as shown in red rectangles. Fig. 13h clearly shows the proposed method's correctness compared to the existing methods.
Fig. 14 represents the luma component analysis of various methods and compare the objective quality using YPSNR metrics for KristenAndSara as well as Blue_sky in 1080p video sequence. These sequences do not contain many complex data, and the performance of the proposed method is exponential. The proposed method shows promising results while reconstructing these video sequences, as shown in Tab. 4. Due to chroma consideration, blocking artifacts and corner outliers are alleviated efficiently with the proposed methods compared to other methods that generally focused on the luma component only, as discussed in this paper. Tab. 4 represents BD_rate for luma and chroma components of the proposed method and compares the results with BD_rate(Y), i.e., luma components of other methods as they are not considered chroma components. We can observe from Tab. 4 that Norkin et al. [23] have reached +3.81% BD_rate (luma), within an acceptable range. The value of BD_rate close to zero or negative shows the better performance of a given system. The proposed method achieves +1.69% BD_rate for the luma component and −0.18% (U) and −1.99% (V) chroma components, which shows the proposed technique's efficiency.   15 shows the subjective analysis of Blue_sky (1080p). The blue_sky video sequence is an exceptionally smooth sequence without any intricate details in frames. The proposed method shows excellent subjective output in terms of the perceptual visual quality of the frame. Fig. 15c shows more details in these regions (A, B, C, D) and outperform other existing methods discussed in this paper. •

Time Complexity of Decoder
To validate the decoder's complexity of the proposed method, we compare our results with the HEVC standard (HM16.9). The mathematical relation for time complexity is given by To evaluate the system's computation complexity, we use different videos sequence, and the comparison is shown in Tab. 5.
From Tab. 5, it is clear that the proposed method's decoding time is quite close to the HEVC standard decoder even after removing blocking artifacts and corner outliers, which shows less computational complexity of the proposed method.

Conclusion
High Efficiency Video Coding is a powerful video compression technique, specifically when HD videos are under consideration. In-loop filter methods discussed in literature reduce blocking artifacts to some extent. However, when some complex data coded at a fast speed, certain blocking artifacts still exist, which significantly degrade the videos' visual quality coded at a low bitrate. We proposed a four-step adaptive deblocking fuzzy-based filter selection method that efficiently removes block artifacts and corner outliers for various video sequences. The proposed method removes all the unwanted quantization noise, which disturbs the video frames natural edges and produces difficulty in selecting the desired filter. Later, we introduce a novel technique to detect blocking artifact as well as corner outliers. Finally, a fuzzy filter selection technique applies to remove blocking artifact along with corner outlier simultaneously in all the regions, namely (Flat-Flat), (Flat-Complex), (Complex-Flat) and (Complex-Complex). The proposed deblocking method is applied both to luma and chroma components. The result section's experimentation clearly shows the proposed technique efficiency in terms of objective and subjective analysis of diverse input videos with tolerable visual quality using HVS. We have achieved average improvement in YPSNR ranges between (0.5-0.86 dB) for SD videos; for HD, it is lying between (0.32-0.47 dB) w.r.t existing techniques. The proposed method achieves +1.69% BD_rate for the luma component, 0.18% (U), and −1.99% (V) for chroma components, which shows the proposed technique's efficiency. The low computational complexity of the proposed method provides a platform for real-time applications in the near future.
Funding Statement: The authors received no specific funding for this study.

Conflicts of Interest:
The authors declare that they have no conflict of intrest to report regarding the present study.