Quality Assessment of High-Resolution Images with Small Distortions after Compression

. Image quality assessment still remains a highly relevant problem, and objective quality assessment methods are being actively developed. The methods, based on the Structural Similarity index method, have become very popular during the last decade. However, their sensitivity has been investigated using only small images and only in the cases of obvious image distortions. In this paper, we have investigated a quality assessment of high-resolution images with low distortions after compression using the Structural Similarity index method. The specific cases, related to the usage of this method for high-resolution images, are analyzed, and the problems that occur when using the method are identified. Experimental investigations have shown that image downsampling is necessary when the image quality is assessed by the Structural Similarity index method. Moreover, a sensitive algorithm suitable for the comparison of the quality of high-resolution images with small distortions is proposed and investigated in the paper.


Introduction
As image processing has become one of the essential parts in the digital information industry, great efforts are directed to the development of image quality assessment methods during the last 20 years.Image quality assessment methods play an important role in a wide range of applications, and they are based on a comparison of the original and distorted (after compression) images, which is represented by the numeric expression.However, a subjective people's impression remains the most important indicator for image quality comparison.The point of the subjective method, called subjective Mean Opinion Score (MOS), is based on the average of the opinion scores, provided by a group of experts on the quality of the compressed image (Ponomarenko et al., 2009a).The main disadvantage of the method is that it is expensive and time consuming (Wang et al., 2004a).Another essential disadvantage of the method is that the human eye fails to notice small artefacts while assessing high-resolution images.For example, the same image could be modified by many variants of the JPEG algorithm, but the human visual perception would not notice any changes in the distorted images.For these reasons, objective methods are often used for image quality assessment (Wang and Bovik, 2009).A comprehensive review of such methods is given in (Thung and Raveendran, 2009), (George and Prabavathy, 2014), (Joy and Sarma, 2014).
The most wide-used method for image quality assessment is Peak Signal-to-Noise Ratio (PSNR) (Hore and Ziou, 2010), (Salomon, 2013).The basis of this method is the calculation of the Mean Square Error (MSE).But it should be noted that a high PSNR value not always guarantees a good image quality (Wang and Bovik, 2009), (Hore and Ziou, 2010).Another popular method for image quality assessment is a Structural Similarity (SSIM) index method (Wang et al., 2004a).It is more complex, but accurate, therefore this method has become of great interest during the last decade.In order to evaluate the objective metrics correspondence to human impression, the image databases are created.A comprehensive survey of such databases is presented in (Winkler, 2012) (Chandler, 2013).The image databases have reference images, and the images, distorted by various noises, as well as MOS values estimating expert opinions on each image (Ponomarenko et al., 2009b).For example, the TID2013 database (Ponomarenko et al., 2013) contains 1700 images, where 68 distorted images (four levels for each of seventeen types of distortions) have been obtained for each reference image.In total, 985 estimations were performed in five countries (Ukraine, Finland, France, USA, and Italy).Researches have shown that, in many cases of distorted images, the SSIM index-based methods provide very good values correlated with the values of the Human Visual System (HVS) (Ponomarenko et al., 2009b), (Ponomarenko et al., 2013).The HVS value is obtained by determining the correlation value between MOS and the "ideal" MOS when the number of experiments approaches infinity.
Nowadays huge sets of high-resolution digital images are used in medicine, astronautics, social networks, etc.Such images must be automatically compressed using JPEG (Wallace, 1992), JPEG2000 (Skodras et al., 2001) or other compression methods in order to reduce the amount of occupied memory while maintaining the same resolution.Moreover, it is often important to keep the sufficient high quality of images, when distortions are hardly noticeable.However, the state-of-art methods, such as the Structural Similarity index method used for image quality assessment, are suitable only in the case of obvious distortions.Moreover, the SSIM index method and its extensions were successfully tested only on small images with big distortions (Ponomarenko et al., 2013).As it is demonstrated in the paper hereafter, the quality assessment problems arise, where large and high-resolution images with hardly noticeable changes are investigated.Therefore, methods for evaluating high-resolution images with even small distortions should be developed.
In this paper, we identify and highlight weaknesses of the existing SSIM index method-based algorithms.The specific cases, related to the usage of the SSIM index method, are analyzed.It is identified here that the original SSIM index method is not suitable for quality assessment of high-resolution images with low distortions.It is shown that downsampling is necessary when the high-resolution image quality is assessed by the SSIM index method.We propose a modified algorithm on the basis of the SSIM index method that is suitable for the comparison of high-resolution images.
The remainder of this paper is organized as follows.The concepts of the SSIM index method and its extensions, as well as their application areas, are discussed in Section 2.
In Section 3, we analyze the suitability of the original SSIM index method for quality assessment of high-resolution images with small distortions and identify the occurring problems.The results of the experimental investigations on image data downsamling before the usage of the SSIM index method are presented in Section 4. The algorithm suitable for quality assessment of high-resolution images with small distortions is also proposed and experimentally investigated here.Finally, conclusions are drawn in Section 5.
The SSIM index method assesses the visual impact of three characteristics of an image: luminance, contrast, and structure.The overall SIMM index value is a multiplicative combination of these characteristics.With two local image patches given (a patch of the original image x and a patch of the distorted image y ), the local SSIM index value is defined as (Wang et al., 2004a) where  1), compose the so-called SSIM index map.The SSIM index value of the whole image is obtained by averaging the local SSIM index values, calculated by using a sliding window of the size 8 8 pixels.The SSIM index method measures the similarity between the two images and the similarity value lies between -1 and 1.If In spite of the fact that the mathematical model of the SSIM index method is rather complex, it is considered that the resulting indicator sufficiently reflects the human visual perception features.Mathematical properties of the method are comprehensively discussed in (Brunet et al., 2012).
It should be noted that the distance between an image and a human can affect his/her perception of image quality.For this reason, a Multi-Scale SSIM (MS-SSIM) index was developed and formulated as follows (Wang et al., 2004b) Here the contrast and structure are calculated at each scale i , and the luminance is computed only at scale P .The overall quality evaluation is obtained by combining the measurements over different P scales.
It is obvious that the MS-SSIM index has a higher correlation with HVS for relatively small images, where the image quality for a human depends on the distance from an eye to the image (scale).However, in the case of large images with small distortions, presented in a high resolution, there is no such dependence.Further, several image quality assessment methods have been developed on the basis of the SSIM index method: a HWSSIM method (Ji et al., 2008), where a discrete Haar wavelet transform is used, a DWT-SSIM method (Yang et al., 2008), in which a discrete wavelet transform is applied, a PSSIM c method (Yang et al., 2008), where the SSIM index values of local image regions are adjusted by perceptual weights defined from the contrasts of the regions.
Despite the variety of the SSIM-based methods, in this paper, we focus on the investigation of the original SSIM index method (Wang et al., 2004a), which is the basis of the others.

Suitability analysis of the SSIM index method
In this section, we analyze the suitability of the original SSIM index method for quality assessment of high-resolution images with small distortions and identify the occurring problems.
In the experimental investigations, a set of 30 different high-resolution images (3882×2608 pixels, 28.9 MB) was used for calculating the image similarity and SSIM index values.The selected images were modified by Spencer Kimball and Peter Mattis JPEG storage procedure which had been implemented in GIMP 2.6.11(WEB, a).The default settings were kept the same, only the Quality Factor (QF) was changed.The QF is the main parameter influencing the image quality after JPEG compression, which determines the compression ratio.This parameter is an integer number between 0 and 100, used to parameterize a quantization matrix.The greater this number is, the less information is lost.
The compression algorithm was implemented by the Independent JPEG Group (IJG) (WEB, b).In the experimental investigation, three different compression quality factor values (85, 70, and 50) are used.The SSIM index values are calculated for each image from a set of 30 different images, using the algorithm developed by Zhou Wang and implemented in MATLAB (WEB, c) (we call it Algorithm 1).The obtained SSIM index values are averaged are presented in Table 1.We can see that file sizes of the images are different, however, the averaged SSIM index values are very close to 1.This is due to the fact that the mathematical model of the SSIM index method and the resulting index values reflect the human visual perception of distortions very well.In this case, a human eye would not easily fix the obvious differences between high-resolution images, therefore the SSIM index values are very high.An illustration of this fact is presented in Fig. 1.Only after zooming one of the fragments of the examined image, we can see that the differences between the original and distorted images are obvious.Here, in the cases a) and b), the SSIM index values are the same, however, the quality of the images is very different (when the values are expressed by four digits after the comma).Moreover, as we see in the case c), the SSIM index value has changed slightly, after a significant change in the image quality.The problem is that the SSIM index values look similar for the human and do not reflect the distortion as much as it should be while exploring the JPEG algorithm variants.This problem is particularly relevant if large groups of images must be processed and quality assessment must be automatic.Although a human could not notice some distortions in the compressed images, however, these images must be of a high quality for further processing, e.g. for some pattern recognition, for pressing the images.Moreover, in the case of medical images, they must be saved of a high quality and high resolution for accurate diagnosis.

Image data downsampling before calculating the SSIM index values
Image reduction (downsampling) is typically used before the assessment of image quality by the SSIM index method (Wang et al., 2004a).The image reduction coefficient F is calculated by the following empirical formula: )), )/ where N is the width and M is the length of the image; f is the dimension size of the downsampled image, which is suggested to be 256 pixels [2].For example, for the image with dimensions of 1536  1536, F value is 6, thus it should be reduced by coefficient 6.The reduction of an image fragment of size F  F is processed by the following formula: is the value of every pixel.The calculation of this formula is repeated from 1 to M and from 1 to N by the step F .This image reduction procedure is used in Algorithm 1 and illustrated in Fig. 2.
The case, where SSIM index value is computed without downsampling, is called Algorithm 2. In Table2, the comparison of Algorithm 1 and Algorithm 2 is presented when assessing the quality of compressed images by the SSIM index method.We see that when using Algorithm 2, the obtained SSIM index values differ more, i.e. they better reflect changes (and distortions) in the images.We conclude that Algorithm 2 is more suitable for quality assessment of images with small distortions, however, it has some disadvantages.
Firstly, Algorithm 2 is time consuming, due to the fact that large amounts of data must be processed.For evidence, the calculation speed of the SSIM index values has been investigated and performance of the algorithms has been compared.The experimental investigation is carried out in the MATLAB R2012b system, using a computer with a Pentium DualCore T4200 processor and 4GB of RAM.A selected set of 30 different images of sizes 3882×2608 and 1941×1304 was used.Experiments are The image downsampling can be implemented by various ways of interpolation.The main interpolation methods used for image downsampling are reviewed by Meijering in (Meijering, 2002), and the most popular methods are as follows: --Nearest-neighbour interpolation, where the output pixel is assigned to the value of the pixel the point of which falls within, and no other pixels are considered.-Linear interpolation, where the output pixel value is a weighted average of pixels in the nearest 2-by-2 neighbourhood.-Cubic interpolation, where the output pixel value is a weighted average of pixels in the nearest 4-by-4 neighbourhood..In this paper, in order to find the most suitable interpolation method for image dowsampling, the experimental investigations have been performed.The averages of the SSIM index values for the selected set of 30 images are presented in TABLE 4.
We see that the differences between the SSIM index values obtained for the original and distorted images are higher, when the nearest-neighbour interpolation is used.Therefore, we propose to use this kind of interpolation for image downsampling when small distortions of high-resolution images are investigated (we call it Algorithm 3). .Despite the fact that the type of downsampling is the only difference from the other investigated algorithms (Algorithm 1 and Algorithm 2), this difference considerably impacts the final SSIM index value (in the cases, where the size of an image is larger than 256×256).
The averages of the SSIM index values for a set of high-resolution 30 images, obtained by the three analysed algorithms, are presented in Table 5.By way of illustration, we present the SSIM maps of different size of images distorted by impulsive noise, obtained by Algorithm 1 and Algorithm 3, in Fig. 6.We see that in both cases the maps are similar, thus the size of an image does not influence the values of the SSIM index when using both algorithms.
Algorithm 3 reflects the differences very well (TABLE 5) contrariwise to Algorithm 1. Experimental investigations of calculation speed of the SSIM index value are also performed using the proposed Algorithm 3, but only slight differences in speed compared with the original algorithm (Algorithm 1) were obtained.
For clarity of the interpretation of the obtained results, we propose to use an inverse index of similarity (ISSIM) instead of the SSIM index.It is calculated by the formula: 100 ) (1 =   SSIM ISSIM . Hence, the higher the quality of the image, the closer to 0 the ISSIM index value is (Table 5).
luminance, the local contrast, and the local structure, respectively; x  , y  , x  , y  , and xy  are the local means, the standard deviations, and the cross-correlation for the local image patches x and y , respectivelyparameters used for adjustment of a relative importance of the three components.The constants 1 C and 2 C are used to avoid instability when the means and deviations are close to zero.The values, obtained by formula (

Fig. 1 .
Fig. 1.Fragments of an image changed by JPEG algorithms, using various values of the image quality level (SSIM index is computed by Algorithm 1).

Fig. 4 .
Fig. 4. The image distorted by increasing the contrast and the SSIM maps of the different size images, obtained by Algorithm 2.

Fig. 5 .
Fig. 5.The image distorted by impulsive noise and the SSIM maps of the different size images, obtained by Algorithm 2.

Fig. 6 .
Fig. 6.The SSIM maps of different size images distorted by impulsive noise, obtained by Algorithm 1 and Algorithm 3.

Table 1 .
Average of the SSIM index values obtained by Algorithm 1

Table 4 .
The SSIM index values obtained using different interpolations