Splicing Image and Its Localization: A Survey

: With the rapid development of information technology, digital images have become an important medium for information transmission. However, manipulating images is becoming a common task with the powerful image editing tools and software, and people can tamper the images content without leaving any visible traces of splicing in order to gain personal goal. Images are easily spliced and distributed, and the situation will be a great threat to social security. The survey covers splicing image and its localization. The present status of splicing image localization approaches is discussed along with a recommendation for future research.


Introduction
Digital image is vulnerable to manipulation and doctoring so that people can tamper with images by image editing software, and apply tampered image in diverse areas. In addition, image tampering is a common task due to the continuous development of science and technology. In 2013, Iran announced that they can produce fighter planes, which could be invisible, in order to enhance the military strength of their country. As shown in Fig. 1(c). Although this image was quickly confirmed to be a spliced image, it still caused bad influence in global world. In 2017, the original author of "Chinese Boys" photographs sued Taiwan artist Jimmy Lin for infringement. The reason is that without the permission of the author of the original work, Jimmy Lin synthesizes bald head photos on the basis of "Chinese Boys" and personal images of Jimmy Lin using the software of mapping without authorization, thus arousing people's fanaticism for idols. As shown in Fig. 2.
(a) Chinese Boys (b) Splicing image There are many influential events, such as the examples of mentioned above, which have caused negative effects in the history. Therefore, tampered images are a potential threat to social stability. In order to protecting the use of digital image, it is important to identify the authenticity of the image, and more conclusive to locate the tampered region directly in the tampered image. In general, there are two types of image tampering: image splicing [Ng and Chang (2004); Yuan and Ni (2017)] and copy-move [Arun (2015)]. However, some post-processing, including blurring, retouching, beautifying Zeng et al. [Zeng, Zhan, Kang et al. (2017)], rotating, etc., make the tampering traces in the tampered image to be eliminated. Considering the diversity of tampering operations [Asghar, Habib and Hussain (2016)], image forensics technology should adopt specific analysis solutions to improve the accuracy of forensics scheme. In this paper, image splicing, and its localization are studied and discussed. The rest of the paper is organized as follows. Section 2 introduces the image splicing and its localization. Section 3 presents the literature survey of splicing image localization. Comparative analysis of different approaches is provided in Section 4. Finally, we conclude this paper in Section 5.

Framework of splicing image localization
Image splicing is a process in which it crops and pastes regions from the same or different images [Bharti and Tandel (2016)]. Namely, copying a part of an image and pasting it onto other images to form a new image using image editing tools like Photoshop, MEITU and so on. The processing of spliced image is shown in Fig. 3. As shown in Fig. 3, the part of the Fig. 3(b) is cropped and pasted onto the Fig. 3(a) to form the Fig. 3(c). It is not difficult to find that in a spliced image, the difference between two image blocks from different images mainly comes from the following aspects. The first aspect is the information carried by the image itself, such as noise Pan et al. [Pan, Xing and Lyu (2012)], light, CFA algorithm Ferrara et al. [Ferrara, Bianchi, Rosa et al. (2012)], blur type [Bahrami and Kot (2014)], camera source [Chen, Fridrich, Goljan et al. (2008)], and so on. The second aspect is the smoothness [Du, Ma, Chen et al. (2015)], which is the operation of removing abrupt edges. The last aspect is the compression characteristics of the original region and the splicing region [Piva (2011); ; ; Mire, Dhok, Mistry et al. (2015)]. The above aspects provide clues for image splicing localization [Wu, Zhu, Li et al. (2005) The active approach is based on watermarking [Potdar, Han and Chang (2005); Yin, Lin, Qiu et al. (2005)]. The watermarking consists of fragile watermarking and semi-fragile watermarking. Both of them are sensitive to image tampering attacks, and difficult to be tampered. The watermarking will be destroyed when the image undergoes tampering, which provides important clues to the localization methods. However, it is needed to know the prior information about the image, so it is mainly used for copyright protection by embedding the digital information to the image.
Input image Pre-processing Image segmentation Hence, passive approach is more favored by researchers, and it is just needed to analyze the traces left on the image by image splicing. The basic framework of image splicing localization is shown in Fig. 4. In the existing localization framework, there are four steps. Firstly, many schemes adopt preprocessing operation to extract the feature effectively, such as color space conversion, discrete wavelet transform (DWT) [He, Wei, Wei et al. (2012)], discrete cosine transform (DCT) [Alahmadi, Hussain, Aboalsamh et al. (2013); Li, Qiang, Xiao et al. (2017)], principle component analysis (PCA) [Pyatykh, Hesser and Zheng (2013)]. Secondly, image is often divided into image blocks with small size with various segment in order to achieve high accuracy of localization. Thirdly, the designed algorithm is used to extract the feature of each image blocks. Finally, all image blocks are classified into two clusters by classification algorithm. In addition, the selection of feature is also the key of the localization.

Splicing image localization techniques
Splicing operation may change the characteristics of the part of image, then feature used to describe the original region and splicing region may be inconsistent. The researcher can track the inconsistent between the original area and splicing area to locate the spliced image blocks. The major image splicing localization schemes fall in one of the following categories. Fig. 5 shows the various splicing image localization techniques.

Pixel based techniques
The splicing operation will destroy the continuity of pixels and neighborhood pixels. Wu et al. [Wu, Zhu, Li et al. (2005)] used the generalized model to construct a frame of image based on pixel level. Du et al. [Du, Ma, Chen et al. (2015)] analyzed whether the continuity between the pixels in the image and the neighborhood pixels was below the threshold, and labeled the location of the spliced region.

Camera based techniques
When a picture is taken, it involves a series of processing steps on the path from sensor to memory, and each image will carry the inherent information of the camera: camera response, sensor noise, color filter array and so on. The splicing operation will make two kinds of camera inherent information existing in one splicing image. The Photo Response Non-Uniformity (PRNU) [Lukás, Fridrich and Goljan (2006); Rosenfeld and Sencar (2009);Liu (2012); Chierchia, Poggi, Sansone et al. (2014)] is the intrinsic fingerprints of the original image, and the basic idea is to estimate a device's PRNU at first, and then evaluate whether the image under question conforms to it. If local deviations appear, the presence of a splice in the corresponding region is posited. Dehnie et al. proposed a wavelet to filter the image in order to further enhance the difference of pattern noise from different camera source. However, calculation of PRNU needs a prior knowledge about the camera source. In addition, most shooting devices are equipped with a sensor that covers a Color Filter Array (CFA), which produces one value per pixel, and the image is then transformed into three channels using interpolation. Thus, for each channel, a number of values originate from the environment, while the rest are interpolated from them. The processing mentioned make the correlation between adjacent pixels. The splicing operation would eliminate the correlation between each pixel and neighborhood pixels by using a color filter array (CFA). The splicing region localization has been implemented by learning, the changing characteristics of the CFA caused by the spliced operation [Cao, Zhao and Ni (2009)]. In Dirik et al. [Dirik and Memon (2009)], two different features are proposed: The first attempts to detect the CFA pattern used during image capture by subsampling the image using various possible selection patterns, reinterpolating it, and comparing it to the original. Having emulated the CFA interpolation process using the estimated parameters, one could then make use of local discrepancies between the interpolated and the observed values to detect local tampering. Sun et al. [Sun, Lang, Gong et al. (2017)] believed that there was a color shift between the splicing region and the original region. The original image could be reconstructed by simulating the CFA model, and the splicing region could be located by analyzing the inconsistency between the image to be tested and the original image in the pixel neighborhood. Although this class of algorithms has gained high accuracy of localization, it needs to simulate and estimate the CFA pattern and interpolation algorithm used in the original image for feature extraction and analysis.

Format based techniques
If the splicing operation is performed in compressed image and uncompressed image, there are different compression characteristics in one splicing image. The splicing region can be located by exploiting the traces left by JPEG compression. Quantization of an image's DCT coefficients is a major step in the JPEG compression pipeline, in which the quantization factor is a function of the chosen compression quality. It has been observed that consecutive JPEG compressions at different qualities lead to specific periodicities in the DCT coefficient distribution. In Amerini et al. [Amerini, Becarelli, Caldelli et al. (2014)], author propose a splicing localization scheme based on DCT variation characteristics, and locate splicing region by analyzing DCT distribution characteristics between different image blocks. Bianchi et al. ] proposed the idea of taking advantage of the inconsistent compression characteristics between the spliced region and the original region to realize image splicing localization.

Noise based techniques
Assuming that the random noise in the image obeys the gaussian distribution, the noise level of the splicing region and the original region will be inconsistent. Mahdian et al. used the technique of wavelet transform to extract the local noise variance in Mahdian et al. [Mahdian and Saic (2009)]. In Pan et al. [Pan, Xing and Lyu (2012)], Pan et al. found that the coefficients of different frequency subbands present a continuous and specific regularity in natural images. According to the clue, they proposed a two-level clustering detection scheme. Based on the research above, Lyu designed an algorithm based on the relationship between noise characteristics and kurtosis in Lyu et al. [Lyu, Pan and Xing (2014)]. The localization schemes mentioned have obtained high localization performance when the noise difference between the splicing region and the original region is large. However, those schemes fail to locate the spliced region when the noise level is quite small. Hence, a localization scheme based on PCA technology was proposed by analyzing the difference of image block noise levels in Zeng et al. [Zeng, Zhan, Kang (2017)]. To preserve the structure information of image content, Chen used the location algorithm combining super-pixel segmentation and noise features in the literature in Chen et al. [Chen, Zhao, Shi et al. (2018)].

Special scenario based techniques
For the blurred image and the shadow image, the splicing operation will make the image have two kinds of blurred kernels or different shadow parameters [Liu, Cao, Chao et al. (2011)]. Bahrami believed that there may be different types of blurring between the splicing region and the original area in the tampered blurred image, so they proposed a method of using the blur type differences to realize blurred splicing image localization [Bahrami, Kot, Li et al. (2015)].

Comparative analysis
All the mentioned techniques above are able to locate the spliced region in one splicing image. Tab. 1 shows the comparison of various splicing image localization techniques. It summarizes parameters, merits and demerits of the methods.

Research prospect
The existing localization schemes have achieved high accuracy of localization, but the accuracy can be further improved and most of these technologies are suitable for gray image. A color image is composed of color information and luminance information, and only the luminance information is used to extract the features for classification in gray images. Therefore, in the localization scheme, it is insufficient to extract the features for classification by only using the luminance information. In order to make full use of color information and improve localization accuracy, researchers should focus on the localization scheme based on color splicing images, and that is what our team has been working on.

Conclusion
In this paper, we have discussed about the image splicing and framework of image splicing localization. The basic flow of how spliced region is located is shown. The overview of different techniques that helps us to locate the spliced region and comparison of different techniques based on different parameters with its merits and demerits are provided. Although the existing localization schemes can effectively locate the spliced region in one splicing image, image splicing localization technology also needs further development to adapt to more application scenarios.