4.3.1. Fuzziness Test Method
In terms of the fuzziness of privacy-protected images generated, because PartialFace retains part of the contours of the face, the recognition accuracy of the differential privacy method was less than 90%. Therefore, the main comparison is between RLLFPR and the Arnold transform method and privacy-protected images produced by AES encryption. RLLFPR, Arnold transform, and AES encryption methods all blur the original face to a large extent, which makes it difficult to identify with human vision.
The privacy-preserved faces generated by RLLFPR cannot be recognized by human vision, and some statistical characteristics of their performance are shown in
Figure 8.
Figure 8 shows the grayscale histogram of the original image, processed by the RLLFPR method, Arnold transform method, and AES encryption method. Compared with the original image (
Figure 8a,e), the gray distribution of the Arnold transform image (
Figure 8c,g) is basically unchanged, which is related to the restoration of the original image after multiple rounds of operation. The gray histogram obtained by the RLLFPR method (
Figure 8b,f) and the AES encrypted image (
Figure 8d,h) show similar distributions for different faces; the RLLFPR method shows a similar bell-shaped distribution; and AES shows a similar white-noise distribution. The similar gray histogram distribution of different faces can improve the blurring performance of facial images.
Figure 9 and
Table 2 show the original image, Arnold, AES, as well as RLLFPR image-adjacent pixel correlation analysis. Specifically, 3000 pairs of adjacent pixels from the three R, G, B channels were selected for analysis. As can be seen in
Table 2, the original image (
Figure 9a) presented a high linear correlation, and the three directional correlation values were all greater than 0.98. Although Arnold transform (
Figure 9c) blurred what can be seen with human vision, it only moved pixels between locations, so it still had high correlation, and the three directional phase property values were around 0.8. RLLFPR (
Figure 9d) and AES (
Figure 9b) showed low correlation in the correlation analysis; RLLFPR showed three correlations around 0.05, and AES had even lower values, around 0.005. Low correlation can indicate that the image pixels are less regular and difficult to predict. In general, the correlation effect of RLLFPR is much higher than that of the Arnold transform and slightly inferior to the AES method.
Table 3 shows a comparison of the PSNR and UACI values of the privacy-protected face images generated by RLLFPR with the Arnold transformed, noisy, and AES encrypted images. The noise-processed face is still recognizable and can be used as an intermediate value for comparison. The PSNR value is the highest and the UACI value is the lowest for the face after noise processing, indicating its high recognizability. The processed image with noise has less distortion and blurring. The RLLFPR, Arnold, and AES encrypted versions of all three images are not visible to the human eye and show better results in terms of PSNR and UACI. The RLLFPR method is similar to the Arnold transform in terms of both data PSNR and UACI, with PSNR values of 11.80 and 11.53 and UACI values of 50.61 and 53.93. All methods resulted in an AES-encrypted ciphertext with values of 8.77 and 75.27. However, AES-encrypted images cannot be recognized.
In addition, we designed three fuzziness tests.
First, we conducted human evaluation experiments by means of questionnaires. The questionnaire consisted of two types of single-choice questions. The experimental group was based on the privacy-preserved faces generated by RLLFPR, and the control group comprised the privacy-preserved images processed by AES encryption. The first single-item choice was entitled “Similarity problem” (
Figure 10a), which judged the degree of similarity between two faces by giving the original facial image and the privacy-preserved image of the original face. Five options were given, respectively, is completely not similar (4 points), a small degree of similarity (3 points), not sure (2 points), most similar (1 points), and completely similar (0 points), using scoring judgment of fuzzy degrees. The higher the score, the higher the fuzziness of the image.
The second type of single-choice question (
Figure 10b) was a matching question. On the basis of the original face, four faces after privacy protection were given, and the privacy-protected face that the participant believed belongs to the same person as the original face was selected. The lower the accuracy, the higher the image fuzziness.
A total of 100 questionnaires (groups 1 and 2 each had 10 questions) were distributed, and 98 were collected, resulting in an effective response rate of 98%. The survey results are considered valid. The statistical results for each question in the questionnaire are as follows (
Table 4).
Discussion 2: The statistical results for each question in the questionnaire are as follows (
Table 4). It can be observed that the privacy-preserved face generated by RLLFPR (SNPF) and the face encrypted with AES show similar results. Both methods received high scores in the first group. The similarity values are 3.51 and 3.55, respectively, which reflects that the privacy-preserved images generated by RLLFPR are visually similar to the effect of the AES-encrypted images and that the processed and original images have a low similarity. In the second group, the matching accuracy is close to 25%, indicating that the respondents were close to randomly selecting their answers. Accuracy close to 25% proves that both images could not be matched to the original image, and even if privacy-preserved images were acquired, they could not be matched to a specific person.The results of the two questionnaires show that the privacy-preserved faces generated by RLLFPR have high ambiguity, similar to encryption methods. PartialFace [
35] achieved high similarity compared to the original face in the first questionnaire, and in the second questionnaire, humans recognized privacy-protected faces with accuracies reaching or even exceeding 70–80%. Therefore, it was judged that the PartialFace method for human eye visual blurring was unsatisfactory, and no subsequent blurring test was performed.
Second, the idea of convolution is used to calculate the mean value of the image block, and the variance (Equations (
9)–(
11)) is calculated for all the mean values. s is the block size, which means the mean value of the s × s region size. Sharp images are colorful, and
tends to be larger, while fuzzy images are smaller.
is generally between 0 and 1.
refers to privacy-protected images and
refers to original images:
Fuzziness 2 is shown in Equation (
11); the larger the fuzziness of the image, the higher the value.
The third kind of fuzziness is the method of edge detection (Equations (
12) and (
13)) + SSIM similarity detection (Equation (
14)). Firstly, the edge of the image is detected, and then the SSIM similarity between the privacy-protected image and the original image is calculated. The lower the similarity, the higher the ambiguity. The Laplacian operator is used for edge detection.
The SSIM algorithm [
41] similarity index measures the similarity of images, ranging from 0 to 1. The larger the value, the higher the similarity of the image;
and
represent the mean and standard deviation, respectively.
The image fuzzy degree Fuzziness3 is obtained by subtracting the SSIM similarity from 1 (Equation (
15)), and the higher the value, the higher the fuzzy degree.
4.3.2. Fuzzy Performance of Privacy-Preserving Face Recognition Based on Randomization and Local Feature Learning (RLLFPR)
Table 5 shows the values obtained by different privacy-protection methods after the fuzziness test methods, and the comparison of privacy-protection performance between different methods is added in
Table 5. The Fuzziness1 results are shown in
Table 4. As can be seen from the table, the ambiguity of RLLFPR measured by Fuzziness2 is in the range of 0.80–0.95, which is obviously better than that of noise addition, Arnold transformation, and other methods. AES is the encryption algorithm with ambiguity in the range 0.6–0.9, and the encrypted image cannot be directly recognized. Measured by Fuzziness3, RLLFPR also has a good effect. RLLFPR (0.997) is better than Arnold (0.996), and noise (0.992) is second to AES (0.999). In summary, the privacy-preserving facial fuzzy image generated by RLLFPR is better than the additive noise method and Arnold method and is similar to the AES encryption method. The generated faces in RLLFPR can effectively obscure human vision, providing protection against unauthorized access to facial data stored on servers. RLLFPR eliminates the need for encryption keys, reducing the burden on individuals concerned about their identification. In addition, RLLFPR can directly recognize privacy-protected images.
Discussion 3: In terms of security, RLLFPR is irreversible as well as revocable. Irreversibility: The core of the SN network is composed of a convolutional layer, a batch normalization layer, and an activation-function and maximum-pooling layer. The convolution operation in the convolutional layer and the pooling operation in the maximum-pooling layer are irreversible operations. The activation function uses a nonlinear ReLU activation function, which is also irreversible. Therefore, with the stacked use of irreversible operations, an irreversible privacy-protected image will eventually be produced, and even if the privacy-protected image is obtained, the probability of recovering from it the original face image is close to 0. The ciphertext encrypted by AES cannot be recognized directly, and the Arnold transformed image can be used to recover the original image after many transformations. This all increases the risk of privacy leakage.
Revocability: Since RLLFPR generates blurred images that cannot be recognized by the human eye, it is impossible to match the face images in the database with the real face images even if the database is stolen. The Arnold transform method is not revocable since its images can be restored to the original images. Combining the two aspects of blurriness and security, RLLFPR is more advantageous than the previous methods.