Skip to main content
SpringerLink
Log in

Privacy-Protective-GAN for Privacy Preserving Face De-Identification

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Face de-identification has become increasingly important as the image sources are explosively growing and easily accessible. The advance of new face recognition techniques also arises people’s concern regarding the privacy leakage. The mainstream pipelines of face de-identification are mostly based on the k-same framework, which bears critiques of low effectiveness and poor visual quality. In this paper, we propose a new framework called Privacy-Protective-GAN (PP-GAN) that adapts GAN (generative adversarial network) with novel verificator and regulator modules specially designed for the face de-identification problem to ensure generating de-identified output with retained structure similarity according to a single input. We evaluate the proposed approach in terms of privacy protection, utility preservation, and structure similarity. Our approach not only outperforms existing face de-identification techniques but also provides a practical framework of adapting GAN with priors of domain knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Newton E M, Sweeney L, Malin B. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(2): 232-243.

    Article  Google Scholar 

  2. Ribaric S, Ariyaeeinia A, Pavesic N. De-identification for privacy protection in multimedia content: A survey. Signal Processing: Image Communication, 2016, 47: 131-151.

    Google Scholar 

  3. Gross R, Airoldi E, Malin B, Sweeney L. Integrating utility into face de-identification. In Proc. the 5th International Conference on Privacy Enhancing Technologies, May 2005, pp.227-242.

  4. Gross R, Sweeney L, de la Torre F, Baker S. Model-based face de-identification. In Proc. the 2006 Conference on Computer Vision and Pattern Recognition, Jun. 2006, Article No. 161.

  5. Gross R, Sweeney L, de La Torre F, Baker S. Semisupervised learning of multi-factor models for face deidentification. In Proc. the 2008 Conference on Computer Vision and Pattern Recognition, Jun. 2008, Article No. 10.

  6. Matthews I, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60(2): 135-164.

    Article  Google Scholar 

  7. Du L, Yi M, Blasch E, Ling H. GARP-face: Balancing privacy protection and utility preservation in face deidentification. In Proc. IEEE International Joint Conference on Biometrics, Sept. 2014, Article No. 67.

  8. Jourabloo A, Yin X, Liu X. Attribute preserved face deidentification. In Proc. IEEE International Conference on Biometrics, May 2015, pp.278-285.

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. Annual Conference on Neural Information Processing Systems, Dec. 2014, pp.2672-2680.

  10. Isola P, Zhu J Y, Zhou T, Efros A A. Image-to-image translation with conditional adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.5967-5976.

  11. Zhu J Y, Park T, Isola P, Efros A A. Unpaired imageto-image translation using cycle-consistent adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.2242-2251.

  12. Hadsell R, Chopra S, LeCun Y. Dimensionality reduction by learning an invariant mapping. In Proc. the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2006, pp.1735-1742.

  13. Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.

    Article  Google Scholar 

  14. Schroff F, Kalenichenko D, Philbin J. FaceNet: A unified embedding for face recognition and clustering. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.815-823.

  15. Zhang K, Zhang Z, Li Z, Qiao Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.

    Article  Google Scholar 

  16. Sun Q, Tewari A, Xu W, Fritz M, Theobalt C, Schiele B. A hybrid model for identity obfuscation by face replacement. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.570-586.

  17. Chen D, Chang Y, Yan R, Yang J. Tools for protecting the privacy of specific individuals in video. EURASIP Journal on Advances in Signal Processing, 2007, 2007(1): Article No. 075427.

  18. Wilber M J, Boult T E. Secure remote matching with privacy: Scrambled support vector vaulted verification (S2V3). In Proc. IEEE Workshop on Applications of Computer Vision, Jan. 2012, pp.169-176.

  19. Chan A B, Liang Z S, Vasconcelos N. Privacy preserving crowd monitoring: Counting people without people models or tracking. In Proc. the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, Article No. 229.

  20. Sun Q, Ma L, Oh S J, van Gool L, Schiele B, Fritz M. Natural and effective obfuscation by head inpainting. In Proc. the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp.5050-5059.

  21. Sun Q, Schiele B, Fritz M. A domain based approach to social relation recognition. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.435-444.

  22. Mirjalili V, Raschka S, Namboodiri A, Ross A. Semiadversarial networks: Convolutional autoencoders for imparting privacy to face images. In Proc. the 2018 International Conference on Biometrics, Feb. 2018 pp.82-89.

  23. Ribaric S, Ariyaeeinia A, Pavesic N. De-identification for privacy protection in multimedia content: A survey. Signal Processing: Image Communication, 2016, 47: 131-151.

    Google Scholar 

  24. Gross R, Sweeney L, Cohn J, de la Torre F, Baker S. Face de-identification. In Protecting Privacy in Video Surveillance, Senior A (ed.), Springer, 2009, pp.129-146.

  25. Frome A, Cheung G, Abdulkader A, Zennaro M, Wu B, Bissacco A, Adam H, Neven H, Vincent L. Large-scale privacy protection in Google street view. In Proc. the 12th International Conference on Computer Vision, Sept. 2009, pp.2373-2380.

  26. Sweeney L. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 2002, 10(5): 557-570.

    Article  MathSciNet  MATH  Google Scholar 

  27. Brkic K, Sikiric I, Hrkac T, Kalafatic Z. I know that person: Generative full body and face de-identification of people. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Jul. 2017, pp.1319-1328.

  28. Meden B, Mallı R C, Fabijan S, Ekenel H K, Struc V, Peer P. Face deidentification with generative deep neural networks. IET Signal Processing, 2017, 11(9): 1046-54.

    Article  Google Scholar 

  29. Meden B, Ziga E, Struc V, Peer P. k-same-net: k-anonymity with generative deep neural networks for face deidentification. Entropy, 2018, 20(1): Article No. 60.

  30. Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. the 32nd International Conference on Machine Learning, Jul. 2015, pp.448-456.

  31. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2015 pp.234-241.

  32. Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In Proc. the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2005, pp.539-546.

  33. Sun Y, Chen Y, Wang X, Tang X. Deep learning face representation by joint identification-verification. In Proc. the 2014 Annual Conference on Neural Information Processing Systems, Dec. 2014, pp.1988-1996.

  34. Schroff F, Kalenichenko D, Philbin J. FaceNet: A unified embedding for face recognition and clustering. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.815-823.

  35. Parkhi O M, Vedaldi A, Zisserman A. Deep face recognition. In Proc. the 26th British Machine Vision Conference, Sept. 2015, Article No. 6.

  36. Wu X, He R, Sun Z, Tan T. A light CNN for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security, 2018, 13(11): 2884-2896.

    Article  Google Scholar 

  37. Lin M, Chen Q, Yan S. Network in network. arXiv:1312.4400, 2013. https://arxiv.org/pdf/131-2.4400.pdf, Sept. 2018.

  38. Du L, Ling H. Preservative license plate de-identification for privacy protection. In Proc. the 2011 International Conference on Document Analysis and Recognition, Sept. 2011, pp.468-472.

  39. Ricanek K, Tesafaye T. MORPH: A longitudinal image database of normal adult age-progression. In Proc. the 7th International Conference on Automatic Face and Gesture Recognition, Apr. 2006, pp.341-345.

  40. Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2015. https://arxiv.org/pdf/1412.6980.pdf, Sept. 2018.

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Sciences, Temple University, Philadelphia, 19122, USA

    Yifan Wu, Fan Yang & Haibin Ling

  2. School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China

    Yong Xu

Authors
  1. Yifan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haibin Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yifan Wu.

Electronic supplementary material

ESM 1

(PDF 643 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Yang, F., Xu, Y. et al. Privacy-Protective-GAN for Privacy Preserving Face De-Identification. J. Comput. Sci. Technol. 34, 47–60 (2019). https://doi.org/10.1007/s11390-019-1898-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-019-1898-8

Keywords

Search

Navigation