Skip to main content
SpringerLink
Log in

Point cloud based deep convolutional neural network for 3D face recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face recognition is a challenging task as it has to deal with several issues such as illumination orientation and variability among the different faces. Previous works have shown that 3D face is a robust biometric trait and is less sensitive to light and pose variations. Also due to availability of inexpensive sensors and new 3D data acquisition techniques it has become easy to capture 3D data. A 3D depth image of a face is found to be rich in information and biometric recognition performance can be enhanced by using 3D face data along with convolutional neural network. However the shortcoming of this approach is the conversion of 3D data to lower dimensions (depth image) which suffer from loss of geometric information and the network becomes computationally expensive. In this work we endeavor to apply deep learning method for 3D face recognition and propose a deep convolutional neural network based on PointNet architecture which consumes point cloud directly as input and siamese network for similarity learning. Further we propose a solution to the issue of a limited database by applying data augmentation at the point cloud level. Our proposed technique shows encouraging performance on Bosphorus and IIT Indore 3D face databases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abate AF, Nappi M, Riccio D, Sabatino G (2007) 2D and 3D face recognition: A survey. Pattern Recogn Lett 28(14):1885–1906

  2. Abbad A, Abbad K, Tairi H (2018) 3D face recognition: Multi-scale strategy based on geometric and local descriptors. Comput Electr Eng 70:525–537

  3. Ahmed E, Saint A, Shabayek AER, Cherenkova K, Das R, Gusev G, Aouada D, Ottersten B (2018) Deep learning advances on different 3D data representations: A survey. arXiv:180801462

  4. Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: Proceedings of European Conference on Computer Vision pp 469–481

  5. Berretti S, Werghi N, Del Bimbo A, Pala P (2013) Matching 3D face scans using interest points and local histogram descriptors. Comput Graph 37(5):509–525

    Article  Google Scholar 

  6. Bhele S G, Mankar V (2012) A review paper on face recognition techniques. Int J Adv Res Comput Eng Technol 1(8):339–346

    Google Scholar 

  7. Blanz V, Vetter T et al (1999) A morphable model for the synthesis of 3D faces. In: Proceedings of SIGGRAPH pp 187–194

  8. Bowyer K W, Chang K, Flynn P (2006) A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Comput Vis Image Underst 101(1):1–15

    Article  Google Scholar 

  9. Bromley J, Guyon I, LeCun Y, Säckinger E, Shah R (1994) Signature verification using a “siamese” time delay neural network. In: Proceedings of Advances in Neural Information Processing Systems pp 737–744

  10. Cao C, Weng Y, Zhou S, Tong Y, Zhou K (2013) FaceWarehouse: A 3D facial expression database for visual computing. IEEE Trans Visual Comput Graph 20(3):413–425

  11. Deng J, Guo J, Xue N, Zafeiriou S (2019) ArcFace: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp 4690–4699

  12. Drira H, Amor BB, Srivastava A, Daoudi M, Slama R (2013) 3D face recognition under expressions occlusions and pose variations. IEEE Trans Pattern Anal Mach Intell 35(9):2270–2283

  13. Dutta K, Bhattacharjee D, Nasipuri M, Poddar A (2019) 3D Face Recognition Based on Volumetric Representation of Range Image pp 175–189

  14. Garcia-Garcia A (2016) 3D object recognition with convolutional neural networks (phd thesis)

  15. Garcia-Garcia A, Gomez-Donoso F, Garcia-Rodriguez J, Orts-Escolano S, Cazorla M, Azorin-Lopez J (2016) PointNet: A 3D convolutional neural network for real-time object class recognition. In: Proceedings of International Joint Conference on Neural Networks pp 1578–1584

  16. Gilani S Z, Mian A, Shafait F, Reid I (2017) Dense 3D face correspondence. IEEE Trans Pattern Anal Mach Intell 40(7):1584–1598

    Article  Google Scholar 

  17. Gupta S, Markey M K, Bovik A C (2010) Anthropometric 3D face recognition. Int J Comput Vis 90(3):331–349

    Article  Google Scholar 

  18. Hassaballah M, Aly S (2015) Face recognition: challenges achievements and future directions. IET Comput Vis 9(4):614–626

    Article  Google Scholar 

  19. He Y, Liang B, Yang J, Li S, He J (2017) An iterative closest points algorithm for registration of 3D laser scanner point clouds with geometric features. Sensors 17 (8):1862

    Article  Google Scholar 

  20. Huang G B, Mattar M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report. University of Massachusetts

  21. Kim D, Hernandez M, Choi J, Medioni G (2017) Deep 3D face identification. In: Proc. of IEEE International Joint Conference on Biometrics pp 133–142

  22. Kingkan C, Owoyemi J, Hashimoto K (2018) Point attention network for gesture recognition using point cloud data. In: Proceedings of British Machine Vision Conference pp 118

  23. Klare BF, Klein B, Taborsky E, Blanton A, Cheney J, Allen K, Grother P, Mah A, Jain AK (2015) Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition pp 1931–1939

  24. Koch G, Zemel R, Salakhutdinov R (2015) Siamese neural networks for one-shot image recognition. In: Proceedings of ICML Deep Learning Workshop vol 2

  25. Le T, Duan Y (2018) PointGrid: A deep network for 3D shape understanding. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition pp 9204–9214

  26. Lei Y, Guo Y, Hayat M, Bennamoun M, Zhou X (2016) A two-phase weighted collaborative representation for 3D partial face recognition with single sample. Pattern Recogn 52:218–237

    Article  Google Scholar 

  27. Leo MJ, Suchitra S (2018) SVM based expression-invariant 3D face recognition system. Procedia Comput Sci 143:619–625

  28. Li H, Huang D, Morvan JM, Chen L, Wang Y (2014a) Expression-robust 3D face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns. Neurocomputing 133:179–193

  29. Li H, Huang D, Morvan JM, Wang Y, Chen L (2014b) Towards 3D face recognition in the real: A registration-free approach using fine-grained matching of 3D keypoint descriptors. Int J Comput Vis 113:128–142

  30. Maturana D, Scherer S (2015) VoxNet: A 3D convolutional neural network for real-time object recognition. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems pp 922–928

  31. Mian A, Bennamoun M, Owens R (2007) An efficient multimodal 2D-3D hybrid approach to automatic face recognition. IEEE Trans Pattern Anal Mach Intell 29 (11):1927–1943

    Article  Google Scholar 

  32. Mian A S, Bennamoun M, Owens R (2008) Keypoint detection and local feature matching for textured 3D face recognition. Int J Comput Vis 79(1):1–12

    Article  Google Scholar 

  33. Ocegueda O, Passalis G, Theoharis T, Shah SK, Kakadiaris IA (2011) UR3D-C: Linear dimensionality reduction for efficient 3D face recognition. In: Proceedings of International Joint Conference on Biometrics pp 1–6

  34. Parkhi OM, Vedaldi A, Zisserman A et al (2015) Deep face recognition. In: Proc. of British Machine Vision Conference pp 6

  35. Patil H, Kothari A, Bhurchandi K (2015) 3D face recognition: features databases algorithms and challenges. Artif Intell Rev 44(3):393–441

  36. Paysan P, Knothe R, Amberg B, Romdhani S, Vetter T (2009) A 3D face model for pose and illumination invariant face recognition. In: Proceedings of Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance pp 296–301

  37. Qi CR, Su H, Mo K, Guibas LJ (2017a) PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition pp 652–660

  38. Qi CR, Yi L, Su H, Guibas LJ (2017b) PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proc. of Advances in Neural Information Processing Systems pp 5099–5108

  39. Rahim R, Afriliansyah T, Winata H, Nofriansyah D, Aryza S, et al. (2018) Research of face recognition with fisher linear discriminant. In: Proceedings of IOP Conference Series: Materials Science and Engineering pp 012–037

  40. Savran A, Alyüz N, Dibeklioğlu H, Çeliktutan O, Gökberk B, Sankur B, Akarun L (2008) Bosphorus database for 3D face analysis. In: Proc. of European Workshop on Biometrics and Identity Management pp 47–56

  41. Schroff F, Kalenichenko D, Philbin J (2015) FaceNet: A unified embedding for face recognition and clustering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition pp 815–823

  42. Sharma PB, Goyani MM (2012) 3D face recognition techniques-a review. Int J Eng Res Appl 2(1):787–793

  43. Soltanpour S, Wu Q M J (2019) Weighted extreme sparse classifier and local derivative pattern for 3D face recognition. IEEE Trans Image Process 28(6):3020–3033

    Article  MathSciNet  Google Scholar 

  44. Soltanpour S, Boufama B, Wu Q J (2017) A survey of local feature methods for 3D face recognition. Pattern Recogn 72:391–406

    Article  Google Scholar 

  45. Taigman Y, Yang M, Ranzato M, Wolf L (2014) DeepFace: Closing the gap to human-level performance in face verification. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition pp 1701–1708

  46. Tan Y, Lin H, Xiao Z, Ding S, Chao H (2018) Face recognition from sequential sparse 3D data via deep registration. arXiv:181009658

  47. ter Haar F B, Veltkamp R C (2010) Expression modeling for expression-invariant face recognition. Comput Graph 34(3):231–241

    Article  Google Scholar 

  48. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition pp 586–591

  49. Wang F, Cheng J, Liu W, Liu H (2018) Additive margin softmax for face verification. IEEE Signal Process Lett 25(7):926–930

    Article  Google Scholar 

  50. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3D ShapeNets: A deep representation for volumetric shapes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition pp 1912–1920

  51. Zhou H, Mian A, Wei L, Creighton D, Hossny M, Nahavandi S (2014) Recent advances on singlemodal and multimodal face recognition: a survey. IEEE Trans Human-Mach Syst 44(6):701–716

    Article  Google Scholar 

  52. Zhou S, Xiao S (2018) 3D face recognition: a survey. Hum-Centric Comput Inf Sci 8(1):35

  53. Zulqarnain Gilani S, Mian A (2018) Learning from millions of 3D scans for large-scale 3D face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition pp 1896–1905

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology Indore, Indore, 453552, India

    Anagha R. Bhople, Akhilesh M. Shrivastava & Surya Prakash

Authors
  1. Anagha R. Bhople
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Akhilesh M. Shrivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Surya Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anagha R. Bhople.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhople, A.R., Shrivastava, A.M. & Prakash, S. Point cloud based deep convolutional neural network for 3D face recognition. Multimed Tools Appl 80, 30237–30259 (2021). https://doi.org/10.1007/s11042-020-09008-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09008-z

Keywords

Search

Navigation