Abstract
Face recognition techniques are widely used in many applications, such as automatic detection of crime scenes from surveillance cameras for public safety. In these real cases, the pose and illumination variances between two matching faces have a big influence on the identification performance. Handling pose changes is an especially challenging task. In this paper, we propose the learning warps based similarity method to deal with face recognition across the pose problem. Warps are learned between two patches from probe faces and gallery faces using the Lucas-Kanade algorithm. Based on these warps, a frontal face registered in the gallery is transformed into a series of non-frontal viewpoints, which enables non-frontal probe face matching with the frontal gallery face. Scale-invariant feature transform (SIFT) keypoints (interest points) are detected from the generated viewpoints and matched with the probe faces. Moreover, based on the learned warps, the probability likelihood is used to calculate the probability of two faces being the same subject. Finally, a hybrid similarity combining the number of matching keypoints and the probability likelihood is proposed to describe the similarity between a gallery face and a probe face. Experimental results show that our proposed method achieves better recognition accuracy than other algorithms it was compared to, especially when the pose difference is within 40 degrees.
Similar content being viewed by others
References
Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041
Arashloo SR, Kittler J (2011) Energy normalization for pose-invariant face recognition based on MRF model image matching. IEEE Trans Pattern Anal Mach Intell 33(6):1274–1280
Ashraf AB, Lucey S, Chen T (2008) Learning patch correspondences for improved viewpoint invariant face recognition. Comput Vision Pattern Recogn. CVPR 2008. IEEE Conference on, 2008. IEEE: 1–8
Asthana A, Gedeon T, Goecke R, Sanderson C (2009) Learning-based face synthesis for pose-robust recognition from single image. British Mach Vision Conf, 2009. British Machine Vision Assoc Soc Pattern Recogn: 1–10
Asthana A, Marks TK, Jones MJ, Tieu KH, Rohith M (2011) Fully automatic pose-invariant face recognition via 3D pose normalization. 2011 Int Conf Comput Vision. IEEE: 937–944
Baker S, Matthews I (2004) Lucas-kanade 20 years on: a unifying framework. Int J Comput Vis 56(3):221–255
Blanz V, Vetter T (2003) Face recognition based on fitting a 3D morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074
Chai X, Shan S, Chen X, Gao W (2007) Locally linear regression for pose-invariant face recognition. IEEE Trans Image Process 16(7):1716–1725
Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. Proc IEEE Conf Comput Vision Pattern Recogn: 3025–3032
Gao Y, Lee HJ (2014) Pose unconstrained face recognition based on SIFT and alignment error. Audio, Language Image Process (ICALIP), 2014 Int Conf. IEEE: 277–281
Gao Y, Lee HJ (2015) Cross-pose face recognition based on multiple virtual views and alignment error. Pattern Recogn Lett 65:170–176
Grgic M, Delac K, Grgic S (2011) SCface–surveillance cameras face database. Multimed Tools Appl 51(3):863–879
Ho HT, Chellappa R (2013) Pose-invariant face recognition using Markov random fields. IEEE Trans Image Process 22(4):1573–1584
Hua G, Yang M-H, Learned-Miller E, Ma Y, Turk M, Kriegman DJ, Huang TS (2011) Introduction to the special section on real-world face recognition. IEEE Trans Pattern Anal Mach Intell 33(10):1921–1924
Jiang Y, Wang B, Zhou Y, Li W, Liao Q (2015) Patterns of Weber magnitude and orientation for uncontrolled face representation and recognition. Neurocomputing 165:190–201
Li S, Liu X, Chai X, Zhang H, Lao S, Shan S (2012) Morphable displacement field based image matching for face recognition across pose. Europ Conf Comput Vision. Springer: 102–115
Li A, Shan S, Gao W (2012) Coupled bias–variance tradeoff for cross-pose face recognition. IEEE Trans Image Process 21(1):305–315
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
Mikolaczyk K, Schmid C Scale and affine invariant interest points detectors. International
Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Van Gool L (2005) A comparison of affine region detectors. Int J Comput Vis 65(1-2):43–72
Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. Europ Conf Comput Vision. Springer: 504–513
Morel J-M, Yu G (2009) ASIFT: a new framework for fully affine invariant image comparison. SIAM J Imag Sci 2(2):438–469
Phillips J (2009) Video challenge problem multiple biometric grand challenge preliminary: results of version 2. Nat Inst Standards Technol
Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16(5):295–306
Prince SJ, Elder JH, Warrell J, Felisberti FM (2008) Tied factor analysis for face recognition across large pose differences. IEEE Trans Pattern Anal Mach Intell 30(6):970–984
Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. Proc IEEE Conf ComputVision Pattern Recogn: 1701–1708
Vu N-S, Caplier A (2012) Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans Image Process 21(3):1352–1365
Wolf L, Hassner T, Taigman Y (2011) Effective unconstrained face recognition by combining multiple descriptors and learned background statistics. IEEE Trans Pattern Anal Mach Intell 33(10):1978–1990
Zhang W, Shan S, Gao W, Chen X, Zhang H (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. Tenth IEEE Int Conf Comput Vision (ICCV’05) IEEE 1:786–791
Zhang Y, Shao M, Wong EK, Fu Y (2013) Random faces guided sparse many-to-one encoder for pose-invariant face recognition. Proc IEEE Int Conf Comput Vision: 2416–2423
Zhu Z, Luo P, Wang X, Tang X (2013) Deep learning identity-preserving face space. Proc IEEE Int Conf Comput Vision: 113–120
Acknowledgements
This work was supported by the Brain Korea 21 PLUS Project and the State Scholarship Fund organized by the China Scholarship Council. This work was also supported by the Business for Academic-Industrial Cooperative establishments that were funded by the Korea Small and Medium Business Administration in 2015 (Grants No. C0221114). This research was also supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2016-R0992-16-1023) supervised by the IITP (Institute for Information & communications Technology Promotion). This work was also supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (GR 2016R1D1A3B03931911). This paper was also supported by the project of local colleges’ and universities’ capacity construction of Science and Technology Commission in Shanghai (No. 15590501300) and by the National Natural Science Foundation of China (No. 61461021).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gao, Y., Lee, H.J. Learning warps based similarity for pose-unconstrained face recognition. Multimed Tools Appl 77, 1927–1942 (2018). https://doi.org/10.1007/s11042-017-4359-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-4359-9