Elsevier

Neurocomputing

Volume 333, 14 March 2019, Pages 339-350
Neurocomputing

Multiview discriminative marginal metric learning for makeup face verification

https://doi.org/10.1016/j.neucom.2018.12.003Get rights and content
Under a Creative Commons license
open access

Abstract

Makeup face verification in the wild is an important research problem for its popularization in real-world. However, little effort has been made to tackle it in computer vision. In this research, we first build a new database, i.e., Facial Beauty Database (FBD), which contains paired facial images of 8933 subjects without and with makeup in different real-world scenarios. To the best of our knowledge, FBD is the largest makeup face database to date compared with existing databases for facial makeup research. Moreover, we propose a new discriminative marginal metric learning (DMML) algorithm to deal with this problem in the wild. Inspired by the fact that interclass marginal faces are usually more discriminative than interclass nonmarginal faces in learning the discriminative metric space, we use the interclass marginal faces to depict the discriminative information. Simultaneously, we wish that those interclass marginal faces without makeup relations are separated from each other as far as possible, so that more discriminative information between facial images without and with makeup can be exploited for verification. Furthermore, since multiple features could provide comprehensive information in describing the facial representations from diverse points of view and extract more informative cues from facial images, we also introduce a multiview discriminative marginal metric learning (MDMML) algorithm by effectively learning a robust metric space such that multiple features from different points of view can be integrated to effectively enhance the performance of makeup face verification. Experimental results on two real-world makeup face databases are utilized to show the effectiveness of our method and the possibility of verifying the makeup relations from facial images in real-world.

Keywords

Makeup face verification
Multiview learning
Distance metric learning
Marginal samples
Face verification

Cited by (0)

Lining Zhang (S’11-M’14) received the B.Eng. and M.Eng. degrees from Xidian University, Xi’an, China, and the Ph.D. degree from Nanyang Technological University, Singapore. He is a Senior Scientist with the Institute for Infocomm Research, Singapore. He has published extensively in top venues, such as the IEEE Transactions on Image Processing, the IEEE Transactions on Circuits and Systems for Video Technology, and the IEEE Transactions on Cybernetics. His research interests include artificial intelligence, computer vision, video/image processing, medical image analysis, machine learning, and computational intelligence. He is a member of the IEEE.

Hubert P. H. Shum is an Associate Professor (Reader) at Northumbria University. Before joining the university, he worked as a Lecturer at the University of Worcester, a post-doctoral researcher at RIKEN Japan, as well as a research assistant at the City University of Hong Kong. He received his Ph.D. degree from the School of Informatics at the University of Edinburgh, as well as his MSc and BEng degrees from the City University of Hong Kong. His research interests include character animation, machine learning, human motion analysis and computer vision.

Li Liu received the B.Eng. degree in electronic information engineering from Xian Jiaotong University, Xian, China, in 2011, and the Ph.D. degree from the Department of Electronic and Electrical Engineering, The University of Sheffield, Sheffield, U.K., in 2014. He is currently with the Inception Institute of Artificial Intelligence, Abu Dhabi, United Arab Emirates. His current research interests include computer vision, machine learning, and data mining.

Guodong Guo is an Associate Professor in the Department of Computer Science and Electrical Engineering at West Virginia University. He is also the Director and Founder of the Computer Vision Laboratory (CVL) at WVU, and affiliated with the Center for Identification Technology Research (CITeR), a unique national Biometric research center funded by NSF. Dr. Guo received his Bachelor degree from Tsinghua University in Beijing, and Ph.D. In Computer Science at the Univsersity of Wisconsin – Madison.

Ling Shao (M ’09-SM ’10) is a professor with the School of Computing Sciences at the University of East Anglia, Norwich, UK. Previously, he was a professor (2014–2016) with Northumbria University, a senior lecturer (2009–2014) with the University of Sheffield and a senior scientist (2005–2009) with Philips Research, The Netherlands. His research interests include computer vision, image/video processing and machine learning. He is an associate editor of IEEE Transactions on Image Processing, IEEE Transactions on Neural Networks and Learning Systems and several other journals. He is a Fellow of the British Computer Society and the Institution of Engineering and Technology. He is a senior member of the IEEE.