3D face alignment and face reconstruction based on image sequence

Yao Wei; Biao Qiao; Huabin Wang; Mengxin Zhang; Shijun Liu; Liang Tao

doi:10.1117/12.2644477

12 October 2022 3D face alignment and face reconstruction based on image sequence

Yao Wei, Biao Qiao, Huabin Wang, Mengxin Zhang, Shijun Liu, Liang Tao

Proceedings Volume 12342, Fourteenth International Conference on Digital Image Processing (ICDIP 2022); 1234216 (2022) https://doi.org/10.1117/12.2644477
Event: Fourteenth International Conference on Digital Image Processing (ICDIP 2022), 2022, Wuhan, China

Abstract

Existing 3D face alignment and face reconstruction methods mainly focus on the accuracy of the model. When the existing methods are applied to dynamic videos, the stability and accuracy are significantly reduced. To overcome this problem, we propose a novel regression framework that strikes a balance between accuracy and stability. First, on the basis of lightweight backbone, encoder-decoder structure is used to jointly learn expression details and detailed 3D face from video images to recover shape details and their relationship to facial expression, and dynamic regression of a small number of 3D face parameters, effectively improve the speed and accuracy. Secondly, in order to further improve the stability of face landmarks in video, a jitter loss function of multi-frame image joint learning is proposed to strengthen the correlation between frames and face landmarks in video, and reduce the difference amplitude of face landmarks between adjacent frames to reduce the jitter of face landmarks. Experiments on several challenging datasets verify the effectiveness of our method.

Citation Download Citation

Yao Wei, Biao Qiao, Huabin Wang, Mengxin Zhang, Shijun Liu, and Liang Tao "3D face alignment and face reconstruction based on image sequence", Proc. SPIE 12342, Fourteenth International Conference on Digital Image Processing (ICDIP 2022), 1234216 (12 October 2022); https://doi.org/10.1117/12.2644477

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

3D modeling

3D image processing

Video

3D image reconstruction

Facial recognition systems

3D video compression

Data modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years