Automated Analysis of Head Pose, Facial Expression and Affect

Abstract

Automated analysis of facial expressions is a well-investigated research area in the field of computer vision, with impending applications such as human-computer interaction (HCI). The conducted work proposes new methods for the automated evaluation of facial expression in image sequences of color and depth data. In particular, we present the main components of our system, i.e. accurate estimation of the observed person’s head pose, followed by facial feature extraction and, third, by classification. Through the application of dimensional affect models, we overcome the use of strict categories, i.e. basic emotions, which are focused on by most state-of-the-art facial expression recognition techniques. This is of importance as in most HCI applications classical basic emotions are only occurring sparsely, and hence are often inadequate to guide the dialog with the user. To resolve this issue we suggest the mapping to the so-called “Circumplex model of affect”, which enables us to determine the current affective state of the user, which can then be used in the interaction. Especially, the output of the proposed machine vision-based recognition method gives insight to the observed person’s arousal and valence states. In this chapter, we give comprehensive information on the approach and experimental evaluation.