In this paper we present a novel technique to automatically synthesise eye blinking from a speech signal. Animating the eyes of a talking head is important as they are a major focus of attention during interaction. The developed system predicts eye blinks from the speech signal and generates animation trajectories automatically employing a “Trajectory Hidden Markov Model”. The evaluation of the recognition performance showed that the timing of blinking can be predicted from speech with an F-score value upwards of 52%, which is well above chance. Additionally, a preliminary perceptual evaluation was conducted, that confirmed that adding eye blinking significantly improves the perception the character. Finally it showed that the speech synchronised synthesised blinks outperform random blinking in naturalness ratings.
Cite as: Dziemianko, M., Hofer, G., Shimodaira, H. (2009) HMM-based automatic eye-blink synthesis from speech. Proc. Interspeech 2009, 1799-1802, doi: 10.21437/Interspeech.2009-153
@inproceedings{dziemianko09_interspeech,
author={Michal Dziemianko and Gregor Hofer and Hiroshi Shimodaira},
title={{HMM-based automatic eye-blink synthesis from speech}},
year=2009,
booktitle={Proc. Interspeech 2009},
pages={1799--1802},
doi={10.21437/Interspeech.2009-153}
}