Speech synthesis applications have become an ubiquity, in navigation systems, digital assistants or as screen or audio book readers. Despite their impact on the acceptability of the systems in which they are embedded, and despite the fact that different applications probably need different types of TTS voices, TTS evaluation is still largely treated as an isolated problem. Even though there is strong agreement among researchers that the mainstream approaches to Text-to-Speech (TTS) evaluation are often insufficient and may even be misleading, there exist few clear-cut suggestions as to (1) how TTS evaluations may be realistically improved on a large scale, and (2) how such improvements may lead to an informed feedback for system developers and, ultimately, better systems relying on TTS. This paper reviews the current state-of-the-art in TTS evaluation, and suggests a novel user-centered research program for this area.
Cite as: Wagner, P., Beskow, J., Betz, S., Edlund, J., Gustafson, J., Eje Henter, G., Le Maguer, S., Malisz, Z., Székely, É., Tånnander, C., Voße, J. (2019) Speech Synthesis Evaluation — State-of-the-Art Assessment and Suggestion for a Novel Research Program. Proc. 10th ISCA Workshop on Speech Synthesis (SSW 10), 105-110, doi: 10.21437/SSW.2019-19
@inproceedings{wagner19_ssw,
author={Petra Wagner and Jonas Beskow and Simon Betz and Jens Edlund and Joakim Gustafson and Gustav {Eje Henter} and Sébastien {Le Maguer} and Zofia Malisz and Éva Székely and Christina Tånnander and Jana Voße},
title={{Speech Synthesis Evaluation — State-of-the-Art Assessment and Suggestion for a Novel Research Program}},
year=2019,
booktitle={Proc. 10th ISCA Workshop on Speech Synthesis (SSW 10)},
pages={105--110},
doi={10.21437/SSW.2019-19}
}