A common design pattern in spoken dialog systems is to reject an input when the recognition confidence score falls below a preset rejection threshold. However, this introduces a potentially non-optimal tradeoff between various types of errors such as misunderstandings and false rejections. In this paper, we propose a data-driven method for determining the relative costs of these errors, and then use these costs to optimize state-specific rejection thresholds. We illustrate the use of this approach with data from a spoken dialog system that handles conference room reservations. The results obtained confirm our intuitions about the costs of the errors, and are consistent with anecdotal evidence gathered throughout the use of the system.
Cite as: Bohus, D., Rudnicky, A.I. (2005) A principled approach for rejection threshold optimization in spoken dialog systems. Proc. Interspeech 2005, 2781-2784, doi: 10.21437/Interspeech.2005-820
@inproceedings{bohus05_interspeech,
author={Dan Bohus and Alexander I. Rudnicky},
title={{A principled approach for rejection threshold optimization in spoken dialog systems}},
year=2005,
booktitle={Proc. Interspeech 2005},
pages={2781--2784},
doi={10.21437/Interspeech.2005-820}
}