Direct error rate minimization of hidden Markov models

Joseph Keshet, Chih-Chieh Cheng, Mark Stoehr, David McAllester

We explore discriminative training of HMM parameters that directly minimizes the expected error rate. In discriminative training one is interested in training a system to minimize a desired error function, like word error rate, phone error rate, or frame error rate. We review a recent method (McAllester, Hazan and Keshet, 2010), which introduces an analytic expression for the gradient of the expected error-rate. The analytic expression leads to a perceptron-like update rule, which is adapted here for training of HMMs in an online fashion. While the proposed method can work with any type of the error function used in speech recognition, we evaluated it on phoneme recognition of TIMIT, when the desired error function used for training was frame error rate. Except for the case of GMM with a single mixture per state, the proposed update rule provides lower error rates, both in terms of frame error rate and phone error rate, than other approaches, including MCE and large margin.

doi: 10.21437/Interspeech.2011-172

Cite as: Keshet, J., Cheng, C.-C., Stoehr, M., McAllester, D. (2011) Direct error rate minimization of hidden Markov models. Proc. Interspeech 2011, 449-452, doi: 10.21437/Interspeech.2011-172

@inproceedings{keshet11_interspeech,
  author={Joseph Keshet and Chih-Chieh Cheng and Mark Stoehr and David McAllester},
  title={{Direct error rate minimization of hidden Markov models}},
  year=2011,
  booktitle={Proc. Interspeech 2011},
  pages={449--452},
  doi={10.21437/Interspeech.2011-172}
}