This paper describes the use of a discrete Ergodic Hidden Markov Model (HMM) for automatic acquisition of a language model from a large amount of text data and discusses the possibility of using the HMM to acquire the concept of POS (part-of-speech). A discrete-output Ergodic HMMs has a similar structure to a stochastic network grammar (SNG), so automatic acquisition of an SNG from a large amount of text data is possible by using an Ergodic HMM and the Baum-Welch training procedure. In this model, the HMM state transition probabilities can be interpreted as the SNG transition probabilities. And the HMM state output probabilities will refrect the distributions of POSs (part-of-speeches). The results of experiments using text data show that a high similarity is found between high output probability words and the POS. This means that an Ergodic HMM has the ability to automatically acquire both an SNG and the concept of POS simultaneously from training text data. Experiments were also performed on sentence speech recognition. In these experiments, an Ergodic HMM outperformed the word bigram grammar for text-open data.
Keywords: Language modeling, Stochastic network grammar, Ergodic HMM, Part-of-speech tagger
Cite as: Murakami, J., Yamatomo, H., Sagayama, S. (1993) The possibility for acquisition of statistical network grammar using ergodic HMM. Proc. 3rd European Conference on Speech Communication and Technology (Eurospeech 1993), 1327-1330, doi: 10.21437/Eurospeech.1993-310
@inproceedings{murakami93_eurospeech,
author={Jin'ichi Murakami and Hiroki Yamatomo and Shigeki Sagayama},
title={{The possibility for acquisition of statistical network grammar using ergodic HMM}},
year=1993,
booktitle={Proc. 3rd European Conference on Speech Communication and Technology (Eurospeech 1993)},
pages={1327--1330},
doi={10.21437/Eurospeech.1993-310}
}