Abstract
In this paper, we present a new search algorithm for sequential labeling tasks based on the conditional Markov models (CMMs) frameworks. Unlike conventional beam search, our method traverses all possible incoming arcs and also considers the “local best” so-far of each previous node. Furthermore, we propose two heuristics to fit the efficiency requirement. To demonstrate the effect of our method, six variant and large-scale sequential labeling tasks were conducted in the experiment. In addition, we compare our method to Viterbi and Beam search approaches. The experimental results show that our method yields not only substantial improvement in runtime efficiency, but also slightly better accuracy. In short, our method achieves 94.49 F(β) rate in the well-known CoNLL-2000 chunking task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Charniak, E.: A maximum-entropy-inspired parser. In: 6th Applied Natural Language, Processing Conference, pp. 132–139 (2001)
Collins, M.: Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms. In: Empirical Methods in Natural Language Processing, pp. 1–8 (2002)
Daumé, H., Marcu, D.: Learning as search optimization: approximate large margin methods for structured prediction. In: 12th International Conference on Machine Learning, pp. 169–176 (2005)
Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: 15th International Conference on Machine Learning, pp. 408–415 (2008)
Giménez, J., Márquez, L.: SVMTool: A general POS tagger generator based on support vector machines. In: 4th International Conference on Language Resources and Evaluation, pp. 43–46 (2004)
Kazama, J., Tsujii, J.: Evaluation and extension of maximum entropy models with inequality constraints. In: Empirical methods in natural language processing, pp. 137–144 (2003)
Kudo, T., Matsumoto, Y.: Chunking with support vector machines. In: North American Chapter of the Association for Computational Linguistics on Language Technologies, pp. 192–199 (2001)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: 8th International Conference on Machine Learning, pp. 282–289 (2001)
McCallum, A., Freitag, D., Pereira, F.: Maximum entropy Markov models for information extraction and segmentation. In: 7th International Conference on Machine Learning, pp. 591–598 (2000)
Ng, H.T., Low, J.K.: Chinese part-of-speech tagging. one-at-a-time or all-at-once? word-based or character-based? In: Empirical methods in natural language processing, pp. 277–284 (2004)
Platt, J.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in Large Margin Classifiers (1999)
Ratnaparkhi, A.: A Maximum Entropy Model for Part-Of-Speech Tagging. In: Empirical Methods in Natural Language Processing, pp. 133–142 (1996)
Suzuki, J., Fujino, A., Isozaki, H.: Semi-supervised structural output learning based on a hybrid generative and discriminative approach. In: Empirical Methods in Natural Language Processing, pp. 791–800 (2007)
Suzuki, J., Isozaki, H.: Semi-supervised sequential labeling and segmentation using giga-word scale unlabeled data. In: 46th Annual Meeting of the Association of Computational Linguistics, pp. 665–673 (2008)
Takeuchi, K., Collier, N.: Use of support vector machines in extended named entity recognition. In 6th Conference on Computational Natural Language Learning, pp. 119-125 (2002)
Taskar, B., Guestrin, C., Koller, D.: Max-margin Markov networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 25–32 (2003)
Tjong Kim Sang, E.F., Buchholz, S.: Introduction to the CoNLL-2000 shared task: chunking. In: 4th Conference on Computational Natural Language Learning, pp. 127–132 (2000)
Yang, Y., Liu, X.: A re-examination of text categorization methods. In: 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 42–49 (1999)
Wu, Y.C., Lee, Y.S., Yang, J.C.: Robust and efficient multiclass SVM models for phrase pattern recognition, Pattern Recognition, vol. Pattern Recognition 41(9), 2874–2889 (2008)
Zhang, Y., Clark, S.: Chinese segmentation with a word-based perceptron algorithm. In: 45th Annual Meeting of the Association of Computational Linguistics, pp. 840–847 (2007)
Zhao, H., Kit, C.: Incorporating global information into supervised learning for Chinese word segmentation. In: 10th Conference of the Pacific Association for Computational Linguistics, pp. 66–74 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, YC., Lee, YS., Yang, JC., Yen, SJ. (2010). An Integrated Deterministic and Nondeterministic Inference Algorithm for Sequential Labeling. In: Cheng, PJ., Kan, MY., Lam, W., Nakov, P. (eds) Information Retrieval Technology. AIRS 2010. Lecture Notes in Computer Science, vol 6458. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17187-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-17187-1_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17186-4
Online ISBN: 978-3-642-17187-1
eBook Packages: Computer ScienceComputer Science (R0)