Abstract
Text categorisation in commercial application poses several limiting constraints on the technology solutions to be employed. This paper describes how a method with some potential improvements is evaluated for practical purposes and argues for a richer and more expressive evaluation procedure. In this paper one such method is exemplified by a precision-recall matrix which sacrifices convenience for expressiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barak, L., Dagan, I., Shnarch, E.: Text categorization from category name via lexical reference. In: Proceedings of Human Language Technologies: Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 33–36. Association for Computational Linguistics (2009)
Gliozzo, A., Strapparava, C., Dagan, I.: Improving text categorization bootstrapping via unsupervised learning. ACM Trans. Speech Lang. Process. (TSLP) 6(1), 1 (2009)
Ko, Y., Seo, J.: Text classification from unlabeled documents with bootstrapping and feature projection techniques. Inf. Process. Manage. 45(1), 70–83 (2009)
Liebeskind, C., Kotlerman, L., Dagan, I.: Text categorization from category name in an industry-motivated scenario. Lang. Resour. Eval. 49(2), 227–261 (2015)
McCallum, A., Nigam, K., Rennie, J., Seymore, K.: A machine learning approach to building domain-specific search engines. In: IJCAI, vol. 99, pp. 662–667. Citeseer (1999)
Qiu, Q., Zhang, Y., Zhu, J., Qu, W.: Building a text classifier by a keyword and wikipedia knowledge. In: Huang, R., Yang, Q., Pei, J., Gama, J., Meng, X., Li, X. (eds.) ADMA 2009. LNCS, vol. 5678, pp. 277–287. Springer, Heidelberg (2009)
Sahlgren, M., Gyllensten, A.C., Espinoza, F., Hamfors, O., Holst, A., Karlgren, J., Olsson, F., Persson, P., Viswanathan, A.: The Gavagai living lexicon. In: 10th Language Resources and Evaluation Conference, Portoroz (2016)
Schohn, G., Cohn, D.: Less is more: active learning with support vector machines. In: ICML, pp. 839–846. Citeseer (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Karlsson, V., Herman, P., Karlgren, J. (2016). Evaluating Categorisation in Real Life – An Argument Against Simple but Impractical Metrics. In: Fuhr, N., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016. Lecture Notes in Computer Science(), vol 9822. Springer, Cham. https://doi.org/10.1007/978-3-319-44564-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-44564-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44563-2
Online ISBN: 978-3-319-44564-9
eBook Packages: Computer ScienceComputer Science (R0)