Abstract
Evaluating the effectiveness of XML retrieval requires building test collections where the evaluation paradigms are provided according to criteria that take into account structural aspects. The INitiative for the Evaluation of XML retrieval (INEX) was set up in 2002, and aimed to establish an infrastructure and to provide means, in the form of large test collections and appropriate scoring methods, for evaluating the effectiveness of content-oriented XML retrieval. This paper describes the evaluation methodology developed in INEX, with particular focus on how evaluation metrics and the notion of relevance are treated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Clarke, C.: Range results in XML retrieval. In: Proceedings of the INEX Workshop on Element Retrieval Methodology (2005)
Denoyer, L., Gallinari, P.: The Wikipedia XML Corpus. SIGIR Forum 40(1) (2006)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM TOIS 20(4), 422–446 (2002)
Kekäläinen, J., Järvelin, K.: Using graded relevance assessments in IR evaluation. JASIST 53(13), 1120–1129 (2002)
Ogilvie, P., Lalmas, M.: Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation. In: Proceedings of ACM CIKM, pp. 84–93 (2006)
Piwowarski, B., Lalmas, M.: Providing consistent and exhaustive relevance assessments for XML retrieval evaluation. In: Proceedings of ACM CIKM, pp. 361–370 (2004)
Kazai, G., Lalmas, M.: eXtended Cumulated Gain Measures for the Evaluation of Content-oriented XML Retrieval. ACM TOIS 24(4), 503–542 (2006)
Tombros, A., Malik, S., Larsen, B.: Report on the INEX 2004 interactive track. ACM SIGIR Forum 39(1) (2005)
Trotman, A., Sigurbjornsson, B.: Narrowed extended XPATH I (NEXI). In: Proceedings of the INEX Workshop on Element Retrieval Methodology (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lalmas, M., Tombros, A. (2007). INEX 2002 - 2006: Understanding XML Retrieval Evaluation. In: Thanos, C., Borri, F., Candela, L. (eds) Digital Libraries: Research and Development. DELOS 2007. Lecture Notes in Computer Science, vol 4877. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77088-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-540-77088-6_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77087-9
Online ISBN: 978-3-540-77088-6
eBook Packages: Computer ScienceComputer Science (R0)