Skip to main content
SpringerLink
Log in
Book cover

Workshop of the Cross-Language Evaluation Forum for European Languages

CLEF 2003: Comparative Evaluation of Multilingual Information Access Systems pp 376–390Cite as

The GIRT Data in the Evaluation of CLIR Systems – from 1997 Until 2003

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3237))

Abstract

The motivations behind the creation of the German Indexing and Retrieval Test database (GIRT) are described and an overview of the structure of the different versions of GIRT is given. The way in which GIRT has been employed in various TREC and CLEF campaigns is then illustrated with a short description of methods and procedures used. The paper concludes with a summary of the trends in the GIRT tracks of these evaluation campaigns.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Womser-Hacker, C.: Das MIMOR-Modell. Mehrfachindexierung zur dynamischen Methoden-Objekt-Relationierung im Information Retrieval. Habilitationsschrift. Universität Regensburg (1996)

    Google Scholar 

  2. Buckley, C., Singhal, A., Mitra, M., Salton, G.: New Retrieval Approaches Using SMART. In: Harman, D. (ed.) The Fourth Text Retrieval Conference (TREC-4) [1995]. NIST, Gaithersburg (1996)

    Google Scholar 

  3. Haas, S.W.: Disciplinary Variation in Automatic Sublanguage Term Identification. Journal of the American Society for Information Science 48(1), 67–79 (1997)

    Article  Google Scholar 

  4. Kluck, M.: Die Evaluation von Cross-Language-Retrieval-Systemen mit Hilfe der GIRT-Daten des IZ. Ein Bericht über die Entwicklung im Zeitraum von 1997 bis 2003. Informationszentrum Sozialwissenschaften, Bonn (2003)

    Google Scholar 

  5. Frisch, E., Kluck, M.: Pretest zum Projekt German Indexing and Retrieval Testdatabase (GIRT) unter Anwendung der Retrievalsysteme Messenger und freeWAISsf, 2nd edn. Informationszentrum Sozialwissenschaften, Bonn (1997)

    Google Scholar 

  6. Kluck, M.: German Indexing and Retrieval Test Data Base (GIRT): Some Results of the Pre-test. In: Dunlop, M.D. (ed.) The 20th BCS IRSG Colloquium: Discovering New Worlds of IR (IRSG 1998), Grenoble, France, March 25-27 (1998) (= electronic workshops in computing), see: http://www.ewic.org.uk/ewic/workshop/view.cfm/IRSG-98

  7. Krause, J., Mutschke, P.: Indexierung und Fulcrum-Evaluierung. Informationszentrum Sozialwissenschaften, Bonn (1999), see: http://www.gesis.org/Publikationen/Berichte/IZ_Arbeitsberichte/pdf/ab17.pdf

  8. Binder, G., Stahl, M., Faulbaum, L.: Vergleichsuntersuchung MESSENGER – FULCRUM. Informationszentrum Sozialwissenschaften, Bonn (2000), see: http://www.gesis.org/Publikationen/Berichte/IZ_Arbeitsberichte/pdf/ab18.pdf

  9. Womser-Hacker, C. (ed.) u.a.: Projektkurs Informationsmanagement: Durchführung einer Evaluierungsstudie, Vergleich der Information-Retrieval-Systeme (IRS) DOMESTIC - LARS II - TextExtender. Universität Konstanz (1998)

    Google Scholar 

  10. Käter, T., Rittberger, M., Womser-Hacker, C.: Evaluierung der Text-Retrievalsysteme Domestic, Intelligent Miner for Text, Lars II und TextExtender. In: Semar, W., Kuhlen, R. (eds.) Information Engineering. Proceedings des 4. Konstanzer Informationswissenschaftlichen Kolloquiums (KIK 1999), pp. 63–73. UVK, Konstanz (1999)

    Google Scholar 

  11. Käter, T.: Evaluierung des Text-Retrievalsystems Intelligent Miner for Text von IBM - Eine Studie im Vergleich zur Evaluierung anderer Systeme. Universität Konstanz (1999), http://www.ub.uni-konstanz.de/kops/volltexte/1999/283/

  12. Griesbaum, J.: Evaluierung hybrider Suchsysteme im WWW. Diplomarbeit Informationswissenschaft Universität Konstanz (2000)

    Google Scholar 

  13. Eibl, M.: Visualisierung im Document Retrieval. Theoretische und praktische Zusammenführung von Softwareergonomie und Graphik Design. Informationszentrum Sozialwissenschaften, Bonn (2000)

    Google Scholar 

  14. Mandl, T.: Tolerantes Information Retrieval. Neuronale Netze zur Erhöhung der Adaptivität und Flexibilität bei der Informationssuche. UVK, Konstanz (2001)

    Google Scholar 

  15. Kluck, M., Gey, F.C.: The Domain-Specific Task of CLEF - Specific Evaluation Strategies in Cross-Language Information Retrieval. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 48–56. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  16. Schott, H. (ed.): Thesaurus Sozialwissenschaften – Thesaurus for the Social Sciences, Ausgabe edn. (1999), [Bd. 1:] Deutsch-Englisch – German-English, [Bd. 2] Englisch-Deutsch – English-German. Informationszentrum Sozialwissenschaften, Bonn (1999)

    Google Scholar 

  17. Schott, H. (ed.): Thesaurus Sozialwissenschaften (Ausgabe) (1999), [Bd. 1:] Alphabetischer Teil, [Bd. 2] Systematischer Teil. Informationszentrum Sozialwissenschaften, Bonn (1999)

    Google Scholar 

  18. Basarnova, S., Magaj, H., Mdivani, R., Schott, H., Sucker, D. (eds.): Thesaurus Sozialwissenschaften Bd.1: Deutsch-Englisch-Russisch, Bd. 2: Russisch-Deutsch-Englisch, Bd. 3: Register. Informationszentrum Sozialwissenschaften / Institut für wissenschaftliche Information in den Gesellschaftswissenschaften (INION RadW), Bonn, Moskau (1997)

    Google Scholar 

  19. Kando, N.: CLIR at NTCIR Workshop 3: Cross-Language and Cross-Genre Retrieval. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 485–504. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  20. Harman, D., Braschler, M., Hess, M., Kluck, M., Peters, C., Schäuble, P., Sheridan, P.: CLIR Evaluation at TREC. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 7–23. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  21. Vorhees, E.M., Harman, D.K. (eds.): The Eighth Text Retrieval Conference (TREC8) (1999). NIST, Gaithersburg (2000), at: http://trec.nist.gov/pubs/trec8/t8_proceedings.html

  22. Cooper, W., Chen, A., Gey, F.: Full Text Retrieval based on Probabilistic Equations with Coefficients fitted by Logistic Regression. In: Harman, D.K. (ed.) The Second Text Retrieval Conference (TREC-2), pp. 57–66. NIST, Gaithersburg (1994)

    Google Scholar 

  23. Gey, F.C., Jiang, H.: English-German Cross-Language Retrieval for the GIRT Collection - Exploiting a Multilingual Thesaurus. In: Vorhees, E.M., Harman, D.K. (eds.) The Eighth Text Retrieval Conference (TREC8) [1999], pp. 301–306. NIST, Gaithersburg (2000)

    Google Scholar 

  24. Braschler, M., Schäuble, P., Kan, M.-Y., Klavans, J.L.: The Eurospider Retrieval System and the TREC-8 Cross-Language Track. In: Vorhees, E.M., Harman, D.K. (eds.) The Eighth Text Retrieval Conference (TREC8) [1999], pp. 367–376. NIST, Gaithersburg (2000)

    Google Scholar 

  25. Sheridan, P., Ballerini, J.P.: Experiments in Multilingual Information Retrieval Using the SPIDER System. In: Proceeding of the 19th Annual International ACM SIGIR Conference, pp. 58–65 (1996)

    Google Scholar 

  26. Gey, F.C., Jiang, H., Petras, V., Chen, A.: Cross-Language Retrieval for the CLEF Collections – Comparing Multiple Methods of Retrieval. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 116–128. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  27. Gövert, N.: Bilingual Information Retrieval with HyREX and Internet Translations Services. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 237–244. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  28. Gey, F.C., Jiang, H., Perelman, N.: Working with Russian Queries for the GIRT, Bilingual, and Multilingual CLEF Tasks. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 235–243. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  29. Gey, F.C.: Research to Improve Cross-Language Retrieval – Position Paper for CLEF. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 83–88. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  30. Kamps, J., Monz, C., de Rijke, M.: Combining Evidence for Cross-Language Information retrieval. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 111–126. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  31. Petras, V., Perelman, N., Gey, F.C.: Using Thesauri in Cross-Language retrieval for German and French. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 349–362. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  32. Renders, J.-M., Déjean, H., Gaussier, E.: Assessing Automatically Extracted Bilingual Lexicons for CLIR in Vertical Domains: XRCE Participation in the GIRT Track of CLEF-2002. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 363–371. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  33. Hackl, R., Kölle, R., Mandl, T., Womser-Hacker, C.: Domain Specific Retrieval Experiments with MIMOR at the University of Hildesheim. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 343–348. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  34. Kamps, J., Monz, C., de Rijke, M., Sigurbjörnsson, B.: Language-Dependent and Language-Independent Approaches to Cross-Lingual Text Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 152–165. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  35. Petras, V., Perelman, N., Gey, F.C.: UC Berkeley at CLEF-2003 – Russian Language Experiments and Domain-Specific Retrieval. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 401–411. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  36. Alderuccio, D., Bordoni, L., Loretto, V.: Data Compression Approach to the Monolingual GIRT Task: An Agnostic Point of View. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 391–400. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  37. Leveling, J.: University of Hagen at CLEF 2003: Natural Language Access to the GIRT4 Data. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 412–424. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  38. Braschler, M., Peters, C.: Cross-Language Evaluation Forum: Objectives, Results, Achievements. Information Retrieval 7, 1–2, 7–31 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Informationszentrum Sozialwissenschaften (IZ), Lennéstr. 30, 53113, Bonn, Germany

    Michael Kluck

Authors
  1. Michael Kluck
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. ISTI-CNR, Area di Ricerca, Pisa, Italy

    Carol Peters

  2. No Affiliations,  

    Julio Gonzalo  & Martin Braschler  & 

  3. German Institute for International and Security Affairs, Stiftung Wissenschaft und Politik (SWP), Ludwigkirchplatz 3-4, 10719, Berlin, Germany

    Michael Kluck

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kluck, M. (2004). The GIRT Data in the Evaluation of CLIR Systems – from 1997 Until 2003. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds) Comparative Evaluation of Multilingual Information Access Systems. CLEF 2003. Lecture Notes in Computer Science, vol 3237. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30222-3_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30222-3_37

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-24017-4

  • Online ISBN: 978-3-540-30222-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation