Skip to main content
SpringerLink
Log in

Using Comparable Corpora to Improve the Effectiveness of Cross-Language Information Retrieval

  • Conference paper
Advances in Natural Language Processing (NLP 2010)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6233))

Included in the following conference series:

Abstract

Large-scale comparable corpora became more abundant and accessible than parallel corpora, with the explosive growth of the World Wide Web. From the Cross-Language Information Retrieval point of view, limitation of translation resources as well as ambiguity arising due to failure to translate query terms is largely responsible for large drops in the effectiveness below monolingual performance. Therefore, strategies on bilingual terminology extraction from comparable texts must be given more attention in order to enrich existing bilingual lexicons and thesauri and to enhance Cross-Language Information Retrieval. In the present paper, we focus on the enhancement of Cross-Language Information Retrieval using a two-stage corpus-based translation model that includes bi-directional extraction of bilingual terminology from comparable corpora and selection of best translation alternatives on the basis of their morphological knowledge. The impact of comparable corpora on the performance of the Cross-Language Information Retrieval process is evaluated in this study and the results indicate that the effect is clearly positive, especially when using the linear combination with bilingual dictionaries and Japanese-English pair of languages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Buckley, C., Allan, J., Salton, G.: Automatic Routing and Ad-hoc Retrieval using SMART. In: Proceedings of the Second Text Retrieval Conference TREC-2, pp. 45–56 (1994)

    Google Scholar 

  2. Cancedda, N., Dejean, H., Gaussier, E., Renders, J.M., Vinokourov, A.: Report on CLEF-2003 Experiments : Two ways of Extracting Multilingual Resources from Corpora. In: Proceedings of LEF 2003 Evaluation Campaign, Norway, Trondheim, August 21-22 (2003)

    Google Scholar 

  3. Dagan, I., Itai, I.: Word Sense Disambiguation using a Second Language Monolingual Corpus. Computational Linguistics 20(4), 563–596 (1994)

    Google Scholar 

  4. Dejean, H., Gaussier, E., Sadat, F.: An Approach based on Multilingual Thesauri and Model Combination for Bilingual Lexicon Extraction. In: Proceedings of COLING 2002, Taiwan, pp. 218–224 (2002)

    Google Scholar 

  5. Diab, M., Finch, S.: A Statistical Word-Level Translation Model for Comparable Corpora. In: Proceedings of the Conference on Content-based Multimedia Information Access RIAO (2000)

    Google Scholar 

  6. Dunning, T.: Accurate Methods for the Statistics of Surprise and Coincidence. Computational Linguistics 19(1), 61–74 (1993)

    Google Scholar 

  7. EDR. Japan Electronic Dictionary Research Institute, Ltd. EDR electronic dictionary version 1.5 technical guide. Technical report TR2-007. Japan Electronic Dictionary research Institute, Ltd. (1996)

    Google Scholar 

  8. Fox, A.E., Shaw, A.J.: Combination of Multiple Searches. In: Proceedings of the Second Text Retrieval Conference TREC-2, pp. 243–252 (1994)

    Google Scholar 

  9. Fuhr, N., Pfeifer, U., Bremkamp, C., Pollmann, M., Buckley, C.: Probabilistic learning Approaches for Indexing and Retrieval with the TREC-2 Collection. In: Proceedings of the Second Text Retrieval Conference TREC-2, pp. 67–74 (1994)

    Google Scholar 

  10. Fung, P.: A Statistical View of Bilingual Lexicon Extraction: From Parallel Corpora to Non-Parallel Corpora. In: Véronis, J. (ed.) Parallel Text Processing (2000)

    Google Scholar 

  11. Gaussier, E., Renders, J.M., Matveeva, I., Goutte, C., Dejean, H.: A Geometric View on Bilingual Lexicon Extraction from Comparable Corpora. In: Proceedings of ACL 2004, Barcelona, Spain, pp. 526–533 (2004)

    Google Scholar 

  12. Grefenstette, G.: The WWW as a Resource for Example-based MT Tasks. In: ASLIB 1999 Translating and the Computer 21 (1999)

    Google Scholar 

  13. Hedlund, T.: Compounds in dictionary-based cross-language information retrieval. Information Research 7(2) (January 2002)

    Google Scholar 

  14. Kaji, H.: Word Sense Acquisition from Bilingual Corpora. In: Proceedings of HLT-NAACL 2003, Edmonton, Canada, pp. 32–39 (2003)

    Google Scholar 

  15. Kando, N.: Overview of the Second NTCIR Workshop. In: Proceedings of the Second NTCIR Workshop on Research in Chinese and Japanese Text Retrieval and text Summarization, Tokyo (2001)

    Google Scholar 

  16. Klavens, J., Tzoukermann, E.: Combining Corpus and Machine-Readable Dictionary Data for Building Bilingual Lexicons. Machine Translation 10(3-4), 1–34 (1996)

    Google Scholar 

  17. Knaus, D., Shauble, P.: Effective and Efficient retrieval from large and Dynamic Document Collections. In: Proceedings of the Second Text Retrieval Conference TREC-2, pp. 163–170 (1994)

    Google Scholar 

  18. Knight, K., Graehl, J.: Machine Transliteration. Computational Linguistics 24(4) (1998)

    Google Scholar 

  19. Koehn, P., Knight, K.: Learning a Translation Lexicon from Monolingual Corpora. In: Proceedings of ACL 2002 Workshop on Unsupervised Lexical Acquisition (2002)

    Google Scholar 

  20. Matsumoto, Y., Kitauchi, A., Yamashita, T., Imaichi, O., Imamura, T.: Japanese morphological analysis system ChaSen manual. Technical Report NAIST-IS-TR97007, NAIST (1997)

    Google Scholar 

  21. Nakagawa, H.: Disambiguation of Lexical Translations Based on Bilingual Comparable Corpora. In: Proceedings of LREC 2000, Workshop of Terminology Resources and Computation WTRC 2000, pp. 33–38 (2000)

    Google Scholar 

  22. Nie, J.Y., Simard, M., Isabelle, P., Durand, R.: Cross-Language Information Retrieval based on parallel texts and automatic mining of parallel texts from the Web. In: Proceedings of the 22nd ACM SIGIR Conference, pp. 74–81 (1999)

    Google Scholar 

  23. Oard, D., Diekema, A.: Cross-Language Information Retrieval. In: Annual Review of Information Science and Technology (ARIST), vol. 33, pp. 223–256 (1998)

    Google Scholar 

  24. Peters, C., Picchi, E.: Capturing the Comparable: A System for Querying Comparable Text Corpora. In: Proceedings of the Third International Conference on Statistical Analysis of Textual Data, pp. 255–262 (1995)

    Google Scholar 

  25. Pirkola, A., Hedlund, T., Keskustalo, H., Järvelin, K.: Dictionary-based cross-language information retrieval: problems, methods, and research findings. Information Retrieval 4(3/4), 209–230 (2001)

    Article  MATH  Google Scholar 

  26. Rapp, R.: Automatic Identification of Word Translations from Unrelated English and German Corpora. In: Proceedings of European Chapter of the Association for Computational Linguistics, EACL (1999)

    Google Scholar 

  27. Renders, J.M., Dejean, H., Gaussier, E.: Assessing Automatically Extracted Bilingual Lexicons for CLIR in Vertical Domains: XRCE Participation in the GIRT Track of CLEF 2002. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, Springer, Heidelberg (2003)

    Google Scholar 

  28. Sadat, F., Maeda, A., Yoshikawa, M., Uemura, S.: Exploiting and Combining Multiple Resources for Query Expansion in  Cross-Language Information Retrieval. IPSJ Transactions of Databases 43(SIG 9) (TOD 15), 39–54 (2002)

    Google Scholar 

  29. Sadat, F., Yoshikawa, M., Uemura, S.: Enhancing Cross-language Information Retrieval by an Automatic Acquisition of Bilingual Terminology from Comparable Corpora. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2003, Toronto, Canada (2003)

    Google Scholar 

  30. Sadat, F., Yoshikawa, M., Uemura, S.: Learning Bilingual Translations from Comparable Corpora to Cross-Language Information Retrieval: Hybrid Statistics-based and Linguistics-based Approach. In: Proceedings of the Sixth International Workshop on Information Retrieval with Asian Languages IRAL 2003, Sapporo, Japan,

    Google Scholar 

  31. Sadat, F., Yoshikawa, M., Uemura, S.: Bilingual Terminology Acquisition from Comparable Corpora and Phrasal Translation to Cross-Language Information Retrieval. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, ACL 2003, Sapporo, Japan (2003)

    Google Scholar 

  32. Sadat, F.: Knowledge Acquisition from Collections of News Articles to Cross-language Information Retrieval. In: Proceedings of RIAO 2004 Conference (Recherche d’Information Assisté par Ordinateur), Avignon, France, April 26-28, pp. 504–513 (2004)

    Google Scholar 

  33. Salton, G.: The SMART Retrieval System, Experiments in Automatic Documents Processing. Prentice-Hall, Inc., Englewood Cliffs (1971)

    Google Scholar 

  34. Salton, G., McGill, J.: Introduction to Modern Information Retrieval. Mc Graw-Hill, New York (1983)

    MATH  Google Scholar 

  35. Savoy, J.: Cross-Language Information Retrieval: Experiments based on CLEF 2000 Corpora. Information Processing and Management 39(1), 75–115 (2003)

    Article  MATH  Google Scholar 

  36. Sekine, S.: OAK System– Manual. New York University (2001)

    Google Scholar 

  37. Shahzad, I., Ohtake, K., Masuyama, S., Yamamoto, K.: Identifying Translations of Compound Using Non-aligned Corpora. In: Proceedings of the Workshop MAL, pp. 108–113 (1999)

    Google Scholar 

  38. Tanaka, K., Iwasaki, H.: Extraction of Lexical Translations from Non-Aligned Corpora. In: Proceedings of COLING (1996)

    Google Scholar 

  39. Utsuro, U., Horiuchi, T., Chiba, Y., Hamamoto, T.: Semi-automatic Compilation of Bilingual Lexicon Entries from Cross-Lingually Relevant News Articles on WWW News Sites. In: Proceedings of the Association for Machine Translation in the Americas (AMTA 2002), pp. 165–176 (2002)

    Google Scholar 

  40. Utsuro, T., Horiuchi, T., Hamamoto, T., Hino, K., Nakayama, T.: Effect of Cross-Language IR in Bilingual Lexicon Acquisition from Comparable Corpora. In: Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2003), pp. 355–362 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science department, University of Quebec in Montreal, 201 President Kennedy avenue, Montreal, QC, Canada

    Fatiha Sadat

Authors
  1. Fatiha Sadat
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Science, Reykjavik University, Kringlan 1, 103, Reykjavik, Iceland

    Hrafn Loftsson

  2. Department of Icelandic, University of Iceland, Árnagardur v/Sudurgötu, 101, Reykjavik, Iceland

    Eiríkur Rögnvaldsson

  3. Arni Magnusson Institute for Icelandic Studies, Neshagi 16, 101, Reykjavik, Iceland

    Sigrún Helgadóttir

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sadat, F. (2010). Using Comparable Corpora to Improve the Effectiveness of Cross-Language Information Retrieval. In: Loftsson, H., Rögnvaldsson, E., Helgadóttir, S. (eds) Advances in Natural Language Processing. NLP 2010. Lecture Notes in Computer Science(), vol 6233. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14770-8_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-14770-8_36

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-14769-2

  • Online ISBN: 978-3-642-14770-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation