Skip to main content
SpringerLink
Log in

Translated Texts Under the Lens: From Machine Translation Detection to Source Language Identification

  • Conference paper
  • First Online:
Advances in Intelligent Data Analysis XXI (IDA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13876))

Included in the following conference series:

  • 820 Accesses

Abstract

Machine Translation Systems are today used to break down linguistic barriers. People from different countries and languages can now interact with each other thanks to state-of-the-art translators from prominent software companies like Google and Microsoft. However, these tools are also used to expand the audience for phishing attacks, scam emails or to generate fake reviews to promote a product on different e-commerce platforms. In all these cases, detecting whether a text has been translated can be crucial information. In this work, we tackle the problem of the detection of translated texts from different angles. On top of addressing the classic task of machine translation detection, we investigate and find common patterns across different machine translation systems unrelated to the original text’s source language. Then, we show that it is possible to identify the machine translation system used to generate a translated text with high performances (F1-score 88.5%) and that it is also possible to identify the source language of the original text. We perform our tasks over two datasets that we use to evaluate our models: Books, a new dataset we built from scratch based on excerpts of novels, and the well-known Europarl dataset, based on proceedings of the European Parliament.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aharoni, R., Koppel, M., Goldberg, Y.: Automatic detection of machine translated text and translation quality estimation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Vol. 2: Short Papers), pp. 289–295. Association for Computational Linguistics, Baltimore (2014). https://doi.org/10.3115/v1/P14-2048, https://aclanthology.org/P14-2048

  2. Arase, Y., Zhou, M.: Machine translation detection from monolingual web-text. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers), pp. 1597–1607. Association for Computational Linguistics, Sofia (2013). https://aclanthology.org/P13-1157

  3. Battiti, R., Masulli, F.: Bfgs optimization for faster and automated supervised learning. In: International Neural Network Conference, pp. 757–760. Springer, Dordrecht (1990). https://doi.org/10.1007/978-94-009-0643-3_68

  4. Bhardwaj, S., Alfonso Hermelo, D., Langlais, P., Bernier-Colborne, G., Goutte, C., Simard, M.: Human or neural translation? In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 6553–6564. International Committee on Computational Linguistics, Barcelona (2020). https://doi.org/10.18653/v1/2020.coling-main.576, https://aclanthology.org/2020.coling-main.576

  5. Bizzoni, Y., Juzek, T.S., España-Bonet, C., Dutta Chowdhury, K., van Genabith, J., Teich, E.: How human is machine translationese? comparing human and machine translations of text and speech. In: Proceedings of the 17th International Conference on Spoken Language Translation, pp. 280–290. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.iwslt-1.34, https://aclanthology.org/2020.iwslt-1.34

  6. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    Article  MATH  Google Scholar 

  7. DeepL: Deepl translator (2021). https://www.deepl.com/pro-api

  8. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  9. Forman, G., et al.: An extensive empirical study of feature selection metrics for text classification. J. Mach. Learn. Res. 3, 1289–1305 (2003)

    MATH  Google Scholar 

  10. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  11. Gellerstam, M.: Translationese in swedish novels translated from English. In: Wollin, L., Lindquist, H. (eds.) Translation Studies in Scandinavia: Poceedings from the Scandinavian Symposium on Translation Theory (SSOTT) II, pp. 88–95. no. 75 in Lund Studies in English, CWK Gleerup, Lund (1986)

    Google Scholar 

  12. Google: Google translator (2021). https://cloud.google.com/translate

  13. Google: Managing multi-regional and multilingual sites (2021). https://developers.google.com/search/docs/advanced/crawling/managing-multi-regional-sites

  14. van Halteren, H.: Source language markers in EUROPARL translations. In: Proceedings of the 22nd International Conference on Computational Linguistics (2008), pp. 937–944. Coling 2008 Organizing Committee, Manchester, UK (2008). https://aclanthology.org/C08-1118

  15. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)

    Article  MATH  Google Scholar 

  16. Juuti, M., Sun, B., Mori, T., Asokan, N.: Stay on-topic: generating context-specific fake restaurant reviews. In: Lopez, J., Zhou, J., Soriano, M. (eds.) ESORICS 2018. LNCS, vol. 11098, pp. 132–151. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99073-6_7

    Chapter  Google Scholar 

  17. Kacmarcik, G., Gamon, M.: Obfuscating document stylometry to preserve author anonymity. In: Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pp. 444–451. Association for Computational Linguistics, Sydney (2006). https://aclanthology.org/P06-2058

  18. Koehn, P.: Europarl: a parallel corpus for statistical machine translation. In: Proceedings of Machine Translation Summit X: Papers, pp. 79–86. Phuket, Thailand (2005). https://aclanthology.org/2005.mtsummit-papers.11

  19. Koehn, P., et al.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pp. 177–180. Association for Computational Linguistics, Prague (2007). https://aclanthology.org/P07-2045

  20. Koppel, M., Ordan, N.: Translationese and its dialects. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 1318–1326. Association for Computational Linguistics, Portland (2011). https://aclanthology.org/P11-1132

  21. La Morgia, M., Mei, A., Nemmi, E., Raponi, S., Stefa, J.: Nationality and geolocation-based profiling in the dark (web). IEEE Trans. Serv. Comput. 15(1), 429–441 (2019)

    Article  Google Scholar 

  22. Labbé, C., Labbé, D.: Duplicate and fake publications in the scientific literature: how many SCIgen papers in computer science? Scientometrics 94(1), 379–396 (2013). https://doi.org/10.1007/s11192-012-0781-y

    Article  Google Scholar 

  23. Li, Y., Wang, R., Zhao, H.: A machine learning method to distinguish machine translation from human translation. In: Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation: Posters, pp. 354–360, Shanghai, China (2015). https://aclanthology.org/Y15-2041

  24. Lynch, G., Vogel, C.: Towards the automatic detection of the source language of a literary translation. In: Proceedings of COLING 2012: Posters, pp. 775–784. The COLING 2012 Organizing Committee, Mumbai, India (2012). https://aclanthology.org/C12-2076

  25. Mahmood, A., Ahmad, F., Shafiq, Z., Srinivasan, P., Zaffar, F.: A girl has no name: Automated authorship obfuscation using mutant-x. Proc. Priv. Enhancing Technol. 2019(4), 54–71 (2019)

    Article  Google Scholar 

  26. Microsoft: Microsoft translator (2021). https://www.microsoft.com/translator/

  27. Nguyen-Son, H.Q., Nguyen, H.H., Tieu, N.D.T., Yamagishi, J., Echizen, I.: Identifying computer-translated paragraphs using coherence features. In: Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation. Association for Computational Linguistics, Hong Kong (2018). https://aclanthology.org/Y18-1056

  28. Nguyen-Son, H.Q., Thao, T., Hidano, S., Gupta, I., Kiyomoto, S.: Machine translated text detection through text similarity with round-trip translation. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5792–5797. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.naacl-main.462, https://aclanthology.org/2021.naacl-main.462

  29. Nguyen-Son, H.Q., Tieu, N.D.T., Nguyen, H.H., Yamagishi, J., Zen, I.E.: Identifying computer-generated text using statistical analysis. In: 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1504–1511. IEEE (2017)

    Google Scholar 

  30. Padró, M., Padró, L.: Comparing methods for language identification. Procesamiento del lenguaje natural 33 (2004)

    Google Scholar 

  31. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics, Philadelphia (2002). https://doi.org/10.3115/1073083.1073135, https://aclanthology.org/P02-1040/

  32. Parmar, Y.S., Jahankhani, H.: Utilising machine learning against email phishing to detect malicious emails. In: Montasari, R., Jahankhani, H. (eds.) Artificial Intelligence in Cyber Security: Impact and Implications. ASTSA, pp. 73–102. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88040-8_3

    Chapter  Google Scholar 

  33. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  34. Popescu, M.: Studying translationese at the character level. In: Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pp. 634–639. Association for Computational Linguistics, Hissar (2011), https://aclanthology.org/R11-1091

  35. St, L., Wold, S., et al.: Analysis of variance (ANOVA). Chemom. Intell. Lab. Syst. 6(4), 259–272 (1989)

    Article  Google Scholar 

  36. Stamatatos, E.: Authorship attribution using text distortion. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Vol. 1, Long Papers, pp. 1138–1149 (2017)

    Google Scholar 

  37. SystemsLab: Book dataset. https://github.com/SystemsLab-Sapienza/books-dataset

  38. Tetreault, J., Blanchard, D., Cahill, A.: A report on the first native language identification shared task. In: Proceedings of the 8th Workshop on Innovative use of NLP for Building Educational Applications, pp. 48–57 (2013)

    Google Scholar 

  39. Wright, R.E.: Logistic regression (1995)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the MIUR under grant “Dipartimenti di eccellenza 2018-2022” of the Department of Computer Science of Sapienza University.

Author information

Authors and Affiliations

  1. Sapienza University of Rome, Rome, Italy

    Massimo La Morgia, Alessandro Mei, Eugenio Nerio Nemmi, Luca Sabatini & Francesco Sassi

Authors
  1. Massimo La Morgia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eugenio Nerio Nemmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luca Sabatini
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francesco Sassi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Massimo La Morgia , Alessandro Mei , Eugenio Nerio Nemmi or Francesco Sassi .

Editor information

Editors and Affiliations

  1. Université de Caen Normandie, Caen, France

    Bruno Crémilleux

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Sibylle Hess

  3. UCLouvain, Louvain-la-Neuve, Belgium

    Siegfried Nijssen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

La Morgia, M., Mei, A., Nemmi, E.N., Sabatini, L., Sassi, F. (2023). Translated Texts Under the Lens: From Machine Translation Detection to Source Language Identification. In: Crémilleux, B., Hess, S., Nijssen, S. (eds) Advances in Intelligent Data Analysis XXI. IDA 2023. Lecture Notes in Computer Science, vol 13876. Springer, Cham. https://doi.org/10.1007/978-3-031-30047-9_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-30047-9_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-30046-2

  • Online ISBN: 978-3-031-30047-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation