Abstract
Named Entity Recognition (NER) is the task of identifying and classifying named entities in large-scale texts into predefined classes. NER in French and other relatively limited-resource languages cannot always benefit from approaches proposed for languages like English due to a dearth of large, robust datasets. In this paper, we present our work that aims to mitigate the effects of this dearth of large, labeled datasets. We propose a Transformer-based NER approach for French, using adversarial adaptation to similar domain or general corpora to improve feature extraction and enable better generalization. Our approach allows learning better features using large-scale unlabeled corpora from the same domain or mixed domains to introduce more variations during training and reduce overfitting. Experimental results on three labeled datasets show that our adaptation framework outperforms the corresponding non-adaptive models for various combinations of Transformer models, source datasets, and target corpora. We also show that adversarial adaptation to large-scale unlabeled corpora can help mitigate the performance dip incurred on using Transformer models pre-trained on smaller corpora.
A. Choudhry and I. Khatri—Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Choudhry, A., et al.: Transformer-based named entity recognition for French using adversarial adaptation to similar domain corpora (2022). https://doi.org/10.48550/ARXIV.2212.03692. Accessed 11 Jan 2023
Copara, J., Knafou, J., Naderi, N., Moro, C., Ruch, P., Teodoro, D.: Contextualized French language models for biomedical named entity recognition. In: Actes de la 6e conférence conjointe Journées d’Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Atelier DÉfi Fouille de Textes. pp. 36–48. ATALA et AFCP, Nancy, France (2020), https://aclanthology.org/2020.jeptalnrecital-deft.4. Accessed 11 Jan 2023
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1423. Accessed 11 Jan 2023
Ganin, Y., Lempitsky, V.: Unsupervised Domain Adaptation by Backpropagation (2014). https://doi.org/10.48550/ARXIV.1409.7495. Accessed 11 Jan 2023
Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2096–2030 (2016). http://jmlr.org/papers/v17/15-239.html. Accessed 11 Jan 2023
Gong, C., Tang, J., Zhou, S., Hao, Z., Wang, J.: Chinese named entity recognition with Bert. DEStech Trans. Comput. Sci. Eng. (2019). https://doi.org/10.12783/dtcse/cisnrc2019/33299. Accessed 11 Jan 2023
Gridach, M., Haddad, H.: Arabic named entity recognition: a bidirectional GRU-crf approach. In: Gelbukh, A. (ed.) CICLing 2017. LNCS, vol. 10761, pp. 264–275. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77113-7_21
Habibi, M., Weber, L., Neves, M., Wiegandt, D.L., Leser, U.: Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14), 37–48 (2017). https://doi.org/10.1093/bioinformatics/btx228. Accessed 11 Jan 2023
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270. Association for Computational Linguistics, San Diego, California (2016). https://doi.org/10.18653/v1/N16-1030. Accessed 11 Jan 2023
Le, H., et al.: FlauBERT: unsupervised language model pre-training for French. In: Proceedings of The 12th Language Resources and Evaluation Conference, pp. 2479–2490. European Language Resources Association, Marseille, France (2020). https://www.aclweb.org/anthology/2020.lrec-1.302. Accessed 11 Jan 2023
Liu, Y., et al.: RoBERTa: a Robustly optimized bert pretraining approach (2019). https://doi.org/10.48550/ARXIV.1907.11692. Accessed 11 Jan 2023
Liu, Z., Jiang, F., Hu, Y., Shi, C., Fung, P.: NER-BERT: a Pre-trained model for low-resource entity tagging (2021). https://doi.org/10.48550/ARXIV.2112.00405. Accessed 11 Jan 2023
Liu, Z., et al.: CrossNER: evaluating cross-domain named entity recognition. Proc. AAAI Conf. Artif. Intell. 35(15), 13452–13460 (2021). https://doi.org/10.1609/aaai.v35i15.17587. Accessed 11 Jan 2023
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization (2017). https://doi.org/10.48550/ARXIV.1711.05101. Accessed 11 Jan 2023
Lothritz, C., Allix, K., Veiber, L., Bissyandé, T.F., Klein, J.: Evaluating pretrained transformer-based models on the task of fine-grained named entity recognition. In: Proceedings of the 28th International Conference on Computational Linguistics. pp. 3750–3760. International Committee on Computational Linguistics, Barcelona, Spain (Online) (2020). https://doi.org/10.18653/v1/2020.coling-main.334. Accessed 11 Jan 2023
Martin, L., et al.: CamemBERT: a tasty french language model. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7203–7219. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.645. Accessed 11 Jan 2023
Neudecker, C.: An open corpus for named entity recognition in historic newspapers. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)., pp. 4348–4352. European Language Resources Association (ELRA), Portorož, Slovenia (2016), https://aclanthology.org/L16-1689. Accessed 11 Jan 2023
Nothman, J., Ringland, N., Radford, W., Murphy, T., Curran, J.R.: Learning multilingual named entity recognition from wikipedia. Artif. Intell. 194, 151–175 (2013). https://doi.org/10.1016/j.artint.2012.03.006. Accessed 11 Jan 2023
Peng, Q., Zheng, C., Cai, Y., Wang, T., Xie, H., Li, Q.: An entity-aware adversarial domain adaptation network for cross-domain named entity recognition (Student Abstract). Proc. AAAI Conf. Artif. Intell.35(18), 15865–15866 (2021). https://doi.org/10.1609/aaai.v35i18.17929. Accessed 11 Jan 2023
Ramshaw, L.A., Marcus, M.P.: Text chunking using transformation-based learning (1995). https://doi.org/10.48550/ARXIV.CMP-LG/9505040. Accessed 11 Jan 2023
Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pp. 147–155. Association for Computational Linguistics, Boulder, Colorado (2009). https://aclanthology.org/W09-1119. Accessed 11 Jan 2023
Roy, A.: Recent trends in named entity recognition (NER) (2021). https://doi.org/10.48550/ARXIV.2101.11420. Accessed 11 Janu 2023
Sang, E.F.T.K.: Introduction to the CoNLL-2002 shared task: language-independent named entity recognition (2002). https://doi.org/10.48550/ARXIV.CS/0209010. Accessed 11 Jan 2023
Tedeschi, S., Maiorca, V., Campolungo, N., Cecconi, F., Navigli, R.: WikiNEuRal: combined neural and knowledge-based silver data creation for multilingual NER. In: Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2521–2533. Association for Computational Linguistics, Punta Cana, Dominican Republic (2021). https://doi.org/10.18653/v1/2021.findings-emnlp.215. Accessed 11 Jan 2023
Wang, J., Kulkarni, M., Preotiuc-Pietro, D.: Multi-domain named entity recognition with genre-aware and agnostic inference. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8476–8488. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.750. Accessed 11 Jan 2023
Wang, J., Xu, W., Fu, X., Xu, G., Wu, Y.: ASTRAL: adversarial trained LSTM-CNN for named entity recognition. Knowl.-Bsed Syst. 197, 105842 (2020). https://doi.org/10.1016/j.knosys.2020.105842. Accessed 11 Jan 2023
Yan, R., Jiang, X., Dang, D.: Named entity recognition by using XLNet-BiLSTM-CRF. Neural Process. Lett. 53(5), 3339–3356 (2021). https://doi.org/10.1007/s11063-021-10547-1. Accessed 11 Jan 2023
Acknowledgments
This research was enabled by support provided by Calcul Québec, The Digital Research Alliance of Canada and MITACS.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Choudhry, A. et al. (2023). Adversarial Adaptation for French Named Entity Recognition. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-28238-6_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28237-9
Online ISBN: 978-3-031-28238-6
eBook Packages: Computer ScienceComputer Science (R0)