Skip to main content
SpringerLink
Log in

A Survey of Pretrained Embeddings for Japanese Legal Representation

  • Conference paper
  • First Online:
Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence (IEA/AIE 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13343))

  • 1531 Accesses

Abstract

Pretrained embeddings have proven effective in legal problems in English. Even so, working well in one language does not guarantee that these models have an advantage in other languages. Understanding the characteristics of these models in a particular language helps us to make more accurate decisions when choosing technology for problems in that language. This paper provides an analytical perspective on pretrained embeddings in the legal field in Japanese. These models are measured on quantitative numbers as well as visualized in terms of their ability to represent Japanese legal terms. With such contributions, this paper may be useful to researchers and engineers who are building Japanese legal embeddings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    japaneselawtranslation.go.jp.

  2. 2.

    https://github.com/google-research/bert/blob/master/multilingual.md.

  3. 3.

    https://nlp.ist.i.kyoto-u.ac.jp/?ku_bert_japanese.

  4. 4.

    https://github.com/cl-tohoku/bert-japanese.

References

  1. Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1532–1543. Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/D14-1162

  2. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

  3. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  4. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019)

    Google Scholar 

  5. Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

  6. Lewis, M., et al.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)

  7. Rabelo, J., Kim, M.-Y., Goebel, R., Yoshioka, M., Kano, Y., Satoh, K.: COLIEE 2020: Methods for Legal Document Retrieval and Entailment (2020)

    Google Scholar 

  8. Hannes, W., Jaromir, S., Karim, B.: Paragraph similarity scoring and fine-tuned bert for legal information retrieval and entailment. In: COLIEE 2020 (2020)

    Google Scholar 

  9. Nguyen, H.-T., et al.: Jnlp team: deep learning for legal processing in COLIEE 2020. arXiv preprint arXiv:2011.08071 (2020)

  10. Hsuan-Lei, S., Yi-Chia, C., Sieh-Chuen, H.: Bert-based ensemble model for the statute law retrieval and legal information entailment. In: COLIEE 2020 (2020)

    Google Scholar 

  11. Nguyen, H.-T., et al.: Paralaw nets-cross-lingual sentence-level pretraining for legal text processing. arXiv preprint arXiv:2106.13403 (2021)

  12. Nguyen, H.-T., et al.: Transformer-based approaches for legal text processing. Rev. Socionetwork Strat. 16, 1–21 (2022). https://doi.org/10.1007/s12626-022-00102-2

    Article  Google Scholar 

  13. Thanh, N.H., Binh, D.T., Quan, B.M., Le Minh, N.: Evaluate and visualize legal embeddings for explanation purpose. In: 2021 13th International Conference on Knowledge and Systems Engineering (KSE), pp. 1–6. IEEE (2021)

    Google Scholar 

  14. Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., Joulin, A.: Advances in pre-training distributed word representations. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018) (2018)

    Google Scholar 

  15. Chalkidis, I., Kampas, D.: Deep learning in law: early adaptation and legal word embeddings trained on large corpora. Artif. Intell. Law 27(2), 171–198 (2018). https://doi.org/10.1007/s10506-018-9238-9

    Article  Google Scholar 

  16. Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., Androutsopoulos, I.: Legal-bert: the muppets straight out of law school. arXiv preprint arXiv:2010.02559 (2020)

  17. Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019)

Download references

Ackowledgement

This work was supported by JSPS Kakenhi Grant Number 20H04295, 20K20406, and 20K20625.

Author information

Authors and Affiliations

  1. Japan Advanced Institute of Science and Technology, Nomi, Japan

    Ha-Thanh Nguyen & Le-Minh Nguyen

  2. National Institute of Informatics, Chiyoda City, Japan

    Ken Satoh

Authors
  1. Ha-Thanh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Le-Minh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ken Satoh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ha-Thanh Nguyen .

Editor information

Editors and Affiliations

  1. i-SOMET, Inc., Morioka-shi, Iwate, Japan

    Hamido Fujita

  2. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, Guangdong, China

    Philippe Fournier-Viger

  3. Texas State University, San Marcos, TX, USA

    Moonis Ali

  4. Shanghai University of Finance and Economics, Shanghai, China

    Yinglin Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nguyen, HT., Nguyen, LM., Satoh, K. (2022). A Survey of Pretrained Embeddings for Japanese Legal Representation. In: Fujita, H., Fournier-Viger, P., Ali, M., Wang, Y. (eds) Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence. IEA/AIE 2022. Lecture Notes in Computer Science(), vol 13343. Springer, Cham. https://doi.org/10.1007/978-3-031-08530-7_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-08530-7_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-08529-1

  • Online ISBN: 978-3-031-08530-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation