초록

In this paper, our focus is the connection and influence of language technologies on the research in neurolinguistics. We present a review of brain imaging-based neurolinguistic studies with a focus on the natural language representations, such as word embeddings and pre-trained language models. Mutual enrichment of neurolinguistics and language technologies leads to development of brain-aware natural language representations. The importance of this research area is emphasized by medical applications

키워드

neurolinguistics, neuroimaging data, EEG, fMRI, natural language representations, word embeddings, distributional semantics models, word2vec, GloVe, BERT, brain-aware embeddings

참고문헌(79)open

  1. [학술대회] Abnar, S. / 2018 / Experiential, distributional and dependency-based word embeddings have complementary roles in decoding brain activity / Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics(CMCL 2018) : 57 ~ 66

  2. [학술대회] Abnar, S. / 2019 / Blackbox Meets Blackbox : Representational Similarity & Stability Analysis of Neural Language Models and Brains / Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP : 191 ~ 203

  3. [학술대회] Bartunov, Sergey / 2016 / Breaking sticks and ambiguities with adaptive skip-gram / Artificial Intelligence and Statistics : 130 ~ 138

  4. [학술지] Bartusiak, Roman / 2019 / WordNet2Vec: Corpora agnostic word vectorization method / Neurocomputing 326 : 141 ~ 150

  5. [학술지] Bojanowski, Piotr / 2017 / Enriching word vectors with subword information / Transactions of the Association for Computational Linguistics 5 : 135 ~ 146

  6. [학술지] Buckner, R. L / 1998 / Event-related fMRI and the hemodynamic response / Human Brain Mapping

  7. [학술대회] Cer, Daniel / 2018 / Universal sentence encoder for English / Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations : 169 ~ 174

  8. [학술대회] Chung, Yu-An / 2018 / Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech / Proc. Interspeech : 811 ~ 815

  9. [학술지] Collobert, R. / 2011 / Natural language processing (almost) from scratch / The Journal of Machine Learning Research 12 : 2493 ~ 2537

  10. [학술지] Ciaran Cooney / 2018 / Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface / iScience 8 : 103 ~ 125

  11. [기타] Dai, Zihang / 2019 / Transformer-xl: Attentive language models beyond a fixed-length context / arXiv preprint arXiv:1901.02860

  12. [학술대회] Deriu, Jan / 2017 / Leveraging large amounts of weakly supervised data for multi-language sentiment classification / Proceedings of the 26th international conference on world wide web : 1045 ~ 1052

  13. [학술대회] Devlin, Jacob / 2019 / BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1 : 4171 ~ 4186

  14. [학술대회] Eger, Steffen / 2019 / Pitfalls in the Evaluation of Sentence Embeddings / Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) : 55 ~ 60

  15. [학술대회] Ettinger, A. / 2016 / Modeling N400 amplitude using vector space models of word representation / CogSci., 2016 2016 : 1445 ~ 1450

  16. [학술대회] Faruqui, Manaal / 2015 / Retrofitting Word Vectors to Semantic Lexicons / Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies : 1606 ~ 1615

  17. [학술지] Gerasimos Fergadiotis / 2016 / Algorithmic Classification of Five Characteristic Types of Paraphasias / American Journal of Speech-Language Pathology 25 (4S)

  18. [학술지] Fiebach, C. J. / 2004 / Processing concrete words : fMRI evidence against a specific right-hemisphere involvement / Neuropsychologia 42 (1) : 62 ~ 70

  19. [학술지] Frank Stefan L / 2013 / Reading time data for evaluating broad-coverage models of english sentence processing / Behavior Research Methods 45 (4) : 1182 ~ 1190

  20. [학술지] Kathleen C. Fraser / 2019 / Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment / Computer Speech & Language 53 : 121 ~ 139

  21. [학술지] Friederici, A. D. / 2000 / Segregating semantic and syntactic aspects of processing in the human brain : an fMRI investigation of different word types / Cerebral cortex 10 (7) : 698 ~ 705

  22. [학술대회] Fyshe, Alona / 2014 / Interpretable semantic vectors from a joint model of brain-and text-based meaning / Proceedings of the conference. Association for Computational Linguistics. Meeting 2014 : 489

  23. [학술지] Alona Fyshe / 2019 / The lexical semantics of adjective–noun phrases in the human brain / Human Brain Mapping 40 (15) : 4457 ~ 4469

  24. [기타] Goldberg, Yoav / 2019 / Assessing BERT's Syntactic Abilities / arXiv preprint arXiv:1901.05287

  25. [기타] Jain, S. / 2018 / Incorporating context into language encoding models for fmri / bioRxiv : 327601

  26. [학술대회] Jauhar, Sujay Kumar / 2015 / Ontologically grounded multi-sense representation learning for semantic vector space models / Proceedings of the 2015 conference of the north American chapter of the Association for Computational Linguistics: human language technologies : 683 ~ 693

  27. [학술지] Harris, Zellig S / 1954 / Distributional structure / Word 10 (2-3) : 146 ~ 162

  28. [학술지] Michael Hanke / 2014 / A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie / Scientific Data 1 (1) : 1 ~ 18

  29. [인터넷자료] Hollenstein, N. / CogniVal: A Framework for Cognitive Word Embedding Evaluation

  30. [학술지] R. J. Huster / 2012 / Methods for Simultaneous EEG-fMRI: An Introductory Review / Journal of Neuroscience 32 (18) : 6053 ~ 6060

  31. [학술지] Alexander G. Huth / 2016 / Natural speech reveals the semantic maps that tile human cerebral cortex / Nature 532 (7600) : 453 ~ 458

  32. [인터넷자료] Ivanova, A. A. / The language network is recruited but not required for non-verbal semantic processing / bioRxiv

  33. [학술대회] Kiros, Ryan / 2015 / Skip-thought vectors / Advances in neural information processing systems : 3294 ~ 3302

  34. [학술지] Gina R. Kuperberg / 2007 / Neural mechanisms of language comprehension: Challenges to syntax / Brain Research 1146 : 23 ~ 49

  35. [학술지] Landauer, Thomas K. / 1997 / A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge / Psychological review 104 (2) : 211

  36. [학술대회] Le, Quoc / 2014 / Distributed representations of sentences and documents / International conference on machine learning : 1188 ~ 1196

  37. [단행본] Lan, Zhenzhong / 2019 / ALBERT: A Lite BERT for Self-supervised Learning of Language Representations / International Conference on Learning Representations

  38. [학술대회] Linz, N. / 2017 / Using Neural Word Embeddings in the Analysis of the Clinical Semantic Verbal Fluency Task / IWCS (2017) 12th International Conference on Computational Semantics

  39. [학술대회] Liu, Nelson F. / 2019 / Linguistic Knowledge and Transferability of Contextual Representations / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1 : 1073 ~ 1094

  40. [기타] Liu, Yinhan / 2019 / Roberta: A robustly optimized bert pretraining approach / arXiv preprint arXiv:1907.11692

  41. [학술대회] Mao, Junhua / 2016 / Training and evaluating multimodal word embeddings with large-scale web annotated images / Advances in neural information processing systems : 442 ~ 450

  42. [인터넷자료] McKinney-Bock, K. / Classification of Semantic Paraphasias: Optimization of a Word Embedding Model

  43. [학술대회] Mikolov, Tomas / 2013 / Distributed representations of words and phrases and their compositionality / Advances in neural information processing systems : 3111 ~ 3119

  44. [학술지] Mitchell, T. M. / 2008 / Predicting Human Brain Activity Associated with the Meanings of Nouns / Science 320 : 1191 ~ 1195

  45. [학술지] Kevin Murphy / 2013 / Resting-state fMRI confounds and cleanup / NeuroImage 80 : 349 ~ 359

  46. [학술대회] Murphy, B. / 2012 / Selecting corpus-semantic models for neurolin- guistic decoding / Proceedings of the First Joint Conference on Lexical and Computational Semantics 1,2

  47. [학술대회] Oota, S. R. / 2018 / fMRI Semantic Category Decoding Using Linguistic Encoding of Word Embeddings / International Conference on Neural Information Processing : 3 ~ 15

  48. [학술지] Serguei V.S. Pakhomov / 2016 / Characterizing cognitive performance in a large longitudinal study of aging with computerized semantic indices of verbal fluency / Neuropsychologia 89 : 42 ~ 56

  49. [학술대회] Pelevina, Maria / 2016 / Making Sense of Word Embeddings / Proceedings of the 1st Workshop on Representation Learning for NLP : 174 ~ 183

  50. [학술대회] Pennington, Jeffrey / 2014 / Glove : Global vectors for word representation / Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) : 1532 ~ 1543

  51. [학술지] Pereira, F. / 2018 / Toward a universal decoder of linguistic meaning from brain activation / Nature communications 9 (1) : 963

  52. [기타] Peters, Matthew E. / 2018 / Deep contextualized word representations / arXiv preprint arXiv:1802.05365

  53. [학술대회] Peters, Matthew E. / 2019 / Knowledge Enhanced Contextual Word Representations / Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) : 43 ~ 54

  54. [학술대회] Pires, Telmo / 2019 / How Multilingual is Multilingual BERT? / Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics : 4996 ~ 5001

  55. [기타] Raffel, Colin / 2019 / Exploring the limits of transfer learning with a unified text-to-text transformer / arXiv preprint arXiv:1910.10683

  56. [학술지] PHILIPPE H. ROBERT / 1998 / Clustering and switching strategies in verbal fluency tasks: Comparison between schizophrenics and healthy adults / Journal of the International Neuropsychological Society 4 (6) : 539 ~ 546

  57. [학술대회] Rodrigues, J. A. / 2018 / Predicting brain activation with WordNet embeddings / Proceedings of the Eight Workshop on Cognitive Aspects of Computational Language Learning and Processing : 1 ~ 5

  58. [학술지] Ardi Roelofs / 2019 / Phonological cueing of word finding in aphasia: insights from simulations of immediate and treatment effects / Aphasiology : 1 ~ 17

  59. [학술지] Ardi Roelofs / 1997 / The WEAVER model of word-form encoding in speech production / Cognition 64 (3) : 249 ~ 284

  60. [학술대회] Rustandi Indrayana / 2009 / Integrating Multiple-Study Multiple-Subject fMRI Datasets Using Canonical Correlation Analysis / MICCAI 2009 Workshop: Statistical modeling and detection issues in intra- and inter-subject functional MRI data analysis

  61. [학술대회] Salle, Alexandre / 2016 / Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations / Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics 2016 : 419 ~ 424

  62. [학술지] Riitta Salmelin / 2007 / Clinical neurophysiology of language: The MEG approach / Clinical Neurophysiology 118 (2) : 237 ~ 254

  63. [학술지] Sejnowski, T. J. / 2014 / Putting big data to good use in neuroscience / Nature neuroscience 17 (11) : 1440

  64. [학술대회] Søgaard, A. / 2016 / Evaluating word embeddings with fMRI and eye-tracking / Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP : 116 ~ 121

  65. [학술대회] Sharaev, M. / 2018 / Pattern Recognition Pipeline for Neuroimaging Data / 8th IAPR TC3 Workshop on Artificial Neural Networks in Pattern Recognition 2018 : 306 ~ 319

  66. [학술대회] Sharaev, M. / 2018 / MRI-Based Diagnostics of Depression Concomitant with Epilepsy: In Search of the Potential Biomarkers / IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) : 555 ~ 564

  67. [학술대회] Schwartz, Dan / 2019 / Understanding language-elicited EEG data by predicting it from a fine-tuned language model / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1 : 43 ~ 57

  68. [학술대회] Schwartz, Dan / 2019 / Inducing brain-relevant bias in natural language processing models / Advances in Neural Information Processing Systems

  69. [학술대회] Toneva M. / 2019 / Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) / Advances in Neural Information Processing Systems

  70. [학술지] Angela K. Troyer / 1997 / Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults. / Neuropsychology 11 (1) : 138 ~ 146

  71. [학술지] Angela K Troyer / 1998 / Clustering and switching on verbal fluency: the effects of focal frontal- and temporal-lobe lesions / Neuropsychologia 36 (6) : 499 ~ 504

  72. [학술지] David C. Van Essen / 2013 / The WU-Minn Human Connectome Project: An overview / NeuroImage 80 : 62 ~ 79

  73. [학술지] Wise, R. / 1991 / Language activation studies with positron emission tomography / Ciba Found Symp 163 : 218 ~ 234

  74. [학술지] Leila Wehbe / 2014 / Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses / PLoS ONE 9 (11) : e112575

  75. [학술대회] Wehbe, L. / 2014 / Aligning context-based statistical models of language with brain activity during reading / EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

  76. [기타] Yang, Zhilin / 2019 / XLNet: Generalized Autoregressive Pretraining for Language Understanding / arXiv preprint arXiv:1906.08237 (2019)

  77. [학술지] Vu, M. A. T. / 2018 / A shared vision for machine learning in neuroscience / Journal of Neuroscience 38 (7) : 1601 ~ 1607

  78. [학술대회] Wang, Alex / 2018 / GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding / EMNLP 2018 2018 : 353

  79. [학술대회] Wang, Alex / 2019 / Superglue: A stickier benchmark for general-purpose language understanding systems / Advances in Neural Information Processing Systems : 3261 ~ 3275