A Deep Learning Based Approach for Biomedical Named Entity Recognition Using Multitasking Transfer Learning with BiLSTM, BERT and CRF

Abstract

The named entity recognition (NER) is a method for locating references to rigid designators in text that fall into well-established semantic categories like person, place, organisation, etc., Many natural language e applications, like summarization of text, question–answer models and machine translation, always include NER at their core. Early NER systems were quite successful in reaching high performance at the expense of using human engineers to create features and rules that were particular to a certain domain. Currently, the biomedical data is soaring expeditiously and extracting the useful information can help to facilitate the appropriate diagnosis. Therefore, these systems are widely adopted in biomedical domain. However, the traditional rule-based, dictionary based and machine learning based methods suffer from computational complexity and out-of-vocabulary (OOV)issues Deep learning has recently been used in NER systems, achieving the state-of-the-art outcome. The present work proposes a novel deep learning based approach which uses Bidirectional Long Short Term (BiLSTM), Bidirectional Encoder Representation (BERT) and Conditional Random Field mode (CRF) model along with transfer learning and multi-tasking model to solve the OOV problem in biomedical domain. The transfer learning architecture uses shared and task specific layers to achieve the multi-task transfer learning task. The shared layer consists of lexicon encoder and transformer encoder followed by embedding vectors. Finally, we define a training loss function based on the BERT model. The proposed Multi-task TLBBC approach is compared with numerous prevailing methods. The proposed Multi-task TLBBC approach realizes average accuracy as 97.30%, 97.20%, 96.80% and 97.50% for NCBI, BC5CDR, JNLPBA, and s800 dataset, respectively.