Skip to main content
SpringerLink
Log in

Joint multi-view character embedding model for named entity recognition of Chinese car reviews

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Named entity recognition (NER) has always been an important research task in information extraction and knowledge graph construction. Due to the randomness of Chinese user-generated reviews, character substitution and informal expression are very common. Its widespread phenomenon leads to that Chinese car reviews NER is still a major challenge. In this paper, we propose a joint multi-view character embedding model for Chinese NER (JMCE-CNER) of car reviews. Firstly, deeper character features are extracted from pronunciation, radical, and glyph views to generate the multi-view character embedding. Secondly, a car domain dictionary is constructed for providing accurate word-level information. Thirdly, the multi-view character embedding and the word-level embedding are jointly fed into the deep learning model to perform the Chinese car reviews NER. The experimental datasets of Chinese car reviews are obtained by manual annotation, containing four types of entities, namely brand, model, attribute and structure of the car. The experimental results on the Chinese car review datasets demonstrate that our proposed model achieves the optimal performance compared with the other state-of-the-art models. Furthermore, the model substantially reduces the impact of character substitution and informal expression on performing NER tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, Anning Wang, upon reasonable request.

References

  1. Xu X, Wang X, Li Y, Haghighi M (2017) Business intelligence in online customer textual reviews: understanding consumer perceptions and influential factors. Int J Inf Manage 37(6):673–683

    Article  Google Scholar 

  2. Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46

    Article  Google Scholar 

  3. Liu Z, Qin C, Zhang Y (2020) Mining product competitiveness by fusing multisource online information. Decis Support Syst 143(5):113477

    Google Scholar 

  4. Qi J, Zhang Z, Jeon S, Zhou Y (2016) Mining customer requirement from online reviews: a product improvement perspective. Soc Sci Electron Publ 53:951–963

    Google Scholar 

  5. Goyal A, Gupta V, Kumar M (2018) Recent named entity recognition and classification techniques: a systematic review. Comput Sci Rev 29:21–43

    Article  Google Scholar 

  6. Li D, Yan L, Yang J, Ma Z (2022) Dependency syntax guided bert-bilstm-gam-crf for chinese ner. Expert Syst Appl 196:116682

    Article  Google Scholar 

  7. Sharma R, Morwal S, Agarwal B, Chandra R, Khan MS (2020) A deep neural network-based model for named entity recognition for hindi language. Neural Comput Appl 32(20):16191–16203

    Article  Google Scholar 

  8. Derczynski L, Maynard D, Rizzo G, Erp MV, Gorrell G, Troncy R, Petrak J, Bontcheva K (2015) Analysis of named entity recognition and linking for tweets. Inf Process Manage 51:32–49

    Article  Google Scholar 

  9. Kainan J, Xin L, Rongchen Z (2021) Overview of chinese domain named entity recognition. Comput Eng Appl 57:1–15

    Google Scholar 

  10. Wang Y, Lu L, Wu Y, Chen Y (2022) Polymorphic graph attention network for chinese ner. Expert Syst Appl, 117467

  11. Peng DL, Wang YR, Liu C, Chen Z (2020) TL-NER: a transfer learning model for chinese named entity recognition. Inf Syst Front 22(1):1291–1304

    Article  Google Scholar 

  12. Khalifa M, Shaalan K (2019) Character convolutions for arabic named entity recognition with long short-term memory networks. Comput Speech Lang 58:335–346

    Article  Google Scholar 

  13. Gui T, Ma R, Zhang Q, Zhao L, Huang X (2019) Cnn-based chinese ner with lexicon rethinking. In: Twenty-eighth international joint conference on artificial intelligence IJCAI-19, pp 4982–4988

  14. Shan Z, Rui L, Zhiping C (2022) Survey of chinese named entity recognition. J Front Comput Sci Technol 16(2):296

    Google Scholar 

  15. Zhang N, Li F, Xu G, Zhang W, Yu H (2019) Chinese ner using dynamic meta-embeddings. IEEE Access 92:103133

    Google Scholar 

  16. Wang Q et al (2019) Incorporating dictionaries into deep neural networks for the chinese clinical named entity recognition. J Biomed Inform 92:103133

    Article  Google Scholar 

  17. Fang Z, Qiang Z et al (2021) Referent graph embedding model for name entity recognition of chinese car reviews. Knowl Based Syst 233:107558

    Article  Google Scholar 

  18. Asgari-Chenaghlu M, Feizi-Derakhshi MR, Farzinvash L, Balafar M, Motamed C (2022) Cwi: a multimodal deep learning approach for named entity recognition from social media using character, word and image features. Neural Comput Appl 34(3):1905–1922

    Article  Google Scholar 

  19. Li Y, Du G, Xiang Y, Li S, Chen H (2020) Towards chinese clinical named entity recognition by dynamic embedding using domain-specific knowledge. J Biomed Inform 106:103435

    Article  Google Scholar 

  20. Gaio M, Moncla L (2017) Extended named entity recognition using finite-state transducers: An application to place names. In: International conference on advanced geographic information systems, applications, and services, pp 15–20

  21. Ling L et al (2018) An attention-based bilstm-crf approach to document-level chemical named entity recognition. Bioinformatics 34(8):1381–1388

    Article  Google Scholar 

  22. Li J, Meng K (2021) MFE-NER: Multi-feature fusion embedding for chinese named entity recognition. arXiv preprint arXiv:2109.07877

  23. Liu Z, Zhu C, Zhao T (2010) Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In: International conference on advanced intelligent computing theories & applications, pp 634–640

  24. Zhao J, Xie X, Xu X, Sun S (2017) Multi-view learning overview: recent progress and new challenges. Inform Fus 38:43–54

    Article  Google Scholar 

  25. Jia X, Jing XY, Zhu X, Cai Z, Hu CH (2021) Co-embedding: a semi-supervised multi-view representation learning approach. Neural Comput Appl 34(6):4437–4457

    Article  Google Scholar 

  26. Ding Z, Shao M, Fu Y (2018) Robust multi-view representation: A unified perspective from multi-view learning to domain adaption. In: Twenty-seventh international joint conference on artificial intelligence IJCAI-18, pp 5434–5440

  27. Guo Q, Guo Y (2022) Lexicon enhanced chinese named entity recognition with pointer network. Neural Comput Appl 34:14535–14555

    Article  Google Scholar 

  28. Xiaofeng M, Wei W, Aiping X (2020) Incorporating token-level dictionary feature into neural model for named entity recognition. Neurocomputing 375:43–50

    Article  Google Scholar 

  29. Nie Y, Zhang Y, Peng Y, Yang L (2022) Borrowing wisdom from world: modeling rich external knowledge for chinese named entity recognition. Neural Comput Appl 34(6):4905–4922

    Article  Google Scholar 

  30. Hkiri AOE, Mallat S, Zrigui M (2016) Improving coverage of rule based ner systems. In: International conference on information & communication technology & accessibility (ICTA), pp 1–6

  31. Gerner M, Nenadic G, Bergman CM (2010) Linnaeus: a species name identification system for biomedical literature. BMC Bioinformatics 11(1):85

    Article  Google Scholar 

  32. Pande SD, Kanna RK, Qureshi I et al (2022) Natural language processing based on name entity with n-gram classifier machine learning process through ge-based hidden markov model. Mach Learn Appl Eng Educ Manag 2(1):30–39

    Google Scholar 

  33. Patil N, Patil A, Pawar B (2020) Named entity recognition using conditional random fields. Proced Comput Sci 167:1181–1188

    Article  Google Scholar 

  34. Tarasova O, Rudik A, Biziukova NY, Filimonov D, Poroikov V (2022) Chemical named entity recognition in the texts of scientific publications using the naïve bayes classifier approach. J Chemin 14(1):1–12

    Article  Google Scholar 

  35. Morwal S (2012) Named entity recognition using hidden markov model (HMM). Int J Comput Vision 1(4):15–23

    Google Scholar 

  36. Teixeira J, Sarmento L, Oliveira EC (2011) A bootstrapping approach for training a ner with conditional random fields. In: Progress in artificial intelligence,15th Portuguese conference on artificial intelligence, pp 664–678

  37. Mcdonald R, Pereira F (2005) Identifying gene and protein mentions in text using conditional random fields. BMC Bioinformatics 6:6

    Article  Google Scholar 

  38. Wang J, Lin C, Li M, Zaniolo C (2020) Boosting approximate dictionary-based entity extraction with synonyms. Inf Sci 530(1):1–21

    Google Scholar 

  39. Tran VC, Nguyen NT, Fujita H, Hoang DT, Hwang D (2017) A combination of active learning and self-learning for named entity recognition on twitter using conditional random fields. Knowl Based Syst 132(15):179–187

    Article  Google Scholar 

  40. Shen Y, Yun H, Lipton Z, Kronrod Y, Anandkumar A (2017) Deep active learning for named entity recognition. In: Proceedings of the 2nd workshop on representation learning for NLP, pp 252–256

  41. Lin Y, Hong L, Yi L, Li X, Anwar MW (2015) Biomedical named entity recognition based on deep neutral network. Int J Hybrid Inform Technol 8(8):279–288

    Article  Google Scholar 

  42. Li P, Dong R, Wang Y, Chou J, Ma W (2017)Leveraging linguistic structures for named entity recognition with bidirectional recursive neural networks. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 2664–2669

  43. Gridach M (2017) Character-level neural network for biomedical named entity recognition. J Biomed Inform 70:85–91

    Article  Google Scholar 

  44. Devlin J, Chang M.W, Lee K, Toutanova K (2018) BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  45. Peters M, Neumann M, Iyyer M, Gardner M, Zettlemoyer L (2018) Deep contextualized word representations. arXiv preprint arXiv:1802.05365

  46. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Adv Neural Inform Process Syst 26

  47. Li X, Zhang H, Zhou XH (2020) Chinese clinical named entity recognition with variant neural structures based on BERT methods. J Biomed Inform 107(5):103422

    Article  Google Scholar 

  48. Wang J, Xu W, Fu X, Xu G, Wu Y (2020) ASTRAL: adversarial trained LSTM-CNN for named entity recognition. Knowl Based Syst 197:105842

    Article  Google Scholar 

  49. Akbik A, Blythe D, Vollgraf R (2018) Contextual string embeddings for sequence labeling. In: Proceedings of the 27th International conference on computational linguistics, pp 1638–1649

  50. Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991

  51. Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th annual meeting of the association for computational linguistics, vol 1, pp 1064–1074

  52. Zhang Y, Yang J (2018) Chinese NER using lattice LSTM. arXiv preprint arXiv:1805.02023

  53. Li S.Y, Jiang Y, Zhou Z.H (2014) Partial multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, pp 1968–1974

  54. Liu J, Gao L, Guo S, Ding R, Thiruvady D (2021) A hybrid deep-learning approach for complex biochemical named entity recognition. Knowl Based Syst 221:106958

    Article  Google Scholar 

  55. Lee L, Lu Y (2021) Multiple embeddings enhanced multi-graph neural networks for chinese healthcare named entity recognition. IEEE J Biomed Health Inform 25(7):2801–2810

    Article  Google Scholar 

  56. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  57. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A.N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inform Process Syst 30

  58. Lafferty J, Mccallum A, Pereira F (2001) Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Processing 18th international conference on machine learning, pp 282–289

  59. Forney GD (1993) The viterbi algorithm. In: Proceedings of the IEEE, vol 61, pp 268–278

  60. Dong C, Zhang J, Zong C, Hattori M, Hui D (2016) Character-based lstm-crf with radical-level features for chinese named entity recognition. In: Natural language understanding and intelligent applications, pp 239–250

Download references

Acknowledgements

This work was supported by grants from the National Natural Science Foundation of China (Nos. 72101078, 72071060, 72101075, 72201087, 72171069, and 72188101) and the Fundamental Research Funds for the Central Universities (NOs. JZ2021HGTA0131 and JZ2022HGTB0286).

Author information

Authors and Affiliations

  1. School of Management, Hefei University of Technology, Hefei, 230009, China

    Jiaming Ding, Anning Wang, Shuangyao Zhao & Qiang Zhang

  2. Key Laboratory of Process Optimization and Intelligent Decision-making, Ministry of Education, Hefei, 230009, China

    Jiaming Ding, Anning Wang, Shuangyao Zhao & Qiang Zhang

  3. Weichai Power Co., Ltd, Weifang, 261061, China

    Wenping Xu

Authors
  1. Jiaming Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenping Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anning Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuangyao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anning Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A Analysis of time complexity

Appendix A Analysis of time complexity

First, we analyze our model’s time complexities. Then, the time complexity of our model is compared to the state-of-the-art models described in the main body.

Some important variables are defined as follows: l: the maximum text length; \({d_1}\): character embedding size; \({d_2}\): pronunciation/radical/glyph/word embedding size; \({d_3}\): hidden layer dimension of LSTM; \(\lambda\): the number of hidden layers (BERT); m: the number of labels; N: the total data amount; t: the data amount of a batch; n: the number of iterations; g: the maximum number of glyphs of the Chinese characters.

1.1 A. 1 Time complexity of the JMCE-CNER model

In this section, we analyze the time complexity of the components in the JMCE-CNER model, which is divided into four parts: embedding presentation layer, Bi-LSTM layer, attention mechanism layer, and CRF layer.


Embedding presentation layer. First, the character embedding is generated by BERT pre-trained model, whose time complexity is \(O\left( {\lambda {l^2}{d_1} + \lambda ld_1^2} \right)\); second, the pronunciation/radical/word embedding is generated by the word2vec model, whose time complexities are \(O\left( {ld_2^2} \right)\); Third, the glyph embedding is generated by the word2vec model and Bi-LSTM layer, whose time complexity is \(O\left( {gld_2^2} \right)\). Therefore, the time complexity of embedding presentation layer is \(O\left( {\lambda {l^2}{d_1} + \lambda ld_1^2 + gld_2^2} \right)\).


Bi-LSTM layer. First, the time complexity of each LSTM cell is \(O\left( {d_3^2 + \left( {{d_1} + 4{d_2}} \right) {d_3}} \right)\); second, the time complexity of calculating LSTM output is \(O\left( {{d_3}m} \right)\). Therefore, the time complexity of Bi-LSTM layer is \(O\left( {ld_3^2 + l\left( {{d_1} + 4{d_2}} \right) {d_3} + l{d_3}m} \right)\).


Attention mechanism layer. This layer consists of three steps: similarity calculation, softmax calculation and weighted summation. First, the time complexity of similarity calculation is \(O\left( {{l^2}{d_3}} \right)\); second, the time complexity of softmax calculation is \(O\left( {{l^2}} \right)\); Third, the time complexity of weighted summation is also \(O\left( {{l^2}{d_3}} \right)\). Therefore, the time complexity of attention mechanism layer is \(O\left( {{l^2}{d_3}} \right)\).


CRF layer. During the training stage, the CRF layer needs to calculate \(\sum \nolimits _{{\tilde{y}} \in {Y_x}} {{e^{Score\left( {S,{\tilde{y}}} \right) }}}\), where \({Y_x}\) is all possible tag sequence (\({m^l}\) in total). By using dynamic programming algorithm, its time complexity can be reduced to \(O\left( {l{m^2}} \right)\). In the inference stage, the Viterbi algorithm is used to find the optimal path, and the time complexity is also \(O\left( {l{m^2}} \right)\). Therefore, the time complexity of CRF layer is \(O\left( {l{m^2}} \right)\).


Overall. Considering all the above time complexity, the time complexity of the JMCE-CNER model can be represented as follows:

$$\begin{aligned} O\left( {nNl\left( {\lambda l{d_1} + \lambda d_1^2 + gd_2^2 + d_3^2 + \left( {{d_1} + 4{d_2}} \right) {d_3} + {d_3}m + l{d_3} + {m^2}} \right) } \right) \end{aligned}$$
(A.1)

1.2 A.2 Comparison with the state-of-the-art models

First, we give the time complexity of each state-of-the-art model. Then, the differences of time complexity between our model and the state-of-the-art models are analyzed.

1.2.1 A.2.1 Time complexities of the state-of-the-art models

The time complexity of each state-of-the-art model can be expressed as:

$$\begin{aligned}{} & {} \mathrm{{CR-CNER}}:O\left( {nNl\left( {d_1^2r + d_3^2 + {d_1}{d_3} + {d_3}m + {m^2}} \right) } \right) \end{aligned}$$
(A.2)
$$\begin{aligned}{} & {} \mathrm{{MFE-NER}}:O\left( {nNl\left( {\lambda l{d_1} + \lambda d_1^2 + d_3^2 + \left( {{d_1} + 2{d_2}} \right) {d_3} + {d_3}m + {m^2}} \right) } \right) \end{aligned}$$
(A.3)
$$\begin{aligned}{} & {} \mathrm{{BBMC}}:O\left( {nNl\left( {\lambda l{d_1} + \lambda d_1^2 + d_3^2 + {d_1}{d_3} + {d_3}m + l{d_3} + {m^2}} \right) } \right) \end{aligned}$$
(A.4)
$$\begin{aligned}{} & {} \mathrm{{ME - MGNN}}:O\left( {nNl\left( {\left( {d_1^2 + d_2^2} \right) u + {{\left( {{d_1} + 2{d_2}} \right) }^2} + d_3^2 + \left( {{d_1} + 2{d_2}} \right) {d_3} + {d_3}m + {m^2}} \right) } \right) \end{aligned}$$
(A.5)

where u is the number of convolution kernels, and r is the maximum number of radicals of the Chinese character.

Therefore, the size ordering of time complexity is shown as:

$$\begin{aligned} O\left( {\mathrm{{ME - MGNN}}} \right)> O\left( {\mathrm{{JMCE - CNER}}} \right)> O\left( {\mathrm{{MFE - NER}}} \right)> O\left( {\mathrm{{BBMC}}} \right) > O\left( {\mathrm{{CR - CNER}}} \right) \end{aligned}$$
(A.6)

1.2.2 A.2.2 Comparative analysis

As shown in Eq. (A.6), the time complexity of the JMCE-CNER model is the second-highest, trailing only the ME-MGNN model. The time complexity of the ME-MGNN model is increased due to the employment of an adapted gated graph sequence neural network (GGSNN) in it. Compared with the CR-CNER model, the BERT pre-trained model is used by the JMCE-CNER, MFE-NER, and BBMC models and the increased time complexity of these models is \(O\left( {nNl\left( {\lambda l{d_1} + \lambda d_1^2} \right) } \right)\). The use of the BERT pre-trained model will inevitably increase the time complexity. The BERT is essentially a tool for generating more accurate character vectors; therefore, these models outperform the CR-CNER model. Furthermore, the JMCE-CNER model has a higher time complexity than the MFE-NER and BBMC models, due to more features of the characters are considered. The BBMC model considers only character features and the MFE-NER model considers character, glyph, and phonetic features. However, the JMCE-CNER model extracts deep character information and enhanced semantic information by using multiple embedding methods to fuse character, pronunciation, radical, and glyph features. Therefore, the performance of the JMCE-CNER model is better than those of the state-of-the-art models but increasing the time complexity.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, J., Xu, W., Wang, A. et al. Joint multi-view character embedding model for named entity recognition of Chinese car reviews. Neural Comput & Applic 35, 14947–14962 (2023). https://doi.org/10.1007/s00521-023-08476-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08476-2

Keywords

Search

Navigation