Peiying Zhang
Skip Abstract Section
Abstract
Sentence similarity analysis has been applied in many fields, such as machine translation, the question answering system, and voice customer service. As a basic task of natural language processing, sentence similarity analysis plays an important role in many fields. The task of sentence similarity analysis is to establish a sentence similarity scoring model through multi-features. In previous work, researchers proposed a variety of models to deal with the calculation of sentence similarity. But these models do not consider the association information of sentence pairs, but only input sentence pairs into the model. In this article, we propose a sentence feature extraction model based on multi-feature attention. In addition, with the development of deep learning and the application of nature-inspired algorithms, researchers have proposed various hybrid algorithms that combine nature-inspired algorithms with neural networks. The hybrid algorithms not only solve the problem of decision-making based on multiple features but also improve the performance of the model. In the model, we use the attention mechanism to extract sentence features and assign weight. Then, the convolutional neural network is used to reduce the dimension of the matrix. In the training process, we integrate the firefly algorithm in the neural networks. The experimental results show that the accuracy of our model is 74.21%.
- M. M. Alobaedy, A. A. Khalaf, and I. D. Muraina. 2017. Analysis of the number of ants in ant colony system algorithm. In Proceedings of the 2017 5th International Conference on Information and Communication Technology (ICoIC’17). 1--5. DOI:https://doi.org/10.1109/ICoICT.2017.8074653Google Scholar
- Mohammed Alweshah and Salwani Abdullah. 2015. Hybridizing firefly algorithms with a probabilistic neural network for solving classification problems. Applied Soft Computing 35 (2015), 513--524. DOI:https://doi.org/10.1016/j.asoc.2015.06.018Google ScholarDigital Library
- C. An, J. Huang, S. Chang, and Z. Huang. 2016. Question similarity modeling with bidirectional long short-term memory neural network. In Proceedings of the 2016 IEEE 1st International Conference on Data Science in Cyberspace (DSC’16). 318--322. DOI:https://doi.org/10.1109/DSC.2016.13Google ScholarCross Ref
- H. Bing. 2017. Statistical machine translation algorithm based on improved neural network. In Proceedings of the 2017 International Conference on Robots Intelligent System (ICRIS’17). 294--297. DOI:https://doi.org/10.1109/ICRIS.2017.81Google ScholarCross Ref
- Kang Chen, Xiao Zhong Fan, Jie Liu, and Zheng Tao Yu. 2007. A new approach to compute the semantic similarity of Chinese question sentence. In Proceedings of the 2007 International Conference on Machine Learning and Cybernetics.Google ScholarCross Ref
- J. Fan, J. Jiao, W. Wu, and T. Zhao. 2015. A model-checking oriented modeling method for safety critical system. In Proceedings of the 2015 1st International Conference on Reliability Systems Engineering (ICRSE’15). 1--6. DOI:https://doi.org/10.1109/ICRSE.2015.7366490Google ScholarCross Ref
- Z. Gao, A. Feng, X. Song, and X. Wu. 2019. Target-dependent sentiment classification with BERT. IEEE Access 7 (2019), 154290--154299.Google ScholarCross Ref
- Felix A. Gers, Nicol N. Schraudolph, and Jurgen Schmidhuber. 2003. Learning precise timing with LSTM recurrent networks. Journal of Machine Learning Research 3, 1 (2003), 115--143.Google ScholarDigital Library
- P. Guo, X. Wang, and Y. Han. 2010. The enhanced genetic algorithms for the optimization design. In Proceedings of the 2010 3rd International Conference on Biomedical Engineering and Informatics, Vol. 7. 2990--2994. DOI:https://doi.org/10.1109/BMEI.2010.5639829Google ScholarCross Ref
- M. Hussain and W. K. Jenkins. 2017. Adaptive digital filtering using the bio-inspired firefly algorithm. In Proceedings of the 2017 51st Asilomar Conference on Signals, Systems, and Computers. 816--819. DOI:https://doi.org/10.1109/ACSSC.2017.8335460Google ScholarCross Ref
- N. Jiang, F. Tian, J. Li, X. Yuan, and J. Zheng. 2020. MAN: Mutual attention neural networks model for aspect-level sentiment classification in SIoT. IEEE Internet of Things Journal 7, 4 (2020), 2901--2913. DOI:https://doi.org/10.1109/JIOT.2020.2963927Google ScholarCross Ref
- Nan Jiang, Dong Xu, Jie Zhou, Hongyang Yan, Tao Wan, and Jiaqi Zheng. 2020. Toward optimal participant decisions with voting-based incentive model for crowd sensing. Information Sciences 512 (2020), 1--17. DOI:https://doi.org/10.1016/j.ins.2019.09.068Google ScholarDigital Library
- C. Kening, Z. Boming, W. Wenchuan, and S. Hongbin. 2011. An intelligent checking system for power system operation tickets. In Proceedings of the 2011 4th International Conference on Electric Utility Deregulation and Restructuring and Power Technologies (DRPT’11). 757--762. DOI:https://doi.org/10.1109/DRPT.2011.5993993Google ScholarCross Ref
- Neeraj Kumar, Naveen Chilamkurti, and Joel J. P. C. Rodrigues. 2014. Learning automata-based opportunistic data aggregation and forwarding scheme for alert generation in vehicular ad hoc networks. Computer Communications 39, 3 (2014), 22--32.Google ScholarDigital Library
- Claudia Leacock and Martin Chodorow. 1998. Combining local context and WordNet similarity for word sense identification. In WordNet: An Electronic Lexical Database, Vol. 49. MIT Press, Cambridge, MA, 265--287.Google Scholar
- Z. Li, H. Lin, W. Zheng, M. M. Tadesse, Z. Yang, and J. Wang. 2020. Interactive self-attentive Siamese network for biomedical sentence similarity. IEEE Access 8 (2020), 84093--84104.Google ScholarCross Ref
- H. Liu, X. Sun, L. Xu, S. Gu, and F. Sun. 2019. Short-term load forecasting based on Elman Neural Network optimized by Firefly Algorithm. In Proceedings of the 2019 IEEE Innovative Smart Grid Technologies—Asia (ISGT Asia’19), Vol. 02. 1425--1429. DOI:https://doi.org/10.1109/ISGT-Asia.2019.8881458Google ScholarCross Ref
- J. Ming, L. Zhang, J. Sun, and Y. Zhang. 2018. Analysis models of technical and economic data of mining enterprises based on big data analysis. In Proceedings of the 2018 IEEE 3rd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA’18). 224--227. DOI:https://doi.org/10.1109/ICCCBDA.2018.8386516Google Scholar
- S. Nejati. 2019. Testing cyber-physical systems via evolutionary algorithms and machine learning. In Proceedings of the 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST’19). 1. DOI:https://doi.org/10.1109/SBST.2019.00008Google ScholarDigital Library
- Y. Park. 2019. Recommending personalized search terms for assisting exploratory website search. In Proceedings of the 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL’19). 404--405. DOI:https://doi.org/10.1109/JCDL.2019.00091Google ScholarDigital Library
- X. Qi, S. Zhu, and H. Zhang. 2017. A hybrid firefly algorithm. In Proceedings of the 2017 IEEE 2nd Advanced Information Technology, Electronic, and Automation Control Conference (IAEAC’17). 287--291. DOI:https://doi.org/10.1109/IAEAC.2017.8054023Google ScholarCross Ref
- H. Ruan, Y. Li, Q. Wang, and Y. Liu. 2016. A research on sentence similarity for question answering system based on multi-feature fusion. In Proceedings of the 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI’16). 507--510. DOI:https://doi.org/10.1109/WI.2016.0085Google ScholarCross Ref
- M. Shao and D. Qian. 2016. The application of Levenshtein algorithm in the examination of the question bank similarity. In Proceedings of the 2016 International Conference on Robots Intelligent System (ICRIS’16). 422--424. DOI:https://doi.org/10.1109/ICRIS.2016.88Google ScholarCross Ref
- Z. Shao, Y. Li, and H. Zhang. 2020. Learning representations from skeletal self-similarities for cross-view action recognition. IEEE Transactions on Circuits and Systems for Video Technology 31, 1 (2020), 160--174. DOI:https://doi.org/10.1109/TCSVT.2020.2965574Google ScholarDigital Library
- A. Sharma. 2016. Algorithms simulating natural phenomena and hypercomputation. In Proceedings of the 2016 IEEE Students’ Conference on Electrical, Electronics, and Computer Science (SCEECS’16). 1--6. DOI:https://doi.org/10.1109/SCEECS.2016.7509337Google ScholarCross Ref
- M. Singh, R. Kumar, and I. Chana. 2019. Neural-based machine translation system outperforming statistical phrase-based machine translation for low-resource languages. In Proceedings of the 2019 12th International Conference on Contemporary Computing (IC3’19). 1--7. DOI:https://doi.org/10.1109/IC3.2019.8844915Google Scholar
- Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Targeted sentiment classification with attentional encoder network. In Artificial Neural Networks and Machine Learning—ICANN 2019: Text and Time Series. Lecture Notes in Computer Science, Vol. 11730. Springer, 93--103. DOI:https://doi.org/10.1007/978-3-030-30490-4_9Google ScholarCross Ref
- Y. Sun and F. Hu. 2018. Structure-based convolutional neural networks for Chinese sentence similarity measurement. In Proceedings of the 2018 11th International Symposium on Computational Intelligence and Design (ISCID’18), Vol. 1. 124--127. DOI:https://doi.org/10.1109/ISCID.2018.00035Google Scholar
- C. Wu, J. Wang, X. Wang, and Y. Ma. 2018. Research on Chinese sentence similarity calculation method based on multiple features. In Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic, and Automation Control Conference (IAEAC’18), Vol. 1. 1160--1165. DOI:https://doi.org/10.1109/IAEAC.2018.8577597Google ScholarCross Ref
- L. J. Wu. 2018. The development of the intelligent checking system for nuclear power plant 3D model. In Proceedings of the 2018 International Conference on Power System Technology (POWERCON’18). 4668--4673. DOI:https://doi.org/10.1109/POWERCON.2018.8601798Google ScholarCross Ref
- M. Y. Wang, C. L. Li, J. D. Sun, W. R. Xu, S. Gao, Y. H. Zhang, P. Wang, and J. L. Li. 2018. Learning sentences similarity by multi-head attention. In Proceedings of the 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC’18), Vol. 2. 16--19. DOI:https://doi.org/10.1109/ICNIDC.2018.8525664Google ScholarCross Ref
- Haipeng Yao, Huiwen Liu, and Peiying Zhang. 2018. A novel sentence similarity model with word embedding based on convolutional neural network. Concurrency and Computation: Practice and Experience 30, 23 (2018), e4415. DOI:https://doi.org/10.1002/cpe.4415Google ScholarCross Ref
- B. Ye, G. Feng, A. Cui, and M. Li. 2017. Learning question similarity with recurrent neural networks. In Proceedings of the 2017 IEEE International Conference on Big Knowledge (ICBK’17), Vol. 1. 111--118. DOI:https://doi.org/10.1109/ICBK.2017.46Google ScholarCross Ref
- P. Zhang, X. Huang, and M. Li. 2019. Disease prediction and early intervention system based on symptom similarity analysis. IEEE Access 7 (2019), 176484--176494. DOI:https://doi.org/10.1109/ACCESS.2019.2957816Google ScholarCross Ref
- W. Zhang, D. Zhao, W. Gong, Z. Li, Q. Lu, and S. Yang. 2015. Food image recognition with convolutional neural networks. In Proceedings of the 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, the 2015 IEEE 12th International Conference on Autonomic and Trusted Computing, and the 2015 IEEE 15th IInternational Conference on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom’15). 690--693. DOI:https://doi.org/10.1109/UIC-ATC-ScalCom-CBDCom-IoP.2015.139Google Scholar
- Z. Zhang, W. Su, and K. Zhou. 2019. Airborne radar sub array partitioning method based on artificial bee colony algorithm. In Proceedings of the 2019 IEEE 3rd Information Technology, Networking, Electronic, and Automation Control Conference (ITNEC’19). 484--489. DOI:https://doi.org/10.1109/ITNEC.2019.8729489Google Scholar
- Wenli Zhuang and Ernie Chang. 2017. Neobility at SemEval-2017 task 1: An attention-based sentence similarity model. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval’17). 164--169.Google ScholarCross Ref
Index Terms
- Hybridization between Neural Computing and Nature-Inspired Algorithms for a Sentence Similarity Model Based on the Attention Mechanism
Recommendations
Double Attention Mechanism for Sentence Embedding
Web Information Systems and ApplicationsAbstractThis paper proposes a new model for sentence embedding, a very important topic in natural language processing, using a double attention mechanism to combine of a recurrent neural network (RNN) and a convolutional neural network (CNN). First, the ...
Sentence Similarity-Based Source Context Modelling in PBSMT
IALP '10: Proceedings of the 2010 International Conference on Asian Language ProcessingTarget phrase selection, a crucial component of the state-of-the-art phrase-based statistical machine translation(PBSMT) model, plays a key role in generating accurate translation hypotheses. Inspired by context-rich word-sense disambiguation techniques,...
Bidirectional IndRNN malicious webpages detection algorithm based on convolutional neural network and attention mechanism
A convolutional neural network combined with attention mechanism and a parallel joint algorithm model (CATTB) of bidirectional independent recurrent neural network are proposed. The algorithm extracts the relocation feature and the “texture fingerprint” ...
Comments