Abstract
The development of tourism and technology has given rise to many online hotel booking services that allow users to leave reviews on hotels. Therefore, an analytical model that can comprehensively present the aspects and sentiments in user reviews is required. This study proposes the use of a long short-term memory (LSTM) model with an attention mechanism to perform aspect-based sentiment analysis. The architecture used also implements double fully-connected layers to improve performance. The architecture is used simultaneously for aspect extraction and sentiment polarity detection. Using 5200 Indonesian-language hotel-review data points with labels of five aspects and three sentiments, the model is trained with the configuration of hidden units, dropout, and recurrent dropout parameters in the LSTM layer. The best model performance resulted in a micro-averaged F1-measure value of 0.7628 using a hidden units parameter of 128, dropout parameter of 0.3, and recurrent dropout parameter of 0.3. Results show that the attention mechanism can improve the performance of the LSTM model in performing aspect-based sentiment analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Badan Pusat Statistik: Kunjungan Wisatawan Mancanegara per bulan Menurut Kebangsaan (Kunjungan), Badan Pusat Statistik (2021). https://www.bps.go.id/indicator/16/1470/1/kunjungan-wisatawan-mancanegara-per-bulan-menurut-kebangsaan.html
Startup Ranking: Countries - With the top startups worldwide. Startup Ranking (2021).  https://www.startupranking.com/countries
Liu, B.: Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers (2012)
Ma, Y., Peng, H., Cambria, E.: Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In: 32nd AAAI Conference Artificial Intelligence AAAI, vol. 2018, pp. 5876–5883 (2018)
Priyantina, R.A., Sarno, R.: Sentiment analysis of hotel reviews using Latent Dirichlet Allocation, semantic similarity and LSTM. Int. J. Intell. Eng. Syst. 12(4), 142–155 (2019). https://doi.org/10.22266/ijies2019.0831.14
Fernando, J., Khodra, M.L., Septiandri, A.A.: Aspect and opinion terms extraction using double embeddings and attention mechanism for Indonesian hotel reviews. In: Proceedings 2019 International Conference Advance Informatics Concepts, Theory, Appl. ICAICTA 2019 (2019). https://doi.org/10.1109/ICAICTA.2019.8904124
Yanuar, M.R., Shiramatsu, S.: Aspect extraction for tourist spot review in Indonesian language using BERT. In: 2020 International Conference Artificial Intelligence Information Communication ICAIIC, vol. 2020, pp. 298–302 (2020). https://doi.org/10.1109/ICAIIC48513.2020.9065263
Setiawan, E.I., Ferry, F., Santoso, J., Sumpeno, S., Fujisawa, K., Purnomo, M.H.: Bidirectional GRU for targeted aspect-based sentiment analysis based on character-enhanced token-embedding and multi-level attention. Int. J. Intell. Eng. Syst. 13(5), 392–407 (2020). https://doi.org/10.22266/ijies2020.1031.35
Bangsa, M.T.A., Priyanta, S., Suyanto, Y.: Aspect-based sentiment analysis of online marketplace reviews using convolutional neural network. IJCCS (Indonesian J. Comput. Cybern. Syst. 14(2), 123 (2020). https://doi.org/10.22146/ijccs.51646
Azhar, A.N.: Fine-tuning Pretrained Multilingual BERT Model for Indonesian Aspect-based Sentiment Analysis (2020)
Jayanto, R., Kusumaningrum, R., Wibowo, A.: Aspect-based sentiment analysis for hotel reviews using an improved model of long short-term memory, unpublished
Muhammad, P.F., Kusumaningrum, R., Wibowo, A.: Sentiment analysis using Word2vec and long short-term memory (LSTM) for Indonesian hotel reviews. Procedia Comput. Sci. 179(2020), 728–735 (2021). https://doi.org/10.1016/j.procs.2021.01.061
Smagulova, K., James, A.P.: A survey on LSTM memristive neural network architectures and applications. Eur. Phys. J. Spec. Top. 228(10), 2313–2324 (2019). https://doi.org/10.1140/epjst/e2019-900046-x
Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems, pp. 1–6 (2015)
Sun, X., Lu, W.: Understanding Attention For Text Classification, no. 1999, pp. 3418–3428 (2020). https://doi.org/10.18653/v1/2020.acl-main.312
Nawangsari, R.P., Kusumaningrum, R., Wibowo, A.: Word2vec for Indonesian sentiment analysis towards hotel reviews: an evaluation study. Procedia Comput. Sci. 157, 360–366 (2019). https://doi.org/10.1016/j.procs.2019.08.178
Roh, Y., Heo, G., Whang, S.E.: A survey on data collection for machine learning: a big data-AI integration perspective. IEEE Trans. Knowl. Data Eng. 33(4), 1328–1347 (2021). https://doi.org/10.1109/TKDE.2019.2946162
Mujilahwati, S.: Pre-Processing Text Mining Pada Data Twitter, Semin. Nas. Teknol. Inf. dan Komun., vol. 2016, no. Sentika, pp. 2089–9815 (2016)
Jurafsky, D., Martin, J.H.: Speech and Language Processing: An Introduction to Natural Language Processing,Computational Linguistics, and Speech Recognition. Prentice Hall (2021)
Najjichah, H., Syukur, A., Subagyo, H.: Pengaruh Text Preprocessing Dan Kombinasinya Pada Peringkas Dokumen Otomatis Teks Berbahasa Indonesia, J. Teknol. Inf., vol. XV, no. 1, pp. 1–11 (2019)
Rosid, M.A., Fitrani, A.S., Astutik, I.R.I., Mulloh, N.I., Gozali, H.A.: Improving Text Preprocessing for Student Complaint Document Classification Using Sastrawi, IOP Conf. Ser. Mater. Sci. Eng. 874, 1 (2020). https://doi.org/10.1088/1757-899X/874/1/012017
Kurniasari, L., Setyanto, A.: Sentiment analysis using recurrent neural network-lstm in bahasa Indonesia. J. Eng. Sci. Technol. 15(5), 3242–3256 (2020)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 13–39 (1997). https://doi.org/10.1007/978-1-4757-5388-2_2
Zhang, F., Fleyeh, H., Bales, C.: A hybrid model based on bidirectional long short-term memory neural network and Catboost for short-term electricity spot price forecasting, J. Oper. Res. Soc. 1–25 (2020). https://doi.org/10.1080/01605682.2020.1843976
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 2017(Nips), 5999–6009 (2017)
Parmar, R.: Common Loss functions in machine learning, Towards Data Science (2018). https://towardsdatascience.com/common-loss-functions-in-machine-learning-46af0ffc4d23
Shmueli, B.: Multi-Class Metrics Made Simple, Part II: the F1-score. Towards Data Science (2019). https://towardsdatascience.com/multi-class-metrics-made-simple-part-ii-the-f1-score-ebe8b2c2ca1
Acknowledgment
The data used in this study were generated at Intelligent Systems Laboratory at the Department of Informatics, Diponegoro University. The data are available on request from the corresponding author, Retno Kusumaningrum. Please Email retno@live.undip.ac.id to requests for access.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cendani, L., Kusumaningrum, R., Endah, S. (2023). Aspect-Based Sentiment Analysis of Indonesian-Language Hotel Reviews Using Long Short-Term Memory with an Attention Mechanism. In: Ben Ahmed, M., Abdelhakim, B.A., Ane, B.K., Rosiyadi, D. (eds) Emerging Trends in Intelligent Systems & Network Security. NISS 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 147. Springer, Cham. https://doi.org/10.1007/978-3-031-15191-0_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-15191-0_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15190-3
Online ISBN: 978-3-031-15191-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)