Abstract
Latent Dirichlet Allocation (LDA) and its variant topic models have been widely used for performing text mining tasks. Topic models sometimes produce incoherent topics having noisy words with high probabilities. The reason is that topic model suffers from binary weighting of terms, sparsity and lack of semantic information. In this work, a fuzzy document representation is used within the framework of topic modeling that resolves these problems. Fuzzy document representation uses the concept of Fuzzy Bag of Word (FBoW) that maps each document to a fixed length fuzzy vector of basis terms. Each basis term in the fuzzy vector belongs to all documents in the dataset with some membership degree. Latent Dirichlet Allocation is tailored to use fuzzy document representation and results are compared with regular LDA and term weighted LDA over short and long text documents on document clustering and topic quality tasks. LDA with fuzzy representation generates more coherent topics than other two methods. It also outperforms the other methods on document clustering for short documents and produces comparable results with term weighted LDA for long documents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Blei, D.M.: Introduction to probabilistic topic modeling. Commun. ACM 55, 77–84 (2011)
Wang, D., Zhu, S., Li, T., Gong, Y.: Multi-document summarization using sentence-based topic models (2010)
Yang, G., Wen, D., Kinshuk, Chen, N.S., Sutinen, E.: A novel contextual topic model for multi-document summarization. Expert Syst. Appl. 42, 1340–1352 (2015)
Akhtar, N., Siddique, B.: Hierarchical visualization of sport events using Twitter. J. Intell. Fuzzy Syst. 32(4), 2953–2961 (2017)
Akhtar, N., Zubair, N., Kumar, A., Ahmad, T.: Aspect based sentiment oriented summarization of hotel reviews. Procedia Comput. Sci. 115, 563–571 (2017)
Lin, C., He, Y.: Joint sentiment/topic model for sentiment analysis. In: Proceeding of the 18th ACM Conference on Information and Knowledge Management – CIKM 2009, p. 375 (2009)
Zhai, Z., Liu, B., Xu, H., Jia, P.: Constrained LDA for grouping product features in opinion mining. In: Huang, J.Z., Cao, L., Srivastava, J. (eds.) PAKDD 2011. LNCS (LNAI), vol. 6634, pp. 448–459. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20841-6_37
Ho, K.T., Bui, Q.V., Bui, M.: Dynamic social network analysis using author-topic model. In: Hodoň, M., Eichler, G., Erfurth, C., Fahrnberger, G. (eds.) I4CS 2018. CCIS, vol. 863, pp. 47–62. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93408-2_4
Hu, Y., Zhai, K., Eidelman, V., Boyd-Graber, J.: Polylingual tree-based topic models for translation domain adaptation (2015)
Li, X., Zhang, A., Li, C., Ouyang, J., Cai, Y.: Exploring coherent topics by topic modeling with term weighting. Inf. Process. Manag. 54, 1345–1358 (2018)
Wilson, A.T., Chew, P.A.: Term weighting schemes for latent dirichlet allocation. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL (2010)
Kai, Y., Yi, C., Zhenhong, C., Ho-fung, L., Raymond, L.: Exploring topic discriminating power of words in latent dirichlet allocation. In: Proceedings of COLING 2016, 26th International Conference on Computational Linguistics Technical Paper, pp. 2238–2247 (2016)
Blei, D., Ng, A., Jordan, M.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Zhao, R., Mao, K.: Fuzzy bag-of-words model for document representation. IEEE Trans. Fuzzy Syst. 26, 794–804 (2018)
Jia, H., Li, Q.: Fuzzy bag-of-topics model for short text representation. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11305, pp. 473–482. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04221-9_42
Wang, T., Cai, Y., Leung, H.F., Cai, Z., Min, H.: Entropy-based term weighting schemes for text categorization in VSM. In: Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI (2016)
Google Code Archive - long-term storage for Google code project hosting. https://code.google.com/archive/p/word2vec/. Accessed 15 Mar 2019
Griffiths, T.L.: Gibbs sampling in the generative model of latent Dirichlet allocation. Unpublished note (2002). https://citeseerx.ist.psu.edu/
Gilks, W.R.: Markov chain monte carlo. Encyclopedia of biostatistics. Adv. online Publ. (2005)
UCI Machine Learning Repository: Reuters-21578 Text categorization collection data set. https://archive.ics.uci.edu/ml/datasets/reuters-21578+text+categorization+collection. Accessed 15 Mar 2019
Manning, C.D., Raghavan, P., Schütze, H.: Introduction to information retrieval introduction. Nat. Lang. Eng. 16, 100–103 (2008)
Newman, D., Lau, J., Grieser, K., Baldwin, T.: Automatic evaluation of topic coherence. In: Proceedings of NAACL-HLT (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Akhtar, N., Sufyan Beg, M.M., Javed, H. (2019). Topic Modelling with Fuzzy Document Representation. In: Singh, M., Gupta, P., Tyagi, V., Flusser, J., Ören, T., Kashyap, R. (eds) Advances in Computing and Data Sciences. ICACDS 2019. Communications in Computer and Information Science, vol 1046. Springer, Singapore. https://doi.org/10.1007/978-981-13-9942-8_54
Download citation
DOI: https://doi.org/10.1007/978-981-13-9942-8_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9941-1
Online ISBN: 978-981-13-9942-8
eBook Packages: Computer ScienceComputer Science (R0)