Skip to main content
Log in

Study on suitability and importance of multilayer extreme learning machine for classification of text data

  • Foundations
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The dynamic Web, which contains huge number of digital documents, is expanding day by day. Thus, it has become a tough challenge to search for a particular document from such a large volume of collections. Text classification is a technique which can speed up the search and retrieval tasks and hence is the need of the hour. Aiming in this direction, this study proposes an efficient technique that uses the concept of connected component (CC) of a graph and Wordnet along with four established feature selection techniques [e.g., TF-IDF, Chi-square, Bi-Normal Separation (BNS) and Information Gain (IG)] to select the best features from a given input dataset in order to prepare an efficient training feature vector. Next, multilayer extreme learning machine (ML-ELM) (which is based on the architecture of deep learning) and other state-of-the-art classifiers are trained on this efficient training feature vector for classification of text data. The experimental work has been carried out on DMOZ and 20-Newsgroups datasets. We have studied the behavior and compared the results of different classifiers using these four important feature selection techniques used for classification process and observed that ML-ELM achieved the maximum overall F-measure of 72.28 % on DMOZ dataset using TF-IDF as the feature selection technique and 81.53 % on 20-Newsgroups dataset using BNS as the feature selection technique compared to other state-of-the-art classifiers which signifies the usefulness of deep learning used by ML-ELM for classifying the text data. Experimental results on these benchmark datasets show the stability and effectiveness of our approach over other competing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://www.worldwidewebsize.com/.

  2. http://nlp.stanford.edu/IR-book/html/htmledition.

  3. http://ai.stanford.edu/~rion/parsing/minipar_viz.html.

  4. http://computation.pa.msu.edu/NO/ConnCompPresentation.html.

  5. http://wordnet.princeton.edu/.

  6. Determined through experiment, iterating the values over a range and considered the value at which best results were obtained.

  7. http://www.dmoz.org.

  8. http://qwone.com/~jason/20Newsgroups/.

  9. http://www.gabormelli.com/RKB/F-Measure.

References

  • Bai Z, Huang G-B, Wang D, Wang H, Westover MB (2014) Sparse extreme learning machine for classification. IEEE Trans Cybern 44(10):1858–1870

    Article  Google Scholar 

  • Chen R-C, Hsieh C-H (2006) Web page classification based on a support vector machine using a weighted vote schema. Expert Syst Appl 31(2):427–435

    Article  Google Scholar 

  • Ding S, Xu X, Nie R (2014) Extreme learning machine and its applications. Neural Comput Appl 25(3–4):549–556

    Article  Google Scholar 

  • Ding S, Zhang N, Xu X, Guo L, Zhang J (2015) Deep extreme learning machine and its application in EEG classification. Math Probl Eng 2015

  • Forman G (2003) An extensive empirical study of feature selection metrics for text classification. J Mach Learn Res 3:1289–1305

    MATH  Google Scholar 

  • Gomez JC, Moens M-F (2012) Hierarchical classification of web documents by stratified discriminant analysis. In: Multidisciplinary information retrieval. Springer, pp 94–108

  • Gopal S, Yang Y (2013) Recursive regularization for large-scale classification with hierarchical and graphical dependencies. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 257–265

  • Guo G, Wang H, Bell D, Bi Y, Greer K (2003) Knn model-based approach in classification. In: On the move to meaningful internet systems, (2003) CoopIS, DOA, and ODBASE. Springer, pp 986–996

  • Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  • Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  • Huang G-B (2003) Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans Neural Netw 14(2):274–281

    Article  Google Scholar 

  • Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  • Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42:513–529

    Article  Google Scholar 

  • Im Kim K, Park HR (2009) Svd-lda: a combined model for text classification. J Inf Process Syst 5(1):5–10

    Article  Google Scholar 

  • Kasun LLC, Zhou H, Huang G-B, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34

    Google Scholar 

  • Ke W (2012) Least information document representation for automated text classification. Proc Am Soc Inf Sci Technol 49(1):1–10

    MathSciNet  Google Scholar 

  • Klassen M, Paturi N (2010) Web document classification by keywords using random forests. In: Networked digital technologies. Springer, pp 256–261

  • Liang N-Y, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  • Lingras P, Butz C (2007) Rough set based 1-v-1 and 1-vr approaches to support vector machine multi-classification. Inf Sci 177(18):3782–3798

    Article  Google Scholar 

  • Li L, Song D, Liao L (2012) Vertical classification of web pages for structured data extraction. In: Information retrieval technology. Springer, pp 486–495

  • Liu X, Gao C, Li P (2012) A comparative analysis of support vector machines and extreme learning machines. Neural Netw 33:58–66

    Article  MATH  Google Scholar 

  • Manning CD, Raghavan P, Schütze H et al (2008) Introduction to information retrieval, vol 1. Cambridge university press, Cambridge

  • Mirza B, Kok S, Dong F (2016) Multi-layer online sequential extreme learning machine for image classification. In: Proceedings of ELM-2015 vol 1. Springer, pp 39–49

  • Oh H-S, Choi Y, Myaeng S-H (2011) Text classification for a large-scale taxonomy using dynamically mixed local and global models for a node. In: Advances in information retrieval. Springer, pp 7–18

  • Rifkin R, Yeo G, Poggio T (2003) Regularized least-squares classification. Nato Sci Ser Sub Ser III Comput Syst Sci 190:131–154

    Google Scholar 

  • Rujiang B, Xiaoyue W, Zewen H (2011) A novel web pages classification model based on integrated ontology. In: Software engineering, business continuity, and education. Springer, pp 1–10

  • Song Y, Roth D (2014) On dataless hierarchical text classification. In: AAAI. pp 1579–1585

  • Tang J, Deng C, Huang G-B, Hou J (2014) A fast learning algorithm for multi-layer extreme learning machine. In: International Conference on IEEE Image Processing (ICIP). pp 175–178

  • Wan CH, Lee LH, Rajkumar R, Isa D (2012) A hybrid text classification approach with low dependency on parameter by integrating k-nearest neighbor and support vector machine. Expert Syst Appl 39(15):11 880–11 888

    Article  Google Scholar 

  • Xue G-R, Xing D, Yang Q, Yu Y (2008) Deep classification in large-scale text hierarchies. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 619–626

  • Yang Y, Pedersen JO (1997) A comparative study on feature selection in text categorization. ICML 97:412–420

    Google Scholar 

  • Yang Y, Wu Q (2015) Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Trans Cybern 99:1–14

    Google Scholar 

  • Zhang J, Niu Y, Nie H (2009) Web document classification based on fuzzy k-nn algorithm,” In: CIS’09. International conference on computational intelligence and security, 2009, vol 1. IEEE, pp 193–196

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rajendra Kumar Roul.

Ethics declarations

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

None.

Additional information

Communicated by A. Di Nola.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roul, R.K., Asthana, S.R. & Kumar, G. Study on suitability and importance of multilayer extreme learning machine for classification of text data. Soft Comput 21, 4239–4256 (2017). https://doi.org/10.1007/s00500-016-2189-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2189-8

Keywords

Navigation