Skip to main content
SpringerLink
Log in

FastText and XGBoost Content-Based Classification for Employment Web Scraping

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12416))

Included in the following conference series:

Abstract

The purpose of this paper is to present the design and results of experiments that focus on universal, autonomous data extraction (web scraping) system fed by publicly available online job listings. In particular, methods of automated crawling, preprocessing and classifying data from job offers will be presented together with the aggregation of the acquired data stored in large-scale, structured databases. We tested two models to classify the content of job portals: fastText and XGBoost. We obtained promising results in the experimental phase, with 88% accuracy by both methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmadi, A., Fotouhi, M., Khaleghi, M.: Intelligent classification of web pages using contextual and visual features. Appl. Soft Comput. 11(2), 1638–1647 (2011)

    Article  Google Scholar 

  2. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)

    Article  Google Scholar 

  3. Chen, T., Guestrin, C.: XGboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)

    Google Scholar 

  4. Drozda, P., Talun, A., Bukowski, L.: Emplobot - design of the system. In: Proceedings of the 28th International Workshop on Concurrency, Specification and Programming, Olsztyn, Poland, 24–26th September 2019 (2019)

    Google Scholar 

  5. Dumais, S., Chen, H.: Hierarchical classification of web content. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 256–263 (2000)

    Google Scholar 

  6. Dziwiński, P., Bartczuk, Ł., Paszkowski, J.: A new auto adaptive fuzzy hybrid particle swarm optimization and genetic algorithm. J. Artif. Intell. Soft Comput. Res. 10(2), 95–111 (2020)

    Article  Google Scholar 

  7. Gabryel, M.: The bag-of-words method with different types of image features and dictionary analysis. J. UCS 24(4), 357–371 (2018)

    MathSciNet  Google Scholar 

  8. Gabryel, M., Grzanek, K., Hayashi, Y.: Browser fingerprint coding methods increasing the effectiveness of user identification in the web traffic. J. Artif. Intell. Soft Comput. Res. 10(4), 243–253 (2020)

    Article  Google Scholar 

  9. Gabryel, M., Przybyszewski, K.: The dynamically modified BoW algorithm used in assessing clicks in online ads. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2019. LNCS (LNAI), vol. 11509, pp. 350–360. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20915-5_32

    Chapter  Google Scholar 

  10. Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016)

  11. Koren, O., Hallin, C.A., Perel, N., Bendet, D.: Decision-making enhancement in a big data environment: application of the k-means algorithm to mixed data. J. Artif. Intell. Soft Comput. Res. 9(4), 293–302 (2019)

    Article  Google Scholar 

  12. Korytkowski, M., Senkerik, R., Scherer, M.M., Angryk, R.A., Kordos, M., Siwocha, A.: Efficient image retrieval by fuzzy rules from boosting and metaheuristic. J. Artif. Intell. Soft Comput. Res. 10(1), 57–69 (2020)

    Article  Google Scholar 

  13. Kumar, R., Jain, A., Agrawal, C.: Survey of web crawling algorithms. Adv. Vis. Comput.: Int. J. (AVC) 1(2/3) (2014)

    Google Scholar 

  14. Ludwig, S.A.: Applying a neural network ensemble to intrusion detection. J. Artif. Intell. Soft Comput. Res. 9(3), 177–188 (2019)

    Article  Google Scholar 

  15. Mahdi, D.A.F., Ahmed, R.K.A.: A new technique for web crawling in multimedia web sites. Int. J. Comput. Eng. Res. 4(2) (2014)

    Google Scholar 

  16. Malhotra, R., Sharma, A.: Quantitative evaluation of web metrics for automatic genre classification of web pages. Int. J. Syst. Assur. Eng. Manag. 8(2), 1567–1579 (2017)

    Article  Google Scholar 

  17. Tambouratzis, G., Vassiliou, M.: Swarm algorithms for NLP - the case of limited training data. J. Artif. Intell. Soft Comput. Res. 9(3), 219–234 (2019)

    Article  Google Scholar 

  18. Vijayarani, S., Suganya, M.E.: Web crawling algorithms–a comparative study. Int. J. Sci. Adv. Res. Technol. 2(10) (2016)

    Google Scholar 

Download references

Acknowledgements

This work is a part of the Emplobot project number POIR.01.01.01-00-1135/17 “Development of autonomous artificial intelligence using the learning of deep neural networks with strengthening, automating recruitment processes” funded by the National Centre for Research and Development.

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Computer Science, University of Warmia and Mazury in Olsztyn, Olsztyn, Poland

    Pawel Drozda

  2. Emplocity Ltd., Warszawa, Poland

    Arkadiusz Talun, Pawel Drozda & Leszek Bukowski

  3. Department of Intelligent Computer Systems, Czȩstochowa University of Technology, Czȩstochowa, Poland

    Rafał Scherer

Authors
  1. Arkadiusz Talun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pawel Drozda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leszek Bukowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rafał Scherer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pawel Drozda .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  6. Electrical and Computer Engineering, University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Talun, A., Drozda, P., Bukowski, L., Scherer, R. (2020). FastText and XGBoost Content-Based Classification for Employment Web Scraping. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2020. Lecture Notes in Computer Science(), vol 12416. Springer, Cham. https://doi.org/10.1007/978-3-030-61534-5_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61534-5_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61533-8

  • Online ISBN: 978-3-030-61534-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation