Deliverable 3.1. Scalable NLP pipelines

The Intelcomp NLP pipeline can be defined as a collection of tools that apply the requested
transformations to unstructured textual data, which will be used by the Intelcomp services
(document classification, subcorpus generation, topic modeling, etc.) as a preliminary step to
process the datasets of interest. It has been designed to carry out standard text preprocessing
tasks (e.g. n-grams detection, keywords extraction, lemmatization, etc) in a High Performance
Computing environment, allowing the efficient and scalable processing of large amounts of
documents. The final version of the pipeline will be deployed over the HPC infrastructure
provided by the Barcelona Supercomputing Center and fully integrated with Intelcomp's Data
Space. This document should serve not only as a report of the work performed by the
members of WP3 but also as a complete guide for the targeted users and operators of the
pipeline.