Incorporating Contextual Information in Prediction Based Word Embedding Models

Abstract

A clustering-based language model is proposed for analyzing the performance of context sensitive word embedding models that uses contextual information. The construction of text readability and prediction models faces several shortcomings due to the complex nature and structure of language. The language structure is complicated all the more by words having vastly different meanings and interpretation based on the context in which it is used. This paper aims to resolve this issue by first clustering the sentences based on similarity and then performing word embedding separately on each of these clusters to obtain an enhanced outcome. This would serve to embed the same word separately in varying context as an improvement over the standard existing word embedding models provided by various prediction based models. Comparing the two approaches, our results have showed that clustering improves the performance of the model and discriminate the contextual information based on sense which leads to more accurate representation in vector form.