Word embeddings and their meaning

Embedding models represent techniques that convert texts into numerical vectors in a low-dimensional space, capturing the semantic and syntactic properties of the text. They were developed out of the need to represent textual data numerically so that it can be efficiently processed by machine learning algorithms. Practical applications of embedding models include serving as inputs for classification and predictive models, machine translation, information retrieval, and other applications where understanding the meaning and structure of text is essential. In this dissertation, I focus on three tasks that expand the practical use of embeddings. The first part is devoted to predicting the potential citation impact of scientific articles; using embeddings derived from abstracts, I build models that estimate whether an article may achieve high citation counts in the future. The second part introduces a novel method for explaining embedding models, which precisely determines how individual words contribute to the prediction results, thereby providing a clear interpretation of the model. The third part addresses the explanation of semantic similarity, demonstrating how distributional statistics combined with lexical knowledge enable a better understanding of the relationships between words in the embedding space. The results of this work have the potential to improve the accuracy and efficiency of classification and predictive models in scientific analytics, while also contributing to a deeper understanding of the mechanisms by which text models process natural language.