GPT-2 for email classification of german emails in Azure

Siemens currently uses a machine learning system deployed in the Azure cloud platform that categorizes incoming emails in German into several categories, subcategories and assigns priority to them. However, full try-out of different models has not been performed, and despite great industry-wise performance of the current solution, it is not clear whether it can be improved. This thesis outlines the theoretical pillars required for understanding the practical part, in which it investigates whether improving performance is possible using a different model and an ensemble solution, tackles other relevant tasks such as assessing whether the data changes over time and presents results in comparison with the current solution. While the analysis shows that performance does not improve with a different, state-of-the-art model, it further strengthens the position of the current model as the optimal solution, and the suggested GPT-2 model may outperform the current DistilBERT in future use-cases. Additionally, the ensemble method, consisting of combining the current and proposed solutions, shows promising results in one of the three tasks, improving prediction accuracy by half a percent. Furthermore, several improvements in the code-base as well as further knowledge about the problem as a result of this experiment are likely to help the business in the future.