Performance Comparison of Large Language Models on Customer Complaint Text Classification

This thesis focuses on comparing the performance of selected open–source Large Language Models (LLMs) in the task of classifying ČSOB customer complaints. The main objective is to evaluate which models perform best when processing real data and to assess whether they can outperform existing feed–forward neural network–based solutions currently implemented in the bank. The work will also investigate whether there is sufficient benefit in moving to LLM to replace the existing solution, which is less demanding in terms of computational capacity. The results provide valuable insights into the possibilities of implementing open–source LLMs and its potential for increasing efficiency and automation in the processing of complaints in the banking sector. The theoretical knowledge is applied in the practical part of the thesis, which deals with the issue of fine tuning models using the LoRA technique, which makes it possible to reduce the demands on training capacity while maintaining high classification accuracy. The results of individual models determine their ability to improve the current classification of complaints based on their accuracy and processing speed. In this work, the data is processed in both Czech and English, with the aim of assessing whether the translation of complaints into English leads to an improvement in classification accuracy. For the fine tuning of LLMs, the Python programming language and libraries from the Hugging Face platform are used, which provide wide possibilities for working with open–source models.