Utilizing Large Language Models for Text Analysis

This bachelor's thesis focuses on the use of large language models for text analysis. A set of three prompts was developed in the thesis to evaluate signs of manipulation, assess adherence to ethical principles, and identify emotions in text. The thesis provides a theoretical definition of the concepts of manipulation, ethical principles, emotions, and the related expressivity, and describes current approaches to their detection using LLMs. The functionality of the prompts was tested on selected datasets (MentalManip, Hate Speech and Offensive Language, Emotions) using the primary model Llama 3.1 8B. Compatibility testing of the prompts was also conducted on a smaller data sample with the models GPT-4o, Gemini 2.0 Flash, and Claude 3.7 Sonnet. The results of testing showed that the model, when using the prompt for manipulation detection, correctly identified most manipulative texts (recall 92,31%), but frequently misclassified non-manipulative texts as manipulative. This issue has also been reported by other researchers. The prompt for hate speech detection achieved an overall classification accuracy of 86,5%. The Llama 3.1 8B model, in combination with the emotion detection prompt, identified 172 primary and 202 secondary emotions in the test data. Negative emotions prevailed in the list of detected emotions. The overall classification accuracy when using emotion categories from the dataset was low (47,77%). The prompt is best used in situations where precise classification into predefined categories is not required. It is important to emphasize that LLMs should serve only as a tool, and the final evaluation and responsibility rest with the user.