The Potential for Misusing LLMs to Generate Manipulative Messages in Social Engineering Attacks

This bachelor’s thesis examines the relationship between social engineering and large language models. It focuses on phishing messages as a specific form of manipulative digital communication. The aim of the thesis is to compare selected phishing messages from an open corpus with synthetically created training messages generated using ChatGPT. The thesis does not examine the technical functionality of phishing, message deliverability, or the real behaviour of users. The subject of the analysis is the textual properties of the messages, especially language quality, communication pretext, psychological manipulation techniques, and the degree of detection-related suspiciousness. The theoretical part defines social engineering in the context of cybersecurity, describes phishing as a communication-based attack, and explains the psychological mechanisms that may increase the persuasiveness of manipulative messages. It also introduces large language models as a technology capable of generating stylistically high-quality and contextually adapted texts. The practical part is based on a comparative content analysis of twenty messages: ten messages from an open phishing corpus and ten synthetic messages created using ChatGPT. The results show that the synthetic messages demonstrate higher and more consistent language quality than the corpus messages, while at the same time containing similar manipulation techniques, especially authority, obligation, responsibility, and time pressure. The thesis concludes that linguistic correctness alone cannot serve as an indicator of a message’s legitimacy and that the detection of social-engineering communication must take greater account of content-related and situational context.