Srovnání efektivity LLM při detekci specifických tříd zranitelností v jazyce C

This bachelor’s thesis experimentally compares the effectiveness of selected large language mo- dels in detecting specific classes of security vulnerabilities in C source code. Two approaches are compared: fine-tuning of the encoder-only model CodeBERT and using the decoder-only model Llama 3 via Few-Shot Chain-of-Thought prompting without any retraining. Both ap- proaches are applied to the detection of CWE-119 and CWE-476 vulnerabilities from the MSR BigVul dataset, which contains 23,543 code samples after preprocessing. Results show that fine-tuned CodeBERT for CWE-119 achieves Recall of 83.5%, Precision of 91.5%, and MCC of 0.87, placing it on par with the best published approaches on this dataset. For CWE-476, which has a substantially smaller training set, the model achieved Recall of 50.0% and MCC of 0.64, illustrating the impact of class imbalance under limited positive sample conditions. Llama 3 without fine-tuning achieved critically low Recall in both categories and is therefore unsuitable for production security analysis. The thesis discusses the strengths and weaknesses of both approaches and their practical applicability.