Detection and explanation of misinformation patterns in fact-checked claims using LLMs

This bachelor's thesis explores automated approaches to combating misinformation. It proposes a method for modeling argumentation in fact-checked claims using entity graphs and large language models (LLMs). The source dataset consists of 50 fact-check reports from the PolitiFact website. The argumentation in each report was analysed during an annotation campaign conducted by a team from VŠE. Annotators classified misleading claims based on predefined misinformation patterns in a Catalogue of Misinformation Patterns and created brief summaries of the argumentation in each fact-check report. The automated misinformation pattern detection approach proposed in this thesis achieved an exact agreement with annotators in 46% of reports, with an additional 36% of cases where the model proposed partially matching misinformation patterns. Evaluation of a wide selection of LLMs demonstrated that the Gemini 2.5 Pro model, created by Google, performed best in argumentation modeling. The experiments revealed that some examined claims are more challenging to model automatically than others. Based on this observation, a set of parameters that determine the complexity of a fact-check report is proposed. This metric has the potential to serve as an indicator of how confident we can be in an LLM's automated decision. To improve the interpretability of argumentation in fact-check reports, a method for automatically sketching entity graphs using LLMs is presented. Preliminary experiments indicate that this approach shows promise, with automatically generated graphs often approaching the quality of those created manually by human annotators. This thesis demonstrates that it is possible to model argumentation within fact-check reports fully automatically. The proposed approach has the potential to be used for evaluating the quality of fact-check reports, which could enable fully automated refutation of examined problematic claims in the future. Additionally, it improves the interpretability of argumentation in fact-checked claims.