Large Language Models: Performance in High School Czech Language and Mathematics

In recent years, large language models (LLMs) have been developing at an exceptional pace, and their use in both everyday life and professional practice has grown significantly. Although these models are primarily developed in the United States and Asia, they can communicate in many languages, including Czech. However, the question remains as to how well current LLMs truly understand the Czech language and to what extent they can solve high-school mathematics problems presented in Czech. The aim of this bachelor thesis is to evaluate the level of linguistic and mathematical capabilities of the latest generation of LLMs and to compare their performance with the results of older models. Testing was conducted using tasks adapted from standardized high-school didactic tests and SCIO tests, enabling direct comparison. The analysis incorporates both state-of-the-art closed-source models, such as GPT and Gemini, and powerful open-source models, such as Gemma, Qwen, and Llama. The performance of individual models is cross-compared and compared against data from the bachelor thesis of K. Hrabáčová (Hrabáčová, 2024), which maps the performance of older generations of LLMs. The results demonstrate a significant improvement in the capabilities of current LLMs in both the Czech language and mathematics, although certain weaknesses persist, mainly in the domain of grammatical error detection. This thesis thus contributes to a deeper understanding of the capabilities and limitations of current LLMs and illustrates the rapid evolution of this technological field.