Comparison of generative artificial intelligence translations

The primary objective of this bachelor’s thesis is to compare the capabilities of generative artificial intelligence models Grok, ChatGPT and ChatGPT Plus and neural machine translation systems DeepL and Google Translate in translating specialized texts between Czech and English. In the theoretical section, the principles and history of both technologies are presented, their advantages and limitations are described and the BLEU and METEOR metrics used for automatic evaluation of translation quality are explained. In the practical section, specialized texts of varying lengths from three academic fields were selected and translated from Czech into English and vice versa using the tools mentioned above. Automatic metrics, human evaluation and a questionnaire survey examining user preferences and usage rates were employed to assess the quality of these translations. The main results show that the Grok model achieved the highest average BLEU and METEOR scores, while ChatGPT approached these values particularly on shorter text segments. The neural translators DeepL and Google Translate showed less consistent performance on shorter texts. Overall, it was found that translation quality strongly depends on the text’s subject area and the direction of translation. Text length does not significantly affect translation quality. Human evaluations corresponded with the automatic metrics. The survey revealed results similar to those obtained by the author’s metric-based and subjective assessments. Furthermore, a slightly higher preference for using neural machine translation over generative artificial intelligence was observed.