Hodnocení výsledků klasifikačních úloh

This thesis briefly introduces selected classification methods, with the main emphasis placed on methods for evaluating the results of classification tasks. It describes confusion matrices for both binary and multi-class tasks, along with derived metrics, as well as more advanced measures of performance and agreement, such as Cohen's kappa, Matthews correlation coefficient, and Brier score. The practical part of the thesis contains an analysis of several datasets, including real-world data from various domains as well as artificially generated datasets for both binary and multi-class classification. Selected models are applied to each dataset, and their outputs are evaluated using the metrics introduced. The aim of the thesis is to assess the suitability of these metrics for evaluating classification tasks across different types of data. Depending on the characteristics of the data, some metrics may provide differing or misleading insights. The analysis shows that AUC (Area Under Curve) is a reliable metric for measuring model discriminative ability, especially in binary tasks. Precision and recall proved crucial in interpreting performance in imbalanced classes, as each metric captures a different aspect of classification error. To assess the quality of probabilistic predictions, the Brier score and logarithmic loss were used. In tasks with more than two classes, evaluation was more complex, but metrics such as AUC and logarithmic loss still yielded comparable results. Furthermore, the role of Lasso regularization was explored. It had a positive impact on AUC and Brier score values, especially for imbalanced data, where it contributed to improved recall for underrepresented classes. After applying Lasso, the model results were more stable and less prone to overfitting, leading to better generalization on imbalanced data. In logistic regression and linear discriminant analysis, Lasso improved the model’s ability to distinguish between classes, whereas decision trees showed a decrease in performance due to dimensionality reduction eliminating some important variables. The main contribution of this work is a systematic comparison of evaluation metrics and their interpretations, which can serve as a guide for selecting appropriate metrics for specific classification tasks.