Porovnání Klasifikačních Metod Strojového Učení Pomocí Monte Carlo Simulací

Although comparisons of classification methods are quite common in the literature, they often have a limited perspective, whether in terms of the selected methods, metrics, or datasets. This work tries to go in an unconventional direction by using a simulation study with Monte Carlo simulations. A total of eight scenarios are created for generating datasets with different settings for the number of observations. 500 datasets are created for each setting. First, the logistic regression estimates are evaluated in terms of bias and inconsistency and then the logistic regression is compared with XGBoost, random forests, artificial neural networks, and logistic regression with weight of evidence transformation of predictors using 7 selected metrics. The results show that the bias of coefficient estimates occurs in the presence of defined time dependence, normal distribution of noise, endogeneity, and nonlinearity, with large bias observed in small datasets. When it comes to comparing models, the biggest differences in predictive power of models are caused by the inclusion of (non)linearity in the data generation process and the size of the generated dataset. For small datasets and for large datasets in the presence of linearity, logistic regression performs the best, in other cases artificial neural networks perform the best.