Regularizační Techniky pro Konvoluční Neuronové Sítě v Klasifikaci: Empirické Srovnání

This thesis investigates the effectiveness of regularization techniques for convolutional neural networks, specifically for image classification. Overfitting remains one of the biggest challenges when creating a working neural network, particularly if it is a deep neural network or when working with complex training data. A wide range of regularization techniques have been created, five were selected for evaluation in this thesis: Label Smoothing, DropBlock, SaliencyMix, Stochastic Weight Averaging (SWA) and Sharpness Aware Minimization (SAM). Experiments are conducted across two datasets: Oxford-IIIT Pet and STL10 and two architectures: ResNet-18 and a custom simple CNN. This allows for a fair comparison across different levels of model complexity and dataset difficulty. An additional experiment was conducted using 20% of the training data to test the techniques under the condition of limited training data. The results show that Label Smoothing achieved the best average test accuracy, while DropBlock achieved the best balance between test accuracy and generalization gap. SaliencyMix showed inconsistency, performing well on the simpler dataset and poorly on the more difficult one. SAM severely overfitted when used with ResNet-18, likely due to the short training length, while SWA underperformed relative to expectations, most likely due to the limited number of epochs. When faced with limited training data, data augmentation techniques, SaliencyMix, showed a large improvement over the other techniques, which is consistent with literature. The conclusion of this thesis is that no single regularization technique is the best across all datasets and architectures, and their selection needs to be carefully guided by every aspect of the neural network and dataset.