Regularization methods in Probabilistic Neural Networks and General Regression Neural Networks

This thesis investigates the application of regularization techniques in general regression neural networks (GRNN) and probabilistic neural networks (PNN). L1 and L2 regularization has been integrated into GRNN and PNN architectures using a Bayesian formulation of parameter estimation. In contrast to the standard kernel density estimation (KDE) approach, in this work the models were reformulated using a mixture of Gaussians for density estimation, allowing the models to move from a non-parametric to a parametric formulation with minimal changes in interpretation. The models were implemented in Python and their performance was evaluated on real-word datasets. In regression tasks, L1 and L2 regularization led to improvements in model performance, with mean square error reductions in the range of 3,02% to 4,04%. In classification tasks, accuracy improvements ranged from 0,36% to 4,44%. However, a slight performance decrease was observed in some datasets, suggesting that the effect of regularization is data-dependent.