Možnosti testování moderního stacku pro kvalitu dat a strojové učení

This thesis investigates the design, implementation, and evaluation of an automated data processing architecture in which data quality validation is tightly integrated into a machine learning pipeline. The work addresses a challenge from current data-driven systems: machine learning models often rely on datasets that may contain inconsistencies or violations of domain rules, because traditional pipelines enforce systematic quality controls before model training is not automatically implemented. A modular framework was developed using Docker for reproducible environments to address this gap and software such as Apache Airflow for orchestration, Great Expectations for data validation, and PostgreSQL for storage and tracking of experimental outputs were used. The system incorporates adaptive logic capable of detecting data quality issues and triggering remediation steps automatically. During the research, set of corruption scenarios was created – including missing values, outliers, categorical errors, and conjoined multi-type corruptions so the quantification of how different data defects influence both data quality metrics and model performance was possible. Random Forest and Logistic Regression models were evaluated on corrupted and remediated datasets to measure degradation and recovery effects. The results demonstrate that while machine learning performance is relatively robust to moderate corruption, data validation metrics reveal substantial structural distortions that models fail to expose. The integrated approach provides a more reliable and transparent workflow, ensuring that machine learning is trained only on validated. The thesis concludes by outlining the strengths, limitations, and practical applicability of the proposed architecture.