No-code webscraping with platform Make.com and its use with RAG database

This bachelor's thesis focuses on utilizing the Make.com automation platform for web scraping, storing extracted data in a RAG database, and integrating it with a chatbot powered by large language models (LLM). The goal is to evaluate the effectiveness of this no-code approach and measure the quality of generated responses using a RAG database compared to responses without it. The theoretical part explores the principles of web scraping, automation platforms, and RAG models, including data storage options using vector databases like Pinecone. It also examines the workings of large language models and metrics for evaluating response relevance, such as BLEU, METEOR, RAGAS, and BERTScore. The practical part focuses on designing and implementing a system that automates the retrieval of gaming data from selected sources (e.g., Steam API, IGDB) using Make.com. The collected data is stored in a RAG database and used by a chatbot to respond to user queries. The results of the experiment are analyzed based on test queries, comparing the quality of responses generated with and without RAG. The findings of this work provide a comprehensive insight into the potential of a no-code approach to web scraping and RAG databases. They demonstrate the extent to which this approach improves response relevance and its limitations when integrated with LLM.