AI-powered competitive website search system

This bachelor's thesis aims to design and develop a program that helps users search for competing websites using artificial intelligence. To reach the goal, the main function of the program involves SEO keywords identification from the content of a given webpage with the help of large language models (LLM). First, the thesis defines the terms „keyword analysis“ and „web scraping“. It then presents a research on large language models and prompt engineering. To ensure the correct functionality of the program, various LLMs were analyzed and tested via API calls, with the final chosen model being Gemini 2.0 Flash. Based on the findings from the requirements analysis and a comparison of existing applications, the main functionalities of the program were defined. These functionalities served as the basis for designing the program’s implementation and user interface. The final application was developed in Python, using Selenium as a web scraping tool. For the implementation of the web interface and deployment of the application, the Streamlit framework was chosen. Finally, usability testing and a security test were conducted on the resulting application. The result of this thesis is an application that uses web scraping to extract HTML documents from a given URL and subsequently generates SEO keywords using a large language model. Users can further customize the output by adjusting the number and the language of the keywords.