Usage of Large Language Models for Interactive Chatbot Applications

The integration of Large Language Models (LLMs) into chatbot systems has shifted the human-machine interaction from rule-based programming to natural language communication. This thesis explores the current state of conversational AI systems and their performance on different data access tiers – static public data, dynamic public data, and private data and evaluates its performance across three distinct data access tiers: static public data, dynamic public data, and private data. For each tier, a dedicated chatbot application is developed, focusing on question answering and constraint-based generation tasks using prompt engineering techniques. Furthermore, it establishes an automatic evaluation pipeline that evaluates and compares the chatbot applications across multiple models. The evaluation demonstrates that the LLM-based chatbots perform more reliably on constraint-based generation tasks than in open-ended question answering, particularly when accessing external data sources. The findings highlight the importance of clear prompt structuring, retrieval strategies and model testing to enhance chatbot accuracy across varying data access tiers.