Practical use of natural language processing in education technology

The main focus of this thesis is the development and integration of new functionalities based on natural language processing for Beagle Learning. Beagle Learning is a Boston-based education technology startup that developed its own educative methodology and is looking to improve the current production NLP API endpoint and extend its capabilities to support the needs of their production application that is being used by thousands of students. The main goal of this work was to analyze, design, develop and integrate the solution of three main tasks that were needed as features in the next planned versions of the Beagle application: -Question type classification -Question deduplication -Question Productivity Index scoring automation For these tasks, multiple NLP approaches were adopted and compared to discover the best solutions that would yield satisfactory results in terms of Beagle’s needs. The algorithms were developed and tested using data provided by the company, where some of the data needed to be manually tagged prior to the research work. The thesis consists of three main parts – firstly, a context has been provided for the reader to understand the matters and the approaches. Then, each of the respective tasks is introduced in its own research block, finally followed by the API implementation and integration testing chapter. As the result, the first two of the three tasks have been successfully completed and implemented. As for the QPI problem, series of challenges and limitations have been identified that will be helpful in the future attempts to find a solution.