Vyhodnocení agenta SARSA v testu algoritmického IQ

Evaluating the intelligence of artificial agents has long taken an anthropocentric route, such as the Turing test, but it is not without problems. Its binary nature is not ideal for comparing very similar systems where exact metrics are needed. Such a metric is offered by the AIQ test, which is the main point of this thesis. This bachelor thesis is concerned with the analysis of the performance of reinforcement learning agents, specifically the SARSA and SARSA(λ) algorithms, and their subsequent evaluation using the AIQ test, which serves as a practical approximation to the theory of universal intelligence. The thesis has a theoretical section that introduces the two main topics by synthesizing key literature. The most important sources for the topic of evaluating artificial systems are Legg & Veness (2013) and Vadinsky's dissertation (2018b). For reinforcement learning, Sutton & Barto (2014) was the main source for the synthesis, describing each element of reinforcement learning in depth. Putting the topic in context is necessary to understand the work as a whole. I discuss the practical part of the thesis at the end of chapter two, where I describe how I implemented the agents, which I further elaborate in chapter three. The last chapter is devoted to the practical experiments, where I ask the research questions and analyze the results. •How does SARSA and its SARSA(λ) extension perform relative to other agents tested in the AIQ? •How does SARSA and its extension SARSA(λ) perform specifically against Q-learning agents? •What is the effect of each parameter on the results? The experimental results show an interesting phenomenon that would be worth investigating more in the future, that the SARSA and SARSA(λ) agents perform at different levels, with the SARSA(λ) agent performing slightly worse than the baseline version of SARSA. Apart from the phenomenon mentioned above, the agent behaved as expected and the performance was weaker than the Q-learning agent. It is because of the assumed behaviour that occurred that we could strengthen the argument for the validity of the AIQ test. The analysis of the effect of parameters revealed that parameter values can significantly affect the performance of agents especially at the extremes at the specified interval. The paper concludes by presenting the results and outlining avenues for possible follow-up research on this issue.