Conversational agents in human-machine interaction: reinforcement learning and theory of mind in language modeling

This doctoral thesis addresses the challenges and advancements in the realm of Human-Machine Interaction, specifically focusing on the agency and misalignment of modern Large Language Models. Initially, we examined the potential for artificial agents to manifest agency within an environment inspired by Social Deduction Games, where Multi-Agent System and Reinforcement Learning shape the interactions. Our findings revealed that introducing a communication channel significantly im- proved agents’ performance, indicative of emergent decision-making abilities. Subsequently, the investigation shifted to the capability of machines to convey information in a manner comprehensible to humans. Through a Referential Game, we identified that agents, while capable of collaboration, struggled with performance when faced with knowledge asymmetry. To address this, we implemented a Multi-Agent Reinforcement Learning approach, aligning with contemporary solutions in the literature and show how it ultimately culminated in the issue of misalignment. In response, our final approach integrated elements from psychology and linguistics to propose a solution to both issues of agency and misalignment. We showed how our method improved communication accuracies solving the agency issue and mitigating the misalignment problem. Moreover, we highlight the environmental and interpretability advantages of our solution. We conclude by stressing the importance of interdisciplinary approaches to refine and understand the capabilities of artificial agents in communication-centric tasks.