Gestural Interaction in Extended Reality

This thesis focuses on gesture-based interaction, an important topic of Human-Computer Interaction, within the context of extended reality environments. Recent advancements in sensing technologies, computational power, and extended reality platforms have significantly increased the % added the interest in gesture recognition research. Despite this progress, several challenges remain, including the continuous and accurate detection of gestures and the development of systems capable of real-time adaptation and robust performance in diverse contexts. To address these issues, this work provides a comprehensive review of the state-of-the-art in continuous gesture recognition detection, analyzing the methods proposed, identifying their strengths and limitations and discussing the problems related to their evaluation, providing an analysis of all the publicly available benchmarks. The thesis also introduces novel methods for skeleton-based continuous hand gesture detection and a novel benchmark for evaluating the task, including heterogeneous gestures designed to interact with extended reality applications. The novel methods integrate multi-task learning and enhanced feature extraction. The results obtained on our benchmark show that our methods outperform state-of-the-art techniques in terms of detection accuracy and number of false positives. These contributions provide a bridge between theoretical advancements and practical implementations, highlighting the transformative potential of gesture recognition in real-world scenarios. Additionally, data augmentation techniques were introduced to enhance the robustness and adaptability of recognition systems. The gesture recognition methods developed have been demonstrated practically in industrial extended reality environments, showing how gesture-based interaction can improve user experience, operational efficiency, and workflow design. The outcomes of this work contribute to the understanding and development of gesture-based interaction, setting a foundation for future research. Key directions for further exploration include addressing scalability challenges, improving adaptability to diverse users and environments, and deepening the integration of human-centric factors to fully realize the potential of gesture recognition in extended reality and beyond.