A Framework for User Representation in Virtual Reality: From Hand Interaction to Full Body Embodiment

Virtual, augmented and mixed reality, known as extended reality, have not only transformed the way we perceive and engage with digital content, but have also brought about a paradigm shift in human-computer interactions. In the last decade, virtual reality had a widespread success, thanks to the release of low cost head mounted displays that can provide immersive experiences and intuitive and effective interfaces. This technology has infinite potential and the fields of application are many: physical or cognitive rehabilitation, medical and military training and simulation, learning and teaching, and gaming are just examples. Despite this, achieving a truly immersive and natural user experience remains a complex and multifaceted challenge and current virtual reality systems often fall short in adequately representing users within the virtual space and providing seamless, intuitive interaction. Most of the applications developed usually come with a strong limitation: the absence of a proper representation of the user’s body. Indeed, vision has an important role in the way we perceive and interact with the surrounding environment: We strongly rely on vision in our daily lives, and for every action we accomplish, we expect to receive feedback. Adding a partial or full representation of the user’s body, more or less realistic, would take lots of direct and indirect advantages in terms of self motion estimation and sense of presence, immersion and embodiment in virtual reality. The concept of embodiment in virtual reality is very solid nowadays even if in practical application a body is still missing. Even so, researchers are still studying its effects on the user in different use cases. By implementing a framework to represent the user and allow interaction through hand tracking and controller solution, the primary aim of this research is to address the aforementioned challenges by developing and evaluating advanced methodologies for integrating full body tracking and interaction with the virtual environment into virtual reality systems.