Interactive manipulation of visuomotor contingencies - A visual stimulation paradigm to investigate depth cue integration in static and dynamic conditions

The primary goal of the visual system is to build an accurate representation of the outside three-dimensional (3D) world extracting valuable information from the collection of signals impinging on the retina. In particular, the adult visual system uses a variety of strategies to optimally combine the available depth cues (e.g., binocular disparity, motion, texture gradient, shading. . .) to obtain robust estimates about objects properties in the 3D environment. This combination of sensory information is so efficient that it works even if the cues are noisy and/or conflictual. Considering that, so far cue combination processes have been (mostly) studied under static conditions. Yet, we are not static mannequins and we do not live in a static environment. We are inherently dynamic, living in an ever-changing world that we constantly explore and dynamically interact with. Furthermore, many experimental evidences pointed out that brain heavily relies upon interaction with external stimuli, which eventually molds our perception. During growth, dynamic exploration allows us acquiring sensorimotor abilities and this action-perception loop induces pivotal changes to facilitate perceptual learning, adaptation, and development. With these ideas in mind, the general objective of my PhD research is to investigate how the dynamic interaction with properties of visual stimuli shapes and influences perceptual processing, and whether experiencing new sensorimotor contingencies may change how brain combines information provided by different cues in a way that accounts for the modifications that these cues undergo. Specifically, I addressed the three following main scientific questions: (Q1) are visual depth cue integrated differently under static and dynamic conditions? (Q2) Does a visuomotor training with a stimulus composed of conflictual cues about the orientation of a 3D surface induce a reweighting mechanism that affects in turn static perception? (Q3) Which are the neural mechanisms and cortical areas underlying possible differences in visual cue processing? The first step to answer these questions was the development of a novel approach to administer dynamic visual stimulation – in real time – contingently to subject’s action, by exploiting the potentialities of 3D graphics engines and VR technologies. In particular, I have designed two procedures to statically and dynamically investigate how a subject judges the orientation of a 3D surface in space when this is defined by conflictual depth cues. For the last objective, I took advantage of the excellent temporal resolution provided by electroencephalography (EEG) techniques for conducting a characterization of the visual evoked response to the same stimuli used in the behavioral experiments. The results I found indicate that: (A1) perception of 3D oriented surfaces are different when it occurs statically than when it is possible for the subject to continuously interact with it in real-time. Visuomotor interaction with conflictual visual stimuli seems to (A2) change the relative weights given to depth cues, triggering a mechanism of visual cue-reweighting, which in the literature it has been investigated under static conditions, only, and considered as a feedback-driven learning process, e.g., taking place in presence of cue-reinforcing haptic feedback, only. Behavioral differences were also supported by the analysis of the EEG recordings, which led to the (A3) characterization of the spatio-temporal dynamics of brain activity in response to texture and binocular disparity cues. Preliminary results highlight different latencies in the processing of distinct visual depth cues under static conditions. A more comprehensive analysis of the neural responses under interactive conditions would require further specific investigation. The proposed experimental approach adopted in my research represents a further demonstration in favor of a potential paradigm shift in the methodologies used to investigate perception-in-action, supporting the idea of Continuous Psychophysics, which could stand alongside classical ``yes-no'' or ``forced choice'' approaches. Graphics engines and VR technologies turn out to be excellent tools for the implementation of visual stimulation tasks in dynamic contexts. All these achievements are directly exploitable for developing perceptual and motor training tasks to improve functional vision in subjects with visual impairments (e.g., strabismus, stereo blindness, amblyopia), but offers also novel solutions for the assessment of stereo vision, for a quantitative, and not only qualitative, measure of performance improvement, which are still lacking in classical clinical practice.