Novel Insights into Infrared Thermography Analysis: from Autonomic Modulations to Contactless Psychophysiological State Assessment

INFRARED Thermography (IRT) is a technique to convert the infrared radiation emitted by objects above absolute zero into an image of temperature distribution. This method allows contactless temperature measurement of an entire scene from a distance. With the huge benefit of not having any electrodes or wires to collect data, IRT has been utilized in psychophysiology to record variations in surface skin temperature and infer the subject’s mental state. This doctoral thesis introduces novel tools to analyze thermal signals, establish patterns distinguishing various affective states, and understand the underlying physiology behind these variations. The ultimate objective is to assess the subject’s mental state using a single contactless and portable device, providing objective and non-intrusive indicators of clinical conditions, potentially leading to more accurate diagnoses and treatment assessments. Skin temperature is influenced by various physiological phenomena, such as cardiovascular activity and perspiration, which have been linked to affective states through different Autonomic Nervous System (ANS) dynamics correlates, such as Heart Rate Variability (HRV) and Electrodermal Activity (EDA). Therefore, it is possible to gain important insights into the physiological processes connected to emotional states by monitoring temperature trends throughout the entire face of subjects over time and space. The first phase of the presented work involves the development of algorithms to extract and process one-dimensional thermal signals from facial regions, while extracting informative and reliable features in the time domain. Two studies were conducted using this methodology. The first study proposed infrared thermography as a contactless alternative to wearable devices for stress detection, it investigates the contribution given by thermal features to the autonomic feature typically used in relation to stress and employs a multivariate approach to automatically recognize stress. This study was carried out through subject-independent classifications based on the Support Vector Machine model with an embedded recursive feature elimination algorithm. Furthermore, this thesis proposes an advanced approach to explore the autonomic processes underlying facial skin temperature regulation in response to emotional stimuli, quantifying the nonlinear relationships between thermal, HRV, and EDA time series. In this context, a cross-mapping approach, based on chaos theory, was used to investigate how cardiovascular and sweat gland dynamics (in terms of HRV and EDA) influence thermal modulation of specific regions, at rest and in stressful conditions, with a focus on gender differences. Finally, an original approach was proposed to explore thermograms as two-dimensional signals using Independent Component Analysis. This approach involves fewer assumptions and supports a model-based decomposition of the signal, potentially revealing new information. By skipping the step of region selection, the analysis becomes fully automatic and more robust. Moreover, different facial regions provide complementary information on ANS regulation, facilitating the retrieval of multiple physiological correlates directly from the thermograms, providing a thorough understanding of the person’s physiological condition for increased specificity. Along with the proposed methodologies to analyse the thermal signal of the face, I present experimental tests on groups of healthy volunteers whose mental states were altered as part of ad-hoc constructed experimental protocols. In conclusion, this thesis presents a thorough and in-depth analysis of the facial thermographic signal with regard to its connection to mental states. It employs both conventional techniques for one-dimensional signal extraction from specific facial regions and time domain feature extraction, and it applies advanced nonlinear analysis to explore the physiological phenomena that influence temperature modulation. Moreover, this thesis introduces a novel approach to analyse the entire thermogram, to unveil autonomic correlates that are commonly associated with emotional states. The presented work deepens our understanding of the complex relationship between facial temperature patterns and affective responses, it introduces new Infrared Thermography (IRT) processing tools while acknowledging potential limitations and providing suggestions for further improvements.