Supporting humans with autonomous systems: deep learning for activity, state and environment recognition

Autonomous systems can support human activities in several situations, ranging from daily tasks to specific working activities. All these systems have in common the need of understanding their environment and the state of the human interacting with them. Once such information has assessed by the system, it can either perform autonomously actions, suggest them or simply present additional information about the activity or environment to the user. It is necessary to consider that different activities need different levels of confidence in the decision making process. Critical systems, such as vehicles in autonomous driving scenarios, need the highest possible accuracy given that they take autonomous decisions; on the other hand, critical safe systems, need a lower level of confidence given that they can just provide a feedback to the user to ease the decision making process. All this brings up several challenges given the high variability of both activities, environments and people, making traditional computer vision approaches less adequate. This is due also to the face that often it is not possible to identify clearly the input variables of the system, given the high correlation between them or the high dimensionality of the input space. Machine learning has shown promising results in such scenarios and in particular deep learning is the evolution of machine learning that has shown most effective results in terms of quality and performance of the learning tasks. Deep Learning can cope well with variable scenarios by scaling to highly dimensional decision spaces that typically suffer the problem of feature identification and selection. This work will show some developed applications in activity, state and environment recognition, presenting how human decisions can be supported by autonomous systems using deep learning techniques. The first part of the thesis will present the state of the art solutions to the aforementioned problems along with the latest deep learning techniques. In the second part of this work, we will describe in depth three different developed applications in activity, state and environment recognition. Finally we will present possible future works along with remaining open research questions.