Neuromorphic Computing for Tactile Sensing Systems on Resource-Constrained Edge Devices

Artificial tactile sensing systems have emerged as a promising technology for enhancing human-machine interactions. Integrating artificial touch into machines enables them to perform complex tasks (i.e., texture sensation, dexterous manipulation, and locomotion) and interact more seamlessly with the environment. These systems typically consist of tactile sensors that provide mechanical/physical contact signals, followed by embedded algorithms on resource-constrained devices for signal acquisition and processing. However, in realworld unconstrained scenarios, these systems face substantial hardware challenges, including the need for fast and low-power/energy computations. To address this challenge, there has been growing interest in biologically inspired computing paradigms, particularly neuromorphic computing. Neuromorphic circuits mimic brain-like processing techniques, offering low-power signal processing through event-driven sensation and acquisition, asynchronous communication, and in memory computations, all while minimizing hardware requirements. This thesis explores the integration of neuromorphic computing techniques into tactile sensing systems for real-time classification of tactile patterns. In particular, the thesis emphasizes the direct processing of tactile signals with minimal preprocessing. We first developed a tactile sensing system comprising eight piezoelectric sensors and interface electronics, mounted on a biomimetic fingertip designed for use in robotic and prosthetic hands. This system was used to capture tactile texture signals under various experimental conditions. Specifically, we designed and 3D-printed eight artificial textures with varying degrees of coarseness, ranging from smooth to rough. The experiments involved evaluating different indentation forces and sliding/scanning velocities. Next, we introduce a neuromorphic approach consisting of two key components: 1) neural encoding of tactile signals and spike emission, and 2) a recurrent spiking neural network (RSNN) for spike-based processing and classification. The RSNN is trained using surrogate gradient descent to leverage the spatio-temporal features of the emitted spikes, enabling subsequent inference without relying on any hand-crafted features. The developed neuromorphic framework is employed first to study the impact of individual experimental conditions on classification accuracy, followed by a comprehensive evaluation considering all conditions together. Our results indicate that increasing the indentation force significantly enhances classification accuracy, while the effect of sliding velocity was minimal. We propose an optimization method to control the computational cost of the RSNN while maintaining its classification accuracy by tuning the refractory period parameter in spiking neurons. In particular, we found that adjusting this parameter between 1 and 10 ms during neural encoding reduces the computational cost (i.e., number of synaptic operations) of the RSNN, with only a small drop in accuracy. Subsequently, we implemented the optimized RSNN on the Intel Loihi neuromorphic chip for real-time tactile texture classification. By tuning the refractory period parameter on the Loihi chip, we observed a reduction in energy consumption per inference of approximately 32.5% (i.e. 7.93 micro Joules compared to 11.75 micro Joules for non-optimized networks). Finally, we compare the performance of the presented neuromorphic system with relevant existing machine learning (ML) and deep learning (DL) methods on edge devices for the same tactile texture use-case. We employed a low-power system based on the STM32 microcontroller and an ML accelerator based on the Coral dev board TPU to host the ML classifiers and DL networks. The evaluation involves the preprocessing stage within each approach and their associated classification networks. Experimental results show that the proposed neuromorphic system exhibits an order-of-magnitude gain in terms of energy consumption per inference compared to the other methods, while achieving a classification accuracy of 86.7%, highlighting its effectiveness in performing low-power tactile signal processing.