Searching Neural Architectures at the Edge

The present dissertation introduces a new approach towards the design of neural networks, in which an edge device automatically designs a neural architecture based on the locally collected data. The proposed approach enables a new paradigm of on-device, privacy-preserving neural network design, as sensitive data no longer needs to leave the local network. The proposed approach has been developed thanks to three key contributions. The first contribution, ColabNAS, established the feasibility of low-cost Hardware-Aware Neural Architecture Search (HW-NAS) by utilizing free online GPU services. The second, NanoNAS, eliminated the GPU dependency, enabling the search to run on standard CPUs and single board computers by incorporating self-awareness of the search platform's memory. The culminating contribution, GatewayNAS, achieves fully autonomous HW-NAS on an embedded IoT gateway. This framework introduces a dual-level hardware awareness, adapting the search process to the constraints of both the target sensor node and the gateway's own time and energy budget. The outcomes of a thorough experimental session confirm that on the Visual Wake Words dataset, a TinyML benchmark, the proposed approach can achieve state-of-the-art results by exploiting a search procedure that runs in less than 10 hours on the Raspberry Pi Zero 2, a $15 edge device. In conclusion, this work democratizes access to automatic neural network design, enables privacy-critical applications in sectors like healthcare and industry, and promotes a more sustainable, low-energy approach to AI development.