Neural network compression across the cloud-edge continuum

The deployment of Artificial Intelligence at the Edge is constrained by a structural mismatch: modern neural networks, from Convolutional Neural Networks (CNNs) to Large Language Models (LLMs), are designed for Cloud infrastructures with abundant GPU resources, while Edge devices operate under severe limits on memory, computational power, and energy. This thesis addresses the challenge of bridging this gap through task-aware structural compression of neural networks, organised around three objectives. The first objective concerns compression algorithms. We propose ImproveNet, a training-integrated structural compression method for CNNs that achieves 95% parameter reduction with accuracy degradation below 1% and inference speedups of up to 10x on embedded devices. For LLMs, we propose SiRE, a one-shot saliency-informed structured pruning method for decoder-only architectures, and its evolution TADP, which decomposes neuron importance along the temporal dimension of activations and consistently outperforms state-of-the-art structured pruning baselines on eight models from the LLaMA, Mistral, Qwen, and Minerva families. The second objective concerns the delivery of compression across the Cloud-Edge continuum. We introduce a service-oriented paradigm of reduction as a service, realised in SlimAI4Edge, a unified framework in which a dispatcher routes reduction requests to the appropriate algorithm and returns lightweight models, or pruning instructions, for local execution on heterogeneous Edge devices. The third objective validates the full chain on satellite telemetry anomaly detection, a scenario arising from the industrial collaboration with Thales Alenia Space. Compressed autoencoder models retain detection performance on the ESA-ADB benchmark while becoming deployable on satellite-class embedded hardware, and structured pruning of transformer-based architectures preserves anomaly-relevant task performance. Together, these contributions demonstrate an end-to-end pathway from compression algorithms to deployable Edge intelligence.