From Monolithic to Modular: Compositional Continual Learning for Foundation Models

Artificial Intelligence has entered the era of Foundation Models: large-scale architectures achieving remarkable performance across tasks. However, they are computationally expensive, environmentally demanding, and inherently static, struggling to adapt to evolving data distributions. Concurrently, continual learning research shows neural networks are prone to catastrophic forgetting, making stable long-term adaptation a fundamental challenge. This dissertation argues that compositionality offers a principled solution. Rather than treating networks as monolithic, compositionality constructs intelligent systems through reusable, modular components that can be selectively combined. This exploit reuse, enhances efficiency, reduces task interference, and enables scalable adaptation. We ground this in Continual Learning, analyzing its core challenges and algorithmic families, demonstrating how multi-expert systems mitigate forgetting and enable zero-shot composition. We then extend this to Foundation Models, studying Parameter-Efficient Fine-Tuning and introducing modular adaptation and merging techniques for continual fine-tuning without full retraining. Through theoretical and empirical evaluation, this work shows compositional adaptation reduces forgetting, limits parameter growth, and improves efficiency. We conclude that sustainable AI lies not in scaling models, but in structuring them—embracing compositionality for efficient, responsible adaptation.