Architectures and Algorithms for Learning Without Data Accumulation over Time

The increasing demand for intelligent systems capable of operating in real-world, temporally dynamic environments calls for a fundamental rethinking of conventional machine learning paradigms. Traditional models rely heavily on the accumulation, storage, and repeated access to vast datasets—a process that contrasts sharply with how biological systems learn: incrementally, causally, and without revisiting the past. This thesis addresses the theoretical and practical challenges involved in designing learning systems that operate without data accumulation over time, laying the foundation for a new class of intelligent agents under the umbrella of Collectionless Artificial Intelligence. At its core, this work proposes learning algorithms and architectures designed to function online, in continuous time, and without reliance on stored data, under strict temporal and spatial locality constraints. In this collectionless setting, the agent is prohibited from buffering or revisiting past inputs, reflecting a more biologically plausible and ethically favorable approach to learning, particularly relevant in edge computing scenarios where privacy, energy efficiency, and decentralization are key. A key contribution is the formulation of the perpetual generation problem, which focuses on enabling systems to autonomously produce consistent and meaningful sequences over indefinite time horizons, without external inputs or memory. The thesis provides a detailed analysis of marginal stability in both linear and nonlinear recurrent neural networks, establishing spectral and dynamical criteria that ensure stable internal dynamics suitable for sustained inference and generation. Another major innovation is the introduction of the Hamiltonian Learning framework, which recasts learning as an optimal control problem solved forward in time. Unlike traditional methods such as backpropagation through time, this approach leverages Hamilton equations to derive learning rules that are local in both time and space. This not only enhances biological plausibility but also enables scalable training of deep and recurrent networks in settings where memory and computational resources are limited. The proposed theories and models are supported by comprehensive experimental validation across a range of tasks, including online sequence generation and streaming data classification. These experiments confirm the effectiveness of the methods under continuous-time constraints, where conventional approaches typically fail due to their reliance on data storage. By integrating perspectives from machine learning and control theory, this thesis advances the vision of self-sufficient, robust, and adaptive learning systems. The methods developed here point toward a new generation of intelligent agents—ones that operate efficiently, respect privacy, and more faithfully mirror the continuous, embodied learning seen in natural organisms.