Agent-based models of dynamical formation and emergence of memory on networks

The brain is quite possibly the most complex organ in existence, capable of, and responsible for, receiving and processing information. This ability is the essence of a wide assortment of brain functions, amongst which there is memory, or the capacity to learn from experience and retrieve stored information to affect decisions and behaviour. Hence, understanding the biological mechanisms underlying memory has been a central task in neuroscience. The very earliest theories of memory were largely conceptual, with Semon [1]–[3] being the first to suggest speculatively that the brain might store memory representations as lasting traces of activity, dubbed engrams. It was only in 1949 that Hebb [4] provided a formal theoretical framework to connect memory and the plasticity of synapses (i.e. connection between neurons), hypothesising that the repeated co-activation of neurons would strengthen their connections and in this principle he laid the core of memory, recovering Semon’s memory engram. Hebb’s theory laid the foundation for the development of statistical models of memory. Amongst the most notable and influential examples, Hopfield’s model [5], [6] conceptualises stored memories as stable attractors in an energy landscape and describes their recovery through a network process (but not their creation). Hopfield networks hence provided an early demonstration of how memory recall can be described as a process of energy minimization, drawing direct inspiration from the principles of Statistical Mechanics and giving rise to the use of statistical tools to understand memory and its functioning. Regarding the neuro-biology of memory, beyond pure neuronal plasticity more recent findings revealed how glial cells, i.e. cells other than neurons, have an active an crucial role for several brain capacities including memory, particularly modulating synaptic function [7], [8]. In this thesis, we present two Statistical Mechanics models of dynamical memory emergence through agent-based processes on networks, specifically biased random walk processes. The work is inspired from experimentally motivated questions, within the NanoScience Laboratory of the University of Trento, about the importance of a specific synaptic interaction between neurons and a particular type of glial cell, astrocytes, for proper formation of long-term memory [9], [10]. Through Statistical mechanics, we have been able to construct simple, high-level models that do not delve into the deep intricacies of the brain as detailed neuro-biological models do but are nonetheless able to parallel the essential features of interest to describe memory. As such, they pose as extremely versatile models that can provide a solid basis, to be eventually further expanded upon, to describe a variety of non-Markovian interactions is several complex systems of biological or other nature. In the first part of the PhD, we have developed a hierarchical memory model that incorporates key aspects underlined in recent neuro-biological literature. This model is based on a cyclic feed-forward network that is enriched by a second level, resulting from an edge-grouping procedure: clustering together M neighbouring nodes in each layer, we considered the edges going from one group to another as constituting one edge group. The resulting structure is a coarse-grained network that is self-similar to the initial one. On this two-level structure we put a discrete-time random walk dynamics with two memory contributions. The first is a rapid but short-lived memory that biases the random walk directly: as the walker explores the network, each edge keeps track of the number of times the walker has traversed it in the recent past, with exponential decay time τ . They then update their weights accordingly, affecting probabilistically the future path of the random walk. We have opted for a functional form that is both non-linear and bounded, specifically a logistic function, making the weight reinforcement mechanism mimic Hebbian strengthening of neuronal synapses. The second memory component of the model is a slower but ever-increasing memory that works through the modulation of the first, short-term one, mimicking the synaptic plasticity modulation effect of astrocytes. This form of memory works by increasing the steepness of the logistic function of an edge, in turn requiring less activations for the edge weight to grow to the maximum. We devised this second dynamics to work on the coarser network: the increase in steepness, of value c, is split amongst all edges belonging to an edge group, proportionally to their relative weights so that stronger edges are modulated more. We consider a stochastic activation for this effect, in order not to make any assumption on the exact bio-molecular driving force of the brain phenomenon. Through numerical simulations, we characterise the interplay between short- and long-term memory dynamics as functions of the short-term memory erasure, controlled by τ , and long-term memory effectiveness, controlled by c. By imposing the long-term memory dynamics to be stochastically active only on one closed path on the coarse-grained network, our results show that three regimes exist in the τ -c parameter space, including one where long-term memory can effectively drive memory emergence guiding the random walker into the target coarse-grained path, while an analogous system without the long-term modulation dynamics sees no memory emerging. We further show how our model not only captures memory formation as pattern emergence, but also its retrieval, as the random walker is able to return to the long-term-memorised path even after the complete manual erasure of any and all stored short-term memory from the network. We then extend the parameter space and account for the spatial scale M of the long-term memory dynamics, with larger values of M and hence a coarser memory enlarging the τ-c combinations in which long-term memory proves pivotal for memory emergence. We have elucidated all these results in light of analytical considerations on the exact functional forms of the hierarchical memory dynamics. The results of this first part of the thesis have been summarised in a published manuscript [11]. Given these important results, showcasing the relevance of synaptic plasticity modulation through a high-level statistical model, we have generalized upon our general construction and its specificities, together with aiming at a more general context: we have considered multiple random walkers having shared memory traces encoded in the underlying network and moving in continuous time. A continuous-time process features waiting times, with random walkers each moving according to an internal Poissonian probability distribution. We have generalised the network structure to be a two-dimensional square lattice, lessening the peculiar constraint of feed-forwardness present in the previous model. To avoid any contribution from boundaries, we have considered periodic boundary conditions. We fill the network according to parameter ρW, representing the density of walkers, from sparse to very dense. The model features the same short-term memory functional form developed in the previous model: the movement of each walker is stored in the network edge weights with exponential decay controlled by τ . Beyond the shared memory, walkers also interact by means of excluded volume. However, compared to the hierarchical memory random walk, here we have also introduced an element of “intelligence” to the moving agents. In particular, walkers are either able to jump according to a given pre-determined order, affecting the probability distributions determining their movement, or without any order. This reminisces recent studies on intelligent active matter, programmable units and soft matter agents [12], [13]. We have characterised the impact of order, together with parameter τ of memory erasure and the density ρW, on the emergence of memorised patterns and the coordination of different walkers. Results from our numerical simulations show that when walkers move following the given order it is sufficient to have a less persistent memory in order for movement patterns to emerge from the process across all densities ρW we considered. When no ordering is imposed, memory is required to last significantly longer to affect walkers is a substantial manner. Regardless of ordering, a smaller ρW means less persistent memory traces are required to drive walkers to pattern emergence. Furthermore, even with long-lasting memory ensuring pattern emergence regardless of both ρW and ordering, the behaviour of walkers is largely impacted by whether or not they follow an order: we have quantified this phenomenon though a mean squared displacement analysis. Unordered walkers showed tendency to move less frequently and more sporadically, as collisions due to excluded volumes are significantly more relevant: this jams the random walk process, particularly at large ρW. Ordered walkers on the other hand are able to move significantly more, often close to their intrinsic jumping distributions. In two other key observables we have employed in our analyses, namely the entropy rate of the process and the total strongly connected fraction of the lattice, our system has showed a behaviour that is reminiscent of a phase transition, with the memory decay time τ acting similarly to a critical parameter. For both observables mentioned the transition we have observed is less abrupt, and the states involved less separated, the larger the τ at which it happens, meaning denser configurations undergo softer transitions. To conclude, our memory models take advantage of the generality of Statistical Mechanics and its efficacy in capturing the very essential details of the synaptic interactions behind brain memory. While the first model is focused on elucidating the importance of synaptic-like plasticity modulation for the creation of long-term memory, the second one generalises and further simplifies the dynamics at play to interface a Hebbian-like memory process with intelligent agents from contexts akin to active matter and programmable units. Together, they provide an effective starting point for both more articulate statistical models of the brain, as well as the description of different non-Markovian agent-based phenomena. References: [1] R. W. Semon, Die Mneme als erhaltendes Prinzip im Wechsel des organischen Geschehens. Engelmann, 1911. [2] R. Semon, The Mneme, trans. by L. Simon. G. Allen & Unwin Limited, 1921, isbn: 978-1-5485-4540-6. [3] R. Semon and B. Duffy, Mnemic Psychology, trans. by V. Lee. G. Allen & Unwin, Limited, 1923. [4] D. O. Hebb, The organization of behavior; a neuropsychological theory. (The organization of behavior; a neuropsychological theory.). Oxford, England: Wiley, 1949. [5] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities.,” Proceedings of the National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, Apr. 1982, Publisher: Proceedings of the National Academy of Sciences. doi: 10.1073/pnas. 79.8.2554. [Online]. Available: https://doi.org/10.1073/pnas. 79.8.2554. [6] J. J. Hopfield, “Neurons with graded response have collective computational properties like those of two-state neurons.,” Proceedings of the National Academy of Sciences, vol. 81, no. 10, pp. 3088–3092, May 1984, Publisher: Proceedings of the National Academy of Sciences. doi: 10.1073/pnas.81.10.3088. [Online]. Available: https://doi.org/ 10.1073/pnas.81.10.3088. [7] A. Araque, V. Parpura, R. P. Sanzgiri, and P. G. Haydon, “Tripartite synapses: Glia, the unacknowledged partner,” Trends in Neurosciences, vol. 22, no. 5, pp. 208–215, May 1999, issn: 0166-2236. doi: 10.1016/S0166-2236(98)01349-6. [Online]. Available: https://www. sciencedirect.com/science/article/pii/S0166223698013496. 5 6 BIBLIOGRAPHY [8] M. De Pitt`a, N. Brunel, and A. Volterra, “Astrocytes: Orchestrating synaptic plasticity?” Neuroscience, Dynamic and metabolic astrocyteneuron interactions in healthy and diseased brain, vol. 323, pp. 43–61, May 2016, issn: 0306-4522. doi: 10.1016/j.neuroscience.2015.04. 001. [Online]. Available: https://www.sciencedirect.com/science/ article/pii/S0306452215003188. [9] B. Vignoli, G. Battistini, R. Melani, et al., “Peri-Synaptic Glia Recycles Brain-Derived Neurotrophic Factor for LTP Stabilization and Memory Retention,” Neuron, vol. 92, no. 4, pp. 873–887, Nov. 2016, issn: 0896-6273. doi: 10.1016/j.neuron.2016.09.031. [Online]. Available: https://www.sciencedirect.com/science/article/ pii/S0896627316306328. [10] B. Vignoli, G. Sansevero, M. Sasi, et al., “Astrocytic microdomains from mouse cortex gain molecular control over long-term information storage and memory retention,” Communications Biology, vol. 4, no. 1, p. 1152, Oct. 2021, issn: 2399-3642. doi: 10 . 1038 / s42003 - 021 - 02678-x. [Online]. Available: https://www.nature.com/articles/ s42003-021-02678-x. [11] G. Zanardi, P. Bettotti, J. Morand, L. Pavesi, and L. Tubiana, “Metaplasticity and memory in multilevel recurrent feed-forward networks,” Physical Review E, vol. 110, no. 5, p. 054 304, Nov. 2024, Publisher: American Physical Society. doi: 10.1103/PhysRevE.110.054304. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevE. 110.054304. [12] V. A. Baulin, A. Giacometti, D. Fedosov, et al., Intelligent Soft Matter: Towards Embodied Intelligence, arXiv:2502.13224 [cond-mat], Feb. 2025. doi: 10.48550/arXiv.2502.13224. [Online]. Available: http: //arxiv.org/abs/2502.13224. [13] H. L¨owen and B. Liebchen, Towards Intelligent Active Particles, arXiv:2501.08632 [cond-mat] version: 1, Jan. 2025. doi: 10.48550/arXiv.2501.08632. [Online]. Available: http://arxiv.org/abs/2501.08632.