Associative Networks Beyond Pattern Retrieval

In this thesis, we explore the contributions of statistical mechanics to the field of theoretical artificial intelligence. Specifically, we examine artificial neural networks equipped with a cost function, referred to as energy or Hamiltonian in physics. Our focus is on generalizations of the Hopfield model, an associative network designed to store and retrieve specific information patterns. In the Hopfield model, each node, called neuron or spin, is fully connected to all other nodes via weighted edges, known as couplings or synapses. These weights are pre-determined using Hebb's rule, a mechanism that encodes specific spin configurations, termed patterns, as energy minima. The network can retrieve a pattern if the number of stored patterns is below a threshold. However, artificial intelligence systems aim to do more than pattern retrieval; they should also have capabilities such as generalization and learning. We address these limitations in the context of associative networks in which stored and queried data share the same type. Starting from the traditional Hopfield model, we first increase its descriptive power by enabling neurons to interact in groups larger than pairs. This extension allows the network to store significantly more information compared to the traditional model. To describe the generalizations capabilities of the obtained model, we feed into the network noisy versions of the patterns and analytically derive computational phase diagrams that we corroborate with numerical experiments. Next, we extend the Hopfield model to enable learning from data, an essential capability of artificial intelligence systems. The learning dynamics derived from this extension converge to a straightforward generalization of Hebb's rule. We then explore how the mathematical structure of the Hopfield cost function can be used to implement an effective description of physical systems, even if the elements of the systems do not interact instantaneously. Finally, we establish a direct connection between the energy of a system and the dynamical system governing the evolution of its components. The resulting general mathematical framework enables the construction of dynamical systems that describe the evolution of associative network components towards the minimization of the energy function.