Improving the Effectiveness of Graph Neural Networks in Practical Scenarios

In the past decade, deep learning has given new life to the field of artificial intelligence, providing many breakthroughs in areas like computer vision, natural language processing, audio, game-playing, and biology. The past few years have seen a particular interest in developing and applying deep learning models that can operate on graph-structured data, also called graph neural networks (GNNs). This is not surprising, as graphs are core data structures that are used to model relationships between different entities, which is a common scenario that appears in molecules, social networks, and physical interactions, just to cite a few. GNNs have achieved many successes, and have been studied from both a theoretical, and a practical point of view. However, several questions remain unanswered. In this thesis, we focus on three open questions regarding practical aspects of GNNs, and we propose effective solutions for tackling them. The first question concerns global structural information, i.e., information that depends on the global structure of the graph. This kind of information is particularly difficult to capture for GNNs, which are targeted towards local interactions. While global structural information has been previously overlooked, we show that it has an important impact on practical applications, and we propose a regularization strategy to provide this information to GNNs during training. The second question concerns size-generalization, which is the ability of GNNs to generalize from small to large graphs. While GNNs are designed to operate on graphs of any size, it is observed that when trained on small graphs, they struggle at generalizing to large graphs. This is particularly problematic, as in certain domains, obtaining labels for large graphs is prohibitive. Furthermore, training on large graphs may require expensive computational resources. We propose a novel regularization strategy, that can be applied on any GNN, and that can improve size-generalization capabilities of up to 30%. The third question focuses on multi-task settings. GNNs work by exchanging messages between nodes, and using learnable functions to produce node embeddings that encode structural and feature-related information. During training, GNNs tend to optimize the produced embeddings to the training loss, making it hard to reuse them effectively for different tasks. This requires the training of multiple models, and the use of different embeddings for different tasks. We propose a training strategy based on meta-learning that provides a single set of embeddings that can be used to perform multiple tasks while achieving performance comparable to those of single-task end-to-end trained models.