Optimisation of high-speed interconnects for scientific computing

Ethernet-based or Ethernet-compatible interconnects are gaining traction as the evolution of today’s scientific and technological landscape pushes the integration of High Performance Computing (HPC), Big Data (BD), and Artificial Intelligence (AI). On the other hand, in research-oriented environments, custom HPC solutions are able to reach a higher level of performance with respect to standardized counterparts, by providing an optimized and power efficient architecture tailored to the specific field of application. This thesis explores strategies to improve communication efficiency in modern HPC systems. Following the requirements of relevant scientific applications, it focuses on providing a low-latency communication primitive within a Network Interface Card (NIC) to accompany the throughput-optimized Remote Direct Memory Access (RDMA), while managing congestion through the development of Quality-of-Service (QoS) mechanisms, and a flexible router hosting network offloads. The work also evaluates the integration of a RISC-V core within an FPGA platform, with the aim of assessing its readiness for control and processing responsibilities in the network stack. Throughout this thesis, several aspects of the interconnect will be addressed and accompanied by an in-depth study of the current state of the art, starting from host CPU-NIC communication to efficient packet routing mechanisms. These investigations will be conducted on top of a custom NIC, APEnetX, a high-performance, FPGA-based interconnection network developed as part of ongoing european exascale research initiatives.