Next Generation Computing Systems: Improving Memory and Hardware Accelerators Management

The efficient management of hardware resources is a crucial challenge for modern computing systems, driven by the exponential growth of data processed by contemporary workloads and the proliferation of different and specialized hardware accelerators, which are enabling the scaling of performance and energy efficiency through specialization and integration. One of the most challenging resources to manage is memory, where address translation, in certain scenarios, can significantly impact performance. While increasing page size might mitigate this issue, it introduces challenges in physical memory allocation. Additionally, managing hardware accelerators in heterogeneous systems presents its difficulties. Current solutions, often relying on device drivers, can incur substantial overhead, particularly for fine-grained tasks. This thesis addresses these challenges with innovative solutions. First, we introduce Decoupled Virtual Address Spaces (DeVAS), a memory management technique that inserts an additional address space between virtual and physical address spaces, supported by an extended memory controller. DeVAS effectively reduces address translation overhead and enables efficient management of hybrid memory systems comprising multiple memory technologies. For hardware accelerators management, we propose the ISA Extension for Integrated Accelerator Management (IXIAM) framework. IXIAM employs an ISA extension with specialized instructions to optimize operations such as accelerator reservation, task offloading, synchronization, and fine-grained control of accelerator-local memory, that significantly improve performance in heterogeneous System-on-Chip architectures. Lastly, recognizing the critical role of convolution in convolutional neural networks, we present enhancements for executing convolutions on multi-core systems and optimizing address translation support in convolutional accelerators operating within a shared virtual address space. These contributions collectively address key challenges in memory and hardware accelerators management, pushing the boundaries of modern computing systems efficiency.