Architectures and algorithms for packet processing and network monitoring

As internet is becoming a critical infrastructure and the amount of traffic carried on it is rapidly growing, along with the potential security threats, monitoring is becoming more and more a crucial activity to the correct operations of networks and network based services. However, the amount of data to be analyzed, the extreme variety of the analysis to be supported, along with the need to correlate data from different sources and the limitations imposed by the privacy legislation make network monitoring a difficult and challenging task. In this work we explore several research fields, all of them related to network monitoring and testing. First of all, we propose tomographic techniques, that allow to infer the internal state of the network by applying statistical analysis to measurements carried out by the end–hosts, with no cooperation from the internal nodes. We then illustrate novel algorithms and data structures for speeding up expensive packet processing tasks, such as deep packet inspection. Subsequently, we move on to architectural topics and show how general purpose processors and special purpose devices can complement each other in order to build monitoring and testing systems offering an optimal trade–off between flexibility and performance. Moreover, we also investigate on the potential that the modern commodity hardware (which is highly parallel) provides and on how this can be leveraged for the benefit of the network monitoring applications. Finally, we delve into the topic of distributed monitoring and propose novel solutions for building an overlay of monitoring probes which can efficiently correlate the observed data, thus avoiding the scalability bottleneck of an architecture based on a single collection point.