Performance Engineering in Agile/DevOps Development Processes: Ensuring Software Performance While Moving Fast

Agile principles and DevOps practices play a pivotal role in modern software development. These methodologies aim to improve software organization productivity while preserving the quality of the produced software. Unfortunately, the assessment of important non-functional software properties, such as performance, can be challenging in these contexts. Frequent code changes and software releases make impractical the use of classical performance assurance approaches. Moreover, many performance issues require highly specific conditions to be detected, which may be difficult to replicate in a testing environment. This thesis investigates and tackles problems related to performance assessment of software systems in the context of Agile/DevOps development processes. Specifically, it focuses on three aspects. The first aspect concerns practical and management problems in handling performance requirements. These problems were investigated through a 6-months industry collaboration with a large software organization that adopts an Agile software development process. The research was conducted in line with ethnographic research, which guided towards building knowledge from participatory observations, unstructured interviews and reviews of documentations. The study identified a set of management and practical challenges that arise from the adoption of Agile methodologies. The second aspect concerns the impact of refactoring activities on software performance. Refactoring is a fundamental activity in modern software development, and it is a core development phase of many Agile methodologies (e.g., Test-Driven Development and Extreme Programming). Nevertheless, there is little knowledge about the impact of refactoring operations on software performance. This thesis aims to fill this gap by presenting the largest study to date that investigates the impact of refactoring on software performance, in terms of execution time. The change history of 20 Java open-source systems was mined with the goal of identifying commits in which developers have implemented refactoring operations impacting code components that are exercised by performance benchmarks. Through a quantitative and qualitative analysis, the impact of (different types of) refactoring on execution times were unveiled. The results showed that the impact of refactoring on execution time varies depending on the refactoring type, with none of them being 100% "safe" in ensuring that there is no performance regression. Some refactoring types can result in substantial performance regression and, as such, should be carefully considered when refactoring performance-critical parts of a system. The third aspect concerns the introduction of techniques for performance assessment in the context of DevOps processes. Due to the fast-faced release cycle and the inherently non-deterministic nature of software performance, it is often unfeasible to proactively detect performance issues. For these reasons, today, the diagnosis of performance issues in production is a fundamental activity for maintaining high-quality software systems. This activity can be time-consuming, since it may require thorough inspection of large volumes of traces and performance indices. This thesis introduces two novel techniques for automated diagnosis of performance issues in service-based systems, which can be easily integrated into DevOps processes. These techniques are evaluated, in terms of effectiveness and efficiency, on a large number of datasets generated for two case study systems, and they are compared to two state-of-the-art techniques and three general-purpose clustering algorithms. The results showed that baselines were outperformed with a better and more stable effectiveness. Moreover, the presented techniques showed to be more efficient on large datasets when compared to the most effective baseline.