Transparent Artificial Intelligence Approaches to Model Disease Progression using Real-World Clinical Registry Data

The integration of Artificial Intelligence (AI) in the healthcare sector has significant implications for the enhancement of clinical workflows, especially concerning the management of chronic and complex diseases, such as amyotrophic lateral sclerosis (ALS). Despite its potential, AI models encounter several challenges, including the necessity for transparency, interpretability, and compliance with regulatory frameworks. This thesis seeks to address these challenges by devising transparent AI methodologies and implementing them within real-world clinical registries, sourced from the BRAINTEASER project. The research is structured around three key objectives. The first objective advocates for the principles of open science in data management and model development. In the domain of data management, it addresses critical obstacles inherent in real-world clinical datasets, such as the presence of missing values, inconsistent follow-up intervals, and the integration of temporal dimensions into AI models. All the data processing steps were developed in collaboration with clinical experts and the data fairification team. Concerning model development, this objective focuses on formulating a framework for training predictive models that comply with the European Union’s AI Act, thereby ensuring that AI predictions maintain transparency, robustness, and generalizability. The second objective is centred on the development of an interpretable methodology for the characterisation of patient data, named DYNamic Archetypal Analysis for MIning Disease TrajEctories (DYNAMITE). This methodology leverages two descriptive techniques: archetypal analysis for deriving a limited set of representative disease states, and process mining for the characterisation and examination of the sequences of disease states that patients undergo. In developing DYNAMITE, the ease of use in clinical settings was assumed as a crucial factor, as the method is designed to be intuitively interpretable at each phase, enabling clinicians to monitor disease progression longitudinally, both at the individual and population levels. Additionally, DYNAMITE is adaptable with respect to follow-up durations and the number of disease states under consideration. The third objective emphasises model explainability. The model training framework aforementioned described, incorporates mechanisms to assess the reliability of the predictions generated, thereby responding to the increasing demand for transparency in AI applications. A multi-faceted approach is proposed to ensure explainability in driving factors of reliability of model predictions. In summary, this thesis significantly advances the field of AI in healthcare by constructing frameworks that advocate for open science, improve the characterisation of patient data, and ensure transparency concerning the reliability and explainability of AI-generated predictions. These contributions are aimed at facilitating the integration of AI into clinical practice, with a specific emphasis on enhancing the care and management of ALS patients.