Multimodal artificial intelligence promises richer and more reliable predictions by combining complementary sources of information, yet real-world data are rarely complete. This thesis investigates how multimodal learning systems can remain accurate, efficient, and robust when one or more expected modalities are unavailable, with particular emphasis on biological and biomedical applications, where data are heterogeneous, costly, and often scarce. The work develops complementary strategies to address missing modalities across multiple domains. For multimodal Transformers, semantically conditioned prompts are introduced to mitigate the representational disruption caused by missing inputs, while a dedicated benchmark and a retrieval-augmented generation framework extend this investigation to multimodal large language models. In medical image segmentation, Transformer- and Mamba-based architectures are developed to preserve global context while improving computational efficiency, culminating in IM-Fuse, a model robust to arbitrary combinations of missing MRI modalities. In computational oncology, OXA-MISS enables on-demand integration of histopathological and transcriptomic information while learning from incomplete datasets. Finally, in regulatory genomics, efficient models for gene expression prediction are introduced, the value of integrating DNA methylation with sequence is demonstrated, and Fiber-seq-derived m6A chromatin accessibility signals are inferred directly from DNA to enhance gene expression prediction when experimental measurements are unavailable. Across these diverse settings, a common principle emerges: missing modalities should be treated as an expected operating condition rather than an exception. By jointly addressing robustness, computational efficiency, and multimodal integration, this thesis advances AI models better aligned with the constraints of real-world biological and biomedical data.
Towards Robust Multimodal Learning for Biological and Biomedical Data Science: Mitigating the Effects of Missing Data
PIPOLI, VITTORIO
2026
Abstract
Multimodal artificial intelligence promises richer and more reliable predictions by combining complementary sources of information, yet real-world data are rarely complete. This thesis investigates how multimodal learning systems can remain accurate, efficient, and robust when one or more expected modalities are unavailable, with particular emphasis on biological and biomedical applications, where data are heterogeneous, costly, and often scarce. The work develops complementary strategies to address missing modalities across multiple domains. For multimodal Transformers, semantically conditioned prompts are introduced to mitigate the representational disruption caused by missing inputs, while a dedicated benchmark and a retrieval-augmented generation framework extend this investigation to multimodal large language models. In medical image segmentation, Transformer- and Mamba-based architectures are developed to preserve global context while improving computational efficiency, culminating in IM-Fuse, a model robust to arbitrary combinations of missing MRI modalities. In computational oncology, OXA-MISS enables on-demand integration of histopathological and transcriptomic information while learning from incomplete datasets. Finally, in regulatory genomics, efficient models for gene expression prediction are introduced, the value of integrating DNA methylation with sequence is demonstrated, and Fiber-seq-derived m6A chromatin accessibility signals are inferred directly from DNA to enhance gene expression prediction when experimental measurements are unavailable. Across these diverse settings, a common principle emerges: missing modalities should be treated as an expected operating condition rather than an exception. By jointly addressing robustness, computational efficiency, and multimodal integration, this thesis advances AI models better aligned with the constraints of real-world biological and biomedical data.| File | Dimensione | Formato | |
|---|---|---|---|
|
PhDThesis_Pipoli_FINAL_PDFA_3.pdf
embargo fino al 09/11/2029
Licenza:
Creative Commons
Dimensione
14.07 MB
Formato
Adobe PDF
|
14.07 MB | Adobe PDF |
I documenti in UNITESI sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/20.500.14242/385550
URN:NBN:IT:UNIPI-385550