Generative AI Models and Automated Validators for Deep Learning Test Input Generation

Deep learning (DL) models have achieved remarkable performance across visual recogni- tion tasks; however, their reliability under diverse and previously unseen inputs remains a major concern. Traditional testing approaches are insufficient to systematically reveal model failures when these models are deployed in real-world settings, as they often pro- duce invalid inputs (i.e., outside the classification task’s domain). This thesis addresses these challenges by advancing both the generation and validation of artificial test inputs for DL image classifiers. Leveraging recent advances in Generative AI (GenAI), we benchmark different gen- erative architectures to assess their effectiveness, efficiency, and output quality in terms of domain validity and label preservation. We conduct a large empirical study involving three GenAI architectures (VAEs, GANs, and Diffusion Models), five image classification tasks of increasing complexity, and 364 human evaluations. Our results show that sim- pler architectures such as VAEs are sufficient for low-complexity datasets like MNIST, whereas feature-rich datasets such as ImageNet require more expressive models, with Diffusion Models generating a substantially higher number of valid, misclassification- inducing inputs. To address the limitations of existing automated input validators, which often fail to align with human judgments of validity, this thesis introduces DeepNaqqal, a su- pervised test input validator that learns validity directly from human-annotated data using transfer learning on deep vision models. We empirically compare DeepNaqqal against six state-of-the-art validators across multiple image classification tasks and TIG families, using independent human evaluation as ground truth. The results demonstrate that DeepNaqqal achieves the highest agreement with human judgments, generalizes to unseen TIGs, and remains effective even with limited labeled data. Finally, we integrate GenAI test input generation and validation into GIFTbench, a modular benchmarking framework for DL image classifier testing. GIFTbench combines pretrained generative AI models with genetic algorithms and a user-friendly Gradio in- terface to support automated, reproducible, and interpretable robustness testing. By perturbing latent representations in VAE, GAN, and Diffusion models, GIFTbench ef- ficiently exposes classifier misbehavior while reducing manual implementation effort. Designed for both researchers and practitioners, GIFTbench enables systematic eval- uation of classifier robustness, generative model effectiveness, and mutation strategies within a unified experimental environment.