Towards continual and few-shot learning in generative adversarial networks (GANs)

Deep learning models, inspired by the human brain, aim to replicate the capabilities of continual learning and few-shot learning. Continual few-shot learning focuses on the ability to learn knowledge over time continuously from a small number of training samples (e.g., 10 samples). Although deep learning models demonstrate impressive intelligence and achieve remarkable performance, their performance deteriorates when it comes to continual and few-shot learning without proper consideration, which are important aspects for many real-world applications. Unlike humans, continual learning is challenging due to the inherent problems of catastrophic forgetting, caused by gradient updates. At the same time, few-shot learning leads to overfitting in these models. Therefore, both scenarios recently attracted attention to eliminate these issues. Compared to discriminative models, generative adversarial networks (GANs) are not explored extensively. In this thesis, we propose novel solutions for GANs that tackle the challenges of continual and few-shot learning together. Specifically, the first method proposes a teacher-student architecture to achieve more diverse image generation. In this approach, we apply cross-domain correspondence loss to learn from few-shot samples, rather than relying on a limited set of training samples (e.g., 50 to 5000 samples). We also analyze the progressive growing of adapter modules for the continual image generation in GANs. In contrast, the second technique presents a parameter-efficient method for continual learning and few-shot image generation with less parameters and more efficient training than the state of the art.