Learning For Computational Image Sensing

This thesis deals with two challenging computational imaging problems: spectral and phase imaging. Standard image sensors heavily down-sample spectral information via color filtering and limited spectral sensitivity and are sensitive only to light intensity, which may result in the loss of valuable information related to object composition or the direction of incoming light rays. Consequently, vision and sensing applications such as spectral imaging, interferometry, and 3D imaging are hindered. The aim of this work is to provide ways to computationally recover this information from compressed, multiplexed, and decimated measurements captured using ad-hoc devices. To this end, two learning-based approaches are proposed. The first approach tackles the problem of hyper-spectral image reconstruction from compressed sensor measurements captured using a CTIS prototype, which is a snapshot imaging device that captures three-dimensional hyper-spectral data cubes as two-dimensional multiplexed signals. Computational post-processing is then needed to recover the latent data cube. However, iterative algorithms typically used to solve this task require large computational resources as the CTIS system matrix is quite wide and can become intractable with a higher spatial resolution of the input measurement. Furthermore, these approaches are very sensitive to the assumed systems and noise models. In addition, the poor spatial resolution of the 0th diffraction order image limits the usability of CTIS in favor of other snapshot spectrometers, even though it enables higher spectral resolution. A novel approach, dubbed Hyper-Spectral and Super-Resolution Network (HSRN) and its subsequent variant HSRN+ are proposed in this regard to recover high-quality hyper-spectral images leveraging complementary spatio-spectral information scattered across the sensor image, furthermore a reconstruction capability beyond the spatial resolution limit of the 0th diffraction order is achieved with quasi real-time performance. The second approach focuses on Quantitative Phase Imaging (QPI) and recovers a high-quality complex light field from in-line holographic measurements, the phase of which can be used to reveal the contrast in transparent and extremely thin microscopic specimens. Despite the limitation of image sensors, which detect only light intensity, phase information can still be recorded within a two-dimensional interference pattern between two distinct light waves. This work introduces HoloADMM, an interpretable, learning-based approach designed for in-line holographic image reconstruction. HoloADMM enhances the phase imaging capability with spatial image super-resolution, offering a versatile framework that accommodates multiple illumination wavelengths and supports extensive refocusing ranges with up to 10 mum precision. HoloADMM can achieve a substantial improvement in reconstruction quality over existing methods and demonstrates effective adaptation to real holographic data captured by a custom-made DIHM prototype.