3D Reconstruction and Novel View Synthesys with Sparse Multi-View Stereo Rigs

The increasing demand for capturing high-quality 3D representations of real-world objects and scenes has been driven by advancements in fields such as visual effects, 3D printing, augmented and virtual reality, and autonomous driving. These applications rely on accurate 3D reconstruction techniques, such as multi-view stereo (MVS) and novel view synthesis, to enhance realism, enable new forms of interaction, and improve environmental perception. While professional multi-view stereo (MVS) systems with dense camera arrays offer high precision, they remain costly. This work focuses on sparse multi-view stereo setups, which provide a more affordable alternative while maintaining competitive reconstruction quality. Our contributions address key challenges in sparse MVS by introducing: (1) an automatic calibration system for such setups, (2) a pyramidal scheme approach integrated into COLMAP to enhance reconstruction efficiency and robustness; (3) a segmentation network trained on synthetic data to improve reconstruction accuracy, removing noise from background; (4) a set of regularization techniques for Neural Radiance Fields (NeRF) that improve performance in sparse-view scenarios; and (5) a novel multi-view photometric stereo method capable of delivering high-quality results even when fewer lighting conditions are available. By leveraging these advancements, this research enhances the practicality of cost-effective multi-view 3D reconstruction systems, broadening their accessibility for a wide range of applications.