SIGNAL MODELS, ANALYSIS ALGORITHMS, AND SOFTWARE TOOLS FOR MODAL AUDIO RESYNTHESIS

This thesis delves into topics related to modal sound synthesis, a technique that generates sounds by simulating the physical interactions within resonating objects. It proposes novel methods for analyzing audio recordings and extracting the modal parameters. A central algorithm, SAMPLE, estimates these modal parameters by finding trajectories in the spectrogram of the input audio. However, SAMPLE encounters challenges with specific sounds, like acoustic beats, where two close frequencies interact and create a beating effect. To overcome this limitation, the thesis introduces BeatsDROP, an auxiliary algorithm that complements SAMPLE and models the trajectories in the spectrogram as the amplitude and frequency modulations of beats. This thesis also details the reasons why most signal analysis models fail with beats. Furthermore, the thesis presents the Generalized Mixture Space (GMS) model, which aids in representing sounds with multiple channels. GMS allows SAMPLE's simplified analysis to be applied while retaining the original channel distribution information for later resynthesis. Beyond the theoretical framework, the thesis details the development of software tools to make these methods readily usable. SAMPLE is a Python package that implements the algorithms and models, along with additional functionalities previously defined in the audio DSP literature. The thesis also includes the re-engineering and expansion of the existing SDT (Sound Design Toolkit), written in C and available as an external library for Pure Data and Max. Functionalities are implemented within SDT to enable interoperability with other software, including the possibility to import modal analysis results from SAMPLE directly into SDT's modal synthesis models.