Enhancing Sound, Music, and Movement Computing with Differentiable DSP
Date:
Impact of Differentiable Digital Signal Processing (DDSP) on the recent work at the Multisensory Experience Lab Hints: We can deploy DDSP-based real-time plugins fast because
- we started the work about 25 years ago :-)
- we are implementing the principles of MLOPs.
Do these, and the future will be yours :-)
In the Multisensory Experience Lab, we traditionally rely on physics-based audio and haptic models for multisensory processing. Currently we focus on physics-based deep learning and differentiable DSP for constructing real-time models or estimating the model parameters. More recently we realized the importance of 🚀 Putting these ML models in Production as real-time plugins. The talk compares this approach to what was used two decades ago with shallow-learning models, self-recoreded or synthetic data, and self-written code and frameworks-