Real-time modelling of audio for creative sound transformation
- Members: Mr Jez Wells, Dr Damian Murphy
- Start Date: June 2008
This work aims to provide an intuitive and flexible spectral model for real time processing of audio in a studio environment. Fourier analysis is used to extract the features of spectral peaks. A novel analysis of time reassignment data is used to estimate non-stationarity of components and to determine whether their behaviour is sinusoidal. The sinusoidal part of the signal is then separated to leave a residual which is analysed with complex B-spline wavelets in order to characterise broader band components (both long term stationary and impulsive). This analysis offers time varying estimates of the mean instantaneous frequency, spectral width and magnitude of components in each analysis band. A 'partially decimated' wavelet transform has been developed which offers mediation between model accuracy and shift invariance and its computational cost.
Parameters from the first part of model are used to control a bank of sinusoidal oscillators whilst those from the second part control a bank of parametric equalisers applied to a noise source. Such a model can be directly understood by music studio engineers, sound designers and many musicians. Applications include pitch scaling, hybridisation\morphing, feature emphasis\de-emphasis and consonance based mapping (adapting timbre to tuning or vice versa). The process is implemented as a real-time analysis\resynthesis VST plug-in which can be used with many DAW and other audio processing and synthesis systems.