Enhacing spectral sintesis techniques with performance gestures using the violin as a case study

Abstract

In this work we investigate new sound synthesis techniques for imitating musical instruments using the violin as a case study. It is a multidisciplinary research, covering several fields such as spectral modeling, machine learning, analysis of musical gestures or musical acoustics. It addresses sound production with a very empirical approach, based on the analysis of performance gestures as well as on the measurement of acoustical properties of the violin. Based on the characteristics of the main vibrating elements of the violin, we divide the study into two parts, namely bowed string and violin body sound radiation. With regard to the bowed string, we are interested in modeling the influence of bowing controls on the spectrum of string vibration. To accomplish this task we have developed a sensing system for accurate measurement of the bowing parameters. Analysis of real performances allows a better understanding of the bowing control space, its use by performers and its effect on the timbre of the sound produced. Besides, machine learning techniques are used to design a generative timbre model that is able to predict spectral envelopes corresponding to a sequence of bowing controls. These envelopes can then be filled with harmonic and noisy sound components to produce a synthetic string-vibration signal. In relation to the violin body, a new method formeasuring acoustical violin-body impulse responses has been conceived, based on bowed glissandi and a deconvolution algorithm of non-impulsive signals. Excitation is measured as string vibration and responses are recorded with multiple microphones placed at different angles around the violin, providing complete radiation patterns at all frequencies. Both the results of the bowed string and the violin body studies have been incorporated into a violin synthesizer prototype based on sample concatenation. Predicted envelopes of the timbre model are applied to the samples as a time-varying filter, which entails smoother concatenations and phrases that follow the nuances of the controlling gestures. These transformed samples are finally convolved with a body impulse response to recreate a realistic violin sound. The different impulse responses used can enhance the listening experience by simulating different violins, or effects such as stereo or violinist motion. Additionally, an expressivity model has been integrated into the synthesizer, adding expressive features such as timing deviations, dynamics or ornaments, thus augmenting the naturalness of the synthetic performances.