Multimedia Application of Time Compress/Stretch of Sound by Granulation
The technique for time-stretch/compression of sound by granulation (or windowing) was originally proposed in 1946 [Gabor 1946]. The original aim of the technique was for frame-rate conversion of cinematic films by optical method. Despite the conceptional simplicity, the signal processing demands had restricted the realisation in digital media, until high-power DSP chips and CPUs were cheaply available [Truax 1988 and 1994, Itagaki et al. 1995]. The technique has been applied for sound processing mainly for music composition [Roads 1978, Xenakis 1971]. The technique, however, has not been exploited for the original purpose in digital media.
In variable-speed functions of video playback systems, audio output is usually muted during the operation [Watkinson 1994]. If audio were to be output, this results in the transposition of the pitch. For example, in case of a slow motion sequence, as the tape speed slows down by repeating a frame, the pitch of the sound track is lowered. In the opposite case, fast forward, the pitch is shifted upward. These effects usually accompanied with distortion in harmonic contents, due to the pitch shift, as appeared in the wave table synthesis technique. Using the sound granulation method for time-stretch/compression, a slow motion and fast forward video sequences can be replayed with sound in respectable speed in original pitch without heavy distortion in timbrel structure.
Examples
|
|
References
Gabor, D. (1946) "Theory of Communication." Journal of Institution of Electrical Engineers. 17(3): 429-457.
Itagaki, T., Manning, P. D., and Purvis, A. (1995) "An Implementation of Real-Time Granular Synthesis onto a Multi-Processor Network." In the Proceedings of International Computer Music Conference 1995. Banff, CANADA, pp. 493-494.
Itagaki, T., and Knox, D. (1999) "Multimedia Application of Time Compress/Stretch of Sound by Granulation." In the Proceedings of International Computer Music Conference 1999, October 1999, Beijing, CHINA, pp. 512-514.
Itagaki, T. (2000) “Sound Compression/Interpolation by Granulation.” presented at the 108th Audio Engineering Society Convention, February 2000, Paris, FRANCE, preprint No. 5126 (J-5).
Knox, D., Itagaki, T., Stewart, I., Nesbitt, A. and Kemp, I.J. (1999) “Preservation of Local Sound Periodicity with Variable-Rate Video.” In the Proceedings of the 7th ACM Multimedia Conference, October 1999, Orlando, USA, pp. 299-302.
Roads, C. (1978) "Automated Granular Synthesis of Sound." Computer Music Journal 2(2):61-62.
Truax, B. (1988) "Real-Time Granular Synthesis with a Digital Signal Processor." Computer Music Journal. 12(2): 14-26.
Truax, B. (1994) "Discovering Inner Complexity: Time Shifting and Transposition with a Real-time Granulation Technique." Computer Music Journal. 18(2): 38-48.
Watkinson, J., (1994) The Art of Digital Video. Boston: Focal Press
Xenakis, I. (1971) Formalized Music. Bloomington: Indiana University Press. ISBN: 0-25-4332378-9.