Spectrograms

• Complex waveforms can be viewed as the sum of simple sinusoids Even though we rarely find a pure sinusoidal tone in real life situations, the sine wave can be considered the fundamental component of all sounds which we do hear. A complex waveform can be defined as any sound wave which is not sinusoidal. By the theorem of Fourier, any complex periodic waveform can be decomposed into a series of simple sinusoids that differ in the three defining attributes of amplitude, frequency, and phase. Figure 7 shows how six sinusoid waves with different frequencies and amplitudes can be added to obtain a waveform which is no longer sinusoidal but is still periodic, in this case an approximation to the sawtooth waveform. For periodic signals such as the vowels in speech, we can measure the period; just as with a sinusoidal wave, the inverse of the period is the frequency of repetition of the complex wave, which for speech is called the fundamental frequency or . A typical value for a male speaker is 125 Hz, while for a female speaker it is closer to 250 Hz. Aperiodic speech sounds such as /s/ do not have a fundamental frequency, since they are not repetitive.
• A spectrum shows amplitude or intensity as a function of frequency The term spectrum is borrowed from optics, where it refers to the various wavelengths which compose a given light. We have already seen that a waveform shows pressure variations as a function of time. In contrast, a spectrum shows the pressure changes (or more often power or intensity values, which are proportional to the square of the pressure changes) for the different frequency components of a complex waveform.
• A spectrogram is a collection of spectra When we create a spectrum, we assume that a signal is periodic in a mathematical sense; i.e., that it consists of exact repetitions of a periodic waveform going to infinity in both directions. Obviously, most real-world signals do not conform to this model; however, for an appropriately sized waveform segment, the spectrum can provide a great deal of information even when we consider aperiodic segments of speech to be periodic. Hardware and software which create spectrograms take a window of contiguous samples, assume that these samples represent a periodic signal, and calculate and save the spectrum for the current window. Then the window is moved forward in time, the next spectrum is calculated, and so on.
• The original sound spectrograph machine The sound spectrograph was first described in Koenig, Dunn, and Lacey in an article by that name in the Journal of the Acoustical Society of America (18, 19-49) in 1946. As late as Flanagan's Second Edition (1972), the dominant mode of spectrogram production was still the sound spectrograph, an analog device with a bank of bandpass filters which after recording and analyzing the speech sample eventually burned the output spectrogram onto facsimile paper.
• Current spectrogram creation methods Given the performance of modern desktop computers, the calculation of a spectrogram from a waveform file can be performed entirely by discrete digital methods. Each spectrum constituting the spectrogram is calculated by an application of Fourier analysis in the discrete domain, the Discrete Fourier Transform (DFT), according to the following formula which calculates one frequency component:

  

  In practice, it is the Fast Fourier Transform (FFT), a more efficient equivalent of the DFT given by the above equation, which is used. The FFT results can be used to calculate both a magnitude spectrum and a phase spectrum; both spectra are necessary in order to resynthesize the waveform using the Inverse Discrete Fourier Transform (IDFT). The magnitude spectrum gives the distribution of energy among the different frequency components making up the waveform, while the phase spectrum gives the starting phases of these same frequency components. As mentioned before, in speech processing phase contributions are largely ignored. What we usually see as the graphic output of a spectrogram program is a two-dimensional array, having time as the columnar axis, frequency as the row axis, and intensity as the value being plotted. This array may be written to a disk file and stored for later use; however, DFT data is about one order of magnitude greater than waveform data, so that in this course we will often generate the DFT data without saving it.

• Spectrograms are not all created equal In designing a spectrogram algorithm, there are a number of decisions to make. Some spectrogram programs involve preprocessing, such as DC removal and pre-emphasis. In almost all cases, a windowing algorithm such as the Hamming or Hanning window is applied to the samples for which a spectrum is to be calculated. The user of a spectrogram program must also specify certain parameters for the DFT calculation. For example, many of the spectrograms to be shown in this course are computed with the following three parameters:
  • Window Size = 6.0 This number is the length in milliseconds of the window. With a sampling frequency of 16 kHz, 6.0 milliseconds will include 96 samples. If we choose a longer window size, we enhance the frequency resolution at the expense of the time resolution, whereas a shorter window size improves the time resolution at the expense of frequency resolution. The window size chosen depends on the purpose of the analysis.
  • Window Increment = 1.0 This number is the length in milliseconds of the distance between successive windows. Again, with a sampling frequency of 16 kHz, we slide the window forward 16 samples for each successive window.
  • Number of coefficients = 128 This is the number of separate frequency readings which will be returned for each window. If the number of coefficients is greater than one half the number of samples in the window size, the waveform in the window is extended to twice the number of coefficients requested and zero-filled; this is because by the Nyquist sampling theorem we must have at least samples to calculate n frequency coefficients.

  Figure 8 shows a waveform and spectrogram for the word ``adoring" with the above parameter settings. Figure 9 shows an expanded view of a small piece of the spectrogram in Figure 8. Figure 10 is a ``3-D" view of approximately the same section of the word as Figure 9.

)