Complex Numbers, Fourier and Signals

Complex rotations, DFT and FFT, windows, sampling, spectrograms, mel filterbanks and MFCCs. Python from the lessons.

Cooley-Tukey FFT, 8 points#

waveform, sr = sf.read("clip.wav", dtype="float32")float32 in [-1, 1]

spectrum = np.fft.fft(signal)

freqs = np.fft.fftfreq(256, d=1/256)frequency of each bin

power = np.abs(spectrum) ** 2power spectrum

window = windows.hann(256)from scipy.signal

result = fftconvolve(signal, kernel, mode='full')

frequencies, times, Zxx = stft(signal, fs=sample_rate, nperseg=256)

spectrogram = np.abs(Zxx) ** 2(n_frequencies, n_time_frames)

print(cmath.exp(1j * cmath.pi))Euler: close to -1

def hz_to_mel(f):

return 2595.0 * math.log10(1.0 + f / 700.0)

mag = stft_magnitude(signal, frame_len=400, hop=160)Hann + DFT per frame

fb = mel_filterbank(n_mels, n_fft, sr)80 mels, n_fft=400 gives (80, 201)

mel_spec = apply_filterbank(cmag, fb)one weighted sum per mel bin

lm = log_transform(mel_spec)eps=1e-10

mfcc = dct_ii(lm[len(lm) // 2], 13)coefficient 0 is overall energy

Multiply: $inline$ . Magnitudes multiply, angles add.
Polar: $inline$ and $inline$ .
Euler: $inline$ . Multiply by it to rotate by $inline$ .
DFT: $inline$ .
Inverse DFT: positive exponent and a $inline$ factor. Reconstruction is exact.
Bins: $inline$ , resolution $inline$ , maximum $inline$ .
Convolution theorem: $inline$ .
Mel: $inline$ .

Command	What it does
8 kHz	Telephony. Nyquist at 4 kHz kills consonants. Avoid for ASR
16 kHz	ASR standard. A 10 s clip is 160,000 floats
22.05 kHz	TTS vocoder training for older models
24 kHz	Modern TTS
44.1 kHz	CD audio, music
48 kHz	Film, pro audio, high-fidelity TTS

Command	What it does
Rectangular	No window. Side lobes -13 dB. Signal exactly periodic in N
Hann	0.5 (1 - cos(2 pi n / (N-1))). -31 dB. General purpose
Hamming	0.54 - 0.46 cos(2 pi n / (N-1)). -42 dB. Speech analysis
Blackman	Wide main lobe. -58 dB. Side lobe suppression is critical

Symptom	Cause	Fix
Garbage output that looks like a model bug	Sample rate mismatch	Match sample rate first. 16 kHz mono float32
7 kHz tone peaks at 3 kHz	Aliasing above Nyquist	Low-pass filter before downsampling
Spurious high frequencies	Spectral leakage at the frame edges	Apply a Hann or Hamming window
Silent failure at inference	Mel count mismatch, 80 against 128	Log the feature shape at both ends
Wrong input for Whisper	dB-mel, not log-mel	Feed log-mel
WER doubles in production	Normalization drift	Use the same normalization in training and inference
Flat spectrum in the last frames	Zero padding at the end	Pad symmetrically or replicate
FFT convolution wraps around	The DFT computes circular convolution	Zero-pad to N + M - 1

Does zero-padding add resolution?: No. Resolution needs $inline$ seconds of data.
Why keep only the first N/2 + 1 bins?: For real input, $inline$ is the conjugate of $inline$ .
Why is i a rotation?: Multiply by i to turn 90 degrees. Two turns give $inline$ .
How does RoPE use this?: It rotates queries and keys. Relative position becomes an angle.
Why mel, not Hz?: People hear pitch on a log scale. 100 to 200 Hz sounds like 1000 to 2000 Hz.
What does Parseval say?: Energy is the same in the time and frequency domains.