Flow Matching & Rectified Flows
Diffusion models take 20-50 sampling steps because they walk a curved path from noise to data. Flow matching (Lipman et al., 2023) and rectified flow (Liu et al., 2022) trained straight paths. Straighter paths mean fewer steps mean faster inference. Stable Diffusion 3, Flux.1, and AudioCraft 2 all switched to flow matching in 2024. DDPM's reverse process is a 1000-step stochastic walk from N(0, I) back to the data distribution. DDIM collapsed it to 20-50 deterministic steps. You want fewer steps — ideally one. The blocker is that the ODE solving the reverse process is stiff; the path is curved. If you could train the model such that the path from noise to data was a straight line, a single Euler step from t=1 to t=0 would work. Flow matching builds this directly: define a straight-line interpolation from x1 ∼ N(0, I) to x0 ∼ data, train a vector field vθ(x, t) to match its time derivative, integrate at inference. Rectified flow (Liu 2022) goes further: iteratively straighten the paths with a reflow procedure that produces a progressively closer-to-linear ODE. After two reflow iterations, a 2-step sampler matches 50-step DDPM quality. Flow matching: straight-line interpolation between noise and data Define: where x0 data and x1 N(0, I). The time derivative along this straight line is constant: Define a neural vector…
Flow Matching & Rectified Flows: Diffusion models take 20-50 sampling steps because they walk a curved path from noise to data. Flow matching (Lipman et al.,…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.