Diffusion Models — DDPM from Scratch
Ho, Jain, Abbeel (2020) gave the field a recipe it could not quit. Destroy the data with noise over a thousand small steps. Train one neural net to predict the noise. Reverse the process at inference. Today every mainstream image, video, 3D, and music model runs on this loop, possibly with flow matching or consistency tricks on top. You want a sampler for pdata(x). GANs play a minimax game that often diverges. VAEs produce blurry samples from a Gaussian decoder. What you really want is a training objective that is (a) a single stable loss (no saddle point, no minimax), (b) a lower bound on log p(x) (so you have likelihoods), and (c) samples that match SOTA quality. Sohl-Dickstein et al. (2015) had a theoretical answer: define a Markov chain q(xt | x{t-1}) that gradually adds Gaussian noise, and train a reverse chain pθ(x{t-1} | xt) to denoise. Ho, Jain, Abbeel (2020) showed the loss could be simplified to one line — predict the noise — and cleaned up the math. In 2020 this was a curiosity. In 2021 it produced state-of-the-art samples. In 2022 it became Stable Diffusion. In 2026 it is the substrate. DDPM: forward noise, reverse denoise Forward process q. Add Gaussian noise in T small steps. The closed form — the reason the math is tractable…
Diffusion Models — DDPM from Scratch: Ho, Jain, Abbeel (2020) gave the field a recipe it could not quit. Destroy the data with noise over a thousand small…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.