Stochastic Processes
Randomness with structure. The math behind random walks, Markov chains, and diffusion models. Language: Python Simulate 1D and 2D random walks and verify the sqrt(n) scaling of displacement. Build a Markov chain simulator and compute its stationary distribution via eigendecomposition. Implement Metropolis-Hastings MCMC and Langevin dynamics for sampling from target distributions. Connect the forward diffusion process to Brownian motion and explain how the reverse process generates data. Many AI systems involve randomness that evolves over time. Not static randomness -- structured, sequential randomness where each step depends on what came before. Language models generate tokens one at a time. Each token depends on the previous context. The model outputs a probability distribution, samples from it, and moves on. That is a stochastic process. Diffusion models add noise to an image step by step until it becomes pure static. Then they reverse the process, denoising step by step until a new image emerges. The forward process is a Markov chain. The reverse process is a learned Markov chain running backward. Reinforcement learning agents take actions in an environment. Each action leads to a new state with some probability. The agent follows a random policy in a random world. The whole thing is a Markov decision process. MCMC sampling -- the backbone of Bayesian inference -- constructs a Markov chain whose stationary…
Stochastic Processes: Randomness with structure. The math behind random walks, Markov chains, and diffusion models. Language: Python
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.