Keypoint Detection & Pose Estimation
A pose is a set of ordered keypoints. A keypoint detector is a heatmap regressor. Everything else is bookkeeping. Distinguish top-down and bottom-up pose estimation and state when each is used. Regress heatmaps for K keypoints with a Gaussian-per-keypoint target and extract keypoint coordinates at inference. Explain Part Affinity Fields (PAFs) and how bottom-up pipelines associate keypoints into instances. Use MediaPipe Pose or MMPose for production keypoint estimation and understand their output format. Keypoint tasks hide under many names: human pose (17 body joints), face landmarks (68 or 478 points), hand (21 points), animal pose, robotic object pose, medical anatomy landmarks. Every one of them shares the same structure: detect K discrete points on an object and output their (x, y) coordinates. Pose estimation is the foundation of motion capture, fitness apps, sports analytics, gesture control, animation, AR try-on, and robotic grasping. The 2D case is mature; 3D pose (estimating joint positions in world coordinates from a single camera) is the current research frontier. The engineering question is scale. A single-image, single-person pose is a 20ms problem. Multi-person pose in a crowd at 30 fps is a different problem with different architectures. Top-down — detect people first, then run a per-person keypoint model on each crop. Highest accuracy; scales linearly with number of people. Bottom-up — one forward pass predicts…
Keypoint Detection & Pose Estimation: A pose is a set of ordered keypoints. A keypoint detector is a heatmap regressor. Everything else is bookkeeping.
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.