Semantic Segmentation — U-Net
Segmentation is classification at every pixel. U-Net makes it work by pairing a downsampling encoder with an upsampling decoder and wiring skip connections between them. Distinguish semantic, instance, and panoptic segmentation and pick the right task for a given problem. Build a U-Net from scratch in PyTorch with encoder blocks, a bottleneck, a decoder with transposed convolutions, and skip connections. Implement pixel-wise cross-entropy, Dice loss, and the combined loss that is the current default for medical and industrial segmentation. Read IoU and Dice metrics per class and diagnose whether a bad score comes from small-object recall, boundary accuracy, or class imbalance. Classification outputs one label per image. Detection outputs a handful of boxes per image. Segmentation outputs one label per pixel. For an input of size H x W, the output is a tensor of shape H x W (semantic) or H x W x Ninstances (instance). That is millions of predictions per image, not one. The structure of segmentation is why it powers almost every dense-prediction vision product: medical imaging (tumour masks), autonomous driving (road, lane, obstacle), satellite (building footprints, crop boundaries), document parsing (layout zones), robotics (graspable regions). None of those tasks can be solved by putting a box around the object; they need the exact silhouette. The architectural problem is simple to state and not simple to…
Semantic Segmentation — U-Net: Segmentation is classification at every pixel. U-Net makes it work by pairing a downsampling encoder with an upsampling decoder…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.