Conditional GANs & Pix2Pix
The first big unlock of 2014-2017 was controlling what a GAN makes. Attach a label, or an image, or a sentence. Pix2Pix did the image version and it still beats every generic text-to-image model on narrow image-to-image tasks. An unconditional GAN samples arbitrary faces. Useful for a demo, useless in production. You want: map a sketch to a photo, map a map to an aerial photo, map a daytime scene to nighttime, colorize a grayscale image. In all of these, you are given an input image x and must output y with some semantic correspondence. There are many plausible ys per x. Mean-squared error flattens them into mush. An adversarial loss doesn't, because "looks real" is sharp. Conditional GAN (Mirza & Osindero, 2014) adds a condition c as an input to both G and D. Pix2Pix (Isola et al., 2017) specialized this: condition is a full input image, generator is a U-Net, discriminator is a patch-based classifier (PatchGAN), and loss is adversarial + L1. That recipe outperforms from-scratch text-to-image models on narrow image-to-image domains even in 2026 because it is trained on paired data — you have exactly the signal you need. Pix2Pix: U-Net generator, PatchGAN discriminator Conditional G. G(x, z) → y. In Pix2Pix, z is dropout inside G (no input noise — Isola found explicit noise got ignored).…
Conditional GANs & Pix2Pix: The first big unlock of 2014-2017 was controlling what a GAN makes. Attach a label, or an image, or a sentence. Pix2Pix did the…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.