Convolutions from Scratch
A convolution is a tiny dense layer you slide across an image, sharing the same weights at every location. Implement 2D convolution from scratch using only NumPy, including the nested-loop version and a vectorised im2col version. Compute output spatial size for any combination of input size, kernel size, padding, and stride, and justify the (H - K + 2P) / S + 1 formula. Hand-design kernels (edge, blur, sharpen, Sobel) and explain why each one produces the pattern of activations it does. Stack convolutions into a feature extractor and connect the depth-of-the-stack to the size of the receptive field. A fully connected layer on a 224x224 RGB image would need 224 224 3 = 150,528 input weights per neuron. A single hidden layer with 1,000 units is already 150 million parameters — before you have learnt anything useful. Worse, that layer has no notion that a dog in the top-left and a dog in the bottom-right are the same pattern. It treats every pixel position as independent, which is exactly wrong for images: translating a cat by three pixels should not force the network to relearn the concept. The two properties an image model needs are translation equivariance (the output shifts when the input shifts) and parameter sharing (the same feature detector runs everywhere). Dense layers give you neither. Convolution…
Convolutions from Scratch: A convolution is a tiny dense layer you slide across an image, sharing the same weights at every location.
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.