The Perceptron
The perceptron is the atom of neural networks. Split it open and you find weights, a bias, and a decision. Implement a perceptron from scratch in Python, including the weight update rule and step activation function. Explain why a single perceptron can only solve linearly separable problems and demonstrate the XOR failure case. Construct a multi-layer perceptron by composing OR, NAND, and AND gates to solve XOR. Train a two-layer network with sigmoid activation and backpropagation to learn XOR automatically. You know vectors and dot products. You know that a matrix transforms inputs into outputs. But how does a machine learn which transformation to use? The perceptron answers this. It's the simplest possible learning machine: take some inputs, multiply by weights, add a bias, and make a binary decision. Then adjust. That's it. Every neural network ever built is layers of this idea stacked together. Understanding the perceptron means understanding what "learning" actually means in code: adjusting numbers until the output matches reality. A perceptron takes n inputs, multiplies each by a weight, sums them up, adds a bias, and passes the result through an activation function. The step function is brutal: if the weighted sum plus bias is >= 0, output 1. Otherwise, output 0. This is a linear classifier. The weights and bias define a line (or hyperplane…
The perceptron is the atom of neural networks. Split it open and you find weights, a bias, and a decision. Implement a perceptron from scratch in Python,…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.