Linear Systems
Solving Ax = b is the oldest problem in mathematics that still runs your neural network. Language: Python Solve Ax = b using Gaussian elimination with partial pivoting and back substitution. Factor matrices with LU, QR, and Cholesky decompositions and explain when each is appropriate. Derive the normal equations for least squares and connect them to linear and ridge regression. Diagnose ill-conditioned systems using the condition number and apply regularization to stabilize them. Every time you train a linear regression, you solve a linear system. Every time you compute a least-squares fit, you solve a linear system. Every time a neural network layer computes y = Wx + b, it is evaluating one side of a linear system. When you add regularization, you modify the system. When you use Gaussian processes, you factor a matrix. When you invert a covariance matrix for Mahalanobis distance, you solve a linear system. The equation Ax = b appears everywhere. A is a matrix of known coefficients. b is a vector of known outputs. x is the vector of unknowns you want to find. In linear regression, A is your data matrix, b is your target vector, and x is the weight vector. The entire model reduces to: find x such that Ax is as close to b as possible. This lesson builds every…
Linear Systems: Solving Ax = b is the oldest problem in mathematics that still runs your neural network. Language: Python Solve Ax = b using Gaussian…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.