Norms and Distances
Your distance function defines what "similar" means. Choose wrong and everything downstream breaks. Language: Python Implement L1, L2, cosine, Mahalanobis, Jaccard, and edit distance functions from scratch. Select the appropriate distance metric for a given ML task and explain why alternatives fail. Connect L1 and L2 norms to LASSO and Ridge regularization and their geometric constraint regions. Demonstrate how the same dataset produces different nearest neighbors under different metrics. You have two vectors. Maybe they are word embeddings. Maybe they are user profiles. Maybe they are pixel arrays. You need to know: how close are they? The answer depends entirely on which distance function you pick. Two data points can be nearest neighbors under one metric and far apart under another. Your KNN classifier, your recommendation engine, your vector database, your clustering algorithm, your loss function -- they all depend on this choice. Get it wrong and your model optimizes for the wrong thing. There is no universal best distance. L2 works for spatial data. Cosine similarity dominates NLP. Jaccard handles sets. Edit distance handles strings. Mahalanobis accounts for correlations. Wasserstein moves probability mass. Each one encodes a different assumption about what "similar" means. This lesson builds every major distance function from scratch, shows you when each one is the right tool, and demonstrates how the same data produces completely…
Norms and Distances: Your distance function defines what "similar" means. Choose wrong and everything downstream breaks. Language: Python
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.