Practice track
Classical ML
The models that come before deep learning, written from the equations. 6 problems, about 160 minutes. Do them in order: each problem uses ideas from the problems before it.
- 01Linear regression by gradient descent
Mean squared error, its gradients for weights and bias, and a batch gradient descent loop that records the cost after each update.
From Linear Regression
ImplementEasy20 min - 02Train softmax regression on four blobs
Train multinomial logistic regression with gradient descent to reach a held-out accuracy and log loss target on four Gaussian blobs.
From Logistic Regression
TrainEasy25 min - 03K-means assignment and update
The k-means assignment and update steps with a tie rule and an empty cluster rule, and a full run until the assignments stop changing.
ImplementMedium25 min - 04Logistic regression from scratch
A stable sigmoid, the binary cross-entropy loss and its gradient, and a gradient descent loop that records the loss.
From Logistic Regression
ImplementMedium25 min - 05Measure bias and variance
Fit polynomials of degree 1 to 12 on resampled training sets and measure bias squared, variance and test error at each degree.
LabMedium30 min - 06Gini impurity and the best split
Gini impurity and the best decision tree split over all features, with midpoint thresholds, weighted child impurity and a fixed tie rule.
ImplementHard35 min