Practice track
Train tiny models
Train a small model in your browser to a target score inside a step budget. 5 problems, about 175 minutes. Do them in order: each problem uses ideas from the problems before it.
- 01Train softmax regression on four blobs
Train multinomial logistic regression with gradient descent to reach a held-out accuracy and log loss target on four Gaussian blobs.
From Logistic Regression
TrainEasy25 min - 02Train a 2-4-1 network on XOR
Train a small sigmoid network with full-batch gradient descent until it labels noisy XOR points with 95 percent held-out accuracy.
TrainEasy30 min - 03Fit a sine curve with a tanh network
Train a 1-16-1 tanh network on noisy samples of sin(x) until its held-out mean squared error against the true curve is below 0.01.
From Activation Functions
TrainMedium35 min - 04Train a tanh network on two spirals
Train a 2-24-24-1 tanh network with mini-batch Adam until it separates two noisy spirals with 95 percent held-out accuracy.
TrainMedium45 min - 05Train a character bigram language model
Train a table of next-character logits with gradient descent until the held-out negative log-likelihood beats a unigram model.
From Text Generation Before Transformers — N-gram Language Models
TrainMedium40 min