Cheat sheets

Probability and Statistics for ML

Distributions, Bayes, tests, bootstrap, sampling from logits and Markov chains. From-scratch Python and SciPy.

  • Concepts and terms
  • Tasks and commands
  • Annotated code
  • Diagrams
  • Defaults and numbers
  • Troubleshooting
  • Questions and tips

Source

Logits to a token#

probsthenLogitsone per tokenTemperaturedivide by TSoftmaxsubtract max firstTop-kkeep k, renormalizeTop-psmallest set with cumsum >= pSamplesample_from_probsprobsthenLogitsone per tokenTemperaturedivide by TSoftmaxsubtract max firstTop-kkeep k, renormalizeTop-psmallest set withcumsum >= pSamplesample_from_probs

From Lesson 01.16, 01.16 code/sampling.py

Bayesian A/B test#

1Priorstart each variant at Beta(1, 1). It is uniform.2DataA gets 50 clicks in 1000 views. B gets 65 clicksin 1000 views.3Updatebeta_update(alpha, beta_param, successes,failures) adds the counts.4PosteriorsA is Beta(51, 951), mean 0.051. B is Beta(66,936), mean 0.066.5Monte Carlodraw 100,000 samples from each. P(B > A) is thefraction where B wins.6Decideship B above 0.95. Ship A below 0.05. Elsecollect more data.

From Lesson 01.07, 01.07 code/bayes.py

Tasks in code#

TaskCode
Posterior from one testbayes(prior=0.0001, likelihood=0.99, false_positive_rate=0.01)
Repeat the testsequential_bayes(prior, likelihood, false_positive_rate, num_tests)
Spam filter with add-one smoothingNaiveBayes(smoothing=1.0), then classifier.train(train_docs, train_labels)
Loss from logitscross_entropy_loss(logits, target_index)
Bootstrap confidence intervalbootstrap_statistic(data, stat_func, n_bootstrap=5000, ci=95)
Welch t-test, paired t-testtwo_sample_ttest(data1, data2), paired_ttest(data1, data2)
Effect sizecohens_d(data1, data2), then interpret_cohens_d(d)
Correct for many testsbonferroni_correction(p_values, alpha=0.05)

From Lesson 01.07, 01.07 code/bayes.py, 01.15 code/statistics.py, 01.06 code/probability.py

Sample from scratch#

x = sample_exponential_inverse_cdf(lam)-log(u) / lam
x = rejection_sample(target_pdf, proposal_sample, proposal_pdf, M)accept rate 1/M
est = importance_sampling_estimate(f, target_pdf, proposal_pdf, proposal_sample, n)
idx = temperature_sample(logits, temperature)z / T, then softmax
idx = top_k_sample(logits, k)keep the k largest logits
idx = top_p_sample(logits, p)stop when cumsum >= p
z = reparam_sample(mu, sigma)mu + sigma * epsilon
y = gumbel_softmax_sample(log_probs, temperature)soft one-hot

From Lesson 01.16, 01.16 code/sampling.py

Library calls#

normal = stats.norm(loc=0, scale=1)
normal.cdf(1.96)P(X < 1.96)
normal.ppf(0.975)inverse CDF
samples = normal.rvs(size=10000)
probs = softmax(logits)from scipy.special
rng = np.random.default_rng(42)
token = rng.choice(len(logits), p=probs)
distribution = distribution @ Ppower method, repeat 100 times
eigenvalues = np.linalg.eigvals(P)
spectral_gap = 1 - sorted(np.abs(eigenvalues))[-2]mixing time near 1/gap

From Lesson 01.06, Lesson 01.16, Lesson 01.22

Key formulas#

Bayes
P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B)}. P(B) is the normalizer.
Variance
\mathrm{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2. Sample variance divides by n-1.
Softmax with T
p_i = \exp(z_i / T) / \sum_j \exp(z_j / T).
Laplace smoothing
add 1 to each count and the vocabulary size to the total.
Beta update
prior Beta(a, b) with s successes and f failures gives Beta(a + s, b + f).
Importance weight
w(x) = p(x) / q(x). Average f(x_i) \, w(x_i) over x_i \sim q.
Metropolis-Hastings
accept with \min(1, a). A symmetric proposal gives a = p(x') / p(x).
Langevin
x_{t+1} = x_t - dt \, \nabla U(x_t) + \sqrt{2 T \, dt} \, z_t.

From Lesson 01.06, Lesson 01.07, Lesson 01.15, Lesson 01.16, Lesson 01.22

Distributions#

CommandWhat it does
BernoulliMean p, variance p(1-p). Binary classification
CategoricalProbabilities sum to 1. Softmax output
PoissonMean and variance both lambda. Event counts
Uniform(0, 1)E[U] = 0.5, Var(U) = 1/12. Start of every sampler
Normal68%, 95%, 99.7% within 1, 2, 3 sigma
Beta(a, b)Mean a/(a+b). Conjugate prior for Bernoulli
ExponentialInverse CDF gives x = -ln(u) / lambda
Brownian motionB(t) is normal with mean 0 and variance t

From Lesson 01.06, Lesson 01.07, Lesson 01.16, Lesson 01.22

Pick a test#

CommandWhat it does
One-sample t-testMean against mu_0. Degrees of freedom n - 1
Welch t-testTwo independent groups. No equal variance assumption
Paired t-testSame folds for both models. Test d_i against 0
Chi-squaredObserved against expected frequencies
Mann-Whitney UTwo groups without a normal assumption
Wilcoxon signed-rankPaired data. Good for 5 or 10 folds
Spearman rhoMonotonic relation on ranks. Robust to outliers
BootstrapAny metric. 95% CI from the 2.5th and 97.5th percentiles

From Lesson 01.15

Numbers and defaults#

SettingDefaultNote
Significance level alpha0.05Reject H0 when p is below it
z for a 95% interval1.96x_bar plus or minus z times s / sqrt(n)
n_bootstrap5000The lesson range is 1000 to 10000
CLT sample sizen >= 30Use n >= 100 for skewed data
Cohen's d, large effect0.80.2 small, 0.5 medium
MH acceptance, Gaussian proposal0.234High dimensions
Top-k k40Typical setting
Temperature with top-p0.7 + top-p 0.9General purpose

From Lesson 01.07, Lesson 01.15, 01.15 code/statistics.py, Lesson 01.16

Troubleshooting#

SymptomCauseFix
exp(102) overflows in softmaxLarge logitsSubtract the max logit first
Sequence probability is 0.0Underflow after about 30 termsAdd log probabilities
Naive Bayes score is 0Unseen word has MLE count 0Add-one Laplace smoothing
99% test gives about 1% sickRare prior dominatesUse the base rate. Run a second test
One of 20 metrics is significant64% chance of a false positiveBonferroni: alpha / m
Tiny p-value on a 0.03% gainVery large nReport the effect size too
MH chain stuck or slowProposal scale too large or too smallTune toward about 23% acceptance
A few samples dominate the estimateProposal q far from pSelf-normalize the weights

From Lesson 01.06, Lesson 01.07, Lesson 01.15, Lesson 01.16, Lesson 01.22

Questions#

Is the p-value the chance that H0 is true?
No. It is P(data this extreme given H0).
What does a 95% CI mean?
95% of intervals from repeated experiments contain the true mean.
Why the reparameterization trick?
Sampling blocks gradients. z = mu + sigma * epsilon is differentiable.
Why top-p over top-k?
Top-p adapts the candidate set to the model confidence.
How fast does a chain mix?
A spectral gap of 0.2 needs about 5 steps. 0.01 needs about 100.
How does MAP relate to regularization?
A Gaussian prior is L2. A Laplace prior is L1.

From Lesson 01.15, Lesson 01.16, Lesson 01.22, Lesson 01.07

Tips#

  • Evaluate both models on the same test set and the same folds.
  • Report the p-value and the effect size together.
  • Monte Carlo error is O(1 / sqrt(N)) in any dimension.
  • Discard burn-in samples. Run several chains from different starts.
  • A random walk drifts about sqrt(n) from the origin after n steps.
  • Use T = 0.0 (greedy) for factual answers and deterministic tasks.

From Lesson 01.15, Lesson 01.16, Lesson 01.22