Cheat sheets

Bias, Fairness and Differential Privacy

Measure bias, pick a fairness criterion, train with DP-SGD and audit the privacy claim. Stdlib Python toys.

  • Concepts and terms
  • Tasks and commands
  • Annotated code
  • Diagrams
  • Comparisons
  • Defaults and numbers
  • Troubleshooting
  • Questions and tips

Source

Three fairness families side by side#

FactorGroup fairnessIndividual fairnessCounterfactual fairness
RuleEqual rates across protected groupsSimilar individuals get similar decisionsDecision unchanged when sensitive attributes change
SourceDemographic parity, equalized odds, CUAEDwork et al. 2012Kusner et al. 2017
NeedsGroup labels and outcomesA similarity metric dA causal DAG
Formal testEqual rates per groupLipschitz constant LCounterfactual alteration
Weak pointCriteria conflict under unequal base ratesDefining d is a policy questionOnly as justified as the DAG

Use each criterion when#

  • Use demographic parity when the claim is equal acceptance rates.
  • Use equalized odds when the claim is equal TPR and FPR across groups.
  • Use conditional use accuracy equality when the claim is equal predictive value.
  • Use individual fairness when you can define a task-specific metric d.
  • Use counterfactual fairness when you have a causal DAG of the population.
  • Use backtracking counterfactuals when the law forbids interventions on protected attributes.

Formulas#

Demographic parity
P(Y=1 | A=a) = P(Y=1 | A=a') for all groups.
Equalized odds
P(Y=1 | Y=y, A=a) = P(Y=1 | Y=y, A=a').
Conditional use accuracy
P(Y=y | Y=y, A=a) = P(Y=y | Y=y, A=a').
Individual fairness
|f(x) - f(x')| <= L * d(x, x')
(ε, δ)-DP
P(M(D) \in S) \le e^{\varepsilon} \cdot P(M(D') \in S) + \delta
WEAT term
s(w, X, Y) = mean cosine(w, X) - mean cosine(w, Y)
WEAT effect
mean_a(s) - mean_b(s) over identity groups.

DP-SGD recipe#

1Samplersample a mini-batch.2Trainercompute per-example gradients.3Clipperclip each per-example gradient to a threshold C.4Noisesum the clipped gradients and add Gaussian noisewith std $\sigma \cdot C$.5Optimizeruse the noisy sum to update parameters.6Accountanttrack the privacy cost with Moments Accountantor Rényi DP accountant.

Where bias is measured#

representationbehaviourmodelEmbedding-basedWEAT on pre-RLHFembeddingsmodelProbability-basedstereotype log-likelihoodprocessGenerated-text-basedresume scoring, dialogueRepresentational harmstereotypes, erasureAllocational harmunequal material outcomesrepresentationbehaviourmodelEmbedding-basedWEAT on pre-RLHFembeddingsmodelProbability-basedstereotypelog-likelihoodprocessGenerated-text-basedresume scoring,dialogueRepresentationalharmstereotypes, erasureAllocational harmunequal materialoutcomes

Toy WEAT probe#

A = ["he", "his", "man"]identity group A
B = ["she", "her", "woman"]identity group B
X = ["engineer", "programmer", "scientist"]tech / career
Y = ["nurse", "teacher", "caregiver"]care / family
pre = weat_score(A, B, X, Y)positive: A closer to X
EMB = debias(EMB)project out the gender direction
gender_dir = [1.0, -1.0, 0.0, 0.0]axis 1 minus axis 0
post = weat_score(A, B, X, Y)drops, not to zero

Call the toy code#

TaskCode
Score a WEAT probeweat_score(<identity_a>, <identity_b>, ...)
Project out a bias directiondebias(<emb>)
Train with reweightingtrain(data, sample_weights=<w>)
Demographic parity per groupdemographic_parity(preds)
Equalized odds per groupequalized_odds(preds)
Clip one gradientclip(<g>, <C>)
Train with DP-SGDdp_sgd(data, epochs, lr, sigma, C)
Epsilon for a runanalytical_epsilon(sigma, steps)

Group metrics and DP reweighting#

train_data = gen(1000)A=0 base 0.3, A=1 base 0.6
test_data = gen(500)
baseline = train(train_data)steps 200, lr 0.1
preds = predict(baseline, test_data)(pred, label, group)
demographic_parity(preds)rate0, rate1
equalized_odds(preds)(TPR, FPR) per group
conditional_use(preds)(PPV, NPV) per group
dp_reweighted = train(train_data, sample_weights=weights)a=0 y=1: 2.0, a=1 y=1: 0.5

DP-SGD toy#

grad_w = clip(grad_w, C)scale to norm C
grad_b = max(-C, min(C, grad_b))clip the bias gradient
noise_b = random.gauss(0.0, sigma * C)noise std sigma * C
b -= lr * (grad_b + noise_b)noisy update per example
for sigma in (0.0, 0.5, 1.0, 2.0, 4.0):sweep the noise multiplier
model = dp_sgd(train_data, epochs=epochs, lr=0.05, sigma=sigma, C=C)
acc = accuracy(model, test_data)
eps = analytical_epsilon(max(sigma, 0.01), steps=epochs * len(train_data), delta=delta)

Toy defaults#

SettingDefaultNote
Clip norm C1.0Lesson 22 toy
delta1e-5Default of analytical_epsilon
epochs10DP-SGD toy, 500 train rows
DP-SGD lr0.05Fixed in the sweep
sigma0.0Standard SGD, infinite epsilon
Published ε[1, 10]Illustrative, not a recommended default
Fairness trainsteps: int = 200lr: float = 0.1

Troubleshooting#

SymptomCauseFix
WEAT effect does not drop to zeroDebiasing reduces but does not eliminate itAdd probability and generated-text metrics
Diverse portrayals, unequal decisionsOnly representational harm measuredMeasure allocational harm too
Gender eval passes, bias remainsBias fires on (gender, race) pairsRun intersectional tests
DP gap shrinks, EO and CUAE worsenUnequal base ratesState the chosen criterion as policy
Canary MIA shows little leakageCanaries are not "most extractable" dataAdd training-data extraction tests
Outputs safe, individuals re-identifiedConfidence scores leakDo not expose, or truncate/quantize them

Questions#

Can one model meet all three group criteria?
No. Unequal base rates prevent it (Chouldechova, KMR 2017).
Why backtracking counterfactuals?
They reason from actual features. They do not intervene on the attribute.
Why do MIA and extraction disagree?
They measure different things: inserted canaries vs "most extractable" data.
What does LoRA + DP-SGD protect?
The adapter. The base model is held fixed.
Why uncertainty in WinoIdentity?
Equal error with unequal uncertainty still changes allocation.