Data & representations · Glossary term
What is Data Leakage?
Unintended use of information during training or feature construction that would not be available at the real prediction point or belongs to a held-out evaluation boundary.
Why does Data Leakage matter?
Leakage produces optimistic metrics that collapse when the system encounters genuinely unseen inputs.
Data Leakage in practice
Split data before fitting preprocessors, keep future information out of historical features, and isolate test labels and benchmark answers from prompts and tuning loops.
What is the common confusion about Data Leakage?
Leakage is not limited to duplicate rows. Global normalization statistics, timestamps, target-derived features, and repeated test-driven prompt edits can all leak information.
Learn Data Leakage in the course
Lessons that name Data Leakage in a title or section
- ML Pipelines
A model is not a product. A pipeline is. The pipeline is everything from raw data to deployed prediction, and every step must be reproducible.
Covered in Phase 02: ML Fundamentals.
Related terms
- Dataset SplitA documented partition of examples into separate subsets for fitting, development decisions, and final evaluation.
- Benchmark ContaminationOverlap or information leakage between evaluation examples and data used to pretrain, tune, prompt, select, or otherwise improve the…
- Eval SetA versioned collection of inputs, expected properties, scoring rules, and metadata used to measure an AI system against a defined…
- Data ProvenanceTraceable information about where data originated, who or what transformed it, which versions were used, and how derived artifacts relate…
- Membership InferenceAn attack that estimates whether a particular record or example was included in a model's training data by observing model outputs or…
Sources
More terms in Data & representations
This entry comes from glossary/terms.md on GitHub. Browse all 250 glossary terms.