Prompting & context · Glossary term
What is Prompt Sensitivity?
Variation in model output or measured performance caused by changes to prompt wording, order, formatting, or examples that preserve the intended task.
Why does Prompt Sensitivity matter?
A system that succeeds under one convenient phrasing may be unreliable for real users or misleading in evaluation.
Prompt Sensitivity in practice
Create semantically equivalent prompt variants, measure variance by case, and keep variants in regression tests instead of optimizing one prompt against one eval set.
What is the common confusion about Prompt Sensitivity?
Sensitivity is not always a prompt defect; it can reveal ambiguity, weak model robustness, unstable decoding, or an inadequate scoring rule.
Learn Prompt Sensitivity in the course
No lesson links to this term yet. Search the course catalog for it.
Related terms
- Prompt EngineeringDesigning model-facing instructions, examples, constraints, and output requirements to improve behavior on a defined task.
- Eval SetA versioned collection of inputs, expected properties, scoring rules, and metadata used to measure an AI system against a defined…
- Regression TestA repeatable check that protects behavior known to work, especially after code, prompt, model, retrieval, or tool changes.
- Few-ShotIn-context learning that includes a small set of demonstrations before the target input so the model can infer the desired task, format,…
Sources
More terms in Prompting & context
This entry comes from glossary/terms.md on GitHub. Browse all 250 glossary terms.