Models & inference · Glossary term
What is Top-k Sampling?
A decoding method that restricts the next-token distribution to the k highest-scoring candidates, renormalizes their probabilities, and samples from that set.
Why does Top-k Sampling matter?
It removes the long low-probability tail from sampling while keeping a fixed maximum candidate count.
Top-k Sampling in practice
Evaluate k together with temperature, top-p, and stop settings, and record the complete sampler configuration with generated results.
What is the common confusion about Top-k Sampling?
Top-k uses a fixed candidate count, while top-p uses a probability-mass threshold whose candidate count changes by step.
Learn Top-k Sampling in the course
Lessons that name Top-k Sampling in a title or section
- Sampling Methods
Sampling is how AI explores the space of possibilities. Language: Python Implement inverse CDF, rejection, and importance sampling from scratch using only uniform random numbers.
Covered in Phase 01: Math Foundations.
Related terms
- Nucleus Sampling (Top-p)A decoding method that samples from the smallest set of next-token candidates whose cumulative probability reaches a chosen threshold.
- TemperatureA decoding parameter that rescales logits before a probability distribution is formed.
- Decoding StrategyThe algorithm that converts a model's sequence of next-token scores into selected tokens and a completed output.
- LogitsThe model's unnormalized numeric scores for candidate outcomes before a normalization function or decoding rule converts them into…
Sources
More terms in Models & inference
- Attention
- Autoregressive
- CNN (Convolutional Neural Network)
- CUDA
- Decoder
- Decoding Strategy
- Diffusion Model
- Encoder
- GAN (Generative Adversarial Network)
- GPT
- Inductive Bias
- Inference
- KV Cache
- LLM (Large Language Model)
- Logits
- MoE (Mixture of Experts)
- Nucleus Sampling (Top-p)
- Parameter
- Perplexity
- Quantization
- Self-Attention
- Speculative Decoding
- Stop Sequence
- Streaming
- Temperature
- Time to First Token (TTFT)
- Transformer
- VAE (Variational Autoencoder)
This entry comes from glossary/terms.md on GitHub. Browse all 250 glossary terms.