Use this page to look up a flag that a section names.
Defaults are for vllm serve at 0.31.0 on a CUDA GPU, read from the configuration classes in the source.
Server
| Flag | Default | What it sets | Page |
|---|
--host | not set, so every network interface | the address the server listens on | server options |
--port | 8000 | the HTTP port | server options |
--served-model-name | the model name | the name that clients send in model | server options |
--api-key | not set | the key that the /v1 routes require | server options |
--chat-template | the template of the model | the chat template, as a file or as text | server options |
--default-chat-template-kwargs | not set | template values for every request | reasoning models |
--reasoning-parser | not set | the parser that moves thinking out of the answer | reasoning models |
--enable-log-requests | off | one log line for each request | server options |
Memory and length
| Flag | Default | What it sets | Page |
|---|
--gpu-memory-utilization | 0.92 | the share of total GPU memory for one instance | startup |
--kv-cache-memory-bytes | not set | the KV cache size in bytes, which overrides the share for the cache | startup |
--max-model-len | the model config | the longest prompt plus answer for one request | long contexts |
--block-size | 16 | tokens in one KV block, or the attention backend's preference | block pool |
--kv-cache-dtype | auto | the data type of the KV cache, such as fp8 | attention backend |
--num-gpu-blocks-override | not set | a fixed number of KV blocks, for tests | startup |
Scheduling
| Flag | Default | What it sets | Page |
|---|
--max-num-batched-tokens | 2048 for vllm serve on a GPU under 70 GiB or an A100, 8192 from 70 GiB, 16384 from 160 GiB | the token budget for one step | token budget |
--max-num-seqs | 256 on a GPU under 70 GiB or an A100, 1024 from 70 GiB | the most requests that run at once | token budget |
--enable-chunked-prefill | on | splitting a long prompt across steps | token budget |
--async-scheduling | on for most generate models | planning the next step while the GPU runs | Model Runner V2 |
--scheduling-policy | fcfs | first come first served, or priority | token budget |
--long-prefill-token-threshold | not set | a cap on the tokens of one prefill in one step | token budget |
Cache
| Flag | Default | What it sets | Page |
|---|
--enable-prefix-caching | on | reuse of cached prefix blocks | block pool |
--prefix-caching-hash-algo | sha256 | the hash of each block | block pool |
--disable-hybrid-kv-cache-manager | off | one group for every layer kind | long contexts |
--kv-transfer-config | not set | a KV connector and its role | KV connectors |
Execution
| Flag | Default | What it sets | Page |
|---|
-O0 to -O3 | -O2 | the compile and CUDA graph level | compile and graphs |
--enforce-eager | off | no compile, no CUDA graphs, and no kernel warmup | compile and graphs |
--max-cudagraph-capture-size | set by the level | the largest batch with a captured graph | compile and graphs |
--attention-backend | chosen from a list | the attention kernel, by enum name | attention backend |
--speculative-config | not set | the speculation method and its options, as JSON | speculative decoding |
Parallelism
| Flag | Default | What it sets |
|---|
--tensor-parallel-size, -tp | 1 | GPUs that split each layer |
--pipeline-parallel-size, -pp | 1 | GPUs that split the layers into stages |
--data-parallel-size, -dp | 1 | full copies of the model |
--enable-expert-parallel | off | experts spread across GPUs for mixture-of-experts models |
Environment variables
| Variable | Default | What it sets |
|---|
VLLM_ENABLE_V1_MULTIPROCESSING | 1 | the engine core in its own process |
VLLM_USE_V2_MODEL_RUNNER | not set | a pinned choice of model runner |
VLLM_BATCH_INVARIANT | 0 | the batch-invariant mode, when set to 1 |
VLLM_API_KEY | not set | the API key, as --api-key sets it |
VLLM_LOGGING_LEVEL | INFO | the vLLM log level |
VLLM_NO_USAGE_STATS | 0 | no usage statistics, when set to 1 |
vLLM 0.31.0 · db9527a · 2026-10-05 · Edition 2026.10