How this manual was made

The version

The manual pins SGLang 0.5.22, tagged on 2026-10-13 at commit baa9befa381511cd4d56f83980951e5a45fffc6d. The facts were verified on 2026-10-16. SGLang releases about every three weeks, and several defaults changed between 0.5.20 and 0.5.22. A recipe written for an older version can be wrong at this one, and the sections say where.

The sources, ranked

Facts come from seven sources, ranked by authority. When two of them disagree, the higher one wins and the text says so.

The documentation ranks below the code on purpose. At 0.5.22 the documentation still lists two attention backends that the code removed. It also gives other HiCache prefetch timeouts than the code uses. When we say the default, we mean the value the code sets.

Four kinds of evidence

Every number in this manual carries one of four labels in the text.

We measured: the number comes from our runs on the Mac or on the GPUs, recorded on the machines in the tables below. You can run the kit and get the same facts.

The code: the default or the mechanism comes from the SGLang source at the tag. The Sources line names the files.

A field report: an engineer or a company outside the SGLang team ran SGLang and published the result. The text names them, the month, the version when they stated it, and what they tuned.

The team reports or the vendor reports: the SGLang team, RadixArk, or a hardware vendor published the number. We repeat it as their claim and set it against independent results where they exist.

DeepEP and prefill-decode disaggregation across many GPUs need hardware that we did not have. For those sections, the code and the field reports carry the argument, and the label says so.

The capture kit

The capture kit starts the real SGLang server on Apple silicon through its MLX backend. It sends requests, runs the official benchmark, and writes what it sees to its output folder. python3 capture/run.py records everything. python3 capture/run.py --check records again into a temporary directory and compares the facts that must not change. Those facts are the generated text at temperature 0, the token and cached-token counts, and the request counts. The KV bytes per token and the metric names are facts too.

Timings change on every run, so the check does not compare them.

The kit needs SGLang 0.5.22 with its MLX extra on Apple silicon, so it cannot run in the repository CI. Without that setup, --check prints skipped with the reason and exits 0. Every file of the Mac runs was recorded on one machine:

The kit starts four servers. They are the default configuration, one without the radix cache, one without the overlap scheduler, and one with a 4,096-token pool. The server without the overlap scheduler crashed on its first real batch at 0.5.22, and the kit records that crash on purpose. Section 2.4 reads it.

The GPU kit

The GPU kit runs the same idea on an NVIDIA GPU. Its run script starts eight server configurations and writes the GB10 run logs. The sweep script repeats each benchmark point with three seeds and records the GPU clock and temperature of every run. The summary script writes one summary for each machine: the GB10 summary data, the L4 summary data, and the throttled L4 summary data. The Miles summary script writes the Miles summary data. The charts read those four files.

Each row below is one sweep of the GPU kit. The link opens its summary file, and each section gives the date and the clocks of the runs that it uses.

Every GB10 run log and the GB10 summary data were recorded on one machine:

The kit also has an l4 profile for Qwen3-8B on an NVIDIA L4, which wrote the L4 run logs and the L4 summary data. Part 1 uses that L4 server for its listings. Our first L4 sweep ran hot and throttled, and section 5.1 tells it as an incident. We then ran the L4 again with a cool-down to 72°C before each run. Section 4.3, section 4.4, and section 5.1 use those runs, and section 5.1 shows that the cooled card still lowered its clock under long load.

Conventions

sglang serve starts the server in every command. The older python -m sglang.launch_server still works at 0.5.22 and prints a deprecation warning. Ports are the default 30000. ~ stands for the home directory in recorded output.

Citations name a place. {{issue #35129}} names an issue in the SGLang repository. {{field Jarvislabs 2026-05}} names a field report by its author and month, and the text links it. The SGLang source files a section was written from are listed once, in the Sources line at its end. A listing copies lines from a capture file unchanged, and a … marks a cut that the note explains.

Figures animate on the web, where a Replay button runs the steps again. Print and reduced motion show the final frame.