Size the weight update
On one GPU the weight update is a small fixed cost, so it is not the step to tune first.
After each training phase, the trainer copies its new weights into the rollout engine. On one GPU this copy stays inside the GPU, and it takes under a second for the models we ran.
When you finish this section, you can measure the weight update and decide if it matters for your run.
Our result: the weight update took 0.64 s for Qwen3-0.6B and 0.91 s for Qwen3-1.7B, under one percent of a step.
Read the update time
Every step logs perf/update_weights_time in seconds. Read it next to perf/step_time. We ran 4 short steps for two model sizes, with answers cut at 256 tokens:
| Model | Weight update, 4 steps | Trainer offload, steps 2 and 3 | Trainer onload, steps 2 and 3 |
|---|---|---|---|
| Qwen3-0.6B | 0.63 to 0.66 s | 1.08 to 1.20 s | 0.77 to 0.86 s |
| Qwen3-1.7B | 0.89 to 0.95 s | 2.63 to 3.20 s | 1.42 to 1.45 s |
Steps 0 and 1 are warm-up steps. In these 4-step runs, their trainer offload took up to 4.1 s for 0.6B and 11.8 s for 1.7B. The table leaves them out.
Trainer offload: the time the trainer takes to move its weights and optimizer state to host memory after training, logged as perf/sleep_time. Trainer onload: the time it takes to move them back before the next log probabilities, logged as perf/wake_up_time.
The larger model is about 2.8 times the size, and its update took about 1.4 times as long. The trainer offload and onload grew faster than the update did. A likely reason is that the update sends only the weights, while the offload also moves the gradients and the optimizer state.
The short answers cut almost every reply, so the reward in these runs stayed near 0. These runs measure the copy, not the training.
Compare it with the step
In our 48-step run with Qwen3-0.6B, a full step took 152.6 s at the median, and the update took 0.64 s. That is under half a percent of the step. The 1.7B model gave a similar share.
So tune the rollout and the training phase first. Come back to the update only when it passes a few percent of the step.
Change the bucket size only with a reason
The trainer sends the weights in buckets. --update-weight-buffer-size sets the bucket size in bytes. Our runs used 536870912, which is 512 MiB.
A larger bucket means fewer transfers and a higher memory peak during the update. On one GPU, with updates under a second, we left it at 512 MiB.
Miles radixark/miles:latest, Miles main of 2026-10-15 · 08752a1 · 2026-10-15 · Edition 2026.10