Sources
The manual ranks its sources by authority, with our own recorded runs first, and this page lists each kind and what the sections take from it.
The ranks match how this manual was made. When two sources disagree, the higher one wins and the section says so. Every source was read or recorded on 2026-11-12, one week after the release of Agent Substrate v1.0.0.
The ranked sources
| Rank | Source | What the manual takes from it |
|---|---|---|
| 1 | our recorded runs on six clusters on four machines: five with Agent Substrate v1.0.0 and one with the Substrate distribution of kagent v1.0.0. Two of the six ran agentgateway v1.7.0 | every listing, and every number in a paragraph that starts with "We measured" |
| 2 | the Agent Substrate source code at the release tag v1.0.0 | every default, state, flag, and mechanism, and the reference tables |
| 3 | the Agent Substrate documentation at the same tag | install steps, the upgrade order, and documented behaviour that we did not run |
| 4 | Agent Substrate issues and pull requests on GitHub | known failures, limits, and the history of a feature |
| 5 | the kagent source code at its tag v1.0.0 | how a session becomes an actor, and the egress policy that kagent writes |
| 6 | the agentgateway source code at its tag v1.7.0 | the Substrate policies of the agentgateway dataplane |
| 7 | the Agent Executor code and documentation at one commit, built and run with no model and then with a small open model on CPU | how a task becomes an actor in 7.5 |
Our runs rank above the code on purpose. The code says what the system is built to do. A run says what it did on a real cluster, with the timers, the images, and the installer of the release. Where a run contradicts the documentation, the section reports both.
Rank 1: our recorded runs
Twenty-four experiments, each with its scripts and raw output in the capture folder. How this manual was made describes the machines and the method.
| Experiment | What it recorded | Used in |
|---|---|---|
| A, the quickstart from source | each install step with its seconds, and the buildx failure | Part 2, 8.3 |
| B, the lifecycle | 900 s with no request, then suspend, resume, crash, and revert | Part 1, Part 3 |
| C, resume against memory | resume and suspend times for actors that hold 0 to 1,000 MiB, from the node and from the bucket, and the disk use | Part 3, 8.1 |
| D, durability | the snapshot timers, a lost node copy, an atelet restart, and a node restart | Part 3, 8.2 |
| E, dependency loss | one request each second while Postgres, ate-api-server, and the Kubernetes API server were away | 1.2, 8.2 |
| F, egress | default deny, allowed hosts, interception, pass-through, and credential injection on both dataplanes | Part 6 |
| G, the dataplanes | the install from release images, and the same inventory for Envoy and for agentgateway | Part 2, Part 6, 7.2 |
| I, capacity | a full pool, and the request parking demo | Part 4 |
| J, death in a sandbox | a process that exits, and one that the kernel kills for memory | 3.5, 8.3 |
| K, kagent | the documented kagent install, 98 turns, parallel turns, and what a files-only snapshot keeps | 7.1 |
| L, the gaps | the microVM class on the first guest, node stops on a cluster with two worker nodes on the second guest (16 vCPU, 23 GB), and worker recovery after a node restart | 4.2, 4.3, 3.5, 8.2 |
| M, Apple silicon | the install from release images and three actor sizes on a laptop | Part 2, Part 3 |
| N, the demos | an idle suspender, a stateful agent, a tool sandbox, and a failure drill, on their own install on the second guest with 16 vCPU and 31 GB | Part 9 |
| P, a node that stops | 13 cases on the cluster with two worker nodes: reverts during and after the outage, a rescue from a tag, an async-suspended actor, a running actor, and the return of the node | 3.4, 3.5, 4.3, 8.2 |
| Q, network checks | eight checks on the second guest (16 vCPU, 23 GB): dependency loss on agentgateway, a reused connection after a rule removal, a model gateway behind egress over TLS, the egress log level, a suspend as a cut-off, a changed Secret, the router ports, and the CPU limit | 4.1, Part 6, 7.2, 8.2 |
| R, Agent Executor | the install of its head commit on Agent Substrate v1.0.0, what each of its objects becomes, lifecycle times, and where its documents and the run disagree | 7.5 |
| S, a real model | a small open model on CPU, kagent turns with it, and five Agent Executor follow-ups: a model call, the example agent image, a clone that succeeds, worker loss under a task, and release v0.3.1 | 7.1, 7.5 |
| T, a drain and a bucket outside the cluster | five drains of one worker node, a node stop with the other node full, and a bucket outside the cluster with added delay and with an outage, on the second guest (16 vCPU, 23 GB) | 4.3, 8.1, 8.2 |
| U, Postgres, upgrade, and scale | an install with a Postgres outside the cluster and its outage, one upgrade from v1.0.0-rc.3 to v1.0.0, and bursts of resumes on one node with up to 10,750 actor rows, on the first guest | 1.1, 4.5, 8.1, 8.4 |
| X, a bucket and a Postgres on another machine | bucket cycles with TLS, with a rate limit, and with added delay, two bucket outages with actors that hold data, an install with Postgres on a second guest, and two outages of that database, on the second guest (16 vCPU, 23 GB) | 4.4, 8.1, 8.2, 8.3 |
| W, bounds and open commands | two settings that bound a kagent turn, and Agent Executor again: a worker loss with a command open, the skills and MCP servers of a workspace, the example with its own prompt and repository, and second runs of the clone and of the agent image | 7.1, 7.5 |
| Y, second runs, a rollback, and v0.6.0 | three outages of the bundled Postgres and one each of ate-api-server and the Kubernetes API server, a second upgrade from v1.0.0-rc.3 to v1.0.0 and a rollback, and an install of v0.6.0 with a direct upgrade, a reinstall, and Agent Executor v0.3.1, on the first guest | 4.3, 7.5, 8.2, 8.4 |
| Z, bursts on one node | a pool grown to 300 workers in steps of 50, bursts of 150, 200, and 300 resumes, the restore time inside each worker, the CPU of each component, and 1 to 4 replicas of ate-api-server and of the router, on the first guest | 1.1, 4.5, 8.2, 8.3 |
| V, validation | twelve checks on the first guest: memory that holds no data, a Python runtime, resumes at the same instant, fidelity, templates, access policies, actor JWTs, a preview feature, a process exit, and a suspend during a request | Part 3, 4.4, 4.5, Part 5, 8.5 |
Rank 2: the code
| Source | Used in |
|---|---|
Agent Substrate, tag v1.0.0, commit b34fbee742906494b5eb4f1f36987176bf8770a4, released 2026-11-05, github.com/agent-substrate/substrate | every section |
The same repository at tag v0.1.0, for the comparison of old and new instructions | 8.4 |
kagent, tag v1.0.0, released 2026-11-09, github.com/kagent-dev/kagent | 7.1, 8.1 |
agentgateway, tag v1.7.0, github.com/agentgateway/agentgateway | 6.6, 7.2 |
The manual reads seven parts of the Substrate code most. They are the API definition, the lifecycle workflows of the API server, the snapshot tracker of atelet, and the request parking code of the router. The others are the database migration, the install manifests, and the command-line plugin. The manual source records the files that each section used, and the audit checks that record.
Rank 3: documentation at the tag
These pages of the Agent Substrate documentation are cited by their titles. We checked each statement that we repeat against the code, and against a run where we had one.
The project site, ate.dev, serves the same pages at version 1.0. We read all 48 pages of the site on 2026-11-12 and compared the 38 that have a file at the tag with that file. We found differences in rendering only, such as link text and placeholders, and no difference in content. The first page of the site is a source of its own, because it states results with no conditions. What Substrate does, and what it leaves to you sets each of those statements beside our runs.
| Page | Used in |
|---|---|
| the quickstart | Part 2, 8.4 |
| the architecture page | Part 1 |
| the API guide | Part 3, Part 4, 8.1 |
| the resource model and the glossary | Part 3, the glossary |
| the request parking page | 4.4 |
| the sandbox classes page | 4.2 |
| the guides for authentication, authorization, and actor JWTs | Part 5 |
| the guides for network egress, the trust bundle, and credential injection | Part 6 |
| the PostgreSQL guide and the node storage guide | 8.1 |
| the upgrade runbook and the version skew policy | 8.4 |
| the page on preview features | 8.5, the flags reference |
| the key rotation guide for actor JWTs | 8.5 |
| the installer command reference and the operator install guide | Part 2, the flags reference |
the kubectl ate reference | 2.2, the command reference |
Three statements in these pages do not match the release. The quickstart says that Substrate suspends an idle actor, and our run and the code show that it does not. The upgrade runbook gives an example version that does not exist. The help text of one installer flag says that a credential provider requires Envoy, and the provider worked with agentgateway.
Rank 4: issues
The issues are reports by users and maintainers. We name an issue where it explains a limit that we did not run, and we say so in the text.
| Issue | What it reports | Used in |
|---|---|---|
| #3101 | node-local snapshots filled a node disk, and the kubelet evicted the workers | 8.1 |
| #3775, #2972 | the design for detecting a dead process inside a living sandbox | 8.5 |
| #3441 | a proposal to reserve less than the declared limit for each actor | 8.5 |
| #3589 | ingress authorization is the same for every atespace | 8.5 |
| #3203 | a proposal for other storage backends, which the maintainers declined before the release | 8.5 |
| #3602 | a durable directory has no size limit | 8.5 |
| #3347 | the reuse of a name is not safe | 8.5 |
| #3285 | a change of egress policy does not close open connections, which our run confirmed | 6.6, 8.5 |
Other sections cite more issues, and each names them in its own text.
The project marks in the figures belong to their owners, and we use them only to identify the projects.
Agent Substrate 1.0.0 · b34fbee · 2026-11-05 · Edition 2026.11