Access policies per atespace
An access policy grants one of three roles on one atespace, and the atespace is the smallest unit that Substrate can protect.
Two teams share a cluster. Team A must not suspend, read, or copy the actors of team B. You look for a way to grant access to one actor or one template, and the API has none. The unit of access is the atespace, and that fact shapes how you divide your work.
When you finish this section, you can write an access policy, say what each role allows, and name the things that a policy cannot separate.
Most of our runs used the default owner. For this section we used one ServiceAccount, alice, and the atespace lab-v. We gave it no role, then the viewer role, then the editor role, and we recorded what the API allowed. The denial texts and the policy listings below come from that run. The rules that we did not run are marked as code or documentation.
How a call is checked
ate-api-server checks every call in two steps. Authentication turns the credential into a principal, as client authentication describes. Authorization then allows the call in one of three cases:
- the principal is a bootstrap owner
- an access policy gives the principal a role that permits the call
- the caller is a Substrate component with the system role that the call needs
In every other case the call fails with PermissionDenied. Authorization cannot be turned off. The model runs inside ate-api-server on an embedded OpenFGA engine, and it stores its data in the same Postgres database as the actors.
Access policy: the list of role bindings for one scope. Each scope has at most one policy, and its name is default.
Two scopes, five roles
A policy covers the global scope or one atespace. The global scope has two roles and an atespace has three. Roles inherit: an owner is also an editor, and an editor is also a viewer.
| Role | Can do | Cannot do |
|---|---|---|
| global owner | all of the rows below, create atespaces, manage workers | mint an actor JWT |
| global viewer | read every atespace, list workers | change anything |
| atespace owner | all that an editor can, write the policy of the atespace, delete the atespace | touch another atespace |
| atespace editor | create, resume, suspend, revert and delete actors, write templates, egress policies and tags | change the policy |
| atespace viewer | read actors, templates, tags, egress policies and the policy | change anything |
Workers have no atespace. Only a global role can list them, so a team with an atespace role cannot run kubectl ate get workers.
Bootstrap owners
Bootstrap owner: a principal from the server flag --authz-bootstrap-owners that always holds the global owner role, whatever the policies say.
The server reads the list at startup and does not start with an empty list. With ate-setup, the default is the shared ServiceAccount ate-client, and the installer prints a warning about it. The kagent distribution of Substrate adds its own controller to the list. We read this argument from the ate-api-server Deployment of our kagent cluster:
- --authz-bootstrap-owners=spiffe://cluster.local/ns/kagent/sa/kagent-controller,system:serviceaccount:ate-system:ate-clientTwo principals own that install. The first is the pod certificate of the kagent controller. The second is still ate-client, so the warning from client authentication applies to a kagent install as well.
For a shared cluster, set the list to the principals of your operators and keep it short. Grant daily administrators the global owner role in the global policy. You can remove a role in a policy through the API. You can remove a bootstrap owner only with a restart of the server.
Write a policy
A policy is a short manifest. This is the file that gave the ServiceAccount alice the viewer role on lab-v.
metadata:
name: default
bindings:
- role: viewer
members:
- user:system:serviceaccount:ate-users:aliceA member is the principal with the prefix user:. Three commands create, read, and change a policy. They follow the pattern of the other kubectl ate objects:
kubectl ate create access-policy -a lab-v -f policy-viewer.yaml
kubectl ate get access-policy -a lab-v -o yaml > current.yaml
kubectl ate update access-policy -a lab-v -f current.yamlThe fourth command removes every binding of the scope. Run it only when you mean that:
kubectl ate delete access-policy -a lab-vFor the editor role we changed one word in the output of get -o yaml and sent that file with update. This is the stored policy after the update.
bindings:
- members:
- user:system:serviceaccount:ate-users:alice
role: editor
metadata:
createTime: "2026-11-12T12:51:47.968679335Z"
name: default
uid: a06fc82d-e269-4ce3-8471-00c1b537adff
updateTime: "2026-11-12T12:51:51.605795572Z"
version: "2"Use --global in place of -a for the global policy. Four rules save you a failed command:
- A global policy accepts
ownerandviewer. An atespace policy also acceptseditor. updatereplaces the whole policy. It needs themetadata.uidandmetadata.versionthatget -o yamlprints, and it fails if the policy changed after yourget.- You cannot delete the global policy. Update it with no bindings.
- A change applies to the next call.
Egress policies follow the same update rule. Our first update egress-policy failed until the manifest carried both fields.
What each role was refused
We ran the same ServiceAccount through four stages. Each denial has the gRPC code PermissionDenied and the same sentence, with the missing permission and the scope in it.
…Error: failed to create actor: rpc error: code = PermissionDenied desc = permission denied: principal "user:system%3Aserviceaccount%3Aate-users%3Aalice" lacks "can_create_actor" on "atespace:lab-v" Role of alice on lab-v | Call | Result |
|---|---|---|
| none | list actors in lab-v | denied: lacks "can_list_actors" on "atespace:lab-v" |
| none | list atespaces | denied: lacks "can_list_atespaces" on "global:root" |
| viewer | list actors and templates in lab-v | allowed |
| viewer | create an actor in lab-v | denied: lacks "can_create_actor" on "atespace:lab-v" |
| viewer | list actors in the atespace lab-c | denied: lacks "can_list_actors" on "atespace:lab-c" |
| viewer | list workers | denied: lacks "can_list_workers" on "global:root" |
| editor | create, suspend, and delete an actor in lab-v | allowed |
| editor | create an actor in lab-c | denied: lacks "can_create_actor" on "atespace:lab-c" |
| editor | create an access policy on lab-c | denied: lacks "can_create_access_policy" on "atespace:lab-c" |
| none, after the owner deleted the policy | list actors in lab-v | denied: lacks "can_list_actors" on "atespace:lab-v" |
We measured the delay of a role change on the x86 server guest (24 vCPU, 47 GB, nested KVM), once for each change. The first call 2 seconds after the create was allowed, and so was the first call 2 seconds after the update. The call 0.2 seconds after the delete was denied. So a change took effect within 2 seconds, and the delete within 0.2 seconds. Raw runs
Two details of the denial help you. The error names the principal with each colon written as %3A, so decode it before you paste it into a policy. And the scope global:root in a denial means that no atespace role can allow the call.
After the install, the global scope had no policy at all: kubectl ate get access-policy --global answered global has no access policy. The shared ate-client identity still owned everything, because it is a bootstrap owner.
What a policy cannot separate
The atespace is the only boundary. Five limits follow, and you must design around each of them.
- No grant for one actor or one template. An editor of an atespace can change every actor in it.
- No groups. A member is one principal. A team of ten needs ten members.
- An editor can copy an actor. A tag is a copy of the memory and the disk of an actor, with every secret in it. Any editor can create a tag and start a new actor from it.
- Names carry the grants. If you delete an atespace and create one with the same name, a late request for the old one can apply to the new one. Never use an atespace name two times. Atespaces and names explains the naming rules.
- The router does not read policies. A client on the list of the router reaches actors in every atespace.
Our kagent run adds a practical case. kagent put every session of every agent into one atespace named kagent. In that install, Substrate policies cannot keep two kagent users apart. That job belongs to kagent.
System roles
The components of Substrate hold roles that no policy can grant. Each role is bound to the pod certificate of one component by a flag of ate-api-server.
| Role | Held by | Allows |
|---|---|---|
ingress_router | atenet-router | resume any actor, list actors in all atespaces |
egress_gateway | atenet-egress, and any other identity that you name with the repeatable flag --egress-gateway-spiffe-id | read actors and egress policies, mint actor JWTs |
atelet | atelet on every node | read the egress policy of any actor |
controller | ate-controller | create, get, list, update, delete and drain workers |
api_server_sidecar | the ate-api-server pod, which its sidecars share | read actors, so that the object store plugin can limit a storage credential to one actor |
Two rows deserve a second look. The router can resume every actor, which is why the list of the router in client authentication matters so much. And only the egress gateway can mint an actor JWT. A global owner cannot, because the token lets its holder act as the actor. Actor JWTs follows that token.
Our position
Use one atespace for each group of people that must not read the others' actors, and accept that this is the only line you get. Replace the default bootstrap owner before a second person uses the cluster. Do not count on Substrate to separate the end users of your product. Your own service has to do that, in front of both the API and the router.
Agent Substrate 1.0.0 · b34fbee · 2026-11-05 · Edition 2026.11