5.2

Access policies per atespace

An access policy grants one of three roles on one atespace, and the atespace is the smallest unit that Substrate can protect.

Two teams share a cluster. Team A must not suspend, read, or copy the actors of team B. You look for a way to grant access to one actor or one template, and the API has none. The unit of access is the atespace, and that fact shapes how you divide your work.

When you finish this section, you can write an access policy, say what each role allows, and name the things that a policy cannot separate.

Most of our runs used the default owner. For this section we used one ServiceAccount, alice, and the atespace lab-v. We gave it no role, then the viewer role, then the editor role, and we recorded what the API allowed. The denial texts and the policy listings below come from that run. The rules that we did not run are marked as code or documentation.

How a call is checked

ate-api-server checks every call in two steps. Authentication turns the credential into a principal, as client authentication describes. Authorization then allows the call in one of three cases:

  • the principal is a bootstrap owner
  • an access policy gives the principal a role that permits the call
  • the caller is a Substrate component with the system role that the call needs

In every other case the call fails with PermissionDenied. Authorization cannot be turned off. The model runs inside ate-api-server on an embedded OpenFGA engine, and it stores its data in the same Postgres database as the actors.

Access policy: the list of role bindings for one scope. Each scope has at most one policy, and its name is default.

Fig. 5.2two scopes of access policy, and the roles of eachtree
Two scopes of access policy, and the roles of each A tree. At the top, the bootstrap owners from the flag --authz-bootstrap-owners always hold owner on the global scope. Below, the global access policy named default grants owner and viewer. Under it, two atespaces named team-a and team-b each have one access policy named default that grants owner, editor and viewer. Workers belong to the global scope. System roles for atenet-router, atenet-egress, atelet, ate-controller and the ate-api-server pod come from pod certificates and no policy can grant them. --authz-bootstrap-owners default: the shared ate-client global access policy name: default roles: owner, viewer always owner atespace team-a access policy default owner, editor, viewer atespace team-b access policy default owner, editor, viewer a global role holds in every atespace workers global scope only SYSTEM ROLES atenet-router resumes actors atenet-egress mints actor JWTs atelet reads egress policy ate-controller manages workers ate-api-server sidecars read actors from pod certificates no policy grants them
A role on the global scope holds in every atespace, and a role on one atespace holds only there. Read from the top down. Blue boxes are the scopes that a policy can cover. The dashed box holds the system roles, which come from pod certificates. The atespace names are examples.

Two scopes, five roles

A policy covers the global scope or one atespace. The global scope has two roles and an atespace has three. Roles inherit: an owner is also an editor, and an editor is also a viewer.

RoleCan doCannot do
global ownerall of the rows below, create atespaces, manage workersmint an actor JWT
global viewerread every atespace, list workerschange anything
atespace ownerall that an editor can, write the policy of the atespace, delete the atespacetouch another atespace
atespace editorcreate, resume, suspend, revert and delete actors, write templates, egress policies and tagschange the policy
atespace viewerread actors, templates, tags, egress policies and the policychange anything

Workers have no atespace. Only a global role can list them, so a team with an atespace role cannot run kubectl ate get workers.

Bootstrap owners

Bootstrap owner: a principal from the server flag --authz-bootstrap-owners that always holds the global owner role, whatever the policies say.

The server reads the list at startup and does not start with an empty list. With ate-setup, the default is the shared ServiceAccount ate-client, and the installer prints a warning about it. The kagent distribution of Substrate adds its own controller to the list. We read this argument from the ate-api-server Deployment of our kagent cluster:

yamlthe bootstrap owners in a kagent installFull log
        - --authz-bootstrap-owners=spiffe://cluster.local/ns/kagent/sa/kagent-controller,system:serviceaccount:ate-system:ate-client
One argument of the ate-api-server container.

Two principals own that install. The first is the pod certificate of the kagent controller. The second is still ate-client, so the warning from client authentication applies to a kagent install as well.

For a shared cluster, set the list to the principals of your operators and keep it short. Grant daily administrators the global owner role in the global policy. You can remove a role in a policy through the API. You can remove a bootstrap owner only with a restart of the server.

Write a policy

A policy is a short manifest. This is the file that gave the ServiceAccount alice the viewer role on lab-v.

yamlthe policy file that we createdFull log
metadata:
  name: default
bindings:
- role: viewer
  members:
  - user:system:serviceaccount:ate-users:alice
The complete file.

A member is the principal with the prefix user:. Three commands create, read, and change a policy. They follow the pattern of the other kubectl ate objects:

bash
kubectl ate create access-policy -a lab-v -f policy-viewer.yaml
kubectl ate get access-policy -a lab-v -o yaml > current.yaml
kubectl ate update access-policy -a lab-v -f current.yaml

The fourth command removes every binding of the scope. Run it only when you mean that:

bash
kubectl ate delete access-policy -a lab-v

For the editor role we changed one word in the output of get -o yaml and sent that file with update. This is the stored policy after the update.

yamlthe stored policy after the update to editorFull log
bindings:
- members:
  - user:system:serviceaccount:ate-users:alice
  role: editor
metadata:
  createTime: "2026-11-12T12:51:47.968679335Z"
  name: default
  uid: a06fc82d-e269-4ce3-8471-00c1b537adff
  updateTime: "2026-11-12T12:51:51.605795572Z"
  version: "2"
The complete file. The server added the uid, the times, and the version.

Use --global in place of -a for the global policy. Four rules save you a failed command:

  • A global policy accepts owner and viewer. An atespace policy also accepts editor.
  • update replaces the whole policy. It needs the metadata.uid and metadata.version that get -o yaml prints, and it fails if the policy changed after your get.
  • You cannot delete the global policy. Update it with no bindings.
  • A change applies to the next call.

Egress policies follow the same update rule. Our first update egress-policy failed until the manifest carried both fields.

What each role was refused

We ran the same ServiceAccount through four stages. Each denial has the gRPC code PermissionDenied and the same sentence, with the missing permission and the scope in it.

outputa viewer tries to create an actorFull log
…Error: failed to create actor: rpc error: code = PermissionDenied desc = permission denied: principal "user:system%3Aserviceaccount%3Aate-users%3Aalice" lacks "can_create_actor" on "atespace:lab-v" 
One line of the trace. The time and the step name at the start of the line are cut.
Role of alice on lab-vCallResult
nonelist actors in lab-vdenied: lacks "can_list_actors" on "atespace:lab-v"
nonelist atespacesdenied: lacks "can_list_atespaces" on "global:root"
viewerlist actors and templates in lab-vallowed
viewercreate an actor in lab-vdenied: lacks "can_create_actor" on "atespace:lab-v"
viewerlist actors in the atespace lab-cdenied: lacks "can_list_actors" on "atespace:lab-c"
viewerlist workersdenied: lacks "can_list_workers" on "global:root"
editorcreate, suspend, and delete an actor in lab-vallowed
editorcreate an actor in lab-cdenied: lacks "can_create_actor" on "atespace:lab-c"
editorcreate an access policy on lab-cdenied: lacks "can_create_access_policy" on "atespace:lab-c"
none, after the owner deleted the policylist actors in lab-vdenied: lacks "can_list_actors" on "atespace:lab-v"

We measured the delay of a role change on the x86 server guest (24 vCPU, 47 GB, nested KVM), once for each change. The first call 2 seconds after the create was allowed, and so was the first call 2 seconds after the update. The call 0.2 seconds after the delete was denied. So a change took effect within 2 seconds, and the delete within 0.2 seconds. Raw runs

Two details of the denial help you. The error names the principal with each colon written as %3A, so decode it before you paste it into a policy. And the scope global:root in a denial means that no atespace role can allow the call.

After the install, the global scope had no policy at all: kubectl ate get access-policy --global answered global has no access policy. The shared ate-client identity still owned everything, because it is a bootstrap owner.

What a policy cannot separate

The atespace is the only boundary. Five limits follow, and you must design around each of them.

  • No grant for one actor or one template. An editor of an atespace can change every actor in it.
  • No groups. A member is one principal. A team of ten needs ten members.
  • An editor can copy an actor. A tag is a copy of the memory and the disk of an actor, with every secret in it. Any editor can create a tag and start a new actor from it.
  • Names carry the grants. If you delete an atespace and create one with the same name, a late request for the old one can apply to the new one. Never use an atespace name two times. Atespaces and names explains the naming rules.
  • The router does not read policies. A client on the list of the router reaches actors in every atespace.

Our kagent run adds a practical case. kagent put every session of every agent into one atespace named kagent. In that install, Substrate policies cannot keep two kagent users apart. That job belongs to kagent.

System roles

The components of Substrate hold roles that no policy can grant. Each role is bound to the pod certificate of one component by a flag of ate-api-server.

RoleHeld byAllows
ingress_routeratenet-routerresume any actor, list actors in all atespaces
egress_gatewayatenet-egress, and any other identity that you name with the repeatable flag --egress-gateway-spiffe-idread actors and egress policies, mint actor JWTs
ateletatelet on every noderead the egress policy of any actor
controllerate-controllercreate, get, list, update, delete and drain workers
api_server_sidecarthe ate-api-server pod, which its sidecars shareread actors, so that the object store plugin can limit a storage credential to one actor

Two rows deserve a second look. The router can resume every actor, which is why the list of the router in client authentication matters so much. And only the egress gateway can mint an actor JWT. A global owner cannot, because the token lets its holder act as the actor. Actor JWTs follows that token.

Our position

Use one atespace for each group of people that must not read the others' actors, and accept that this is the only line you get. Replace the default bootstrap owner before a second person uses the cluster. Do not count on Substrate to separate the end users of your product. Your own service has to do that, in front of both the API and the router.

Agent Substrate 1.0.0 · b34fbee · 2026-11-05 · Edition 2026.11