Actor and its seven states

A request to your actor returns 503. kubectl get pods shows every pod as Running, and the worker pod log does not name your actor. Kubernetes cannot tell you more, because an actor is not a pod. Its state is a row in Postgres, and kubectl ate get actors reads that row.

When you finish this section, you can read the state of an actor and name the call that moves it. You can also predict what a request does in each state.

The Actor object

Actor: one instance of an ActorTemplate, with a name that is unique in its atespace. An actor is the unit that Substrate suspends and resumes. It can run on a different worker after each resume.

This is the counter actor of the quickstart while it runs.

actorTemplate:
  atespace: ate-demo-counter
  name: counter
metadata:
  atespace: ate-demo-counter
  createTime: "2026-11-12T10:13:40.987568669Z"
  name: my-counter-1
  uid: 6f80f612-7547-4f5b-b3ab-831595621806
  updateTime: "2026-11-12T10:30:18.103768100Z"
  version: "16"
status:
  assignedNode: kind-control-plane
  lastAssignedGeneration: 3
  snapshots:
  …
  state: ACTOR_STATE_RUNNING
  workerAssignment:
    nodeName: kind-control-plane
    …
    workerPod: counter-7cc7dcd596-fps97
    …
    workerPodUid: c572e96d-dd9b-4500-b0e3-880a98d9f61d
    workerPool: counter

workerAssignment and assignedNode answer different questions. The first says where the actor runs now. The second also says where its last snapshot is after a suspend, and section 3.4 shows how long that lasts.

The seven states

The enum ActorState has seven values at v1.0.0. The state diagram on the architecture page of the project draws four of them, so trust the enum.

Three of these states are stable. ACTOR_STATE_SUSPENDED lasts until a request or a resume call arrives. ACTOR_STATE_CRASHED lasts until a revert or a delete, and a request does not end it. ACTOR_STATE_RUNNING lasts until a caller suspends the actor or Substrate marks it crashed. Section 3.3 proves that Substrate does not suspend it. The other four states are steps of one operation.

ACTOR_STATE_RUNNING does not prove that the process is alive. Two actors whose process had exited read ACTOR_STATE_RUNNING for the 20 minutes of our watch. Section 3.5 shows that run.

Old tutorials name a state ACTOR_STATE_PAUSED and a command kubectl ate pause actor. Both are gone at v1.0.0.

The transitions we saw

We drove my-counter-1 through a suspend, a resume, a crash, and a revert in 13 seconds. A script read the state after each step.

2026-11-12T10:30:09.378Z	state	ACTOR_STATE_SUSPENDED	kind-control-plane	9	SNAPSHOT_OWNER_ACTOR:SNAPSHOT_STORAGE_STATE_COMPLETED:gen3
2026-11-12T10:30:10.310Z	state	ACTOR_STATE_RUNNING	kind-control-plane	11	SNAPSHOT_OWNER_ACTOR:SNAPSHOT_STORAGE_STATE_COMPLETED:gen3
…
2026-11-12T10:30:11.590Z	state-after-crash	ACTOR_STATE_CRASHED	kind-control-plane	12	SNAPSHOT_OWNER_ACTOR:SNAPSHOT_STORAGE_STATE_COMPLETED:gen3
…
2026-11-12T10:30:17.724Z	state	ACTOR_STATE_SUSPENDED	-	14	SNAPSHOT_OWNER_ACTOR:SNAPSHOT_STORAGE_STATE_COMPLETED:gen3
…
2026-11-12T10:30:18.333Z	state	ACTOR_STATE_RUNNING	kind-control-plane	16	SNAPSHOT_OWNER_ACTOR:SNAPSHOT_STORAGE_STATE_COMPLETED:gen3

Look at the version column. A resume moved it from 9 to 11, and the revert moved it from 12 to 14. Each operation writes the actor two times: one write for the step state, and one for the stable state. Our script read the state after each command returned, so it never saw the state between.

The log of ate-api-server does show those states. The listing in section 3.1 has resuming before running, suspending before suspended, and deleting before deleted.

We measured the step states in the kagent run, on a second x86 server guest (16 vCPU, 31 GB). There a harness resumes and suspends an actor for every turn. Over 98 turns, the actor spent a median of 0.38 seconds in ACTOR_STATE_RESUMING. Over 99 suspends, it spent a median of 0.14 seconds in ACTOR_STATE_SUSPENDING. Those actors keep files only, so the times are low. Raw runs

A step state can also last. In our dependency test, one actor was still in ACTOR_STATE_RESUMING five minutes after Postgres came back. Section 8.2 has that case.

Who moves an actor

Substrate itself makes one move: it marks an actor ACTOR_STATE_CRASHED when the sandbox, the worker, or a restore is lost. Section 3.5 lists every way that we reached that state.

A call in the wrong state does not wait. ResumeActor on a crashed actor returns FailedPrecondition at once, and a request gets the same text as a 503. SuspendActor on a suspended actor returns the actor with no change.

The router calls ResumeActor for every request, also when the actor runs. On a running actor the call succeeds, changes nothing, and returns resumed=false. That is why a plain HTTP request is enough to start a suspended actor. Section 1.2 follows one such request.

Our position: read the state before you debug the network. kubectl ate get actors -a <atespace> costs one call. It separates a crashed actor from a full pool and from a dead process, and each of those needs a different fix.