Runtime model

This page is the flow: how a message becomes work, one turn end to end. The plain-language version is The life of one request; the wire-level protocol lives in the Worker pages. Here — the sequence, the modes, and what survives the turn.

The turn at a glance

message
  → orchestrator   persist message · load agent · rehydrate context
  → enforcement    quota / limit gate — before any spend
  → LLM loop       one merged MCP toolset:
       ├─ no tool calls   → stream the answer → persist → done  (no pod — most turns)
       ├─ brain tool      → runs in-process   (memory · secret · cron · knowledge …)
       └─ worker tool     → ensure a worker session (once) → tools/call over the tool channel
  → wrap-up        persist assistant message + summary · emit usage · let go

Step by step

1. Intake. orchestrator persists the user message (agent/chat), loads the agent config (agent) and rehydrates the working context from Postgres — tail window + running summary (the context engine, see Short memory).
2. Enforcement. setting/enforcement checks the team's quota/limits before any work.
3. The loop. llm.stream() runs in the brain with one merged toolset: brain-MCP (in-process tools), worker-MCP (attached only while a worker session is live), and connector-MCP (the store's APIs). The worker has no LLM — the loop never leaves the brain.
4. No tools needed → stream the answer, persist it, done. No pod is created; this is the cheap path and most turns take it.
5. Hands needed — the first worker tool call triggers provisioning:
   • reuse a live AgentRuntimeSession for (agent, chat) if one is running or idle — the queue is hit at most once per session, never per tool call;
   • else enqueue a provision task (BullMQ); the runtime manager (runtime/worker) picks a RuntimeProfile, builds the k8s Job, injects short-lived creds;
   • the worker boots, dials back over WebSocket, MCP initialize + tools/list — its tools join the session toolset (Deploy);
   • from here the brain drives tools/call over the tool channel; progress and logs stream to the frontend via Redis pub/sub (runtime/event).
6. Wrap-up. Outputs persist via system/file; the assistant message and updated summary are saved; usage BillingEvents are emitted; the brain lets go. The worker session survives its idle window — a follow-up turn reuses it — until the reaper tears it down (Destroy).

What ends a turn

The socket closing cancels the turn. Closing the tab, a reload, leaving the app, losing the network, and the chat's own stop button (which aborts the browser's fetch, and so closes the socket) all end the same way: the abort travels down through the loop, into the tool that is executing and into the model call itself, so the turn stops in milliseconds instead of burning tokens for a reader who has gone.

SPA navigation does not cancel it, and that is deliberate. Switching a card's tab, opening another page, unmounting the chat component — the browser keeps the in-flight request alive, the turn runs to completion and, for an agent, persists. The user who switched screens has not left; the answer is still theirs, and killing the turn would throw away work they are about to read. "Cancel on unmount" is not the missing half of cancellation, it is a regression waiting to be written.

A stopped agent turn keeps the text already delivered and is marked interrupted, so the thread after a reload says the same thing the screen did. A turn that fails persists nothing.

One turn per agent

An agent runs one turn at a time, guarded by a Redis hold:

• the hold expires after 5 minutes, and is pushed back out to a full 5 minutes at roughly a third of that while the turn is alive, up to a ceiling of 60 minutes — so a long turn keeps its hold, and a lost one cannot hold the agent forever;
• the hold is best-effort: an unreachable Redis must never delay or fail a turn;
• autostart reads it — a busy agent is skipped, never interrupted.

Ceilings on one turn

• 8 iterations of the tool loop. Hitting the ceiling keeps the partial output rather than discarding it.
• 3 data-changing tool calls per turn, in any turn — an agent's as much as the concierge's. Every tool that writes spends from a per-turn budget: creating and archiving an agent, rewriting an autostart prompt, changing a cadence, writing a note to memory. The fourth attempt is refused with a sentence the model can read rather than with silence. Without it, one unlucky phrasing could make eight changes in a single reply, because the loop counts round-trips and has no idea which tools write anything. One call is one change even when it moves three fields at once — the budget counts changes to the workspace, not fields.
• An upstream 429 from the model provider surfaces as 429, and a provider outage as 503 — never an opaque 500, so a client can tell "retry shortly" from "we are broken".

Runtime modes → k8s primitives

The orchestrator/mode sub-slice picks how much machine a turn gets:

The mode comes from the agent's own runtimeProfile column, and none is the default — an agent nobody has given hands to never reaches this table at all. Within a mode, nothing is raised until the model actually calls a worker tool; see Hands.

Mode	What it is for	Primitive	Idle
None	no hands — the default	no pod — LLM + memory only	n/a
Light	bash, scripts, file ops	k8s Job, restartPolicy: Never, ttlSecondsAfterFinished	60–120s
Browser	browser tool used	worker Job with in-pod headless Chromium (Playwright, per-tenant --user-data-dir)	until browser idle
Heavy	scraping, big compute	k8s Job on the workers nodepool, high limits	30–60s
Warm	premium	Light/Heavy kept alive past idle	10–30 min

When a session fails, the record says which way

A worker session can fail six different ways, and each one points at a different thing to go and look at, so the row carries a reason and not just the word failed:

reason	what to suspect
dial_timeout	provisioning — the pod never called back
handshake_timeout	the image, or the pass it was given
ready_timeout	the pod is up but never finished starting
session_mismatch	the caller is not the session it claims to be
channel_version	a rollout — the two sides speak different versions
heartbeat_lost	a zombie: it was alive and stopped answering

Version disagreement is on that list deliberately. The two sides ship separately, so they will not always agree — and a handshake that cannot say "I speak a different version" fails by hanging instead of refusing, which is the failure that takes hours to diagnose rather than seconds.

What survives the turn

The brain and the worker vanish; these don't:

State	Lives in
messages + the running summary	agent/chat (Postgres) — see Short memory
durable & long-term memory	agent/memory — see Memory
Task · AgentRuntimeSession rows	runtime (Postgres) — see DB schema
files & artifacts	system/file → object storage
spend	usage BillingEvents → billing

Queue & scaling

• BullMQ on Redis — provisioning + scheduled tasks; delayed jobs double as idle timers; pub/sub carries the event stream.
• Native k8s Jobs — the worker image bundles Playwright + Chromium (browser runs in-pod, no separate pool); ttlSecondsAfterFinished gives scale-to-zero. No KEDA/Knative for MVP.
• Short-lived session tokens — Core mints a JWT scoped to one AgentRuntimeSession (user/auth); no long-lived secrets ever enter the pod (security).

See also

• Worker — Deploy / Use / Destroy — the session lifecycle and the MCP tool channel in wire-level detail.
• Short memory — how the context engine assembles the prompt each turn.
• DB schema — Task and AgentRuntimeSession shapes.
• How it works — the same story in plain language.