Use

How the brain (api + LLM) drives a live worker once it's up: the session model, the communication protocol, and which tools run where.

Lifecycle: Deploy → Use → Destroy.

A session ≠ a tool call

One user request → one live worker session → many tool calls over a persistent channel → one release. The queue is hit at most once (to provision), never per tool call.

Unit	Through the queue?	Transport
Task / session ("I need a worker for (agent, chat)")	yes — once (provision), or 0× if a warm session is reused	BullMQ
Tool call (exec, browser_play, …)	no	live WebSocket to the running worker

Why a living session, not a Job per tool

Required, not just an optimization: tools in a turn share state — the working directory (cd), a logged-in browser context, env, background processes. A fresh Job per tool call would lose all of that and pay pod spin-up every time. So the worker stays alive for the session and the brain drives many tool calls into it.

Who runs the loop: the brain

The orchestrator (in api) owns the LLM reasoning loop; the worker is a stateful tool-execution server with no LLM. The brain decides "call exec", sends it, gets the result, feeds it to the LLM, repeats. Security bonus: the loop stays in the brain, so LLM API keys never enter the worker pod — the worker holds only tool executors.

The protocol — two planes, three channels

The wire contract is written once, elsewhere

This section is the map — two planes, three channels, what rides where. The wire contract for channel 3 (the initialize / ready / heartbeat / release frames, what answers each, what happens on silence, and what the two sides do when their versions differ) is Tool channel — written once so the gateway and the worker cannot drift.

Two planes — Mind (api + LLM, persistent) and Hands (worker, on-demand) — and exactly three channels. Don't conflate them.

#	Channel	Between	Transport	Carries	User sees?
1	Chat stream	User ↔ Mind	SSE/WebSocket (streaming endpoint, agentId)	answer tokens, "thinking" status, activity cards, chosen artifacts	Yes
2	Provision	Mind → queue → dispatcher	BullMQ on Redis, once	"give me a worker" { agentId, sessionId, caps } + short-lived token	No
3	Tool channel — MCP	Mind (MCP host) ↔ Worker (MCP server)	MCP over a worker-initiated WebSocket, persistent	initialize · tools/list · tools/call · progress · resource_link	No

The mind↔worker conversation rides channel 3 — a separate internal channel, never the user chat. Only curated status and summaries cross from 3 into 1.

The worker is an ephemeral MCP server. Channel 3 is plain MCP (Model Context Protocol — the same JSON-RPC tool standard we use for store connectors). The brain (api) is the MCP host: it aggregates several MCP servers and exposes their tools to the LLM as one toolset —

• brain-MCP (in-process): memory · secret · cron · channel …
• worker-MCP (the pod): exec · fs · browser … — attached on ready, detached on release
• connector-MCP (external): the store's API (OpenCart), integrations

A tool's locus is just which MCP server hosts it. The worker has no LLM; MCP sampling (server-initiated model calls) is disabled — the reasoning loop stays in the brain.

The handshake (after the pod boots)

LLM emits a tool_call (e.g. browser.*)
   │  channel 2 — once
   ▼  Mind enqueues provision { agentId, sessionId, caps } + mints WORKER_TOKEN (sub = sessionId)
dispatcher provisions the worker pod (env: SESSION_ID · CONTROL_URL · WORKER_TOKEN_FILE · allowlist · storage scope;
                                     the pass itself is a projected Secret mounted at that path, never an env value)
   │  channel 3 — worker-initiated
   ▼  worker boots → dials CONTROL_URL over WebSocket → MCP `initialize` (presents WORKER_TOKEN)
api worker-gateway (MCP host) validates the token, matches sessionId, runs `tools/list`,
       registers the worker's MCP server into the session toolset → "ready"
   ▼
the LLM now sees the worker's tools; the host routes `tools/call` to THIS worker; results stream back

Worker-initiated dial-back means the api never needs the pod's IP — it survives reschedules and needs no inbound routing. Heartbeats (ws ping/pong) + idle-timeout → an orphaned worker self-terminates.

Transport — decided (option A): MCP runs over the reverse WebSocket (the worker dials the host). The worker is the TCP initiator yet still the MCP server; the gateway is the MCP client. This keeps "the api never holds a pod IP, no inbound into the worker" — at the cost of a small custom MCP transport instead of the stock Streamable-HTTP one. The MCP session id ↔ our sessionId.

On the wire (MCP)

MCP is JSON-RPC 2.0, so what we sketched before maps straight onto it:

// host → worker — discover tools (once, on connect) — allowlist/profile-gated
{ "method": "tools/list" }
//  → { "tools": [ { "name": "browser.click", "inputSchema": { … } }, … ] }

// host → worker — call a tool
{ "method": "tools/call", "params": { "name": "browser.click", "arguments": { "selector": "#buy" } } }
//  → { "content": [ { "type": "text", "text": "ok" } ],
//      "structuredContent": { "url": "/cart" },
//      "_meta": { "screenshot": { "type": "resource_link", "uri": "store://art_88" } } }

// worker → host — progress during a long action
{ "method": "notifications/progress", "params": { "progressToken": "c12", "message": "navigated to /cart" } }

• Discovery: tools/list is the allowlist/profile made explicit; notifications/tools/list_changed if it changes mid-session.
• Sticky state: cwd, browser context, env and background processes persist across calls — the MCP server instance is stateful for the session's lifetime.
• Big payloads by reference: files / screenshots / dumps go to object storage via system/file and travel as an MCP resource_link, never inline.

Each tool result feeds back into the LLM, which decides the next tools/call. This loop runs autonomously for seconds to ~an hour over the open MCP session, with zero extra queue traffic.

What reaches the chat (and what doesn't)

The user chat (channel 1) is curated by the LLM, not a raw feed of channel 3:

In the chat (channel 1)	Internal only (channel 3 / control plane)
The agent's prose answer	Raw MCP tools/call traffic
A "thinking / working…" state	Worker logs and progress notifications
Activity cards — "🌐 Browsing example.com…", "✅ found the price"	Intermediate screenshots / dumps
Artifacts the agent chooses to show	The full action trace (debug/trace view only)

The order is: the agent thinks → drives the hands over channel 3 → gets facts → decides what to say to the human over channel 1. The worker never writes to the chat; the LLM does, after summarizing.

The tool split — what runs where

A tool runs in the worker if and only if it needs the worker: a mutable filesystem, process spawning, a browser, or arbitrary/untrusted network egress (SSRF isolation). Everything else stays in the brain — LLM calls, agent-domain state, orchestration, presentation.

In MCP terms: worker tools are served by the worker-MCP, brain tools by the brain-MCP — the LLM sees one merged toolset, the host routes each call to the right server.

Worker tools (worker)

Tool	Needs	What it does
exec / process_exec	processes	run / manage shell commands and background processes
file / unzip	working FS	read/write the pod's working dir (≠ system/file); extract archives
browser / browser_screenshot / browser_play	browser	in-pod headless Chromium automation
http / web_fetch	egress (SSRF)	fetch an arbitrary URL → response / clean text
pdf_analyze	binary + FS	extract text via pdftotext over a file on disk

Brain tools (not in the worker)

web_search, image_analyze, tts (LLM/external) · memory_* (agent/memory) · secret_* (agent/secret — brain only; KEK never ships to a pod) · cron_* (agent/cron) · channel_* · access · skill_write · spawn_agent (orchestrator sub-task) · render_form / telegram_send (presentation) · resource_status.

Borderline calls (decided)

1. http / web_fetch → worker — arbitrary URLs are an SSRF risk; the brain (DB + KEK) must not fetch them. The worker does, behind the egress allowlist.
2. pdf_analyze → worker, image_analyze → brain — PDF needs a local binary + file on disk; image analysis is a vision LLM call.
3. spawn_agent → brain/orchestrator — a sub-agent is another ephemeral api run (a sub-task), not a worker subprocess.
4. file tool ≠ system/file service — the tool touches the ephemeral workspace; the service persists artifacts. On task end, the workspace syncs into system/file.

Security notes

• No secrets in the worker. Secret CRUD is brain-only; the dispatcher JIT-injects just the secrets a task needs as a short-lived projected Secret — the KEK never leaves api.
• Egress allowlist — in the tool today, not yet in the network. http and web_fetch check a destination list before every hop, not once at the start: a list consulted only on the address the model wrote is not a list, it is a formality a 302 steps over. Entries are hosts (example.com, *.example.com) — not URLs, not ports, and * is not accepted, because "allow everything" one character away from a legitimate entry is how it would happen by accident. An entry that cannot be parsed is dropped rather than widened, and named in the boot log. To mean "anywhere", there is one exact word: internet:unrestricted. It admits any host, it is understood identically by the api and by the worker, and it is a word rather than a symbol for the reason * is refused — nobody types twenty-one characters with a colon in them by accident. It counts on both sides: an agent asking for it under a ceiling of named hosts is refused, loudly, naming both; ordinary agents with no destinations now request public internet (AGNT2-551). The list has two sources and the narrower one wins. The deployment's ceiling (RUNTIME_WORKER_EGRESS_ALLOWLIST) is everywhere a worker of this installation may ever go and no tenant can move it; the agent's own request (config.egress) is what its sessions ask for. Every entry of the request must fall inside the ceiling or the session is refused before it exists, naming both. An absent installation ceiling defaults to public internet (AGNT2-479); an explicit empty ceiling still means nowhere. An absent ordinary agent request means public websites. Organization policy and private/metadata address checks continue to apply. Neither source alone would do: the ceiling alone is one list for everybody forever, and the agent's alone puts the decision in the hands of whoever can edit an agent, which here is the tenant. The list rides down in the pod's manifest, per session, set by the provisioner — so nothing on the tool channel can widen it, and it is there before the first call rather than fetched after boot. The key is written present and empty rather than left out, so a manifest never again says "closed" by saying nothing. This check does not restrain the pod. It lives in the tool, which is code running inside the worker, in the same process that executes the model's commands: exec can open a socket the list never sees (the image ships busybox wget, not curl), and a host that is on the list but resolves to a link-local address is admitted. The second level — a default-deny egress NetworkPolicy — exists as of AGNT2-204, as k8s/runtime/worker-egress.template.yaml. It does not carry the list above and cannot: a NetworkPolicy matches CIDRs and pod selectors rather than host names, so a worker under it reaches the cluster's resolver and the control channel home and nothing else. Bridging the two — a CNI that speaks FQDN policy, or an egress proxy — is still open, and until it is bridged a granted destination is admitted by the tool and refused by the node.
• Acting on a page is a SEPARATE grant (AGNT2-221). browser_play can type into a form and press a button, and a button is a purchase, a deletion, an accepted set of terms. Opening a page and waiting on it are governed by the egress list above; typing and pressing are governed by a second list — BROWSER_PLAY_ALLOWLIST in the manifest, config.browserPlay on the agent — and nothing an owner writes in the egress list puts a single host into it. Being allowed to read a site is not permission to press things on it: an owner who wrote a shop into config.egress said "you may look at the shop", and reading that as "you may buy from it" would silently upgrade every grant anybody has ever written. Its grammar is narrower on purpose. Exact hostnames only: *.example.com is refused, because a wildcard is a statement about a family of sites most of which the person writing it has never seen; and internet:unrestricted is refused in every position, because reading the web has an "everywhere" and acting on it deliberately does not. Every entry must also fall inside that agent's own egress grant — you cannot act where you may not go. Unset means nowhere, and that is what every agent row says today. The check is made again before every acting step, against the address the page has actually reached, so following a link to a site you may only read leaves you able to read it and nothing more.
• Ephemeral workspace. Worker tools touch only the pod's emptyDir; it's wiped on stop, outputs copied to system/file before Job deletion.

See also

• Deploy — how the worker is provisioned and boots.
• Tool channel — the wire contract behind this section, written once.
• Destroy — release, idle reaper, TTL, cleanup.
• Implementation — the build prompt for this subsystem.
• Runtime model — the per-turn flow end to end.
• Agent secrets — why secret tools stay in the brain.
• RLM — the pattern for working over data bigger than the context window: the brain drives the loop, the worker is the environment.

Public web default (AGNT2-551)

Ordinary agents with absent or empty config.egress use internet:unrestricted at read time. This covers forms, concierge creation, package imports and existing agents without rewriting their configuration. Every nonempty host list is preserved: there is no reliable way to distinguish an old suggested list from an owner's deliberate restriction. Malformed values remain closed. In Worker settings, replace the default with named hosts to narrow access; clear the list to restore public websites.

Draft interviews do not ask for domains or derive restrictions from task URLs. Browser action consent is still a separate, explicit list. Partner provisioning inherits the current approved organization's worker egress when the template has no destinations. Organization deny-all remains deny-all; existing provisioned empty grants inherit the current policy at worker-state and provisioning reads. The billing-owned lookup is registered through an agent-layer port, so the agent slice never imports billing. Explicit agent hosts are intersected with the organization grant. Queued provisioning reads policy again.

The deployment ceiling defaults to public internet when absent; an explicitly empty value still denies everything. Private addresses, cloud metadata and MCP address checks remain blocked. Named deployment ceilings must admit the requested agent grant or session provisioning refuses it. Deploy the coordinated GitOps ceiling change before enabling these API defaults. Neither PR is auto-merged.

AGNT2-551 also closes the public-network gap for unrestricted sessions: the Job gets agentfy2.dev/public-internet=true only after its effective grant and the deployment ceiling permit unrestricted access. The matching NetworkPolicy adds public IPv4/IPv6 ranges, excluding every special range in the worker address floor. Restricted sessions never get this label. Their existing network policy still needs an FQDN-aware CNI/proxy to permit named external sites; this change does not replace those deployment-specific rules. Grants apply to newly provisioned sessions; allow an existing worker to stop before checking a changed grant. The GitOps counterpart includes DNS and API control access as well.

Enter a password without showing it to the model

A trusted caller creates a short-lived fill reference with POST /v1/agents/:agentId/fill-secrets:

{ "value": "synthetic-example-only", "hosts": ["login.example.com"], "ttlSeconds": 600, "uses": 1, "label": "Team login" }

The response is { "ref": "fill_<32 hex characters>", "expiresAt": "<ISO date>" }. Give the model only this reference, never the value. It can use:

{ "url": "https://login.example.com", "steps": [
  { "action": "type_secret", "selector": "input[type=password]", "secretRef": "fill_<32 hex characters>" },
  { "action": "click", "selector": "button[type=submit]" }
] }

type_secret forbids text; only input password/text/email or textarea fields are accepted. The current page must match the secret's host or a subdomain, and must independently pass the worker's egress and browser-play consent rules. A redirect to a different origin during redemption refuses insertion. Refusals name expired, used, host_mismatch or unknown; operational/field failures are also explicit. A consumed use is spent even if the page then changes. Revoke with DELETE /v1/agents/:agentId/fill-secrets/:ref (idempotent).

The model sees the reference and masked step/page results. The API encrypts the value in Redis with TTL; only the isolated browser broker receives plaintext at entry time. Shell commands cannot read its pass, profile, memory or CDP pipes. The broker's socket accepts browser tools only. It serializes calls and masks known plaintext and URL/base64 forms in returned strings. Screenshots use a separate fresh profile, not the filled tab. Approved sites must be trusted with the password: a malicious recipient can deliberately disclose what it receives.

Runtime contract revision 2 requires the updated browser image and Kubernetes native sidecars (1.29+ with SidecarContainers enabled, stable in 1.33). The browser broker uses the browser image's normal resource budget; exec adds a request of 100m CPU and 128Mi memory. Its browser storage is memory-backed. Both containers are nonroot, read-only, without Linux capabilities; only the IPC volume is shared. Legacy workers refuse type_secret; they do not silently type a raw value. The API route uses the existing worker/skills authentication level. See secret slice for bounds and storage.

Large MCP catalogues are separate: a direct call to a tool hidden behind load_mcp_tools tells the model to load its instructions and retry. It does not ask for worker permission or claim an administrator blocked that MCP tool.

Continuing a browser play conversation (AGNT2-615)

browser_play retains a private tab per conversation while its worker lives. Start with {"steps":[{"action":"navigate","url":"https://fixture.example/login"}]}, then call {"steps":[{"action":"click","selector":"#open-form"}]} to continue. fresh: true resets your tab. tabReset: true also signals first use, eviction or restart; navigate before acting on an empty tab. The shared worker limit is three tabs; cookies remain shared. The API supplies conversation identity, which is never a tool argument.

click/type wait up to 3s for an actionable element (timeoutMs up to 10000). Every attempted step reports its number and outcome; the result includes the current URL/title and up to 2 KiB of visible text with field values omitted. Use type_secret with the existing secretRef contract for secret entry. A screenshot step photographs that same signed-in tab with fields masked; the API returns a team-only /agents/<id>/worker/image/<uuid>.png link retained for seven days. A separate browser_screenshot starts with a fresh profile.

Human browser observation

The service-only browser view shows the current signed-in tab without an agent tool call, stored screenshot or idle-clock update. Public API and SDK usage: worker browser view.