Model keys

A pool of vendor keys the installation can answer with, and a screen an operator adds to, switches off and revives from. Before it, the product had one key, it lived in the api's environment, and replacing a burnt one was a deployment. Added in AGNT2-366; since AGNT2-367 a key can also say which model levels it serves.

Who sees it: the platform administrator. The screen is /model-keys in the admin panel.

An empty pool changes nothing

This adds a source. When nothing in the pool can serve a call, both vendors fall back to the environment key they always used — ANTHROPIC_API_KEY for completions, OPENAI_API_KEY for embeddings. So:

• an installation that adds no keys behaves exactly as it did;
• turning the pool on is adding a row on a screen, not a release;
• and the failure mode of the whole feature is «back to how it was».

The screen above is an installation with nothing in the pool — the state every installation starts in. The api answers with its environment key and the screen says so, above the table, because a table of eight resting keys and a table of no keys describe the same running product. (The shot predates the Levels column below.)

What a row says

column
Vendor	anthropic or openai
Name	whatever you called it — the workspace or the account it came from
Levels	which model levels this key serves; «all levels» when it names none
Ends in	the last four characters, which is what you match against the vendor's own console
State	one of the four below
Last used, Calls, Tokens	what has actually gone through it

One key is one LlmCredential row, and a plain read of that table shows exactly what the screen shows:

SELECT vendor, label, tiers, last4, "restingUntil", "attentionAt", "disabledAt", "requestCount"
FROM "LlmCredential";

The value is not in that list because it is not readable: it is stored as an AES-256-GCM envelope, sealed to the row's own id, so a blob copied into another row will not open. The cipher is the same EnvelopeCipher that keeps an agent's own secrets — moved down into setup/core by AGNT2-366 precisely so a second way to encrypt would not exist.

The value of a key is never shown. Not to a platform administrator either. There is no field on the response that could carry one, so the screen could not render it if it tried — and the real response bodies are read by a test that proves it, with a control showing the same bodies do contain the vendor and the name.

Which levels a key serves

The product runs on three model levels — simple, smart, genius — described on Model tiers and the prompt cache. A key may name the levels it serves, and the pool is asked for a key per vendor and level.

• Naming none means every level. That is what every key added before levels existed holds, so an installation that has never assigned a key to a level behaves exactly as it did — the same rule as the empty pool itself.
• One key serving several levels is the ordinary case: simple and smart usually sit on one key at one vendor.
• A level may be served by a different vendor's keys than the level below it. Today only Anthropic answers completions, so that road is open in the data and not yet walked.

The levels are the one thing about a key you change after adding it: tick them in the row and the change binds on the next model call, in every api replica, because they are read from the database per call. The environment key serves every level.

The four states

state	what it means	comes back
Working	in the pool; calls are being routed through it	—
Resting	the vendor answered 429 — the quota is spent	by itself, at the instant the vendor's retry-after named — or LLM_CREDENTIAL_REST_MS later (a day) when it named none
Needs attention	the vendor answered 401 / 403 — the key itself is refused: revoked, mistyped, from the wrong account	never on its own. Only a person
Switched off	somebody took it out of service. Its history is kept	when somebody puts it back

The middle two are drawn differently on purpose. A resting key needs nobody; a refused key needs you. Colouring both amber would put your eye on the row that needs nothing. Merging them is the defect worth naming: a revoked key that «heals» after a day breaks every answer for as long as it exists, and does it quietly.

And a third case marks nothing at all. A timeout, a reset socket, a 5xx or a bad request of our own making is not the key's fault. Marking on those would empty the pool for a day because the provider had a bad minute.

A rest is as long as the vendor said, not longer (AGNT2-381). The configured day used to be a floor as well as a default, so a burst limit Anthropic clears in twenty seconds put a key to sleep for twenty-four hours — and on an installation with one key that is the whole model asleep for a day. Now the vendor's word wins whenever it is there; the day is the guess for when it said nothing.

LLM_CREDENTIAL_MARKING=off switches the marking off altogether. It exists as the control for the measurement and for nothing else: an installation left on it walks into the same exhausted key on every call, for ever, and says so in the log each time.

The three things you can do

the button
Add a key	vendor, a name, the key, and optionally the levels it serves. The value is stored encrypted and never comes back out
Switch off / Switch on	take it out of service, or put it back
Put back in service	end a rest early, or clear a refusal you have dealt with

There is no delete, and that is deliberate. Switching a key off keeps the row, because the row is what past spend is attached to — and a deleted one makes «what did this key cost us» unanswerable for exactly the key somebody removed because it was expensive.

Behind the screen, six routes, all under the admin/ prefix — the guard reads the address rather than a decorator somebody has to remember to add:

GET   /admin/llm-credentials              every key: vendor, name, levels, last four, state, traffic
POST  /admin/llm-credentials              add one
PATCH /admin/llm-credentials/:id/tiers    change which levels it serves; an empty list means all
POST  /admin/llm-credentials/:id/disable  take it out of service — this is what "delete" means here
POST  /admin/llm-credentials/:id/enable   put it back
POST  /admin/llm-credentials/:id/revive   end a rest early, or clear a refusal a person has dealt with

Which key the next call takes

Set by LLM_CREDENTIAL_ROTATION, and the default matters more than it looks:

sticky — per conversation (the default)	a conversation stays on one key, chosen by rendezvous hashing over the usable set. Two api replicas reach the same answer with nothing shared, and a key going to rest moves only the conversations that were on it
request — per request	the least-recently-used key, every call, queued by lastUsedAt in the database

The default is per-conversation because the provider's prompt cache belongs to the workspace the key belongs to. Eight keys from eight workspaces are eight separate caches, and changing key inside a conversation throws the cached prefix away and pays to write it again. Measured on the real api: rotating per request costs 3.4× per turn against a warm cache; per-conversation rotation costs the same as a single key and still spreads the load.

What that means in practice is on Model tiers and the prompt cache.

The 3.4× is from AGNT2-366's report, which lives on that ticket and not in this repository; the investigation that led to the pool is specs/AGNT2-369-openrouter-or-own-keys/research.md.

Exhaustion has to arrive quickly, or the carousel buys nothing

A pool that rotates away from a spent key is worthless if «spent» arrives as an hour of waiting rather than as an error — the person has already gone. So the vendor SDK's own uninterruptible retry is off: it honours the provider's «wait this long» verbatim, and on an exhausted quota that instruction names a time hours away, with no way to interrupt it. The api has its own bounded retry instead (AGNT2-377): a pause between attempts is capped at ANTHROPIC_RETRY_WAIT_MAX_MS, and the whole blocking call at ANTHROPIC_CALL_BUDGET_MS. The four numbers are on Model tiers and the prompt cache.

A provider asking for a long wait is not asking us to retry. It is telling us this key is done for now — which is a fact to act on, not to sleep through.

Two replicas may pick the same key

Nothing prevents it, and nothing tries to. Everything about a key's standing lives in the database — the marks, the counters, the queue, the levels — because at N api replicas a per-process anything is N independent copies of a fact that has to be one. But the purpose is to spread load, not to ration it, and a strict queue would need a lock on the hot path of every model call to buy a fairness nobody asked for.

Check it: add a second key

1. Sign in to the admin panel as a platform administrator → Model keys.
2. Add a key — a second anthropic one, from a different workspace, with no level ticked. The row appears as Working, Levels reads «all levels», Ends in matches the last four characters in the vendor's console, Calls at 0.
3. Have two different conversations with an agent, several turns each.
4. Re-open the screen. Calls has moved on both rows, and each conversation stayed on one key — that is the default rotation doing its job.
5. Tick only genius on the new key. From the next call on, ordinary turns leave it alone and its Calls stops moving until a turn is assessed as genius.
6. Switch off one key. Its state becomes Switched off and its counters are still there.
7. Switch on again; new traffic returns to it.

The one thing you cannot do at any point in that sequence is read a key back. That is the property, not an omission.

Where the code is

the pool, the states, the levels, the rotation	api/src/slices/system/llmCredential
the six routes and their DTOs	api/src/slices/admin/llmCredential
the screen	admin/slices/platform/llmCredential

See also Secrets and Platform role.