Overview

"Storing the chats" is really three different data jobs with different access patterns, volume and retention. Conflating them into one table is the mistake — telemetry would bloat the transactional store, and search would fight writes. Keep them separate, each behind a slice so the physical backend stays swappable.

Job	What it is	Where	Slice
Storage	the chat of record (what the UI shows)	Postgres (shared → dedicated instance at scale)	agent/chat
Short memory	the working context window for this turn (window + rolling summary)	derived from Postgres; summary persisted on the chat	agent/chat + agent/orchestrator
Debug & tracing	prompts · tool calls · tokens · the full trace	Langfuse (separate; ClickHouse-backed)	instrumented from api
Memory	the notes the agent keeps across conversations, plus looking a conversation up	notes in Postgres; vectors in infra/vector	agent/memory

The principle

• One source of truth (Storage) — durable, relational, paginated for the UI.
• Telemetry is not truth (Debug) — append-only, high-volume, short retention, its own system.
• Memory is derived (Memory) — an async index built from the chats, not the chats themselves.

"Not in the api" — to be precise: the chat domain logic stays in the agent/chat slice (it's just a gateway); the data can live in a separate database. The gateway makes "in the api now, separate later" a config change, not a rewrite.

→ Then: Storage · Short memory · Debug & tracing · Memory · Implementation.