Architecture

Binaries

The listener does not change broker state. It exists so the usage fanout has a reader; a queue nobody consumes fills until it dead-letters.

Layout

services/broker/
  domains/usage/events/       ← public event payload
  internal/
    domains/                  ← usage, budget, routing
    people/                   ← upsert on first sight
    credential/               ← a person's client id, secret hash, and token signing
    pricing/                  ← cost and savings from a price row
    relay/                    ← Anthropic and OpenAI-compatible adapters
    handlers/                 ← per-operation types and bodies
    authz/                    ← Cedar
    command/                  ← idempotency + transaction
    publisher/                ← drain the usage outbox
    testdb/
  messageBus/
  migrations/
  db/                         ← sqlc generated queries

The platform mechanics are in common/ — see ADR 0018 and ADR 0019.

Identity

common/cfaccess.VerifyIdentity returns email plus name claims. Verify still returns only the email, so intelligence, workspace, and the MCP gateway are unchanged.

After authentication, internal/people upserts the person on every request: insert on first sight, refresh last_seen, and set first_name / last_name when the request supplies them. The principal key is the normalized email, which is already what ADR 0013 resolves a request to. Usage rows and budget overrides use that same string.

A person can also mint one API credential for themselves (docs/decisions/0079-a-person-mints-their-own-api-credential.md↗). The broker stores the client id and a hash of the secret. The JWT it signs names that person's email. Intelligence, workspace, and foundry verify the signature against /.well-known/jwks.json and use the email as the caller. In the cluster that address is plain HTTP, so those services set BROKER_JWKS_ALLOW_HTTP. The flag allows HTTP only for an in-cluster host. A public key-set URL still has to be HTTPS. The shared service token is still how a harness calls in. It is not handed to a person.

Foundry also POST /v1/people when an agent is created or renamed, and once for every agent when it starts. The person is ${slug}@ai.flow.life (prod) or ${slug}@dev-ai.flow.life (dev), first name agent, last name the agent name, and creator_email the agent's owner. A service token identifies Foundry.

A credential is personal or service. A personal credential's JWT names the owner, so it spends the owner's budget. A service credential's JWT names its own identity, <client_id>@service.broker, which gets its own budget. Broker creates that person row with the credential, first name service and last name the credential name.

people.creator_fid is set for a service identity and for an agent mailbox. It names the person who created them. GET /v1/usage?by_creator=true sums those identities under that person; the Usage page calls this Group by creator. Budgets are still per identity.

Name claims are best-effort. A browser Access assertion may carry only email. The UI falls back to the email.

Catalog and prices

services/broker/migrations/0002_catalog.sql↗ and services/broker/migrations/0003_seed_catalog.sql↗ seed Anthropic, OpenAI, xAI, and OpenRouter models with stable UUIDs. Later migrations add models the same way. 0008_grok_4_7.sql adds grok-4.7. Prices are USD micros per million tokens, versioned by effective_from. A price change inserts a new row; it does not rewrite history. A model with no price still records tokens; cost is null.

Model limits

Each model row carries the limits a caller needs before it builds a request. services/broker/migrations/0009_model_capabilities.sql↗ added the two that are not about price.

reasoning_effort_levels is a JSON array, and empty means the model takes no effort parameter at all. A caller must omit the field for those rather than send a value: Anthropic models take a thinking block with a token budget instead, and a non-reasoning model rejects the field outright. The broker relay forwards request bodies as it receives them, so it does not translate or strip the field on a caller's behalf — this catalog is how a caller knows not to send it.

GET /v1/catalog returns every provider and model with these limits. GET /v1/catalog/model returns one model, taking provider and wire_id as query parameters. The model is a query parameter rather than a path segment because an OpenRouter wire id contains a slash, as in z-ai/glm-5.3. The coding harness reads that route once per run to size its context gauge and decide whether to send reasoning_effort; see Foundry architecture.

Usage

services/broker/migrations/0004_usage.sql↗ follows ADR 0015:

broker_routing_decisions has one row per automatic routing decision. See Routing.

broker_usage_daily is a denormalized read shape in the same database, not a second store (ADR 0002). GET /v1/usage reads that rollup. The default group=day returns those rows as stored. group=person_model, person, model, and provider sum the same rows across the requested date range.

Pricing

services/broker/internal/pricing/pricing.go↗ takes token counts and a price row:

• Cost — input, output, cache-read, and cache-write tokens at their rates
• Cache savings — cache_read × (input_rate − cache_read_rate). Realized; the provider already charged the cheaper rate
• Routing savings — the same tokens priced against the requested model, minus actual cost. Estimated; stored on the row so the UI does not recompute it

Routing

Routing is on for every caller. people.auto_route defaults to true, and a platform admin turns it off for one person or service. Off means the caller always gets the model they asked for: neither static rules nor automatic routing run. A caller broker has not seen yet is routed.

A row in broker_routing_rules substitutes one named model for another. Its condition can limit it to requests with at most max_input_tokens estimated input tokens, at most max_turns user turns, or no_tools. A matching rule wins, and relay does not grade the request.

When no rule matches, relay grades the request light, standard, or frontier, in this order:

1. Quick check, in Go, with no model call. A request that asks for reasoning, whose earlier turns used tools, or whose estimated input is over 32,000 tokens is frontier. A single turn with no tools, a user message of at most 200 characters, and a system prompt of at most 1,000 characters is light. Anything else goes to the next step.
2. Jev, when both JEV_API_KEY and JEV_BASE_URL are set. With either one missing, this step is skipped. Broker sends the excerpt below as a score question over the three tiers, with an 800 ms timeout. The grade is used when Jev's confidence is at least BROKER_ROUTE_MIN_CONFIDENCE (default 0.8). Below that, the requested model is kept and the chat model is not asked. A timeout or error goes to the next step.
3. Chat model. A small model answers with one word, with a three second timeout, a small output cap, and temperature 0 (models that take reasoning_effort get their lowest effort instead of a temperature). When BROKER_ROUTE_CLASSIFIER_MODEL is set, that model always answers, whatever the requested model's family or provider. Dev and prod set it to z-ai/glm-5.3-flash on OpenRouter. When it is unset, the cheapest enabled model in the requested model's family answers. A configured model that fails is not replaced by the family's cheapest model; the requested model is kept.

The excerpt is the system prompt and last user message, cut short, the tool names offered, the number of user turns, whether earlier turns called tools, and the estimated input size. Long conversations and tool loops are graded frontier.

The substitute stays on the same provider and in the same route_family, and it must cost no more than the requested model. light is the cheapest model that still fits the prompt size, the completion cap, and reasoning_effort when the request set one. standard is the middle of that list. frontier keeps the requested model. A null route_family is left alone. When nothing answers, the requested model is kept.

gpt-5.4-fast, gpt-5.4-mini-fast, and gpt-5.5-fast are disabled and have no family. Fast is a service tier, not a model id, so those names are not models the API serves.

Each grading call is its own usage row with request_kind route; a Jev call is recorded against the jev catalog provider, and a configured classifier's call against that model's provider. The routed call's row has routing_reason rule (with routing_rule_id) or auto. Routing savings on that row are the requested price minus the actual cost. GET /v1/savings reports them net of the grading cost.

Every automatic decision is a row in broker_routing_decisions, written in the same transaction as the usage row and linked to it: the tier, which step graded it (quick, jev, chat, or none), Jev's confidence, the grading time, and the outcome (downgraded, kept_frontier, kept_low_confidence, kept_no_fit, or classifier_failed). GET /v1/routing/stats counts them. A call the provider rejected is not recorded.

The decisions are ADR 0098 and ADR 0100.

Budgets

One seeded default applies to everyone. An override is a row in broker_budget_overrides; its absence is the default. Relay checks spend against the effective limit before the upstream call. Report records the row even when over budget.

Relay

services/broker/internal/relay/relay.go↗ starts from the agents inference adapters. xAI and OpenRouter reuse the OpenAI-compatible path with a different base URL and key. Keys come from BROKER_*_API_KEY. Streaming is accepted on the request but the upstream call asks for JSON so the usage block can be parsed.

The provider call happens outside command.Execute. A transaction must not wait on another network (docs/architecture.md).

Tests

go test ./services/broker/...