Architecture

Scope is region and subscription

The process loads the whole registry at startup, filtered to the pipeline environment named by ENV. GET /v1/environments returns that environment (dev or prod) and the region/subscription leaves the catalog lists for it.

GET /v1/environments/{region}/{subscription}/...

Dev and prod are where a service or app is deployed. They come from the generated catalog (the pipeline stages) and are shown as tags in the UI. They are not a path segment and they do not appear in environment-variable names. ENV on the process is only where this tool itself is running.

The registry

Three files under services/devops/registry↗:

logfacets.yaml is not part of the scope tree — log search is org-wide — but it loads from the same directory because it is the same kind of thing: hand-maintained configuration, mounted the same way, read once at startup. It is optional; an absent file falls back to a built-in list.

The catalog is derived from every azure-pipelines.yml under services/, frontend/, and mcp/, because those files already state what deploys where. A pipeline under services/ is tagged service; one under frontend/ is tagged app; one under mcp/ is tagged mcp. A stage that names no region or subscription lands in us/core. CI regenerates the file and fails on a difference, so a stale catalog stops a deploy rather than quietly misleading an operator.

scopes.yaml names variables; it holds no credentials and no hostnames. The file is baked into the image and the same image is deployed to dev and prod, so anything environment-specific written into it would be correct in at most one of them: credentials come from secrets.<env>.yaml and hosts from config.<env>.yaml. The Postgres entry names an environment variable; the process reads that variable when a request asks for that scope. How the path maps onto a name is on the operations page. Kubernetes logs in with an Azure AD service principal per subscription (AKS API server URL plus tenant, client ID, and client secret). Service Bus uses workload identity, or a named connection-string variable.

A subscription that exists in the tree but has no provisioned infrastructure is marked ready: false and still listed. A rollout in progress should look like one rather than like a bug.

Roles

Identity comes from Cloudflare Access, verified against the team's JWKS on every request. There are three roles. Everyone who passes Access can read; the other two divide what a request may change.

The split is what the change acts on. Turning a flag on is a release decision about one application, and the people who own that decision are the ones writing the code behind it. A shell in a pod, a rollback, and a republished event act on the deployment itself, which is a smaller group and a different kind of mistake. The roles are a ladder rather than separate grants: an admin can do everything an engineer can.

Reads are cluster-wide because an operator chasing a crash loop should not have to name the namespace first. Every write path is logged before the attempt, not after: an audit trail that records only successes is not one, and a rollback that hangs is exactly what someone will come looking for.

Who holds a role is configuration, not Cedar

Every domain service on the platform authorizes with Cedar, evaluated in process against a replicated copy of principals and their grants, so adding someone is a data change rather than a deploy. This service does neither. A caller is whoever Cloudflare Access says they are, and the role is that address against DEVOPS_ADMIN_EMAILS or DEVOPS_ENGINEER_EMAILS in services/devops/.azure/env/config.dev.yaml↗. An address on both lists is an admin.

That is a deliberate exception, for the same reason this service reads other services' databases: it owns no database, so it has nowhere to keep the copy of identity data Cedar would evaluate against, and standing one up for two role lists that change a few times a year would be the larger mistake. The cost is real and worth naming — granting a role is a committed diff and a deploy, and these roles have nothing to do with the platform-admin role the domain services know about.

Feature flags are proxied because the credential cannot be shared

FeatureHub is deployed from the infrastructure repository as a management server, which serves its admin UI and a management API under /mr-api, and a set of edge replicas the SDKs poll for values. Only the management server is called from here; the console never touches the edge, and shows its URL so an operator can see where a key is used.

Its management API has no long-lived token. A caller logs in with an email and a password and is handed a session. There is no credential that could be given to a browser that would not also be full administrative access to every flag in that environment, so the login stays in this process, the session is held in memory, and every call is proxied — the same reasoning as the Datadog Application key above.

Which instance a request reaches is the ordinary scope: scopes.yaml binds a management server and a login to each subscription, exactly as it binds a database and a cluster. A subscription with no featureHub block, or one whose management URL is unset in this environment, has no FeatureHub, and those endpoints answer 501 rather than failing.

FeatureHub has environments of its own — development and production inside an application — and they are not the region and subscription in the path. The scope chooses the instance; a FeatureHub environment is a division inside it, and appears as a column of the feature grid.

Every write to a feature value reads it first. FeatureHub's value endpoint takes the whole value rather than the fields being changed, and a field left out of the body is stored as cleared — so a write that sent only the value and the lock would delete any rollout strategy the feature had. Reading first also supplies the version FeatureHub compares, which is what turns two operators editing the same flag into a 409 instead of one of them silently losing. A 409 and a 422 are both passed on as themselves rather than reported as a gateway failure: they are FeatureHub saying what is wrong with the edit, not FeatureHub being unavailable.

API keys are the one thing the read endpoints withhold. The service account list returns names and permissions but never a key, and reading one is a separate request, open to the engineer role and logged with who asked and never with the key itself. A client-eval key reads every flag value in its environment, so a list that carried keys would hand them to anyone who opened the page.

Log search is scoped by environment, not by subscription

/v1/logs/config, /v1/logs/search, and /v1/logs/facets are the only read paths here without a region and subscription in the path. Datadog indexes every cluster in one place, so there is no per-subscription resource to address.

What does apply is the environment. Every outbound query is rewritten as env:<ENV> (<operator query>) before it reaches Datadog, with the operator's half parenthesised so an OR inside it cannot widen the result. This happens in the handler rather than in the browser because it is a boundary, not a default — see ADR 0021.

Both Datadog credentials stay in this process. The Logs API requires an Application key alongside the API key, and an Application key delivered to a browser is an Application key published; the proxy exists for that reason. Where the keys are unset, the endpoints answer 501 and the console renders a "not configured" state instead of failing.

Two endpoints are not JSON

A log tail is server-sent events and a terminal is a WebSocket. Neither fits a request/response model, so both are plain handlers outside the OpenAPI document, and the document says so.

Neither can carry an Authorization header, so the Access token travels as a query parameter for those two routes only.

Helm is a subprocess

helm list, helm history, and helm rollback are invoked as commands rather than through the Go SDK. The SDK pulls in most of Kubernetes and Helm's own dependency tree to reproduce three calls that already have stable JSON output, and helm history --output json is a steadier contract than Helm's internal release structs. The release name is validated as a Helm name before it becomes a process argument.