Operations
The billing API release, its develop deploy, and the in-cluster Accounting URL.
Billing deploys as one Helm release, billing-api. The image entrypoint is /service, built from
services/billing/cmd/api/main.go. That process serves HTTP and also runs the
outbox publisher, the lease consumer, the deposit-accounting consumer, and the invoice scheduler.
services/billing/.azure/azure-pipelines.yml triggers on changes under
services/billing. A merge to develop deploys to dev. A merge to main deploys to prod once
secrets.prod.yaml exists. That file uses the prod Key Vault key and is not in the repo yet.
Replica count, the node pool, and the dev ingress host are in services/billing/.azure/k8s/values/api.dev.yaml.
The scheduler posts invoices to Accounting. That URL is in
services/billing/.azure/env/config.dev.yaml. DEV_AUTH is not set. The
process refuses to start when that variable is 1 outside local.
Service Bus domains charge, billing_account, invoice, and collection are declared under
services/billing/.azure/bus/topology.yaml. The lease consumer reads
leases-lease-queue-billing. The leases topology names that subscriber; the queue exists once
that topology is applied. The deposit-accounting consumer reads
billing-deposit_accounting-queue-collection. That queue is created when the deposit-accounting
domain lists collection as a subscriber. Until that topology is applied, the consumer logs the
missing queue and retries. HTTP still serves.
First-time database user, secrets, and pipeline registration are in docs/devops/deploying-a-service.md.
Every domain in the bus topology needs fanout and queue settings in both
services/billing/.azure/bus/bus.dev.yaml and
services/billing/.azure/bus/bus.prod.yaml. These environment files also
include deposit_accounting and collection. Missing domain settings fail bus configuration
validation, including local emulator generation.
Invoice reversal queue
An invoice with withdrawn_by_cancellation=true is withdrawn through its charge corrections.
Its reversal_attempts is deliberately one above the shared reversal attempt limit to keep
older schedulers from reversing the entire invoice. This is healthy and does not require an
operator to reset the count. Pending original postings and charge corrections still run.
When looking for stuck full invoice reversals, exclude withdrawn_by_cancellation=true rows.
Inspect reversal_blocked_reason on voided invoices with a journal entry and no reversal entry.
For a full reversal that exhausted its limit, correct the Accounting rejection before setting
reversal_attempts back to zero. Do not apply that recovery procedure to parked withdrawals.
Their original posting errors are in posting_error, and individual charge correction errors
are in charges.cancellation_blocked_reason.
The shared limit is voiding.MaxReversalAttempts in
services/billing/internal/voiding/void.go.
Migrations
The withdrawal constraint is added without scanning existing invoices in migration 0050.
Migration 0052 validates existing rows in a separate transaction, after the first migration
has released its exclusive lock. Validation allows ordinary invoice writes to continue.
Tracing
The API supports opt-in OpenTelemetry export. Its configured internal HTTP clients propagate trace context and request IDs. Outbox events preserve trace context through Service Bus publication and consumption. See HTTP and messaging tracing for configuration, coverage, and local tests.
Automatic late fees
Late fee policies and assessment describe configuration, payment submission deadlines, net-rent attribution, accounting posting, and retry behavior.
Transfer posting work
The invoice scheduler also resumes Billing account transfers, opposite transfer journals, and
transaction-date corrections. Inspect billing_account_transfer.status and posting_error, and
billing_account_transfer_date_correction.completed_at and posting_error, for unfinished work.
The account's Transfers tab shows the same errors. Accounting selects its current Open month when the automated journal posts, and Billing records
the confirmed month. Transaction dates are retained across a delayed posting or retry.
Outside local development, the Accounting client uses Billing's own OAuth client credentials.
BILLING_TOKEN_URL and BILLING_SERVICE_CLIENT_SECRET enable machine-authenticated Accounting calls;
BILLING_SERVICE_CLIENT_ID defaults to svc-billing. Without them Billing still starts, but Accounting-dependent operations remain unavailable and
posting work retries safely. Accounting must configure ACCOUNTING_TOKEN_JWKS_URL and
ACCOUNTING_TOKEN_ISSUER to verify svc-billing bearer tokens for
https://accounting.flow.internal. The machine principal must be active; token verification
never revives a disabled principal. Local APP_ENV=local with DEV_AUTH=1 uses the Billing
development identity. Browser credentials are not forwarded to Accounting.
See Billing account transfers for reservations, provenance, and recovery behavior.