Flow
Decisions

0027 — How trusted a claim is, kept separate from where it is in the workflow

A claim carries a workflow column and a trust column that do not depend on each other, and nothing is ever stored as simply true.

Decided. Every claim carries two columns that do not depend on each other.

status says where the claim sits in the workflow: candidate, active, disputed, superseded, rejected, archived.

epistemic_state says how much the claim has earned our trust: asserted, corroborated, human_confirmed, authoritative, disputed, stale.

Neither can be worked out from the other, and both come back on every read.

No claim is ever stored as simply true. A read returns the trust state, how many independent sources back it, how recent it is, and the quotes it rests on, alongside the text.

Alternatives

One column, with trust folded into the workflow states. Fewer columns.

Rejected because it forces false choices. A claim can be published and thinly supported. A rejected candidate can be something three people independently confirmed. Merging the two columns means losing one of those facts.

A single confidence number. Easy to sort by, and it tells you nothing. A score of 0.72 does not say whether that came from one hedged sentence or four independent people, and those two need to be treated differently. We keep a confidence number, but as one signal rather than the answer.

Have a person review everything before it counts. This works at the scale of one person's notes.

Rejected because at thousands of observations a day the review queue never empties, and a review queue that never empties is a knowledge base that stays empty.

Why

  • Checking a quote is cheap. Every proposed claim comes with a quote and the exact character positions it was taken from. We check that the quote is really there. That costs nothing, always gives the same answer, and catches an invented quote outright. Only quotes that pass get the more expensive check of whether they actually support the claim.
  • Corroboration only counts if it is independent. Two pieces of evidence count as two only when they come from different people in different conversations. One person saying the same thing twice is one source.
  • An AI's output can support a claim but never corroborate it. Otherwise a knowledge base built from AI conversations would talk itself into agreement, turning an assistant echoing you into what looks like independent evidence. A single-user system never hits this. This one hits it immediately.
  • Showing where a claim came from beats trying to block bad ones. A wrong claim that gets through is still visibly weak, because its trust state and its sources travel with it.

Consequences

  • Each citation records the character range it covers, when the quote was last verified, how strongly it supports the claim, who said it, and whether that was a person or an AI. The last two are what make independent-source counting possible without a second table.
  • Each observation records a source tier. Only a system-of-record source can make a claim authoritative — a real deploy event outranks somebody saying they deployed.
  • Extraction labels hedged wording as a hypothesis, never as a fact or a decision.
  • Challenging is easier than asserting, on purpose. Anyone can challenge a claim, and it becomes disputed straight away with no vote. Cheap to flag and expensive to assert means the knowledge base drifts toward "we are not sure" rather than toward confident nonsense.
  • Each kind of claim has its own half-life. When it passes, the claim becomes stale, which keeps it out of context packs without deleting it.
  • The thresholds — how strong a quote has to be, how many sources trigger automatic promotion, the half-lives, when a candidate expires — are configuration, not constants in the code. They have to be measured against real traffic and changed without a deploy.

Source: docs/decisions/0027-epistemic-state.md.

On this page