Flow
ServicesIntelligence Service

Architecture

How intelligence decides what is known — the pipeline, how a claim earns trust, the job queue, the entity graph, and why the search index can always be rebuilt.

Binaries

BinaryPathRole
APIcmd/apiHTTP server, outbox publisher, the enqueue endpoints
Enrichment workercmd/intelligenceEnrichmentEvent-driven jobs. The one thing here that scales with load
Batch workercmd/intelligenceBatchClustering, rollups, prospecting, maintenance. Never scaled past one
Ingest listenercmd/intelligenceListenerOther services' events → queue rows. Calls no model

All four share one image and one set of configuration, because they have to agree on which embedding model is in use. Vectors written by one model and searched with another do not produce an error. They produce confident nonsense.

Domains

DomainTablesOwns
knowledgeknowledge_practice, knowledge_observation (partitioned), knowledge_insight, knowledge_citation, knowledge_candidate, knowledge_link, three knowledge_rollup_*, knowledge_project_packClaims, their evidence, and the derived signals
retrievalretrieval_chunk, retrieval_embedding (partitioned), retrieval_query_cacheThe index. The only place vector appears
entityentity, entity_alias, entity_edgePeople, teams, buildings, deals, vendors, and weighted edges
portfolioportfolio_statement, portfolio_property, portfolio_metric, portfolio_metric_observation, portfolio_metric_baseline, portfolio_entity_rollup, portfolio_attention, portfolio_decision, portfolio_finding, portfolio_brief, portfolio_audience_segment, portfolio_email_list, portfolio_memoThe compiled company model, the terrain the harness reads, and the executive memo
platformenrichment_job, enrichment_budget, principalsThe queue and the spend caps
attentionattention_profile, attention_item, attention_brief, attention_deliveryWhat one person should see today, kept as an issue across days

No SQL joins across domains (ADR 0023).

The pipeline

Each stage is its own kind of queue job. That means each one can be retried on its own, given its own spend cap, and seen on its own when it stops moving.

Observe

Fetch the text through the Derivation surface: The internal routes intelligence reads conversation text through, authenticated by a service token and a named principal. Evidence stays in workspace and is fetched per request rather than copied, so a redaction takes effect everywhere at once. and write down one observation for each piece of it. No model is called, so this step is cheap and always safe to repeat.

A source can be a conversation, a note, a meeting, a Slack thread, an uploaded document, a decision written on a project, the project's own write-up, a saved output, a Company fact: A statement about the business that a platform admin has decided is settled — a legal entity name, what a product is, which market the company sells into. It is a record in workspace's fact domain, written as markdown or uploaded as a file, readable by anyone and writable only by a platform admin. Intelligence derives claims from it the ordinary way and then confirms them instead of queueing them for review, and those claims go into every project's context pack. Distinct from an insight whose kind happens to be `fact`, which is any claim the pipeline extracted from a conversation and which still goes to a reviewer. a platform admin recorded, or the evidence on a project task. A task is never read as a whole, only the part of it that states something settled (Projects). Before anything reaches a model, all of them are converted into the same shape: a list of segments in order, each with a speaker, a sequence number, and a source tier. The rest of the pipeline reads only that, so nothing after this point has to care where the text came from.

Three things are never read: a scratchpad, a project or record that opted out of knowledge, and a project under legal hold. Everything else in workspace is evaluated. None of the three applies to a curated fact — it has no project, cannot opt out, and is read whenever it is active — so the only thing that stops one being observed is having been superseded or archived.

Gate

Cheap checks: is it long enough, does it say anything we do not already know, is this source allowed to feed the knowledge layer, and is the text the kind of thing worth reading at all. Most conversations stop here. That is the point. A model is only asked about text likely to contain something worth keeping.

The minimum depends on the kind of source. A conversation needs four turns, 800 characters, and at least two turns from a person; a meeting or a Slack thread is held to the same bar. On a meeting, a line counts as a person at Flow when the line names them, or when the roster already does and that display name matches exactly one of those people. A guest, or two people who share a display name, stays an outside speaker. When the roster or a line names at least one Flow person for certain, the meeting is read even if the lines themselves have no principal. The claim points at the transcript, and that person points at the transcript. When a quoted line names a Flow person, that person is also named as the author of the claim. A note needs 200 characters and nothing else. A note is one block of text somebody sat down and wrote, and holding it to a conversation's minimum would throw away every note there is.

A document takes the note's bar for a different reason. Its sections have no speaker, so they can never satisfy a human-turn floor and no document would ever be extracted from. Sections with no speaker are recorded at the external tier, which means a document can ground a claim but never corroborate one.

A decision, a saved output, and the project's own write-up take the note's bar too, and for the note's reason: each is one block of text somebody wrote on purpose. A project task's evidence has its own minimum for each type, and its tier follows whoever wrote the version of the task that holds it, so text the assistant wrote is agent_output. Both are on the Projects page.

Two of the checks are about what the text is rather than how much of it there is, and they exist because of agent sessions. An hour of an agent running tests, reading files, and reporting linter output clears four turns and 800 characters without difficulty, and what a model extracts from it is claims like "the test suite is run with go test" — true, grounded, quotable, and worthless. So a source that is almost entirely command output, diffs, stack traces, and tool narration is dropped, and so is one where the only human contribution is "ok" and "go ahead".

A curated Company fact: A statement about the business that a platform admin has decided is settled — a legal entity name, what a product is, which market the company sells into. It is a record in workspace's fact domain, written as markdown or uploaded as a file, readable by anyone and writable only by a platform admin. Intelligence derives claims from it the ordinary way and then confirms them instead of queueing them for review, and those claims go into every project's context pack. Distinct from an insight whose kind happens to be `fact`, which is any claim the pipeline extracted from a conversation and which still goes to a reviewer. is held to none of this. There is no length floor, because "Flow Technology Ltd is the contracting entity in Ireland" is sixty characters and is exactly what the domain is for, and the noise checks do not run either. Every check here is a way of asking whether unsolicited material is worth a model call, and for a fact a platform admin already answered that. The only fact that stops here is one with no text, which workspace refuses anyway.

Both thresholds are set high, because the two mistakes are not symmetrical. Extracting from one bad session wastes a model call that the later checks may reject anyway. Dropping a real conversation loses it silently, and nothing downstream notices. A design discussion with a lot of pasted code in it is exactly the material worth keeping, and it passes.

Extract

A model proposes claims. Each one comes with the exact stretch of source text it was drawn from.

There are two prompts, and which one is used depends on the source. The ordinary one is a discovery instruction: it describes the material as a conversation, a thread, or a note, tells the model that returning nothing is a valid answer, and asks it to prefer a few strong claims to many weak ones. That is right for a transcript and wrong for a Company fact: A statement about the business that a platform admin has decided is settled — a legal entity name, what a product is, which market the company sells into. It is a record in workspace's fact domain, written as markdown or uploaded as a file, readable by anyone and writable only by a platform admin. Intelligence derives claims from it the ordinary way and then confirms them instead of queueing them for review, and those claims go into every project's context pack. Distinct from an insight whose kind happens to be `fact`, which is any claim the pipeline extracted from a conversation and which still goes to a reviewer., which has already been through exactly that judgment — a fact listing five entity names would come back as two claims, with nothing recording that the other three were dropped.

So a fact gets a compaction instruction instead: every distinct statement in the source becomes its own claim, bodies are one to three sentences, and an empty reply is only correct for text that says nothing about the company. The citation rules, the ban on outside knowledge, and the rule that a sentence inside the source is never an instruction to the model are identical in both. A fact can be an uploaded document whose text nobody here wrote.

Both prompts also ask for officers — the formal appointments in a board consent, a set of bylaws, or a delegation of authority. A document like that is exactly what somebody records as a fact, and reading officers from it as an evidence upload but not as a fact would make the record depend on which screen the file was uploaded on.

Each officer is recorded as an office: a person, the company the office is at, and the title, such as Secretary or Manager. The company is the legal name the document gives ("Flow Global Holdings LLC"). It becomes an entity of kind company, found by name or created the first time it appears, and the person is linked to it by an officer_of edge. That edge does not fade in the nightly decay and is never pruned, because an appointment does not become less true when nobody mentions it. When a document names a group of companies at once ("the Flow entities, collectively"), the office is kept with no company rather than creating an entity for the group.

Offices are shown on the person's page, on the company's page, and on the Authorities page in the Officers tab, grouped by company. A reviewer can take an office off in two ways. Remove is for an office the person held and no longer does. Dismiss is for one the extractor got wrong. Either way the row is kept with the reason, so reading the same document again does not add it back.

Offices describe how the company is structured and nothing more. They grant no access, and they are not evidence of who decides things day to day, so they do not feed the practice rankings; see Authority and practices.

Both prompts also ask for influence: a statement in the text that one named person defers to, escalates to, or consults another. Each one needs a quote, and Ground checks the quote and resolves both names to people before it is kept. A name can be an email address, which matches the person whose work email it is.

Each claim is filed under one Practice: The area of expertise a claim belongs to: marketing, engineering, design, construction. Knowledge is company-scoped, so practice rather than project is what partitions the corpus.. The prompt lists every practice except unclassified as its slug, its name, and its description, and the model answers with a slug. The description is where the edges between practices are drawn, for example that Software Engineering covers Flow's software and not the engineering of buildings. A reviewer edits names and descriptions on the Practices page. An edit applies to claims extracted afterwards. To apply it to claims already filed, the reviewer re-files the practice from the same dialog: a background job asks the model where each live claim in the practice belongs, 500 claims per job, and moves the ones that belong elsewhere along with their authority signals. When the last page finishes, the authority rankings are rebuilt.

The prompt asks for a short title, but the model does not always give one. Before a claim is stored, line breaks and repeated spaces in its title become single spaces, and a title longer than the database allows (300 characters) is cut to fit and ends with "…". Without this, one long title would make the insert fail, and every claim from the same source would be lost with it.

Ground

Two checks that ask different questions.

The first asks whether the quote is really there, at the position the model said. The second asks whether the quote actually supports the claim.

A quote that passes the first check and fails the second is kept as background, not as evidence. If a claim ends up with no supporting quote at all, it is dropped rather than published without one.

Route

Session journal entries publish automatically, and so do claims backed by independent people. Facts and decisions with only one source go into a review queue as Candidate: A proposed insight waiting on a decision. Most are resolved without a person: well-grounded corroborated claims promote automatically, poorly grounded ones auto-reject, and anything unreviewed past its deadline expires rather than accumulating.. Anything whose quotes did not hold up is rejected automatically. Anything nobody reviews before its deadline expires.

A claim extracted from a curated fact is the exception, and it is the only one. It is confirmed as human_confirmed on ingest and published without review, because the write it came from was already gated to a handful of named people — sending it to a queue would be running the same check twice, and the one thing the admin was trying to say is that this is settled (docs/decisions/0037-admin-curated-company-facts.md). The state is human_confirmed rather than authoritative: that word is reserved for a system of record, and an admin is a person who is usually right, which is what human_confirmed already means.

Publishing a claim then kicks off two follow-up jobs. A contradiction check looks for nearby claims that might conflict, asks a model about any that do, marks both sides disputed if they really conflict, and publishes a notification back to workspace over the bus. Separately, every citation is scored for whether its quote entails the claim, and a claim left with no entailed citation is withdrawn — not disputed, since nobody disagreed with it; it simply never had evidence.

Routing is what makes the volume manageable. At thousands of observations a day, a queue where every candidate needs a person to look at it never empties, and a review queue that never empties promotes nothing. Reviewers get one summary a day, never one notification per candidate.

Meeting next steps

Workspace writes a summary of every Zoom, phone, and Plaud meeting, and the summary lists next steps: a thing someone in the meeting said would be done, who took it on, and the deadline as it was said. Extraction does not produce these, because none of its kinds means "a person said they would do this".

When workspace publishes evidence.summary_ready, a promote_commitments job reads the next steps and writes each one as a commitment claim. There is no second model call. Workspace has already checked each step against the transcript lines it cites, keeping an owner only if that person spoke or is named in one of those lines. The job quotes those lines as the claim's citations. That way the claim is grounded the same way an extracted one is, and a redaction of a line reaches it. A step whose cited lines are all redacted or missing is skipped.

Commitments publish without review and lose weight after 30 days with nothing restating them. When a newer summary of the same meeting no longer lists a step, that meeting's quotes are removed from the claim, and a claim with no quotes left is expired. If a later summary lists the step again, the same claim is published again. A commitment a reviewer rejected stays rejected. Extraction and discovery may not propose a commitment themselves.

Conversations nobody filed

A conversation recorded without a project goes to its author's scratchpad, and the pipeline above never touches it: the Derivation surface: The internal routes intelligence reads conversation text through, authenticated by a service token and a named principal. Evidence stays in workspace and is fetched per request rather than copied, so a redaction takes effect everywhere at once. refuses a scratchpad, so there is nothing to observe. It becomes part of what the company knows only after somebody moves it onto a project.

That move is the step that gets forgotten, so a separate job proposes it. Twenty minutes after an unfiled conversation goes quiet, its text is matched against the claim corpus, and each project that produced a matching claim is written back to workspace as a suggestion with a confidence and a sentence naming what matched. The author accepts or dismisses it, and accepting is not the move — that is a separate call, with the ordinary membership check.

The question being answered is "which project produced claims that look like this conversation", not "which project description does this resemble", because a description is written at kickoff and stops being true within weeks. The cost of that choice: a project with no published claims yet is never proposed, so filing into a new project stays a manual choice until it has been worked on.

Only projects the author is already a member of are ever proposed, and workspace decides that list rather than intelligence — so a suggestion can never reveal a project its recipient could not already see. docs/decisions/0035-suggesting-a-home-reads-a-scratchpad.md covers the rest of what bounds this, including why reading a scratchpad here does not weaken the rule that nothing derives knowledge from one.

Where a claim is, and how much we trust it

status says where a claim sits in the workflow. Epistemic state: How much credit a claim has earned — asserted, corroborated, human confirmed, authoritative, disputed, or stale — tracked separately from where it sits in the review workflow. Independence is counted by source, so ten quotes from one conversation is one source. says how much it has earned our trust (ADR 0027):

StateMeans
assertedOne source said it
corroboratedIndependent sources agree
human_confirmedA person vouched for it
authoritativeIt came from a source tier that outranks discussion — an approved decision, a signed document
disputedSomething credible contradicts it
staleNothing has said it again within this kind's half-life

The two columns move independently. A published claim can go stale. A draft can already be backed by several people. Merging them into one column is what produces a knowledge base that treats "somebody said this once" and "four people independently confirmed it" as the same thing.

A confirmed claim resting only on curated facts is the one thing that never goes stale. A half-life answers "has anybody restated this lately", and nobody restates the company's legal name. Such a claim leaves a pack when the fact behind it is superseded or archived, or when somebody challenges it — which anyone still can. "Settled" means it starts trusted, not that it is unfalsifiable.

Independence is counted by source, not by quote. Ten quotes from one person's one conversation is one source. Counting it as ten is exactly the mistake that makes a knowledge base confidently wrong.

The source tier is decided at the edge of the system, where we still know the facts. A Flow person speaking is human_conversation. An AI is agent_output. Someone in a meeting with no Flow account — a customer, a vendor — is external. An external line can support a claim about what that person said. It can never corroborate a claim about how the company works, because three outsiders agreeing is three people outside the building agreeing.

The job queue

The queue is a Postgres table, not a broker queue. A worker takes a job by holding a lease on it. Jobs are deduplicated on the pair of kind and target, retried with a delay that doubles each time, and set aside once they have failed too often. Hitting the day's token cap is not a failure: the worker puts that job off until the next UTC midnight without using a retry. The enrichment worker's lease is 15 minutes. It cancels a job 10 seconds before that so a slow verify_grounding fails once instead of being reclaimed and run twice.

It is a table because of how we wait for a conversation to settle. Forty turns arriving in one conversation have to collapse into a single job, and collapsing them is an update to a row. A broker can deliver a message and can delay one, but it cannot merge two into one.

Recovering jobs whose lease expired happens inside the API rather than as another queued job. A queue whose workers have all died cannot be repaired by adding a row to it.

How the kinds feed each other — who queues whom, and which nightly job has to finish before the next can read its output — is on Job kinds.

The search index can always be rebuilt

Chunks and their embeddings are calculated from text that lives in workspace, so the whole index can be rebuilt from source whenever we want. That is the argument that made the exception to the no-extensions rule acceptable (ADR 0022). Losing the index costs a rebuild, not data.

Search runs two rankings and merges them: one by meaning, one by keyword. It also folds in workspace's own search over raw text. Each is good at what the other misses. Embeddings find the sentence that says the same thing in different words. Keywords find the ticket number nobody rephrases.

The entity graph

An edge can connect any two things — a person to a project, a project to an idea, an idea to another idea — and each has a weight that fades over time. The fading is done by a scheduled job rather than worked out from a timestamp at query time. A graph where every connection ever made is still at full strength describes the company's history, not its present.

Two endpoints, because there are two questions. GET /v1/graph/subgraph answers "what is around this thing", from a focus of the form kind:id out to a chosen depth. GET /v1/graph/overview answers "what is this corpus about" and needs no focus: it returns the most connected nodes and the edges among exactly those nodes, so the result is a closed graph rather than a set of edges pointing at nodes the caller was never told about.

The graph page in intelligence-app opens on the overview and switches to a neighbourhood when a node is picked or clicked. The canvas defaults to a 3D force layout and can switch back to the original 2D one: node colour is kind, size is degree. It is not the whole graph — at company scale that is neither drawable nor readable — but a ranked sample of it is a starting point, and a focus chosen for the reader is a focus chosen arbitrarily.

Extraction writes three kinds of edge as it grounds an insight: mentions to every entity the insight names, derived_from to each project whose evidence supports it, and works_on from those entities to those projects. works_on carries a lower weight per observation than the others, because being named in the same insight as a project is weak evidence of working on it; it takes several before the edge clears the traversal floor.

Areas

An area is a group of decisions and facts about the same subject, such as "Lease Pricing". Areas are stored in induced_area, with their members in induced_area_member. Each area has an origin: induced when the nightly clustering found it, manual when a reviewer created it.

Induced areas. A Kestra flow calls /internal/jobs/induce-domains each night, which queues the induce_domains job. The job takes up to 2,000 decisions and 2,000 facts, groups them by the embedding of their first chunk with k-means (between 2 and 24 groups, depending on how many there are), and asks the model to name each group from its twelve titles nearest the centre. It then replaces every induced area in one transaction. Manual areas are not touched. An induced area's id is only stable until the next run.

Blocking. Deleting an induced area writes a row to blocked_area with the area's slug, name, and centre (the average embedding of its members). On later runs a group is dropped if its centre is within cosine similarity 0.9 of a blocked centre, or if the name the model gives it has a blocked slug. Checking the centre catches the same subject under a new name; checking the slug catches the same name on a group that drifted. Centres are only compared when they come from the same embedding model. Unblocking deletes the row, and the area comes back on the next run if the clustering still finds it.

Manual areas. Creating an area stores it as pending and queues populate_area for it in the same transaction. The job embeds the name and description together and adds every decision and fact whose embedding has a cosine similarity of at least 0.55, up to 500, closest first. It then marks the area ready. A manual area can share members with induced areas. The nightly run queues populate_area again for every manual area, so new decisions and facts are picked up. A manual name cannot reuse a blocked slug or the slug of an existing area. Deleting a manual area does not block anything.

On the graph. GET /v1/graph/area/{areaId} returns an area's members and, by default, the nodes one edge away from them, with only the edges that touch a member. The graph page uses it when the focus is area:<id>, linked from each area on the areas page. Members are drawn in full colour and the nodes around them faded. Any areas can also be drawn over the ordinary graph as regions: a padded outline in 2D, a sphere in 3D. The page asks POST /v1/knowledge/areas/memberships which areas the drawn insights belong to. Only insights are ever inside an area.

Titles, managers, and who is who

Three tables answer "who is this person" and each has a different owner.

  • entity and entity_alias hold the person and the names they go by.
  • person_title and person_reporting hold the formal title and the manager, each with effective_from and effective_to. A person has at most one current row in each (effective_to is null). A change closes the current row and opens a new one, so a title or manager from last quarter is still there. Each row has a source: directory for an org import, user for a person correcting their own record, inferred for something worked out from evidence. A directory import does not replace a title the person set themselves.
  • spine_identity_link maps an external account (a Slack id, a GitHub login, a work email) to a person. A link is active (reported as accepted) or pending_review (reported as review). A link in review is inert: nothing resolves through it and nothing is attributed through it.

The current manager is also kept as one reports_to edge in the entity graph, because the decision-rights model reads edges. The edge follows person_reporting: when the manager changes, the old edge is removed and the new one written.

Org import loads these tables. It is covered under operations. The people lookups that read them are under API.

Two rules keep a wrong answer out of reports:

  1. A name is not an identity. If two active people share a display name, the lookup says ambiguous and asserts nothing, and the importer skips the row and puts it in the identity review queue rather than choosing one.
  2. Private personal context never becomes a company claim. Workspace keeps each person's interests, threads, and commitments. This service does not subscribe to that domain. intelligenceListener reads four Workspace domains (conversation, evidence, fact, project), and a test fails if a person-context domain is added to that list or if a person-context event, were one to arrive, is turned into an observation, a job, or an insight.

Authority and practices

A title or a legal-entity office says who is allowed to decide. It often does not say who actually decides, or whose view the people who decide rely on. This service reads that from the graph instead and calls it observed authority. It is advisory: an observed claim grants nothing. The reasoning is in docs/decisions/0106-authority-is-influence-not-title.md.

Signals

Each signal says that one person treated another as the authority in a Practice: The area of expertise a claim belongs to: marketing, engineering, design, construction. Knowledge is company-scoped, so practice rather than project is what partitions the corpus.. Some come from a single source's text, and some are worked out from the graph on every rebuild:

KindWhere it comes fromSender
decision_signoff, slack_approvalThe decider on an extracted decisionNone
defers_to, escalates_to, consultsAn influence statement in the textThe person who deferred, escalated, or consulted
participantThe authors and attendees of a decision who were not the deciderEach of them, sharing one vote
citedThe author of a citation a decision relies onThe decider
question_assigneeThe person a company question in the practice was given toNone
decision_rightA has_decision_right belief from the operating model, in the practice its domain maps toNone

The last four are derived and replaced in full on each rebuild. A signal read from a decision only counts while the decision still stands, so rejecting a decision removes the authority it implied. Every signal is weighted by its kind and by how it was observed, and halves in weight every 180 days.

Ranking

infer_authority ranks each practice separately, except the unclassified practice, which holds what extraction could not place and is not an area anyone has authority in. Flow's own sending mailboxes (addresses on m.flow.life, such as the meeting summary sender) are recorded as people but never ranked: they pass on other people's decisions. A signal one of them sent counts as a signal with no sender. They get no practice tags and no activity either. Signals with a sender form a graph from sender to recipient, and each sender's outgoing weight is split into one vote. PageRank, with a damping factor of 0.85, runs over that graph. When it restarts, it lands on people in proportion to their activity in the practice plus the signals about them that have no sender. The result is influence: the person's share of the practice, which sums to 1 across everybody ranked in it.

Two more numbers say how much to trust that share:

  • evidence_mass is the total decayed weight of the signals about the person.
  • effective_sources is how many independent senders that weight comes from. Ten equal senders give 10, and one sender with all the weight gives 1. A signal with no sender counts its source record as the sender.

Confidence rises with evidence and is discounted when the evidence comes from fewer than three independent sources. A claim is corroborated once it has at least 3 of evidence from at least 2 independent sources, and asserted before that. A person with less than 0.5 of evidence is not ranked. The reporting line is the one input from the org chart. A person gives their manager a small vote of 0.1, and only when both already appear in the practice, so observed evidence outweighs it quickly.

The Authorities page shows each practice's people by influence, with confidence, a breakdown by signal kind, and every signal behind a claim. The operatingmodel command reads the same claims when it builds the review forms, grouped through the domain map in services/intelligence/config/practice-domains.yaml.

To check the ranking against people who know the company, run cmd/authorityeval. It scores each practice's top five against a reviewer's ordered list (NDCG@5). It also resamples the practice's source records and reports how far each person's rank moves. It builds the same graph the job does, inside a transaction it rolls back, so it writes nothing.

Practice tags on people

infer_person_practices tags each person with the practices they work in. Activity is counted per practice, with older activity weighted less:

ActivityWeight
Authored a standing insight1
Decided it1
Attended a meeting it was read from1 per meeting, split across that meeting's insights
Works on a project it belongs to0.5
Mentioned in it0.3

A practice becomes a tag when the person has at least 2 of activity in it, at least 15% of their activity is in it, and either they lean into it at least 1.5 times as much as the company does or it is their largest practice. A person gets at most three tags, and the unclassified practice is never one.

A reviewer can confirm a tag, add one, or remove one. Inference keeps the reviewer's choice and only refreshes the numbers. A removed tag is kept as removed, so the next run does not add it back.

Ordering people by activity or influence

The Authorities page and the Practices page both list people per practice, and both have the same switch: Activity (the default) or Influence.

  • Activity is the person's share of all activity in the practice. A person's activity is the weighted, age-decayed total of what they authored, decided, attended, worked on, and were mentioned in there. Both pages divide it by the total for everyone in the practice, so the shares in one practice add up to 100%. The API also returns the raw amount as activity.
  • Influence is the person's share of the practice's authority, from the authority rebuild.

Both measures are shares, so the two pages show the same unit for the same switch.

The tagging run saves every person's activity in every practice in person_practice_activity, not only the pairs that earned a tag. The table is rewritten on each tagging run.

Both pages list the same people in the same order. Each one reads /v1/knowledge/practices/{slug}/people with the chosen sort, so a practice shows only people tagged with it, and never a tag a reviewer removed. The Authorities page then adds each person's authority claim from /v1/knowledge/authorities to show the evidence behind their standing. A tagged person with no claim is still listed, with 0% influence and no evidence to open. Both pages leave off anyone whose share for the chosen order rounds to 0%. The Practices page shows the top eight in each practice, and the Authorities page shows up to a hundred.

People who have left

A reviewer marks someone who has left the company with the status button in the top right of their entity page, choosing "Left the company" in the dialog. That sets the person's status to departed and removes their authority and practice tags straight away. Later rebuilds leave them out: they are never ranked or tagged, and a signal they sent counts as having no sender. They stay in the graph, their relationships are kept, and the claims they made or are named in still count. The graph's entity list still shows them; other lists do not.

Choosing "Active" in the same dialog sets them back to active and queues an authority rebuild. Practice tags come back at the next tagging run, except tags a reviewer had confirmed or removed, which are not restored.

What goes in a context pack

A context pack is a project's briefing: the claims that are still current, the questions nobody has answered, work happening elsewhere that bears on this, and who to ask. It is prepended to a prompt, so every section is small — a pack that fills the window with background leaves no room for the conversation.

It is filled in two passes plus one section that is not selected at all, and the order is the point.

The first pass is provenance. Every live insight supported by a citation from this project's own evidence, newest observation first. These are the only entries that are certainly about the project.

The second pass is similarity, and only for the room the first pass left. It searches the company corpus for claims close enough to this project's subject matter to be worth reading beside its own, and marks each one related rather than project. The query is built from the titles of the project's own insights rather than from its name: a name is two or three words and an insight body is a paragraph, and comparing them scores far apart however relevant they are.

A project the first pass found nothing for gets no second pass. There is no query to run — its name is the only description available, and comparing a name to a body is what the title-based query exists to avoid. The closest match to three words in a small corpus is noise, so padding from it gave every empty project the same arbitrary neighbours and stated a relationship nothing had established. An empty pack is the honest answer for a project that has not produced anything yet.

Beside both passes sits company_facts: the claims derived from what a platform admin settled about the business. It is the same list in every pack, compiled once for the company and cached in its own row rather than copied into each project's, so one fact changing does not invalidate every cached pack. It is deliberately not chosen by similarity — the company's legal name is not topically close to a pricing project, and that is exactly the project that must not get it wrong. It has its own caps, a row count and a character budget, because what a prompt can afford is measured rather than guessed.

Doing similarity alone answers "what resembles this project", which is a different and much weaker question than "what did this project learn". Answering the first while presenting it as the second is how a project's briefing fills up with decisions other projects made, and it is why the source field exists on every entry rather than the two being merged.

Layout

services/intelligence/
  domains/*/events/         ← public event payload contracts
  internal/
    domains/                ← knowledge, retrieval, entity, platform, portfolio
    pipeline/               ← observe, extract, ground, entail, contradict, cluster, rollup, prospect
    inference/              ← Embedder and Extractor, their adapters, the budget meter
    ingest/                 ← other services' events → queue rows
    jobs/                   ← queue kind → handler, shared by both workers
    worker/                 ← claim, run, settle, back off
    workspace/              ← client for the derivation surface
    config/                 ← one loader, shared by all four binaries
    handlers/               ← routing, HTTP types, the enqueue endpoints
  messageBus/               ← generated fanout topology
  migrations/               ← platform, knowledge, retrieval, entity, pack, company facts, portfolio
  db/                       ← sqlc output, one package per domain

Company questions

The daily question_generate job reads recent Slack and Zoom evidence across the practices already on file, as DISCOVERY_PRINCIPAL. The nightly run reads that list. The Admin page can leave the subject blank for the same list, or type a phrase to limit that one run. A record belongs to a practice when its project practice matches, or when the practice name is a word in the title or the passages. Unfiled conversations have no practice, so the words are what include them. A long transcript is read from the start, and the rest of that record is left unread so one meeting does not stop the run. It groups that evidence by project, adds a few older passages from the same project when they help read a new signal, and adds a group when the same words show up in more than one project. It proposes questions from those bounded groups of human passages. Agent text is not evidence. A quote that is not in the cited passage is dropped, and a question that cites only the older passages is dropped. Before saving, it checks the inquiry catalogue, published decisions, open questions, and discovery packets, so a repeat, including the same question in different words, is filed as a duplicate or as already answered instead of being shown. A passage from the same discussion can answer it too. The answering sentence has to be copied from that passage. A sentence that only restates the problem, or that only asks something else, does not. Notes and Plaud transcripts are searched too. A passage counts only when it is itself about the question. A longer record that merely mentions the words does not. When a passage does settle it, the question is filed as already answered and is not shown. The first run reads the last week, or the last 90 days when that week is empty. Later runs continue from the previous one. The job may save nothing. Every conversational question that clears the checks is presented. A subject typed on the Admin page limits that one run to that phrase. Questions that need operational data stay in a separate queue and are not investigated.

When a question is saved, it is assigned to the person who spoke the most of the cited new lines and whose email is already on those lines. A tie goes to the later line. A speaker with no email is skipped, and the question is still saved with nobody assigned. The status does not change. The Mine queue lists questions assigned to the caller. Once a day, workspace sends each open assigned question as a Slack direct message. The message names the channel, meeting, or thread the question was drawn from. A reply in that thread is saved as a note in the assignee's name and the question is marked already answered. A message in the direct message is saved the same way against the latest question there that is still open. It does not start an investigation. The Slack tab on the analytics page counts questions posted and replies received, by person. Selecting a person lists the questions. GET /v1/discovery/slack-replies is the count. GET /v1/discovery/slack-questions is the list, and a question the caller cannot read is left out. Approving a question in the app freezes that version and runs it through the existing investigator. A later edit is a new version. The resulting card keeps origin generated.

Conversation discovery

The scheduled investigator starts from recent workspace evidence observations. The default trigger window is 14 days; DISCOVERY_WINDOW_DAYS accepts 1–730 days for a controlled backfill. The observation query counts a record's full history and uses its latest observation to qualify the trigger. A new reply can therefore bring an older discussion into the run. The default record limit is 200 (DISCOVERY_MAX_THREADS accepts 1–10,000); reaching this limit is reported, not presented as exhaustive coverage.

The investigator reads original Slack and meeting utterances through workspace, after checking the reading principal's access and the reviewed source-exclusion registry. It does not transcribe Zoom recordings. Recordings missing from workspace or the observation index remain outside this path. The adapter requests records and utterances separately, so conversation turns and project artifacts are not retrieved and discarded as unsupported record hits. Those source types remain outside this path. Unnamed transcript speakers are retained. Known notification senders are excluded; content length is a lightweight seed filter, not a requirement for three participants.

At most 20 seeds, spread across available source systems, retrieve related historical records. Natural-language query expansion supplies aliases and business concepts; the underlying workspace search is lexical, not an embedding index. Identifier matching remains another candidate source. Their candidates are interleaved and deduplicated before the 40-pair investigation limit, with at most four selected pairs per record to prevent a long document dominating the run. Overlapping-work and culture pairs are placed ahead of that interleave. The batch worker stops each sweep after 30 minutes. Four investigations run at the same time. Pairs not opened stay on the run as a queue of record ids. The next sweep continues that queue when the researcher version is unchanged. Generic format, role and discipline acronyms do not create identifier pairs; they remain available to text search. Each candidate then searches both seed records and makes up to three model-planned challenge/context searches. Generated queries request two to five terms: whitespace is AND in workspace search. An over-specific query with no hits receives at most two shorter queries, preserving the first entity and quoted phrases; explicit exclusions are never relaxed. The actual challenge queries are recorded.

At most six additional records enter an evidence context. Each record has a 12,000-byte text and message-overhead allowance (8,000 for candidate planning). Larger records are selected as original contiguous excerpts, with overlap, topical matches, and opening and closing context. Original source sequences are retained; citation indices refer to the displayed evidence packet. Selection is labelled PARTIAL in the model input, search log and card scope. Missing text cannot establish absence or that an issue remains unresolved. The combined evidence context retains a 180,000-byte guard. These are bounded searches, not proof that all relevant company evidence was found.

The model may propose overlapping work, commitments, disagreement, operating friction, observable help, and practices that may transfer, as well as contradictions and handoffs. It must add something beyond a topical summary. Net operating income is not the shared question. Profit is the default inquiry lens. Overlapping work and culture are separate lenses: overlap may report a coordination check, and culture reports what a person did. A culture card is one person, or people the sources show doing the same work. It can record a lift or a drop from that person's own earlier words. It does not join separate people's separate work, and it is not a judgment of the person. The packet list can be limited by lens, exact inquiry slug, entity text, and an exclusive created_after or updated_after instant. Each row includes the latest review mark, the source record ids, created_at, updated_at, and input_revision. Detail adds utterance_seq on each passage. occurred_at is when the line was spoken. created_at is when the investigation was stored. An admin run can search one of those categories. That run skips identifier pairing, and its queue continues only when the next run selects the same category. The scheduled sweep still covers every category. A question keeps the other search hits that use its terms, and a pair already quoted together on a visible card is left for a different discussion. Discussions cited by a claim linked to one of those hits are opened as well. The link chooses the record. The claim text is not evidence. A lead may rest on one source record and is saved for review with its uncertainty. The card names the lens, then a short headline and a short story of the business situation. Quotations stay under that story. Limits and the search log stay folded. A finding still needs two independent origins. Citation text must match a source substring and fit the 2,000-character passage limit. A separate verification pass sees both the citations and retrieved context to check negation, chronology, populations and qualifications. A malformed citation or unsupported draft can receive one revision using the exact validation feedback. The replacement must pass all citation and verification checks again; contradicted claims remain refused. Quote errors distinguish a missing quote, an overlong quote, and a substring mismatch. Invalid verdicts are refusals, not negative findings. JSON-mode requests always carry an explicit JSON instruction at the HTTP adapter.

Opposing quotations are optional. When none was found, the card must describe the challenge search and its limitations. A failed retrieval is not an empty successful search. Conversational findings remain attributed hypotheses; discovery does not infer measured savings or causal effects.

DISCOVERY_MODEL and DISCOVERY_REASONING_EFFORT override the shared extraction model settings for the batch investigator, while retaining its provider connection. Each scheduled investigation run has a fresh 2,000,000-token allowance across candidate planning, investigation, repair, and verification. It replaces the discovery daily cap; other job kinds keep their daily budgets. Requesting an immediate discovery run brings any pending investigation forward to now, including jobs deferred by the old daily cap. It reuses the pending job rather than adding a duplicate. Input estimates plus the full output allowance are reserved before calls and reconciled to reported usage, so a call can be stopped before the displayed usage reaches 2 million. Failed calls with unknown usage retain their reservation. Absent overrides inherit extraction settings. Each run records its model and algorithm version. A model change still requires comparison on frozen inputs and business review of withheld cases.

Every scheduled run is recorded before research starts. Its report retains source coverage, selection limits, model-call latency and tokens, considered pairs, search descriptions and refusal reasons. Packets, their source references, considered pairs and the completed run report commit in one transaction. A revision fingerprint suppresses unchanged reruns. All records consulted remain access dependencies, including uncited context. A failed run is distinct from a completed run with zero findings; a worker crash leaves a visible unfinished run. Pair starts and model-call starts and finishes are logged with stage, latency and token counts, without source text. The UI counters are still a completion report, not live progress. Budget exhaustion stops new work, commits already completed packets with a partial report, and finishes the queue job. It does not automatically retry with a fresh allowance. A separately started run gets its own 2-million-token budget, even on the same day. This is not a no-finding verdict or a resumable cursor. A crash followed by a new run ID starts a new allowance; reported usage remains in the completed run report.

Novelty checks look for readable prior investigations sharing source records. They do not establish what every employee knows. Business usefulness and actionability still require reviewer judgments. The first evaluation should replay a verified useful historical case and withheld cases from several business areas, then compare candidate recall, claim support, known-result rate and cost. Do not use a nonzero saved-card count as the acceptance criterion.

On this page