--- id: 01KXS4J44AWVNJMYZSWKYD5CC9 name: derived-progress-report version: 1.0.1 title: The daily progress report that checks itself — derived numbers, a page for people, a record the knowledge base ingests problem: A recurring report is read by people who cannot verify a single figure in it, and later by machines that cannot parse the prose it is written in. prerequisites: [read-files, edit-files, run-shell, read-version-control-history, http-client, scheduled-jobs] derived_from: [the build record of one live implementation, its release history, its fix commits, and one downstream consumer's misdiagnosis] stack: { language: go, runtime: "a single statically-linked binary + a coding agent", why: "the numbers must be derived by something with no imagination, and the reader of the recipe must be able to run the collector with no checkout of its source" } verified: { project: internal-harness, release: v0.88.1, date: 2026-07-16 } effort: "large — the collector and the wire client are a few days of ordinary code; the budget goes to the two halves that look free and are not: deciding what each number is allowed to be called, and the gates that keep the answer true after you leave" --- # The daily progress report that checks itself ## The problem You ship work every day and the people who fund it, depend on it, or plan around it cannot see any of it. So somebody writes a summary. That summary is the one artifact your team produces that **nobody checks**: every other output is consumed by a machine that rejects it when it is wrong, and a report is consumed by a reader who has no repository to open. Now add a second reader who arrives later — a shared knowledge base that must answer *"what shipped on that project last week, and who worked on it"* — and prose alone stops being enough. What you build: for each period, one **durable record of derived facts**, one **narrative document** carrying a machine-derived header, one **self-contained page** for people, one **regenerated aggregate** across every period ever reported, and one **idempotent transmission** of the period into a knowledge base that lives outside your repository and parses what you send. **The acceptance, in one sentence:** every quantity on the page can be traced to the facts record a script derived (there is no third source), the period a reader saw yesterday is byte-for-byte the same document today, and re-sending a period changes nothing on the other side — while a period nobody could measure says so, on the page, instead of showing a zero. ## Why the obvious approach fails The next agent will do the obvious thing, and it is worth naming precisely because it produces something that **looks right**: ask a capable model to read the period's changes and write the report — counting the work items by the ticked boxes in their documents, rendering it to a page, and posting that page to the knowledge base. Every part of that breaks, and none of the breaks is visible to the reader: - **A recalled number is plausible, and plausible is indistinguishable from true.** Asked how many changes landed, a model produces a number of the right magnitude. It is wrong, the reader cannot check it, and nothing in the pipeline ever disagrees with it. - **Counting ticked boxes reports the opposite of the truth, not an approximation of it.** In the verifying build, one live work item carried **0 of 41** ticked acceptance boxes in its documents while **7 of its 17** tracked phases were done. The box-counting report says **0%** — to a stakeholder, on a real project, about work that was more than a third finished. Not imprecise: inverted, in the direction that destroys trust fastest. - **Rendering first makes the page the source of truth, and the page is prose a model wrote.** Everything downstream — the cumulative view, the header the knowledge base reads — then has to parse generated prose to recover numbers that existed, structured, thirty seconds earlier. A generated artifact that parses generated prose is a house of cards. - **The consumer on the other side does not want your page.** It wants a shape it can key, dedup, and index. Posting a document and hoping means every project on the fleet invents its own dialect, and the divergence is found by the consumer, live. The fork: **derive the facts into a durable record first, write the judgment against that record, and render both the page and the wire shape from it.** Everything below is that inversion. ## The principles 1. **The numbers are derived; the judgment is written; there is no third source.** A script computes every quantity from the repository into a persisted record, and the prose cites that record. A number not in the record does not go on the page. (The genre's own division of labour: the machine gathers, the author abstracts.) 2. **Completion comes from the work items' own machine-readable state — never from a mark inside a human-written document.** Boxes are not maintained by anyone; the state a check or a board already reads is. 3. **Every proxy carries its method into the document.** A quantity that is not the thing it stands for ships with the sentence that says so, and the sentence lives **inside the facts record** — so the page cannot be rendered without the disclosure being available to state. 4. **Absence is a state, never a zero.** A source that is missing, unreadable, or not kept on this machine is reported as an absence with its reason. A silent zero is a lie told with a number, and it is worse than the proxy it replaced. 5. **A total is not a measurement, and two questions never share one name.** Concurrent effort summed is not elapsed time; volume handled is not volume produced. Where both are worth having, both are printed, each under its own name. 6. **A published period is immutable; the aggregate is regenerated.** The period contains prose (irreproducible → frozen); the aggregate contains only what the facts say (reproducible → rebuilt, and gated like any other generated artifact). The aggregate is built from the facts and never by parsing the prose. 7. **A derived number is safe to embed; a derived string is not.** Change subjects, titles and model-written headlines are untrusted the moment they reach markup, a structured header, or a wire body — regardless of how trustworthy their source is. Escaping is achieved **by construction** (a typed value handed to a serializer), never by remembering to escape. 8. **The wire shape belongs to its consumer, not to you.** Outside the repository, field names are somebody else's parser. Copy the shape exactly, key it so that re-sending replaces rather than duplicates, and treat a change to it as a breaking change. 9. **A period is named explicitly, never inferred from a window's edge.** A window and the day it describes are two different facts; deriving one from the other is how a record lands under the wrong key. 10. **A credential is read at use time and rides in the transport header only** — never in the body, never in a log, never on disk. This is what lets the sender print its entire payload for inspection without leaking anything. 11. **Every artifact names which half of it was machine-derived, and which agent produced which half.** Who *sent* a record and who *built* it are different facts; a reader who cannot tell them apart will diagnose the wrong component (see the scars). 12. **A gate that cries wolf is worse than no gate.** A generated artifact carries no clock, and a drift check compares the substance the project can re-derive, never the shared skin it merely inlines. ## The ground it needs Five things must be standing before step one. Each is a **ladder**: probe for it, build the smallest one if it is missing, or degrade — and label the degradation where the report's own reader can see it. There is no fourth rung. ### 1. A machine-readable history of changes, with timestamps and authors **Why it is load-bearing:** it is the only source in the building that cannot be flattered. Every delivery number and the effort proxy come from here. - **Probe:** anything that can answer "which changes landed between these two instants, by whom, touching what" without a human transcribing it. - **Build the smallest:** if changes are not recorded with timestamps, stop — this capability has no floor beneath it, and you are building a different thing first. - **Degrade:** there is no honest degradation. A report whose delivery numbers are typed is the failure this recipe exists to prevent. ### 2. A machine-readable state for the work items **Why it is load-bearing:** progress is the number a stakeholder reads first, and principle 2 says it may not come from prose. Without this, the report has no progress at all — which is a supportable outcome, and inventing one is not. - **Probe:** look for state that something already *acts on* — a check that fails when it is wrong, a board that renders from it. Not "a file that looks like a tracker". - **Build the smallest:** one optional target the project implements, which prints the normalized rows in the contracts section below. It is twenty lines. **The capability must never parse the project's own schema** — that would carry one project's tracker into every future project forever. The shared side declares the contract; the project prints it. - **Degrade:** no target → the report shows **no progress bars and states why**, carrying the reason as a first-class value in the facts record (a stated absence, not an empty list). The reason reaches the page. ### 3. A durable per-period home, and a check the project already runs **Why it is load-bearing:** the record must survive as the unit of history (principle 6), and the gates that keep the report honest must ride inside a command the project **already** runs. A new standalone gate needs every project to wire it up, and the project that forgets is exactly the one whose published history quietly stops matching its facts. - **Probe:** one command that the team believes means "green". A per-period directory convention anywhere in the docs tree. - **Build the smallest:** a directory per period holding the facts record and the documents; the existing check gains two lines. - **Degrade:** no check runner → the gates become a scheduled job that reports loudly. If they are not run at all, say so in the report's own honesty note: this report is ungated. ### 4. External access: a place to send the record, a credential, and a clock **Why it is load-bearing:** the second reader is a machine outside your repository. Without a sink this capability is still worth building — but its ingestion half does not exist, and the recipe's exact contract is the part you skip. - **Probe:** does the fleet have a knowledge base that ingests documents, and does it publish a wire shape? If it does, that shape wins over the one below. - **Build the smallest:** an endpoint that accepts the envelope in the contracts section, upserts on (project, period), and requires a scoped write credential. The unattended run needs a scheduler on a machine that has the credential. - **Degrade:** no sink → the reports stay local and the aggregate is the whole deliverable; say in the human note that this project's history is not queryable by the fleet, so nobody assumes it is. ### 5. A presentation layer for the page **Why it is load-bearing:** the page is forwarded — attached, mailed, opened on a phone with no access to your network. It must be **one self-contained file with no external fetches and no scripting**, or it will render as a broken skeleton in exactly the situation it was made for. - **Probe:** the project's own design tokens or shared stylesheet. - **Build the smallest:** one stylesheet, inlined at build time. - **Degrade:** no styling layer → an unstyled but self-contained page. Never a page that fetches its style at open time. ## The contracts ### The wire envelope The one shape a machine **outside** the emitting repository parses: the fleet's shared knowledge base keys, dedups and indexes it. Copy it byte-for-byte. **Binding: exact** ```json { "kind": "progress-report", "schema": 1, "project": "", "date": "", "title": "", "source": "/", "reviewed": false, "content_format": "markdown", "content": "", "metadata": null, "facts": null } ``` Field notes — three of these carry traps: | field | meaning, and the trap | |---|---| | `kind` | routes future record kinds; the consumer branches on it rather than sniffing the body | | `schema` | the **wire** version. Version it on its own axis, never coupled to your local record's version — otherwise a local shape change silently misreports the wire contract | | `project` + `date` | **the dedup key.** Re-sending the same pair must replace, never duplicate. `date` is the period the report is *about*, not when it was sent (principle 9) | | `title` | untrusted free text (it is derived from the document) — it is serialized, never concatenated | | `source` | **names the SENDER, not the generator.** Both facts matter and the field carries one; carry the generating agent's version separately or a downstream reader will diagnose the wrong component (see the scars) | | `reviewed` | `false` = an unattended run produced it, nobody confirmed the scope; `true` = a human ran it and read the output. The consumer should surface this on any answer it derives — an unreviewed record is a weaker source | | `content` | the **entire** narrative, header included. This is the primary ingestible document | | `metadata` | declared **nullable on purpose**: the consumer requires the key and treats its interior as a loose index, because projects legitimately carry different keys. The shape it *should* hold is the adapt-bound block below | | `facts` | the structured record, verbatim, when the sink opts in. Also nullable: a consumer that reads a missing key as starvation and an explicit `null` as "not sent" gives the reader one unambiguous answer — **emit the key always**, `null` when facts are withheld or dropped to fit the size cap | **Fixture: valid** ```json { "kind": "progress-report", "schema": 1, "project": "acme-web", "date": "2026-07-16", "title": "acme-web — progress report, 2026-07-16", "source": "acme-reporter/1.4.2", "reviewed": false, "content_format": "markdown", "content": "---\nkind: progress-report\nproject: acme-web\n---\n\n# acme-web — progress report\n\nCheckout redesign reached its third phase.\n", "metadata": { "kind": "progress-report", "project": "acme-web", "date": "2026-07-16" }, "facts": { "schema": 1, "project": "acme-web" } } ``` **Fixture: invalid** — the sender renamed a key into its own language; the consumer's dedup starves ```json { "kind": "progress-report", "schema": 1, "slug": "acme-web", "date": "2026-07-16", "title": "acme-web — progress report, 2026-07-16", "source": "acme-reporter/1.4.2", "reviewed": false, "content_format": "markdown", "content": "# acme-web\n", "metadata": null, "facts": null } ``` **Fixture: invalid** — the confirmation flag arrived as a string, so every record reads as truthy and "unreviewed" stops being distinguishable ```json { "kind": "progress-report", "schema": 1, "project": "acme-web", "date": "2026-07-16", "title": "acme-web — progress report, 2026-07-16", "source": "acme-reporter/1.4.2", "reviewed": "false", "content_format": "markdown", "content": "# acme-web\n", "metadata": null, "facts": null } ``` **Fixture: invalid** — an extra top-level field is a private dialect the consumer never agreed to; it belongs in the loose index block ```json { "kind": "progress-report", "schema": 1, "project": "acme-web", "date": "2026-07-16", "title": "acme-web — progress report, 2026-07-16", "source": "acme-reporter/1.4.2", "reviewed": false, "content_format": "markdown", "content": "# acme-web\n", "metadata": null, "facts": null, "internal_run_id": "8f21" } ``` ### The index block — the header the narrative carries and the envelope repeats Derived wholly from the facts record and stamped atop the narrative, so a machine can classify the period without parsing the prose beneath it. It is the **same block** the envelope's `metadata` carries, pre-parsed, so the consumer need not read the document's header format. **Binding: adapt** — the consumer requires the key, not the interior; different projects legitimately carry different measurements, and a record rejected over a schema quibble is a record that silently stops arriving. ```yaml kind: progress-report schema: 1 project: date: window: { since: , until: , label: } author: reviewed: false lang: repos: [] metrics: commits: 0 code_commits: 0 insertions: 0 deletions: 0 files: 0 hours_derived: 0.0 # the PROXY (principle 3) — always present # the four below are emitted ONLY when a runtime was actually read. # absent key = not measured; a zero here means "measured, and it was zero". agent_hours: 0.0 # effort: concurrent spans SUMMED wall_clock_hours: 0.0 # elapsed: the same spans, overlaps MERGED tokens_input: 0 # fresh input only tokens_output: 0 epics: [{ slug: , percent: 0, phases: /, status: }] tags: [, progress-report, ] headline: ``` The trap is the four optional metrics: make them **nullable/absent-capable in your types**, or "not measured" and "measured as zero" collapse into the same output — which is principle 4 broken in the one half where nobody can see it. ### The facts record — the durable local unit **Binding: adapt** — nothing outside the repository parses it; the aggregate and the header are both derived from it, so its shape is yours to name as long as it carries these roles. ```json { "schema": 1, "project": "", "lang": "", "date": "", "window": { "since": "", "until": "", "label": "" }, "git": { "commits": 0, "code_commits": 0, "insertions": 0, "deletions": 0, "files": 0, "hours": 0.0, "authors": [">"] }, "repos": [{ "name": "", "commits": 0, "insertions": 0, "deletions": 0 }], "epics": [{ "slug": "", "title": "