# SignalGraph

Produces deterministic evidence for conversion delivery and marketing-journey
health from sanitized aggregates, then attaches DataHub owner and lineage
context when configured.

Built for [DataHub's Agent Hackathon](https://datahub.com/blog/build-with-datahub-agent-hackathon/),
**Agents that do real work** category: read DataHub for context, evaluate a
deterministic rule, write one explicit action back. See [Project
status](#project-status) for what is verified today and
[docs/hackathon-demo.md](docs/hackathon-demo.md) for the judge path.

## The problem

Someone clicks your ad. The click carries a `gclid`. Days later that person
books, and the booking has to travel back to the ad platform as an offline
conversion so the platform learns which clicks turned into money.

When that last hop fails, nothing errors.

The booking is still in your database. The dashboard still renders. The ad
platform simply never hears about the sale, so it keeps bidding as if those
clicks were worthless. There is no alert, because from every individual
system's point of view nothing went wrong. Teams usually find out weeks later,
by noticing that reported revenue drifted away from real revenue.

Two questions follow, and neither is quick to answer during the incident. How
much money is affected? And who owns the thing that broke? The counts live in
a warehouse. The ownership lives in a catalog. Nobody is joining them at 9pm.

## How it works

```text
sanitized aggregate evidence (operator JSON, or local sanitized CSV rows)
  -> one deterministic rule (conversion delivery, marketing-journey health,
     signal admissibility, or delivery-window pricing)
  -> DataHub readback for owner and lineage (skipped only with --offline)
  -> stable JSON stdout + Markdown evidence file
  -> optional, explicit, idempotent DataHub incident (--apply-incident)
```

Four independent evaluators share this shape; each is documented in its own
section below and in [docs/architecture.md](docs/architecture.md). None of
them share state, open a connection you did not ask for, or read a clock.
DataHub is asked three read-only questions through its [official MCP
server](https://github.com/acryldata/mcp-server-datahub) — `search`, then
`get_entities`, then `get_lineage` — never mutated except through the two
opt-in write actions below.

## Use cases

| Situation | Evaluator | Section |
| --- | --- | --- |
| An offline conversion may not have reached the ad platform | `revenue_signal_integrity` | [What it tells you](#what-it-tells-you) |
| A marketing signal's trustworthiness is unknown before any rule touches it | signal admissibility gate | [Deciding whether a signal is usable at all](#deciding-whether-a-signal-is-usable-at-all) |
| A whole delivery window needs a price, a likely failure stage, and an owner to route to | `conversion_delivery` | [Pricing the whole delivery window](#pricing-the-whole-delivery-window) |
| Click-context, consent, and delivery health across a full marketing journey needs a read | `marketing_health` | [Marketing journey health](#marketing-journey-health) |
| *Roadmap, not built:* the same aggregate-contract shape applied to a ClickHouse web-events warehouse, a BigQuery-hosted CRM, HubSpot, or Typeform | — | [docs/roadmap.md](docs/roadmap.md) |

## What it tells you

SignalGraph compares how many conversions were eligible at the source against
how many the destination actually received, succeeded, and left pending. It
sorts that comparison into one of five outcomes, then reads DataHub for the
ownership and lineage of the asset involved.

Here is committed synthetic output, from `examples/broken-report.md`:

```markdown
## `missing_upload`

- Failure code: `MISSING_DELIVERY`
- Severity: `medium`
- Affected conversions: `1`
- Declared exposure estimate: `120.00 EUR` (not measured loss)
- Next action: Reconcile missing offline conversions and retry destination delivery.
```

The five outcomes:

| Outcome | Severity | What it means |
| --- | --- | --- |
| `NO_RECENT_UPLOAD` | critical | Supplied counts show no received, successful, or pending conversions. This does not prove the upload job is down. |
| `HIGH_MISMATCH` | high | Supplied counts show more than half of eligible conversions as unsuccessful. |
| `MISSING_DELIVERY` | medium | Supplied counts show some eligible conversions did not arrive or succeed. |
| `PENDING_OVER_TOLERANCE` | low | Conversions are sitting in the destination queue past the tolerance you set. |
| `HEALTHY` | none | The counts line up. |

When DataHub is configured, the same report also carries the destination
asset's owner and how many assets sit downstream of it, so the finding arrives
with a name attached instead of a dataset URN you then have to look up.

## Marketing journey health

The DataHub-first extension checks aggregate health across the path from
landing event to mature conversion record: provider click-context capture,
storage context, UTM/click-context mismatches, consent state, unauthorized tag
firing, context propagation, and destination delivery. It accepts counts only;
it never accepts raw `gclid`, `fbclid`, cookie values, customer identifiers, or
probabilistic attribution.

An upstream data-lake or warehouse job is expected to emit the sanitized
contract. SignalGraph evaluates it, then uses DataHub to answer the operational
questions: which asset represents the evidence, who owns it, and what is
downstream of it. DataHub is context, not the event source of truth.

```sh
python3 -m signalgraph.marketing_stub \
  --source-dir demo/stubs/marketing_health_alert \
  --output /tmp/marketing-health.json

python3 -m signalgraph.cli \
  --offline \
  --marketing-health /tmp/marketing-health.json \
  --markdown-out /tmp/marketing-health.md \
  --json-out /tmp/marketing-health.json
```

Denied consent is reported as context, not automatically treated as a defect.
Observed tags firing without required consent are `critical`. UTM/click-ID
mismatches are surfaced as `observe` until a source-specific policy says they
are an incident. Aggregate counts remain explicitly unverified.

## What it does not do

This matters more than the feature list, so it is not buried at the bottom.

**An upstream producer supplies aggregate evidence.** SignalGraph reads a
small JSON file emitted by a local fixture or a future data-lake/warehouse
adapter. It does not query your warehouse, browser, Google Ads, Meta, or CRM
today. Producing trustworthy aggregates from those systems remains outside
this repository.

**You supply the value assumption.** `declared_exposure_estimate` is
`conversion_value` multiplied by the number of affected conversions, and
`conversion_value` is a field in the file you wrote. It is not measured lost
revenue, and nothing looks up a real price.

**The evidence is labeled.** The finding records `counts` as
`operator_supplied`, `conversion_value` as `operator_declared_per_conversion`,
and `independent_verification` as `false`. DataHub context is separate
metadata readback; it does not verify the conversion counts.

**It is read-only by default.** The MCP server's mutation, user, and document
paths remain switched off. Two independent, opt-in flags are the only
writes: `--apply-incident` may raise one idempotent operational incident on
the validated destination asset, and `--apply-assertion` may upsert one
DataHub Custom Assertion and report the current finding as its result.
Neither retries, repairs, replays, or changes conversion data. It reports by
default; you must opt into either audited action.

**There is no model in it.** No LLM, no autonomy, no planning, no tool
selection. One rule, five branches, the same input always producing the same
bytes out.

**It refuses rather than guesses.** If DataHub returns an ambiguous search
result, a malformed entity, an asset with no owner, or a missing lineage edge,
SignalGraph exits non-zero. It will not fall back to a cached answer or a
fixture and present that as evidence.

The committed examples in `examples/` are synthetic offline output, marked
`datahub_evidence: false`. They are proof that the rule behaves, not proof of
a live DataHub run.

## Try it

Python 3.11 or newer. No dependencies to install, nothing to configure, no
DataHub needed:

```sh
git clone https://github.com/Atroci/signalgraph.git
cd signalgraph
./demo/run_demo.sh
```

That runs a healthy fixture and a broken one, and prints the paths to the JSON
and Markdown evidence it wrote. The broken one is the `MISSING_DELIVERY`
report shown above.

You can also read the output without running anything, at
<https://atroci.github.io/signalgraph/>.

## The file you have to write

One fixture is one comparison. This is the whole input format:

```json
{
  "fixture_id": "missing_upload",
  "source": { "eligible_count": 2 },
  "destination": {
    "received_count": 1,
    "success_count": 1,
    "pending_count": 0
  },
  "conversion_value": "120.00",
  "currency": "EUR",
  "lineage": {
    "source_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.booking_events,PROD)",
    "queue_asset": "urn:li:dataset:(urn:li:dataPlatform:kafka,signalgraph.qualified_conversion_queue,PROD)",
    "destination_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.offline_conversions,PROD)"
  }
}
```

Two eligible conversions, one of which arrived and succeeded. The arithmetic
shows one affected conversion and a declared exposure estimate of 120.00 EUR;
it does not prove a loss. The three URNs tell SignalGraph which DataHub assets
to ask about.

To derive counts from local sanitized rows instead of typing them into the
fixture, run the CSV path in [docs/demo-script.md](docs/demo-script.md#local-csv-derived-counts).

## Adding DataHub context

With DataHub configured, SignalGraph starts the official
[DataHub MCP Server](https://github.com/acryldata/mcp-server-datahub) and asks
it three questions, in order: `search`, then `get_entities`, then
`get_lineage`. It reads ownership and lineage for the three assets your
fixture named, and attaches the destination owner and downstream count to the
finding.

You need `uv` on your PATH, which provides `uvx`, and a reachable DataHub
instance holding the demo assets.

```sh
export DATAHUB_GMS_URL=http://localhost:8080
# Optional, only if your instance needs it:
# export DATAHUB_TOKEN='<your-token>'
./demo/run_mcp_demo.sh
```

The generic script runs one already-seeded fixture. For the complete operator
proof, `./demo/run_vps_mcp_demo.sh` reseeds two conversion fixtures, one
CSV-derived reconciliation fixture, and one marketing-health aggregate fixture
on an existing private VPS DataHub, opens an SSH tunnel, and verifies the
reports.
The full procedure is in [docs/datahub.md](docs/datahub.md). Direct GraphQL is
still supported as a legacy mode, and it is never used as a fallback when MCP
fails. To demonstrate the explicit incident action after validated MCP context:

```sh
./demo/run_vps_mcp_demo.sh --apply-incident
```

To demonstrate the DataHub Custom Assertion action instead — one
`upsertCustomAssertion` plus one `reportAssertionResult`, run for every
finding rather than only a failing one — use the CLI directly, since the VPS
runner does not forward this flag yet:

```sh
python3 -m signalgraph.cli --mcp --apply-assertion \
  --markdown-out /tmp/evidence.md --json-out /tmp/evidence.json \
  demo/fixtures/missing_upload.json
```

Both flags are optional, apply only to conversion fixtures, and are not used
by the default judge path. Marketing-health reports remain read-only in this
slice. See [docs/datahub.md](docs/datahub.md#explicit-custom-assertion-action)
for the field-by-field detail.

## Deciding whether a signal is usable at all

`--signal` is a separate, optional gate that answers a different question:
before any rule reads a marketing signal, is that signal admissible as
evidence?

```sh
python3 -m signalgraph.cli \
  --offline \
  --markdown-out /tmp/evidence.md \
  --signal demo/fixtures/marketing_signal_admit.json
```

It returns `admit`, `quarantine`, or `deny` with a reason code. It denies raw
and derived identifiers, personal and health data, identity resolution, CRM
synchronisation, attribution modelling, and probabilistic scores. It collects
nothing, resolves no identity, keeps no state between runs, and opens no
connection. See [docs/marketing-signal-contract.md](docs/marketing-signal-contract.md).

## Pricing the whole delivery window

The rule above answers one comparison. The commercial question is bigger: over
a whole delivery window, how many conversions did not make it, what were they
worth, and where did they stop?

`signalgraph.conversion_delivery` answers that from one sanitized aggregate
document:

```text
warehouse / CRM / booking platform
  -> eligible conversions
  -> aligned time window
  -> upload attempts
  -> normalized destination diagnostics
  -> pending / rejected / duplicated / missing
  -> affected count and affected value
  -> likely failure location
  -> DataHub owner
  -> recommendation-only alert / incident / optional retry actions
```

The committed golden incident, `demo/fixtures/conversion_delivery_golden.json`,
reads like this. Ten conversions were eligible and worth 1000.00 EUR. Nine were
attempted, eight were normalized by the destination, four were accepted, and
three came back successful in both the CRM and the booking platform. An excerpt
of what comes back, alongside a full `evidence` block and recommended actions:

```json
{
  "affected_count": 7,
  "affected_value": "700.00",
  "currency": "EUR",
  "failure_location": "upload_job",
  "owner_routing": { "label": "growth_operations" },
  "reason_codes": ["duplicated", "missing", "pending", "rejected"],
  "state": "alert"
}
```

Seven conversions with an affected value of 700.00 EUR are reported, the first
place to look is the upload job, and the finding routes to
`growth_operations`. The `recommended_actions` list contains alert, open
incident, and retry suggestions; every one carries `executed: false` and
`requires_approval: true`. Nothing is sent, opened, or enqueued.

The same boundaries apply here as everywhere else in this repository. It
consumes aggregate evidence you produce; it is not a warehouse, CRM, booking
platform, GA4, Ads, destination connector, CDP, attribution warehouse, or
identity system. Aggregate CRM and booking outcomes are allowed as evidence;
rows, references, and identifiers are refused outright. DataHub stays read-only
context for lineage, contracts, owners, and diagnostics. There are no live
destination adapters and nothing runs on a schedule.

One flag runs it, offline, against the committed golden fixture:

```sh
python3 -m signalgraph.cli \
  --offline \
  --conversion-delivery demo/fixtures/conversion_delivery_golden.json \
  --markdown-out /tmp/evidence.md
```

It is fail-closed. Only a `healthy` or `alert` evaluation renders a report;
contradictory or excluded evidence exits non-zero with its reason code rather
than publishing a partial finding.

Full explanation, the step-to-evidence table, the privacy boundaries, and the
MVP-versus-later split are in
[docs/conversion-delivery-assurance.md](docs/conversion-delivery-assurance.md).

## Project status

SignalGraph was built for the DataHub Agent Hackathon and is a working
prototype rather than a product. Two deterministic evidence paths, four
synthetic demo assets, no packaging for production use.

The rule engine, the offline path, the MCP adapter, the signal gate, the
conversion delivery evaluator, the CSV reconciliation path, the marketing
health evaluator, the lake-shaped marketing stub, and the explicit incident
action are built and covered by 159 tests. Legacy
direct-GraphQL readback passed against DataHub 1.7.0 on 2026-08-05. The fresh
MCP path passed against DataHub 1.7.0 on 2026-08-06 through a private SSH
tunnel: the two conversion fixtures, CSV-derived reconciliation, and
marketing-health fixture returned validated metadata, one owner, one
downstream asset, and their expected deterministic findings. The live JSON and
Markdown remain outside Git; committed examples remain synthetic offline
output. Scope and acceptance gates are
in [MVP.md](MVP.md); the specifications are in
[specs/001-datahub-presentation-readiness/spec.md](specs/001-datahub-presentation-readiness/spec.md),
[specs/003-conversion-delivery-assurance/spec.md](specs/003-conversion-delivery-assurance/spec.md),
and [specs/005-marketing-journey-health/spec.md](specs/005-marketing-journey-health/spec.md).

Do not commit tokens, live owner data, raw DataHub responses, or live reports
to this repository.

## Roadmap

Everything above is built, tested, and passed a live DataHub gate. The
longer-term idea is bigger than the MVP: a DataHub-grounded signal-detection
layer that reads sanitized, aggregate-only signals from wherever a marketing
or growth team already has them — ad platforms, email, organic/search,
website and backend events, a ClickHouse web-events warehouse, a
BigQuery-hosted CRM, HubSpot, Typeform — and attaches DataHub owner and
lineage context to whatever it finds.

None of that is built. [docs/roadmap.md](docs/roadmap.md) states the
long-term category, the candidate sources, and exactly what has to happen
before a candidate stops being a roadmap bullet: its own feature spec, its
own sanitized-aggregate contract, tests, and the same live-DataHub
verification gate every shipped feature in this repository already passed.
The constitution's ["Vision and roadmap (not
built)"](.specify/memory/constitution.md#vision-and-roadmap-not-built)
section is the binding version of this boundary.

## Checks

```sh
python3 -m unittest discover -v
./demo/judge_demo.sh
```

Reviewers need no credentials and no DataHub for the offline gate.
`demo/judge_demo.sh` checks the suite, syntax, byte-stable offline artifacts,
and the CSV-derived path. `demo/run_vps_mcp_demo.sh` is the separate operator
gate for live MCP context and optional incident writeback.

## Code map

`signalgraph.contracts` defines the fixture and finding shapes.
`signalgraph.rules` holds the conversion integrity rule.
`signalgraph.mcp_context` performs read-only MCP normalization.
`signalgraph.datahub` performs the legacy GraphQL readback and the explicit,
opt-in incident and Custom Assertion actions.
`signalgraph.signal_contract` gates signal admissibility.
`signalgraph.conversion_delivery` evaluates one delivery window over sanitized
aggregates. `signalgraph.marketing_health` evaluates sanitized marketing
journey-health aggregates. [docs/architecture.md](docs/architecture.md)
describes the evidence boundary.

The repository root also carries a conformant [Agent
Plugins](https://github.com/agentplugins/agent-plugins-spec) v1.0.0 package
(`plugin.json` plus `skills/signalgraph-evidence-check/`), so a compatible
agent client can discover this CLI as a portable skill. See
[docs/agent-plugin.md](docs/agent-plugin.md) for the conformance checklist;
[tests/test_agent_plugin.py](tests/test_agent_plugin.py) enforces it.

## License

Apache 2.0. See [LICENSE](LICENSE).