# dsh-technocore-watch benchmark — W1, 3 runs per mode (2026-09-14)

Unofficial community integration — not affiliated with or endorsed by FLOP Labs. Local Technocore server only; no provider API key.

- **date**: 2026-09-14
- **commit**: b9680d0 (technocore-watch-core 3a47f92), no uncommitted code changes during any run (src, bench/*.ts, tests, package files)
- **scale**: 1 (15 workload minutes)
- **runs**: 3 per mode, run one at a time in the order W B B W B W (W = watcher, B = baseline), after excluding r3-watcher (see Excluded runs); each run boots its own local technocore-chat, proxy and harness
- **hardware**: Apple M5 x10, 16 GiB, darwin 25.6.0 (macOS 26.6.2)
- **node**: v26.0.0
- **uv**: uv 0.11.14 (Homebrew 2026-05-12 aarch64-apple-darwin)
- **dsh**: @deepseek-ai/dsh-agent-loop 0.1.5-rc.2 (in-process agent loop + shipping DeepSeek adapter against a local model double)
- **technocore**: technocore-chat v0.13.0 @20a4457 (local, CHAT_WAIT_POLL=0.5, CHAT_RATE_READ=600, CHAT_MAX_WAITERS_PER_IP=4)
- **watcher**: dsh-technocore-watch defaults: quiet 2 s / max 7 s / min wake 8 s, flood re-poll pacing on (lead 500 ms, min nap 1000 ms), 3 long-poll slots, wake delivery
- **baseline**: B0: per session, an ideal model looping on wait_for_message(room, since, 10) round-robin over its 3 rooms (the loop model answers each request at once with the next tool call: 0 ms added per turn)
- **workload**: W1: 2 sessions x 3 subscriptions (1 shared), idle 0-3, 8-burst at 3, 5 msg/s flood 4-9, trickle every 45 s, restart at 10, singles
- **method**: wake latency = post 200 OK -> the notice followup is enqueued to the session (watcher) / post 200 OK -> the wait_for_message tool result returns to the loop (baseline); neither includes model time. Reads are requests through the counting proxy excluding posts and config.

## Medians

Each cell is the median of the per-run values, with the range (min–max) across runs; a single number means every run agreed. Latency percentiles are computed per run, then the median of those is taken (not a pooled percentile).

| metric | watcher (median of 3) | baseline (median of 3) |
|---|---|---|
| wake latency p50 (ms, post 200 -> delivered to the session; see method) | 4413 (4411–4432) | 9181 (9165–9229) |
| wake latency p95 (ms) | 8076 (8066–8078) | 18914 (18913–18934) |
| wake latency max (ms) | 10883 (10230–12210) | 20150 (20095–20766) |
| flood wake latency p50 (ms) | 4491 (4486–4510) | 9364 (9362–9375) |
| flood wake latency p95 (ms) | 8075 (8073–8095) | 18962 (18946–18968) |
| burst wake latency p50 (ms) | 2116 (2018–2458) | 393 (187–518) |
| trickle wake latency p50 (ms) | 2211 (2201–2385) | 5121 (5088–5172) |
| seen by poll p50 (ms; watcher stage) | 3344 (3325–3363) | - |
| origin read requests / min (whole run) | 30.57 (30.51–30.7) | 14.11 |
| origin read requests / min (minutes 0-3: 45 s trickle only) | 26.06 (26.06–26.74) | 12.69 |
| origin read requests / min (flood, minutes 4-9) | 38.2 (37.4–38.4) | 15.8 |
| reads per watched-room-minute (whole run; /5 rooms watcher, /2 lanes baseline) | 6.11 (6.1–6.14) | 7.05 |
| reads per watched-room-minute (minutes 0-3) | 5.21 (5.21–5.35) | 6.34 |
| reads per watched-room-minute (flood) | 7.64 (7.48–7.68) | 7.9 |
| reads per subscribed-room-minute (whole run; same 5 rooms for both) | 6.11 (6.1–6.14) | 2.82 |
| model requests (total) | 54 | 221 |
| model requests in minutes 0-3 (45 s trickle only) | 3 | 37 |
| model requests during the flood (minutes 4-9) | 37 | 79 |
| deliveries to sessions (notices / non-empty wait results) | 54 | 42 |
| context characters added by deliveries | 39849 | 59069 (58905–59075) |
| largest model request body (bytes; baseline only) | - | 93826 (93659–93832) |
| duplicate delivered seqs (target 0) | 0 | 0 |
| deliveries after the restart | 9 | 9 |
| post x session pairs never delivered (target 0) | 0 | 0 |
| post x session pairs | 1525 (1525–1530) | 1526 (1520–1529) |
| tool calls | 0 | 221 |
| empty tool calls (target 0) | 0 | 175 |
| 429 responses | 0 | 0 |
| wait_held:false replies | 4 | 0 |

## Fairness note

- **Coverage differs.** Both modes serve the same 6 subscriptions (2 sessions, 5 unique rooms). The watcher follows all 5 rooms at once: up to 3 held long-polls plus the other rooms swept (every 15 s or more, promoted on activity), and the shared room is read once for both sessions. The baseline is 2 serial lanes (one per session), each holding one `wait_for_message` at a time round-robin over its 3 subscriptions, so a subscription sits under a held wait only about a third of the time and its messages wait for their turn.
- **Reads per watched-room-minute** divides reads by the rooms under watch at any moment times minutes: 5 for the watcher, 2 (lanes) for the baseline. **Reads per subscribed-room-minute** divides both by the same 5 rooms (raw reads/min / 5). Neither is the whole story: the first credits the watcher's wider coverage, the second ignores it.
- **Delivered content differs.** A watcher notice carries counts and seq ranges, never message text (738 characters per notice here), and the bench model answers every notice turn with text: it never calls `technocore_watch_read`, so the watcher's tool calls and empty tool calls are 0 by construction of the model double, not a measurement. A baseline `wait_for_message` result carries the message bodies. The watcher's model requests and context characters therefore leave out reading the messages; a session that reads them adds at least one tool call and one model request per read, plus the message text as context.
- **Latency endpoints differ.** Watcher: post 200 -> the notice followup is enqueued to the session. Baseline: post 200 -> the `wait_for_message` result returns to the loop. Neither includes model time.
- **The baseline is ideal.** Its looping model answers instantly (0 ms added per turn) and never stops or errs; a real model spends seconds per turn, which raises baseline latency and lowers its read rate, and pays tokens for every empty wait.
- **Scale.** Below scale 1 the workload minutes are compressed but rates and product timings are not (5 msg/s flood, 10 s long-poll, coalesce windows, 15 s sweeps); at 0.25 the trickle arrives every 11.25 s instead of 45 s.

## Wake latency by event kind (ms, median p50 / p95 / max, n)

| event | watcher | baseline |
|---|---|---|
| trickle | 2211 / 7090 / 7090 (n=19) | 5121 / 15646 / 15646 (n=19) |
| burst | 2116 / 2166 / 2166 (n=16) | 393 / 539 / 539 (n=16) |
| flood | 4491 / 8075 / 8498 (n=1487) | 9364 / 18962 / 20150 (n=1488) |
| single | 6402 / 10883 / 10883 (n=3) | 315 / 10454 / 10454 (n=3) |
| seen by poll (watcher stage) | 3344 / 7085 / 8881 (n=1525) | - |

## Per-run values

In execution order (sequential, one run at a time). Start is the UTC wall time the measured workload began; load is the 1-minute load average just before the run; max gap is the longest stretch without an origin request (over 60 s marks a stalled run).

| # | run | start (UTC) | load | max gap (s) | wake p50 / p95 / max (ms) | flood p50 / p95 (ms) | reads/min all / idle / flood | reads per watched-room-min | model requests | deliveries | context chars | duplicates | undelivered | tool calls / empty | 429 / wait_held:false |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | watcher 1 | 12:08:12 | 3.34 | 10 | 4411 / 8066 / 10883 | 4486 / 8073 | 30.57 / 26.74 / 38.2 | 6.11 (/5) | 54 | 54 | 39849 | 0 | 0 of 1525 | 0 / 0 | 0 / 4 |
| 2 | baseline 1 | 12:23:54 | 4.55 | 10.6 | 9229 / 18913 / 20766 | 9362 / 18962 | 14.11 / 12.69 / 15.8 | 7.05 (/2) | 221 | 42 | 58905 | 0 | 0 of 1520 | 221 / 175 | 0 / 0 |
| 3 | baseline 2 | 12:39:35 | 5.26 | 10.1 | 9181 / 18934 / 20150 | 9375 / 18946 | 14.11 / 12.69 / 15.8 | 7.05 (/2) | 221 | 42 | 59075 | 0 | 0 of 1529 | 221 / 175 | 0 / 0 |
| 4 | watcher 2 | 12:55:16 | 4.39 | 10 | 4413 / 8078 / 12210 | 4491 / 8075 | 30.51 / 26.06 / 38.4 | 6.1 (/5) | 54 | 54 | 39849 | 0 | 0 of 1530 | 0 / 0 | 0 / 4 |
| 5 | baseline 3 | 13:31:06 | 11.72 | 10.1 | 9165 / 18914 / 20095 | 9364 / 18968 | 14.11 / 12.69 / 15.8 | 7.05 (/2) | 221 | 42 | 59069 | 0 | 0 of 1526 | 221 / 175 | 0 / 0 |
| 6 | watcher 3 | 14:03:04 | 2.54 | 10 | 4432 / 8076 / 10230 | 4510 / 8095 | 30.7 / 26.06 / 37.4 | 6.14 (/5) | 54 | 54 | 39849 | 0 | 0 of 1525 | 0 / 0 | 0 / 4 |

## Excluded runs

Not part of any median above; kept here so nothing is dropped silently.

- **r3-watcher** (max request gap 369.6 s, 20.1 min): Invalid: the laptop lid was closed and the Mac went to clamshell sleep twice during it (pmset log: 6 s at 14:22:26 +0100, then 371 s from 14:24:13 +0100, workload minute 13.3), so for 369.6 s no request, post or delivery happened and the run lasted 20.1 minutes instead of 15.67. Re-run as the last run, with a no-sleep assertion (caffeinate -s -i) held for its whole duration; the max request gap column above shows that no kept run stalled. Its numbers: wake p50 / p95 4452 / 8086 ms, reads/min 21.69, model requests 52, duplicates 0, undelivered 0 of 1531.

## Origin requests by class (per run)

- watcher 1: {"read":129,"read-wait":350,"config":1} over 15.67 min
- baseline 1: {"read-wait":221} over 15.67 min
- baseline 2: {"read-wait":221} over 15.67 min
- watcher 2: {"read":128,"read-wait":350,"config":1} over 15.67 min
- baseline 3: {"read-wait":221} over 15.67 min
- watcher 3: {"read":129,"read-wait":352,"config":1} over 15.67 min