# Herald OS Architecture

Herald OS is an agent-native operating system: it makes Hermes Agent the primary interface between
the user and the computer. Herald OS Linux is the whole system (Fedora underneath, the Herald shell
as the session); on macOS the same shell runs fullscreen over the macOS desktop. It is a separate
product from Hermes Desktop: it consumes the upstream Hermes runtime unchanged and grows its own
shell on top.

## Three authorities

The same seam rules upstream Hermes Desktop uses apply here:

| Party | Owns | Lives in |
| --- | --- | --- |
| Electron main | The machine: window/fullscreen lifecycle, backend process, PTYs, filesystem, app discovery, system stats, notifications, typed IPC | `apps/desktop/electron` |
| Renderer (React) | The experience: surfaces, navigation, presentation, ephemeral interaction state | `apps/desktop/src` |
| Hermes runtime | The work: sessions, model calls, tools, skills, memory, cron, approvals | the user's Hermes install (`$HERMES_HOME/hermes-agent`) |

The renderer never touches Node or Electron directly; native power arrives through a narrow,
typed preload bridge (`window.heraldOS`). Agent behaviour is never re-implemented in React.

```mermaid
flowchart LR
  subgraph shell [apps/desktop Electron shell]
    Main[Electron main]
    Preload[preload bridge]
    Renderer[React renderer]
    Main --> Preload --> Renderer
  end
  subgraph runtime [Hermes runtime in HERMES_HOME]
    Serve["hermes serve (JSON-RPC + REST)"]
    Agent[AIAgent, tools, skills, memory]
    Bridge[herald-os-bridge plugin]
    Serve --> Agent --> Bridge
  end
  Renderer -->|"WebSocket JSON-RPC (@herald-os/client)"| Serve
  Main -->|"REST with session token"| Serve
  Bridge -->|"request_tool_approval"| Serve
  Bridge -->|"mdfind, open, lsof, ps, osascript"| macOS[(macOS)]
  Bridge -->|"plocate, gio, ss, ps, nmcli, journalctl"| Linux[(Linux)]
```

## Host platforms

Both seams (`HostPlatform` in Electron main, `HostAdapter` in the bridge plugin) are implemented per
host. The renderer and the agent see identical tool names and return shapes on every platform.

| | macOS | Linux (Herald OS Linux) | Windows |
| --- | --- | --- | --- |
| Shell runs as | fullscreen over the macOS desktop | the whole session: `greetd` -> `niri` -> shell (`cage` as a fallback) | stub |
| `HostPlatform` | `platform/darwin.ts` (`plutil`, `sips`, `qlmanage`, `mdfind`, EventKit JXA) | `platform/linux.ts` (`/proc`, `.desktop` entries, icon themes, `nmcli`, `recently-used.xbel`, GNOME thumbnail cache) | `platform/generic.ts` |
| `HostAdapter` | `host/darwin.py` (`osascript`, `lsof`, `mdfind`, `system_profiler`, `pmset`) | `host/linux.py` (`ps`, `ss`, `plocate`/`fd`, `gio`, `xdg-open`, `nmcli`, `bluetoothctl`, `wpctl`, `gsettings`, `journalctl`, `loginctl`) | `host/windows.py` stub |
| App launch / reveal | `shell.openPath` / `showItemInFolder` | `gio launch <.desktop>` / `org.freedesktop.FileManager1` | |
| Calendar | EventKit with permission state | unavailable (EDS later) | |

Setup, session model and dev loop for Linux: `docs/LINUX.md`.

## Backend lifecycle

1. Resolve a runtime through an ordered ladder (`apps/desktop/electron/backend/resolve.ts`):
   `HERALD_OS_HERMES_ROOT` -> `$HERMES_HOME/hermes-agent` managed install (the `.hermes/bin/hermes`
   launcher today's installers publish, else the `venv/bin/python` of earlier ones) -> `hermes` on
   `PATH` or in `~/.local/bin`. Each candidate is probed before use. `herald-os` looks the same way
   (`hermes_install`; `herald-os hermes-command` for scripts such as `firstboot.sh` and
   `herald-os-update`). With no runtime the shell waits and looks again every few seconds; on the
   image it also waits while `/var/lib/herald-os/hermes-pending` says the first boot's Hermes install
   is still to come (`herald-os-hermes.service`).
2. Spawn `hermes serve --host 127.0.0.1 --port 0` with `HERMES_DASHBOARD_SESSION_TOKEN` (random per
   launch), `HERALD_OS=1`, and `HERMES_HOME`.
3. Read `HERMES_BACKEND_READY port=N` from stdout, then poll `GET /api/status` with
   `X-Hermes-Session-Token` until it answers.
4. Hand `{ wsUrl, baseUrl }` to the renderer. The renderer dials `ws://127.0.0.1:N/api/ws?token=...`
   with the upstream `JsonRpcGatewayClient`; REST calls go through main so the renderer only ever
   sees a capability, not a credential.
5. On exit the child is restarted with bounded backoff; after the budget is exhausted the shell
   shows a recoverable failure screen instead of spinning.

## Wire contract

The wire is declared in Python (`tui_gateway/contracts`) and generated into
`apps/shared/src/gateway-contract.generated.ts` upstream. Herald OS re-exports that file through
`packages/hermes-client`, so a field the backend stops sending fails `tsc` here instead of drifting.

Sessions are created with `source: "herald_os"`. Upstream only folds Desktop-only GUI tools into
sessions whose source is `desktop`, so a Herald OS session never receives tools whose client half
lives in Hermes Desktop.

A conversation can also run as another Hermes profile, a bot (`src/store/bots.ts`; the roster is
`profiles.list`, a bot's picture `profiles.get_asset`). Each profile keeps its sessions in its own
`state.db`, so `session.create`, `session.list`, `session.resume` and `session.delete` carry the
bot's `profile`: the gateway opens that profile's store and home (an unknown profile is error
4064), and a resume without it looks in the launch profile's store and finds nothing. The live
session keeps its profile, so `prompt.submit` and the other calls by runtime id need none, and
Hermes's own sessions still go without `profile`. Stored ids can repeat across profiles, so the
renderer keys a bot's sessions by profile and id.

Server-to-client requests the shell answers: `approval`, `clarify`, `sudo`, `secret`. Anything else
is declined with `-32601` so the backend treats it as unanswered rather than hanging.

## Renderer

Paths in this section and the next are relative to `apps/desktop`.

The renderer draws a whole desktop: wallpaper, menu bar, dock, command bar (`Cmd+K`), notifications
and a small window manager. Its apps are registered in `src/shell/apps.ts`:

- **Pages** of the main Hermes window, picked from its sidebar: Overview, Hermes (chat), Missions,
  Memory, Files, Automations, Connections, Settings.
- **Floating apps** with their own window: Terminal, System, Web, Studio, Herald Canvas, Herald
  Docs, Sheets and Slides, and a chat pop-out.

It runs in one of two modes, chosen by Electron main (`electron/shell/mode.ts`):

| Mode | Used by | Windows |
| --- | --- | --- |
| `desktop` | macOS, Linux under `cage` | One fullscreen Electron window; the renderer's own window manager (`src/store/windows.ts`, `src/shell/wm/`) lays out every app inside it. |
| `panels` | Linux under `niri` (`HERALD_OS_SHELL_MODE=panels`) | One Electron window per surface (menu bar, dock, main window, command bar, wallpaper, each floating app), tiled by the compositor. `src/shell/ShellRoot.tsx` picks what each window shows. The Dock's menu for a running app is a window the Dock opens and draws into itself. |

Code layout (details in [`apps/desktop/README.md`](../apps/desktop/README.md), conventions in
[`apps/desktop/DESIGN.md`](../apps/desktop/DESIGN.md)): `src/shell` is the desktop chrome,
`src/features/<app>` holds one folder per app, `src/store` the nanostores state and actions,
`src/lib` pure helpers with their tests, `src/commands` the OS command catalogue.

Renderer faults land in the same log as main's own, `~/.hermes/logs/herald-os.log`.
`src/lib/report.ts` writes each one on the window's console as one `[herald-os]` line with its stack:
a call nothing waits on names itself when it fails (`logFailure`), and a net that `main.tsx` installs
before anything else loads reports uncaught errors and unhandled rejections in place of Chromium's
own line. Electron main copies each shell window's warnings and errors into the log as
`[renderer] <surface>` lines (`electron/renderer-console.ts`), cut at 4,000 characters and at most 30
a minute per window. Pages that are not Herald's are left out: web windows, Studio previews, file
viewers and plugin frames.

## OS control

Everything a user can do in the shell is also a registered command (`src/store/os-commands.ts`,
catalogue in `src/commands/`), so three callers drive the UI the same way:

1. The command bar.
2. The voice fast path (`src/lib/voice/intents.ts`), which runs simple requests such as "open
   missions" or "hide the sidebar" locally, without a model round-trip.
3. Hermes itself, through the bridge's `os_ui` tool. Electron main serves a JSON-lines Unix socket
   (`~/.hermes/herald-os/control.sock` in desktop mode, `$XDG_RUNTIME_DIR/herald-os/control.sock`
   in panels mode) and hands the backend its path and a per-launch token
   (`HERALD_OS_CONTROL_SOCKET`, `HERALD_OS_CONTROL_TOKEN`). The socket is readable only by the
   user (mode 0600) and refuses requests without the token. In panels mode it also serves the
   `herald-os` CLI that the compositor's hotkeys call; those commands need no token.

Voice (engines, wake word, barge-in) is described in [`VOICE.md`](VOICE.md).

## Pick up where you left off

The Overview leads with up to three threads of work the user may want to resume.

- **Electron main** (`electron/context/snapshot.ts`, `electron/ipc/context.ts`) takes a context
  snapshot on request: documents opened or changed in the last three days (the platform's recents
  plus new files on the Desktop and in Downloads), folders under the usual project roots with
  activity this week (branch, uncommitted changes and last commit for git repositories, read with
  `--no-optional-locks` so looking never rewrites the index), and the GUI apps running. Folders and
  words the user excluded are dropped before anything leaves main. `powerMonitor` turns a wake, an
  unlock or the end of a 15-minute idle stretch into a `context:returned` event when the user was
  away for at least ten minutes.
- **The renderer** (`src/store/continuity.ts`) adds recent Hermes conversations and today's
  calendar, numbers every entry (`f1`, `p1`, `c1`) and sends it with fixed instructions through the
  gateway's `llm.oneshot`: one stateless model call with no session, no tools and no transcript.
  `src/lib/continuity.ts` parses the JSON answer and keeps only items whose ids were in the
  evidence, so a card cannot point at a file the model made up. Threads are stored in prefs and
  refreshed on boot (when older than ten minutes), when the user returns, and on demand.
- **Acting stays with the user.** Continue reopens a thread's conversation (in Studio when it built
  the thread's project), its folder and its documents. The suggested next step is a button that
  sends the prompt Hermes proposed. A return only updates the cards, or posts a notification when
  another page is showing. `continuity.catchUp`, `continuity.continue` and `continuity.nextStep`
  expose the same actions to voice, the command bar and Hermes.

The feature stays off until the user turns it on, because the names of files, projects,
conversations and events go to the model provider.

## Herald Canvas

The image editor (decision record: ADR-020; using it: [the manual](manual/canvas.md)) lives in
`src/features/canvas`, with the project format in `shared/canvas` and file access in
`electron/canvas`.

- **The document** (`engine/document.ts`) is an immutable state of layers in `.comp` order with
  shared rasters, so undo swaps states; pixel edits keep their own history. `shared/canvas/comp-format.ts`
  reads and writes the manifest, holding every value to the ranges Compositor accepts.
- **The compositor** (`engine/gpu/`) draws a document, or any area of it, on WebGL2: layers, blend
  modes, folders, masks, clipping, adjustment and effect passes. The view (`gpu/view.ts`) composites
  what is on screen at the screen's resolution and drafts at half resolution on software renderers
  while things move; `engine/tiles.ts` plans the tiles that exports, previews and the AI tools render
  in, and `gpu/gl.ts` holds rasters past the GPU's largest texture in pieces.
- **Main** reads and writes projects atomically and watches them (`electron/canvas/package-io.ts`),
  converts HEIC, TIFF and RAW through the system's tools (`convert.ts`), streams PNG exports a band
  of rows at a time (`png.ts`), reads Photoshop files in parts, and downloads and verifies the
  on-device models (`model-store.ts`).
- **Workers** run what would stall the window: the ONNX models (`ai/ml-worker.ts`), PatchMatch
  (`ai/inpaint-worker.ts`) and Photoshop files (`psd/psd-worker.ts`, mapped by `psd/psd-map.ts`).
- **Hermes** edits through the `canvas.*` commands (`src/commands/canvas.ts`, `agent.ts`), which change
  the open document as one step and save it, or change the project on disk; the bridge's `canvas`
  tool maps its actions to them. In panels mode the commands run in the Hermes window and the Canvas
  window, its own process, reloads the project when it changes.

## Herald Office

Herald Docs, Sheets and Slides (decision record: ADR-021; using them:
[the manual](manual/office.md)) live in `src/features/office`, with the formats in `shared/office`
and file access in `electron/office`.

- **One frame for three apps** (`shell/OfficeWindow.tsx`): the menus (File and Edit shared, then
  the app's own) and their shortcuts in one table (`shell/commands.ts`), the document tabs, the
  status bar with the Ask Hermes bar, and the dialogs. Each app's session (`session.ts`) opens,
  saves, watches, prints and closes its documents through an adapter that reads, writes and prints
  its formats (`docs/adapter.ts` and so on); `shared/office/files.ts` says which formats each app
  opens and saves, and main builds its dialogs from it.
- **The save policy.** Before the first save over a file, the fidelity report lists what Herald
  showed differently and what the format will not keep; main copies the original into
  `office-backups` under the Herald OS data folder the first time Herald writes over it in a session
  (`electron/office/backups.ts`). Herald saves a document by itself only once the person has saved
  it, and stops when a save would lose something new. A change on disk loads by itself when there
  are no unsaved edits and is a question when there are.
- **Docs** is TipTap 3 (`docs/`): one schema (`docs/schema.ts`) for the editor, the converters, the
  print view and the document API (`docs/model.ts`), which works on a live editor or a file's JSON.
  The page view (`docs/pages/`) lays the one editor's text out on pages, with headers, footers and
  notes edited in place on them (`parts.ts`, `PartEditor.tsx`) and sections that each have a page
  setup. Comments (`comments.ts`, `CommentsPanel.tsx`) and live tables of contents (`toc.ts`,
  `toc-view.ts`) are operations of the same API, and templates (`docs/templates/`) and statistics
  (`statistics.ts`) build and read documents of the same shape. Word files go through Herald's own
  reader and the docx package (`shared/office/docx`), Markdown and plain text through
  `shared/office/doc-text.ts`, and the page spell-checks with suggestions on right-click
  (`electron/office/spelling.ts`).
- **Sheets** is Univer's open-source packages (`univer/`), one instance per open workbook, in
  Herald's palette (`palette.ts`, `theme.ts`, `univer.css`); formulas are worked out in a worker,
  `.xlsx` is read and written in another (`sheets/xlsx-worker.ts`), and `sheets/headless.ts` runs
  Univer without a window for the workbook API (`sheets/model.ts`). Herald's own features attach to
  each workbook's Univer (`sheets/features.ts`): charts drawn by ECharts in Univer's floating
  objects (`sheets/charts/`), comments and notes, named ranges and validation, each in a folder of
  its own, and the data tools and the summaries built of formulas (`sheets/tools/`).
  `shared/office/xlsx` writes charts, comments and notes into the package after ExcelJS
  (`finish.ts`), carries the parts of the opened file that Herald does not model, such as pivot
  tables, pictures and slicers (`keep/`), and keeps what only Herald reads back in a part of its own
  (`herald-part.ts`).
- **Slides** is a DOM slide editor: a deck in points with PowerPoint's model (`slides/deck.ts`),
  one view that draws a slide for the editor, the slide list, presenting and PDF export
  (`slides/view/`), TipTap for the text in boxes, shapes and table cells (`slides/editor/`), and the
  deck API (`slides/model.ts`, applied by `slides/live.ts`). The master and its layouts open as a
  deck of their own in the same editor (`slides/masters.ts`, `layouts.ts`); themes (`themes.ts`,
  custom ones in `theme-store.ts`), transitions (`transitions.ts`), groups (`groups.ts`) and
  connectors glued to shapes' sites (`sites.ts`) are parts of the deck. Presenting
  (`slides/present/`) is one state that every view's keys move: the show, the presenter view and the
  audience window, which main lets open full screen on another display
  (`electron/office/presenter.ts`). `from-document.ts` and `from-sheet.ts` lay out slides from a
  Docs document and tables from a Sheets range by rule. PowerPoint files are written by PptxGenJS
  with a finishing pass and read by Herald's own DrawingML reader (`slides/pptx/`), and each file
  Herald saves carries its deck for exact reopening (`slides/pptx/herald-part.ts`).
- **Hermes** works through the `docs.*`, `sheets.*`, `slides.*` and `office.list` commands
  (`src/commands/docs.ts`, `docs-depth.ts`, `sheets.ts`, `sheets-depth.ts`, `slides.ts`,
  `office.ts`), which find their document in `agent.ts`: open in this window, the change is one
  transaction (Docs), one Univer undo group (Sheets) or one step of the deck's history (Slides), so
  it is one step to undo and saves the way the person's edits do; a file that is not open is
  changed and written back only when Herald keeps everything in it. In panels mode each Office app
  is its own window, and main relays a command from the Hermes window to the window that has the
  document open (`officeRun`). Each window's report carries its documents' selections, which
  `office.list`, the `os_ui` state and spoken requests pass on to Hermes. The Ask Hermes bar and the
  inline actions (`hermes/`) build each request from the live document and follow it in the
  document's own Hermes session; a request marks the text it is about (`docs/marked.ts`), so the
  answer lands there even after the person clicks elsewhere, and Review with Hermes leaves its
  comments in one `docs.addComments` call (`hermes/docs-review.ts`). Dictation types into the page,
  the cell or the text box in front (`typing.ts`).
- **Main** (`electron/office/`): `ipc.ts` shows the open and save dialogs, reads and writes files
  whole and atomically (inside the home folder, up to 512 MB), keeps what each window has open for
  Hermes and relays `officeRun`. `file-watch.ts` watches each open file's folder, with a poll as a
  safety net, and `watches.ts` ends each watch with its window. `print.ts` gives Export as PDF and
  Print one path: the window's print view is loaded, scripts off, in a hidden window that prints it
  to PDF bytes or through the system's print dialog, so paper shows what the PDF does.
  `templates.ts` keeps the templates the person saves, in `office-templates` under the Herald OS
  data folder. `convert.ts` can convert OpenDocument files through headless LibreOffice when it is
  installed; no window converts through it yet, so `.odt`, `.ods` and `.odp` stay off in
  `shared/office/files.ts`.

## Herald desktop pet

Hermes's petdex pet on the desktop (decision record: ADR-022; using it:
[the manual](manual/pet.md)) lives in `src/features/pet`, with its contract in `shared/pet.ts` and
main's side in `electron/pet`. Which pet, its size and on or off are Hermes's `display.pet`, changed
only through Hermes's gateway (`pet.select`, `pet.scale`, `pet.disable`); where the pet stands and
what it reacts to are Herald's prefs.

- **Main** (`electron/pet/`): `config.ts` reads `display.pet` from Hermes's `config.yaml` and the
  pets in its `pets` folder the way Hermes resolves them, and logs nothing it read. `ipc.ts` checks
  their file stats every 4 seconds, sends what it found to every window, hands out the shown pet's
  spritesheet (from that pet's own folder, up to 32 MB), and in panels mode relays between the
  Hermes window and the pet's window: snapshots only from the first, actions and placements only
  from the second. `window.ts` is the pet's window in panels mode, sized through its size hints,
  moved through niri's IPC, and handing the keys back whenever niri focuses it.
- **The brain** (`brain.ts`) runs in the Hermes window, because gateway events and approvals reach
  only the window that owns a session. It folds Hermes's sessions, the pending approval, voice, the
  command log, system status and the notes into one `PetSnapshot`, and acts on what the pet sends
  back (an answer to the approval, a question, a dropped file, a note opened). `transport.ts`
  carries both ways: an atom in desktop mode, main's relay in panels mode. `chat.ts` runs the
  pet's own background session.
- **The pet** is drawn by `PetOverlay.tsx` over the desktop in desktop mode, or by `PetSurface.tsx`
  in a window of its own (`Herald OS · Pet`, with a window rule in the repository's
  `linux/niri/config.kdl`) in panels mode. Both render `PetView.tsx` (the sprite on a canvas, the
  approval card from `src/shell/RequestHost.tsx`, the chat bubble, the drop choices) and share the
  pure modules: `sprite.ts` (the spritesheet's rows), `state.ts` (what it shows), `stage.ts` and
  `placement.ts` (where it stands), `hide.ts` (when it hides), `drops.ts`, `moods.ts` and
  `chat-model.ts`.
- **Commands and settings.** The `pet.*` commands (`src/commands/pet.ts`), Settings > Pet
  (`PetSection.tsx`) and the menu bar's paw (`PetMenuBarToggle.tsx`) change the pet through
  `actions.ts`: Hermes's settings through its gateway, Herald's own in prefs.

## System bridge

`plugins/herald-os-bridge` is a regular out-of-tree Hermes plugin. It registers a narrow toolset
(`herald_os`) that exposes the host machine through explicit, permission-tiered tools. Execution
happens on the backend host behind a `HostAdapter` abstraction (`darwin` and `linux` implemented;
`windows` is a stub). Sensitive operations route through upstream's own approval gate
(`tools.approval.request_tool_approval`) so the shell renders one approval card for everything.
See [`SYSTEM-BRIDGE.md`](SYSTEM-BRIDGE.md).

## Upstream compatibility

- The Python runtime is never forked: it is whatever `hermes update` installs.
- Only `apps/shared/src` is compiled into Herald OS, from a pinned snapshot
  (`upstream/UPSTREAM.lock`, fetched by `scripts/sync-upstream.sh`).
- The plugin uses only public plugin APIs (`register`, `ctx.register_tool`, `ctx.register_skill`,
  `tools.approval.request_tool_approval`), and reads each turn's session source the way Hermes's own
  tools do (`gateway.session_context.get_session_env`) to keep its tools to Herald OS sessions.
- Anything Herald OS needs from core that does not exist yet (for example a generic plugin
  server-request hook for client-side execution) is proposed upstream rather than patched locally.