# iOS companion architecture The iOS app is a thin, native client for the OpenMausBot instance running on your Mac. The Mac remains the only machine that persists agent processes, credentials, SQLite data, transcripts, and computers. The iPhone trusts a Mac by scanning the QR code shown in desktop **Settings → Phone & devices**; it does not need an OpenMausBot account of its own. ## Use My Cloud on your phone With MausBot Cloud, pair the phone with your Cloud instead of your computer, and it keeps working while the computer is off. 1. In the desktop app, open the menu under your name and choose **Get the phone app**: the App Store for iPhone, or the APK for Android. Scan its code with the phone's camera. 2. In the same menu, choose **Connect your phone · to your Cloud (always on)**. (**Settings → MausBot Cloud → Use My Cloud on your phone** does the same.) The app opens your Cloud in its window, at the Cloud's phone pairing. 3. Choose **Create pairing code** and scan the QR code with the phone app. On your computer, **Connect your phone** offers your Cloud first and this computer second, once your plan is paid and your Cloud is Ready. Until the Cloud is Ready it offers only *to this computer*, with a note that the Cloud will show there. With the Cloud open in the app window, the entry reads *to your Cloud* and pairs with it directly. On another server it pairs with that server, and it is shown only to a session allowed to make pairing codes there. ## Current status The first version includes: - QR-first pairing from desktop **Settings → Phone & devices**, with the computer name confirmed on the iPhone before it connects. - Hosted HTTPS for the default QR after the desktop owner enables it, with dedicated Tailscale and trusted-local QR routes only after an explicit choice. - Nearby computers, a manual address, and a six-digit code under **Other ways to connect** when the QR path is unavailable. - Secure per-device trust, device listing, and revocation. - Multiple saved computers with a one-tap switcher. Each pairing has its own Keychain credential, and only the selected computer is connected at a time. - Bot and room lists, paged transcripts, sending, interruption, and unread state. - Approvals and questions, including narrow “always allow” grants. - Resumable SSE, streamed reply text, reconnect hydration, and an opt-in live Boat computer view. The loopback-only VPS SSH viewer remains desktop-only. - On-demand stills of a bot's Local VM, idle or working, for a phone the computer owner has allowed computer access. - Markdown rendering and Keychain storage for the phone's pairing trust. - Secure completion of supported credential-request cards using iOS Password AutoFill and QR-pinned HPKE encryption. Apple Passwords/iCloud Keychain is sufficient; 1Password, Bitwarden, and other AutoFill providers are optional. Alerts work while the app is open or for the short period it remains connected after moving to the background. Once iOS suspends or closes the app, new alerts cannot arrive. Closed-app push delivery, voice, and App Store release automation are not part of this version. The optional hosted transport connects to the user's own computer; it is not a cloud transcript store and cannot wake a terminated iOS app. The Mac must be running OpenMausBot and must not be asleep. Desktop **Settings → Phone & devices** offers an off-by-default **Keep this computer awake** switch that prevents system sleep while device access is on; the display may still turn off. A sleeping or powered-off computer cannot receive phone requests or run its local routines, including through the optional hosted transport. ## Markdown Bot and room replies are rendered. Messages the person typed stay literal. Rendered blocks are paragraphs, headings, lists, task lists, quotes, thematic rules, fenced code, and top-level tables indented by at most three spaces. A table inside a list item, a blockquote, or a fence stays text or code, as does a row indented by four spaces or a tab. Inline emphasis, code, strikethrough, and links use Foundation attributed text. A table scrolls horizontally inside the bubble. A task item shows a checkbox, and an ordered task keeps its number beside that checkbox. The in-app spoken form reads each table row and each task's words. It skips a delimiter-only paragraph and the checkbox token. Closed-app voice is not part of this version. ## Runtime architecture ```text iPhone (pairing trust in Keychain; credential plaintext only while editing) │ │ │ trusted LAN/Tailscale │ optional hosted HTTPS ▼ ▼ sidecar :8810 Cloudflare Tunnel (outbound connector) │ ▼ guardian gateway 127.0.0.1:8812 │ exact per-launch socket/pipe └──────────────┐ ▼ companion sidecar (pairing auth, default-deny allowlist, response/SSE scrubbing, authenticated endpoint refresh; credential ciphertext only) │ loopback only ▼ OpenMausBot harness :8799 HTTP API + event stream agent processes and approvals │ private Electron utility-process channel ▼ OS-encrypted desktop credential store │ ▼ SQLite message store + local configuration ``` There are three deliberately separate trust surfaces: | Surface | Bind | Purpose | |---|---|---| | Harness | `127.0.0.1:8799` | Existing app API; remains loopback-only | | Companion | `0.0.0.0:8810` | Paired native devices; authenticated and allowlisted | | Companion control | `127.0.0.1:8811` | Start pairing, cancel pairing, list devices, revoke | | Hosted gateway | `127.0.0.1:8812` | Guardian-owned route to one exact sidecar generation | The desktop app owns the sidecar lifecycle through `electron/companion.mjs`. The renderer only receives narrow IPC operations; it does not fetch the control port directly. Every control route needs a token the app makes for each sidecar launch and hands over Electron's private port, so a page served from `127.0.0.1:8811` itself, as a bot's browser could open it, cannot start pairing or change a device's access. A standalone sidecar prints its pairing page link with that token. ## SQLite compatibility SQLite does not move onto the phone. It is an implementation detail behind the harness API: - `server/message-db.ts` and `server/store.ts` persist and page transcripts. - The phone asks for `GET /api/bots?messages=50` and `GET /api/threads/:threadId/messages?before=…&limit=50`. - SQLite ordering and cursors are therefore tested at the server boundary, while the Swift package tests decoding and prepend/deduplication using responses captured through the real sidecar. - A storage migration may change the bytes on disk without changing the app. If an API payload changes, regenerate the fixtures with `node scripts/capture-companion-fixtures.mjs` and review the diff. The sidecar keeps its device registry in `~/.openmausbot/devices.json`. That is security state owned by the network boundary, not transcript data, so it does not belong in the message database. ## Connectivity ### Same Wi-Fi The QR code is still the primary path on the same Wi-Fi. If it is unavailable, the user can open **Other ways to connect** and choose a nearby computer or enter the address shown in desktop Settings → Phone & devices. Nearby discovery does not run until the user opens that fallback. Nearby discovery uses Bonjour and direct LAN traffic. Use it only on a network you trust. Choosing a nearby computer or manually entering a LAN address is therefore an explicit fallback. Once the app is using a hosted or Tailscale route, automatic reconnection stays within those protected transports. Moving back to direct LAN requires choosing that computer or address again. ### Tailscale Tailscale is an optional route away from home and on Wi-Fi networks that isolate clients. In desktop **Settings → Phone & devices**, open **More ways in** and turn **Tailscale** on: until you do, the app never runs Tailscale. Install or open Tailscale on both devices, sign in to the same tailnet and leave MagicDNS enabled, then choose **Pair a device**. OpenMausBot places the computer's tailnet name in the QR beside its other ways in; it never silently replaces the default route. Manual entry remains available as a fallback. With the switch on, the link also answers on this computer's tailnet address (found by Tailscale's own interface, never by a 100.64/10 address alone), and the app asks Tailscale for the computer's MagicDNS name once: when you turn the switch on, and again only when you choose **Check again**. It runs only Tailscale's command-line tool (`tailscale status --json`), never the macOS app's executable: with only the app installed it asks you to choose **Install CLI** in the Tailscale app's settings. It never opens the Tailscale app or signs it in. The URL is `http://..ts.net:8810`, encrypted and authenticated by WireGuard inside the tailnet. Use the MagicDNS name rather than the `100.64.0.0/10` address: App Transport Security exceptions are domain-based, and `ios/project.yml` narrowly allows insecure HTTP for `ts.net` subdomains. Turning the switch off stops answering on the tailnet address; it never changes anything in your Tailscale. Each way in (Same Wi-Fi, Anywhere through MausBot, Tailscale, the HTTPS address) is its own switch and stays off until you turn it on. While nothing is paired and no code is open, the phone link sleeps: nothing listens and there is no relay connection. A desktop that is only a client of another computer starts none of it. Tailscale is optional. The direct path does not use an OpenMausBot-operated relay or create a cloud copy of local transcript data. ### Optional hosted HTTPS In desktop **Settings → Phone & devices**, **Use your phone anywhere** accepts a passwordless email code and provisions one HTTPS address for that computer. This desktop sign-in is only for hosted HTTPS. The iPhone never signs in; it trusts the computer through the same pairing QR. Nearby, manual, and Tailscale connections continue to work without an account. The desktop runs an outbound connector to Cloudflare, so no inbound router configuration or Tailscale installation is required. The hosted address is included in a pairing invitation only after it is ready. The default setup waits for that HTTPS address instead of silently substituting Tailscale; Tailscale pairing remains an explicit choice in its own optional card. The address is held only while something needs it. The desktop creates it when you sign in, choose Retry, or start pairing (a few seconds under the "Creating secure remote access" spinner), keeps it while any phone or desktop is paired, and gives it back one to four hours after Phone & devices shows nothing paired and no pairing open. The next pairing gets the same address back. If no HTTPS address is free, the panel says when to try again, and Wi-Fi or Tailscale pairing still works. Cloudflare terminates and proxies the encrypted connection to the connector. The OpenMausBot control plane stores account and installation metadata plus opaque tunnel/DNS identifiers in D1, but not bots, transcripts, approvals, screen frames, pairing tokens, or connector tokens. See `docs/ios-privacy.md` for data and deletion details. The connector does not point at the reusable LAN port. Electron launches one private sidecar socket (or Windows pipe) and a guardian that owns both the fixed loopback gateway and `cloudflared`. If Electron or that sidecar exits, the guardian first makes forwarding unavailable, confirms the connector is dead, and only then releases the gateway. Another process that later binds a local port cannot inherit the public route. The hop from this computer to Cloudflare runs on the home upload, uncompressed, so the sidecar gzips JSON replies of 1 KB or more on that private socket when the phone accepts gzip. The event stream, file downloads and the LAN port are sent as they always were. ## Pairing and device security 1. On the Mac, open **Settings → Phone & devices** and choose **Pair a phone**. The app starts device access as part of setup. 2. On the iPhone, choose **Connect my computer** and scan the QR. 3. Confirm the computer name and the displayed transport — **HTTPS connection**, **Tailscale connection**, or **Trusted local connection**. The phone stores its trust securely in Keychain; no iPhone account is required. A camera QR also pins that computer's public credential-encryption key. Manual and old pairings can still chat, but must scan a fresh QR before entering a key on the phone. 4. If scanning is unavailable, open **Other ways to connect** for a nearby computer, manual address, or six-digit code. 5. Revoking the phone on the Mac removes its access and lets it pair again. To add another Mac, open **Settings → Computers → Connect another computer** on the iPhone and scan that Mac's QR. The existing computer stays usable if the new pairing fails or is cancelled. Switching computers replaces the live event stream and in-memory chat state, but keeps every saved pairing; removing one computer deletes only that computer's Keychain credential from the phone. An app upgrade migrates the previous single saved pairing automatically. The Mac must remain awake with OpenMausBot running for chats, approvals, and routines to work, including through hosted HTTPS or Tailscale. After pairing, the phone periodically reads the authenticated, sidecar-owned `GET /api/companion/endpoints` snapshot. This lets an existing phone learn a new hosted address—or its withdrawal—without another pairing ceremony. The route never reaches the harness and returns only the computer name plus a bounded list of connection origins. An OpenMausBot account is not required for nearby, manual, or Tailscale connections. Only the desktop owner signs in when enabling the optional hosted HTTPS route; the iPhone always uses the same QR trust flow. ### Secure credential entry This is Password AutoFill, not a password-vault integration. A native `SecureField` marked as a password lets the user explicitly choose Apple Passwords or any enabled third-party AutoFill provider. OpenMausMobile does not enumerate a vault, receive a provider token, or save the entered value in its own Keychain. The packaged desktop creates one stable P-256 recipient key pair and keeps the private JWK inside its operating-system-encrypted credential document. Only the 65-byte uncompressed public point is added to a camera QR. The phone validates and pins that point with the connection; manual and older pairings remain fully usable for chat but cannot submit a credential until they scan a fresh QR. Credential submission is enabled only while the phone is using hosted HTTPS or a Tailscale route. Local and Bonjour chat pairing still work as before, but their cleartext HTTP transport would expose the reusable device token to anyone who could observe that Wi-Fi network, so the app directs the user to finish the credential request on the computer instead. Each submission uses RFC 9180 base-mode HPKE with P-256/HKDF-SHA256/AES-GCM-256 and authenticates this exact newline-separated context: ```text openmausbot-phone-credential-v1 ``` The companion replaces any caller-supplied device header with the identity of the bearer token it authenticated. The harness accepts only an allowlisted target on the exact pending card, opens the ciphertext through its private Electron channel, and waits for the encrypted desktop store and live config to commit before it marks the card complete. Replays are idempotent; moving a ciphertext to another device, bot, task, card, target, or computer fails authentication. The native field is cleared immediately after local encryption. If the response is interrupted, the app retains only that exact ciphertext in memory and reuses it for Retry, so HPKE randomness cannot turn a network retry into a second credential write. Nothing is persisted on the phone. Development servers deliberately have no recipient key and fail closed instead of advertising an unusable secure-entry route. The device-facing socket rejects browser `Origin` headers before reading a token. Its route policy in `companion/src/routes.ts` is default-deny: a new harness route remains unreachable until it is deliberately added. Allowed in the first release: - Read the fleet, rooms, instances, configuration status, and transcripts. - Fetch settled screen images and opt into live screen frames. - Request a fresh interactive cloud-desktop viewer only when the computer owner has enabled that capability for this specific paired phone. - With the same capability, fetch a still of a bot's Local VM (`POST /api/bots/:id/local-computer/screenshot`). The app always passes the conversation's `threadId`, so the harness answers 409 for a conversation that is not on the Local VM. This reads a picture; it cannot start, stop, remove the VM. Like the desktop panel's preview, each capture counts as use of the VM, so an open computer view keeps it from being reclaimed as idle; leaving the view or backgrounding the app stops that. - With the same capability, drive the Local VM: take the bot's computer under a device control lease (`POST /api/bots/:id/computer/control`), then `POST /api/bots/:id/local-computer/join?controlLeaseId=…`. The harness answers that only for a loopback caller, only for a conversation on the Local VM, and only to the lease that holds the computer. The sidecar rewrites the loopback noVNC address into its device-scoped relay path, then asks the harness every few seconds whether that lease still holds (`action: "check"`, read-only) and closes the relay when it does not. The phone speaks RFB over the relay. Hand back closes the viewer and releases the lease, and so does backgrounding the app. - A phone paired with the server directly (`openmausbot serve` behind Tailscale Serve or a tunnel, no sidecar) drives the Local VM the same way, but the join answers it with the server's own authenticated desktop proxy (`/api/desktop-viewer/local//websockify`), bound to its control lease, and the VNC password; never the loopback address. The proxy re-checks the lease and the session every few seconds and closes the socket when either lapses; hand back closes it at once. Computer access is the pairing's scope: Full access (`openmausbot pair`) may, chat-only (`--client`) is answered 403 and the app shows computer access as off. - Phone control requires per-bot Local VMs. Shared and pool modes are refused on both connection paths until a viewer can reserve its desktop against other bots; a control hold on one bot alone cannot do that. Stills remain available in every mode. - Send messages, interrupt bots, answer approvals/questions, and mark chats read. - Create a basic bot. - Submit a supported pending credential card as an RFC 9180 HPKE envelope bound to the authenticated device, bot, task, message, request, and target. Only the paired packaged desktop's embedded server and Electron private process channel receive the private key or plaintext; the sidecar, hosted relay, chat transcript, and SQLite store see ciphertext or status only. The write surface uses purpose-built `read` and `always-allow` endpoints. The general bot and room `PATCH` endpoints are not reachable through the sidecar. An always-allow request succeeds only when its server-issued key is still on a pending approval for that bot, so possession of a device token is not enough to invent a broad execution grant. Intentionally refused: - Reading credentials, arbitrary credential targets, and general provider configuration. The only credential write is the exact pending-card envelope above, and it feeds the existing desktop OS-encrypted save path. - Pairing, device revocation, or companion lifecycle control. - Local VM lifecycle, webhooks, connectors, routines, team import/export, and internal peer-agent routes. - Cloud computer provisioning, sleep, shell execution, and screenshot APIs. The phone receives only the fresh `join` viewer URL, never the provider key. - New harness routes that have not been reviewed for device access. ## Stream and state model `CompanionCore` contains the wire models, client, raw-byte SSE parser, and pure state fold. The SwiftUI target owns lifecycle and presentation only. On connection, the server sends a `hello` frame containing a cursor and whether the requested gap was replayed. The client: 1. resumes from its last `:` cursor; 2. folds replayed and live frames when the gap is available; 3. hydrates the newest page of each visible conversation when it is not; and 4. paginates older transcript pages on demand. Unknown message and frame kinds degrade safely instead of failing an entire response, and one malformed fleet record does not hide every healthy chat. Screen frames are off by default and enabled only while a computer view is visible. Backgrounding keeps the stream for only the short grace period iOS allows, then closes it; foregrounding reconnects from the saved cursor. A hello cursor is committed only after a cold hydration succeeds; replayed streams advance it one folded frame at a time, so a disconnect during recovery cannot skip the remaining gap. ## Phone apps as bot tools The phone registers the apps it allows with `PUT /api/phones/self`. Behind the sidecar it is identified by the companion header; on a Cloud home it uses its own session. It long-polls `GET /api/phones/self/jobs` (20 s), leases a job with `POST /api/phones/self/jobs/:id/lease`, and answers with `POST /api/phones/self/jobs/:id/result`, sending `{ ok: true, result }` or `{ ok: false, error, code }`. Results over 256 KB are refused as `too_large`. The SSE frame `{ kind: "device.job", phone, id }` tells an open app to poll now. Bots see these apps as `PHONE_*` tools through the `device` MCP kind. Reads run silently and writes show up as activity rows. A send opens a sheet on the phone and goes only when the person taps Send there; OpenMausBot adds no card of its own. A turn that came from the phone does phone things on the phone; a turn at the computer works the usual way and changes the phone only when the person names it. Only the owner's phone is reachable, so team Clouds never expose it. The contract fixture `server/testing/fixtures/phone-jobs.json` is what the phone apps copy into their own test fixtures. ## A bot's MCP servers The bot panel's Apps page reads `GET /api/bots/:id/mcp-servers`. It answers `{ servers: [{ name, enabled, on, offFor? }] }` for every bot's servers and the bot's own, in the order MCP server settings list them. `enabled` is the switch in those settings. `on` is the bot's switch: the server is enabled and in the bot's `mcpServers` list (no list means every enabled server). `offFor: true` means the server doesn't start for this bot. A bot that doesn't exist is a 404. Like the bot's overview it needs the admin scope, so a Team Cloud member's session is refused whichever bot it names. `GET /api/mcp/servers` stays closed to phones, since it carries each server's command, arguments and address. ## Source layout ```text companion/ src/routes.ts device-facing allowlist src/devices.ts pairing and token registry src/proxy.ts HTTP/SSE forwarding and scrubbing src/origin.ts private per-launch hosted origin listener src/control.ts loopback-only control plane src/mdns.ts Bonjour advertisement ``` ## Verification contract The merge gate for this feature is: ```sh pnpm typecheck pnpm test pnpm build:companion pnpm check:electron ``` The simulator validates compilation, launch, layout, manual address parsing, and failure states. Bonjour, Local Network permission, Tailscale routing, Keychain behavior across a reboot, and approval delivery still require a real iPhone pass. ## Follow-on releases Keep the foundational merge separate from capabilities that widen security or distribution scope: 1. **Foundation:** sidecar, desktop controls, Swift core/app, pairing, chat, approvals, reconnect, simulator and contract CI. 2. **Desktop conversation parity:** task create/switch/rename/delete, SQLite search with exact-message landing, transcript export/share, reactions, and edit/version controls. Archived or hidden chat management remains desktop-only. 3. **Notifications:** native permission, live/replayed alerts, time-sensitive approvals, badges, and a brief background grace period are in the app. Closed-app delivery still requires project-owned APNs credentials and a hosted relay; Tailscale cannot wake a terminated iOS process. 4. **Distribution:** signing, bundle ownership, privacy declarations, TestFlight, and App Store review material. Swift tests and an unsigned simulator build already run in the repository CI. 5. **Optional expansion:** voice/call mode or Local VM/host-computer interaction. Each requires its own threat-model review.