# Desktop client and video

Status as of 2026-11-07, branch `claude/desktop-client-380250`. The client was built against a scripted fake gateway, then run against the real gateway (`internal/gateway`) on `internal/demo`'s drawn phones with scripted agents; see [Against the real gateway](#against-the-real-gateway). The third pass added unlocking with a saved PIN, pairing over Wi-Fi, the companion app, background displays, recordings, export, remote agents and keeping the Mac awake; see [Round three](#round-three). Nothing here touched a real phone.

## What is built

| Path | What it is |
| --- | --- |
| `ui/` | The desktop client on MyGo v0.7.4 native UI. Entry point `ui.Run(api gateway.API, opts ui.Options) error`. It uses nothing of the gateway but `gateway.API`. |
| `ui/fakeapi/` | A scripted `gateway.API`: the prototype's scenarios with ffmpeg test streams for the mirrors. It also has the gateway's extra methods: step screenshots, agents, read-only, deleting history, and round three's pairing, companion app, background displays, drawn recordings, retention and export. |
| `ui/demogw/` | The real gateway on `internal/demo`'s phones, with their drawn screens streamed through `video.FrameStream`, and scripted agents that call it as an MCP client would. For `pinky-ui-demo -gateway` and the end-to-end tests. |
| `internal/video/` | H.264 and HEVC decoding through FFmpeg 9 (libavcodec 63, libavutil 61, libswscale 10) called with purego, VideoToolbox hardware decoding with software fallback, Annex B helpers, a live test stream from the `ffmpeg` command, and a player that decodes on its own goroutine and keeps the newest picture. |
| `cmd/pinky-ui-demo/` | Runs the client against the fake (`go run ./cmd/pinky-ui-demo`) or the real gateway (`-gateway`), a `bench` subcommand for the video numbers, an `e2e` build tag that links MyGo's automation plugin, and `mygo.json` to package it. |
| `assets/` | The brand files (SVG sources, `Pinky.icns`, `pinky.ico`, Linux icons, menu bar template images) and `assets.go`, which embeds the four the client draws. `BRAND.md` lists them. |

Screens, after prototype v18 (`*.dc.html`):

- **Sidebar.** Logo and name in the header next to the window controls; the phones with a status dot and a plain status line; "All phones" once two or more phones are known; Add phone; Activity, Settings and Stop all (with its shortcut) at the bottom.
- **Phone.** The live mirror in a phone bezel sized to the window, with the agent's overlay (the target outlined, the touch point with two ripples while a step is in progress, the way of a swipe, drag or pinch that waits for the person), the lock (screen dimmed, "Pixel 9 is locked" pill, both gone while the pointer is on the mirror), the "You're in control" banner with Hand back above the phone, the gray ring while the person is in control, and the keys below (Back, Home, Recents on Android; Home on iPhone) and ⋯ (Take screenshot, Rotate and Work in the background work; Record is shown disabled). While an agent works on a background display, the mirror shows that display, with a switch to the phone's own screen.
- **All phones.** Live tiles converted at tile size (an idle iPhone's tile is a still, see "Stills of idle iPhones" below), amber edge on phones that need the person, Take over and Stop on hover, the offline tile, and the tile of a phone that needs setup (an amber alert where the screen would be: it has no picture before it is open); a click selects the phone.
- **A phone that needs setup** (`gateway.DeviceNeedsSetup`: connected, but its driver lists it as `unauthorized` or `needs_setup`, waiting for a step such as Allow USB debugging, Trust, Developer Mode or the developer disk image). The sidebar, its tile and the menu bar flyout say "Needs setup" with an amber dot. Its page says "<phone> needs setup" and the step in the words its checkup gives the step (`setupStep`, the "authorized" item: `setupWords` for an iPhone, Allow USB debugging or the computer's USB access for Android), the driver's own words under them only where the client's leave something out (which USB driver to install, why the disk image did not mount), then See what's wrong (the checkup) and, for an iPhone, Show the steps (Add phone › iPhone) or How to install it (the helper). On a Mac with Xcode an iPhone whose helper is missing or ran out gets Install Pinky for iPhone (or Install again) instead, with Checkup as a link, and the page shows the install's progress while it runs ([ios.md](ios.md#from-pinkys-window)). Unplugged, it is offline like any phone, without the step, and one Pinky never opened leaves the list two minutes later. An agent's task on it says "Phone needs setup" (amber, no Take over), and a handoff the agent opens for the step says the agent goes on by itself once the phone is ready, with Decline and Done instead of Take over (`handoffControls`; the gateway ends it when the phone is open). An iPhone that waits for its passcode before the WebDriverAgent helper may control it (`driver.SetupPasscode`) says so in words on its page, in its checkup and in Add phone › iPhone, and so does an iPhone below iOS 27 set to connect through CoreDevice (`driver.SetupConnection`), with Choose the connection, which opens Settings › Phones (`chooseConnection`). An iPhone that iOS 27 will not mirror while an app on it uses the camera or microphone (`driver.SetupMediaInUse`, 2026-11-11) reads "Camera or microphone in use" in the sidebar, tile and flyout; its page takes that as its title, says what to do (the green or orange dot, Control Center, end the call or close the app) and, by a spinner, that Pinky connects by itself, without See what's wrong or Show the steps; its checkup and Add phone › iPhone say the same (`TestIPhoneMediaInUseStep`).
- **Right panel.** Needs you (approval cards with Deny, Allow and More options: Allow for this task, Always allow *kind* in *app*, Take over, Deny with a note; unlock cards with "Unlock with Touch ID" once the phone's PIN is saved, or the PIN form, or why it cannot be unlocked from here; handoff cards, where the agent asked the person to do something, with Done, Take over, Decline and Add a note; step cards (`gateway.NeedSetup`), without buttons, while a task waits for the person to enter an iPhone's passcode on the phone before the WebDriverAgent helper starts, hidden behind the agent's handoff card for that phone and opening the phone from the menu bar and notifications, like unlock cards; first-connection cards with what the agent may do, Deny and Approve), then Tasks: the selected phone's task expanded (status, Take over, Stop, title, agent · phone · time, steps with their marks, a click on a step with a screenshot opens it under the step; in Developer mode a Details switch that shows the tool calls), the other phones' tasks one row each. On an Android phone without the companion app, an Install card above Tasks.
- **Approval panel.** A MyGo panel window (`WindowOptions.Panel`) shown with `ShowInactive`, so it never takes the keyboard. It shows while approvals wait and the main window is not in front, at the top right of the display the pointer is on: time-left bar, "1 of 2" paging, "Denies in 42 s", the eyebrow, title and detail, the screenshot cropped around the target with the target outlined, Deny, Allow and More options, the note editor, and a short "Allowed once" / "Denied" confirmation. After the person clicks it, Return allows and Esc denies.
- **Menu bar.** A template icon in the macOS menu bar (the color icon in the Windows and Linux trays) with the number of things waiting as its title; a click opens a flyout (Needs you with Review, the phones, Open Pinky, Stop all ⌥⌘.); right click opens a menu. Linux tray icons report no clicks, so there the menu holds "Show Phones…".
- **Global shortcut.** `CmdOrCtrl+Alt+.` (⌥⌘. on macOS, Ctrl+Alt+. elsewhere) runs Stop all from anywhere.
- **Notifications.** A system notification for each new need while the main window is not in front; a click opens the approval panel or the phone. macOS shows notifications of packaged apps only, so `go run` skips them.
- **Activity.** Finished tasks by day with app letter icon, outcome dot (Done, Stopped, Denied) and approval count; the detail with summary, steps with their screenshots as thumbnails (a click opens the screenshot; the person's steps with "Answered in 14 s"); Watch recording (the task's screen recording with a mark per step to jump to) or, without one, Watch replay (the steps one at a time with their screenshots, Play, Previous and Next, on the step timeline); Export… for one task and Export all, as an HTML report and JSON in a folder on the Desktop. Search goes to `History` as the person types.
- **Settings.** Phones (each phone's saved PIN, Work in the background, its companion app), Agents (the gateway's agents with what each may do and Change, first-connection requests, and Remote agents: HTTP, the address, URL, token and the `claude mcp add` command), Safety (the three presets, blocked apps with add and remove, Always allowed with a × that makes a rule ask again, Read-only mode, Unlock from this computer, Hide codes from agents, and Rules: the rules file's rules to add, change, move and remove through `Rules`/`SetRules`, a confirmation before removing or changing one that blocks, and the built-in ones read-only. `SetRules` gets the rules a change was made from, and refuses it (`gateway.ErrRulesChanged`) after a hand edit of `rules.yaml`; the list then shows the file's rules, and the form stays open on its rule. While `FileProblems` lists a rules file edited by hand that the gateway cannot use, a note says what is wrong, here and under Advanced › Rules file, and the controls that would change that file are off), Privacy (Keep activity for, Record the screen, Delete all activity) and Advanced (Developer mode, backends, the video decoder in use, the rules file).
- **Add phone.** Which phone → Android: "Looking for your phone…", "Found Pixel 8a, but USB debugging is off" with the three steps and Try again, "Found Pixel 8a" and Done; Pair over Wi-Fi by QR code or with a pairing code; iPhone with the six-digit code, and the older-iPhone choices. With no phone connected the client opens straight into it.
- **Closing the window** hides it; the gateway keeps running in the menu bar. Cmd+W hides too.

## How to run

```sh
go run ./cmd/pinky-ui-demo                       # one phone: Pixel 9 reorders lunch, asks to pay
go run ./cmd/pinky-ui-demo -scenario three       # and an iPhone (HEVC stream) and an offline Galaxy
go run ./cmd/pinky-ui-demo -scenario locked      # the phone is locked; unlock from the card
go run ./cmd/pinky-ui-demo -scenario takeover    # you are in control
go run ./cmd/pinky-ui-demo -scenario empty       # no phones: opens Add phone
go run ./cmd/pinky-ui-demo -theme dark -dev -approval-timeout 10m -speed 2
go run ./cmd/pinky-ui-demo -scenario three -page settings   # or activity, add-phone
go run ./cmd/pinky-ui-demo -scenario many        # eight phones: All phones and the sidebar scroll
go run ./cmd/pinky-ui-demo -scenario handoff     # the agent asks you to sign in to Dinely
go run ./cmd/pinky-ui-demo -scenario agent       # Codex connects for the first time
go run ./cmd/pinky-ui-demo -scenario companion   # Pixel 9 lacks the companion app: Install
go run ./cmd/pinky-ui-demo -scenario background  # the agent works on a background display
go run ./cmd/pinky-ui-demo -gateway              # the real gateway on internal/demo's phones
go run ./cmd/pinky-ui-demo -gateway -hidden      # starts in the menu bar; the window opens on the first request
go run ./cmd/pinky-ui-demo -gateway -video=false # demo phones without video, as pinky --fake has them
```

With `-gateway`, Claude Code connects to Pixel 9 (fake) two seconds after start, which the gateway asks the person to allow, then opens Dinely, reorders lunch, adds a note and asks to pay. Touch the mirror to take over; it waits with `phone_handoff` until Hand back. The Demo menu starts more agents (a sign-in handoff on the iPhone, Codex for another first connection) and locks or unplugs the iPhone. History and screenshots go to a temporary folder deleted on quit.

Needs Homebrew's FFmpeg 9 (`brew install ffmpeg`): the `ffmpeg` command makes the test streams and its libraries decode them. The **Demo** menu switches scenarios, restarts the one playing (⇧⌘R), plugs in a new phone for Add phone and has the fake phone scan the pairing QR code (it does by itself after 6 s). PINs saved in the demo stay in memory; its agent token is made up. Approvals deny after 60 s, as designed; `-approval-timeout` lengthens that for a slow walk-through.

A walk-through that covers the brief: start with `-scenario three`; watch the Pixel's steps and ripples; Allow (or More options › Deny with a note) on "Pay $12.80?"; click All phones and a tile; Take over on the iPhone, Hand back; hide the window (⌘W) to see the approval panel; click the menu bar icon; Demo › Locked phone and Unlock with Touch ID; Demo › One phone to see the sidebar without "All phones".

Other commands:

```sh
go test ./internal/video ./ui/...                # decoding, the fake, and the views in ui.Tester
go run ./cmd/pinky-ui-demo bench                 # video CPU and latency (below)
go build -tags e2e -o /tmp/pinky-ui-demo ./cmd/pinky-ui-demo && \
  (cd /tmp && MYGO_AUTOMATION=pinky.sock ./pinky-ui-demo)   # drive it over MyGo's automation socket
go tool mygo build cmd/pinky-ui-demo             # Pinky Demo.app and a .dmg in build/pinky-ui-demo
GOOS=windows go build ./... && GOOS=linux go build ./...
```

The automation socket path must be short (macOS limits Unix socket paths to 104 bytes), hence the relative name. The e2e build also has a Debug menu that opens the flyout and the approval panel, which automation cannot reach through the tray.

## Video

`internal/video` loads `libavutil.61`, `libavcodec.63` and `libswscale.10` with purego: from `PINKY_FFMPEG_DIR`, next to the executable (and `Contents/Frameworks` in the app bundle), then Homebrew's and the system's folders. Missing or of another major version, `video.Load` returns an error that names the file, where it looked and how to install FFmpeg 9 (`video.ErrNotFound`); the client calls it at start, logs it, and every mirror shows it.

The struct offsets (AVFrame, AVPacket, AVCodecContext, AVCodec, AVCodecHWConfig) are FFmpeg 9.0's on 64-bit systems, printed with `offsetof` from Homebrew's headers. `checkLayout` verifies them at load against what FFmpeg itself writes (a frame it allocates, a packet it sizes, a context's codec id and thread count), so another layout fails with a clear error instead of corrupting memory. Pixel formats, hardware device types and decoders are looked up by name. `AVERROR(EAGAIN)` follows the platform's errno.

Decoding:

1. A configuration packet (SPS/PPS/VPS) is held and prepended to the next packet, as scrcpy's client does.
2. With `hw_device_ctx` set to a VideoToolbox device (D3D11VA on Windows, VA-API on Linux), FFmpeg's default `get_format` picks the hardware format, and falls back to software by itself when the hardware refuses a stream. Software decoding uses slice threads only, which add no latency, and `low_delay`.
3. A hardware picture is mapped (`av_hwframe_map`, no copy on VideoToolbox) or else copied (`av_hwframe_transfer_data`). For mirrors and recordings, `Decoder.ConvertYUV` copies its planes as they are, at the stream's size: NV12 from the hardware decoders, I420 from the software one, and pictures in another layout (10-bit HEVC) converted to I420 by `sws_scale_frame`. It names the matrix swscale reads from the stream's tag: BT.601 for a stream that names none, BT.709 for one whose matrix swscale cannot convert (the Mi 10 Pro's YCgCo). `Decoder.Convert` converts to RGBA with `sws_scale_frame`, into an `image.RGBA` of any size, for the gateway's screenshots from video (`internal/gateway/frameshot.go`) and the bench, 10-bit pictures too (`TestConvertTenBit`).
4. `video.Player` decodes on a goroutine of its own and triple-buffers: the picture on screen, the newest one waiting, the one being converted. A player that falls more than 30 packets behind skips to the next key frame. Once the picture waiting has waited for a second (the window is covered or the screen locked, so it paints nothing), it goes on decoding but converts nothing. It used to stop once nobody had called `Take` for a second, which lost a picture of a mirror that stopped painting on a still screen (below; `TestPlayerAfterPause`).
5. The mirror element takes the newest picture in its `Draw` callback, hands its planes to `Bitmap.UpdateYUV` (MyGo's, from egoist/mygo#1487: NV12's planes are taken without a copy, I420's chroma is interleaved into a plane the bitmap keeps, a picture of the same size keeps the textures, and the GPU converts and scales the planes as it samples them) and asks for the next paint with `AnimationFrameRate(60)` while pictures come and for 250 ms after the last one (`mirrorPaintFor`). Then it stops painting, and the player's `OnFrame` after the next picture posts an update that paints the window again (`mirror.wake`; `TestMirrorSleepsOnStillScreen`). So video frames repaint the window without rebuilding the view, a still screen costs no frames once the phone stops sending new pictures (the player hands over no picture that is the same, sample for sample, as the one before, `PlayerOptions.SkipRepeats`, as phones go on sending a still screen for a while; `TestMirrorSleepsThroughRepeats`), and nothing is converted while the window is hidden. Recordings play the same way, but paint at 60 fps for as long as they play. [mirror-smoothness.md](mirror-smoothness.md) has what this costs and saves. A screenshot from the mirror copies the planes on the main thread and converts them to RGBA off it (`video.YUV.RGBA`, as MyGo's CPU renderer converts them), at the phone's full resolution.

Until 2026-11-08 pictures were converted to RGBA by libswscale at the pixel size they were shown at (the bezel's screen box times the display scale): about 600×1332 for a 1080×2400 stream shown 300 points wide on a 2× display, and 316×700 for All phones' tiles. The planes go over at the stream's size instead: copying them costs a fifth of scaling them, and the GPU samples them bilinearly, two taps each way, as libswscale's fast bilinear scaled them. MyGo makes no halved copies of YUV pictures (its author's answer on #1487: apps pass frames at the size they show), so a mirror shown at a quarter of its stream or less would alias more than it did; the smallest tiles are about a third.

### Measurements

Machine: Apple M4 Pro (14 cores), macOS 27.2, Homebrew FFmpeg 9.0.2 (libavcodec 63.1.102). `go run ./cmd/pinky-ui-demo bench` decodes ffmpeg test streams (testsrc2 or testsrc; libx264 ultrafast or hevc_videotoolbox; 8 Mbit/s, a key frame each second, no B-frames) in real time at 60 fps, 15 s per run after 3 s of warm-up. CPU is the process's user and system time over all its threads, as a share of one core, plus VideoToolbox's decoder service, which decodes in a process of its own. Latency runs from the packet reaching the player to its RGBA picture being ready; the GPU upload and the display's refresh come on top.

| Stream → picture | Decoder | CPU, process + VT service | Packet → RGBA, median / p95 | Decode + convert, means |
| --- | --- | --- | --- | --- |
| H.264 1920×1080 → 1920×1080 | VideoToolbox | 6% + 3% | 2.2 ms / 3.5 ms | 2.3 ms + 0.36 ms |
| H.264 1920×1080 → 1920×1080 | software | 15% | 0.74 ms / 1.0 ms | 0.44 ms + 0.32 ms |
| H.264 1080×2400 → 1080×2400 | VideoToolbox | 29% + 3% | 4.3 ms / 4.9 ms | 2.2 ms + 2.3 ms |
| H.264 1080×2400 → 600×1332 (a mirror at 2×) | VideoToolbox | 15% + 3% | 3.2 ms / 4.0 ms | 2.3 ms + 1.1 ms |
| H.264 1080×2400 → 600×1332 | software | 22% | 1.3 ms / 2.0 ms | 0.40 ms + 1.1 ms |
| H.264 1080×2400 → 1080×2400 | software | 23% | 1.4 ms / 2.0 ms | 0.37 ms + 1.1 ms |
| HEVC 1206×2622 → 600×1304 | VideoToolbox | 16% + 3% | 3.5 ms / 3.9 ms | 2.4 ms + 1.2 ms |
| HEVC 1920×1080 → 1920×1080 | VideoToolbox | 6% + 3% | 2.1 ms / 3.5 ms | 2.1 ms + 0.29 ms |

About 4 of 900 pictures were replaced before they were taken, all at the start, when ffmpeg sends its first half second at once.

What the numbers say:

- **1080p60 costs about 9% of one core with VideoToolbox, and a picture is ready 2.2 ms after its packet.** FFmpeg waits for VideoToolbox frame by frame, which is most of that.
- **Converting to RGBA is most of the CPU.** VideoToolbox hands over NV12, which libswscale converts at about half the speed of YUV420P from the software decoder (it seems to miss the fast path), so at a phone's full size software decoding is both cheaper (23% against 32%) and quicker (1.4 ms against 4.3 ms) on this machine. At the size a mirror shows, where scaling dominates, both cost about the same. Hardware decoding stays the default: phones' real screens are harder to decode than test patterns, and smaller machines lean on the decoder more.
- **Two conversion threads** halve the conversion time for no more CPU: at mirror size, one thread took 15% and 2.1 ms a picture, two 14% and 1.0 ms, fourteen (libswscale's automatic choice) 20% and 0.4 ms. The decoder uses two (`DecoderOptions.ConvertThreads`).
- **Converting at the shown size matters:** a 1080×2400 stream shown at 600×1332 costs half the CPU of converting it whole.
- **Not measured:** the upload in `Bitmap.Update`, painting, and the whole client's CPU while it mirrors. The screen was locked for most of this overnight session, and macOS draws no windows then, so the client paints nothing. Earlier interactive runs played smoothly; Activity Monitor next to `go run ./cmd/pinky-ui-demo` shows the client's share. Painting and the client's CPU while it mirrors an emulator were measured on 2026-11-09: [mirror-smoothness.md](mirror-smoothness.md).

Ways to cut the CPU further, in the order I would try them: ask VideoToolbox for planar YUV or BGRA output through `hw_frames_ctx` (needs a `get_format` callback through purego), which removes the slow NV12 path or the YUV conversion; MyGo's NV12 or I420 bitmap updates converted in a shader (the design's first upstream ask); and zero-copy CVPixelBuffer to Metal (#973). The second is done (below); MyGo's author left the third for later.

### Planes instead of RGBA (2026-11-08)

The same machine and bench, with and without `-yuv`, two runs each, for a 1080×2400 H.264 stream at 60 fps shown at 600×1332:

| Handing a picture to MyGo | Decoder | CPU, process + VT service | Packet → picture, median | Convert or copy, mean |
| --- | --- | --- | --- | --- |
| RGBA at 600×1332, `Bitmap.Update` (before) | VideoToolbox | 17–18% + 3% | 3.5 ms | 1.2 ms |
| Planes at 1080×2400, `Bitmap.UpdateYUV` | VideoToolbox | 4% + 3% | 2.4 ms | 0.12 ms |
| RGBA at 600×1332, `Bitmap.Update` (before) | software | 25–26% | 1.7 ms | 1.2 ms |
| Planes at 1080×2400, `Bitmap.UpdateYUV` | software | 12% | 0.58 ms | 0.15 ms |

Drawing costs the main thread a little more: MyGo's Metal renderer encodes a full-size NV12 picture, its upload included, in 0.43 ms of CPU, against 0.28 ms for an RGBA picture of 600×1332 (its offscreen bench, measured for egoist/mygo#1487), and I420 from the software decoder costs another 0.25 ms to interleave its chroma. At 60 fps a mirror then takes about 7% of a core with VideoToolbox instead of 19%, and about 16% in software instead of 27%, besides VideoToolbox's service. Not measured: the whole client with a window showing the mirror. MyGo's `TestDrawsVideoAsTheCPURenderer` checks on this Mac that Metal draws NV12 planes as its CPU renderer does, and captures of `pinky-ui-demo` over the automation socket, which MyGo draws on the CPU, show the demo's video in its colors. `TestMirrorShowsPlanes` plays a phone through the mirror as a USB stream (H.264, 30 fps) that ends and opens again as a phone moving to Wi-Fi or Tailscale does, at another size, in HEVC at 15 fps and 1 Mbit/s: each shows in the colors ffmpeg made, within 3 levels, at its aspect, also between the slower stream's pictures, and a screenshot has the stream's size.

## How the client reads gateway.API

`gateway.API` leaves some things to convention. The client assumes these; the fake follows them, and the gateway should too (or tell me what to change):

- **Device coordinates.** A mirror opens `Stream` with `MaxSize: 0` and takes the decoded picture's size as the device's coordinate space: touches are sent in those pixels (as `driver.Toucher` defines them since 5e83552; `Key` and `Text` carry no coordinates) and steps' `Target` and `Point` are drawn in them. On an iPhone screen pixels and picture pixels differ by about 1%, so the overlay is clipped to the screen and an outline at the edge cannot spill onto the bezel. That matches the design, where screenshots are decoded from the stream. Tiles open the stream at full size too and scale it down while converting, so their overlays stay right without a `DeviceView` screen size.
- **Step marks.** Status `approval_pending` is "waiting for you" (amber dot); status `user`, or no tool and text starting "You ", is the person's step (person icon); status `""` or `running` on the last step of a running task is in progress (green pulse, and the mirror draws its touch); everything else is done (check). A step's text may hold a second line after a newline, shown smaller ("Your note: …").
- **Outcomes.** A done task with a "You denied" step (or status `approval_denied`) ended "after your answer", shown as Denied in Activity unless something was allowed after it.
- **Summaries and app icons.** Activity shows the `summary` the agent gave `phone_release` (any step's `Args` with a `summary` field), and takes the app for the letter icon from the first `phone_launch` step's `app` argument, or an "Opened X" step.
- **Approvals.** `Decide` gets `allow_once`, `allow_task`, `allow_always`, `deny` (with the note) or `take_over`. The approval card names the rule in "Always allow *payments* in *Dinely*" from `NeedView.Rule` and `App`, and offers it only when `NeedView.AlwaysAllow` is set (the gateway can keep it), and Allow for this task only when `NeedView.AllowTask` is set (not on the agent's own questions), in both the card and the floating panel; Settings › Safety › Always allowed lists the kept answers (`Exemptions`, `RemoveExemption`). `NeedView.Screenshot` must be in the same pixels as `Target`.
- **Input.** Pointer down, move (throttled to 125 Hz) and up go to `Touch`; right click is Back and middle click is Home; the wheel and the touchpad become a finger dragging the content (down at the pointer, moves against the scroll, lifted and put back at the edges): a touchpad's scroll moves it as it comes and lifts it 140 ms after the last event, while a mouse wheel's notches glide it (`ui/mirrorwheel.go`, issue 75) in a move every 8 ms over 90 ms, the notches of a spin keep one finger moving and lift it still moving so the phone flings, and a notch on its own rests 120 ms before the lift; Enter, Tab, Backspace, Delete, Escape, the arrows and Page Up/Down go to `Key`; typed and input-method text goes to `Text`; ⌘V pastes the computer's clipboard as text. The gateway is expected to pause the agent on any of them (the API's comment says so); the client shows the banner when the device reports `in_control`, or while a handoff is open on it. Calls run off the main thread, one queue per device, so they keep their order.

## Gaps in gateway.API (for the main session)

None of these block the demo; each is a small additive change when the gateway is ready for it. I did not change `api.go`. The gateway's methods beyond `gateway.API` (`MediaPath`, `Agents`, `SetAgentScope`, `SetReadOnly`, `WipeHistory`, and round three's in `internal/gateway/extras.go`) are used through optional interfaces, see below. `ui/contract_test.go` fails to compile when one of them stops matching the gateway's signature.

- **Settings.** No methods to read or change the safety preset, blocked apps, unlock, code hiding, retention or recording. The client keeps them (`ui.Options.Settings`) and reports each change through `ui.Options.OnSettings(old, s)`, with the settings as they were before it, and `cmd/pinky` passes on what differs and persists them. What the gateway keeps of them (the preset, Read-only mode, Hide codes, the blocked apps) comes again from `ui.Options.GatewaySettings` while Settings shows, once the policy's version changed (a hand edit of the rules files, or a change from Settings); it is shown and becomes the next change's `old`, so a change never passes on a value that a hand edit replaced. A read that a change overtakes is dropped and made again. A rules file without a preset shows none of the three.
- **Pairing.** Android pairs by QR code or pairing code through the extras (round three). iPhone pairing with a code still has no method: Pair waits for the gateway to report the iPhone.
- **Unlock.** `Unlock(deviceID)` is called after the person clicks "Unlock with Touch ID"; the gateway asks the keychain for the PIN, which asks for Touch ID. The client only saves and removes PINs.
- **Rules file path.** `ui.Options.RulesFile` opens it from Settings › Advanced.
- **Phone controls.** Record screen (a manual recording) has no API; it shows disabled, its tooltip pointing at Settings › Privacy. Rotate is a row of buttons, one per way the screen turns (five on Android, four on an iPad, three on an iPhone: no Auto-rotate on either, no upside down on an iPhone), the current one marked: `DeviceSwitch` reads the rotation while the menu is open (every 2 s, not while a turn is on its way) and `SetDeviceSwitch` turns it (`ui/rotate.go`), on phones with `Features.Rotate`; an iPhone or iPad over CoreDevice has it when the helper is installed (the gateway moves the phone there on a click), and without the helper, or with CoreDevice chosen, shows it disabled with a tooltip pointing at Settings › Phones.
- **One task's history.** Activity's per-task Delete stays disabled: the gateway can only wipe all of it.
- **Remote agents' state.** The client cannot tell whether the HTTP server listens (a port in use, say); `cmd/pinky` could report it.

## Against the real gateway

The second pass (2026-11-07) ran the client against `internal/gateway` itself and closed the gaps it showed. `go run ./cmd/pinky-ui-demo -gateway` is the way to see it; `go test ./ui -run TestGateway` clicks through it.

**What changed**

- **Handoffs** (`NeedHandoff`): the agent called `phone_handoff` and waits. The card's title is the agent's reason. Done → `Decide(id, AllowOnce, note)` (hand back with the note), Take over → `Decide(id, TakeOver, "")`, Decline → `Decide(id, Deny, note)`; Add a note opens the note field for Done and Decline. The floating panel and the flyout show handoffs too; the panel has no Return and Esc for them, since their buttons are not Allow and Deny.
- **First connections** (approval with rule `New agent`): a card with "Let *agent*", a select of what it may do (All apps except blocked ones, Only the apps I choose with an apps field, Read only: browse, never commit, Watch only: no taps or typing) with a line under it that says what the chosen one allows, Deny and Approve. When the agent asked for a read-only task (`NeedView.ReadOnly`), the card says so and the select starts on Read only. It shows in Needs you, in the floating panel and at the top of Settings › Agents, with an amber dot on the Agents tab.
- **The gateway's extra methods**, found with type assertions on the API value (`ui/extras.go`), so `gateway.API` is unchanged and the client still runs on anything that only implements it: `MediaPath` for step screenshots (task cards, Activity thumbnails and previews, replay); `Agents` and `SetAgentScope` for Settings › Agents (reloaded every 5 s while shown, and at once when a first-connection request opens or closes); `SetReadOnly("", on)` when the Read-only switch changes; `WipeHistory` behind Settings › Privacy › Delete…, with a confirmation.
- **In control without a task**: the banner reads "You're in control. Hand back to let agents use this phone again." and Hand back calls `HandBack`; the All phones tile shows Hand back on hover too.
- **`ui.Options.Hidden`**: on macOS the app starts as an accessory (menu bar icon, no Dock icon, no window); the first Needs you item opens the main window without taking the keyboard (`ShowInactive`) and makes the app regular. Opening it from the menu bar does the same.
- **App Nap**: `ui.Run` holds an `NSProcessInfo` activity for the app's lifetime (then `ui/appnap_darwin.go`, purego's objc; since 2026-11-08 MyGo's `Power.KeepRunning`, `keepRunning` in `ui/power.go`; a no-op elsewhere).

**Decisions**

1. **App Nap flags.** The brief asked for `NSActivityUserInitiated | NSActivityLatencyCritical`. `NSActivityUserInitiated` includes `NSActivityIdleSystemSleepDisabled`, so holding it for the app's lifetime would keep the Mac from ever idle-sleeping while Pinky runs, and Pinky lives in the menu bar all day. I used `NSActivityUserInitiatedAllowingIdleSystemSleep | NSActivityLatencyCritical`: App Nap, timer coalescing and background throttling stay off, which is what slowed the streams, and the Mac may still sleep when idle. Round three adds `NSActivityUserInitiated` while a task runs (round three, item 7). Since 2026-11-08 MyGo's `Power.KeepRunning` (egoist/mygo#1486) holds `NSActivityUserInitiatedAllowingIdleSystemSleep` without `NSActivityLatencyCritical`, which nothing measured needed. Measured on macOS 27.2, an app with no activity is napped after 35 to 40 s with its windows out of sight, but not while its menu bar icon shows, so Pinky naps only where the menu bar is out of sight too, as behind the locked screen; `Power.KeepAwake` also keeps App Nap off, but keeps the Mac awake as well.
2. **The scope chosen for a new agent.** Approve calls `SetAgentScope(name, scope)`, which also approves the agent, and then `Decide(id, AllowOnce, "")`, so the person's choice wins. The gateway's `approveAgent` currently sets `DefaultScope` again when a call that was still waiting picks up the answer; the main session asked the gateway session to keep a scope already set instead. Until that lands on main, a narrower scope chosen in the card can be lost on the first approval; the client does not poll to put it back. `TestGatewayAgentScope` checks the order of the two calls.
3. **The request's detail.** The gateway's "Allowing gives it everything access." would contradict a narrower choice in the select, so with the select shown the client drops that sentence and adds the time ("Asked at 03:51.").
4. **A handoff on a locked phone** is one card: the handoff's reason, "Unlock with Touch ID" (which ends the handoff, as decision 11 says), and Decline and Done for after an unlock on the phone. The separate unlock card for that phone is hidden, so two amber cards never ask for one thing.
5. **A handoff outranks control.** The gateway reports `needs_you` for a phone with an open handoff even while the person is in control (the agent waits with `phone_handoff` after it got `user_took_over`). So the phone screen shows the banner whenever a handoff is open: "You're in control. *Agent* is paused." when the task is paused, else "Your turn. *Agent* waits for you: *reason*". Hand back answers the handoff through `HandBack`. When the person already holds the phone, the card says "You're in control. Click Done to hand the phone back" and offers Decline instead of Take over.
6. **Step wording** now reads the gateway's conventions as well as the fake's: "Waiting for you: Tapped “Pay”" waits as "Waiting for you: tap “Pay”", then reads "You denied: tap “Pay”" (`approval_denied`), "Tap “Pay”" with "Not done: you took over" (`user_took_over`), "… stopped", "… the agent did not come back for it" (`expired`); handoffs read "Asked you to take over: …", then "You handed it back" or "You declined" with "Asked: “…”"; `ask` steps read "You agreed" or "You said no"; failures name their code. `device_locked` no longer marks a step as waiting: the gateway's locked phone is an unlock need, not a step. Between the person's answer and the end of the action, the gateway keeps the step `approval_pending` while the task is already working again; the client words that as "You answered: tap “Pay”" instead of showing an amber step with no card.
7. **Step screenshots** load with `MediaPath`, `os.ReadFile` and `image.Decode` off the main thread, at most 400 kept. The target is outlined on a screenshot only when the phone's live picture size is known (the gateway scales screenshots down by a whole factor), and never on 54-point thumbnails, where it said nothing.
8. **Read-only** calls `SetReadOnly("", on)` from the switch as well as reporting the change through `OnSettings`. If `cmd/pinky` also applies it, the two calls agree.
9. **Demo phones get video** through `video.FrameStream`, which encodes `Screenshot()` twelve times a second with the `ffmpeg` command. `ui/demogw` wraps `internal/demo`'s provider for that, so the gateway still sees the demo's `Toucher` and `PersonWatcher`.

10. **The mirror's states.** The first picture is taken while the mirror paints, and painting does not build the view again, so the "Connecting…" overlay from the build before it stayed over the live picture until something else changed (an idle iPhone sends no frames while its screen is still). Taking the first picture now asks for a build. A stream that fails shows why ("This phone has no live picture." for `ErrUnsupported`) and is opened again after 3 s (2 s after it ends) on a timer, not on the next build, which an idle window or a locked screen never makes. While it tries again the mirror keeps showing that reason, not the Connecting spinner, so a phone without a picture does not flicker every 3 s. Updates that goroutines post before the main window exists are kept and run once it does, instead of being dropped.

**For `cmd/pinky` (main session)**, done on main since: `Hidden` from `pinky mcp` (`--background`), and demo phones streaming through `video.FrameStream` in `internal/demo`.

- With `TrustNewAgents: true` in `--fake` mode the first-connection card never shows; leave it false in app mode if people should see it.
- Gateway wording seen on the way: the payment approval's title is "Pay: “Place order · $12.80”?" where the prototype says "Pay $12.80?", and the first-connection detail says "everything access".

## Round three

The third pass (2026-11-07) built what the acceptance checklist listed as missing on the client's side, after [round3.md](round3.md). Every new control is backed by a method in `internal/gateway/extras.go` (or `DeviceView.Features`) that the client finds with a type assertion and leaves out when it is missing; `TestGatewayRound3` runs them against the real gateway on the demo phones.

**What changed**

1. **Unlock from this computer.** Settings › Phones has, for each phone whose driver can unlock (`Features.Unlock`), "Unlock from this computer" with Save PIN…: a password field (`Password()`; its state is thrown away after each save so no editor keeps the PIN) that saves with `secret.PINAccount(id)` into `Options.Secrets` (nil is `secret.System()`), then "PIN saved" with Change and Remove. The locked phone's card shows "Unlock with Touch ID" (`api.Unlock`) only when the phone has `Features.Unlock`, its PIN is saved and Unlock from this computer is on. When only the PIN is missing, the card has the same form; otherwise it says why: "Unlock it on the phone.", "…is off in Settings › Safety.", or "Unlocking from this computer is not available on this system." (also in Settings › Safety). Unlock's errors read in words: a wrong PIN (`driver.ErrLocked`), no PIN saved, no keychain, a gateway without unlocking, offline; a cancelled Touch ID (`secret.ErrCancelled`) says nothing. After a refused PIN the cards stop offering Unlock until the phone is seen unlocked or the PIN changes, and offer Try again instead, which enters the PIN once more only when the person clicks it: a right PIN can be refused, say when the phone asked for a fingerprint first (#89). The client never reads a PIN back.
2. **Pairing.** Add phone › Android › Or pair over Wi-Fi calls `StartPairing`, draws `QR()` with `rsc.io/qr` (BSD, pure Go), its modules snapped to whole pixels with the quiet zone, and waits on `Wait`; leaving the step closes the pairing. On success the client reads the devices and shows the new phone. "Pair with a code instead" takes the IP address and port and the six-digit code for `PairWithCode` (45 s), with errors in words: a wrong code, an address that does not answer, no adb on this computer, a timeout. A gateway that cannot pair says so.
3. **The companion app.** On an Android phone's page (`Features.Companion`), `CompanionState` is read every 30 s while it shows. Missing, off or outdated puts a card above Tasks: Install, Turn on or Update, why (faster, steadier control, typing without the clipboard), "If the phone asks, allow the install.", and Not now (for the session). Install calls `InstallCompanion` (up to 5 minutes, for the phone's prompt) under an indeterminate progress bar, then reads the state again. Its errors read in words: Play Protect, Xiaomi's Install via USB, a timeout, the phone going offline. After a refusal over USB (`INSTALL_FAILED_USER_RESTRICTED`) the card and the checkup's companion fix add **Show pinky-android.apk** and **How to install it by hand** (`companionByHand`; the README's steps, in Chinese when Pinky is), as a Xiaomi without a SIM card cannot turn on Install via USB (`TestXiaomiThatRefusesUSBInstalls`). Settings › Phones names each phone's companion state.
4. **Background work.** "Work in the background" per phone (`Features.Background`) in Settings › Phones and in the mirror's ⋯, through `SetBackground`; `Background` is read with the devices. While a task runs on such a phone, the mirror shows `AgentStream`: the agent's display, without input and without the lock, with the agent's outline. A segmented control under it switches to the phone's own screen, which works as before with no outline. When the gateway has no agent display open (`ErrNotFound`: the task began before the switch), the phone's screen shows without the switch, and the client asks again every 3 s. The page header adds "· Claude Code works in the background"; tiles show the agent's display.
5. **Recordings and history.** Activity asks `Recording(taskID)`. A task with one offers Watch recording: the recording in a phone frame through `recording.Open` (or a fake's `OpenRecording`) and the mirror's decoder, with Play and Pause, a timeline with a mark per step (larger where the person answered; click, or ← and → by 5 s), the time, the steps with their times to jump to, and the step's target outlined for 2 s after its mark. A task without one keeps Watch replay with the screenshots, which ⋯ also offers for recorded tasks. ⋯ › Export… and Export all call `ExportHistory` into a new folder on the Desktop and show the report in the Finder; a gateway without it, or with the stub's `ErrUnsupported`, gets the client's own `report.html` and `tasks.json`. Keep activity for (7, 30, 90 days, 1 year, Forever) and Record the screen are saved through `OnSettings` as before, and the client passes them to `SetRetention` and `SetRecording` at start (off the main thread) and on each change.
6. **Remote agents.** Settings › Agents › Remote agents: "Let agents connect over HTTP" (`Settings.RemoteAgents`), the address (`RemoteAddr`, checked: a port from 1 to 65535 and an IP address of this computer or localhost, with a warning when other computers can reach it), the URL `http://<address>/mcp` (`ui.RemoteURL`), the agent token (`Options.AgentToken`, dots until Show) and `claude mcp add --transport http pinky <URL> --header "Authorization: Bearer <token>"`, each with Copy. Since 2026-11-10, while Tailscale is connected (`Options.RemoteTailnet`, read every 5 s off the main thread, from `tailscale status --json` and `tailscale serve status --json` through `internal/tailnet`), it also shows "Agents on your tailnet" with `tailscale serve --bg <port>` to copy (`serveCommand`; Pinky never runs it), and once Tailscale serves the address, the Tailnet URL `https://<computer>.<tailnet>.ts.net/mcp` instead, for Grok Bot and other agents on the tailnet ([mcp.md](../mcp.md#grok-bot-over-tailscale)). `ui/remote_tailnet_test.go`; `PINKY_SCREENSHOTS=<dir> go test ./ui -run TailnetScreenshots` draws both.
7. **The Mac awake while tasks run.** While any task runs (working, waiting or paused), `ui/appnap_darwin.go` holds `NSActivityUserInitiated | NSActivityLatencyCritical`, named "Pinky: an agent task is running": no idle system sleep, while the display may sleep. It ends with the last task, and on quit. The lifetime activity that keeps App Nap off still allows idle sleep. `TestKeepMacAwake` checks both with `pmset -g assertions`. Other systems do nothing yet. Since 2026-11-08 `keepAwake` (`ui/power.go`) holds MyGo's `Power.KeepAwake` with that name instead, on every system: an `NSActivityIdleSystemSleepDisabled` activity on macOS, a system-required power request on Windows, the desktop portal's inhibition of suspend on Linux. `keepAwakeAway` holds another, "Pinky: a phone is connected through Tailscale", while a phone is away (`docs/away-from-home.md`). MyGo's `TestPower` checks the Mac's with `pmset -g assertions`, `TestKeepAwake` what the client asks MyGo for, and `TestAwakeWhileTasksRun` when. `pinky daemon` has no MyGo app, whose `Power` needs its main loop, so it holds the same two reasons through `internal/awake` (`awake.ReasonTask`, `awake.ReasonAway`, which `ui/power.go` uses too): an IOKit `PreventUserIdleSystemSleep` assertion on macOS (purego), the same `PowerCreateRequest` on Windows, and `systemd-inhibit --what=idle:sleep --mode=block cat` on Linux, whose `cat` ends when Pinky closes its stdin or exits. `Hold` returns there once `cat` sends back a byte, so once logind has given the lock: it used to wait 500 ms for a refusal and call the lock held otherwise, and on CI logind once refused only after about a second, while polkit started (2026-11-08); `TestHoldWaitsForLogind` checks this with stand-ins for systemd-inhibit. `cmd/pinky/awake.go` decides when from the gateway's events and a look every 30 s, with the app's `KeepAwakeAway` from `settings.json`, and not for `--fake`. `TestHold` checks the Mac's with `pmset -g assertions` (on Linux with `systemd-inhibit --list`, skipped where logind refuses), `TestKeepDaemonAwake` when.
8. **fakeapi and the demo.** The fake implements every extra: features per phone, pairing (a Pixel 8a over Wi-Fi; the code is scanned after 6 s, or from Demo › Scan the Pairing QR Code; code 000000 is wrong and port 1 refuses), the companion app (scenario `companion`), background displays (scenario `background`; the agent's display is SMPTE bars), recordings drawn per step with the step's number and the time in seven-segment figures (the boarding pass task has none), retention, recording on and off, and export. `pinky-ui-demo` keeps PINs in a `secret.Memory` and shows a made-up token.

**Decisions**

1. **Where the PIN is saved.** In Settings › Phones and on the locked phone's card, the same form, at least four characters. The mirror's lock has no Unlock button of its own: inside the mirror's input handler it took clicks unreliably, and it repeated the card.
2. **Is there a keychain?** `secret.Supported` asks the store, which says so without asking the person anything; no store is probed by writing or deleting.
3. **Where exports go.** A new folder on the Desktop per export ("Pinky activity 2026-11-07 at 09.30.12"), shown in the Finder, as screenshots go to the Desktop. No save dialog, which a locked screen could not test either.
4. **The agent's display is for watching.** Input there would reach the phone's own screen, so the keys are disabled with a tooltip and touches are not taken; the person switches to the phone's screen to use the phone.
5. **A recording's position** comes from the frames' PTS, which `recording.Open` counts from the recording's start, as `RecordingMark.At` does. A fake's `OpenRecording` stream that counts from where it opened (its first PTS more than 10 s below the start asked for) is offset; `recording.Open`'s never is, since it opens at the key frame at or before the start, which can be more than 10 s earlier (iOS sends one every 20 s at most, and a still screen sends few frames). That stream sends the frames from the key frame up to the start at once: the player shows none of them and keeps the time at the start. `Player.Take` only returns the newest picture, so catch-up on a real file is taken in `OnFrame` before the next decode overwrites it (a channel closed after `NewPlayer` returns so `OnFrame` does not race on a nil player). A paused seek shows that lead as the picture at the start when the next taken frame follows it (`Seq+1`); if frames were dropped in between, it shows the later frame at its own time instead of an earlier lead. When the stream ends before a paint (a seek past the last frame), `ended` takes once more so the pending picture is not left behind. `TestRecordingPausedSeekPicture` checks the shown frame's PTS for a seek to 1.899 s on a 10 fps file and past the end. The timeline seeks once its press ends, wherever the pointer is then (clamped to the ends); a press the system cancels seeks nowhere.
6. **Record and Keep activity reach the gateway from the client**, through `SetRecording` and `SetRetention`, so they apply whether or not `cmd/pinky` passes them too; the calls agree.
7. **The agent token comes from `Options.AgentToken`**: the client reads no files of the gateway's.
8. **The companion card** sits above Tasks in the neutral panel color, not amber and not in Needs you: nothing waits on it.

**For the gateway session (`cmd/pinky`)**

- Pass `ui.Options.AgentToken` (`api.EnsureToken(dirs.TokenFile())`), and serve MCP at `ui.RemoteURL(settings.RemoteAddr)` (path `ui.RemotePath`, `/mcp`) while `RemoteAgents` is on; `OnSettings` reports changes to either. `Options.Secrets` can stay nil.
- `recording.Open`: PTS from the recording's start, and `io.EOF` at its end.
- `Unlock`: wrap `secret.ErrCancelled`, `secret.ErrNotFound` and `driver.ErrLocked` with `%w`, so the client's words apply.
- Background tasks and the person: when the person takes over, or a handoff is open, the mirror shows the phone's own screen, and `Touch`, `Key` and `Text` go there as always. A handoff that needs the person on the agent's display (signing in inside the app it works in) has no way to send input there yet.
- `pinky --fake` runs the real gateway on `internal/demo`'s phones, which have none of the optional drivers, so these controls stay hidden there until the demo phones gain fakes of them; `pinky-ui-demo` shows them all on `fakeapi`.

## i18n

The client speaks English, Simplified Chinese (2026-11-07, branch `claude/ui-zh-hans-465ea6`) and Japanese (2026-11-09, DIYgod/pinky#1 by tarou-jp, brought up to date with main). Everything the person sees in the app is in the catalogs: the windows, the floating approval panel, the menu bar flyout and the tray, the menus, notifications, toasts, tooltips and accessibility labels. What agents read (MCP tool descriptions and results, `docs/SKILL.md`, `docs/mcp.md`) stays English.

**How it works.** MyGo's catalogs (`docs/i18n.md` in the MyGo module): `cmd/pinky/mygo.json` has `"i18n": {"dir": "locales", "source": "en"}`, so `cmd/pinky/locales/` holds `en.json` (every message, with an `@key` description and typed placeholders), `zh-Hans.json` and `ja.json`, in ARB with ICU MessageFormat. `go tool mygo generate cmd/pinky` (which `mygo build` and `mygo dev` also run) writes `catalog_gen.go` there: a Go function per message, named after its key (`sidebarStopAll` → `locales.SidebarStopAll()`), which says it in the language the app shows. The `ui` package calls them; the generated code embeds the catalogs and imports MyGo's own Simplified Chinese and Japanese strings (the menus by role, the Edit and Window menus, Quit, text fields' menus, dialog buttons). A packaged build declares every catalog's language (`CFBundleLocalizations`, `zh-Hans.lproj`, `ja.lproj`), so System Settings › General › Language & Region › Applications lists Pinky; `cmd/pinky/resources/darwin/<lang>.lproj/InfoPlist.strings` translate what Info.plist says to the person (the reason for Local Network access, `NSLocalNetworkUsageDescription`), and a new language needs one.

**Which language.** MyGo picks the one of the app's languages the person reads best, from the system's preferred languages in order (any Chinese written in simplified characters, such as zh-Hans-CN or zh-Hans-SG, gets Chinese; ja and ja-JP get Japanese; Traditional Chinese and everything else get English). Settings › Advanced › Language overrides it: System, English, 日本語 or 简体中文 (each named in its own language, as `i18n.Autonym` writes it). The choice applies at once, without a restart, and MyGo keeps it as the app's language in the macOS defaults (`AppleLanguages` of the app's domain), which System Settings shows too; System clears it. For one run, `MYGO_LOCALE=zh-Hans` forces a language and saves nothing (`go tool mygo dev -lang zh-Hans` does the same). The e2e build in Chinese:

```sh
go build -tags e2e -o build/pinky-e2e ./cmd/pinky
mkdir -p /tmp/pkzh && PINKY_HOME=/tmp/pkzh MYGO_AUTOMATION=/tmp/pkzh/auto.sock MYGO_LOCALE=zh-Hans build/pinky-e2e --fake
```

**The gateway's words.** The gateway records steps, approval titles and status lines as finished English ("Tapped “Pay”", "Pay: “Place order · $12.80”?", "Claude Code · Reorder lunch · in the background"), and history keeps them that way. In Chinese and Japanese, `internal/gwtext/gwtext.go` (until 2026-11-13 `ui/gwtext.go`, which keeps its names as aliases; Pinky on Android's core uses it too, [android-app.md](android-app.md#the-gateways-words-in-the-apps-language)) reads them back by the patterns of `internal/gateway` (`describe`, `approvalTitle`, `statusLocked`, the files and agent steps) and says them with the catalog's `gw…` messages. An approval that asks for the action itself does so in the imperative since 2026-11-12 ("Tap at 540,1200?"), and in history's past tense before ("Tapped at 540,1200?"); `parseQuestion` reads both, since an earlier daemon and the approvals it stored still send the old words. `stepDisplay` words the client's own step states (`step…`), and `ruleName` and `ruleAction` name the policy's rules. A step's second line is read the same way, by `stepSub`: "Your note: …", and "Always allowed: “Payments” in Dinely" under a step that ran without asking because of the person's Always allow, which an uncertain step keeps under "May have happened". Inside them, agents' words (task titles, handoff reasons, questions, notes, summaries) and the phone's (labels in quotes, app and phone names) stay as they are. Text that matches no pattern shows in English, and in English the gateway's text always shows unchanged. When the gateway words something new, add a pattern and a case to `ui/gwtext_test.go`. `internal/update`'s reasons why a copy of Pinky cannot replace itself (`Location.Problem`) are read back the same way, by `whereProblem`.

**Adding a message.** Add it to `en.json` with an `@key` description (where it shows, what its placeholders hold; `type` is `String`, `int` or `DateTime`), then its translations to `zh-Hans.json` and `ja.json` in the same commit (and recheck both when the English changes), run `go tool mygo generate cmd/pinky`, and call `locales.Key(…)`. Name keys after the area they belong to (`settingsPrivacy…`, `needs…`, `tasks…`), and make one message of a sentence with placeholders rather than gluing pieces, so a translation can reorder it; counts that change the English words are ICU plurals. `go test ./cmd/pinky/locales` fails when a key lacks a translation or a translation's arguments differ; `go tool mygo i18n check cmd/pinky` reports the catalogs' errors and warnings. Strings that sessions add in English work as before and show in English in the Chinese and Japanese UI until they get keys.

**Chinese style.** Short, plain Simplified Chinese as in Mac apps, addressing the person informally; full-width punctuation, a single-character ellipsis, a space between Chinese and Latin letters or digits, and the terms `zh-Hans.json` already uses (look a term up there before adding one). Product and feature names that are names stay as they are (Pinky, Claude Code, Touch ID, Wi-Fi, USB); Pinky for Android is “Android 版 Pinky”.

**Japanese style.** Polite, plain Japanese as in Mac apps (です・ます), with the terms `ja.json` already uses: スマホ for phone, パソコン for computer, エージェント, タスク, 「設定 › スマホ」. 「」 quote the app's and the systems' names for menus, buttons and settings; “” keep the quotes around the person's and the phone's words that English has (“{label}” をタップ). Full-width punctuation, a single-character ellipsis, and a space between Japanese and Latin letters, digits or placeholders. Name the phones' settings as their Japanese systems do: iOS's デベロッパモード, 「プライバシーとセキュリティ」, 「このコンピュータを信頼しますか？」; Android's 「開発者向けオプション」, 「USBデバッグを許可しますか？」, 「ペア設定コードによるデバイスのペア設定」, 「QR コードによるデバイスのペア設定」, 「IP アドレスとポート」; Finder's 「Wi-FiがオンになっているときにこのiPhoneを表示」. The Japanese README ([README.ja.md](../../README.ja.md)) is a community translation; it names the app's menus, buttons and messages as `ja.json` does, so a renamed message is worth a look there too.

**Tests.** The `ui` tests pin the client's words to English (`ui/main_test.go`), since they find controls by their English labels; `inLanguage(t, "zh-Hans", testers…)` switches a test to Chinese, as `TestGatewayWordsInChinese`, `TestStepDisplayInChinese` and the language setting's test do (`TestLanguageSetting` picks 日本語 too, and `TestLanguageFollowsAJapaneseSystem` reads Pinky on a computer that prefers Japanese). Tests that check a layout or draw screenshots loop over `testLanguages()`, every catalog's language, so a longer translation that no longer fits fails there (`TestApprovalPanelFits`, `TestControlBannerAboveTheScreen` and the like); `PINKY_SCREENSHOTS=dir go test ./ui -run Screenshots` writes their pictures, named `-en`, `-zh` and `-ja`. `PINKY_SCREENSHOTS_ZH=dir go test ./ui -run 'TestChinese.*Screenshots'` draws every screen in Chinese, the update card, What's new, Settings › Advanced › Updates and Send feedback included (`PINKY_SCREENSHOTS_DARK=1` adds dark to the first). The tests of what the views say about when an iPhone's helper stops working set the time with `clockAt` and make their dates with `wallTime` (`ui/main_test.go`), of what the clocks where they run show, so that neither the hour nor the time zone they run in changes the words they check or lay out. `clockAt` sets the tests' clock (`testNow`), which stands in for `timeNow`, read by the helper's words only, and for the clock of every test's fake (`fakeapi.Options.Now`). Other views tell dates by `Context.Now` or `time.Now`, which a test cannot set to a date: MyGo's Tester starts its own clock at the computer's time and only moves it on (`Advance`).

**What stays English.** Agent-facing text; the gateway's error details inside a toast or a card (the words around them are translated), drivers' details (`Info.Detail`, pairing and install errors from adb); the export files (the gateway's report, and the client's fallback `report.html` and `tasks.json`); the release notes of a version that `CHANGELOG.zh-CN.md` lacks (What's new, and the update card, show the Chinese notes of the changelog's translation and of `latest.json` where they exist, and fall back to `CHANGELOG.md`'s), and all release notes in Japanese, which has no translation of the changelog; a feedback issue's version line and diagnostics, and the usage report What Pinky shares shows, for whoever reads them (`keepLabelIn` and `videoLineIn` say their parts in English); logs; the e2e build's Debug menu and the demo's Demo menu; the system's power assertion name; and the fake's scripted wording in `ui/fakeapi` (steps such as "Found last week's order"), which only the demo and tests show.

**Seen for real.** The e2e build ran with `--fake` and `MYGO_LOCALE=zh-Hans`, an MCP agent (`pinky mcp`) reordering lunch on the demo Pixel through the real gateway, and every screen was captured over the automation socket: the steps and the payment approval come from `internal/gateway` and read in Chinese; the language was switched to English and back in Settings without a restart. `TestChineseScreenshots` drew the states the demo phones cannot reach (locked with the PIN form, Touch ID, handoffs, a new agent, the companion card, the agent's screen, pairing, the recording player). One layout changed for Chinese: step thumbnails no longer print "Loading…" in a box 25 points wide, which Chinese stacked a character a line. Elsewhere the Chinese is shorter than the English and fits where it does, at the window's minimum width too. A known compromise: placeholders that hold names get a space next to Chinese text, as Latin names such as "Claude Code" need, so the rare fallback names in Chinese (an agent without a name, "this app") show with a space too many. The same flow in both languages:

| Screen | English | Simplified Chinese |
| --- | --- | --- |
| A payment waiting, the gateway's steps | `live-payment-en.png` | `live-payment-zh.png` |
| The floating approval panel | `live-panel-en.png` | `live-panel-zh.png` |
| In control, the step not done | `live-takeover-en.png` | `live-takeover-zh.png` |
| Activity, the finished task | `live-activity-en.png` | `live-activity-zh.png` |
| Settings › Advanced › Language | `live-language-en.png` | `live-language-zh.png` |

## The approval's screenshot

The floating panel used to show a strip 100 points high around the target, scaled to the target's width: for a small button that was the button alone, enlarged, and the person could not tell the app, the page or what the agent wanted to press. It now shows, from `ui/approvalshot.go`:

- **The whole screen, small,** in the screenshot's shape, with the target outlined in the overlay's indigo (12 points at least, so a small button still shows), a tap without an element as a ring, and a swipe as an arrow from a dot where the finger goes down. It says which app and which page.
- **Beside it, the part around the target, larger,** at about 0.85 DIP a point of the phone's interface (phones count as 400 points across and tablets, by their shape, as 800), the rest dimmed: the button's words read as on the phone. A wide target zooms out to show whole while its words still read (0.55 DIP a point), at most to the screen's width; one too large for that shows around the finger, without the outline, which would be two lines across. A portrait screen has the zoom on its right; a landscape one, phone or tablet, takes the width with the zoom under it. An action without a position (a key, opening an app, the agent's own question) shows the screen alone, on a tray.
- **A click shows it larger:** beside the floating panel in a window of its own, a panel that floats over every app and takes the keyboard without making Pinky the active app (Esc or × closes it), or over the main window as a dialog. It follows the approval the panel shows and closes when that approval is answered or withdrawn.
- **The panel is as tall as what it shows** (`panelHeight`): 400 wide, room for a title and a detail of two lines each (both are cut at two lines now) and the screenshot, which grows into what shorter words leave. 450 for a portrait screen, 530 to 540 for a landscape one, 330 without a screenshot, at most the display's work area. The answer keeps that height for its moment, with the screenshot it answered.
- **Without a screenshot**, which the gateway takes of no blocked app and a phone may fail to send, an action's panel says so in a line: the phone is locked, Pinky takes no screenshots of an app Settings › Safety blocks, or there is no screenshot of the screen. Copying files and the agent's own questions say nothing.
- **Needs you cards** in the main window show the same screenshot, smaller, where the middle does not show that phone's screen with the target marked: another phone or All phones selected, the phone locked, or its mirror without video.

The gateway now says where the finger goes: `NeedView.Point` (a tap's point, a swipe's start) and `NeedView.To` (a swipe's end), which `ask` puts in `approval.Request`. Before, a swipe's approval had only the element under its start.

Tests: `TestShotLayout` (the split and what the zoom shows), `TestApprovalPanelFits` (portrait and landscape phones and tablets, a swipe, a tap by point, no target, no screenshot on a locked phone and in a blocked app, in English and Chinese, with short and long words: the countdown, Deny, Allow and More options stay in the window and the screenshot keeps 150 points), `TestApprovalShotOpens`, `TestApprovalShotOnCard`, and `TestApprovalPoint` in the gateway. Their screens are a checkout page MyGo draws in the test (`ui/fakescreen_test.go`), not a phone's. `PINKY_SCREENSHOTS=dir go test ./ui -run TestApprovalShotScreenshots` draws each case in both languages (`approval-shot-en.png`, `approval-shot-zh.png`). `go test -tags e2e -run TestApprovalShotInTheApp ./cmd/pinky` runs the e2e build in English and Chinese with an MCP agent paying on the demo Pixel: the panel is taller than one without a screenshot, a press on the screenshot opens the large view left of the panel, its × closes it, and Allow pays (`live-panel-en.png`, `live-panel-zh.png`; passed on this Mac with the screen unlocked).

## Brand colors, and keeping amber and red apart

The prototype's colors stay, quiet as they were: neutral grays, indigo for primary buttons, links, switches and the agent's overlay, amber for what needs the person, red for Stop, green for working. The brand appears only where the eye looks for identity:

- **Logo:** the 32-point icon (`pinky-32.svg`, lemon field, phone face, watermelon ring) in the sidebar header, next to the name, right of the window controls. Lemon appears nowhere else.
- **Selection:** the selected row of the sidebar, Settings' sections and Activity keeps the prototype's gray fill and gains a 3-point watermelon bar at its start.
- **Focus:** the keyboard focus ring is watermelon (`#EE4D6C` light, `#FF8A9B` dark; 3.6:1 and 7.6:1 against the background).

The prototype's amber and red sat too close to the brand. In OKLCH, dark mode's amber `#F2BB59` (L 0.82, hue 80°) was almost the logo's lemon `#FFBE1A` (L 0.84, hue 83°), and dark mode's red `#FF8A80` (hue 26°) had the same lightness as the watermelon highlight `#FF8A9B` (hue 12°). So I moved the semantic colors away from the brand's hues, and kept each color to its own shapes:

| Role | Light (was) | Dark (was) | OKLCH hue |
| --- | --- | --- | --- |
| Needs you: dot | `#DF8601` (`#E09100`) | `#F7A74E` (`#F2BB59`) | 65° (lemon is 83–96°) |
| Needs you: text | `#975800` (kept) | `#F7A74E` (`#F2BB59`) | 64–66° |
| Needs you: card, line, tile edge | `#FFF6EB`, `#F6D7B1`, `#E79733` | `#342210`, `#674520`, `#B6762A` | 66–72° (were 83–85°) |
| Stop: text, border, hover | `#C32C18`, `#F0C6BD`, `#FEF0ED` | `#F87A60`, `#723B31`, `#3A1E19` | 31–33° (watermelon is 11–13°) |
| Brand: selection bar, focus ring | `#EE4D6C` | `#FF8A9B` | 13° / 12° |

The trade-off: amber now leans orange and red leans vermilion, so amber and red are closer to each other than in the prototype (hue 65° against 31°, but lightness 0.70 against 0.54 in light mode), still clearly apart, while neither can be mistaken for the logo or the selection mark. Shapes back the hues up: amber is dots, card tints and the tile edge; red is only the text and border of Stop; watermelon is only a thin bar and the focus ring; lemon is only inside the logo. No brand color is used as a fill, so selected rows never look like Needs you cards.

## The look: tokens and parts (2026-11-11)

Every screen is built from the tokens and parts in `ui/style.go`, so the app reads as one. Use them instead of numbers of your own; a size, gap or radius that is not on the scale is a bug.

- **Layout.** A stage and two wings: the page in the middle is the stage (`bg`), the sidebar and the panel on the right are the wings (`wing`, gray), and the cards on the wings are raised (`card`, white in light mode, with `raised`'s two-layer shadow). On a page, settings and lists sit in macOS-style groups (`group` and `groupRow`: rows on one rounded field, divided by a hairline inset from the left). The sidebar shows the window's material where there is one (`c.Vibrancy()`: macOS, and Mica on Windows 11), and the wings' gray elsewhere. The sidebar's brand row and every page's header are `headerHeight` (52) tall, and rows start their text at `rowInset` (12).
- **Type.** One scale, `fsCaption` 12, `fsSmall` 13, `fsBody` 14, `fsHeadline` 15, `fsTitle3` 17, `fsTitle2` 20, `fsTitle1` 26, and four weights, `fwRegular` to `fwBold`. Body text, rows and buttons are `fsBody`; descriptions `fsSmall` in `ink2`; meta, counts and eyebrows `fsCaption` in `ink3`; card and group titles `fsHeadline` semibold; a page's header `fsTitle3`, its content's title and dialogs' `fsTitle2`. Section labels above a group are `sectionLabel`: caption, semibold, `ink3`, sentence case (no capitals).
- **Space.** `sp1` 4, `sp2` 8, `sp3` 12, `sp4` 16, `sp5` 20, `sp6` 24, `sp8` 32. There is no `sp7`.
- **Radii.** `rSmall` 6 for small buttons, chips and thumbnails; `rControl` 8 for buttons, rows and menu items; `rCard` 12 for cards, tiles, popovers and notes; `rSheet` 16 for dialogs.
- **Color carries meaning, nothing else.** Amber (dots, card tints, the tile edge) is what needs the person; red only Stop and Deny (and errors' text); green allows and works; watermelon (`brand`) only the selection mark and the focus ring; indigo (`accentBtn`) the primary button and links. Removing a phone, the one other destructive action, is a plain button in Settings and red text in the ⋯ menu, and asks first.
- **Parts.** `button` (34 tall; `btnPrimary`, plain, `small`, with an icon), `linkButton`, `quietButton`; `card`, `raised`, `group`/`groupRow`; `sheet` for every dialog (a 16 radius, 24 padding, a soft shadow and a 0.32 backdrop, with the primary button rightmost); `menuPanel`, `menuItem` and `menuText` for popovers such as the phone's ⋯ menu (every row with an icon, and what goes under a row lined up with its words, `menuIndent`); `pageHeader`, `pageTitle` and `settingsHeading`; `emptyWell` for an empty list and `stateTitle`/`stateDetail` for a page's empty, error and first-run states; `tileIcon` for Add phone's choices; `appIcon`, `dot`, `hollowDot`, `spinner`, `selectionMark`. Activity's steps hang on a timeline: a rail with each step's picture, or `stepThumb`, a sketch of a screen, where it has none.
- **States.** Whatever takes a click shows hover (`bg3`, or the ink faint on the wings) and pressed (`sel`, or a deeper tint of its color), and keyboard focus draws MyGo's ring. A disabled control is MyGo's half opacity; nothing else dims it.
- **Icons.** MingCute's, bundled unchanged in `ui/icons/mingcute`: the free Apache-2.0 set there, and in `pro/` the few of MingCute Icons System Pro v1.60 that the free set lacks (the eye with a slash, the open lock, the cable, the checklist), used under the owner's purchased license. Its README says how to add one. Line icons at rest, the same icon filled for a selected row or tab (`icGridOn`, …, with `pick`). One grid of sizes: `iconMicro` 12 in badges, `iconSmall` 14 beside small text, `iconBody` 16 in rows, buttons, tabs and menus, `iconLarge` 20 on the keys under the phone and in banners, `iconBig` 24 over the phone's screen, `iconHero` 28 at the head of an empty or blocking state. Icons take the text's color, so they follow the theme. The keys of Android's navigation and the rotation picks keep glyphs of our own (`icon`), drawn as the systems draw them, at MingCute's stroke: neither set has a left triangle and a plain circle for Back and Home, or a phone turned four ways. An offline phone shows a cable when it was last on USB, no Wi-Fi when it was on the network (`offlineIcon`), and Unlock shows a fingerprint on the Mac, an open lock elsewhere (`unlockIcon`).
- **Motion** (`ui/motion.go`). Short and calm, easing out, showing what changed: a page, a Settings section and a step of Add phone fade in where the last one was (`swapIn`, 160 ms); a request in Needs you comes down to its place while what is under it slides (`cardIn`, `slide`, 200 ms); a task's step and a phone in the sidebar grow in (`rowIn`, `listRow`); the You're in control banner fades in (`bannerIn`); dialogs rise in over a fading backdrop (`sheet`); dots and switches fade to a new color (`colorTo`); fills follow the pointer (`fillFade`, 120 ms). Nothing bounces, loops for show or waits: an arriving card's buttons and Stop take clicks from the first frame. Under the system's reduced motion MyGo only fades, in place. Tests run with `stillMotion` set, since their clock stands still; `withMotion` turns it back on for the tests of motion. MyGo's dialogs (`DialogBase`) have no exit of their own, so a dialog goes at once.

`PINKY_SCREENSHOTS_ZH=<dir> PINKY_SCREENSHOTS_DARK=1 go test ./ui -run 'TestChinese.*Screenshots'` draws every screen in both appearances, the quickest way to see a change everywhere.

## Differences from the prototype

- The phone in the middle grows with the window (the prototype draws it 300 × 632); its bezel radius scales with it.
- The "You're in control" banner sits above the phone, which shrinks to leave it room. The prototype floats it over the top of the middle, where its small phone leaves room; over a phone that fills the window it covered the screen's top (see Nothing over the phone's screen).
- The mirror shows ffmpeg test patterns (testsrc2 for the Pixel, testsrc in HEVC for the iPhone), not the prototype's drawn Dinely, Chatter and Tally screens. Targets and touch points are where those apps' buttons would be.
- "Replay this demo" links are gone (the client is not a demo); the Demo menu restarts scenarios.
- More phone controls: Record screen and Background display are disabled (no API).
- Add phone's Wi-Fi QR is a placeholder, since the gateway cannot start wireless pairing yet; the prototype's QR encoded nothing either. The older-iPhone choices say this build cannot set them up yet.
- Activity's step thumbnails are the steps' screenshots where the gateway kept one, and the prototype's sketch otherwise; Watch replay is a slideshow of those screenshots, not a video.
- Settings › Agents shows first-connection requests as the prototype does, and the same request is also a card in Needs you, so it can be answered from the phone screen and the floating panel.
- The tray shows the number of waiting needs as its title on macOS.
- The approval panel is 400 wide and as tall as what it shows, rather than 360 × 320: see The approval's screenshot.

## Known issues

- Unpackaged builds (`go run`) show no notifications; macOS allows them for packaged apps only. The packaged demo (`go tool mygo build cmd/pinky-ui-demo`) builds, signs ad hoc and starts its streams, but its notifications were not tried: the screen was locked by then, so nobody could answer macOS's permission question.
- The approval panel shows when approvals wait and the main window is not focused. While Pinky itself is the active app with only the panel and the flyout open, macOS may make the panel the key window when the main window hides; it never takes the keyboard from another app.
- A recording that plays repaints the window at 60 fps, even through still parts. The live mirror stops 250 ms after its last picture since 2026-11-09 ([mirror-smoothness.md](mirror-smoothness.md)).
- While the screen is locked MyGo draws nothing, so a window builds its view only when something asks it to (the automation socket's input, a capture). The client reads the gateway's views before its window shows, so the first frame already has the phones.
- A wheel's notch drags the content by 15% of the mirror's height on every platform (MyGo sends a notch as 40 DIPs on macOS, 99 on Windows at the default three lines, 50 on Linux; `wheelNotches`); touchpads map one to one. Before issue 75 a notch was one jump of 2.5× its DIPs (about a third of the screen on Windows) and the finger lifted only after resting, so the phone never flung. Tune on real phones.
- The panel's Return and Esc work only after the person clicked it, by design.
- Windows and Linux builds compile but were not run; D3D11VA and VA-API decoding are untested.
- FFmpeg 9 only; other major versions are refused rather than supported with other offsets.
- Approving a new agent with a narrower scope can lose it on the first approval until the gateway keeps a scope set before the answer (decision 2 above); 79d1c2a on main fixed this in the gateway.
- Recordings, export, retention and background displays are stubs in the gateway, so against it the client shows its fallbacks; the player and the agent's display were checked on the fake only (the player through the real decoder).
- Keeping the computer awake while tasks run was tried on macOS only; Windows and Linux get MyGo's power requests, untried there.
- Durations ("38 s", "1 min 02 s") are joined with no-break spaces, so a task card's "Claude Code · sdk_gphone64_arm64 · 38 s" moves "38 s" to the next line together instead of leaving the "s" alone. MyGo honors them (`TestDurationsDoNotWrap`), except that a no-break run wider than its whole box still breaks inside, as browsers do.
- With `-hidden`, the first request opens the main window and, because that window is not focused, the floating panel as well.
- `video.FrameStream` costs a full-size RGBA copy and an x264 encode per frame; it is for demo phones only.
- **Automation with the screen locked was unreliable until MyGo fixed it.** While the screen is locked a window counts as occluded and MyGo holds its frames (`engine.holdFrame`). In a 15-second run the main window was built twice, once at start and once on the first capture; a press (`act`), a pointer click (`input`), pointer moves and later captures built nothing. A press only bumps the button's click count, which the next build consumes, so it was lost until a build happened to come along, and which presses "worked" depended on timing, not on the card: Approve, Allow and the sidebar's Activity were lost the same way. The floating panel is built once when it is made (hidden, nothing waiting) and never again while the screen is locked, so its capture was an empty card with an empty tree. Since 2026-11-08 Pinky uses a MyGo whose automation builds the frame a window out of sight waits for when it acts on it, reads its tree, waits for it to idle or captures it (egoist/mygo#1485): in `pinky-ui-demo -hidden`, whose window never shows, a press on the sidebar's Settings over the socket shows Settings in the next tree and capture. `idle` never settled while a mirror played, as it repainted 60 times a second; since 2026-11-09 it settles once the phone has sent no new picture for 250 ms (a repeat of the one before does not count). `ui.Tester` builds every frame and stays the way to test these flows in unit tests (`TestGateway*`, `TestApprovePaymentWithRealGateway`).

## Verification

Each screen was driven over MyGo's automation socket (`e2e` build) and captured in light and dark; the captures are in `docs/notes/screenshots/` (test patterns and the fake's data only, no real screen). The flows were clicked through with the screen on: approving, denying, taking over by touching the mirror, handing back, unlocking, three phones and All phones, the flyout, Add phone with a phone plugged in. The screen locked partway through the night, so the final set was captured with it locked: each capture is preceded by a nudge and a throwaway capture so the view builds with the current state (which MyGo's automation does itself since egoist/mygo#1485), the approval panel was drawn by `ui.Tester` (`PINKY_SCREENSHOTS=dir go test ./ui -run TestPanelScreenshots`), and Add phone is shown in the first-run state (`-scenario empty`).

| Screen | Light | Dark |
| --- | --- | --- |
| Phone, one phone, payment waiting | `phone-one-light.png` | `phone-one-dark.png` |
| Phone, three phones | `phone-three-light.png` | `phone-three-dark.png` |
| All phones | `all-phones-light.png` | `all-phones-dark.png` |
| Phone locked, unlock card | `locked-light.png` | `locked-dark.png` |
| You're in control (drawn again by `ui.Tester`, see below) | `takeover-light.png` | `takeover-dark.png` |
| Approval panel, 1 of 2 | `approval-panel-light.png` | `approval-panel-dark.png` |
| Approval panel, a drawn checkout (English, Chinese) | `approval-shot-en.png` | `approval-shot-zh.png` |
| Menu bar flyout | `flyout-light.png` | `flyout-dark.png` |
| Activity | `activity-light.png` | `activity-dark.png` |
| Settings › Safety | `settings-light.png` | `settings-dark.png` |
| Add phone, first run | `add-phone-light.png` | `add-phone-dark.png` |

Against the real gateway, `ui/gateway_e2e_test.go` runs the views in `ui.Tester` on `internal/gateway` with the demo phones and scripted agents: the first-agent approval clicked in Needs you, the payment approval with its screenshot and target, a takeover by touching the mirror (pointer events through `mirrorInput`, so `Touch` reaches the demo phone), the banner and Hand back, the payment allowed, the finished task in Activity with step screenshots from `MediaPath` and Watch replay; a narrower scope for a new agent that survives the agent picking up the answer, and Settings › Agents showing it; a handoff answered with Done and a note; a handoff on a locked phone as one card; Stop and Stop all; and a phone kept after the agent left, handed back. `go test -race ./ui/...` passes.

The same flow was clicked through in the real app over the automation socket with the screen locked (`-gateway -hidden`, e2e build): no window at start, the window opening by itself on the first-agent request, Approve, the live demo screen with the payment's target outlined on the video, a click on the mirror taking over, Hand back, Allow, and the task done. These were captured with `ui.Tester` against the real gateway (`PINKY_SCREENSHOTS=dir go test ./ui -run TestGatewayScreenshots`), the mirrors showing still screenshots of the demo phones:

| Screen | Light | Dark |
| --- | --- | --- |
| First connection, in Needs you | `gateway-new-agent-light.png` | `gateway-new-agent-dark.png` |
| Payment approval from the gateway | `gateway-payment-light.png` | `gateway-payment-dark.png` |
| Handoff, with the "Your turn" banner | `handoff-light.png` | `handoff-dark.png` |
| Handoff in the floating panel | `handoff-panel-light.png` | `handoff-panel-dark.png` |
| Settings › Agents, changing a scope | `settings-agents-light.png` | `settings-agents-dark.png` |
| Activity with step screenshots | `activity-shots-light.png` | `activity-shots-dark.png` |

Round three's screens, drawn by `ui.Tester` on the fake (`PINKY_SCREENSHOTS=dir go test ./ui -run TestRound3Screenshots`; the mirrors show stills of the test source and of a demo phone's app):

| Screen | Light | Dark |
| --- | --- | --- |
| Pair over Wi-Fi, QR code | `pair-qr-light.png` | `pair-qr-dark.png` |
| Pair with a code, wrong code | `pair-code-light.png` | `pair-code-dark.png` |
| Install Pinky for Android | `companion-light.png` | `companion-dark.png` |
| The agent's screen in the mirror | `background-light.png` | `background-dark.png` |
| Locked phone, saving its PIN | `unlock-pin-light.png` | `unlock-pin-dark.png` |
| Locked phone, Unlock with Touch ID | `unlock-touch-id-light.png` | `unlock-touch-id-dark.png` |
| Settings › Phones | `settings-phones-light.png` | `settings-phones-dark.png` |
| Settings › Agents › Remote agents | `settings-remote-light.png` | `settings-remote-dark.png` |
| Settings › Privacy | `settings-privacy-light.png` | `settings-privacy-dark.png` |
| Recording player | `recording-light.png` | `recording-dark.png` |

Round three's tests (`ui/round3_test.go`, `ui/recording_test.go`, `ui/appnap_darwin_test.go` (since 2026-11-08 `ui/power_test.go`), `ui/fakeapi/round3_test.go`, and `TestGatewayRound3`): a PIN saved on the card, never shown, used to unlock and removed in Settings; a short PIN refused; no keychain; the QR code drawn module by module as `rsc.io/qr` encodes it and the paired phone shown; a pairing code after a bad address, a wrong code and a refused port; the companion card through Install and Not now; the background switch; the agent's screen with its switch and the fallback without one; recordings offered or not, jumping to a step's mark, and a recording played through the real decoder, jumping to a step's picture (checked by its color), pausing and going on from there; export through the gateway, without it and with its stub; Record and Keep activity applied; remote agents' switch, Copy, Show and the address check; the Mac awake while a task runs, and the assertion seen by `pmset`; the controls against the real gateway on the demo phones.

The `ui` tests run the views in `ui.Tester` against the fake: approving, denying with a note, taking over and handing back, the lock card, one and three phones, All phones, the offline phone, Activity, Settings, Developer mode's details, Add phone with a new phone plugged in, the approval panel's paging and Esc, the flyout's Stop all, and mirror input (touch mapping, Back, Enter, Chinese text, wheel and touchpad to swipe; the wheel's glide timing in `TestWheelGlide`).

- **Blocked apps for one task** (`NeedView.Unblock`): when an agent opens a blocked finance app, or one the person blocked, its approval card says what allowing does ("Wise is blocked by “Finance apps”. Allowing lets Claude Code open it and read its screen until this task ends…") and offers Deny and Allow for this task, nothing else; Return and Esc in the floating panel answer it too. `NeedView.App` is the app to open. Task cards say "Read only" for tasks an agent asked to run read-only (`TaskView.ReadOnly`).

## Removing a phone

**Remove this phone…** (`ui/removephone.go`) is in the ⋯ menu under the mirror, on the page of a phone that is offline or needs setup (a link under its detail), on each tile in All phones (a ⋯ button that shows while the pointer is on the tile, and the tile's right-click menu), in a phone's right-click menu in the sidebar, and on each phone's row in Settings › Phones ("Remove…"). Every entry opens one dialog, which says:

- what goes: the phone leaves the list and agents can no longer use it; its saved PIN and its settings in Pinky are deleted; the task running on it stops (with the agent's name and the task);
- what stays, unless the person ticks **Also delete its history: tasks, screenshots and recordings** (off by default): its history in Activity;
- what happens next, by how the phone is connected now: on a cable it shows up again at once as a new phone, unless unplugged first; an Android phone on Wi-Fi is disconnected and not reconnected by itself, and comes back when paired again in Add phone or plugged in; an iPhone on Wi-Fi comes back when plugged in; an offline phone comes back when plugged in (or, on Android, paired again over Wi-Fi). And that the phone keeps trusting the computer, so other apps are not affected: Pinky offers no way to unpair an iPhone.

**Remove** (red) calls the gateway's `RemoveDevice` and, once the phone is gone, deletes `pin:<id>` from the keychain; a missing PIN or a system without a keychain is fine. The dialog stays open, with a spinner, until the gateway answers; an error shows in it. Afterwards a toast says "Removed <name>", or that the history or the PIN could not be deleted; the client forgets what it kept about the phone (PIN state, background switch, companion state, the connection choice Settings › Phones read, its popovers), and reloads Activity when the history went. The selection moves to another phone (`afterRefresh`). Without `RemoveDevice` on the API (an older gateway), none of the entries show. The fake removes phones too (`fakeapi/remove.go`, `Removals`, and `SetTransport` to put a phone on a cable or on Wi-Fi).

Strings: the `removePhone*` keys in both catalogs. Tests: `ui/removephone_test.go` (each entry, the notes for a cable, Wi-Fi and an offline phone on both platforms, the history box, Cancel, the PIN deleted, Chinese) and `TestRemovePhoneInTheApp` in `cmd/pinky` (tag `e2e`: the app over the automation socket with the fake phones). `PINKY_SCREENSHOTS=dir go test ./ui -run TestRemovePhoneScreenshots` draws the dialog and its entries in English and Chinese (`remove-*.png`).

## Nothing over the phone's screen

A person reported on Discord (2026-11-08) that the "You're in control. Codex is paused. Hand back" banner covered a button of the phone's app, which they could then press only on the phone. The banner was attached to the top of the Phone screen's middle, 16 below the header, over a phone that the window sizes to fill the middle: it covered the screen's top 47 points at the window's default size (more in a smaller window), the status bar and most of an app's top bar, where back buttons and menus are, and it took the clicks there.

What `ui/` draws over a phone's picture, and what happens to it now:

| What | Where | Now |
| --- | --- | --- |
| "You're in control" / "Your turn" banner with Hand back (`controlBanner`) | Phone screen, over the top of the phone; took clicks | Above the phone, in the layout: the phone shrinks to leave it room. This is the one reported |
| Lock: the screen dimmed, "Pixel 9 is locked" pill | Mirror, whole screen; the pill a little below the middle, where a PIN pad comes up; clicks went through | Both fade out while the pointer is on the mirror (Phone screen only), so the lock screen shows as it is; they come back when it leaves |
| Connecting spinner, "no live picture" | Mirror, while there is no picture | Unchanged: there is nothing under them |
| Agent's overlay: target outline, touch ripples | Mirror, painted while the agent works; clicks go through | Unchanged |
| Input method's composition | Mirror, near the bottom, while composing; clicks go through | Unchanged |
| Take over, Stop, Hand back and ⋯ on a tile | All phones, on hover | Unchanged: tiles take no input, a click selects the phone |
| Toasts | MyGo, near the window's bottom | Unchanged: they can cover the keys under the phone for a few seconds, never its screen |

How the banner works now (`ui/phone.go`):

- It is the first child of the middle column, before the phone. Its height comes from its words: `controlBannerHeight` wraps them with `Context.MeasureText` in the width the padding, border, icon, gaps and Hand back leave, which is what the row lays them out in, so the phone is sized for the banner it gets (the banner's height is set to it). Words longer than three lines end with an ellipsis; the Needs you card has the agent's words in full.
- It comes and goes only between the person's gestures on the mirror (`mirror.down`, `mirror.scrolling`): a touch on the mirror starts the takeover that shows the banner, and a mirror that moved or shrank under a finger still down would end the touch elsewhere on the phone. `app.control` keeps the banner each phone shows until the finger lifts.
- The phone and its keys are keyed (`phone-<id>`, `keys-<id>`), so they stay the same elements as the banner comes before them: the mirror keeps the keyboard's focus, and what the person types after the click that took over goes to the phone.

The old banner also broke the takeover itself: built before the phone without a key, its arrival made the mirror a new element in the middle of the press that caused it, so MyGo dropped the press and the touch-up never reached the phone (the phone was left with a finger down until the next touch), and the mirror lost the keyboard's focus.

Tests (`ui/controlbanner_test.go`): `TestControlBannerAboveTheScreen` (in control and a handoff, English and Chinese, windows 1280 × 820, 960 × 620, 1100 × 1300 and 1900 × 820: the banner above the phone, everything within the window, and clicks on the screen's top 1%, 3.5% and 6.5% at its left, middle and right, its middle and its bottom reach the phone at the matching pixel), `TestControlBannerHeight` (short and long words, English and Chinese, every 9 points from 960 to 1500 wide: the words stay inside the banner, at most three lines), `TestControlBannerWaitsForTheFinger` (the banner waits for the finger, the touch ends where it started, the mirror keeps the focus and typing reaches the phone, a scroll holds the banner too) and `TestLockGivesWayUnderThePointer` (the lock screen dimmed at rest, shown as it is under the pointer, dimmed again when the pointer leaves; a click goes to the phone). All four fail on the code before this change.

`PINKY_SCREENSHOTS=dir go test ./ui -run TestControlBannerScreenshots` draws `mirror-bar-<en|zh>-<light|dark>.png`, the middle of the Phone screen in four states side by side (in control; a handoff's "Your turn"; locked; locked with the pointer on the mirror; the phones show a checkout page and a PIN pad drawn in the test), and `takeover-<light|dark>.png`, which replaced the automation captures above. `handoff-<light|dark>.png` were drawn again by `TestGatewayScreenshots`. The live captures `live-takeover-en.png` and `live-takeover-zh.png` predate the change and still show the banner over the phone.

## iPhones and iPads over Wi-Fi (2026-11-08)

`ui/wifi.go`. Settings › Phones ends with **Use iPhones and iPads over Wi-Fi** (`Settings.IOSWiFi`, on by default), which reaches the gateway through `wifiSetter` (`SetWiFi`) and the app's `applySettings`.

- **Found on Wi-Fi.** A phone listed with `driver.SetupConfirmWiFi` gets its own page (`foundPage`: the name, model and iOS version, why Pinky waits, **Use** and **Ignore**, and what each does), **Use**/**Ignore** on its row in Settings › Phones instead of its settings, and a box above Add phone › iPhone's steps (`foundBox`). The steps do not follow a found phone until the person chooses Use; then they do. **Ignore** calls `IgnoreFoundPhone`, forgets what the client kept about the phone and says how to use it after all. Both go through `foundChooser`; without it the buttons are disabled.
- **Which steps need the cable.** While Wi-Fi is on, Add phone › iPhone's steps carry a hint under their titles (`guideRowHint`): Trust "Needs the cable", Developer Mode "Done on the iPhone; if the switch isn't there, plug it in once", Prepare "Also works over Wi-Fi, more slowly"; the plug-in step says a phone paired over Wi-Fi shows up by itself. A phone on Wi-Fi waiting for Trust is told to plug in (`setupWords`, also on its page and in the checkup), one waiting for Developer Mode that the switch appears only over the cable.
- **Locked on Wi-Fi.** A phone that waits to be unlocked on Wi-Fi (`lockedOnWiFi`: `needs_setup`, `SetupUnlock`, Wi-Fi) shows a lock and "<name> is locked" with why Pinky waits, the line "Locked on Wi-Fi" in the sidebar, Settings and the header (which now says iPad for iPads), and "Unlock <name>" in Add phone.
- The fake API: `FindOnWiFi` (also Demo › Find an iPad on Wi-Fi in `pinky-ui-demo`), `SetWiFi`, `WiFi`, `UseFoundPhone` (prepared, then online), `IgnoreFoundPhone`.

Strings: `settingsPhonesWiFi*`, `found*`, `gwStatusFoundOnWiFi`, `gwStatusLockedOnWiFi`, `phoneLockedWiFi*` and the `addPhoneIPhone…WiFi`/`…Found…`/`…Hint` keys, in both catalogs. Tests: `ui/wifi_test.go` (the page and Use, Ignore from Settings, the switch both ways, Add phone with the found box, the hints and Use, Trust over Wi-Fi, a locked phone). `PINKY_SCREENSHOTS=dir go test ./ui -run TestWiFiScreenshots` draws the found phone's page, Settings › Phones and Add phone in English and Chinese.

Later the same day: a phone being opened or moved (`gateway.DeviceConnecting`) gets a page with a spinner ("Connecting to …", "Moving … to Wi-Fi…", and that a task on it waits), the line in the sidebar, header and Settings, and no Take over. A phone macOS's device service lost (`driver.SetupDeviceService`, "Lost on Wi-Fi") gets a page that explains it and offers Restart device service… on a Mac (also in Add phone › iPhone and the checkup's words): a dialog says every iPhone and iPad disconnects for a second or two, then the gateway's `RestartDeviceService` runs with the reason macOS shows in its password request; declined, nothing is said. The fake API: `SetConnecting`, `LoseOnWiFi`, `RestartDeviceService`, `Restarts`, `SetRestartError`. Tests: `TestConnectingPage`, `TestLostOnWiFiPage` in `ui/wifi_test.go`.

## All phones with many phones (2026-11-08)

All phones scrolls, and its tiles sit in columns (`allPhonesGrid` in `ui/phone.go`). Before, the tiles were a wrapping row in the middle of the page: more phones than the window showed were out of reach (the row overflowed without a scroll view, and tiles squashed to nothing in a narrow window), and a last row that was not full sat centered, its tiles wider than those above.

- The page is a scroll view under the header, between the sidebar and the right panel, which stay put. The tiles are a `mg.Grid` of `Fixed` columns centered in it: `tileColumns` fits as many tiles of at least 200 points as the room between `sidebarWidth` and `rightPanelWidth` takes (known from the window's width, as `phoneCenter` sizes the mirror), sharing it up to 232 points each, as the wrapping row did. A last row that is not full starts in the first column. Phones that all fit in one row take one column each, so one row stays centered as before; with three phones the page is laid out exactly as before at 960, 1280 and 1700 points.
- Every tile is built in every frame, so each phone keeps its one stream while the page scrolls. A tile out of view is not painted, so its player converts nothing (`Player.convert` keeps the waiting picture while nobody takes one), and it shows the newest picture as it scrolls into view. Tab reaches tiles out of view, and MyGo scrolls the one it focuses into view.
- The sidebar had the same problem: with a dozen phones in a short window, Add phone, Activity, Settings and Stop all went out of the window. The phones' rows are now a scroll view of their own (`sidebarPhones` in `ui/view.go`) between "Phones" and Add phone, which stays in view with the rest; with few phones nothing moves. A margin and padding of 3 points keep the rows' focus rings from being clipped. When the selection changes, as by a tile of All phones, its row scrolls into view, once.
- No other page had either problem: the other wrapping rows are short rows of buttons or chips, which start at the left, and the long pages already scroll.

The fake API has `-scenario many` (`fakeapi.ManyPhones`, "Eight phones" in the Demo menu): Three phones' and Pixel 8a, iPhone 16e, OnePlus 13 (locked), iPhone 15 Pro and Moto G (offline). Tests (`ui/allphones_test.go`): `TestAllPhonesGrid` (one, three, four and six columns at 960, 1280, 1500 and 1920 points, every tile under the one above it in the first row, the columns centered; three phones in one row, centered), `TestAllPhonesScrolls` (the last tile out of view, then in view after a touchpad scroll and clicked, the header in place; Tab brings it into view), `TestAllPhonesLivePictures` (real test streams: a tile in the third row has its stream open but is not painted, then shows its bars once scrolled into view without its stream opening again), `TestSidebarScrolls` and `TestSidebarFollowsSelection` (twelve phones in a 960 × 620 window). All but the three-phone check fail on the code before this change. `PINKY_SCREENSHOTS=dir go test ./ui -run TestAllPhonesScreenshots` draws `all-phones[-end]-<en|zh>[-dark].png` and `sidebar-many-phones-en.png`.

## Add app picks from the phones (2026-11-09)

Settings › Safety › Add app lists, under its field, the connected phones' apps that match what is typed (`gateway.InstalledApps`, through the `installedAppLister` interface; read when the field opens and again after 30 s), leaving out those blocked already or listed in Settings. Clicking one, or typing its exact name, alias or id, blocks it by its id (`BlockedApp.ID`). Text that matches no app on the phones gets a warning ("No app on your phones is named “…”. Pinky blocks only that exact name: …"), and Block still blocks the name as typed, for an app that is not on the phones yet. Without phones, the field works as before and says so. Test: `ui/blockapps_test.go`.

## Pinching on the mirror (2026-11-09)

`ui/mirrorpinch.go`. The person pinches on the phone's picture in two ways, both sent as two-finger touches (`gateway.Gateway.Touch2` → `driver.MultiToucher`), which pause the agent like any of the person's input:

- **A touchpad's pinch** (`mg.InputZoom`). MyGo gives an incremental scale per event at the pointer and no end event; it reports pinches on macOS, under Wayland and in browsers, and on Windows a touchpad's pinch arrives as Ctrl+scroll, which MyGo delivers as `InputZoom` too. The first event puts two fingers down around the pointer, along the screen's axis with more room, inside margins of 7% and 8% of the screen (the edges' system gestures, as the agent's pinches keep); each event moves them apart or together by its scale; they lift 150 ms after the last event. They start at a quarter of the room when zooming in and at all of it when zooming out, and a pinch that reaches either end lifts the fingers and puts them down again at the other, so a long pinch keeps zooming. A pinch lifts a scroll's finger first, and scroll events are dropped while its fingers are down.
- **Option-drag** (Alt elsewhere; `mg.Alt`), as in Xcode's Simulator: one finger at the pointer and the other opposite it across the screen's centre (`picW-1-x`, `picH-1-y`), so moving toward or away from the centre zooms and moving around it turns. A click that is not a drag is a two-finger tap.

Both only where `DeviceView.Features.Pinch` is set (the driver is a `driver.MultiToucher`, or an iPhone or iPad over CoreDevice with the helper installed, which the first pinch moves there: `internal/gateway/helperneed.go`); elsewhere `InputZoom` is not taken and Option-drag is a plain drag. While fingers are down, `drawFingers` shows them as two white dots, as Simulator does. An approval's screenshot of a pinch draws both fingers' arrows and zooms on both paths (`NeedView.Point2`, `To2`), and history and the step overlay say "Zoomed in on …", "Zoomed out …" or "Turned … with two fingers" (`gwDid*`/`gwDo*` messages). Tests: `TestMirrorPinch`, `TestPinchMarks`, and the `gwtext` cases.

Not tried by hand: a real touchpad on Windows and Linux, and the feel on a real phone (the emulator's and the Simulator's apps followed the touches, see android.md and ios.md).

## Dragging (2026-11-10)

The agent's `phone_drag` (docs/notes/gateway.md, "Dragging") shows in the client in three places:

- **History and the step overlay** (`internal/gwtext/gwtext.go`): "Dragged “<label>”" is `gwDidDrag`, "Dragged from X,Y to X,Y" `gwDidDragFromTo`, and a bare "Dragged" (no element and no points, as when the element was gone) `gwDidDragBare`; a step waiting for the person or one that failed takes the `gwDoDrag*` forms ("drag …"). `imperative` turns "Dragged " into "drag " for English.
- **The approval's screenshot** (`ui/approvalshot.go`): `NeedView.Path` holds every point of the drag; `shotMarks.route()` draws one arrow through them all with a ring at each stop, and the zoom takes in the whole route. A drag without stops draws like a swipe.
- **The live mirror** (`ui/mirror.go`): while a swipe, a drag or a pinch waits for the person, the mirror draws its way with the same arrow as the screenshot (`waitingWay` finds the step's approval in `a.needs` by task and by the step's time or target; `StepView` carries only the target and the first point). The card in the main window has no screenshot while the mirror shows the phone (`cardShot`), so before this the person saw only the outline of the icon the drag presses, not where it goes. Background tasks draw it on the agent's display only, as the outline.
- **Tests:** `TestDragMarks` (the route and the zoom), `TestMirrorShowsWaitingDrag` (the route on the mirror), the `gwtext` cases for all three forms in Chinese.

The person's own drags on the mirror are unchanged: they were already played back with their timing (`Touch`).


## Blocked apps that share a name (2026-11-10)

The owner saw two rows named "Trust" in Settings › Safety › Blocked apps: Trust Bank's iPhone app (`sg.trust`) and Trust Wallet's Android app (`com.sixdays.trust`). What went wrong, and what changed:

- **The crash.** Rows were keyed by name, so the two rows had one key and shared one state in MyGo (a Tester panics on that; the app only logs it). Both remove buttons answered one click, and each removed its row by index, the second past the end of the list. Rows are now keyed by id and name and removed by id and name (`withoutBlocked`); `applySettings` compares rows by id and name too, where a name-keyed map saw no change after one of the two went. Test: `TestBlockedAppsShareAName` (ui), `TestUnblockOneOfTwoSameNamedRows` (cmd/pinky).
- **No crash report.** The app and the daemon deferred `crashes.Stop()`. A panic runs deferred calls on its way up, so Stop removed the crash file before the runtime wrote the panic to it: no report and no unclean-exit marker. They now defer `crashes.Exit(recover())`, which stops on a clean exit and panics again otherwise; the runtime writes the panic, with the first panic's stack, to the file. Test: `TestPanicThroughExitBecomesAReport`.
- **Unblocking by id.** `SetAppBlocked(app, false)` excluded the app's name as well as its id, which reached every app of that name. Now an app with an id is excluded by its id, and an excluded id takes the app out of every category whatever its name (`Apps.Categories`); the name is excluded too only when the name alone is still blocked by the table (Wise, "DBS digibank"). A name the person typed is one entry for every app of that name, and goes when a row it blocks is removed. Test: `TestUnblockOneOfTwoSameNamedApps`.
- **The missing row.** Settings read the blocked apps at start, before Wi-Fi iPhones were online, and again only when the policy changed; and `Gateway.BlockedApps` listed the phones' apps one after another within one 10-second budget, so a slow phone listed last could run out of it. Settings now reads the list again when the phones online change (`settingsVersion`), and the gateway lists all phones at once. The iOS driver still drops an app list that arrives after its caller gave up (`catalogWithin`); keeping it for the next call would be the iOS driver session's change.

## Stills of idle iPhones in All phones (2026-11-11)

An open view of an iPhone keeps its display stream running (`internal/driver/ios/idle.go` stops it 30 s after the last view or input), and while it runs iOS shows screen sharing and never auto-locks the phone. All phones is Pinky's home page, so its live tiles kept an idle iPhone's screen on for as long as Pinky showed it.

- `tileStill` (`ui/phone.go`): an iPhone's tile shows a still when no task runs on it (`running`: working, waiting or paused), the person is not in control and no handoff waits. Otherwise the tile is live as before.
- `stillFor` (`ui/mirror.go`): a mirror that has a picture holds it (`hold`: state `"still"`, the player closed, the bitmap kept) once its stream has shown pictures for `stillAfter` (1 s); a mirror without a picture opens the stream for one first. `mirrorFor` (the phone's page, a task starting) opens a still's stream again and shows the held picture until the first new one, with no spinner. A tile whose reopened stream fails, or sends no picture for `stillGiveUp` (10 s), keeps the picture it has.
- `sweepMirrors` now holds a phone's own mirror that has a picture instead of closing it, while the phone is online, so that coming back to All phones shows the still without opening the stream again. Agents' displays, and mirrors of phones gone or offline, close as before. Hiding or minimizing the window still closes every mirror; shown again, All phones takes a fresh picture (the stream runs about a second, then the driver's 30 s idle stop).
- Android tiles stay live: scrcpy runs with `power_on=false` and does not turn the screen on or keep it on, so the phone dozes and locks as usual, and Android shows no screen-sharing indicator for it.
- `DeviceView.Streaming` needed no change in the gateway: once the tile lets go, the hub closes and the driver reports `Capturing` false after its idle stop. The fake now counts its open streams, so the demo's "streaming its screen" follows the mirrors too.
- Tests: `TestTileStill`, `TestStillKeepsItsPicture`, `TestAllPhonesIPhoneStill` (real ffmpeg test streams: live while the iPhone's task runs, still and no stream after Stop, live on its page, still again on All phones, kept across another page). Not checked on a real iPhone.