# iOS driver notes Status of `internal/driver/ios`, the CoreDevice backend, and what step 0 found on a real iPhone. Measured on 2026-11-07 with an iPhone 13 mini (iPhone14,4) on iOS 27.0.1 (24A446), connected by USB to a Mac with macOS 27.2 and Xcode 27.2; the developer disk image was mounted and Developer Mode on. A second pass the same night added element frames through `remoteAXService`, the provider's inspection cache, and touches in video pixels. The third (round three, the next morning) added typing by paste, `Waker` and `Unlocker`, measured the phone over Wi-Fi, and read Calculator's display as its value. The fourth (the evening of 2026-11-07) added iPhones below iOS 27; see "iPhones below iOS 27". The fifth (the night of 2026-11-07) fixed what the field test of 2026.1107.1 found (`docs/notes/field-test-iphone.md`): refs, app ids, pickers, covered rows, SpringBoard's alerts, unlocking, the stream while nobody watches. The sixth (2026-11-08) made Pinky prepare a phone itself, without Xcode; see "Preparing a phone without Xcode". ## Step 0 results | Item | Result | Numbers | | --- | --- | --- | | Tunnel without root | **Works.** A userspace tunnel in Pinky's own process, next to macOS's CoreDevice tunnel | Tunnel up in 0.25–0.35 s, RSD handshake 0.13–0.15 s, 85 services, MTU 16000 | | Screen stream | **Works.** HEVC over RTP through the userspace tunnel, depacketized to Annex B | 1136x2464 HEVC Main, up to 60 fps, 1–3 Mbit/s typical (cap 6), first packet 0.6 s after the start request, keyframe 70 ms after a request, 0 packets lost over USB | | Touch | **Works.** Tap, swipe, long press and raw touches through the universal HID service, on one long-lived session. Raw touches take video pixels; the video is the screen at 1:1 plus black padding | Touch-down to the first changed video frame: 100–131 ms, median 109 ms (still Settings page) | | Keys | Home **works** (Indigo button event); Back is a left-edge swipe and **works**; Recents returns `unsupported` | | | Text | **Works for any text**, pasted: the pasteboard is saved, set to the text, Command-V goes through the virtual keyboard, and once the field shows the text the pasteboard is put back. Checked in Settings' search with the Pinyin keyboard active: English, Chinese, emoji, appending, and the pasteboard byte for byte the same afterwards. Without the pasteboard service, or when it does not answer, keys: printable ASCII on a U.S. hardware keyboard layout only, switching a composing keyboard (Pinyin) to the Latin one with Control-Space and back. The field is read back after typing, and text that reads otherwise is an error | Paste: 0.26–0.46 s per call (the save 2–70 ms, the paste confirmed in the field 0.1–0.36 s, the restore 25–50 ms), about 1 s for the first call in a session. Keys: 4–5 s for "clock" with the switch | | Element tree (pass/fail) | **Pass.** Through `remoteAXService`, the service Xcode 27's Device Hub uses: label, role, value and frame for every element on screen, nothing installed on the phone. (The inspector API, the first attempt, gives no frames on iOS 27, as go-ios found on 18.4.1.) | Act, settle, `Observe` in one session: Settings 33–261 ms (12 elements, all with frames), Calculator 340–481 ms (23; it waits for the push that carries the display's number), once 1.07 s, over 18 launches in round three (the second pass had seen 2.2 s and 3.0 s in 2 of 9; they did not come back). Without settling, right after a launch: 0.8–1.6 s, which the phone sets. A walk alone: 0.19 s (Settings), 0.29–0.31 s (Calculator), 0.32 s (home screen). Before, inspector and OCR: 1.35 s for Settings with 13 of 38 elements placed, 0.65–0.70 s for Calculator with 8 of 23, and 13.5 s for the first OCR in a process | | OCR placement | **Fallback only.** macOS Vision through purego places elements by label when `remoteAXService` fails, and adds text the tree lacks when asked | Vision 0.1–0.4 s per screenshot (13.5 s the first time in a process); 9 of 10, 12 of 13, 18 of 26 elements placed on Settings, Clock, home screen | | Screenshot | **Works**, two services with fallback | 1125x2436 PNG, 0.35 s (Settings) to 0.6–0.75 s (home screen, 5–7 MB) over USB | | Wake, lock state, unlock | **Works.** On iOS 27 the device is a `Waker` (the screen's state from CoreDevice's backlight state; `Wake` presses Home only on a dark screen, so it leaves the app in front alone) and an `Unlocker`. Below iOS 27 `WDADevice` is an `Unlocker` but no `Waker` (its `Unlock` wakes the screen with Home itself), and `AXDevice` is neither. The lock state read right on the awake, unlocked phone. `Unlock` unlocked the locked iPhone 13 mini on 2026-11-08 through both paths, the owner typing the passcode, and woke its dark screen by itself; `Wake` on its own on a dark screen was not tried. See "What the owner should try" | Screen state 12 ms | | Wi-Fi | **Works, and on by default since 2026-11-08.** The same tunnel, video, HID, `remoteAXService` and pasteboard run through usbmuxd's network connection; phones Pinky has not used wait for the person, and locked ones are left alone | See "Wi-Fi" below. iPad Pro, iPadOS 27.0.1: swipes 77–212 ms to a changed frame, screenshots 5 s, 0.24 % of video packets lost | | Windows and Linux | **Not tested.** The package cross-compiles without cgo for both | | What these mean for the design: - The userspace tunnel removes the "tunnel may need administrator rights" prerequisite on every OS: no utun, no wintun, no sudo. It still needs usbmuxd (built into macOS; Apple Devices on Windows; the `usbmuxd` package on Linux). - The element tree, the part step 0 feared, works without WebDriverAgent: `remoteAXService` gives frames, and the phone even pushes the whole visible tree after every change, so `Observe` after an action usually costs tens of milliseconds. It needs `clientNeedsAccessibility`, which turns on the phone's accessibility runtime for the session (Device Hub does the same). OCR stays as the fallback. - USB matters: an iPhone that also syncs over Wi-Fi is listed twice by usbmuxd, and a tunnel over Wi-Fi is slower, screenshots most of all (see "Wi-Fi"). The driver picks the USB entry while there is one and moves the phone to Wi-Fi when the cable is pulled. ## What is built `internal/driver/ios` implements `driver.Provider`, `driver.Device`, `driver.Toucher`, `driver.LockReporter`, `driver.Settler`, `driver.Waker` and `driver.Unlocker` on iOS 27. Below iOS 27, `WDADevice` implements `driver.Device`, `driver.Toucher`, `driver.LockReporter`, `driver.Settler` and `driver.Unlocker` (`wda_power.go`), and `AXDevice` `driver.Device`, `driver.LockReporter` and `driver.Settler` (see "iPhones below iOS 27"). | File | What | | --- | --- | | `tunnel.go` | Userspace CoreDevice tunnel: the `CoreDeviceProxy` lockdown service carries bare IPv6 packets; a gVisor netstack (TCP and UDP) runs them in-process. go-ios dials services through a 127.0.0.1 forwarder that speaks go-ios's userspace-tunnel preamble and only reaches the device's tunnel address | | `conn.go` | Tunnel plus RSD handshake. The first handshake announces the host (`HandshakeAsHost`), once per tunnel, as iOS 27.2 requires | | `coredevice.go` | CoreDevice RemoteXPC requests, always with protocol 629.3 / DDI version 2 (some features refuse the old version), with dial and request timeouts | | `offer.go`, `display.go` | The media stream offer (adapted from go-ios, MIT) and start/stop. Payload type 100's feature list is `FLS;SW:1;` instead of Xcode's `FLS;VRAE:0;SW:1;`: VRAE:0 makes the encoder drop frames under motion (pymobiledevice3's measurement) | | `rtp.go`, `sps.go` | RTP parsing; RFC 7798 HEVC (single NAL, AP, FU; PACI skipped) and RFC 6184 H.264 (single, STAP-A, FU-A) depacketizers; the codec is sniffed from the NAL header because payload type 100 carries HEVC. The display service's 14-byte slice trailer is stripped. SPS parsers for the picture size | | `media.go` | The long-lived stream: access-unit assembly, `driver.Packet`s (a config packet before each keyframe, PTS from the 24 kHz RTP clock), RTCP receiver reports every second (the device stops after 10 s without them), a per-frame receipt like Xcode's, picture loss indications when a subscriber needs a keyframe (repeated every 600 ms until one comes), loss handling, fan-out to `Stream` subscribers | | `hid.go`, `keys.go` | Universal HID touch reports (surface 257), Indigo button events (Home), a virtual HID keyboard registered with `createService` on a connection of its own for the length of one call (issue #65), ASCII to US-layout usages | | `multiinput.go` | On a phone with several displays: whether each input changed the picture, an error after touches that never show on the display in use, Home with the phone's answer and a swipe up when it does nothing | | `pasteboard.go`, `type.go`, `smartspace.go` | The pasteboard service (PULL and SET); `Type` by paste with the pasteboard saved and put back, the spaces iOS's smart insert adds to a paste taken off again, and the keys fallback with the Latin keyboard switch | | `power.go` | `Waker` (screen state from the backlight, Home) and `Unlocker` | | `device.go`, `idle.go` | The session: one tunnel, and one media stream with its HID connection while anyone watches or Pinky sends input; taps and swipes in screenshot pixels and raw touches in video pixels, mapped to the panel's 0..65535 range | | `axp.go`, `rax.go`, `observe_rax.go` | `remoteAXService`: a keyed-archive reader and writer for AccessibilityPlatformTranslation's classes, the session (requests one at a time, the walk, the trees the phone pushes, what walks told of each element), and `Observe` from it (pushed tree or walk, roles, labels, frames in pixels, other processes' elements, the keyboard, what covers the app) | | `appid.go` | The app on screen: by process, by name, and beneath Face ID's prompt and other system UI | | `springboard.go` | SpringBoard's alerts and banners over the app, as events, and banners as elements too | | `capture.go` | Elements the screenshot does not show (`not_in_screenshot`) | | `ax.go`, `caption.go`, `observe.go` | The fallback: the inspector walk, caption parsing into label, role and states, OCR placement, keyboard detection, the foreground app | | `capturegate.go` | When a read may take its picture for OCR (`MayCapture`): as soon as the app in front and the lock state are known, or after the whole read | | `ocr.go` | Names for the shared OCR engine in `internal/ocr` (since 2026-11-08, shared with the Android driver): Vision through purego on macOS, Windows.Media.Ocr through WinRT on Windows, `unsupported` elsewhere; `ocrExtras` and `keyboardFromOCR` are `ocr.Extras` and `ocr.KeyboardTop` | | `screenshot.go`, `apps.go`, `lock.go`, `settle.go`, `provider.go`, `display_info.go` | The rest of the interface | | `diag.go`, `cmd/iosctl` | Diagnostics and the debugging command | | `version.go` | Which way to drive a phone, by its iOS version | | `wda.go`, `wda_observe.go`, `wda_device.go`, `wda_run.go`, `replaytouch.go` | The WebDriverAgent path (iOS 17.4 to 26): the HTTP client over usbmuxd, its tree as elements, the device, starting the runner's test, the mirror's touches played back as gestures | | `ax_device.go` | The accessibility path (iOS 17.4 to 26, nothing installed) | | `ddi.go`, `mount.go`, `prepare.go`, `host.go` | Preparing a phone without Xcode: finding the developer disk image on the computer, mounting it, the Developer Mode switch, pairing on Windows, the background jobs and the steps in `Info.Setup`, and what the computer lacks (usbmuxd) | | `framestream.go`, `mjpeg.go`, `h264_darwin.go`, `h264_windows.go`, `h264_ffmpeg.go`, `annexb.go`, `h264_other.go`, `settle_frames.go` | Frames (JPEG from WDA, screenshots) to H.264 — VideoToolbox on macOS, FFmpeg's Media Foundation encoder on Windows — for the mirror and the recorder, and settling by them | Coordinates are screenshot pixels, the screen the right way up in any orientation (see "Orientation"): 1125x2436 on an iPhone 13 mini in portrait, which is the size UIKit renders (375x812 points at 3x); the panel is 1080x2340 and the video 1136x2464. `Toucher` points are video pixels (the contract since 5e83552). Matching decoded frames against screenshots (row and column brightness profiles, correlation 0.998 and better on Settings and Calculator) showed the video is the screen at 1:1 in the top-left corner with black padding, 11 columns on the right and 28 rows at the bottom, so a video point is the same screen point and `Touch` only clamps points in the padding to the screen's edge. A video smaller than the screen would be scaled back up; no phone tested does that. ### Decisions - **One media stream while anyone watches or Pinky sends input.** HID input is dropped unless a stream from the same client runs, so `Open` starts the stream; `Stream` returns a subscriber view that starts at a fresh keyframe, and closing it does not stop the device's stream. While it runs, iOS tells apps the screen is shared (`UIScreen.isCaptured`), shows the screen-sharing indicator and keeps the camera and microphone from apps, and some banking apps refuse to run (DBS digibank, China Merchants Bank; field test item 5). So the stream stops after 30 s with no view open (the mirror, a task's recording) and no input, and the next view or input starts it again: 0.8 s measured, for a Home press. Restarting streams often can wedge the device's media daemon until a reboot (pymobiledevice3 reports it); a task's recording keeps a view open, so the stream rarely stops in the middle of one. - **Own display, HID and CoreDevice clients on top of go-ios's transport** instead of go-ios's `display` and `hid` packages: go-ios discards the stream answer (SSRCs and the RTCP port, needed for receiver reports), refuses userspace tunnels for media, and its HID package has no buttons or keyboard. - **go-ios from main** (`v1.3.3-0.20261103131045-6910f3b585ad`): the display and HID packages and the RSD host announcement are not in a release yet. Pin a release once one has them. Since 2026-11-09 go.mod replaces it with the same commit plus one, from the owner's fork (`github.com/DIYgod/go-ios`, branch `pin-device-certificate`, 53a1fc8), which pins the phone's TLS certificate (next item). The change is sent to go-ios as [danielpaulus/go-ios#892](https://github.com/danielpaulus/go-ios/pull/892); drop the `replace` once go-ios has it. - **The phone's TLS certificate is pinned to the pair record.** go-ios accepted any certificate in lockdown's TLS (`InsecureSkipVerify`, nothing else), so something on the network that answered in a paired phone's place (a taken-over address, or a replayed `_apple-mobdev2._tcp` record whose TXT and MAC are broadcast in the clear) got every lockdown session, every service and the tunnel over CoreDeviceProxy, and with it the input sent to the phone. The fork's go-ios accepts the peer only when its certificate carries the public key of the pair record's `DeviceCertificate`, which the host issued for the phone's own key when they paired; the handshake proves the peer holds the private half. Every TLS client connection goes through it: lockdown sessions, services, and Pinky's tunnel, whose RemoteXPC traffic has no TLS of its own. A pair record without a `DeviceCertificate` is accepted as before. On the iPhone 13 mini (iOS 27.0.1, 2026-11-09), over USB and over Wi-Fi, the phone presented the `DeviceCertificate` byte for byte on a lockdown session and on CoreDeviceProxy, and connected with the pinning on (`TestDeviceCertificate`, build tag `device`, read only). Not covered: WebDriverAgent's HTTP below iOS 27 where usbmuxd carries it to the phone's port 8100 without TLS (on Wi-Fi that is only a WDA already running when Pinky opened the phone: the one Pinky starts there it reaches through its tunnel, GitHub PR #53), and go-ios's remote pairing tunnel, which Pinky does not use (its pair-verify does not check the phone's signature). Against the first, Pinky's own search on Linux and Windows keeps a listed phone at its address while lockdown answers there and skips records that name an address no phone can have ("A record is no proof of an address" below); macOS's usbmuxd picks the address itself. - **HID timestamps in 24 MHz ticks.** go-ios and pymobiledevice3 stamp reports in nanoseconds. With those, a 12 ms key press opened the press-and-hold accent menu (as if held for half a second) and swallowed the rest of the text; stamping in 24 MHz ticks, the phone's mach time unit, fixed it. So the phone reads mach ticks and nanoseconds make every interval look 41.7 times longer. Touch looked fine only because paging goes by distance; fling velocities were affected too. - **Element frames from `remoteAXService`, the inspector and OCR as the fallback.** See "remoteAXService" below. `Observe` falls back when the service fails to connect or answer, and tries it again after 30 s. - **A pushed tree when it is recent, else a walk.** The phone builds and pushes the screen's tree after every change, and requests wait while it does (1–4 s on screens with many images), so a walk right after an action is slow. `Observe` walks and waits for the push at once and takes whichever comes first. A push counts only if its first tree arrived at least 1 s after Pinky's last input or launch, the Observe comes within 6 s of that input, and the push's app is the frontmost app the walk's first request found: a push can arrive late, after the next input, and then describes the screen before it (a regression run caught a late home-screen push answering for Settings before this rule). After 6 s the walk alone answers, because the screen may have changed in ways the phone does not push (a timer's text). - **Roles from AutomationType.** The walk and the second push give `XCUIElementType` (`Key` 20, `Icon` 44, `Cell` 75, `PickerWheel` 39, ...), the same types XCUITest reports. A first push lacks it; then traits and the translator's own role decide (`AXPAttributeRole` 2 button, 3 switch, 4 container or status bar item, 6 heading, 7 image, 9 a tab of iOS 27's floating tab bar, 13 adjustable, 14 static text, 15 text and search fields, as seen). The tab role makes a tab whatever the type (the floating tab bar lists its items as buttons, outside any tab bar, with a value of 0 or 1 that is not shown); the adjustable trait (picker wheels, sliders, the home screen's page pill) adds the state `adjustable`. An element's value reads as its value description when it has one: a picker wheel's value is its row's index in a long looping list ("5003"), the description the row ("12 o’clock"; field test item 22). - **First pushes only with elements walks have described.** A first push lacks every element's type and identifier, containers' included, so a read from it gave elements other roles, ids and ancestry than the next read, and so new refs (field test item 13: Ethena's nav bar 55 <-> 77, Clock's Delete Alarm button -> text). The session remembers each element's type and identifier by its object ID, which is the same in pushes, walks and later sessions, from every walk and second push, and fills them into first pushes. A first push with an element no walk or second push has described is not used: the read waits for the second push (0.55–0.65 s after the first, on Clock) or the walk. Checked: 12 of 12 elements of Settings kept their refs from a read of a push to a read of a walk. - **Covered elements are left out.** Elements whose middle lies under the keyboard (its keys' box), iOS 27's floating tab bar (its tabs' box with a 6 pt margin, down to the bottom edge, where the home indicator is) or an alert or banner SpringBoard shows over the app are left out, or keep only the part left uncovered when that is at least a quarter of them, as Android's companion does for windows above an app: a tap by ref goes to the middle of the bounds (field test item 23: a swipe on a row behind the tab bar switched tabs). - **Roles from captions on the fallback path.** The audit daemon answers attribute queries (label, value, traits) only for the element holding the inspector focus, which would cost a round trip per element on top of the move. The walk keeps its moves pipelined and parses VoiceOver's caption ("Alarms, Button, Selected, Tab") with English and Simplified Chinese trait words; other languages get the whole caption as the label and the role `other`. - **The provider asks a phone again only when needed.** The gateway lists devices every 2 s. `List` keeps each phone's answer until usbmuxd's entry for it changes (a new `DeviceID` on every attach) or, for a phone that is not online, for 30 s; an inspection cut short by its context is not kept. `Open` reuses the answer, and refuses a phone reachable only over Wi-Fi while Pinky does not use Wi-Fi or the person has not said the phone is theirs (see "Wi-Fi"). - **Foreground app from the elements.** The frontmost application's process id (on the inspector path, an element handle's first four bytes) goes through the process list (`listprocesses`, its executable URL decoded) and the installation proxy's bundle paths to a bundle id and name. A process the catalog does not list reads the catalog again once (an app installed since); one that is no app is not looked up again for 30 s; a failed lookup falls back to the catalog by the element's label when one app has that name (field test item 27: the first read of a session said "Plasma One" without its id, and blocking goes by id). `Open` reads the catalog in the background. - **The app beneath system UI.** While Face ID's prompt or its fade-out is in front of an app, the frontmost application is SpringBoard or a process with no name, while the screen still shows the app (field test item 14: OCR read Trust's balances under "Home Screen"). When the frontmost process is not an app from the home screen and the phone is not locked, the read names the app the system has in the foreground: one the inspector session saw come to the foreground since the last read that named the app in front, else that app, unless the daemon said since that it left; SpringBoard with its icons on screen is the home screen. On iOS 27 the inspector's app monitoring sends no states when it turns on, only changes after that, and it reports apps iOS wakes in the background (China Merchants Bank, seen while Settings was in front) in the foreground too, hence the rules; `Open` connects the inspector session for it. Turning monitoring off and on again does not list the apps again (it only empties the states). - **No picture of a screen the gateway refuses.** A read takes a picture for OCR (and on the inspector path to place the elements) only once `driver.ObserveOptions.MayCapture` allows the screen it reads: not a locked phone, not a blocked app the person did not allow for the task, not an app Pinky cannot name (`capturegate.go`, 2026-11-08). Before, the picture was taken while the tree was read, and the gateway dropped the text of a blocked app. Waiting for the whole tree would add the walk to every read with OCR (0.2–0.3 s on the cable, seconds over Wi-Fi with screenshots), so the remoteAXService path asks as soon as the walk's first request names the frontmost process (18 to 19 ms over USB) and lockdown the lock state, which run side by side, and the picture starts while the tree is still read. It asks for every app the read may name (`appCandidates`, `screenApp`'s cases without the tree): the app, or for SpringBoard both the home screen and the app beneath it, since only the tree's icons tell the home screen from Face ID's prompt over a bank's app; the picture starts early only when each may be captured. A process the process list does not name waits for the whole read, which finds the app by the name in the tree. WebDriverAgent names the app in its own request, long before the page source. The read asks again with the screen it returns and drops OCR's text when the answer changed (the app changed while it read). The inspector path, a fallback, decides after its walk. The gateway sets `MayCapture` on every read, OCR or not, because the inspector path always takes a picture. `ObserveStats.Capture` and `CaptureWait` say when the picture started. Unit-tested (`TestCaptureGate`, `TestAppCandidates`, which checks `screenApp`'s answer is always among them, `TestWDAObserveCapturesOnlyWhatMayBe`, which counts a fake WebDriverAgent's screenshots: none of the blocked app or a locked phone); not yet timed on a phone. - **Screenshots from the instruments service first**, CoreDevice's screen capture second. Both got stuck on the device at times (most likely after a test process was killed mid-request); the fallback and dial timeouts keep calls from hanging. - **Lock state from lockdown's `PasswordProtected`**, the value `devicectl` reports as `passcodeRequired`. iOS 27's developer disk image does not implement CoreDevice's `getlockstate` (the feature is listed but `dtdeviceinfod` has no implementation). - **Typing pastes** (the owner's decision). `Type` saves the pasteboard, sets it to the text, presses Command-V on the virtual keyboard, waits until the focused field's value changes (read through `remoteAXService`: the frontmost app's focused element, attribute 19, then its value), and puts the pasteboard back. It chooses before touching the field, so text it cannot type leaves the field as it was. Restoring waits for the paste because restoring first would paste the person's own clipboard into the field (and show it to the agent); without `remoteAXService` it waits 0.5 s. What is saved stays in memory for one call and is never logged or shown; `TypeStats` says only how it went. - **Pinky's pasteboard writes stay on the phone.** The phone's daemon (the DDI's `dtpasteboardd`) writes with `UIPasteboard setItemProviders:localOnly:expirationDate:` and passes `localOnly` exactly when the SET carries `sourceMetadata` (disassembled from the DDI's `CoreDeviceUtilities`: `localOnly = sourceMetadata != nil`, the branch that also records "sync metadata"). pymobiledevice3 sends none, so its writes go out through Universal Clipboard and replace the clipboards of the owner's Mac and iPad. Pinky always sends `{nonce, pasteboardName, changeCount}`. Checked: the Mac's pasteboard change count (a number; its contents were never read) stayed the same through every write. - **What the pasteboard keeps.** `Type` reads each item's first representation (`promiseSecondary`, always quick), then all of them (`allResolved`) for up to 3 s; if that read stalls (an app listing a type it cannot produce makes the daemon stall for about 64 s, pymobiledevice3 found with Notes) it keeps the first representations. The daemon refuses to read an item marked as coming from another device through Universal Clipboard (`com.apple.is-remote-clipboard`), a password (`org.nspasteboard.ConcealedType`) or transient (`org.nspasteboard.TransientType`, `de.petermaurer.TransientPasteboardType`, `org.nspasteboard.AutoGeneratedType`), with CoreDeviceError 26006; such a pasteboard cannot be saved, so `Type` empties it after pasting instead of leaving the text there. The test phone held a Universal Clipboard item when this round started. A pasteboard that changed since Pinky's own SET (someone copied, or a Universal Clipboard item arrived) is left as it is. - **Keys only when the pasteboard does not do** (no pasteboard service, a read that timed out, or `iosctl type -keys`): printable ASCII through the virtual keyboard, **as a U.S. hardware keyboard layout has them**: keyboard reports carry key codes, and the phone puts its own layout's characters on them (Colemak, Dvorak, AZERTY, QWERTZ put other letters there). Before typing, a probe letter (deleted again) shows whether the active keyboard composes: Pinyin puts a candidate bar into the app's own tree (cells whose identifier is the candidate, "啊"), sometimes only after a few hundred milliseconds, so the probe looks for up to 1 s. If it composes, Control-Space switches to the other keyboard for the text and back afterwards, 300 ms after the last key (switching back at once let Pinyin take the English word just typed into a composition, "Cloc k", in one run); if the other keyboard composes too, the call returns `unsupported`. Latin keyboards capitalize the first letter of a field (the hardware keyboard's Auto-Capitalization setting), which keys cannot avoid; pasting is not affected. On a Latin keyboard a second probe, semicolon's key (deleted again), checks the layout: ";" on a U.S. layout, a letter on Colemak ("o"), Dvorak ("s"), AZERTY ("m"), QWERTZ ("ö") and most others; the first probe catches AZERTY already ("a" arrives as "q"). Another layout switches to the other keyboard as a composing one does, and if that is no U.S. Latin keyboard either, `Type` types nothing and says why (`layoutError`, `unsupported`): keys would enter other text, so try again, as the pasteboard may answer then, or set the hardware layout to U.S. The probes go before the field is cleared, at the cursor, and what they typed is found by comparing the field before and after (`arrived`). `TypeStats.Trace` records the probes. - **The field is read back after every `Type`** (`checkTyped`), by paste or by keys, until it reads as the text (for up to 1.5 s): the whole value, or for appended text the text inside the value and no more than the old value and the text together (an empty field's value is its placeholder, which goes away). Text that reads otherwise is an error, which agents get as `uncertain` with why: by keys, iOS's corrections and capitalization or a layout the probes missed; pasted, a field that formats or cuts what it takes. A field that has no focus, or whose value did not change or reads empty, is an error too (GitHub issue #65). A focused field that cannot be read (an app that draws its own: Xianyu's search box has no value, and reads as the text of its placeholder, or as a button with the text, GitHub issue #76) is checked by what the app shows instead (`typeseen.go`): a walk before typing (after the field is cleared) and after, of the app's own process outside the keyboard (the keyboard's suggestion bar offers the pasted item, "Paste from dtpasteboardd"). One more element showing the text counts as typed (`Checked` "shown"); the app showing exactly what it did, both walks complete, is an error that names iOS's Allow Paste prompt: such an app reads the pasteboard itself, so iOS asks the person, and unanswered nothing is pasted. The prompt shows in neither the app's tree nor SpringBoard's overlays nor the screenshot (the reporter of #76 looked), so Pinky cannot tell it is there; the error asks for `phone_handoff`, Don't Allow (the pasteboard is already put back, so Allow Paste would paste the person's own clipboard) and Settings › the app › Paste from Other Apps › Allow. Standard text fields take Command-V as the person's own paste and never ask. Anything else, and a secure field (its value is bullets), leaves a note on the settled result instead (`driver.InputNoter`, additive: the gateway adds the note of a `type` to its message). OCR is not used for the comparison: a screenshot in `Type` would bypass the capture gate. `TypeStats.Checked` says which. - **Waking with Home, never the side button.** The screen's state is CoreDevice's display info `backlightState`, read from the primary display's entry and, when that has none, from the top of the reply, where iOS 27.2 moved it (seen on an iPhone17,1 and an iPad13,4): `activeOn` and `activeDimmed` count as on, `off` and `inactiveOn` (the always-on display of newer phones) as asleep. `Wake` does nothing on a screen that is on: Home would leave the app. - **Unlocking.** `Unlock` wakes the screen, returns at once if lockdown says the phone is unlocked, gives a screen that just lit up 0.5 s, swipes up from the bottom edge, and waits up to 3 s for the passcode pad (buttons for all ten digits, or a secure field for an alphanumeric passcode; it also stops there if Face ID unlocks the phone). Without the pad it swipes again, up to three times, waking the screen again if it went dark: a lock screen with notifications took the first swipe for its notification list once (field test item 17), and a swipe types nothing. Without the pad after that it returns `ErrPINNotEntered` having typed nothing. It types the passcode on the virtual keyboard 120 ms per key, checking before each key that lockdown still says locked (once it does not, the keys would go to the app in front), presses Return unless the passcode is four or six digits (those submit themselves), and waits up to 4 s for lockdown to say unlocked, else `ErrLocked`. It never types the passcode twice. - **Dropped connections are dialled again.** The phone closes service connections at times: when it locks in a long session (deviceinfo, which waking and unlocking use; field test item 9) and after the pasteboard service sat idle (item 6). A CoreDevice request that fails drops its connection and the next one dials again (`cdService`); `ScreenOn` retries once at once, and `Type` reads the pasteboard a second time on a new connection before giving up. When the pasteboard service is there but still does not answer, `Type` says to try again instead of `unsupported`: the agent in the field test took "unsupported" to mean the iPhone can only type English. - **Smart insert's spaces are taken off.** iOS adds a space when text is pasted next to a word, so `Type` with Append made "Wall" and "paper" read "Wall paper", "Wi" and "-Fi" "Wi -Fi", and text pasted between two words got a space on each side (reported by a person using Pinky, reproduced on the iPhone 13 mini, iOS 27.0.1, Settings' search, 2026-11-08). Chinese next to anything got none, nor did text that already starts with a space; `Type` without Append clears the field first, so nothing is next to the paste. `Type` compares the field's value after the paste with the value before (`smartSpaces`): when it reads as the text with such spaces added, and in only one way, it takes them off with the keyboard, Backspace for one after the text (the cursor ends up after it) and Left over the text, Backspace and Right back for one before it, and checks the value again. On the iPhone that gave "Wallpaper", "Wi-Fi", "hellobigworld", the cursor where it belongs (a further paste landed after the text), for 0.2 to 1.1 s more per call. Text the arrow keys cannot be counted over (combining marks, emoji sequences such as a skin tone, flags) keeps a space before it; `TypeStats.Spaces` says what happened. The WebDriverAgent path types with XCTest's keyboard, not by paste. - **A locked phone that cannot be read reads as locked.** A locked phone can stop answering accessibility requests, and the inspector walk then times out too (field test item 9). When `remoteAXService` fails and lockdown says locked, `Observe` returns a locked screen with no elements, so the agent hands the phone over instead of failing. - **SpringBoard's alerts and banners are events.** They are not in the frontmost app's tree (field test item 7: a timer that went off showed only as OCR fragments). SpringBoard's application element answers whichever app is in front (request type 1 with its pid); while an app is in front its children are the status bar's items, which carry trait bit 23, and what SpringBoard draws over the app. Each read looks at them in the background, and each overlay becomes one event while it stays: an alert (`Alert`/`Dialog` type) a `dialog` event, whose elements are listed too so that an agent can press its buttons; anything else with a button a `notification` event, with the bundle id of the app it names first when one app has that name; the rest a `toast` event. Since 2026-11-09 a notification is also listed as one element on every read while it shows: a button with its whole text, at the overlay's place, with ID `pinky:notification:` (`screen.NotificationID`), so that the gateway hides a blocked app's banner and covers it in screenshots, and masks a code in it (docs/notes/gateway.md, "iPhone banners"). Only the phone's quiet screens were checked (no false events in Settings, Clock and the home screen): no banner or alert could be raised during the test, so their shape in SpringBoard's tree is not confirmed. - **Elements the screenshot does not show.** Some apps draw part of their screen in a layer iOS leaves out of screenshots, recordings and mirroring (field test item 21: Trust's passcode keypad was in the tree and blank in the mirror). A read with OCR compares its screenshot with the tree: when at least three labelled elements cover areas of one flat color (every pixel or two sampled), with no OCR text in them, they carry the state `not_in_screenshot`; reads without a screenshot keep what the last such read of the app found, so an element's states do not change with the kind of read. No false marks on the home screen, Settings, Clock and Calculator (whose thin operator symbols a sparse grid missed at first); the keypad itself is covered by unit tests only (banking apps are off limits). Taps on such a keypad are refused under the credentials rule (`policy.HiddenKeypad`: nine or more digits among `not_in_screenshot` keys; the area of the digit keys plus a quarter key takes no taps, Face ID and delete included): in the field run of 2026-11-10 a tap on Trust's FaceID key through the helper cost a passcode attempt. - **Wi-Fi is on by default, and asks first.** See "Wi-Fi" below. `Open` uses the USB entry unless `Info.Connection` is "wifi". - **Values in identifiers.** Calculator's display has no label or value, only the identifier `StandardInputView;value:144`. An identifier with `;value:` gives its name as the identifier (stable, for refs) and the rest as the value, and such an element reads as text, so the display is `text "144"`. ## How to run Hold the device lock for every command that touches the phone. ```sh go build -o /tmp/pinky-iosctl ./internal/driver/ios/cmd/iosctl scripts/devlock.sh /tmp/pinky-iosctl -udid rsd scripts/devlock.sh /tmp/pinky-iosctl -udid observe -ocr scripts/devlock.sh /tmp/pinky-iosctl -udid stream -seconds 10 -out /tmp/pinky-stream.hevc scripts/devlock.sh /tmp/pinky-iosctl -udid latency -vertical # on a still list such as Settings scripts/devlock.sh /tmp/pinky-iosctl -udid tap 285 2283 scripts/devlock.sh /tmp/pinky-iosctl -udid swipe 900 1218 225 1218 -ms 300 scripts/devlock.sh /tmp/pinky-iosctl -udid key home scripts/devlock.sh /tmp/pinky-iosctl -udid type -append "hello 你好" scripts/devlock.sh /tmp/pinky-iosctl -udid launch Calculator scripts/devlock.sh /tmp/pinky-iosctl -udid screenshot -out /tmp/pinky-shot.png scripts/devlock.sh /tmp/pinky-iosctl -udid screen # is the screen on scripts/devlock.sh /tmp/pinky-iosctl -udid locked # lockdown only, no session /tmp/pinky-iosctl mux # usbmuxd's entries, no phone contacted ``` `iosctl unlock` reads a passcode from standard input and unlocks: only for the owner (see "What the owner should try"). `iosctl orient` prints what the phone says about its orientation (the display info, the frontmost app's frame, the first elements, Observe's screen, the RTP header extensions) and saves a screenshot and a few seconds of video in a folder under `/tmp/pinky-*`; `iosctl stream` counts the packets by the turn they carry. `iosctl devices` lists every phone usbmuxd sees and opens a tunnel to each to check its services, so do not run it while other sessions use their phones. Keep captures under `/tmp/pinky-*` and delete them afterwards. `iosctl observe` prints where the tree came from (`remoteax`, `remoteax-push` or `inspector`) and the timings; `ObserveStats.Trace` says what the remoteax path waited for. `iosctl type` prints how it typed (paste or keys) and what happened to the pasteboard, never the text. Each `iosctl` command is a new process with no earlier input, so it always walks; the device tests below measure the push path. Tests: `go test ./internal/driver/ios/` runs the unit tests (RTP and SPS parsing, depacketizers, access-unit assembly, the offer against a captured one, the keyed-archive reader and writer, trees from pushes, elements and roles from the tree including other processes and the keyboard, caption parsing, keyboard detection, OCR placement, HID reports, the provider's cache, video-to-screen points) without a phone. The real-phone tests share one session and use only the home screen, Settings, Calculator and Clock: ```sh PINKY_DEVICE= scripts/devlock.sh go test -tags device -count=1 -v ./internal/driver/ios/ ``` All 16 pass on the iPhone 13 mini (inspect, stream, home screen observe, swipe and raw touch, apps, launch Settings by name, Calculator tap, screenshot, lock state, unsupported keys, latency, typing by paste into Settings' search with the pasteboard checked after each call, typing by keys with the keyboard switch, waking, a Calculator key tapped where Observe puts it and the display read back as `7`, Observe timing after launches). `TestDeviceConnection` measures one connection in a session of its own: `PINKY_CONN=usb` or `wifi`, run alone with `-run TestDeviceConnection`. `PINKY_NO_SETTLE=1` makes the timing test observe right after each launch; `PINKY_SHOTS=1` saves the typing test's screenshots under `/tmp/pinky-ios` (delete them afterwards). Links (`driver.LinkOpener`, `links.go`, 2026-11-09). `phone_launch` with a link no longer goes through `Launch`. `LinkApp` names the app a link opens in before anything opens, and the gateway judges that app; `OpenLink` opens the link in that app by bundle id and nowhere else (`not_found` when `LinkApp` now names another). - Web links (`http` and `https` with a host) open in Safari by bundle id, whatever browser the person made the default: that is the app the rules judged. A universal link opens in Safari too, not in the app that claims its domain. - Another scheme opens in the app whose `CFBundleURLTypes` declares it, from the installation proxy's list (the catalog's `schemeApps`, read again once when no app declares the scheme, for an app installed since). When several declare it, Apple's own app (`ApplicationType` System) wins; among the others, which one iOS picks is not defined, so `LinkApp` returns `driver.LinkChoiceError` and the gateway refuses. A scheme no app declares is `not_found`, also for a scheme iOS may handle without an app declaring it (`tel:` may be one): refused rather than guessed. - CoreDevice (Device Hub on iOS 27, the developer services on 17.4 to 26): the app's launch request carries the link in `payloadURL` (the field `devicectl device process launch --payload-url` fills; the same key inside `platformSpecificOptions` as `__PayloadURL` did nothing). The contributor checked web links this way (pull request #3) on an iPhone with iOS 27: Safari showed the link's host. An app's own link through `payloadURL` is not checked on a phone yet. The device test was left out, since it opens Safari, which holds the owner's tabs and history (docs/WORKING.md). - WebDriverAgent: `POST /url {"url", "bundleId"}` (XCTest's `openURL:usingApplication:`), which WebDriverAgent has from 4.11.0 (Xcode 14.3 and later). Earlier ones ignore `bundleId` and open the link wherever iOS would, which the rules did not judge, so `OpenLink` reads `/status`'s build version and returns `unsupported` for them. - A simulator has no installation proxy: `WDADevice.SetLinkSchemes` gives it the scheme table, and without it a simulator opens web links only. `simSchemes` in `wda_sim_test.go` builds the table from `simctl listapps` and each app's Info.plist; Simulators added as phones should do the same. - Shortcuts runs a shortcut from a link under `shortcuts:`, `shortcuts-production:`, `workflow:` and `workflow:` (its Info.plist on the 26.5 Simulator). The policy refuses those schemes, and any link that would open in `com.apple.shortcuts`. Other schemes Apple's apps declare there: Passwords `otpauth`, Wallet `wallet`, Settings `prefs` and `app-prefs`, Files `shareddocuments`, Calendar `calshow`. `TestWDASimLinks` (iOS 26.5 Simulator, WebDriverAgent 16.14.0): `https://example.com/` opens in Safari in 0.85 to 2.3 s, also with Settings in front and with Safari not running; `calshow://` in Calendar in 2.1 to 5.5 s; a scheme no app declares is `not_found`, and `calshow://` in Safari is `not_found`. Calendar asks for the location and to send notifications when it first opens, and the alerts would cover the screen for the tests after it: the test grants the location (`simctl privacy grant location com.apple.mobilecal`), turns the notifications down (`POST /alert/dismiss`) and quits Calendar. Right after an app opens another, WebDriverAgent can report the new app's bundle id with the old app's tree (Safari named Settings, one element); the gateway's settling waits that out, and the test waits for the id and the name to agree. ## Details and data ### Tunnel - The device gives each tunnel its own /64 (macOS's own tunnel was `fd3e:…`, ours `fd4f:…`), so both coexist. macOS 27's own tunnel is not routable from other processes (no utun route), so Pinky cannot borrow it. - The device grants the MTU asked for (16000), which keeps per-packet overhead low. Over USB a 6.9 MB screenshot arrives in about 0.75 s including the phone's PNG encoding, so the userspace stack moves at least 9 MB/s; gVisor's TCP receive autotuning is on. - Sending the device much data needs a small MTU. With 16000, the first full-size packets of an upload (the developer disk image) went out and the device then sent nothing more on the tunnel, not even their ACKs: the tunnel's counters stopped, gVisor retransmitted with backoff, and 18 s later the device's TCP gave up ("Operation timed out", 2026-11-08, iOS 27.0.1). The tunnel that mounts the image asks for 1280 (`uploadMTU`), as go-ios's tunnels do; the image then goes over in half a second. Other tunnels keep 16000: their packets to the device are small. - `HandshakeAsHost` once per tunnel matters: without the host's RemoteXPC version flags, services that send large replies (screenshots) crash on the device, and cryptexd logs "Invalid or missing remote device connection version flags". The phone takes the announcement only once it has settled TLS on that RSD connection; closing the connection right after sending it lost it on about half of the tunnels ("Device must renegotiate TLS"). The connection now stays open for 2 s (`rsdLinger`), and every tunnel got its flags. ### Video - The answer's `streamConfig`: `RxPayloadType` 100 (HEVC inside), `Framerate` 60, `KeyFrameInterval` 0 (no periodic keyframes), `RXMaxBitrate` 6000000, `RTCPTimeoutEnabled` with 10 s, `CustomWidth`x`CustomHeight` 1136x2448 while the SPS says 1136x2464 (the SPS is right; ffprobe agrees). - The phone also creates a virtual "Wireless" display (id 2, 1136x2448) for the stream. - The offer asks for `avcMediaStreamNegotiatorMode` 5, the mode Xcode's Device Hub, go-ios and pymobiledevice3 use: 4:2:0 with a 6 Mbit/s cap. Mode 7 (Remote Desktop) streams HEVC 4:4:4 at up to 60 Mbit/s, which Device MCP uses. The frame rate is 60 either way; [mirror-smoothness.md](mirror-smoothness.md) has what mode 7 would take. Over Wi-Fi (2026-11-09, `pinky-mirrorbench`) the iPhone 13 mini gave 50 to 54 changed pictures a second while a list moved, in bursts, 278 to 283 ms from touch to picture with a second or more now and then, and took 1.05 to 1.52 s to open its stream again on the open phone (once no picture in 20 s). - RTCP is multiplexed on the receiver port; the phone sends a sender report about every second. Its NTP time against our arrival gives 55–95 ms one way, if both clocks agree. - After a change the phone sends 60 fps for about 5–7 s, then stops sending frames until the screen changes again (a status bar clock tick restarts them); its RTCP sender reports keep coming every second throughout. While frames do come, a still screen gives identical frame sizes and motion varying ones (5–75 KB). The size of a still frame depends on the encoder's current bitrate: about 1–2 KB when idle, but after motion the encoder raises its bitrate and pads still frames to a constant 10 KB for seconds. So "no frames, or frames of one size" marks a still screen, not small frames; the settle detector uses that (step 0 item 4). The stream's watchdog counts RTCP too: only silence on both for 5 s means the phone stopped the stream. - A subscriber that joins while frames have stopped gets the parameter sets and a fresh keyframe within about 25 ms of its request. - Settle data from Settings: tapping "General" changed the first frame 331 ms after touch-down (the tap is held 60 ms, so about 270 ms after touch-up, including the app's own response); the push animation's frames were 58–74 KB; frame sizes became constant about 2.3 s after the tap. A fling in a list keeps the picture moving for about 2.5 s (measured as frame-to-frame pixel difference in the decoded video). Home screens with widgets (Photos, Clock) never look still. - The audit daemon's screen and layout change notifications arrived in one session and not in others, so they are only a hint for settling. - Without loss there are no keyframes after the first. Subscribers that join later ask for one; the phone ignores a request that comes right after a keyframe, hence the retry. - Packet loss happened only over Wi-Fi and while large screenshots shared the tunnel; the gVisor UDP buffer never overflowed. ### Input - Touch positions are 0..65535 on each axis of the touch panel, in the frame of the device held the usual way up, whatever way it is turned; see "Orientation". - `Key(back)` swipes from the left edge at mid-height over 250 ms. It starts at x=0 so the system gesture wins; a stray drag cannot select anything, but avoid screens where a drag at mid-height matters. - The virtual keyboard makes iOS treat the phone as having a hardware keyboard while it is attached, and hides the on-screen keyboard, for the person too (GitHub issue #65, below); the software keyboard still showed in Settings until a key was pressed. It goes away only when the connection that registered it closes, so it has a connection of its own and lasts one call: registered at the first key of a `Type`, `Key` or `Unlock` (about 0.35 s with the spare connection, the 300 ms wait included, and about 1 s without), removed when the call ends. `Type` without `Append` sends Command-A and Delete first. - Keys go through the active input method. On the test phone the Pinyin keyboard was active: "clock" typed as keys became a composition (Command-A did not select it, one Delete left "c lo c" with U+2006 spaces between the syllables), and Escape appeared to cancel it. That is why `Type` pastes (the owner's choice between pasting and switching keyboards); keys remain only as the fallback above. Command-V and Command-A work with Pinyin active as long as nothing is being composed. ### Pasteboard - The service: `com.apple.coredevice.pasteboardservice` (DDI `dtpasteboardd`, launchd label `com.apple.coredevice.dtpasteboardd`, entitlement `com.apple.private.CoreDevice.canAccessPasteboard`), bare XPC dictionaries with `command`, replies on the server-to-client stream; requests need the wants-reply flag. Messages: `PULL {pasteboardName, dataPolicy}` → `PULL_REPLY {pasteboard: {items, metadata, sourceMetadata}}`; `SET {pasteboardName, items, sourceMetadata}` → `SET_REPLY` with the new snapshot. Items are `{types: [UTI], data: {UTI: {data}}}`; a promised representation comes as `{isPromised, isAvailable, size}`. Metadata is `{nonce (the phone's boot session), pasteboardName, changeCount}`, and a PULL returns the `sourceMetadata` of the SET that made the current contents, so Pinky can tell its own write. - Errors come as `{error: {code, domain, userInfo}}` (CoreDeviceError). 26006: "The pasteboard contains type '…' indicating its contents are transient, confidential, or already synchronized by another mechanism, and was not synchronized." - `SET` with no items empties the pasteboard. Timings over USB: the first request on a connection 0.55 s (the RemoteXPC handshake; `Open` now connects the service in the background), then PULL 2–35 ms, SET 8–30 ms. - No paste permission prompt or banner appeared: Command-V from a keyboard counts as the person's paste. ### Keyboards - The test phone's language is English (Singapore); lockdown's `com.apple.international` `Keyboard` names the default keyboard (`en_GB@sw=QWERTY;hw=Automatic`), not the active one, which was Pinyin (Simplified Chinese). - With Pinyin active, a typed letter shows a candidate bar in the app's own tree (`Cell` elements labelled "啊, 叹词的啊" with the identifier "啊", and a "More suggestions" button). `AXPAttributeTextInputMarkedRange` (123) answered nothing; the field's value shows the uncommitted letters. - Control-Space switched Pinyin to English (UK) and back; two presses restore the keyboard. After hardware key presses iOS hides the on-screen keyboard until the field is tapped again. (Seen while the keyboard stayed attached for the whole session. Since Pinky removes its keyboard after each call, the on-screen keyboard comes back by itself about 1.5 s later: issue #65, below.) A probe's walk right after a switch took 1.1–1.8 s (the phone was pushing a tree). ### Wi-Fi Pinky uses iPhones and iPads over Wi-Fi by default since 2026-11-08 (Settings › Phones › Use iPhones and iPads over Wi-Fi; `ui.Settings.IOSWiFi`, `Gateway.SetWiFi`, `driver.WiFiFinder`). usbmuxd lists a phone over the network while it is on the same network as the computer and was paired with it with Wi-Fi allowed (Finder's "Show this iPhone when on Wi-Fi", Xcode's "Connect via network"); everything Pinky uses runs through that connection. **Turning Wi-Fi connections on** (`wifiallow.go`, 2026-11-08). usbmuxd lists a phone over the network only once its lockdown value `EnableWifiConnections` in the domain `com.apple.mobile.wireless_lockdown` is true, which is what Finder's checkbox and Xcode set; nothing sets it by itself, so a person who never ticked the box found that Pinky's Wi-Fi did nothing (reported on Discord after the release). While Pinky uses phones over Wi-Fi, `List` turns it on in the background for a phone on a cable whose lockdown values it read (it trusts this computer), once per attach: one lockdown session reads the value, sets it to true only when it is not, and reads it back; a failure is tried again a minute later (`wifiAllowRetry`). Turning Pinky's switch off, or removing the phone, leaves the value alone. Add phone › iPhone and Settings › Phones say so. Unit-tested (`TestWiFiAllowedOnCable`); read on the owner's iPhone 13 mini over Wi-Fi: already true, as on every phone usbmuxd lists over Wi-Fi. Setting it on a phone that has it off is not yet tried on a real phone. **Tried on the owner's iPhone 13 mini** (2026-11-09, Pinky.app 2026.1109.1, read only but for the owner's own clicks). Unticking Finder's "Show this iPhone when on Wi-Fi" (Apply) did not turn `EnableWifiConnections` off: a lockdown read after a replug gave true, Pinky wrote nothing (it logs a write), and once usbmuxd was restarted it listed the phone over Wi-Fi and Pinky used it there, Finder's box still unticked. The phone's `com.apple.mobile.wireless_lockdown` holds `EnableWifiConnections` true, `EnableWifiDebugging` false, `SupportsWifi`, `SupportsWifiSyncing`, `WirelessBuddyID`, `BonjourFullServiceName`, `InstanceName`; Finder's box is a Mac-side setting as far as usbmuxd is concerned. Xcode 27's Device Hub has no "Connect via network" switch. So the write itself is still untried on a phone that has the value off. What kept the phone off Wi-Fi after unplugging was usbmuxd, which had lost it already before it was plugged in (not listed over Wi-Fi at Pinky's start): Pinky said "Lost on Wi-Fi" a minute after the unplug, and Restart device service brought it online over Wi-Fi in 9 s. **Where it stands** (2026-11-09, after a person found it did not work for them and nobody could tell why). Every outcome is kept per phone and logged once when it changes: on already, turned on, failed (with the error), and not looked at with the reason (Pinky's switch off, the phone removed from Pinky, not trusting this computer yet, lockdown not answering); a phone listed over Wi-Fi or no longer is logged too. `List` puts it in `driver.Info.WiFi` (`WiFiConnection`, additive: `Connections`, one of `WiFiConnections*`, `Detail`, and `Seen`, usbmuxd listing the phone over the network now). The gateway turns it into the checkup's Wi-Fi item (`gateway.WiFiCheck`), also while the phone is offline, so an unplugged phone that does not show up on Wi-Fi says why: pass when it is on Wi-Fi or seen there, a warning to check the network (`FixSameNetwork`) when its connections are on but this computer does not see it, a failure with the error (`FixReconnect`), skipped while Pinky's switch is off (`FixWiFiSetting`) or before Pinky could look. Diagnostics list it on the phone's line ("wifi connections turned_on, not seen on wifi"). `TestWiFiConnectionState`, `TestWiFiCheck`, `TestDiagnostics`. **Asking first.** usbmuxd lists every phone ever paired that way, a family member's iPad among them, and Pinky used to prepare and open whatever its provider listed. Now the provider leaves a phone it reaches only over Wi-Fi alone until the gateway says the phone is the person's: one in the history store's `devices` table (phones Pinky opened, and phones it saw on a cable), one listed on a cable now, or one the person chose to use. Any other phone is listed with `driver.SetupConfirmWiFi` ("Found on Wi-Fi"), after one lockdown session that reads `DeviceName`, `ProductType` and `ProductVersion` (kept 10 minutes, `foundReread`); no Developer Mode switch, no disk image, no tunnel, and agents do not see it. **Use** (`Gateway.UseFoundPhone`) saves it among the phones Pinky has seen and prepares it at once; **Ignore** (`IgnoreFoundPhone`) removes it as Remove this phone does, so it stays off the list over Wi-Fi until it is plugged in (the provider is a `driver.Forgetter` now). A found phone that drops off usbmuxd's list stays listed for 3 minutes (`foundHold`), since an iPad that sleeps comes and goes; one that is gone longer leaves the gateway's list, as it was never the person's. **One phone on both connections.** The phone keeps its UDID, so its settings, PIN and history follow it. When the provider lists an open phone over another connection than the one it was opened over (`device.over`), the gateway closes the session and opens it again over the new one at once (`updateDevice`), waiting up to 6 s for the old session to close first (`closeWait`), and calls wait for it meanwhile (`device.moving`, 20 s; `gate`) instead of failing. A session whose cable is pulled fails its calls as `driver.ErrOffline` once its tunnel stopped (`Device.gone`); for a phone the person uses over Wi-Fi the gateway then waits 6 s for it (`dropWait`), and `phone_observe` reads again once it is back. **Locked phones are left alone.** An iPad locked on Wi-Fi sleeps: usbmuxd lists it for 10 to 30 s about once a minute, each time as a new attach. With the gateway opening it each time, its screen lit up every time (the person watching saw it), and the session ended with the listing. So while a phone is locked on Wi-Fi (`PasswordProtected` in lockdown's values, which inspection reads anyway) the provider lists it as waiting for a step, `needs_setup` with `driver.SetupUnlock` ("Locked on Wi-Fi" in the app while no agent waits for it), and opens nothing; it looks again every 3 s while usbmuxd lists it, keeps it listed for 3 minutes after usbmuxd drops it (`sleepHold`), and opens it once it is unlocked. An agent that connects gets `device_needs_setup` with `setup: unlock` and hands the phone to the person; the handoff ends once the phone is open. A phone without a passcode reads as unlocked, and would still be opened while it sleeps. A session that is already open stays open when the person locks the phone (checked: locked, then unlocked 10 s later, the session stayed). **Phones without a passcode** never read as locked, also asleep, so the rule above does not hold them back. A phone Pinky has never seen locked, back on usbmuxd's Wi-Fi list less than 3 minutes after it dropped off (`dozeWindow`), is not listed until usbmuxd has listed it for 30 s in a row (`awakeAfter`): an awake phone stays listed, a sleeping one shows up for 10 to 30 s (44 s once). A phone seen locked once has a passcode and does not wait. Unit-tested only. **Connecting.** While the gateway opens a phone (`device.opening`), it shows `DeviceConnecting` (additive in `api.go`): "Connecting", or "Moving to Wi-Fi" / "Moving to the cable" during a move, instead of offline; `Info.State` still reads offline. After Use, an iPad over Wi-Fi was in this state for about 3 s. **Phones usbmuxd lost** (`lost.go`). See "A phone usbmuxd lost" below for the bug. While Pinky uses phones over Wi-Fi and one the person uses is missing from usbmuxd's list, the provider reads that phone's HostID and Wi-Fi address from this computer's pair record (through usbmuxd; Android ids have none) and looks for `_apple-mobdev2._tcp` itself with the mdns package (which only asks). It matches instances to phones with `netmux.Match`, as the Finder below does: by the TXT record's `identifier` and `authTag`s first, then by the Wi-Fi address in the name, "@-supportsRP-26" (checked with `dns-sd -B`: the iPhone's 08:87:c7:2a:8a:db, the iPad's f0:2f:4b:e8:d3:4a, both matching their pair records). Matching by the address alone missed phones with a rotating private Wi-Fi address, whose name holds an address the pair record does not, and called them not on Wi-Fi instead of lost (#88). A phone on the network for a minute in a row while usbmuxd does not list it (`lostAfter`) is listed `needs_setup` with `driver.SetupDeviceService`, under the name it had (the gateway's saved record after a restart of Pinky). The search stops 5 minutes after nothing is missing. macOS only, where the bug is. **Right after a pull** (2026-11-09, after an owner waited 75 s for Lost on Wi-Fi). The provider notes when usbmuxd last listed each phone on a cable (`lostState.cabled`). For `unplugRecent` (10 minutes) after a pull, and only for a phone whose Wi-Fi connection it knows is on (wifiallow.go: read so or turned on at that attach) and that usbmuxd has not listed over the network since the pull (one it has came through, and may be asleep now: `lostState.netted`), it does not wait `lostAfter`, which is there for phones that sleep: usbmuxd lists a phone over Wi-Fi about 2 s after its cable is pulled, so one the network shows that usbmuxd does not list `unplugGrace` (8 s) after the pull is lost, and one that neither shows `notSeenAfter` (15 s) after the pull, with the search running for 5 s at least, is listed offline with `driver.SetupNotOnWiFi` (new, additive) and Detail "Not seen on Wi-Fi", which the gateway's status line says and the Phone screen explains: asleep, another network, one that keeps devices apart, a VPN, or Pinky without Local Network access. Agents get `device_offline` for it, as before. Once the network shows it, it is lost; once usbmuxd lists it, neither. `TestUnpluggedPhone`. Not yet tried on a phone. - **Getting around usbmuxd was tried and dropped** (iPhone 13 mini, 18:23, while usbmuxd had lost it): it answered ping, but lockdown took no TCP connection by its address: 62078 timed out, and the port its `_apple-mobdev2` record advertises (32498) refused. go-ios can run lockdown over any `net.Conn` (`NewDeviceConnectionWithConn`, `StartSession` with the pair record), so with an open port a session would work; but the port is not reliably open, and every service Pinky uses outside the tunnel goes through usbmuxd (lockdown values, the image mounter, AMFI, the inspector, the installation proxy), so a second way would be a large part of its own and still leave Finder and Xcode without the phone. - **The fix is the person's**: the phone's page offers Restart device service. Pinky asks first, then quits usbmuxd through macOS's own request for an administrator's password (`osascript` `do shell script "/usr/bin/killall usbmuxd" with administrator privileges with prompt …`), and launchd starts it again; every iPhone and iPad disconnects for a second or two. Declined, nothing happens (`driver.ErrCanceled`). `driver.DeviceServiceRestarter`, `Gateway.RestartDeviceService`. Not yet tried by the owner on the Mac (it needs their password). **On Linux** (`netmux.go`, the package `netmux`, 2026-11-09). libimobiledevice's usbmuxd, which Linux systems have, lists phones on a cable only (1.1.1, its last release, from 2020, has no network code; neither has its master branch), so with Wi-Fi on an unplugged phone just went offline. Now the app's provider (`Provider.WiFiMux`, set in `cmd/pinky`) runs a usbmuxd of its own on Linux once Wi-Fi is on: a socket in a new 0700 folder under `$XDG_RUNTIME_DIR`, and `USBMUXD_SOCKET_ADDRESS` pointing at it for the process, which go-ios reads for every connection (`GetUsbmuxdSocket`). Nothing tells the provider that Pinky ends, so it leaves its folder behind, and the next start removes the folders at whose socket nothing listens any more (`removeOldMuxDirs`, `TestRemoveOldMuxDirs`); before that every start left one more until the person logged out (18 on arch after two days). It hands every request about phones on a cable to the system's usbmuxd (pair records too) and adds the phones a `netmux.Finder` found: `ListDevices` lists them as `Network` with DeviceIDs from 2^28, and `Connect` to one dials the phone's address on the port asked for, as macOS's usbmuxd does. `Listen` is passed on unchanged (Pinky polls `ListDevices`). With `USBMUXD_SOCKET_ADDRESS` set already (netmuxd, usbmuxd2), Pinky uses that one and finds nothing itself; a phone the system's usbmuxd lists on the network itself is not listed twice. - **Finding phones.** The Finder browses `_apple-mobdev2._tcp` (the mdns package) and matches each instance to this computer's pair records, read through the system's usbmuxd, for the UDIDs in `/var/lib/lockdown` and the phones Pinky has seen (only `PINKY_DEVICES` under it). From iOS 26.4 the instance's name carries a private Wi-Fi address; the TXT record has an `identifier` and `authTag`s, each the first 8 bytes of HMAC-SHA256(HKDF-SHA512(HostID), identifier), base64 (Apple's `AMDIsTXTRecordForUDID`; netmuxd matches the same way). Older phones are matched by the Wi-Fi address in the name. The records are no proof either way: they last long (PTR 75 minutes) after a phone left, and one can lack its TXT strings or address for a while though the phone is there. So a record's name, once matched, stays matched, the last address is kept, and lockdown decides: a phone is listed while lockdown (62078) takes a TCP connection, and dropped once it has not for 45 s (`dropAfter`) from the first time it did not, longer than arch's Wi-Fi dropped for while renewing its address (8 s and 36 s). Dropping it after two misses, as at first, emptied the list 8 s into such a drop with no phone on a cable, so `ListDevices` failed (EOF) and Pinky closed its session on the iPad (`TestMuxKeepsAPhoneThroughABlip`). - **How often it asks** (2026-11-09, in review). Every iPhone on the network that has Wi-Fi connections on answers each round of `_apple-mobdev2` questions, and with `MulticastAnswers` it answers to the group, so every device there hears it; the search started at every 5 s for as long as Wi-Fi was on. Now its rounds back off as RFC 6762 §5.2 asks of a long search (`mdns.Options.MaxEvery`): 1 s after the first, then twice as long each time, up to a minute, and from the start when the computer's addresses change (looked at every 5 s); a phone that joins the network announces itself in between. Lockdown was asked every 3 s about each phone whose record matched; now at every look (3 s) only while a phone is new or a listed one has just stopped answering, and twice as long each time, up to 30 s (`probeSteady`), while it keeps answering or keeps not answering. A phone that stops answering is dropped 45 s after the first look that found it so, not after its last answer, which may be 30 s older: a Wi-Fi drop of 39 s keeps it listed wherever it falls (`TestFinderProbePace`). - **Without the system's usbmuxd.** libimobiledevice's usbmuxd runs as `usbmuxd --systemd`, which exits once no phone is on a cable (udev starts it again on the next plug). Pinky's usbmuxd then lists the phones on Wi-Fi alone and reads pair records from `/var/lib/lockdown` itself; at first it failed every request, and both phones went offline the second the last cable came out (`TestMuxWithoutUpstream`). - **mDNS through a firewall.** On arch (ufw active, input DROP but for multicast to 224.0.0.251:5353) the browser found nothing: a responder answers a question from a port other than 5353 by unicast to that port (RFC 6762 §6.7), which ufw drops. `mdns.Options.MulticastAnswers` (additive; Linux only, ignored elsewhere) sends the questions from the group socket, port 5353, without the unicast-response bit, and both phones answered at once, with their TXT records (now in `mdns.Service.TXT`, additive). A TXT record that does not parse is skipped, not the rest of the response after it (`TestBadTXT`). The Android search is unchanged. - **Checked on arch** (Arch Linux, iPhone 16 Pro and iPad Pro 11-inch, iOS 27.2, both also on the cable): the Finder matched both by their TXT tags within 3.2 s, go-ios listed them as `Network` through Pinky's usbmuxd, and lockdown sessions (TLS with the pair record) over Wi-Fi took about 310 ms. A Pinky running through netmuxd on the same machine moved the iPad to Wi-Fi when it was unplugged (online over Wi-Fi 65 s after the cable came out). With Pinky's own usbmuxd (the system's stopped, both phones unplugged): both found on Wi-Fi within 3 s and opened over it, the live picture showing; both online over Wi-Fi for a 12-minute watch, checked every 15 s, with no `ListDevices` error and no offline line; the iPad's Wi-Fi off and on (lockdown silent for 6 s) kept it listed. With the system's usbmuxd starting with the first cable and exiting with the last: plugged in, each moved to USB in 0.4 to 0.5 s; unplugged, the iPad was online over Wi-Fi 1.4 s later and the iPhone 2.8 s later. Not yet tried: a move while an agent's task runs, and another distribution (another firewall, or `/var/lib/lockdown` the user cannot read: then only phones Pinky has seen are found). The slower pace of questions and lockdown checks above came after these checks, from the review. Unit-tested: `TestMux`, `TestMuxWithoutUpstream`, `TestMuxKeepsAPhoneThroughABlip`, `TestFinder`, `TestMatch` (with a tag worked out apart from this code), `TestWiFiMux`, `TestTXT`, `TestQueryMulticastAnswers`. - **The lockdown heartbeat** (`netmux/heartbeat.go`). A usbmuxd that lists a phone on the network keeps a connection to its `com.apple.mobile.heartbeat` service and answers each `Marco` with `Polo` (macOS's, Apple's on Windows and netmuxd do). Without it an iPhone 18 Pro Max on iOS 27.2 still answered lockdown and `syslog_relay` over Wi-Fi, but closed AMFI, the image mounter and `CoreDeviceProxy` right after the TLS handshake, so Pinky could not check Developer Mode or start the tunnel and kept it as "locked on Wi-Fi"; with the heartbeat answered the same requests worked at once, and the phone moved to Wi-Fi 1.5 s after it was unplugged, online with its screen (Windows, 2026-11-09). The Mux keeps one per listed phone (`Mux.Heartbeat`), started again when it ends. From the review: starting it (go-ios's `ConnectToService`, which takes no context) gives up after 20 s or when the phone leaves (`connectWithin`); a read waits a minute before the first `Marco`, then three of the `Interval`s it gives, set through the connection's `Conn()`; a phone the system's usbmuxd lists on the network itself (Apple's on Windows while the phone is on a cable too) keeps that usbmuxd's heartbeat, and the Mux starts none (`TestHeartbeatLeavesTheUpstreams`). - **The cable's own network.** On a cable an iPhone also answers the search on the network it makes with the computer (172.20.10.1, "Apple Mobile Device Ethernet" on Windows), and lockdown answers there too; with both, its address flipped every few seconds, each time a new attach. The Finder prefers an address outside 172.20.10.0/28 and keeps the one a phone is listed at while it is still among the best (`pickAddr`). When a phone's records name only the link, it stays at its last Wi-Fi address only while lockdown answers there, and moves to the link at once otherwise (`TestFinderLastAddressOnlyWhileItAnswers`). - **A record is no proof of an address** (2026-11-09, security hardening). Anyone on the network can send a phone's `_apple-mobdev2._tcp` record, whose name and TXT strings go out in the clear, with another address; sent with the cache-flush bit, it even drops the phone's own address from the search's cache. Lockdown's TLS is pinned now (above), but WebDriverAgent's HTTP below iOS 27 is not where it goes through usbmuxd to the phone's address (a WDA already running when Pinky opened the phone; since GitHub PR #53 Pinky reaches the one it starts on Wi-Fi through its tunnel), so whoever answered at a phone's address saw the taps and the text Pinky sends to it. Before, the Finder moved a phone to whatever address its records named at the next look. Now a phone listed on Wi-Fi stays at its address while lockdown answers there, whatever the records say, and lockdown is not even asked at the other address; the phone goes there only once its own address has not answered for `dropAfter` (45 s), so a nap does not hand it over, and to the cable's link at once as before. A phone that really moved (a new DHCP lease) stays listed at its old address until then and is at the new one right after, without a look where it is not listed: 45 to 75 s instead of one look. A phone that was dropped goes back to the address it was last listed at while its records still name that one among others. Records that name an address no phone can have are skipped: unspecified, loopback, multicast and broadcast addresses and IPv4 written as IPv6 (`mdns.Usable`, which the mdns cache applies too, so such a record is neither dialed nor flushes a host's addresses), and this computer's own (`usableAddrs`), where whatever listens on 62078 would get what Pinky sends. Addresses outside this computer's networks stay, since an mDNS reflector brings records across VLANs. Not covered: before a phone is first listed, or once it has been away longer than `dropAfter`, an address that only forged records name is taken when something answers there. A pinned lockdown handshake at the new address before listing it would close that, at the cost of a TLS session per check. Unit-tested, each failing without the change: `TestFinderStaysWhileItAnswers`, `TestFinderFollowsAMovedPhone`, `TestFinderSkipsAddressesNoPhoneHas`, and `TestUnusableAddresses` in the mdns package. Not tried on a phone or with a forged record. - **On Windows** too (2026-11-09): Apple's usbmuxd (TCP 127.0.0.1:27015) lists an iPhone on Wi-Fi only while it is on a cable as well, so the provider runs its Mux there on a TCP port of its own (127.0.0.1, a random port), and every go-ios connection to an iPhone, on a cable too, goes through it. The Finder reads pair records from `%ProgramData%\Apple\Lockdown` for itself; the Mux does not hand them to its clients from there (`ServePairDir` off): any local user can reach that port, and a Pinky run as an administrator would hand out the computer's private key. Apple's usbmuxd keeps running anyway and answers `ReadPairRecord`. - **The address is not inherited** (from the review). `USBMUXD_SOCKET_ADDRESS` is set for the whole process, so every process Pinky starts inherited it: on Windows the update helper, and the Pinky it starts after an update, took the old Pinky's port, gone by then, for one the person set, ran no Mux of its own and reached no iPhone. Pinky now sets `PINKY_OWN_USBMUXD` to the same address, and a process of Pinky's that starts with both equal unsets them (`forgetInheritedMux`, in the package's `init`, before go-ios reads the address); one the person set stays (`TestForgetInheritedMux`, `TestWiFiMuxAfterAnUpdate`). The same holds on Linux, for its updates to come. - **Phones the person removed** (Forget) are left out of the UDIDs the Finder looks for, so they are neither listed nor asked over Wi-Fi until they are on a cable again. **OCR from the live video.** A screenshot over Wi-Fi costs about 5 s, so a read with OCR (every `phone_observe`) took 5 to 7 s. The gateway now offers reads with OCR a picture decoded from the live video (`driver.ObserveOptions.Frame`, additive, through `frameShot`, which shares the mirror's stream and asks for a key frame), and over Wi-Fi the driver reads OCR's text from it, cropped to the screen (at 1:1 in the picture's corner), JPEG-encoded for Vision; without a frame it takes a screenshot as before. `ObserveStats.ShotFrom` says which. `phone_observe { screenshot: true }` still takes a screenshot. Not yet measured on a phone: `TestDeviceOCRFromVideo` compares both. **Measured, and what changed with it** (2026-11-08, owner present; numbers in the hand-over note). On the iPhone 13 mini over Wi-Fi a frame came in 0.1 to 0.2 s against 4 to 11 s for a screenshot, with the same OCR lines (41 to 43 against 44); six fresh views of the stream got a picture in 0.13 to 0.84 s, none lost. On the iPad Pro 11-inch only 2 of 6 frames came within the 5 s the gateway waits: once 40 to 70 % of the video's packets were lost (9 minutes after a usbmuxd restart; 0.24 % the day before), once the iPad sent nothing for 10 s on a still screen despite 14 key frame requests. Its screenshots took as long as their PNG: 0.8 MB in 1 s, a 15.5 MB lock screen 16 to 26 s. Hence: - **The wait for a frame is bounded** (`videoFrame`): 2.5 s at most (`frameWait`), and after a frame did not come the phone's screenshots are used for a minute (`frameRetry`), so a phone like that iPad costs no extra wait per read. A frame comes at once while the mirror or a task's recording watches the phone: the gateway decodes the stream's last key frame and what followed. - **Approval cards, history pictures and agents' screenshots come from the video too** over Wi-Fi: the gateway asks a `driver.FrameShooter` (new, additive) for the screen from a frame first (`shotOf`), and takes a screenshot only when there is none. - **Screenshots wait long enough over Wi-Fi.** go-ios asks the instruments screenshot channel with a fixed 5 s, which every screenshot of a busy screen over Wi-Fi outlasted (the "first screenshot times out" of the day's tests: the first was the home screen's); Pinky then asked the capture service, which sent the screen again at 16 bits a channel. Pinky now opens the channel itself (`instShot`, as go-ios's `ScreenshotService` does) and waits 30 s over Wi-Fi, 5 s on a cable, where 7 MB take under a second and a longer wait would only hide a stuck service; the capture service gets 30 s and 10 s. - **Walks over Wi-Fi get 8 s** (`raxLimitsWiFi`, 4 s on a cable). The first read of a new session over Wi-Fi gave 5 to 17 of the 19 elements Settings showed, in 4 to 19 s, and later reads of that session all 19 in about 2 s: the walk sends one request per node, one after the other, and the first walk of a session, while the app builds its accessibility tree, ran into the 4 s budget. - **A dark screen reads as locked** (`asleepRead`). A read that found next to nothing (the app's element and one more at most) of a phone that reads as unlocked asks for the screen's state (`ScreenOn`, one CoreDevice request); when the screen is off the read is locked, and its OCR text is dropped. An iPad that went to sleep with a session reading read 0 or 1 elements, unlocked, for 28 s until it left the network. The gateway wakes a phone before an agent's call reads it, so this is for reads that do not (background reads), and phones without a passcode. Unit-tested: `TestVideoFrame`, `TestShotFromFrameFirst` (gateway), `TestWiFiReadLimits`, `TestAsleepRead`. Not yet on a phone: the instruments channel's own wait, the walk budget's effect on a first read, a dark screen through `ScreenOn` over Wi-Fi. **Services that hang after a cut session.** `TestDeviceRecovery` opens the phone again every 20 s and logs the open (tunnel, stream, HID) and a read, to measure how long the phone takes to recover after a session was cut off. Not yet run. **What preparing needs over Wi-Fi.** | Step | Over Wi-Fi | Checked | | --- | --- | --- | | Trust | Needs the cable: the phone asks only on a cable, and usbmuxd lists a phone over the network only once it is paired. A phone that no longer trusts the computer reads as "plug it in" (`trustWiFiDetail`) | By how pairing works; not on a phone | | Developer Mode switch (AMFI's reveal, action 0) | **Needs the cable.** `com.apple.amfi.lockdown` starts over Wi-Fi (0.4 s), but the reveal over Wi-Fi gets no answer: the phone closes the connection. The same request over USB answered success in 35 ms. Pinky does not send it over Wi-Fi, and the steps say the switch may need the cable once (a phone used with Xcode shows it already) | iPad Pro, iPadOS 27.0.1, Developer Mode already on, so the reveal changed nothing | | Turning Developer Mode on | On the phone, over Wi-Fi too | (the person's step) | | Developer disk image | **Works over Wi-Fi.** The cryptex install through the tunnel with the 1280 MTU (`uploadMTU`) | The iPad's 27.1.9275.0 cryptex taken off over Wi-Fi in 0.39 s, then the provider mounted Xcode's 27B52 (27.2.52.0) over Wi-Fi in 9.0 s; online again 10.6 s after the take-off (`TestDiskImageRemount`, `PINKY_REMOUNT=any`, with the owner's leave) | | Checkup | Works over Wi-Fi | All four items pass in 5.2 s, most of it the screenshot (`TestCheckupOnPhone`) | **Measured** on an iPad Pro 11-inch (3rd generation, iPad13,4, iPadOS 27.0.1), the same afternoon, unlocked and awake, `TestDeviceConnection` (`PINKY_CONN=wifi`, then `usb`): | | Wi-Fi | USB | | --- | --- | --- | | Session open | 3.9 s from Use to online (inspection with a tunnel, then the session) | 0.28 s | | Touch-down to changed frame | 77, 87, 96, 108, 149, 212 ms | 57, 63, 68, 69, 73, 90 ms | | Home screen read (remoteAX) | 0.61 s, 1.99 s, 2.62 s | 3.49 s (first, pushed tree), 0.52 s, 0.61 s | | Screenshot (1668×2388) | 5.05 s, 4.98 s | 1.09 s, 0.89 s | | JPEG or HEIC capture | refused ("Cannot initialize ScreenshotImageFormat"; devicectl offers only PNG) | refused | | Settings read after launch | 1.72 s | 1.51 s | | Type (paste) | 1.56 s | 2.07 s | | Video | 2033 frames in 55 s (55 to 66 a second while the screen moved, none on a still screen), 3 key frames, 44 of 17999 packets lost (0.24 %), all in one burst after the screenshots, 3 key frame requests, 2.9 Mbit/s | 2024 frames in 43 s, 0 lost, 1 key frame request, 3.6 Mbit/s | Far better than the iPhone's Wi-Fi run of round three (below): 0.24 % lost instead of 1.5–10 %, with the RTCP receiver reports now carrying the loss (merged ios-wifi work), and reads in seconds rather than walks of 1.7–2.3 s. Screenshots are the cost: about 5 s, so a read with OCR (every `phone_observe`) takes 5 to 7 s over Wi-Fi against about 1 s on the cable. Taking OCR's picture from the live video instead removes most of it (done the same day: see "OCR from the live video"). **Moving with a task running** (a read-only agent task reading the screen every 1.5 s, through the gateway, `cmd/wifitry` scratch harness): - Cable pulled: two reads failed at once with "internal" (the inspector's lockdown connection on the gone USB entry), before `Device.gone`; 2 s later discovery saw the phone on Wi-Fi, and it was open over Wi-Fi 2.8 s after that; the lease stayed. The first read over Wi-Fi took 16.7 s (a screenshot timed out once), the next ones 4.6 to 6.7 s. With `Device.gone` the next pull read as offline and the call waited for the move. - Plugged in: discovery saw it within 2 s, open over USB 1.6 s later (6.3 s in a second run, waiting for the Wi-Fi session to close); no call failed. - After several cut-off sessions (pulls during reads, test processes stopped with their sessions open, the iPad sleeping with a session), the iPad's services over Wi-Fi answered slowly or not at all for at least ten minutes: the HID service did not answer within 10 s, remoteAX timed out, the screenshot services did not answer, and reads came back empty after the 60 s observe timeout (now an error, `errEmptyRead`, rather than an empty screen). A fresh session did not help. Whether a restart clears it was not tried; the owner did not restart the iPad. **While it sleeps** (locked, cable out, the gateway holding it open, before the lock rule): opened 5 times in 4.5 minutes, each open lit the screen, each session ended 9 to 44 s later when usbmuxd dropped the phone. With the rule: listed as Locked on Wi-Fi for 3 minutes, never opened, the screen stayed dark; one lockdown look got no answer (`StartSession: EOF`) as the iPad dozed, and now reads as still locked (`stillAsleep`). Unlocked: open over Wi-Fi 5 s later. **A phone usbmuxd lost.** macOS's usbmuxd (macOS 27.2) loses a phone over Wi-Fi after one interrupted connection: it asks mDNSResponder to reconfirm the phone's `_apple-mobdev2._tcp` record, which fails (-65541), releases the phone, and attaches it again only when the record is new to the Mac's cache, which takes until it expires (up to about 75 minutes) while the phone stays on the network. `sudo killall usbmuxd` (launchd starts it again) lists it at once; every phone drops for a second or two. An iPad that sleeps churns its record every 20 to 70 s and so comes back by itself. Found by the iphone-wifi-why session the same day (its hand-over note has the logs). Pinky notices such a phone and offers the restart ("Phones usbmuxd lost" above). How usbmuxd listed the phones over the afternoon: the iPad (asleep) for 15 to 30 s every 30 to 90 s, as a new DeviceID each time; awake, without a gap. The iPhone 13 mini was listed over Wi-Fi without a gap for 40 minutes in the afternoon (it had not been listed in the morning: see "A phone usbmuxd lost"). usbmuxd restarted once at 16:09 (DeviceIDs from 1 again): the owner ran `sudo killall usbmuxd`, which brought the iPhone back. ```sh # Wi-Fi or USB measurements (the phone unlocked; Pinky.app quit, or it takes the phone on a cable): PINKY_DEVICE= PINKY_CONN=wifi scripts/devlock.sh go test -tags device -count=1 -v -run 'TestDeviceConnection$' ./internal/driver/ios/ # The checkup and the remount over whatever usbmuxd lists (Wi-Fi when there is no cable): PINKY_DEVICE= scripts/devlock.sh go test -tags device -count=1 -v -run 'TestCheckupOnPhone$' ./internal/driver/ios/ ``` The device tests that build a provider (`deviceProvider`) limit it to the phone under test (`Provider.Only`, which `PINKY_DEVICES` now sets too, so other phones are not even read) and use it over Wi-Fi when usbmuxd lists it only there. #### Round three (2026-11-07, iPhone 13 mini) With Finder's "Show this iPhone when on Wi-Fi", usbmuxd lists the phone twice (USB and Network). Opening the Network entry runs everything Pinky uses: the tunnel (CoreDeviceProxy through usbmuxd's network connection), video, HID, `remoteAXService`, screenshots and the pasteboard. The same session measured over each (`TestDeviceConnection`, two Wi-Fi runs): | | USB | Wi-Fi | | --- | --- | --- | | Session open | 0.58 s | 1.5–2.6 s | | Touch-down to changed frame | 73–154 ms | 6 ms–1.5 s (median 150–270 ms) | | Observe, walk | 0.3 s (pushes 32–38 ms) | 1.7–2.3 s | | Screenshot (home screen) | 0.67–0.78 s | 3.4–5.3 s | | Type (paste) | 0.26–0.46 s | about 2 s | | Video | 0 packets lost, no keyframe requests | 1.5–10 % of packets lost, 10–27 keyframe requests in 55 s | Everything goes through TCP inside TCP over Wi-Fi (gVisor's TCP in the tunnel, inside usbmuxd's connection), so each round trip costs tens of milliseconds and a walk, which is one round trip per request, pays it 25–40 times; the video is UDP in the tunnel and the phone drops it when the tunnel backs up. So Wi-Fi stayed off then (`Provider.Wireless`, since removed): the mirror would stutter, agents would wait seconds per step, and turning it on would also make the gateway open every iPhone and iPad paired over Wi-Fi (the owner's iPad among them) and show screen sharing on them. The second reason is what "Asking first" above answers; the first improved with the RTCP loss reports (2026-11-08). Apple's own Wi-Fi path is a different tunnel (remote pairing over QUIC, pymobiledevice3's `remote start-tunnel` over Wi-Fi), not tried here; it may still be the way to make Wi-Fi as good as the cable. ### Element tree (inspector, the fallback) - The walk: `deviceInspectorFocusOnElement:` then `deviceInspectorMoveWithOptions:` (first, then next) in batches of 8, until an element repeats; it covers the visible app including off-screen cells of scroll views (Settings lists all 38 rows, only the visible ones get positions), but not the keyboard (another process) or the status bar. It does not scroll the app. - Per element: caption (label, value, traits, hint as VoiceOver speaks them), accessibility identifier, an opaque handle. Captions are localized. - Tried for frames on iOS 27, all empty: attributes `Frame`, `AXFrame`, `frame`, `accessibilityFrame`, `Bounds`, `Position`, `Size`, `ElementRect`, `AXAttribute-2003` and more; hit testing (`deviceFetchElementAtNormalizedDeviceCoordinate:` with an `NSValue` point) answers nothing; `deviceFetchSpecialElement:` 0 and 1 return the first and last element, not the app. The audit (`deviceBeginAuditTypes:`) reports rects only for flagged elements (8 "element detection" issues on the home screen, 3.9 s). - Side effect found and fixed: the inspector focus draws a ring on the focused element (visible in the video) until `deviceInspectorPreviewOnElement:` is sent with null, which every walk now does. - A null element handle in `deviceElement:valueForAttribute:` broke the daemon connection; never send one. ### remoteAXService How it was found: Xcode's `AccessibilityAudit.framework` (in `Xcode.app/Contents/SharedFrameworks`) holds both ends. On the host, `AXAuditRemoteDevice` owns the DTX connection and serves as the transport of an `AXPHostCacheManager` (from the system's `AccessibilityPlatformTranslation.framework`): it sends what the manager hands it with `processDataFromHost:` and feeds what the phone sends through `processDataFromRemoteDevice:` back to it. On the phone, `AXAuditDevicesAppRemoteServer` does the reverse for the phone's `AXPTranslator_iOS`. The Objective-C runtime (through purego) listed the classes and methods, lldb disassembled the few methods that mattered, Apple's own `NSKeyedArchiver` showed the request format, and `AXPTranslatorRequest`'s `description` named the attribute numbers. The protocol, as iOS 27.0.1 speaks it: - DTX on the tunnel (`com.apple.accessibility.axAuditDaemon.remoteAXService`), global channel only. Send `_notifyOfPublishedCapabilities:`; `deviceAPIVersion` answers 26. - `clientNeedsAccessibility:` with YES is required: without it requests get no answer. NO makes the phone close the connection, so it means goodbye. The phone then asks the host `hostAPIVersion` (Xcode answers 7, and so does Pinky). - A request is an `NSKeyedArchiver` archive (secure coding) of `AXPTranslatorRequest` {`requestType`, `attributeType`, `actionType`, `clientType` (Xcode's cached-tree client type, 2), `parameters`, `translation` (`AXPTranslationObject` {`pid`, `objectID`, `isApplicationElement`, ...})}, sent as the `NSData` argument of `processDataFromHost:`. The answer comes as `processDataFromRemoteDevice:` with an archived `AXPTranslatorResponse` {`associatedRequestType`, `attribute`, `resultData`, `associatedTranslationObject`, `notification`, `error`}. - Request types used: 2 one attribute (`attributeType`), 4 the frontmost app's element (18–19 ms), 5 several attributes with `parameters` {`attributes`: [numbers]} (the key `AXPAttributes` gives an empty answer). 11 is the tree dump the phone pushes; asked for by the host, it got no answer. - Answers carry no request ID, so requests go one at a time: 5–60 ms each when the phone is idle, about 10 ms per element in a walk. A request for all 131 attributes at once got no answer within 2 s; a session with a request that timed out is dropped, since its late answer would be read as the next one's. - Attribute numbers (from the description on macOS 27; the walk asks for the marked ones): 1 activation point, 3 attributed label, 8 children*, 9 children in navigation order*, 21 frame* (screen points), 23 help, 25 identifier*, 26 is element*, 27 enabled*, 28 focused*, 30 selected*, 32 visible*, 33 label*, 41 parent, 42 placeholder*, 45 role*, 53 value*, 61 automation type* (`XCUIElementType`), 62 image data (PNG), 64 value description*, 65 host application pid, 66 is remote element*, 69 remote element pid*, 77 traits*, 78 traits as text, 112 visible point, 114 braille frame, 124 responds to interaction, 128 memory address, 129 application orientation. - Pushes: after `clientNeedsAccessibility` and after screen changes, 1.1–1.5 s later, the phone pushes `AXPTreeDumpTypeInitialDump` (the frontmost app's element and every element on screen with about 24 attributes: 21 KB for Settings, 36 KB for Calculator, 32 KB for the home screen), then `AXPTreeDumpTypeAdditionalData` (the same elements with about 60 attributes, AutomationType, identifiers and icon images included: 68 KB, 134 KB and 1.7 MB) and sometimes notifications. Not every change gets a push (Calculator's Clear did not). Decoding takes 0.3–4.5 ms. - While the phone builds a push, requests wait. Walks measured 0.27–0.33 s on an idle Calculator and 1.08 s when one met a push; on screens whose second push carries many images, 2.9 s (home screen) and 4.6 s (Settings' search with the keyboard up). Hence the push rules above. - What is listed: the elements on screen, in reading order (12–14 on Settings' first page, where the inspector walk found 38 rows including those scrolled out of view; 23–25 on Calculator; 26 on the home screen). Elements behind a sheet can be listed too (Settings' rows under its search suggestions). - Other processes: the keyboard's keys (another process) come as children of the app, with frames relative to a host view listed just before them, which has traits bit 34 (`0x400000000`) and the keyboard's frame; the keys are `StaticText` labelled with one character. A home screen widget (Photos, another process) came with a frame relative to itself and no host; `Observe` leaves such elements out rather than place them wrongly. Attributes 66 and 69 stayed empty in both cases. - Calculator's display has no label or value; its identifier carries the number (`StandardInputView;value:7`), which `Observe` shows as a text element with that value (`text "7"`, identifier `StandardInputView`). The identifier is not in a first push (that element has only its frame, traits and role there), so `Observe` waits up to 1.2 s for the second when a first push leaves an element unnamed; requests wait while the phone builds that push, so asking for the identifier would not be sooner. Calculator's keys are `Key` (20) with the keyboard-key trait, so the keyboard is recognized by its process (many one-character labels), not by that trait. - The first read after an app opens, measured again in round three (18 launches): act, settle, `Observe` (what the gateway does) took 33–261 ms on Settings and 340–481 ms on Calculator (the wait for the push with the display's number), once 1.07 s; the 2–3 s seen in the second pass did not come back. `Observe` right after `Launch`, without settling, took 0.8–1.6 s: the phone's first answer (the frontmost app) alone took 0.73–0.83 s while the app started, and its push came 1.1–1.6 s after the launch. - An app that has just launched can show its screen before it lists any element (a walk found only the app element once); `Observe` then waits up to 1.5 s for the push and walks again. - Request types are the cases of `-[AXPTranslator processTranslatorRequest:]`, which macOS 27 ships with the iOS translator (`AXPTranslator_iOS`): 1 `processApplicationObject:` (parameters `{pid}`: that app's element, in front or not), 2 one attribute, 3 can set, 4 frontmost app, 5 several attributes, 6 `processHitTest:` (parameters `point`, `displayId`, `contextId`), 7 action, 8 set attribute, 9 supported actions, 10 supports attributes, 11 tree dump. Found with lldb on a process that loaded the framework. - Request type 1 answers for SpringBoard (its element's ID was the same in every session and run) but not for a process without a user interface: asking for a daemon's or the widget renderer's element got no answer within 2 s, which drops the session. - While Clock is in front, SpringBoard's element has the status bar's items as children (time, signal, Wi-Fi, battery, the location arrow, the "Return to Clock" breadcrumb), all with trait bit 23; nothing of the home screen. - A second push can say an element is not visible that the walk right after says is ("2 min, two minutes" in Clock's recent timers): that element comes and goes between reads. - Traits (attribute 77) are UIKit's, public and private. The private bits, read from `UIAccessibility.framework` on macOS 27.2 (Mac Catalyst; its `_UIAXInitializeConstantValues` sets them): 17 web content, 18 text entry, 19 picker element, 20 radio button, 21 is editing, 22 launch icon, 23 status bar element (the status bar's items carry it), 24 secure text field (`traitSecureText`: native password fields as well as Safari's), 25 inactive, 34 spacer (what `traitRemoteHost` names: the keyboard's host view carries it), 45 touch container, 53 toggle. Looked up for GitHub issue #47 (Notes' body refused as a code field), to tell whether bit 24 marks web pages only: it does not. ### OCR (fallback) - Vision's accurate recognizer with `zh-Hans`, `zh-Hant`, `en-US`: 0.1–0.4 s per 1125x2436 screenshot once loaded. The first use on a Mac took 56 s (model compile), and runs with automatic language detection sometimes took 15 s, so detection is off. - Placement matches labels to text in rounds (equal, truncated with an ellipsis, label inside a line, line inside the label) in reading order. Text that appears twice can be matched to the wrong element (a search field whose value equals a result title); elements without visible text (icons, dock apps, images) get no position. - The on-screen keyboard is detected from OCR (five or more single letters in the lower half) and its keys are dropped from the output. ## Orientation Pinky's coordinates on an iPhone or iPad are the screen as the person sees it, the right way up, in every orientation: element bounds, taps, swipes, screenshots, OCR and the mirror's picture agree, and `Screen.Width`/`Height` swap in landscape, as on Android. iPads rotate everything, the home screen included; iPhones rotate only in apps that allow it (Safari, Calculator), and iPhones with Face ID never turn upside down. Taps that landed in the wrong place on an iPad Pro (GitHub issue #2) are the likeliest result of the old portrait-only mapping. ### On iOS 27 Measured on 2026-11-08 on the iPhone 13 mini (Calculator) and an iPad Pro 11-inch (3rd generation, iPad13,4; Settings › General), iOS 27.0.1, over USB, in every orientation each allows, the owner turning the devices (`iosctl orient`, then taps checked by what opened): | What | iPhone 13 mini | iPad Pro 11-inch | | --- | --- | --- | | Display info `bounds`, `nativeSize` | 1125x2436, 1080x2340, in every orientation | 2388x1668 in every orientation: the panel is mounted landscape | | `nativeOrientation` | `rot0` | `rot270` | | `currentOrientation` | portrait `rot0`; turned counterclockwise `rot270`, clockwise `rot90` | held the usual way up `rot90`; counterclockwise `rot0`; clockwise `rot180`; upside down `rot270` | | Screenshots | Upright: 2436x1125 in landscape | Upright: 1668x2388 in portrait | | remoteAXService's frames | Upright points (Calculator 812x375 in landscape) | Not seen: the service did not answer this iPad at all that night (below) | | Video | The display's frame, 1136x2464, whatever the orientation; turning it `currentOrientation` clockwise puts it upright | The same: 2400x1680, upright after `currentOrientation` | | Touch panel | The display's frame | **The frame of the device held the usual way up**, which the display is turned `rot270` from | So the display info describes the display as mounted, `currentOrientation` turns it into the interface, and `nativeOrientation` turns the frame of the device held the usual way up into the display; the touch panel's 0..65535 is in that last frame, so a tap turns by both (`screenInfo.touchTurn`). On the iPad a tap at the display's own coordinates landed on an empty part of General's header, and one turned by both opened Screen Capture, the row at the point aimed at. Before, Pinky read the display info once at `Open` and mapped taps straight onto the display's frame: on that iPad, Pinky saw a 2388x1668 landscape screen while it was held in portrait, placed elements by the wrong scale and tapped the wrong places in every orientation, which matches GitHub issue #2 (an iPad Pro M5, taps in the wrong place); on iPhones landscape was wrong the same way. Now: - The screen's size and its orientation are the interface's (`screenInfo.upright`, `orientationOf`); element frames, OCR, screenshots, keys and swipes work in it unchanged. - The display info is read with every screen read (beside the walk, ~12 ms over USB), before a gesture when the last read is more than a second old (`freshDisplay`), and every second while a view of the video is open (`orientLoop`). - Taps and swipes go through `panelPoint`: the upright point turned back by `touchTurn` into the touch panel's frame. - The video's packets carry the turn that puts them upright (`driver.Packet.Turn`, new and additive; `media.setTurn`). The player passes it on (`video.Frame.Turn`), and the mirror draws the picture turned (`drawTurned`), lays itself out in the upright shape, sends touches back in the picture's own pixels (`mirror.toPicture`) and saves screenshots upright; the bezel around it takes the same shape, on the phone's page, in All phones and in the recording player, with corners that round with the screen's short side, less on an iPad (`ui/bezel.go`, GitHub issue #16, `TestBezelFollowsOrientation`). `Device.Touch` takes the picture's pixels, so a drag maps through one description of the screen even while the device turns. - OCR from the live video over Wi-Fi cuts the screen out of the frame in the display's frame and turns it upright. - Recordings keep each picture's turn and play back upright (`internal/recording/turn.go`, `TestRecordTurns`, `TestRecordingTurnsPicture`); a recording opened in a player that honours the track header's matrix shows upright as long as the phone was not turned during it (checked with ffprobe only). - Below iOS 27 without the helper (`AXDevice`), which has no display info, each read takes the screen's size from its screenshot, which is upright. Checked with the new code on both devices: the iPhone in portrait and landscape (Calculator's 7 and AC by their bounds, the packets' turn 0 and 1), the iPad in portrait and landscape (Keyboard and Screen Capture opened by their OCR bounds, General back from the sidebar, the packets' turn 1 and 0). The mirror's turning is covered by `TestMirrorTurnsPicture` (a test stream marked with a quarter turn) and not yet looked at with a real iPad. The video's RTP packets carry a one-word header extension (profile `0x9301`, `0x9311` on some), whose data counts up packet by packet (`00 01 60 65`, `00 01 60 66`, …): nothing in it follows the orientation, so Pinky reads the display info instead. Also seen on the iPad that night: remoteAXService timed out on its first request (`deviceAPIVersion`) in every session, over USB, though the tunnel, display info, screenshots, video and HID worked; reads fell back to the inspector and OCR. Its first video frame came 1 to 1.5 s after the stream started, against 0.6 s on the iPhone. ### Through Pinky on the iPhone (WebDriverAgent) Measured on the iPad Pro 11-inch (M5) and iPhone 17e Simulators, iOS 26.5, WebDriverAgent 16.17.6, in all four orientations (`/session/:id/orientation`), in Settings, Safari and the home screen: | What | In landscape | Upside down (iPad) | | --- | --- | --- | | The tree's frames, `/wda/screen`, `/window/size` | The interface's points (1210×834 on the iPad) | The interface's points | | Gestures (W3C actions) | The interface's points: taps on the right half of a landscape iPad opened the rows there | The interface's points (Camera opened) | | `/screenshot` | Upright, 2420×1668 | **The panel as it is** (upside down) | | MJPEG frames | **The panel as it is** (834×1210 at half size), with EXIF orientation 8 (`LANDSCAPE`) or 6 (`LANDSCAPERIGHT`) | The panel as it is, EXIF 1 | | An alert's window (Settings' Enable Dictation?) | **The panel's frame**: a window of 834×1210 points, its buttons tall and narrow | | | `GET /orientation` | `LANDSCAPE` either way | `PORTRAIT` (WebDriver folds the four into two) | So Pinky: - reads the screen's size the right way up from the orientation the last read, gesture or mirror touch saw (`WDADevice.size`): before, it kept the size it read at open, so in landscape every tap was clamped to the portrait width and element frames were scaled by 1210/1668 instead of 2; - turns MJPEG frames upright by their EXIF tag (`exifTurn`, `turnNV12`; 8 is three quarter turns clockwise, 6 one, checked against the screenshots), so the mirror shows landscape the right way up, and maps the mirror's touches by the picture's shape; - turns a window of the size of the screen turned sideways upright, the way the picture turns (`sidewaysWindows`, `turnTree`; the way from the stream's last frame, else one frame of the MJPEG server), so an alert's buttons are where they show: tapping Not Now by its bounds dismissed the alert in both landscapes; - turns the screenshots, OCR's picture and the MJPEG frames a half turn when the interface is upside down. `GET /orientation` folds upside down into portrait, but `GET /rotation` gives the active app's interface orientation as z (0, 90, 180, 270: 270 in `LANDSCAPE`, 90 in `LANDSCAPERIGHT`, the same as the frames' EXIF turns), so Pinky reads it with each screenshot and every second while the picture runs (`interfaceTurn`, `pictureTurn`), and takes the landscape turn for sideways windows from it too. `TestWDASimUpsideDown` (iPad Simulator): OCR on the turned screenshot finds General where the tree puts it, and the picture's frames carry the half turn. - Back taps the page's back button, which UIKit names `BackButton` (its label is the page it goes back to), the widest bar's when there are several: an iPad's split view has a bar for the sidebar and one for the page, and the first bar's first button in the screen's left third was the sidebar search field's Dictate button, which opened the Enable Dictation? alert. Without one, the leading button of a bar outside its search field, else the edge swipe (`TestBackButtonSplitView`; on the iPad Simulator Back went from Keyboards to General). `iosctl -wda … observe`, `tap` and `stream` in landscape on the iPad Simulator: 2420×1668, the detail pane's rows out to x 2388, a tap at x 2200 opened Keyboard, a 1210×834 upright picture. `TestWDASim` still passes on the iPhone 17e Simulator in portrait, and `TestWDASimOrientations` (Safari turned through both landscapes and back: the size, every element on the screen, elements past the portrait width, a landscape picture) passes on the iPad Pro 11-inch (M5) and iPhone 17e Simulators. ## iPhones below iOS 27 Built on 2026-11-07. No iPhone below iOS 27 was at hand, so what follows separates what ran somewhere from what is only built to spec. ### What each iOS offers The developer disk image is one image for every iOS from 17 (Xcode 27.2's `/Library/Developer/DeveloperDiskImages/iOS_DDI`, build 27B52); its daemons are built for iOS 17.0 (`LC_BUILD_VERSION` minos 17.0), and their launchd plists publish the same services on every version. So the question per version is what the phone does when asked. | Piece | iOS 27 | iOS 17.4 to 26 | iOS 17.0 to 17.3 | iOS 16 and earlier | | --- | --- | --- | --- | --- | | Tunnel | Lockdown's `CoreDeviceProxy` | The same (added in 17.4) | Not there: remote pairing over QUIC on the phone's USB network interface, which go-ios and pymobiledevice3 do as root (go-ios pauses macOS's `remoted`) | No tunnel; lockdown services | | Developer services | Pinky mounts Xcode's image as a cryptex (see "Preparing a phone without Xcode") | The same image, as a cryptex or a personalized image | The same image | A `DeveloperDiskImage.dmg` for that exact version; Xcode 27 ships none (its `DeviceSupport` folder is gone), and Pinky will not fetch Apple's images from mirrors | | Screen stream | `displayservice` | Listed, but refused with CoreDeviceError 9021 (go-ios's measurement). Xcode 27's Device Hub refuses too: its DeviceKit says a device "must be running … or above to screen share" | | | | HID input | `universalhidservice` | 9021 on iOS 18 (go-ios) | | | | Screenshots | Instruments, CoreDevice's screen capture | Instruments (`dtservicehub`) | | | | Elements | `remoteAXService`, with frames | The inspector (`axAuditDaemon.remoteserver`, over lockdown, as the iOS 27 fallback uses), without frames (go-ios on 18.4.1). Whether `remoteAXService` exists before 27 is unknown | | | | Apps | CoreDevice's app service | The same (iOS 17 and later) | | | | Lock state | Lockdown's `PasswordProtected` | The same | | | The daemons check the OS version only against iOS 18 and 26 (`__isPlatformVersionAtLeast` in `dtuhidd`, `dtremotedisplayd`, `dtscreencaptured`, `dtpasteboardd`; `dtdeviceinfod` also checks 27), so the 9021 refusals most likely come from what the OS itself lacks, not from the image. Without HID there is no touch at a point, no keyboard and so no pasting, and no Home; without the stream no live picture. XCTest is the remaining way to touch, which needs a test runner on the phone: WebDriverAgent. ### Two ways, chosen per phone `pathFor` (`version.go`) picks by version, and the provider by what is on the phone, unless the person chose the connection in Settings › Phones (see "Choosing the connection"): - **`wda`**, when the WebDriverAgent runner is installed (an app whose bundle id ends in `.xctrunner` and names WebDriverAgentRunner; Pinky's own build would win). Everything works, unlocking from the computer included. `OpenWDA` uses WDA if it already answers (started by Xcode, say), and otherwise starts its test through testmanagerd with go-ios over the tunnel's RSD entry. Pinky reaches WDA's ports, 8100 (HTTP) and 9100 (MJPEG), through usbmuxd, one connection per request: nothing listens on the computer. Requests: `/status`, a session with Pinky's settings, `/source?format=json`, `/screenshot`, W3C `/actions` for taps, long presses, swipes and the mirror's drags, `/wda/keys`, `/element/active` + `/clear`, `/wda/homescreen`, `/wda/pressButton` (Home while unlocking: `/wda/homescreen` brings SpringBoard to the front, which a locked phone refuses), `/wda/apps/activate` (which, unlike `launch`, never restarts a running app), `/wda/activeAppInfo`, `/wda/locked`. - **`accessibility`**, when it is not: nothing installed, and nothing to act with. The picture is screenshots (one every 100 ms while the screen changes, then one a second), the elements the inspector's walk placed by OCR, apps opened through the app service. Taps, swipes, typing and keys return `unsupported` with a sentence that names the helper: the inspector's only input, its Activate action, pressed nothing on a real phone (see "What ran where"). The provider asks such a phone again every 30 s, so a helper installed meanwhile is picked up. - iOS 17.0 to 17.3 and 16 and earlier are `needs_setup`, with the reason in the detail. Both settle by the picture (`settle_frames.go`): WDA's MJPEG frames (15 a second, and a still screen encodes to the same bytes) or screenshots, settled once nothing changed for 250 ms and at least 500 ms passed, as on iOS 27. On the Simulator: 0.52 s on a still screen, 0.93 s after a fling. Both send H.264 encoded with purego — VideoToolbox on macOS (`h264_darwin.go`), FFmpeg's Media Foundation `h264_mf` on Windows (`h264_ffmpeg.go`): Main profile, no B-frames, 2.5 Mbit/s, a key frame first and again after 10 s of a still picture; frames that did not change (same bytes) are skipped, as the phone's own stream does. The picture is the same from either. Linux has no encoder, so its stream fails with `unsupported`: the mirror and the recorder have no picture, but everything else keeps working — reading the screen, taps, swipes, typing, keys and the agent's own tools. The error the log shows names what is missing. WDA's MJPEG is the screen at half size, 15 fps, quality 50; `Toucher` scales the mirror's points back up and plays a drag back when the finger lifts, because WDA performs whole gestures. The mirror and the recorder need no change: they take H.264 already. ### What ran where | What | Where | Result | | --- | --- | --- | | WDA path end to end (`TestWDASim`): home screen, Settings by bundle id, a tap on General and back, typing `Wallpaper` and `蓝牙 é ✓` into Settings' search and clearing it, a screenshot, 3 s of video checked by ffprobe, settling, a mirror drag, the lock state | iOS 18.5 and 26.5 Simulators (iPhone 16 and 17), WebDriverAgent 16.17.6 built by Xcode 27.2 | All pass. Observe 0.5 s (Settings) to 1.5 s (home screen); typing 0.55 to 0.85 s; first video frame after 0.2 to 0.3 s; activating an app 0.8 s (the first Settings launch after the 26.5 Simulator booted took over 30 s) | | VideoToolbox encoder, frame stream, settling by frames, MJPEG parsing, WDA's tree to elements, the client against a fake WDA, the touch recorder, version paths, provider choices | Unit tests | Pass | | Starting WDA through testmanagerd over Pinky's tunnel, usbmuxd port forwarding to a phone | Nowhere | Built on go-ios's `RunTestWithConfig` and usbmuxd `Connect`, as `ios ui run wda` does | | The accessibility path | Its services on the iPhone 13 mini, iOS 27.0.1 (`TestAXProbeDump`, `TestAXProbeActivate`, `TestAXProbeFocusActivate`, device tag); nothing below iOS 27 | Reading works: Calculator's 24 elements in a 0.75 to 1.0 s walk, Settings' 49 in 1.84 s, labels, identifiers and trait words, no frame anywhere. Acting does not: Activate on Calculator's 7 and All Clear returned within 3 to 4 ms without an error and changed nothing on the display (144 before and after), also with the inspector's focus confirmed on the key first. It may act only in apps built for debugging; not tried | | The WDA path on a phone, chosen in place of CoreDevice (`TestDeviceConnectionWay`, `TestDeviceWDAStart`, `iosctl -connection wda`, device tag) | The iPhone 13 mini, iOS 27.0.1, with the runner the owner had installed for another tool (see "Choosing the connection") | Reading, tapping, Back, typing English and Chinese, screenshots, the live picture, the lock state and unlocking all work; timings in "Choosing the connection". The phone asked for its passcode before the runner's first start after a restart, and again half an hour later | | iPhone Mirroring with the inspector, for a tree with Mirroring's picture and input | The iPhone 13 mini, iOS 27.0.1, the person locking it and mirroring Settings to this Mac (macOS 27.2) | No tree: while mirrored, the inspector's focus request and remoteAXService timed out, and the app service's process list closed, twice in a row; lockdown said `PasswordProtected` false and the backlight was off. The services still timed out after Mirroring was closed and the phone unlocked: for 8 minutes, through 90 s with no client connected and through another lock and unlock, while the tunnel, display info, HID and screenshots worked. Only restarting the phone brought them back | What the inspector offers per element, read from its focus messages on iOS 27.0.1: four sections, Basic (Label, Value, Traits, Hint, Identifier, User Input Labels, none settable), Actions (Activate, plus the element's custom actions such as Copy on Calculator's display), Element (class, address, view controller) and Hierarchy (its ancestors). Nothing gives a frame. Found on the Simulators and handled: - The edge swipe that is Back on iOS 27 did nothing on the iOS 18.5 Simulator through XCTest, so Back taps the navigation bar's back button when there is one and only otherwise swipes. - Screenshots are a pixel narrower than points times scale (1178 for 393 points at 3x), so the screenshot's size is the coordinate space, as on iOS 27. - Settings rows are a Cell holding an accessible Button with the title; only accessibility elements (and text fields and switches) become elements, or every row would show twice. - The keyboard is a `Keyboard` element with `Key` children in the app's tree; its keys only mark the keyboard as shown. The globe and dictation buttons beside it are listed. iOS 26 shows a one-time tip about sliding to type over the keyboard. - iOS 26 moved Settings' search to the bottom, with a close button instead of Cancel. Commands, with WDA built for a simulator (`wda-dd` is any build folder): ```sh xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner \ -destination 'platform=iOS Simulator,id=' -derivedDataPath wda-dd CODE_SIGNING_ALLOWED=NO build-for-testing TEST_RUNNER_USE_PORT=18100 TEST_RUNNER_MJPEG_SERVER_PORT=19100 xcodebuild test-without-building \ -xctestrun wda-dd/Build/Products/WebDriverAgentRunner_iphonesimulator*.xctestrun -destination id= PINKY_WDA_URL=http://127.0.0.1:18100 PINKY_MJPEG_URL=http://127.0.0.1:19100 \ go test -tags wdasim -count=1 -v -run TestWDASim ./internal/driver/ios/ PINKY_SIM_UDID= PINKY_WDA_URL=http://127.0.0.1:18100 PINKY_MJPEG_URL=http://127.0.0.1:19100 \ go test -tags wdasim -count=1 -v -run TestWDASimLinks ./internal/driver/ios/ /tmp/pinky-iosctl -wda http://127.0.0.1:18100 -mjpeg http://127.0.0.1:19100 observe ``` WDA on a simulator listens on every interface of the computer while it runs; stop it when done. `iosctl` opens a phone below iOS 27 the way Pinky would and runs the commands every path has (observe, tap, swipe, key, type, screenshot, stream, launch, apps, locked). ### Installing Pinky on the iPhone (Called "Installing Pinky on the iPhone (WebDriverAgent)" before 2026-11-12, and "Installing Pinky for iPhone" until 2026-11-13, when Pinky on the iPhone, Pinky's iPhone app, took the place of Pinky Helper as WebDriverAgent's host; "the helper" below is either.) Once per phone, on a Mac with Xcode, with the iPhone plugged in (or paired over Wi-Fi): **Install Pinky on the iPhone** in Pinky's window (below, "From Pinky's window"), or in Terminal: ```sh pinkyctl helper install ``` Without Homebrew's `pinkyctl`: `/Applications/Pinky.app/Contents/MacOS/Pinky helper install`. With several phones, name one by name or UDID (`pinkyctl helper install "Ann's iPhone"`); flags go before it: `--team `, `--bundle-id `, `--local-signing`, `--helper-only`. With a free Apple ID the phone first asks you to trust the developer: Settings › General › VPN & Device Management, then run the command again. On the phone it shows as Pinky, or, where Pinky Helper is installed instead (step 7), as Pinky Helper with Pinky's icon ("The helper's name and icon"). What it does (`internal/driver/ios/helperinstall.go`, `cmd/pinky/helper.go`): 1. **The phone.** From devicectl's list (`devicectl list devices --json-output`): the physical iOS devices paired with this Mac, the named one or the only one within reach. Developer Mode off stops it with the way to turn it on. 2. **The team and the bundle id.** It reads the helper already on the phone over usbmuxd (`findRunnerApp`), or, for a phone only on Wi-Fi, from devicectl's list of its apps (`devicectl device info apps`), which names no team. The team is `--team`, else that helper's (the prefix of its `application-identifier`), else the only team this Mac holds a valid Apple Development certificate for (the certificate's OU, from `security find-identity` and `find-certificate`); with several it asks for `--team`. The bundle id is `--bundle-id`, else that helper's when the team is the same or unknown, since iOS replaces an app only with one of its team, else `pinky..WebDriverAgentRunner`, which Pinky prefers among several runners. Pinky knows a runner by its executable, WebDriverAgentRunner-Runner, so any bundle id works. When iOS refuses to replace a helper another team signed, it says to name that team with `--team` or a new `--bundle-id`. So running it again replaces the helper in place. 3. **The source.** The release Simulators build (`wdaVersion`, "Where WebDriverAgent comes from"), from the archive in Pinky's data folder, checked and downloaded once. 4. **The build.** `xcodebuild build-for-testing -scheme WebDriverAgentRunner -destination id= CODE_SIGN_STYLE=Automatic DEVELOPMENT_TEAM= PRODUCT_BUNDLE_IDENTIFIER=`, first with the profiles on this Mac alone. Only when none signs the helper for the phone does it build again with `-allowProvisioningUpdates -allowProvisioningDeviceRegistration`, which may register the phone with the team and make a profile; `--local-signing` stops there instead. The embedded profile must include the phone. 5. **Name, icon, signature.** It brands the runner app (below) and signs it again with the certificate Xcode signed it with (`codesign --force --sign --preserve-metadata=identifier,entitlements,flags,runtime --generate-entitlement-der`), then checks it with `codesign --verify --deep --strict`. The frameworks and the test bundle inside keep Xcode's signatures; branding changes none of them. 6. **Install and check.** `xcodebuild test-without-building` installs it and starts WDA as Product › Test does, on the phone's loopback only (`TEST_RUNNER_USE_IP`). Once WDA prints `ServerURLHere` it stops the run and says when the signature runs out; it waits up to 4 minutes, as the phone may first ask for its passcode to allow automation. While Xcode waits for a locked phone ("Waiting for the destination to become ready") it says once to unlock it and keep it unlocked. 7. **Pinky's iPhone app in its place** (2026-11-12). Before installing the runner, it builds Pinky's iPhone app from the release's iPhone app kit with the same team, puts WDA's test bundle into it and installs it the same way, with the app as the test's host; WDA then runs in Pinky's app and the helper is not installed. Whatever fails there installs the helper instead, and `--helper-only` skips it. See [iphone-app.md, "Hosting WebDriverAgent"](iphone-app.md#hosting-webdriveragent). Xcode's output goes to `/WebDriverAgent/helper-install.log`; a failure names Xcode's errors and, when it recognizes them, what to do: trust the developer, turn on Developer Mode, unlock the phone, enter the passcode for automation, add an Apple ID in Xcode › Settings › Accounts. A helper running for Pinky at the time is replaced, and Pinky starts the new one when it next opens the phone. #### From Pinky's window (2026-11-11) On a Mac with Xcode, Pinky's window runs the same install (`driver.HelperInstaller`, `Gateway.InstallHelper`, `ui/helper.go`). It offers **Install Pinky on the iPhone** (**Install again** once the helper has a signature that runs out): - on the Phone screen of an iPhone that needs its helper installed, in place of See what's wrong (the checkup becomes a link under it), with a note that Xcode builds it and that the phone must stay unlocked; - in the reminder on the phone's page two days before the signature runs out, and in the checkup's helper item (soon or ran out) and its install-helper step; - in Add phone › Try an older iPhone › Install Pinky on the iPhone, for the phone Add phone found, or, with none listed yet, the only one Xcode reaches. While it runs, the button gives way to the step (getting ready, building with Xcode, installing on the phone) and Cancel; when Xcode waits for a locked phone, the step asks in bold to unlock it and keep it unlocked. Where the Mac can sign for several teams and the phone's helper names none, it asks which team, a button each. A failure says why in the person's words, by the fix the driver recognized (Xcode missing, phone out of reach, Developer Mode, no Apple ID in Xcode, another team's helper, no way to sign, the developer not trusted, the passcode for automation, a locked phone, the download), with Xcode's words in small print (the first 400 characters) and **Show the log**, which reveals `helper-install.log`. Done, a toast says so and the phone comes back on the helper. - **The phone meanwhile.** From the first step the provider lists the phone as needing its helper installed (`installingInfo`) and starts no runner, and the gateway lets go of it at once, so nothing of Pinky's competes with Xcode; the Phone screen shows the install instead of the phone. After, the provider forgets what it knew of the phone and its helper, and the gateway looks for phones again. - **One at a time.** Xcode installs on one phone at a time here, so the other buttons wait while one runs. The gateway refuses while an agent holds the phone ("An agent is working on …"). An install may take 20 minutes at most, the passcode included. - **Without Xcode or off a Mac** nothing changes: the same places say how to install it by hand and link here. - **The teams' room** (2026-11-12). A team's button is no wider than the page it is on: a longer label keeps its start and its end, where the team's ID is, around an ellipsis, and its tooltip has all of it (`btnOpt.shrink`, which takes the tooltip's cue from the button's width in the last frame, as a tooltip changes no layout). The Phone screen does not scroll: its column is no higher than the stage, and teams that take more room than the rest of the page leaves them scroll (`helperOpt.fit`), in a list that reaches into the stage's padding for its scroll bar, kept showing while they overflow (`KeepScrollbars`), whose buttons draw their focus rings inside themselves (`ringInside`), with Cancel under it, in view. The list takes the height that the page's layout leaves it, unlike the reminder's, which the reminder works out (`layoutReminder`), so it does not always end in half a team: its bar shows that it scrolls. The checkup and Add phone scroll as a whole, so there Cancel follows the last team. Before, a team of an organization ("Shenzhen Example Technology Co., Ltd.") was wider than the smallest window's stage, a longer name ran over the sidebar and the right panel and widened the row, which put the teams beside it out of the checkup's panel, and nine teams at 960×620 pushed Cancel, Checkup and Remove off the window. Unit-tested with fakes: the provider (`helperwindow_test.go`), the gateway (`helper_test.go`) and the window (`ui/helper_install_test.go`: from the Phone screen, a failure with its log, the team choice, Install again in the reminder and the checkup, a signature that ran out, Add phone; `TestIPhoneHelperTeamsFit`: a team wider than the page and nine teams, on the Phone screen and in the checkup, at 1280×820 and 960×620, in each language). Not yet run from the window on a phone. From Xcode, as before (the helper then keeps WDA's name, WebDriverAgentRunner-Runner, and Appium's icon or none): 1. Get WebDriverAgent: `git clone https://github.com/appium/WebDriverAgent` (Appium's copy works too). 2. Open `WebDriverAgent.xcodeproj`. For the WebDriverAgentRunner target, under Signing & Capabilities, pick your team (a free Apple ID works). If Xcode says the bundle identifier is taken, change its prefix (`com..WebDriverAgentRunner`), keeping WebDriverAgentRunner in it so Pinky finds it. 3. Pick the iPhone as the destination and run Product › Test once. Xcode installs WebDriverAgentRunner and starts it; with a free Apple ID the phone first asks you to trust the developer in Settings › General › VPN & Device Management. Stop the test once Xcode's log shows `ServerURLHere`. From then on Pinky starts the helper by itself whenever it opens the phone. With a free Apple ID the signature lasts 7 days, with a paid developer account a year; after that, click Install again in Pinky's window, run `pinkyctl helper install` again or repeat step 3 (see "When the signature of Pinky on the iPhone runs out"). The same from a terminal: ```sh xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner \ -destination id= -allowProvisioningUpdates DEVELOPMENT_TEAM= test ``` ### The helper's name and icon (2026-11-11) The helper used to show on the home screen and in App Library as "WebDriverAgen…" (WebDriverAgentRunner-Runner) with Appium's icon, or with a blank one, as on the owner's iPhone. Xcode makes the app that hosts a UI test, `-Runner.app` with the bundle id `.xctrunner`, from a template of its own (`XCTRunner.app` in the platform's `Developer/Library/Xcode/Agents`): its Info.plist is the template's, with the name and executable `WebDriverAgentRunner-Runner` and no display name, and of WDA's project only the bundle id and the test target's app icon reach it. No build setting names it: `PRODUCT_NAME` would rename the test bundle too. So Pinky brands the built runner app afterwards (`helperbrand.go`, `brandRunner`): - **The name.** `CFBundleDisplayName` is "Pinky Helper", with `InfoPlist.strings` in `zh-Hans.lproj` (「Pinky 助手」) and `ja.lproj` (「Pinky ヘルパー」), so a phone in Chinese or Japanese shows its own. 「Pinky 助手」 rather than the app's 「Pinky 辅助应用」: shorter, it fits under the icon; Japanese uses the app's own ヘルパー. All three showed in full on a Simulator's home screen. - **The icon.** Pinky's 1024-pixel iOS icon (`assets/icon/pinky-ios-1024.png`, embedded on macOS only), compiled by the Mac's `actool` as Xcode does for an app: `Assets.car` and the PNGs older iOS versions read go into the app, over Appium's, and actool's `CFBundleIcons` and `CFBundleIcons~ipad` into its Info.plist. - **What stays.** The bundle id, `CFBundleName` and the executable, so Pinky finds the runner as before (`pickRunnerApp`) and policy still knows it for Pinky's own (`policy.IsPinkyApp`). Helpers installed before keep working as they are. - **The mark.** `PinkyHelperBrand` (1) in its Info.plist: a branded build is not branded again, and a later Pinky that changes the name or the icon raises the number. - **Order.** The icon files and the translations first, the Info.plist last in one rename, so the app is never half branded. - **Signing.** A phone's build is signed again (step 5 above). A Simulator's build is not signed for one (`CODE_SIGNING_ALLOWED=NO`, Apple's template signature of `XCTRunner` stays), and the Simulator runs it as it is. Simulators get the same name and icon: `SimProvider.brandBuild` brands a build once, before its first run; a failure is logged and the run goes on unbranded. What ran where: | What | Where | Result | | --- | --- | --- | | A branded runner on a Simulator: the home screen, and WDA through `xcodebuild test-without-building` | iPhone 17e Simulator, iOS 26.5, Xcode 27.2 (27B52), 2026-11-11 | Pinky Helper with Pinky's icon, also in Chinese and Japanese; WDA printed `ServerURLHere` and answered `/status` | | Building for a phone with the profiles on this Mac, branding and signing again | `generic/platform=iOS`, the owner's team through its wildcard profile, "iOS Team Provisioning Profile: *", 2026-11-11 | The same certificate, team, identifier and entitlements (DER included) as Xcode's; two more sealed files (the translations); `codesign --verify --deep --strict` passes; nothing asked of Apple | | Branding with the Mac's actool, the brand mark, finding the branded runner, the phone and team choice, devicectl's list, certificates, hints | Unit tests (`helperbrand_test.go`, `helperinstall_test.go`) | Pass | | `pinkyctl helper install` on a phone, replacing the helper another tool had installed | The owner's iPhone 13 mini, iOS 27.0.1, on Wi-Fi only (no cable), Xcode 27.2, `--team 492J8Q67PF --local-signing`, 2026-11-11 | usbmuxd did not list the phone, so the helper on it came from devicectl's list (team unknown); built with the local wildcard profile, branded, signed again, installed in place over the old one with the same bundle id; WDA printed `ServerURLHere` and was stopped; the phone then listed it as Pinky Helper. The first try waited on a locked phone until it was unlocked, hence the notice above | ### When the signature of Pinky on the iPhone runs out (Called "When the helper's signature runs out" before 2026-11-12.) The same holds for Pinky Helper and for Pinky on the iPhone hosting WDA: both are signed with the person's Apple ID. Built on 2026-11-08 after a user's report. The helper is signed with the person's Apple ID, and the provisioning profile it is signed with runs out: 7 days after Xcode installed it with a free Apple ID, a year with a paid developer account. iOS then no longer starts it, and Pinky used to say only that the helper did not start. To install it again, on a Mac with Xcode click **Install again** in the reminder on the phone's page or in its checkup, or run `pinkyctl helper install` again (see "Installing Pinky on the iPhone"): it keeps the helper's team and bundle id, signs it anew and installs it over the old one. From Xcode, plug the iPhone in and repeat step 3 (Product › Test, stopped once `ServerURLHere` shows), or run the `xcodebuild … test` command above. Nothing else changes on the phone. While the phone stays plugged in, Pinky reads the new date within a minute and the reminder goes; a phone that needed setup comes back online within 30 seconds. What Pinky does (`wda_profile.go`): - **Reading.** Pinky lists the phone's provisioning profiles with misagent's `CopyAll` (the lockdown service `com.apple.misagent`, over usbmuxd), which only reads. go-ios's `misagent` package sends that request but drops the answer, so Pinky has its own small client. Each profile is a CMS signed message; Pinky takes the property list inside without checking the signature and reads `ExpirationDate`, the `application-identifier` entitlement and `ProvisionedDevices`. It reads BER as well as DER (indefinite lengths, the content in a constructed OCTET STRING), which `encoding/asn1` refuses. - **Matching.** The helper's profile is one that grants the runner on this phone: its `application-identifier` names the runner's bundle id, or a wildcard (`.*`) covers it, for the runner's team, which is the prefix of the runner's own `application-identifier` in the installation proxy's app list. Of several, the one that runs out last counts: iOS starts the app while any of them lasts, and an old profile stays next to a new one. - **The date.** `driver.Info.HelperExpires` (new, additive; zero when unknown) carries it for phones driven through the helper, Automatic below iOS 27 or chosen in Settings › Phones. List reads it while inspecting such a phone, again every 30 minutes while the phone is online, and every minute within the reminder (`driver.HelperReminder`, 48 hours), so a helper installed again while the phone stays plugged in is noticed. - **Once it ran out**, List inspects the phone again and lists it as `needs_setup` with `Setup` `install_helper` (`driver.SetupInstallHelper`, new) and a `Detail` that says when it ran out and how to install it again. The gateway then closes the phone, though a runner already running might have gone on until it ended: Pinky stops at the time it announced. A helper chosen in Settings › Phones but not installed now also gets `install_helper`. - **A profile Pinky cannot read** changes nothing: the phone works as before, without a reminder. Pinky's log gets each read (`iphone: the helper's provisioning profile`, expires), a change of the date, and a read that failed. - **The client.** Two days before, the phone's page shows a reminder above the phone: "The helper on stops working on : install it again with pinkyctl helper install on a Mac with Xcode", with How to install it. Once it ran out, the page of the phone that needs setup says so in the same words, with How to install it in place of the steps of Add phone, and Add phone › iPhone shows it too instead of "stuck, retrying". The checkup's `helper` item warns two days before, notes the day when it is further off, and fails with those words once it ran out (`ui/checkup_iphone.go`). - **The reminder's room** (2026-11-12). The reminder is as wide as the stage, up to 520, and as low as it can be (`layoutReminder`): its buttons beside its words, as a wide window shows them, or under the words where that is lower, as in a narrow stage, where the words beside a button wrapped into a column of ten lines and more. It is never higher than what leaves the phone below it its least height (`phoneFloor`, 240): words that would take more are cut to the lines that fit, with all of them in a tooltip (the checkup has them too), and more rows of teams than fit scroll under the words, the last one half in view, in a list whose buttons draw their focus rings inside themselves, as the list clips what is around them (`ringInside`), and that reaches into the reminder's padding for its scroll bar, which over the buttons' ends would take the presses there. A button wider than the reminder keeps the start and the end of its label, where a team's ID is, around an ellipsis, and its tooltip has the whole label. Asked which team, the reminder has Cancel after the teams, as the needs-setup page has, which forgets them and offers Install again (`reminderLayout.cancel`): beside them, at the end of the last row of them under the words where it fits there, or in a row of its own, which it has under teams that scroll too, so that it stays in view; the least the reminder takes under its words counts that row. Without it, the only way on from the choice was to pick a team. Before, the teams to choose from all went beside the words: with three teams, or two in the smallest window, the words had no width left, wrapped a letter a line and pushed the phone and its keys off the window. The words are a rich text of the span that `Context.MeasureText` measured (`reminderSpan`): a `Text` of their size has lower lines than `MeasureText` tells, which lays spans over the theme's text. Tests, in `ui/helper_test.go`: `TestIPhoneHelperReminderLayout` draws every form of the reminder (how to install the helper, Install again, each reason an install failed for, two teams and nine, with Cancel whole and in view) in each language, at 1280×820 and 900×600, with a date of ordinary length and the widest of a year, for a phone with a short name and one with a long name; `TestIPhoneHelperReminderCut` checks the cut words with their tooltip and the teams that scroll; `TestIPhoneHelperReminderCancel` that Cancel leaves the choice, after two teams and under nine; `TestLayoutReminder` the rule itself, with Cancel in each language and without it, in widths and rooms no window gives. They run on a fixed clock: `clockAt` sets the time the helper's words are told by (`timeNow`) and the tests' fake dates what it does by (`fakeapi.Options.Now`), and `wallTime` makes a date of what the clocks where the test runs show, so the words are the same at every hour and in every time zone. The layout test used to take the computer's time plus 20 hours: the width of the date, and with it a line break at 900×600, changed with the hour. On CI's Linux runner, which has no font for Japanese and measures it as missing glyphs, a two-digit hour made the reminder a line higher, the phone hit its least height and its keys ran off the window; with a cached test result, the same commit passed. What ran where: | What | Where | Result | | --- | --- | --- | | Reading the profiles and matching the helper's (`TestHelperProfileOnPhone`, read only) | iPhone 13 mini, iOS 27.0.1, 2026-11-08 | 59 profiles in 113 to 127 ms, all parsed (DER). The runner the owner had installed for another tool (bundle id ending in `.wda.xctrunner`) is granted by its team's wildcard profile, "iOS Team Provisioning Profile: *", from a paid account: 365 days, running out on 2027-10-08. The installation proxy's list gave the runner's `application-identifier` | | BER profiles, matching, the reminder and the expiry in the provider, the checkup item, the client's words and layout | Unit tests (`wda_profile_test.go`, `ui/helper_test.go`), the client's tester in English and Chinese, light and dark | Pass | | A profile that actually ran out; a free Apple ID's 7-day profile | Not yet | No such phone at hand. Whether iOS keeps an expired profile listed is not seen: if it removes it, Pinky finds no profile and shows the old failure to start | ```sh PINKY_DEVICE= scripts/devlock.sh go test -tags device -count=1 -v -run TestHelperProfileOnPhone ./internal/driver/ios/ PINKY_SCREENSHOTS= go test ./ui -run TestIPhoneHelperScreenshots ``` ### Choosing the connection Settings › Phones shows each iPhone's connection: Automatic (`pathFor`, as above), CoreDevice (iOS 27 and later, nothing installed) or Pinky on the iPhone (WebDriverAgent, iOS 17.4 and later, the helper installed). The provider implements `driver.ConnectionChooser`: `Connections` lists the three ways with whether each works on the phone and why not (`os_version`, `not_installed`, `offline`), and `SetConnection` keeps the choice and forgets what List knew of the phone, so the next List inspects it again. The gateway saves the choice in the history store (`device_connections`), gives it to the provider before its first discovery, and on a change takes the phone offline and runs a discovery pass at once: the phone reconnects the chosen way without a restart. A way the phone cannot use leaves it needing setup, with the reason and what to do: the helper chosen but not installed is `Setup` `install_helper`, CoreDevice chosen below iOS 27 is `connection` (`driver.SetupConnection`, `coreDeviceTooOldState`), which the app words in the person's language with Choose the connection, a way to Settings › Phones. Measured on the iPhone 13 mini, iOS 27.0.1, over USB, both ways (`TestDeviceConnectionWay`, the same steps each way): | | CoreDevice | WebDriverAgent | | --- | --- | --- | | Open | 0.7 s | 4 to 5 s: the runner starts each time | | Read the screen | 0.1 to 0.5 s | Calculator 0.5 to 0.7 s, Settings 1.8 to 2.9 s, the home screen 7.3 s, with a snapshot depth of 30 or 60 alike | | Tap | 0.06 s | 0.5 to 0.8 s | | Type `clock`, `你好` | 0.5 to 1.0 s (pasted) | 0.9 s, 1.1 to 1.3 s (XCTest's keyboard) | | Screenshot | 0.8 s | 0.15 to 0.18 s | | Live picture | The phone's HEVC display stream: 152 frames in 6 s | WDA's MJPEG (XCTest screenshots, 15 a second at half size), encoded to H.264 on the Mac: 562x1218, the first frame after 0.26 to 0.28 s | | Calculator's display after 7 | `7` | `Edit field, 7` | | Unlocking from the computer | Works | Works | | Banking apps that refuse screen sharing | DBS digibank warned of screen sharing while the stream ran | DBS digibank and China Merchants Bank opened without a warning, the MJPEG picture running | The banking apps were only opened, with the owner watching: nothing logged in or tapped, no screenshots. For the helper's run Pinky.app was quit, so no CoreDevice stream ran at the same time; DBS's first open while Pinky.app was still streaming had warned. iOS tells apps the screen is shared when it is mirrored or recorded (the display stream counts); XCTest's screenshots are single pictures and most likely do not count. The passcode for automation: - Before a UI test runs, iOS asks on the phone for its passcode to allow automation: here before the runner's first start after a restart, and again half an hour later, after the runner had stopped and the phone had been locked and unlocked. XCTest waits up to 60 s for it ("enabling automation mode" in the runner's log). Reports from Apple's forums and Appium say passcode-protected phones ask about once a day; when exactly iOS asks again is not known. - A passcode entered after the runner gave up still counts: the runner was stopped 2 s into the wait, the owner entered the passcode on the phone, and the next start, three minutes later, did not ask. - Pinky reads the runner's log for the wait (`runnerLog`). Once it has lasted 5 s (without a request it ends at once), the open returns `errAutomation`. The provider keeps that runner (`wdaWaits`), List reports the phone as needing setup with "Enter the passcode on the iPhone…" (`Setup` `passcode`, `driver.SetupPasscode`, which the app says in the person's words), and once WDA answers List reports it online and the next Open takes the runner over: the gateway no longer starts a runner every discovery pass while the phone waits. When the runner gives up, Pinky waits 2 minutes before starting it again, twice as long after each request in a row nobody answered, up to 30 minutes, and at once after a reconnect or when the person chooses the connection again. On the phone, the request was detected; the takeover and the waits are unit-tested only, and what the runner does once its 60 s are up was not seen. The phone already had a runner, installed by its owner for another WebDriverAgent tool (iPhone Use; executable `WebDriverAgentRunner-Runner`, bundle id ending in `.wda.xctrunner`, signed with the owner's development certificate). `pickWDARunner` finds runners by bundle id, name or executable, so Pinky used it as it was: nothing was reinstalled or signed again. Pinky starts it with its own environment, so it listens on the phone's loopback only while Pinky runs it (see Security notes); a WDA another tool already runs is used as it is. Also found: - Through the helper, `Screen.Keyboard` stays false: on iOS 27 the keys are not in WDA's tree, and XCTest's keyboard query fails on the phone. - Settings' search field drops out of WDA's tree while results show; typing goes on into it. - The runner's test log went to Pinky's stderr; it is now read for the passcode wait and dropped (`wdaLog`, tests print it). ### The helper's runner and auto-lock (2026-11-10) The owner's iPhone 13 mini (iOS 27, on Wi-Fi) did not lock by itself after a task that used the helper (a task connection, about 22:25 to 22:38; "back to the person's choice" at 22:38:27). The suspect: the runner (`WebDriverAgentRunner-Runner`) going on after Pinky ended its test run, with XCTest keeping the phone awake. Not reproduced; what was seen: - **How the runner was stopped.** Cancelling go-ios's run makes it kill the runner through appservice (`killTestRunner`). Pinky waited 5 s for the run to end and then closed the tunnel that request goes through. Over Wi-Fi requests to that phone took 3 to 25 s that evening (`slow screen read` in the log), so the kill could be cut off; a run that ends with its connection sends none. - **No runner afterwards.** At 22:55 appservice's process list had no process in the runner's bundle. The phone was locked by 22:59 (backlight off, `PasswordProtected` true), with Pinky still running and the phone open through CoreDevice since 22:38: the CoreDevice session (its stream stops after 30 s idle, `idle.go`), the lockdown heartbeat and the virtual keyboard did not keep it awake. usbmuxd carried about 1.5 KB/s, so no stream ran. powerd listed no display-sleep assertion. - **One more start and stop** (`TestWDAStopDevice`, Pinky's old stop, 23:06, phone unlocked): the runner started, iOS asked for the passcode to allow automation, the open returned `errAutomation` after 6 s, and go-ios's kill came back at once ("Test runner killed with success"); 4 s later no runner was listed. While the passcode request showed, SpringBoard turned its idle timer off (`SBTransientOverlayScenePresenterRequestedIdleTimer`); 15 s after the kill the request went away (`TransientOverlayDismissed`), and about 90 s later the screen was off by itself. So the request keeps the phone awake for as long as the runner waits on it (XCTest waits up to 60 s, see "The passcode for automation"). Whether a running runner that does not wait keeps the phone awake was not measured. The new stop was not run on the phone: the owner's unlocked minutes went to this run. Pinky now makes sure the runner ends (`wdaRun.stop`): after the 5 s it lists the phone's processes through appservice, kills those whose executable is in the runner's bundle (the installation proxy's `Path`), and looks again every 300 ms until none is left, for up to 30 s, through a new tunnel when the run's is gone; it logs a runner it had to end. `WDADevice.Close` and `wdaStart.stop` stop the run before closing the tunnel, so this covers the end of a task connection, a change of connection, the device closing and Pinky quitting (the gateway closes its devices). Unit-tested with a fake process table (`wda_stop_test.go`). A runner waiting for the passcode is stopped in the background while Pinky runs (`go s.stop()`); when Pinky quits, the gateway closes its providers after its devices, and `Provider.Close` stops those runners and waits for them to end. A runner Pinky did not start this time (WebDriverAgent already answering) is taken over and, before, was always left running when the device closed. Pinky now keeps a record per phone in `HelperRuns` in its data folder (`runRecords`, `Provider.RunDir`): written when it starts a run, removed once `stop` saw the runner gone. A record still there when WebDriverAgent already answers means an earlier Pinky started that runner and did not see it end (it crashed, was killed, or the kill did not get through), so the device ends it on Close and forgets the record. A runner another tool started (Xcode, with no record) is still left alone. The runner app is the person's own build, so its bundle cannot tell whose run it is. **Watching the phone keeps it awake.** (2026-11-10, 23:16 to 23:37) After the change above the phone again did not lock by itself, with no runner on it and no task. Pinky's window showed the iPhone's live picture, and that is the cause: the mirror is a view of the CoreDevice display stream, which runs as long as a view is open (`idle.go`), and iOS does not dim the screen while that stream runs. On the phone it shows as avconferenced's `fvdp_statsReport` once a second (about 42 encodes/s); the stream goes through Pinky's own tunnel, so usbmuxd's traffic does not show it. The owner switched Pinky to another page at about 23:35; the last `fvdp_statsReport` was at 23:35:30 (the stream's 30 s idle stop), and at 23:37:12 SpringBoard dimmed the screen on its idle timer (`ActiveOn->Dimmed: "idle timer"`). Whether the mirror should let go of the stream while Pinky's window is hidden, minimized or behind other windows is a question for the client. To look again: `TestWDAStopDevice` (tag `device`, `PINKY_WDASTOP=1`, `PINKY_STOPMODE=old` for the old stop) starts and stops the helper and lists the runner's processes afterwards; `TestIdleLogProbe` (`PINKY_LOGPROBE=1`, `PINKY_LOGKEYS=idletimer,powerd`) beside it shows what turns the idle timer off. Both need the phone unlocked, and the start may bring the passcode request. ### Task connection (2026-11-10) A phone left on Automatic with the helper installed uses both ways without the person changing the setting: CoreDevice for everyday tasks, the helper for a task that needs an app refusing screen sharing (the owner's weekly bookkeeping reads DBS digibank and China Merchants Bank). `internal/gateway/taskconn.go`: - **Who asks.** The agent, with `phone_connect`'s `connection` (`helper`, `coredevice`, `auto`), or Pinky, when `phone_launch` opens an app in `sharingRefusers` (`com.dbs.sg.dbsmbanking`, `com.cmbchina.MPBBank` on iOS and `cmb.pb` on Android for 招商银行, and the names people use for them), judged by the app the launch resolved (`prep.launch`, and the app a clone copies) as well as the agent's words: the field run of 2026.1110.27 opened 招商银行 by its iOS id and stayed on CoreDevice. `com.cmbchina.*` is 掌上生活, which is left out until it is seen to refuse. The launch hook sits in `execute`, which runs only once the gate (takeover, lock, open approval, watch-only scope) and the rules let the launch through, so a refused or still-pending launch never reconnects; `phone_batch`, `launchUnread` and approved launches (`runApproved`) go through it too, and tapping an app's icon does not. The app list moves nothing: the person may still refuse the app. `phone_connect`'s `connection` checks the same things first (`mayMove`). - **The move is the reconnect of a change in Settings › Phones** (`SetConnection`, `setOffline`, a discovery pass), not saved. Closing the CoreDevice session stops its media stream (`Device.Close` stops it), so no stream runs on the helper and every input goes through WDA; the mirror shows WDA's MJPEG. Calls wait for the phone (`d.moving`, `switchWait` 60 s); the phone's status reads "Switching to the helper". - **The passcode.** When the runner waits for it, `openWDA` returns `driver.SetupError` (`passcode`, wrapping `errAutomation`) and List reports `needs_setup` `passcode` from then on (`wdaState`). The gateway's `open` takes the step from the error at once: before, the failed open woke the switch before the next List, which read "the helper did not start" and fell back to CoreDevice while the prompt timed out on the phone unseen (field run, 2026.1110.27). The call returns `device_needs_setup` (`setupResultLocked`), the agent hands off, and Pinky shows a Needs you card of its own (`NeedSetup`, "Enter the passcode on …", with a notification) while a task waits for that step; the card hides behind the agent's handoff card and goes once the phone opens. The agent never types it. - **Failure.** A helper that is not installed (per `Connections`) is not tried: the message tells the agent to tell the person. One that does not start puts the phone back on the person's choice; the task goes on over CoreDevice and the message gives the open's error (`d.openErr`). - **The lease's grace.** `expireOffline` skips a phone while it switches, and the grace starts when the switch ends, so a helper that takes as long as `ReconnectGrace` to fail still falls back to CoreDevice. - **The person during a switch.** `Gateway.SetConnection` and `switchTo` give the provider a way under `g.mu`, and `switchTo` gives none once the person has chosen one; after its wait it leaves alone a phone whose choice changed meanwhile, and the agent is told. - **Back.** `switchBackAfter` (30 s) after the last lease ends, unless a task holds the phone again, so a round of tasks stays on the helper. A choice in Settings › Phones forgets the task's way; a restart forgets it too, as the provider starts from the saved choice. - **The person's choice wins.** A way chosen by hand (`g.chosen`, from `device_connections`) is kept for every task, and the agent is told. Unit-tested with a fake two-way provider (`taskconn_test.go`): asking, launching (allowed and denied), the app list moving nothing, a chosen way kept, no move while taken over or for a watch-only agent, a missing, failing or slow helper (longer than the grace), the passcode, a choice in Settings during and after a switch, Android. Not yet run on a phone: the switch's timing on the iPhone 13 mini, the stream really stopping (no screen-sharing indicator), DBS and China Merchants Bank opening after the switch, the passcode prompt in the middle of a switch, the switch back, and the mirror while it switches. Not done: the person marking their own apps as refusing screen sharing; telling the agent when Face ID's system prompt covers an app (the tree of that prompt on each path has not been recorded on a phone yet). ### The helper on demand (2026-11-10) CoreDevice has no switches (`driver.Switcher`: rotation, appearance, location) and no second finger (`driver.Pincher`, `driver.MultiToucher`); only the helper does. Rather than drive a phone on iOS 27 through the helper all the time, which is slower for everything (the table above) and asks for the passcode now and then, Automatic stays on CoreDevice and moves to the helper when something needs it, as for an app that refuses screen sharing (`internal/gateway/helperneed.go`): - **`phone_device` changing a switch** moves the task (`helperFor`, `useConnection` with a `helperNeed`) and goes on through the helper; reading the switches moves nothing. A change the agent's scope, a read-only task or read-only mode refuses is refused first (`switchRefusal`), so it never moves the phone. On a real phone the move comes before the "Phone settings" approval, which shows the switch's current value, read through the helper; a denied approval leaves the task there until it ends. The note says "Pinky moved this task to Pinky on the iPhone (WebDriverAgent) for turning the screen". - **`phone_pinch`** moves the task (`pinchHelper`, under the action lock, before the screen is read) and returns `stale` with the screen read through the helper: the agent's refs and points came from CoreDevice's tree. - **A task that changed a switch stays on the helper** until it ends (`useConnection`'s `keep`), so `putBackLocked` puts it back through the helper; `connection: "coredevice"` is refused meanwhile, with a note. Seen on the iPhone Air before this: a task moved back to CoreDevice after a turn left Calculator in landscape when it ended. - **The person's pinch on the mirror.** `Features.Pinch` is true over CoreDevice the same way, and the first `Touch2` (a touch down) starts the move in the background (`personPinchHelper`): that gesture is lost, and so is every touch while the phone moves (`d.switching`), quietly, though the phone is offline meanwhile. The next gesture reaches the phone through the helper. - **Staying while the person uses it.** While the phone waits to go back (`switchBack` set, no lease), each of the person's inputs on the mirror starts `switchBackAfter` again (`personInput`), so it does not go back under them; this holds after a task's move too. - **The person's Rotate.** When an iPhone or iPad comes online, `checkHelper` asks the provider's `Connections` whether the helper is installed (`device.helper`); with it, and the connection on Automatic, `Features.Rotate` is true over CoreDevice too (`helperReadyLocked`). `SetDeviceSwitch` then moves the phone (`personHelper`, `switchTo`), even while the person is in control, and turns it; without a task the phone goes back to CoreDevice `switchBackAfter` later, with one when the task ends. When the helper waits for the passcode first, the turn returns a `driver.SetupError` (`passcode`), which the client words for the person ("Enter the passcode on …", `phoneHelperPasscode`), and the move back is armed all the same, so the phone does not stay on the helper once the passcode is in (found in review: it stayed until the next task ended). `DeviceSwitch` reads nothing over CoreDevice. A turn made through the helper stays when the helper goes: on the iPhone Air (iOS 27.3) Calculator stayed landscape after the phone moved back to CoreDevice, read again 8 s later. Unit-tested with `twoWayPhone`'s `bareCore` (`helperneed_test.go`): a change and a read, the put-back on the helper, no helper, the pinch, the person's Rotate and a chosen CoreDevice turning it off, the person's pinch (the lost gesture, touches while moving, staying while used, no helper), a read-only task and read-only mode moving nothing, the person's turn waiting for the passcode. Tried on the iPhone Air (iOS 27.3, Automatic, the helper installed, Wi-Fi), 2026-11-10: Rotate showed over CoreDevice; Portrait moved the phone to the helper in 6 s, turned it, and the phone went back to CoreDevice 30 s later. An agent's `phone_device` landscape in Calculator moved its task and turned it in one call; `connection: "coredevice"` was then refused with the note; at release the turn was put back through the helper (Calculator portrait in the next task) before the phone went back. In that next task, `phone_pinch` returned `stale` with the screen read through the helper, its refs all new (the reason not to pinch by the old ones), and the next pinch settled. The owner's pinch on the trackpad over CoreDevice moved the phone (online on the helper 5 s later); its touches while it moved were refused as `device offline`, two error toasts, which `Touch2` now drops quietly (`TestPersonPinchWhileMoving` failed the same way before); the second pinch reached the gateway after the move and put off the move back by 2 s, which then came. ### Not done yet - Nothing below iOS 27 has run on a phone. The first such phone should get, in order: `iosctl devices` (the backend it picks), `TestAXProbeDump`, then `iosctl observe` and `stream` on each path and `tap` with the helper. - Mirroring a phone that Pinky drives on iOS 27 can leave its accessibility services stuck, so Pinky reads by OCR alone until the phone restarts (seen once, iOS 27.0.1 and macOS 27.2). Pinky does not notice it yet; a check that tells the person to restart the phone would help. - iPhone Mirroring as a way in: macOS 15 or later, iOS 18 or later, the same Apple Account on both, the phone locked nearby, not in the EU; window capture and mouse and keyboard events would need Screen Recording and Accessibility for Pinky. With no tree from the phone while it mirrors (above), it would read the screen by OCR alone, as DESIGN.md's planned Mirroring backend does; not built. - `Waker` through the helper; `Unlock` wakes the phone by itself. - Installing the helper without Xcode, on Windows and Linux: Pinky's window installs it on a Mac with Xcode ("From Pinky's window"); go-ios can sign with a certificate and profile (`ios/signing`), which would make it possible elsewhere. - The picture below iOS 27 on Windows and Linux (no encoder Pinky can call without cgo). ## Preparing a phone without Xcode Built on 2026-11-08. Before, Pinky needed Xcode for two steps: showing the Developer Mode switch (iOS hides it until a developer tool has seen the phone) and mounting the developer disk image (Xcode's Devices window did it; below iOS 27 nothing did it again after the phone restarted). Now Pinky does both. What remains of Xcode is the image itself, which Pinky does not ship. | Step | Before | Now | | --- | --- | --- | | Trusting the computer | The system asks: macOS when a phone is plugged in, Linux's usbmuxd too | The same on macOS and Linux. On Windows, where nothing else may ask, Pinky asks the phone itself (go-ios `Pair`) while it has no pair record, at most every 4 s | | The Developer Mode switch | Xcode reveals it when it sees the phone | Pinky sends AMFI's reveal (`com.apple.amfi.lockdown`, action 0) once per attach while Developer Mode is off. The person turns it on, the phone restarts and asks them to confirm. Pinky never confirms for them (AMFI's action 2 would): that question is the person's | | The developer disk image | Xcode mounts it. Below iOS 27 a restart drops it and nothing mounts it again until Xcode runs; iOS 27 keeps it (seen on 27.0.1) | Pinky mounts it in the background whenever the developer services are missing (see below): below iOS 27 after every restart, on any iOS when another tool took it off. | | The tunnel | Pinky's own (lockdown's CoreDeviceProxy, userspace) | Unchanged | ### Where the image comes from `findDiskImage` (`ddi.go`) looks, in order, in `PINKY_DDI` (an `iOS_DDI` folder or the `Restore` folder in one; when set, the only place), in Pinky's folder (`DeveloperDiskImages/iOS_DDI` in its data folder: `~/Library/Application Support/Pinky` on macOS, `%LOCALAPPDATA%\Pinky` on Windows, `~/.local/share/pinky` on Linux), and on macOS in Xcode's `/Library/Developer/DeveloperDiskImages/iOS_DDI`. Xcode installs that folder the first time it is opened (from its `MobileDeviceDevelopment` package), and it stays when Xcode is deleted. The newest image wins, by cryptex version, then by Xcode build; a folder missing a file its manifest names is skipped. Apple serves Xcode 27's image only inside Xcode, from the App Store or the developer site, both behind an Apple Account sign-in; `devicectl manage ddis update` copies from an installed Xcode and downloads nothing. Apple's public CDN has older images inside device-support packages (Xcode 16.2 and 26.0.x), which most likely lack the services iOS 27's path uses, and whose cryptex Apple's signing server refuses (26.0.1's 17B48: see "Images Apple no longer signs"). The mirrors other tools use are third-party copies: pymobiledevice3 downloads github.com/doronz88/DeveloperDiskImage (Xcode 27.0 beta 4, 27A5228h, files renamed, so not Apple's files byte for byte) and go-ios deviceboxhq.com (Xcode 26.4 beta, no cryptex). Xcode's license forbids redistributing Apple's software. The owner chose to offer the first mirror as a last resort (`mirror.go`), on these terms: only when no official copy is on the computer, or (the owner's calls on 2026-11-08 and 9) when Apple's signing server refused the official copy, Xcode's or one in Pinky's folder; only when the person clicks Download in Add phone, with every file pinned, the source named on screen as a third-party copy of Apple's files, a clear message when it fails or a file differs, and Pinky hosting and shipping nothing of Apple's. So: - The files come from one commit of the mirror (`e2aaeac8…`), from raw.githubusercontent.com, and each must have the size and SHA-256 Pinky pins, or the download stops with `driver.ErrDiskImageMismatch` and nothing is kept. An earlier download stays until a new one is complete. - The mirror keeps the image's two halves apart under new names (the cryptex, and the personalized image that iOS before 27 mounts). Pinky puts them back together under the names the cryptex half's manifest gives them (`022-22070-072.dmg` and its trust cache for the personalized half), in `DeveloperDiskImages/iOS_DDI-mirror` in its data folder, with a `version.plist` that says where they came from. The manifest's SHA-384 digests match the assembled files for every chip iOS 17 runs on; the 18 identities that differ (A8 and A9 chips, which no iOS 17 runs on) differ the same way in Apple's own 27B52 manifest. - `findDiskImage` takes that folder only when it finds no official copy; the preparer's look (`findDiskImageAvoiding`) also when every official copy here is one Apple's signing server refused. When Apple refused every image here, the look returns the download, whose refusal Add phone reports. - The image is a beta (27A5228h, cryptex version 27.1.5228.8, older than 27B52's 27.2.52.0). Decompressed, its cryptex has the same developer daemons as 27B52's (`dtremotedisplayd`, `dtuhidd`, `dtpasteboardd`, `dtdeviceinfod`, each the same number of times). Whether a phone mounts it and offers everything from it has not been tried here; a user of another tool reported it installed on an iPhone 12 with iOS 27.2 and touch working (JosssphZhou/poki#18, 2026-11-06). Apple's signing server still signed it on 2026-11-08. - The download is about 32 MB. Pinky asks for nothing else from the mirror, and sends only a plain `User-Agent: Pinky`. Clients reach it through `driver.DiskImageDownloader` (the iOS provider) and the gateway's `DiskImageMirror` and `DownloadDiskImage`; Add phone shows it under the official ways when a phone waits with `Setup` "no_disk_image", or with "disk_image_refused" for Xcode's image or one in Pinky's folder (not for `PINKY_DDI`'s, the only image Pinky uses while it is set, nor for the download's own); the checkup's note for those points to Add phone's steps. ### The mount - It starts when `List` finds the services Pinky checks for missing (the screen stream and HID on iOS 27, instruments below), in the background, one at a time per phone, once the phone is unlocked: a locked phone mounts nothing (CoreDevice refuses too, "the device was still locked"), and installing a cryptex on one waits instead of failing, so Pinky reads lockdown's `PasswordProtected` (and the iOS version) first and asks for the phone to be unlocked. While the image is being installed Pinky looks at the lock every 2 s, and a phone that locks stops the mount at once, reported as locked: go-ios would otherwise wait for the locked phone until the mount's timeout. Meanwhile the phone is `needs_setup` with `Setup` "preparing"; when the mount ends the provider forgets the phone and looks at it again at once. - `mountDiskImage` takes one way per phone, by its iOS (`mountWay`), as pymobiledevice3 does: iOS 27 and later get the image as a cryptex through cryptexd (`com.apple.security.cryptexd.remote`), as Xcode 27 installed it on the test phone; that is also the only way for models newer than the personalized identities of 27B52's manifest list (up to iPhone18,5, the iPhone 17 family), such as the iPhone 18 Pro. Earlier iOS gets the personalized image through the image mounter (Xcode 15's way). There is no second way after a failure: the first version tried the personalized image after a cryptex install gave up, in the background, which could only leave an iOS 27 phone with an image Xcode would not put there. Either way Apple's signing server (gs.apple.com) signs the image's digests for this phone with a nonce the phone made, through go-ios, and the phone checks that signature, so only Apple's own image mounts. Each mount sends the phone's ECID and chip identifiers to Apple, as Xcode does. - **Apple's signing server on Linux and Windows** (2026-11-08, Arch Linux, Pinky 2026.1108.18, an iPhone on iOS 27). gs.apple.com's chain ends in Apple's 2006 "Apple Root CA" (through "Apple Server Authentication CA"), which macOS ships among its System Roots but the Mozilla-based CA bundles of most Linux systems do not, so every ticket request failed with `x509: certificate signed by unknown authority` and Add phone said to check the internet connection. Both ticket requests, `cryptex`'s (iOS 27) and go-ios's `imagemounter` (below iOS 27), go through `http.DefaultTransport`, so `appleroot.go` (Linux and Windows) sets its `RootCAs` to the system's roots plus that root, embedded (`apple-root-ca.pem`, from https://www.apple.com/appleca/AppleIncRootCertificate.cer, SHA-256 `B0B1730E…1001F024`, the same as macOS's System Roots; `TestAppleRootCATrusted` pins it). Windows got it on 2026-11-09 (2026.1109.23) after the same error from a Windows computer (Discord): Windows ships with few roots and downloads the others, Apple Root CA among them, the first time a chain needs one (Automatic Root Certificates Update), which does nothing where an administrator turned it off or Windows Update cannot be reached. A pool from `x509.SystemCertPool` keeps Windows' own verifier: Go 1.27 (`crypto/x509/verify.go`) asks Windows first and, only when Windows does not trust the chain, checks it against the roots added to the pool, so nothing Windows trusts is lost. Not on macOS, which trusts it already. TLS verification stays on (go-ios's workaround in danielpaulus/go-ios#875 is plain HTTP). A certificate error now gets the generic "could not mount" detail with the error, not the network one. Checked on that machine with a stock Mozilla bundle (`SSL_CERT_FILE`): a request to gs.apple.com through `http.DefaultTransport` failed with the x509 error before and got HTTP 200 after. Not checked: a full mount on Linux with the change, as both phones there already had the image (the machine's own trust store had meanwhile been given Apple Root CA by hand). - It keeps what is there: a cryptex of the same version is left alone (nothing is signed or sent), an older one is replaced, a newer one is left alone and reported. A personalized image that is not this one is replaced: Pinky gets there only when the services are missing. - After mounting it waits up to 10 s for the services. If they do not come, it says whether the image is older than the phone's iOS (then Xcode needs updating) or asks for a restart of the phone. - A mount gives up after 90 s. A failed mount is tried again by what went wrong: a locked phone after 3 s (`Setup` "unlock": a phone that just restarted mounts nothing until it is unlocked once), no image after 10 s ("no_disk_image", with where to get one), no network after 20 s, other errors after 30 s. What trying again cannot fix (the image does not know the phone, a newer image on the phone, services missing after a mount) waits 10 minutes or until the phone reconnects. An image Apple's signing server refused is not tried again at all: each time `List` looks at the phone (every 30 s), the provider looks for the image again, and mounts at once when it finds one Apple has not refused (Xcode updated, an image copied in or downloaded), none at all, or when the phone reconnects ("Images Apple no longer signs"). Within a mount, a refused image gives way at once to another here that Apple has not refused: Xcode's copy to the mirror's download. The refusals are kept across runs in `refused.json` in Pinky's folder for images (by build and version), so a restart does not go back to a refused copy, and a signature clears one. A restart is a new usbmuxd attach, which starts over. - `List` asks a phone that waits for a step again sooner than other phones that are not online: every 2 s while it waits for Trust, every 3 s while it is being prepared or must be unlocked, so Add phone moves on soon after the person's step. - The developer services are checked once per usbmuxd attach, so a phone that reconnects or restarts is always looked at again, and every minute while a phone is online (`recheckOnline`): an image taken off while the cable stays in is mounted again. A failed look keeps the phone online; only services seen missing count. A CoreDevice `Open` that fails forgets the phone, so the next `List` looks again. Before 2026-11-08 (2026.1108.7) an online phone was looked at only when it reconnected, and a "services there" result was reused for 5 minutes across reconnects. - Each step goes to Pinky's log (`Provider.Log`): `iphone: mounting the developer disk image` (image, folder), `iphone: developer disk image ready` (way, kept, replaced, took), `iphone: could not mount the developer disk image` (way, took, err, retry_in), `iphone: Apple's signing server refused the developer disk image` (image, version, dir, way, took, err), `iphone: the developer services went away`, the Developer Mode switch, the trust request on Windows and the mirror download. - **The cryptex install on iOS 27.0.1** needed three fixes (2026-11-08, iPhone 13 mini; "What ran where"). Before them every install waited until the mount's 90 s timeout, the phone unlocked, while Apple's devicectl installed the same image in half a second. 1. go-ios's `installCryptex` waited on each payload's stream for the phone's FileOpenReply before it wrote the payload (`xpc.FileTransferStream.WaitAccepted`). cryptexd sends that reply only after the payload. Pinky mounts with a copy of go-ios's cryptex code that sends each payload at once (`internal/driver/ios/cryptex`, MIT, its README says what changed); pymobiledevice3, written from devicectl's traffic, does the same and notes that a device resets a stream held open waiting for the reply. A fake cryptexd that behaves this way shows go-ios's install waiting forever and the copy installing. 2. Pinky closed the RSD connection right after announcing the host, sometimes before the phone took the announcement, which left the tunnel a legacy peer's (see "Tunnel"). Fixed with `rsdLinger`. 3. The upload stalled after the first full-size packets with the tunnel's 16000 MTU (see "Tunnel"). The mount's tunnel asks for 1280 (`uploadMTU`). With all three, Pinky's provider mounted Xcode's 27B52 in 1.4 s after the image had been taken off, and the phone was online again 2.3 s after the take-off. ### Images Apple no longer signs Field report, 2026-11-08: a person with iOS 27.3 (a developer beta then) stopped at Add phone's preparation step with `failed to get Cryptex1 ticket from Apple: unexpected status in response 94: "This device isn't eligible for the requested build."` (DIYgod/pinky#6 and on X, by the same person; macOS 27.3 and Pinky 2026.1108.14). The screen showed only that sentence, said Pinky would try again, and tried the same image every 10 minutes. The report did not say which image Pinky had used. The Cryptex1 ticket request (`cryptexTSSRequest`) carries the image's three digests (`Cryptex1,CryptexInfoPlist`, `GenericTrustCache`, `GenericVolume`), the identity's fixed fields (chip 0xFF10, type 3, product class 0xF2, version `39.999.999.0.0,0` in 27.x images), the phone's ECID and chip in `Cryptex1,UDID`, and its nonce. Nothing in it says which iOS the phone runs, so whether Apple signs depends on the image, not on the phone's iOS. Measured on 2026-11-08 from this Mac with requests built as the mount builds them, a made-up ECID and nonce, sent to gs.apple.com with curl: | Image | Apple's answer | | --- | --- | | Xcode 27.2's, 27B52 (cryptex version 27.2.52.0), the newest release of Xcode then | Signed | | The mirror's, 27A5228h (Xcode 27.0 beta 4, 27.1.5228.8) | Signed | | Xcode 26.0.1's, 17B48 (17.2.48.0, identity version `29.999.999.0.0,0`), from Apple's public device-support package for Vision Pro (M5) and iPad Pro (M5) | Status 94, "This device isn't eligible for the requested build." | | 27B52's request with random digests | Status 94, the same sentence | | 27B52 and 27A5228h with chip IDs from 0x8101 to 0x8170, and 0x9999 | Signed for every chip | Another tool's user saw the same refusal for Xcode 26.6's image (17F113) and a signature for 27A5228h (JosssphZhou/poki#18). So an iPhone that takes the image as a cryptex (iOS 27 and later) needs the image of Xcode 27 or later, and a Mac whose only image came with an older Xcode gets status 94 whatever the phone's iOS. That fits the reports (a second person, on iOS 27.0.1, stopped at the same step the same day): Pinky took Xcode's copy before the mirror's, and a person with an old Xcode was never offered the mirror. Xcode 27.3 beta's image (27C5020n) was not available to check; Apple's public index of downloadable components lists no developer disk images for Xcode 27. Since 2026-11-08 a refused mount leaves the phone with `Setup` "disk_image_refused" and `Info.DiskImage` (where the image was found, its build, version and folder). Add phone says which image it was and how to get one Apple signs, by where it came from: Xcode's, install or update Xcode from the App Store and open it once, or download the third-party mirror's image (offered there too since the owner's call, on a click); the mirror's, install or update Xcode, or a Pinky with a newer image; one in Pinky's folder, replace it with the folder of a Mac with the latest Xcode, or download the mirror's image (since 2026-11-09: on Windows and Linux the only other way); `PINKY_DDI`'s, replace it. Pinky tries again as soon as another image is there, and not with the same one ("The mount"). ### Steps for clients `driver.Info.Setup` (new, additive) names the step a phone waits for: `trust`, `trust_record` (the iPhone trusts this computer, but Apple's device service could not replace the pairing record it holds on Windows: see "A pairing record Apple's service cannot replace"), `developer_mode`, `unlock`, `preparing`, `no_disk_image`, `disk_image` (the mount failed; `Detail` says why), `disk_image_refused` (Apple's signing server refused the image; `Info.DiskImage` says which: see "Images Apple no longer signs"), `update` (iOS too old), `install_helper` (the helper is missing, or its signature ran out: see "When the helper's signature runs out") `passcode` (iOS asks for the passcode before the helper may control the phone: see "The passcode for automation") `connection` (the connection chosen in Settings › Phones does not work with the phone: see "Choosing the connection") and `media_in_use` (an app uses the camera or microphone, and iOS 27 does not mirror the screen then: see "An app using the camera or microphone"). `Detail` still says it in a sentence. Add phone › iPhone (`ui/addiphone.go`) shows them as numbered steps, each saying what to do on the phone, and the reason when one is stuck; the Phone screen of an iPhone that needs setup links to them. The steps replace the pairing-code form, which no iPhone used. ### The computer Every connection to an iPhone goes through usbmuxd. macOS has it built in. Windows gets it with Apple's iPhone driver, from the Apple Devices app (Microsoft Store) or iTunes. Linux gets it from the `usbmuxd` package, whose daemon udev starts while an iPhone is plugged in, so at Pinky's launch it is often not running. Before, Pinky left iPhones out for the whole run when usbmuxd did not answer at launch; now the provider always starts, `List` returns no phones while usbmuxd is missing (once it has listed phones, the first two failures in a row are errors instead, so a hiccup does not drop them from the gateway), and `CheckHost` (`driver.HostChecker`, through the gateway's `HostChecks`) says what is missing: `apple_devices` (Windows), `usbmuxd` (Linux, not installed) or `usbmuxd_stopped` (Linux, installed but not running). `pinkyctl devices` prints it too. ### Checkup iPhones answer the checkup page (`driver.Checker`, `checkup.go`), reading only: no taps, no keys, no settings, nothing mounted. | Item | How it is checked | When it fails or warns | | --- | --- | --- | | `developer_mode` | The image mounter's `QueryDeveloperModeStatus` | Fix `developer_mode` | | `disk_image` | What the phone has mounted (cryptexd's list, else the image mounter's), the services the path needs (screen and HID on CoreDevice, `legacyReady`'s below iOS 27), and whether this computer has an image to mount again should it go missing (the preparer's look in Pinky's folder, Xcode's and the download, passing over the images in `refused.json`) | Fails with fix `disk_image` when the services are missing; warns with `no_disk_image` when nothing here could mount it again | | `helper` | WebDriverAgent's `/status`, on the helper's path, and when its signature runs out, read afresh; without the helper it fails | Fix `install_helper`; warns two days before the signature runs out, and once it ran out fails, or warns while the helper still runs | | `screen` | A screenshot | Fix `screenshot` | | `input` | CoreDevice: the HID service accepts a connection, which is closed again without an event (any event would reach the phone). Without the helper: fails | Fix `install_helper` below iOS 27 | The gateway puts the connection items first; for an iPhone that is not online, the `authorized` item's fix is its `Setup` step, which the client puts into the same words as Add phone's steps, with a button to them (`ui/checkup_iphone.go`). ### What ran where | What | Where | Result | | --- | --- | --- | | Finding the image | This Mac, Xcode 27.2 | Xcode's 27B52, cryptex version 27.2.52.0 | | How the phone has its image (`TestDiskImageStatus`, read only) | iPhone 13 mini, iOS 27.0.1 | Developer Mode on, cryptexd offered, one cryptex `com.apple.MobileAsset.DDI` 27.2.52.0 (this Mac's image), no personalized image | | Pinky's mount on a phone that has the image (`TestDiskImageMountKeeps`) | The same | Kept as it is in 7 ms: no signing request, no upload, the same cryptex afterwards | | Taking the image off and letting the provider mount it again, as after a restart (`TestDiskImageRemount`, with the owner's leave) | iPhone 13 mini, 2026-11-08 | Taking it off worked (3 s). Then every cryptex install (the provider's, its retry and the test's own clean-up) waited until its 3-minute timeout: the phone was locked (Apple's devicectl refused at once, "the device was still locked", and read `passcodeRequired` true). go-ios's install waits for the phone to accept each file before it reads the install's reply, so a refusal makes it wait instead of fail. Afterwards the phone listed 64 services instead of 85: no image at all. Instruments (`dtservicehub`) and `remoteAXService` were still listed because iOS 27 offers them itself: neither is in the image (Xcode 27.2's cryptex and personalized images, attached read only on the Mac, publish `displayservice`, `universalhidservice`, `screencaptureservice`, `pasteboardservice` and `deviceinfo` from `/Library/LaunchDaemons/com.apple.coredevice.*.plist`, and nothing else Pinky uses). So none of Pinky's mounts put anything on the phone, and the personalized try in the background did not either: both images carry the screen and HID daemons. Pinky now checks the lock first and watches it during the mount. The coordinating session put the image back with `devicectl` once the phone was unlocked. On an unlocked phone the install waits too (two rows down) | | The checkup (`TestCheckupOnPhone`, read only) | iPhone 13 mini, iOS 27.0.1, 2026-11-08, after the image was put back | All four items pass in 1.0 s: Developer Mode on; the image 27.2.52.0 mounted as a cryptex, and Xcode's 27B52 here to mount again; a 1125×2436 screenshot; the HID service answers (no event sent) | | A restart, Pinky.app 2026.1108.7 running, Xcode quit (read only) | iPhone 13 mini, iOS 27.0.1, 2026-11-08 11:13 | The phone came back with the cryptex 27.2.52.0 already there: Pinky opened it 52 s after boot (launchd's start in the process list), though nothing on the Mac mounted it (no mount in CoreDeviceService's log, no lookup of Apple's signing server by any process). iOS 27 keeps the cryptex image across restarts | | Pinky.app's own mount: the image taken off (3 s) and the cable plugged in again, the phone unlocked, with the owner's leave | iPhone 13 mini, iOS 27.0.1, 2026-11-08 11:43 | While the cable stayed in, Pinky.app did not notice for 2 minutes (it looked at online phones only when they reconnected; now every minute). Plugged in again, it saw the services missing in 11 s and mounted Xcode's 27B52 the cryptex way: three tries, each waiting until its 90 s timeout. cryptexd's log: the request and five file streams arrived, then nothing for 31 s until the connection closed ("image file transfer: Broken pipe" for all five). devicectl then mounted the image in 2.5 s, and Pinky showed the phone online 3 s later | | Finding why the install hung (with the owner's leave; the image taken off and put back with devicectl each time) | iPhone 13 mini, iOS 27.0.1, 2026-11-08 13:25 to 13:59 | Sending the payloads at once alone: still waits. The phone's syslog: cryptexd said "Invalid or missing remote device connection version flags" on the tunnels where remoted had logged "Device must renegotiate TLS" (the RSD connection closed too soon); with the RSD connection kept open every tunnel got its flags, and the install still stalled after 1,047,632 bytes of the image. The tunnel's counters: after the first 27 full-size packets nothing came back from the phone, gVisor retransmitted 7 times, and the phone's TCP timed out after 18 s. The goroutines: the tunnel's reader waiting on the network, its writer idle. Once at 13:46 Apple's signing server failed TLS with an expired certificate, through this Mac's proxy; a few minutes later it was valid again | | Pinky's own mount, all three fixes (with the owner's leave) | iPhone 13 mini, iOS 27.0.1, 2026-11-08 14:03 and 14:06 | `mountOn`: the image (17.2 MB) in 0.5 s, installed 6.7 s after the start. The provider, as the gateway runs it, after the image was taken off: missing services seen in 0.2 s, "iphone: developer disk image ready" way=cryptex took=1.418s, online again 2.3 s after the take-off; then the checkup, all four items pass | | The home screen's tree after the restart (read only) | iPhone 13 mini, iOS 27.0.1, 2026-11-08 11:53 | "Home Screen" (SpringBoard), 25 elements from the tree, no OCR, 0.56 s; no "no frontmost app" | | Downloading the mirror's image (`TestDownloadMirrorLive`) | This Mac, from GitHub | 32 MB in 37 s, every file matched its pin, the assembled image read as 27A5228h, cryptex version 27.1.5228.8 | | The Developer Mode switch | Not yet | The test phone has Developer Mode on, and turning it off is not allowed | | Pairing on Windows | Not yet | Unit tests only | | Pinky's own usbmuxd on Windows, on the cable and over Wi-Fi, with the heartbeat | iPhone 18 Pro Max, iOS 27.2, Windows 11 with Apple Devices, 2026-11-09, by the contributor (DIYgod/pinky#25) | On the cable listed on USB and found on Wi-Fi at its Wi-Fi address only; unplugged, online over Wi-Fi 1.5 s later, then `phone_connect` and a read of its screen. Without the heartbeat, AMFI, the image mounter and `CoreDeviceProxy` closed after TLS. The review's changes (the inherited address, pair records, timeouts) came after this check | | usbmuxd on Linux, on the cable and over Wi-Fi (Pinky's own usbmuxd, `netmux`) | iPhone 16 Pro and iPad Pro 11-inch, iOS 27.2, Arch Linux, 2026-11-09, by the contributor (DIYgod/pinky#20) | Found on Wi-Fi within 3 s, opened over it with the live picture, 12 minutes online; moved to USB in 0.4 to 0.5 s when plugged in, back to Wi-Fi 1.4 to 2.8 s after unplugging. See "On Linux" | | The steps in Add phone | The client's tester, light and dark, English and Chinese | Each step and each stuck case drawn and checked | ```sh PINKY_DEVICE= scripts/devlock.sh go test -tags device -count=1 -v -run 'TestDiskImage(Status|MountKeeps)|TestCheckupOnPhone' ./internal/driver/ios/ # Takes the image off an unlocked phone and lets Pinky mount it again (ask the owner first): PINKY_DEVICE= PINKY_REMOUNT=1 scripts/devlock.sh go test -tags device -count=1 -v -run TestDiskImageRemount ./internal/driver/ios/ # Downloads the mirror's image (32 MB) into a temporary folder: PINKY_MIRROR_LIVE=1 go test -count=1 -v -run TestDownloadMirrorLive ./internal/driver/ios/ ``` The demo (`go run ./cmd/pinky-ui-demo`) has Demo › Plug In a New iPhone, which walks a made-up iPhone through the steps. ## iOS Simulators (2026-11-09) On macOS, every booted iOS Simulator is an iPhone or iPad in Pinky (`sim.go`, `simwda.go`, `SimProvider`), with `Info.Virtual` set: the app and Settings › Phones say "Simulator" where a phone says USB or Wi-Fi. It goes through the gateway like any phone, so approvals, rules, the mirror and recordings are the same. The person starts and shuts Simulators down in **Add phone › Virtual phone** (`driver.VirtualLauncher`: `simctl boot`, with Simulator.app opened on the device so it shows, and `simctl shutdown`); agents have no tool for either. Xcode 27 has no Simulator.app: its Device Hub (`Contents/Applications/DeviceHub.app`, `com.apple.dt.Devices`) opens on the device it showed last, on this Mac the owner's iPhone, and takes no device to show (no `-CurrentDeviceUDID`, and its `devices:` URL scheme is undocumented), so Pinky opens nothing there and the person sees the Simulator in Pinky's mirror. Pinky creates no Simulators and downloads no runtimes. A Simulator shows no app installs, logs or links: other work adds those for phones. ### How Pinky drives one - **Xcode.** Found the way `xcrun` finds it, without running `xcrun` (which on a Mac without the command line tools asks to install them): `DEVELOPER_DIR`, then xcode-select's folder (`/var/db/xcode_select_link`), then the Xcode apps in `/Applications`, `Xcode.app` first. Looked for again every half minute. Without Xcode, Add phone says that iOS Simulators need Xcode (`VirtualPhones.Missing = "xcode"`); a Mac without Xcode still uses iPhones, so `SimProvider` is no `driver.HostChecker`. - **Listing.** `simctl list -j devices`, iPhones and iPads only, and only runtimes that are available (`isAvailable`). It costs 0.2 to 0.4 s, so Pinky reads it again only when the device set changed: CoreSimulator writes each device's state to its `device.plist`, and the names, times and sizes of those files are the stamp (`setStamp`). - **The helper.** Appium's WebDriverAgent, built by Pinky with the Mac's Xcode (below). Until it is built, a booted Simulator needs setup (`SetupPreparing`, "building"); when the build fails, `SetupSimulatorHelper` with Xcode's reason, built again after 5 minutes or at **Try again**. No iPhone steps (Developer Mode, the disk image, trust) apply. - **Running it.** Each Simulator Pinky opens gets its own `xcodebuild test-without-building` run of that build, the way Xcode's Test starts it, with WDA and its MJPEG server on free ports of the loopback (`TEST_RUNNER_USE_IP=127.0.0.1`, `TEST_RUNNER_USE_PORT`, `TEST_RUNNER_MJPEG_SERVER_PORT`, which xcodebuild hands to the test without the prefix): a Simulator shares the Mac's network, and WDA would else listen on every interface. Then the device is the WDA path of "iPhones below iOS 27" (`WDADevice`), with the app list from `simctl listapps`. Closing the phone stops the run with SIGTERM, which ends xcodebuild and WDA within 3 s; SIGINT makes xcodebuild collect diagnostics for minutes. Each run has its own derived data folder in `/WebDriverAgent/runs`, removed when it stops, so Xcode's own DerivedData stays untouched; xcodebuild runs in its own process group, so a Ctrl-C in the terminal that runs Pinky does not reach it. Nor does Pinky's death, as macOS has no parent-death signal: each run has a guard (`runGuard`, `sim_darwin.go`, 2026-11-09), a `/bin/sh` in a group of its own that reads a pipe only Pinky holds open. When the run ends, Pinky writes a line and the guard exits; when Pinky is killed, the pipe closes with nothing written, and the guard sends the run's group SIGTERM, SIGKILL 5 s later if it still runs, and removes the run's folder (`TestKilledPinkyEndsItsRun` kills a Pinky running a fake xcodebuild). ### Where WebDriverAgent comes from Pinky ships no WebDriverAgent and downloads no binary. The first time a Simulator needs it, Pinky downloads the source of one release, [Appium's WebDriverAgent](https://github.com/appium/WebDriverAgent) v16.15.3 (Apache 2.0, the latest release on 2026-10-16), as the archive GitHub makes of its commit (`02cb92f0e93c8b021443bcbb8eeb80932c113220`), and checks it before building anything: - The archive may be at most 16 MB, and so may what it unpacks to. Only regular files and folders are unpacked (no links, which could point out of the folder), and names that would leave the folder are refused. - The files must hash to `wdaSourceTree` (`treeHash`: the SHA-256 of one line per file, sorted by path, with its mode, 644 or 755, its SHA-256 and its path), not the archive's bytes, whose compression GitHub may change. A mismatch is `errSourceMismatch`, and nothing is built. Why this source: Appium's WDA is what Pinky's iPhone helper already is (`pinkyctl helper install` builds the same release for a phone), it is open source under a permissive license, and building it from a pinned, hashed source means nothing Pinky did not check runs on the Mac. A Simulator needs no signing (`CODE_SIGNING_ALLOWED=NO`), so the build needs no Apple ID or team. `xcodebuild build-for-testing -scheme WebDriverAgentRunner -destination 'generic/platform=iOS Simulator' CODE_SIGNING_ALLOWED=NO ARCHS=`, in a folder of its own, builds it; the Products folder moves to `/WebDriverAgent/builds/-xcode-` (`16.15.3-xcode27B52-arm64`), and builds for other keys are removed. A new Xcode build, a new WDA release or another CPU builds again. On this Mac (M-series, Xcode 27.2, 27B52) the build took 20 s; the first build after an Xcode update, when Xcode compiles its Simulator modules too, can take minutes, so the build may run 15. The archive (0.6 MB), the build (25 MB) and the last build's log (3.4 MB) are about 30 MB together. Before a build's first run Pinky gives it the helper's name and icon (`brandBuild`, "The helper's name and icon"), in about half a second. ### iOS 27 Simulators: nothing better than WDA What iOS 27 phones use, and the Simulator lacks: - On a phone, Pinky's iOS 27 path (Device Hub's protocol) runs over a RemoteXPC tunnel to the phone's services: `remoteAXService` for the tree, the HID service for touch, the screen stream. A Simulator has no RSD tunnel and no such services: it shares the Mac's kernel, and CoreSimulator talks to it through its own private frameworks (SimulatorKit, and IndigoHID for touches and buttons, which Simulator.app uses). - The iOS 27.0 Simulator runtime has `axauditd` (`com.apple.accessibility.axAuditDaemon.deviceservice.plist`), whose `remoteAXService` is the service Pinky reads on phones; on a Simulator it is only reachable through RemoteXPC, and its client needs an AppleInternal entitlement. No `dtuhidd` either. - Xcode 27's DeviceKit (Device Hub) pairs Simulators and sets their location; it does not share their screen or send input to them. - `simctl` has no input or accessibility commands: `io screenshot` and `io recordVideo` for pictures, nothing to tap or read. So WDA stays the way: a tree with frames, touches, typing and an MJPEG picture, the same code as the iPhone helper. Driving IndigoHID through private frameworks would give faster input but no tree, and would break with each Xcode. On this Mac the iOS 27.0 runtime (24A434) is listed by `simctl list runtimes`, but its disk image is missing, and booting a Simulator on it fails ("The iOS 27.0 simulator runtime is not available", SimError 401). Pinky leaves Simulators of unavailable runtimes out of the list. Testing used iOS 26.5 and 18.5. ### What ran where | What | Where | Result | | --- | --- | --- | | `TestSimProvider` (`-tags wdasim`): download and check the source, build, list the Simulator as preparing then online, start WDA, list the apps (Settings there, the runner not), open Settings, read the screen, go home, the lock state, the checkup, close (WDA gone, the run's folder removed) | iPhone 17e Simulator, iOS 26.5, Xcode 27.2 (27B52), Apple silicon | Pass. Build 20 s; boot 7 s; WDA answering 18 s after the run started; only 127.0.0.1 listening | | The WDA path itself (`TestWDASim`, "What ran where" above) | iOS 18.5 and 26.5 Simulators | Pass | | The app: Add phone › Virtual phone, Start, the phone online, the mirror, an agent's task through MCP, Shut down, Start from the phone's page | iPhone 17e Simulator, iOS 26.5, Xcode 27.2 (27B52), Apple silicon; see "Virtual phones end to end" below | Pass | | The app with an Android emulator: the same steps | AVD with Android 16 (API 36 `google_apis`, arm64-v8a), emulator 37.2.13, Apple silicon; see "Virtual phones end to end" below | Pass | | Listing, the stamp, Xcode lookup, the archive and tree checks, build keys and retries, starting and stopping, unavailable runtimes | Unit tests (`sim_test.go`) | Pass | ### Virtual phones end to end `cmd/pinky/virtual_e2e_test.go` (`-tags e2e`) drives the e2e build of the app as the person would: Add phone › Virtual phone lists the Simulator or AVD, **Start** starts it, it turns up online in the phone list with its kind ("Simulator", "Emulator") on its page and its live picture, an agent's first `phone_connect` waits for the person (rule "New agent", pressed **Approve**, the same as for a phone), the agent opens Settings in a task and releases it, the task is in history with its recording, **Shut down** in Add phone takes it offline, and **Start** on its page ("It isn't running.") brings it back under the same id. The app runs with `PINKY_DEVICES=none` and usbmuxd out of reach, so only the virtual phone the test started joins it. ```sh # A Simulator that exists and is shut down; the test shuts it down again: PINKY_E2E_SIMULATOR= PINKY_E2E_SIMULATOR_OS=26.5 \ go test -tags e2e -run TestVirtualSimulatorInTheApp -timeout 30m ./cmd/pinky # An AVD that exists and does not run; the test kills its emulator again: PINKY_E2E_AVD= go test -tags e2e -run TestVirtualEmulatorInTheApp -timeout 30m ./cmd/pinky # PINKY_SCREENSHOTS= keeps pictures of the phone's page and the task, and the app's log. # PINKY_E2E_WDA_ARCHIVE= seeds the WebDriverAgent source archive (see below). ``` - **The Simulator** (iPhone 17e, iOS 26.5, Xcode 27.2 27B52, Apple silicon, 2026-11-09): pass, several runs. The first start with nothing in the data folder (download, check, build) was online in 44 s; with the archive seeded, 25 to 27 s (build 17 s, WDA answering 6 s after the run started). Start again from the phone's page, with the helper built: online in 10 s, once 22 s after a short offline at 19.5 s while WDA came up. The mirror showed the Simulator's screen, Settings opened through MCP, the task's recording (`.mp4`) was written, and quitting the app ended the xcodebuild run and removed its folder. WDA's runner app (WebDriverAgentRunner) shows on the Simulator's home screen once it ran, as Xcode's test runners do; since 2026-11-11 as Pinky Helper with Pinky's icon. - **Little Snitch on this Mac** holds each new unsigned test binary's first connection to github.com until someone answers, so the download timed out there (setup "simulator_helper", tried again after 5 minutes as designed). `PINKY_E2E_WDA_ARCHIVE` puts the archive in the test's data folder first; Pinky still checks its tree hash and builds it. The signed app asks once, like any app. - **No Simulator.app.** Xcode 27 has none (see "iOS Simulators"), so the person sees a Simulator in Pinky's mirror only; the first runs logged `opening Simulator.app err="exit status 1"` until Pinky stopped trying. - **The emulator** runs against an adb server of the test's own (port 5097, `ADB_USB=0`, `ADB_MDNS=0`, `emulator_helpers_test.go`), so the owner's Android phone on the shared adb server is never asked about itself: Pinky's Android provider asked every device adb listed for its properties, even ones `PINKY_DEVICES` left out. Since it leaves those alone (android.md, `Provider.Limit`), the test's own server only keeps it apart from what runs on the shared one. It gives the emulator a crash folder of its own (`ANDROID_EMU_CRASH_REPORTING_DATABASE`), so a crash leaves no report that would make the next windowed emulator on this Mac wait for an answer, and fails at once with what Add phone shows when the emulator quits while it starts. - **The emulator results** (an AVD made for the test from the installed image, pixel_6 with Android 16, API 36 `google_apis` arm64-v8a, emulator 37.2.13, Apple silicon, 2026-11-09): pass, three runs. A cold boot (no snapshot) was online 20 s after Start, a quick boot from the AVD's snapshot 3 s; Start again from the phone's page brought it back in 3 to 4 s as the same phone (`emulator-5554`). The phone was named as its AVD, the mirror showed the emulator's screen, Settings opened through MCP, the task's recording was written, and Shut down stopped the emulator. - Before Pinky passed `-no-audio`, every start on this Mac died about 30 s in: the emulator waited for the Bluetooth headset's microphone until its hang detector killed it (android.md, "Android emulators from Add phone", **No sound**). Add phone and the phone's page said so ("Couldn't start pinky-virtual-e2e: signal: segmentation fault: detected a hanging thread 'QEMU2 main loop'…"). - Before Pinky waited for `sys.boot_completed`, the cold boot was online at 15 s as "cmd: Can't find service: settings", and its page showed "No picture: … cannot read the display size" (android.md, **Booting**). - An Android 10 AVD passed every step but the recording: that emulator's video encoder fails (android.md), so the task had no video. The test wants Android 11 or later. - **Killed, not quit.** When the app was killed (a failed test's cleanup), its xcodebuild run outlived it in its own process group and kept the run's folder; a normal quit left neither. Each run's guard now ends it and removes the folder (see **Running it** above). ## GitHub issue #7: WeChat through WebDriverAgent (2026-11-08) A person drove WeChat on an iPhone 16 Pro (iOS 26.7.1, the WDA path, over Wi-Fi) from ChatGPT, on 2026.1108.14. Four reports; what came of them: 1. **微信 opened 微信读书.** Confirmed in the code: the app list has each bundle's own name, which is "WeChat" in every language, while WeRead's is "微信读书"; the gateway took the only name holding "微信". Now the list also has the name the home screen shows (`driver.App.Aliases`, read from SpringBoard's icon layout through `com.apple.springboardservices`, within 3 s, `homeScreenNames`; apps only in the App Library have none), and the gateway takes a name only by id, name, alias or a short table of well-known names (`internal/gateway/appnames.go`); a name that only part of an app's name holds finds nothing, and the result lists those apps as similar (`phone_launch`, the task's `apps`). Unit-tested; the icon layout read from the iPhone 13 mini over Wi-Fi gave 99 names in 1 s (that phone is in English: WeChat's icon says WeChat there). 2. **A first read gave WeChat with only the lock screen's charging line.** Likely the lock screen: the WDA path reads the lock from lockdown's PasswordProtected only and took a failure for unlocked. Now a failure falls back to WDA's `/wda/locked`, and a read whose tree shows nothing of the app asks WDA too and reads as locked, without elements. Unit-tested with a fake WDA; not reproduced on a phone. 3. **Taps of 20 s in WeChat.** Not confirmed. Each action reads the tree twice (before and after), and the read after also takes a screenshot for OCR; a big tree like WeChat's chat list takes WDA seconds, and over Wi-Fi the screenshot too. A read of 3 s or more now logs its parts (`iphone: slow screen read through WebDriverAgent`: source, nodes, lock, screenshot, OCR) so that a report with logs says which. 4. **Rows repeated, and rows under the navigation bar.** WDA lists a row that is one accessibility element and, again, the texts inside it, and counts rows scrolled under the bars as visible. Now a text its element already says is left out, and elements under the navigation bar, a tab bar, a toolbar, the status bar or the keyboard go as on the iOS 27 path (`unoccluded`, a covered element keeps a part that shows when it is big enough). Unit-tested on a made-up tree; WeChat's own tree not seen. ### Apps named in the phone's language (2026-11-12) The issue's last comment: agents given Chinese tasks still saw `WeChat · 微信`, the bundle's English name first. Now every iPhone path names an app as its home screen does and keeps the bundle's name as the alias (`appNames` in `apps.go`): `phone_apps` lists `微信 · WeChat (com.tencent.xin)`. - **A phone (iOS 27's Device Hub path and Pinky on the iPhone (WebDriverAgent) path).** Both list apps through `browseApps`: the installation proxy for the bundles (`CFBundleDisplayName`, else `CFBundleName`, never localized) and SpringBoard's icon layout for the home screen's names (`homeScreenNames`). The home screen's name is the app's name; the bundle's is an alias. The app in front (found by its bundle's path on the Device Hub path) and a URL scheme's apps carry the same name. On the WDA path the app in front takes the name from the app list once it has been read (`cachedAppName`), else WDA's. `byName`, which finds the app in front by its application element's label and a banner's app, matches aliases too, so the localized label finds it. Apps only in the App Library are not in the icon layout and keep the bundle's name: the installation proxy cannot read an app's InfoPlist.strings, and AFC cannot read app bundles. - **A Simulator.** `simctl listapps` gives the bundle's names and its path on the Mac, so `localizedAppName` (`simnames.go`) reads the bundle's localization for the Simulator's languages (AppleLanguages in the device's `.GlobalPreferences.plist`): the first language the app has, as iOS chooses (`zh-Hans-SG` takes `zh_CN.lproj` or `zh-Hans.lproj`), from `.lproj/InfoPlist.strings` (binary, XML or text, UTF-8 or UTF-16) or `InfoPlist.loctable`. On the booted iOS 26.5 Simulator in Chinese it named 24 of 37 apps (文件, 地图, 信息…); the rest have no Chinese in that runtime (Fitness has 18 localizations, none Chinese). - **The gateway.** `phone_apps`' `query` matches aliases too. Agent scopes typed by name in Settings › Agents still match the app's name only (`appListed`, the policy's `ScopeApps`): a scope that says "WeChat" no longer covers WeChat on a Chinese iPhone, one that says 微信 now does. Ids always match. ## Slow reads through WebDriverAgent over Wi-Fi (2026-11-10) A read-only task on an iPhone 13 mini (iOS 27.0.1, the WDA path over Wi-Fi, Claude Desktop, Pinky 2026.1110.27) took 20 to 36 s per step, and calls that ran past a minute made Claude Desktop drop the session. The slow-read log said where: the screenshot for OCR took 3 to 25 s and the page source 1 to 23 s, both at once over the same tunnel. - **The screenshot.** WDA's `GET /screenshot` is the screen at full size as a lossless PNG in base64 whatever `screenshotQuality` says (`FBScreenshot` encodes it to PNG again): 4.6 MB for a photo on the iPhone 17 Simulator, 0.6 MB for a mostly white web page, at about 1 MB/s through the tunnel. Over Wi-Fi, OCR now reads one frame of WDA's MJPEG server instead (`wda_ocr.go`): the screen at half size, JPEG at quality 50, 20 to 300 KB. A new connection's first frame is taken after it connects (the server keeps none). OCR found the same text in it: a Wikipedia page in Safari gave 35 lines from the frame and 39 from the screenshot, the difference a few words of the page under Safari's tip, mangled in both; a read with OCR had 27 OCR elements with the frame and 26 with the screenshot. A frame that does not come within 2.5 s makes OCR use screenshots for a minute. Over a cable the screenshot stays. Unit-tested with a fake WDA; on the Simulator through `OpenWDAURL` with the connection set to Wi-Fi. Not run on a phone. - **The page source.** WDA pretty-prints its JSON: Safari's tree of a Wikipedia page, 667 nodes, was 1.76 MB, about 150 KB of it values and the rest indentation; `?format=xml` gave the same tree in 205 KB and answered sooner (2.1 s against 2.6 s on the Simulator). XML lacks `isFocused` and `rawIdentifier` (its `name` is the identifier, else the label), which the elements use, so the source stays JSON for now. `excluded_attributes` saves little: the time is XCTest's snapshot, not the attributes, and `isVisible` and `isAccessible`, the ones that cost, are needed. ## What does not work yet - Elements scrolled out of view are not listed (the phone lists what is on screen); agents scroll and observe again. - Other processes' elements without a host view (home screen widgets) are left out, since their frames are relative to the widget. The exception is WebKit's web content process (`com.apple.WebKit.WebContent`, found through the process list): Safari lists a page's elements with no host view and with frames in screen points, so they are placed as they are, and a password field of a page is told from a plain one by the trait `1 << 24` (`traitSecureText`). Checked by the contributor (pull request #4) on an iPhone with iOS 27 in Safari (a Wikipedia sign-in page and a `data:` page); the device test that did it was left out, since it opens Safari, which holds the owner's tabs and history (docs/WORKING.md). Not tried: a web view inside another app, and whether native secure fields carry the same trait. - Elements behind a sheet can be listed with the sheet's (Settings' search). - The pushed-tree rules trust timing: two inputs less than about 0.3 s apart (faster than the pacing rule allows) could still get the first one's push. - While Pinky is connected the phone's accessibility runtime is on and it pushes a tree after every change: up to 1.7 MB (home screen) over the tunnel each time. - Orientation (see "Orientation"): remoteAXService's frames on a turned iPad were not seen (the service did not answer that night); the helper's upside-down handling and its Back in split views ran on the iPad Simulator only. - A pasteboard the phone does not let anyone read (a Universal Clipboard item, a password, transient content) cannot be saved; `Type` empties it after pasting. When reading every representation stalls, only each item's first one is put back. Restored contents stay on the phone (no Universal Clipboard) and carry Pinky's source metadata. - Typing by keys (only when the pasteboard does not do): ASCII on a U.S. hardware layout, the first letter capitalized by Latin keyboards where the field capitalizes, 4–5 s per call without a keyboard switch and 7–11 s with one (each probe walks the tree, slowly while the phone pushes one; a switch adds the space check, `spendSwitchSpace`). - Wi-Fi: screenshots take about as many seconds as their PNG has megabytes (1 to 15 MB, only PNG), so OCR, approval cards, history and agents' screenshots take the screen from the live video instead (see "OCR from the live video"). A sleeping phone is reachable only now and then, so Pinky waits until it is unlocked. After several sessions were cut off, the test iPad's services answered slowly over Wi-Fi for at least ten minutes (see "Wi-Fi"). - `Unlock` unlocked the iPhone 13 mini on 2026-11-08 (ios-older's test, the owner typing the passcode). Swiping again when the first swipe brings no passcode pad (a lock screen full of notifications, field test item 17) is unit-tested only: `TestDeviceLockScreenPad`, which types nothing, found the pad in 1.2 s on a lock screen already showing it. Lock detection for phones without a passcode is not tested. - SpringBoard's alerts and banners as events, the app beneath Face ID's prompt and `not_in_screenshot` were checked on quiet screens only (no false events, names or marks); a real banner, a real Face ID prompt and a real capture-proof layer have not been seen by this code. - Banking apps that refuse to run while the screen is shared cannot be used by agents: the stream must run for input. They work for the person 30 s after the last view or input. - iOS 17.4 to 26 is built but not tried on a phone (see "iPhones below iOS 27"); 17.3 and earlier are not supported. Windows and Linux cross-compile, untested. - Instruments (`com.apple.instruments.dtservicehub`) is not in Xcode 27.2's developer disk image, and iOS 27 lists it without one, so its presence does not show that an image is mounted. The check below iOS 27 (`probeLegacyServices`) therefore also wants one of the services the image itself publishes (display, HID, screen capture, pasteboard or device info). Not checked on a phone below iOS 27. - The image downloaded from the mirror, the Developer Mode switch and pairing on Windows are built but not yet tried on a phone (see "Preparing a phone without Xcode"). A computer without Xcode's image (a Mac that never had Xcode, Windows, Linux) needs a copy from a Mac with Xcode, or the mirror's beta image. - `WaitSettled` uses frame sizes (and accessibility notifications when they come); animated widgets keep it from ever settling on such home screens (it times out), and after motion the encoder's slowly falling bitrate can keep it waiting a little after the picture stopped. ## What the owner should try Done on 2026-11-08 for unlocking, both ways: the owner locked the iPhone 13 mini, ran `iosctl -connection coredevice unlock` and `iosctl -connection wda unlock` and typed the passcode in their own terminal, and the phone unlocked each time (steps 1 and 4; `unlock` woke the screen itself). Steps 2, 3 and 5 were not run; no wrong passcode was tried. Through the helper, the first try typed nothing and failed: `/wda/homescreen` is refused while the phone is locked, so Home is now `/wda/pressButton`. For another phone: with the iPhone connected by USB and nobody else using it: 1. Lock it with the side button and leave it until the screen is dark. 2. `scripts/devlock.sh /tmp/pinky-iosctl -udid screen` should print `screen on: false`, and `locked` should print `locked: true`. 3. `wake` should light the screen on the lock screen (Home, as a keyboard's Home key would), and `screen` then print `true`. 4. `unlock`, then type the passcode and Enter: it should print `unlocked` within a few seconds, the phone showing its home screen or last app. Watch the phone: the passcode pad should come up after the swipe and fill one key at a time. 5. Once more with a wrong passcode: it should fail with "device locked" after one attempt (the phone counts one failed attempt) and type nothing more. If the pad does not come up after the swipe (the phone may want Home instead with a keyboard attached), `unlock` stops with "the passcode pad did not appear" and has typed nothing. If the phone accepts the passcode but `unlock` still fails, lockdown's lock state is slow to change and `unlockWait` (4 s) needs raising. ## Security notes - WebDriverAgent serves its HTTP and MJPEG ports on every interface of the phone by default, Wi-Fi included: anyone on the same network could drive the phone through it, past Pinky's approvals. When Pinky starts the helper it sets `USE_IP`, so both listen only where Pinky reaches them: on a cable the phone's loopback, where usbmuxd's connections arrive; on Wi-Fi the phone's address inside Pinky's tunnel, which Pinky then dials through it, since usbmuxd connects to the phone's Wi-Fi address there and a runner on the loopback never answers (iPhone 12 Pro Max, iOS 18.7.7, 2026-11-09). A WDA that is already running when Pinky opens the phone (started from Xcode or Appium) is used as it is, with whatever binding it has: stop it after installing (step 3 of "Installing Pinky on the iPhone (WebDriverAgent)") so Pinky starts its own. - While a session runs, the tunnel's forwarder listens on 127.0.0.1 and lets any local process reach the phone's developer services through it (go-ios's own userspace tunnel does the same). It adds nothing an agent with shell access could not already do through usbmuxd, which the threat model covers, but the gateway's isolation advice applies to it too. - Media streams make the phone show its screen-sharing indicator and block its camera and microphone; `Close` stops the stream, and the phone also stops it by itself 10 s after receiver reports stop (a crashed process). - The virtual keyboard and any held finger are released on `Close`. A killed process cannot release them; the phone should drop them when the HID connection closes with the tunnel, but that is not tested. - `Type` holds the person's pasteboard in memory for one call, never logs it, and its writes stay on the phone (`localOnly`). A process killed between the SET and the restore leaves the typed text on the pasteboard, and the saved contents are lost with the process. - `Unlock` keeps the passcode only for the call and never logs it; it types it only while lockdown says locked and the passcode pad is on screen, so a Face ID unlock on the way cannot send it to an app. `iosctl unlock` reads it from standard input, not the command line. ## Notes for other sessions - `go.mod` gains go-ios (main), gVisor and purego; expect a merge in `go.mod`/`go.sum`. - No shared contract changed by this driver; it follows 5e83552 (Toucher points in video pixels). - For the gateway and the client: `Stream` packets are HEVC Annex B; a `Config` packet (VPS, SPS, PPS) comes before every keyframe a subscriber receives; PTS counts from the stream's start on a 24 kHz clock. A new subscriber waits up to about 100 ms for a requested keyframe. No frames come while the screen is still (from about 7 s after the last change), so the mirror keeps showing the last frame and recordings have gaps. Element bounds and taps are screenshot pixels (1125x2436 here); raw touches are video pixels (1136x2464), which on iPhone are the same numbers inside the screen. The video's right 11 columns and bottom 28 rows are black padding; a mirror can crop the picture to the `Screen`'s width and height. - `Open` connects `remoteAXService` in the background, which turns on the phone's accessibility runtime for the session (no visible change on the phone), the pasteboard service and the inspector session (for apps coming to the foreground), and reads the app catalog. - The device's video stream stops 30 s after the last `Stream` view closed and the last input, and starts again for the next view or input (see Decisions); a view open all along (the mirror, a recording) keeps it running. - iPhone screens can now carry events (`dialog`, `notification`, `toast`, from SpringBoard; see Decisions) and two states of the driver's own: `adjustable` (a vertical swipe changes it: picker wheels, sliders) and `not_in_screenshot` (the screenshot, and so the mirror, does not show it). The provider caches its inspections (see Decisions). - `Type` takes any text and leaves the pasteboard as it was (see Decisions for the cases it cannot). `Waker`: `ScreenOn` is quick (one CoreDevice request); `Wake` presses Home only on a dark screen. `Unlocker` follows the contract: one attempt, `ErrLocked` when the phone stays locked (or the passcode pad never showed, with nothing typed), `ErrUnsupported` for a passcode the keyboard cannot type. `Open` honours `Info.Connection == "wifi"`. - `WaitSettled` returns no sooner than 500 ms after the call, once the last 250 ms brought frames of one size, or no frames while the stream is alive; a fling in a list takes 2–3 s to stop, and animated widgets never stop. - During testing `iosctl devices` probed every phone usbmuxd listed, which included the owner's iPad (iPad13,4, over Wi-Fi): one tunnel and one RSD handshake, nothing else. - The phone was left on its home screen; Settings was only browsed (back navigation, scrolling, searches for "wallpaper" and "clock" that were cancelled, and closed again through the close button). Once a test's cleanup tap landed on a search result and opened Settings > Apps > Clock; nothing there was touched (its Mobile Data switch stayed on), and the test no longer taps after typing. Calculator was left cleared. - Round three: the phone's pasteboard held a Universal Clipboard item, which the phone does not let anyone read, so it could not be kept; the typing tests set their own pasteboard contents and leave it empty. The keyboard was switched to English (UK) and back to Pinyin by the keys test; it was left on Pinyin. The phone stayed unlocked and awake throughout; nothing was locked, no passcode typed, no setting changed. - The on-screen keyboard check from OCR needs single letters spread across the screen: a list's alphabetical section index (Settings > Apps) fooled the first version. - `driver.Packet.Turn` is new (additive, shared contract): quarter turns clockwise that put a packet's picture upright; the iOS 27 driver sets it, everything else leaves 0. `video.Frame.Turn` carries it to the mirror, which turns the picture and maps touches back (`ui/mirror.go`), and recordings keep it (`internal/recording/turn.go`): the first picture's turn in the track header's matrix (FFmpeg reads it as rotation -90 for a quarter turn), later turns in a `uuid` box of Pinky's before their fragment; the recording player turns the picture as the mirror does. - `driver.ConnectionChooser` is new and optional (additive): the iOS provider implements it, and the gateway's `Connections` and `SetConnection` serve Settings › Phones. The choice lives in the history store's `device_connections` table (schema 3). - With the helper, Pinky sees the phone's passcode request for automation in the runner's log and lists the phone as needing setup until it is answered (see "Choosing the connection"); an agent gets that detail when it opens the phone meanwhile. ## Typing on a non-U.S. hardware layout (GitHub issue #29, 2026-11-09) Reported by theniceboy: iPhone 17, iOS 27.0.1, USB, Device Hub, the hardware keyboard layout Colemak. `phone_type` "Display" into Settings' search entered `Sur;iaj` and settled. `iosctl type` showed why: the pasteboard read timed out twice (`pbQuick` 3 s, and once more), so `Type` typed by keys, which went out as U.S. key codes; the same read answers in well under 300 ms at other times. The single probe letter "a" sits where it is on Colemak and Dvorak, so the keys path took the keyboard for a U.S. one, and nothing read the field back. - `savePasteboard` no longer repeats a read that timed out (`context.DeadlineExceeded`): the second wait only doubled the time before typing by keys. A read that failed at once (the phone closed the idle connection) is still tried again once. - The layout probe and the read-back above. - Also reported on the keys path: the first letter capitalized, which is the hardware keyboard's Auto-Capitalization (above; Settings' search capitalizes, Clock's city search does not), and the first space of "one two three four" dropped ("Onetwo three four"). - **The dropped space comes from switching keyboards.** Checked on the iPhone 13 mini (iOS 27.0.1, Pinyin active, English (UK) with the U.S. layout as the other keyboard), typing by keys (`iosctl type -keys`) into Clock › World Clock › Add's city search, reading only that field (2026-11-09, the owner unlocked it for the test): every call that switched with Control-Space lost one Space, the first of the text wherever it was ("ijkl mn", "tehqiuck brwon fox", " ab cd" appended as "ab cd"), the same with a pause of 150 ms, 300 ms or 1 s before and after each space, so no race with corrections; with English already active (no switch) nothing was lost, twice. Spaces pressed right after the switch went nowhere, a letter after them arrived. Not every switch swallows one (the first call of the session, and one into an empty field, lost none). So after a switch, before clearing the field and typing, `spendSwitchSpace` types "x", Space, "y", reads whether the field got "x y" or "xy" (a trailing Space alone would not tell: some fields drop it from their value), deletes them, and tries again while the Space is swallowed (up to three times). With it, five calls in a row typed their text exactly, and the Space was swallowed on the first try and went in on the second each time but once. The reporter's keyboards are not known, but the dropped space says Type switched there too. - Other things the phone test turned up: right after `Open` (iosctl's case) remoteAXService is not connected yet, and the keys path skipped its probes and typed blind; one such call put Pinyin's composition of the letters into the field, which the read-back reported. The keys path now waits up to 8 s for the service (`raxDialWait`). And a replaced value that reads as before (typing the same wrong text again, as an agent trying once more would) is an error, not the "could not check" note, which is kept for appended text that did not appear and for an empty value. - Tests: `TestSavePasteboardTimeoutNotRetried`, `TestArrived`, `TestOtherLatin`, `TestTypedAs` (Colemak's `Sur;iaj` and "Onetwo three" are errors), `TestTypedAsWithoutPrior`, `TestSpaceProbe`, `TestHiddenText`, `TestLayoutError`; the gateway's `TestTypeNote`. ## Phones with several displays, and no video (GitHub issue #32, 2026-11-09) Reported on an iPhone Duo (iPhone19,4, the foldable) with iOS 27.3, from an Apple Silicon Mac with macOS 27.2, Pinky 2026.1109.13 and .14: the image mounted as a cryptex (the mirror's 27A5228h), then the phone stayed at Connecting with `open device: no video from the device: context deadline exceeded`, and `phone_connect` said `device_offline`. The stream start was answered; no key frame came within the wait. Pinky asked for display 1, always, as Xcode and go-ios do for a phone with one screen. **What the display info says.** CoreDevice's own model of a display (`CoreDevice.Display` in Xcode 27.2's `CoreDeviceUtilities`, whose Codable keys are its property names) has `displayId` (UInt32), `uniqueId`, `name`, `bounds`, `frame`, `pointScale`, `logicalScale`, `nativeSize`, `physicalSize`, `currentOrientation`, `nativeOrientation`, `primary`, `active` (optional), `backlightState`, `external`, `type` (`integrated`, `external`, `wireless`, `virtual`, `dfr`, `unknown`) and the display's modes. The replies seen so far (iPhone 13 mini, iPad Pro, iPhone17,1, iOS 27.0.1 to 27.2) list one display, without `displayId`; no reply from a foldable has been seen, and nothing public says what iOS 27 lists for one. **What the reporter saw next** (a later comment, the same phone, still on a build before this work): folded with the outer display on, the live view, elements and taps worked, and the checkup passed but for Wi-Fi; unfolded with the inner display on, every open failed after about 8 s, sometimes with `video receive: EOF`; the phone showed its screen-sharing prompt each time. So display 1 is the outer display, and the inner one has another id, if a stream can ask for it this way at all. A Duo's reply may well name no `displayId`, like every reply seen so far, and may list one display only. **What Pinky does now** (`display_info.go`, `device.go`): - It reads every display of the reply. The display it shows is the best of the phone's own displays (`own`: not `external`, of type integrated or none said): a lit one (`active`, or a lit backlight when `active` is not said), the primary one among several lit ones, the primary one when none is lit. A display of another type (an external screen, the wireless display the phone makes for a video stream, see "Video") never counts, so an iPhone that listed that one beside its screen still has one display. While no display of a foldable is lit (asleep), Pinky stays on the one it showed. - The stream asks for that display by id (`VideoStreamForDisplayID`). A phone with one display asks for 1 first, as before, then for its display's own id when that is another. A phone with several displays of its own, or a model with two screens (`twoScreens`: iPhone19,4) whatever its reply lists, asks for the shown one, then its other displays, then 1; when the ids do not tell its displays apart (a display without `displayId`, or one display listed of two) it asks for 2 and 3 too. Up to three, each with 8 s for a key frame (`readyWait`); a start the phone refuses (a CoreDevice error other than "in use") or a stream it ends before the first picture (`video receive: EOF`) also moves on to the next. Which id brought each display's picture is remembered for that display's next start (`good`, by the display's key), so after the first unfold the inner display streams at once. - It follows a fold. The display info is read before gestures, with every screen read and every second while a view is open (`orientLoop`); when the display to show is no longer the one the stream was started for, the stream starts again on it (`displayMoved`, then `live`), and the screen's size, orientation, touch mapping and element coordinates go by the new display. A display is told from the other by its key: its `uniqueId`, else its id and size, the shorter side first so that turning the phone is no fold; so a reply without ids, or one that lists only the display in use, still tells the inner display from the outer. A display that says it is off, or a phone whose backlight is off, is not moved to. Which listed display a stream shows (`streamShows`) goes first by the picture's shape (`pictureOf`: the ratio of its sides within 3% of one display's only; the inner and outer displays differ far more), since the id a stream asks for may name no listed display, or another one than the reply's ids; without a telling shape, by the id asked for, but only when each display has an id of its own (`idsKnown`). Sizes and touches go by the display shown, also when the stream fell back to another one than Pinky meant, and when the phone moves to the display the stream already shows, nothing restarts. - When no display brings a picture, the error stays short and names the displays asked for (`no video from the device (display 1): …`, `(displays 1, 2, 3)`); a start the phone refuses says its display too (`start display stream (display 2): …`), and the log gets one `iphone: no video from the phone` line per display: the display list, the answer's payload type, size and frame rate and its whole `streamConfig`, and what came over the wire (`wireSummary`): RTP packets and bytes, RTCP packets, packets that were not RTP, when the last came, payload types, whether the payloads read as HEVC or H.264, access units and their NAL types, whether parameter sets came and with what size, key frames, key frames before parameter sets, frames passed over waiting for a key frame, unreadable SPS, loss, incomplete frames and key frame requests. A refused start logs `iphone: the phone did not stream a display`; a stream on a phone with several displays logs `iphone: streaming display`, and a fold `iphone: the phone switched displays`. - The phone's checkup says it too. The open fails, so the gateway has no device to ask: the provider keeps the last open's story (`noteOpen`) and answers through the new optional `driver.OpenChecker`; the gateway then lists the phone as connected and trusting this computer, with a failed screen item (fix: restart the phone) whose detail is that story. The client shows "The phone sends no live picture…" with the detail under it, so that Send feedback carries it. A phone that opened but whose stream later restarts without a picture gets the same item from its own checkup. - Screenshots. On a phone with several displays, a screenshot asks CoreDevice's capture service for the display the stream shows by its `uniqueId` (the key `displayUniqueID`, as go-ios's `ScreenshotOptions`), when the reply gives one: the instruments service takes the main display, which may be the one folded away. If such a screenshot fails, later ones take the main display as before (`shotUIDOff`). **Not known yet**, until someone with a foldable tries: whether the Duo lists both displays, with ids or a `uniqueId`; which id streams the inner display (2 or 3 is a guess; the log line `iphone: streaming display` says which worked, with the picture's size); whether `active` or the backlights follow the fold; whether a stream of display 1 brings nothing while folded (the likeliest reading of the report) or the stream fails for another reason, which the new log lines tell; which display HID touches reach, and whether remoteAXService's frames and the screenshot services follow the display shown. HID touches are mapped onto the display streamed; if they always land on one panel, taps on the other will be wrong. The report of 2026-11-12 (below) says they do: unfolded, nothing Pinky sent did anything. **Input on a Duo (a report from X, 2026-11-12, Pinky 2026.1112.x, iPhone19,4, iOS 27).** Folded, on the outer display, Home did nothing (the mirror's Home button, and so `phone_key home`); unfolded, the live picture followed the inner display and turned with the phone, but no tap, swipe or key did anything, and Pinky reported each as done; agents could not use the phone; a phone with one screen worked. The earlier report (#32) had taps working folded. What the code and Xcode 27.2's files say (`dtuhidd` from the developer disk image 27B52, and `CoreDevice.framework`, read with `strings`): - **Touches name no display.** A touch is a 58-byte report to service 257 on `universalhidservice`: five contact slots with a position from 0 to 65535 on each axis, nothing else (`touchReport`; go-ios's `hid` package is the same, `surfaceMainTouchscreen`). 257 is not the phone's own digitizer but a virtual touchscreen `dtuhidd` creates when it starts ("Adding touchscreenGesture service"); the phone decides which display its events reach. Pinky maps the point onto the display it streams (`panelPoint`), which is right only when the phone routes the virtual touchscreen to that display. The two reports read as: it reaches the outer display only, or the phone applies input only while a stream of display 1 runs (input needs a stream from this computer at all, see "One media stream" above; whether it must be display 1's is not known); unfolded, Pinky streams the inner display by another id. - **No other way to aim a touch is known.** `dtuhidd` also takes Indigo digitizer events (`com.apple.coredevice.hid.indigo`, feature `com.apple.coredevice.feature.remote.hid.digitizer`), which carry a target ("No touchscreen service found for digitizer event target %lu"); their keys are Swift literals that `strings` does not show, so the format is not known, and trying it blind on a person's phone is not worth it. `createService` takes properties for a virtual service, but which property would tie a touchscreen to a display is not known either. - **Home** is an Indigo button event (consumer page, Menu), which `dtuhidd` hands to "the main screen button service"; when it finds none it logs "No main screen button service found for button event" and drops the press. Pinky sent it without asking for a reply, so nothing could tell. - **Reading the screen follows the display shown**: remoteAXService's frames are scaled by the display Pinky shows (`observe_rax.go`), so the elements of the inner display are where the picture shows them. "Agents can't use it" follows from input: every action changed nothing and said it worked. What Pinky does now (`multiinput.go`, `hid.go`), on a phone with several displays only (`multiLocked`: its reply lists them, or the model has two screens); a phone with one display is unchanged: - After each touch (`gestureSteps`; the mirror's touches on lift) it watches the picture for 1.2 s (`watchEcho`, `movedAfter`: an accessibility change, frames of varying size after the input, or frames where none came for the second before) and keeps count per display (`echo`, by the display's key). When the last two inputs on the display in use changed nothing, the next touch is still sent but returns an error that says so (`inputUnseen`, `unseenError`; the gateway answers `uncertain` with it): on a display where no input ever showed, it tells the agent to fold the phone or hand off; on one where an earlier input did, that these may have missed and to observe before touching again. The count goes by display, so the outer display is not judged by the inner one. A change ends a run of still inputs, but no longer vouches for the ones after it (until issue #106, one change cleared the display for good, so taps that did nothing after one that did were all reported as done). - Home asks the phone to answer the button (`buttonListen`, 300 ms) and watches the picture for 1 s; when the phone answers with an error, or nothing changed, it swipes up from the bottom edge (the home gesture of every iPhone without a Home button; the same swipe as the one that brings up the passcode pad, from the last row, 250 ms, to 45 % of the height) and watches the picture for 1 s again. When the swipe changed the picture, Home succeeds and tells the agent to check that the home screen shows (`TakeInputNote`). When it changed nothing either, Home returns an error (`homeSwipeOutcome`), which the gateway answers as `uncertain`: until issue #106 it succeeded, and an agent was told Home settled while the folded Duo still showed Calendar. On the home screen already nothing changes either, so Home there is `uncertain` too, and the error says so. The swipe counts as a touch for `inputUnseen`; the verdict on the touches before it does not decide Home's answer. Waking the screen (`power.go`) still presses the button only. - The log: `iphone: streaming display` now names the display the picture shows (`shows`) and the touch service; `iphone: input on a phone with several displays` for each input, with `moved`, the display shown, the stream's display id, the touch service, the panel point and the counts; `iphone: Home on a phone with several displays` with whether the phone replied, its reply and whether it refused; `iphone: Home by a swipe up`; and once a session `iphone: the phone's HID services`, the answer to a `connectedServiceDescriptors` request on a connection of its own (a guess at the request's shape after "send" and "createService"; `dtuhidd` logs "connectedServiceDescriptors returning serviceIDs"). These lines are in the end of pinky.log and daemon.log that Help › Send Feedback… attaches. What to ask a Duo owner, on this version: (1) unfolded, unlocked, on the home screen, tap an app icon twice in the mirror and press Home in the mirror; (2) fold the phone, wait for the picture of the outer display, tap an app icon and press Home; (3) right away, Help › Send Feedback… with "Attach diagnostics", and paste all of it. The lines above say for each display which id streamed it, whether each touch and Home changed the picture, whether the phone answered the Home button and what, and what HID services it lists; and whether the home swipe worked where the button did not. **Issue #106 (2026-11-12, Pinky 2026.1112.16 on Windows, iPhone Duo, iOS 27.2, USB).** Display 1 (1398×2034) is the outer display, display 2 (2007×2853) the inner one. Unfolded, on the home screen: two taps on Calendar in the Dock, both `settled`, Calendar did not open; Home replied with an empty reply, did not change the picture, and the swipe up changed nothing (the home screen showed already). Folded (the phone dropped and came back over USB; the log warned `no screenshot of the display in use` and that the HID connection closed): a tap on Calendar opened it; Home again did nothing, the swipe up left Calendar in front, and the agent was told `settled · no change` with "Check that the home screen shows". Both reports of success are fixed above. Unfolded, the accessibility tree held the Dock only and the icon grid came from OCR alone; nothing in the report says why. pinky.log does not log screen reads (element counts, whether the walk came back to its start, its trace are `ObserveStats`, shown by `iosctl` only), so the log of that time would say which display id streamed the inner display and at what size (`iphone: streaming display`), whether remoteAXService connected (`iphone: remoteAXService connected`) and what each input did, but not what the tree held. Not changed. Tests: `TestMovedAfter`, `TestPictureMovedSince`, `TestInputUnseen` (two still touches on the inner display report it, the outer display starts afresh, a change ends a run, one still touch after a change is not reported, two are, a phone with one display never), `TestHomeSwipeOutcome` (a swipe that changed nothing is an error the gateway takes as uncertain, one that changed the picture a note, a failed swipe its error), `TestUnseenErrorIsTold`, `TestHIDReplyError` (made-up replies), `TestHoldAndListen`. Not tried on a phone: none of the owner's phones has two displays. **The image.** The mirror's newest image is still the one Pinky pins: Xcode 27.0 beta 4 (27A5228h, cryptex version 27.1.5228.8, mirror commit `e2aaeac`, 2026-10-31); nothing newer is published there. Apple's Xcode 27.2 (27B52, cryptex version 27.2.52.0) is newer and the image Pinky prefers when Xcode is installed. Whether iOS 27.3 wants a newer image than either is not known: the report's mount worked and the developer services answered (the stream start was answered), so the image is not the likeliest cause. Pinning a newer mirror image takes a new commit there, then `mirrorCommit`, `mirrorBuild`, `mirrorXcode` and every file's size and SHA-256 in `mirrorFiles` (`mirror.go`), and the owner's yes. The iPhone Duo is newer than 27B52's personalized identities, which is no matter on iOS 27: the cryptex way is the only one used there. **Models.** From Xcode 27.2's `device_traits.db`: iPhone18,5 (iPhone 17e), iPhone19,3 and 19,7 (iPhone 18 Pro Max), iPhone19,4 (iPhone Duo), and the iPads that were missing (iPad A16, iPad mini A17 Pro and A20 Pro, iPad Air M2 to M4, iPad Pro M4 and M5). Tests: `TestParseDisplayInfoFoldable`, `TestParseDisplayInfoBacklightPerDisplay`, `TestParseDisplaysActiveWins`, `TestParseDisplayInfoKeepsWhileDark`, `TestParseDisplayInfoTopLevelBacklight`, `TestChooseDisplayExternalLast`, `TestStreamDisplayNotOwn`, `TestStreamDisplays`, `TestParseDisplayInfoSingleUnchanged` (made-up replies with CoreDevice's keys); `TestNoteDisplaysFollowsFold`, `TestNoteDisplaysFallback`, `TestNoteDisplaysSingle`, `TestFoldWithoutIDs` (both displays listed without ids: display 1 ends its stream unfolded, 2 shows the inner one, the next unfold goes to 2 at once), `TestFoldOneListed` (one display listed, always id 1), `TestStreamShowsByPicture` (display 1 streams the outer display though the reply calls the inner one 1), `TestPictureOf`, `TestDisplayUniqueID`, `TestTwoScreens`; `TestStreamDisplayFallsBack`, `TestStreamDisplayNoVideoAnywhere`, `TestStreamDisplaySingle` (the message names display 1), `TestStreamDisplayStartFails`, `TestStreamDisplayLinkGone`, `TestStreamDisplayEnded` (fake streams); `TestNoVideo*`, `TestWireSummary`, `TestAnswerSummary` (RTP and RTCP fed by hand); `TestProviderOpenChecks`; the gateway's `TestCheckupOpenFailed`; the client's `TestCheckupOfIPhoneNoLivePicture`. Not tried on a phone: none of the owner's phones has two displays, and the single-display path is unchanged but for the display info's `type`. ## Pinching through WebDriverAgent (2026-11-09) `pinch.go`. `WDADevice.Pinch` sends W3C actions with two touch pointers (`POST /actions`), which XCTest performs as one multi-touch event: both pointers move to their start, go down, move together every 40 ms along `driver.Pinch.At` (at most 60 moves; a turning pinch needs a point every few degrees, as W3C moves are straight lines), and lift. The mirror's `Touch2` records both fingers while they are down (`pinchRecorder`, like `touchRecorder` for one finger) and plays the pinch back when they lift, with its timing, as `Touch` does for drags. `*Device` (Device Hub on iOS 27) and `AXDevice` (no helper, below iOS 27) do not pinch: they implement neither `driver.Pincher` nor `driver.MultiToucher`, so the gateway answers `unsupported` before it asks the person, and the mirror takes no pinch on them. Device Hub's HID reports have five contact slots, but Pinky drives one (`hid.go`); a second contact has not been tried on a phone. **Checked on the iOS 26.5 Simulator** (iPhone 17 Pro, a Simulator of its own, WDA built from Appium's copy with `IPHONEOS_DEPLOYMENT_TARGET=17.0` and run on ports 18900/19900; the Simulator was deleted afterwards), with small test apps built locally with `swiftc`, not committed: | App | What | Result | | --- | --- | --- | | UIKit: `UIPinchGestureRecognizer` and `UIRotationGestureRecognizer` together | `Pinch` 300→900 px | ×2.77 in 1.5 s | | | `Pinch` 900→300 px turning 45° | ×0.34 and 39° | | | `Pinch` turning -90° only | -95° | | | `Touch2` as the mirror sends it, 300→900 px in 21 touches 30 ms apart, played back on lift | ×2.77 | | SwiftUI: `MagnifyGesture` with `@GestureState` | `Pinch` 300→900, 900→300; `Touch2` 300→900 | ×2.99, ×0.33, ×2.99; `onEnded` every time | | SwiftUI: `SimultaneousGesture(MagnifyGesture(), RotateGesture())` | the same | ×2.99 while the fingers moved, then `onEnded` never came and later pinches changed nothing | The last row looks like how that test app combined the two gestures rather than the touches (the same touches work in the other two apps); worth a look on a real iPhone with an app that zooms and turns. ## A pairing record Apple's service cannot replace (Windows, 2026-11-10) **The report.** An iPhone 15 Plus (iPhone15,5) on iOS 27.2, plugged into Windows 10, never got past the trust step: `pinky devices` listed it as unauthorized with "Unlock the iPhone and tap Trust…", and every session failed with lockdown's `InvalidHostID` however often the person tapped Trust. go-ios reached the phone (`goios.ListDevices`, `goios.ReadPairRecord`), so the cable, Apple Mobile Device Service and the driver were fine. **What the files said.** `C:\ProgramData\Apple\Lockdown\.plist` held a pairing record from 2026-05-11 whose ACL was `SYSTEM:(I)(F), Administrators:(I)(F), Users:(I)(RX)`: a record written under SYSTEM, not by the device service. Apple Mobile Device Service runs as LOCAL SERVICE, and while pairing it wrote the new record with today's host id to `.plist.tmp` — with `BUILTIN\Users:(F)` and `LOCAL SERVICE:(I)(F)`, so it could write it — but it never replaced the `.plist`: that file gives LOCAL SERVICE no DELETE, and the folder grants Users `(CI)(WD,AD,WEA,WA)` without DELETE_CHILD. A pairing therefore succeeds all the way through (`SavePair` to usbmuxd answers success, as go-ios's `Pair` needs), the phone shows Trust, and the record this computer then sends the phone is the old one still. The phone answers `InvalidHostID`, and the old record is what Pinky kept reading: `goios.ReadPairRecord` answers through usbmuxd from the same folder. **The service account**, as anyone can check (`sc qc` needs no administrator): `sc qc "Apple Mobile Device Service"` answered `SERVICE_START_NAME : NT AUTHORITY\LocalService`, and `Get-CimInstance Win32_Service -Filter "Name='Apple Mobile Device Service'"` said the same. That is what makes a record written under SYSTEM one the service cannot replace: it may read the `.plist` and write a `.plist.tmp` beside it, but not delete the `.plist`. **The repair** (administrator; the folder's DACL lets no normal user do either): stop `Apple Mobile Device Service`, keep the `.plist` aside as `.plist.stale-`, move `.plist.tmp` to `.plist`, start the service. Without stopping the service, letting the service replace the file does as well: `icacls "$env:ProgramData\Apple\Lockdown\.plist" /grant "*S-1-5-19:(M)"` (LOCAL SERVICE by SID, so it reads the same in every language), then unplug and plug the phone back in. `$env:ProgramData` rather than `C:\ProgramData`, because ProgramData can be on another drive than C:; the command is written for PowerShell, where the person can paste it as is. Modify rather than full control: replacing the file needs write and delete, not the power to change its permissions and owner, and the record holds this computer's private key for the phone, so `(F)` would let every LOCAL SERVICE process take it. Given Modify, the service's next pairing replaced the record, and the new record's ACL grants `Users:(F)` as well as the inherited entries, so later pairings land. The phone connected at once afterwards, and Pinky prepared it as any other: the developer disk image it downloaded from the mirror mounted as a cryptex. **What Pinky does now** (`prepare.go`). `askTrust` takes the lockdown error. A record the phone refuses (`staleRecord`: `InvalidHostID` and the like) is asked for again — before, any record that `ReadPairRecord` could read was taken as good, and Pinky waited for a Trust that could never help. After a pairing the phone accepts, `askTrust` reads the record again: a pairing is a new host id every time, so a record that did not change is one Apple's device service could not write (`recordStuckError`). Then Pinky pairs no more while that record stays: another pairing would make the phone ask "Trust This Computer?" again every 4 s, and its pending-dialog error would flip the step back to "tap Trust". The stuck state remembers the host id the record was stuck with and the usbmuxd attach it was seen through, and leaves it when the record's host id changes (an administrator let the service replace it) or the phone is plugged in again (a new attach, a new `DeviceID`), so either repair starts pairing again by itself. `trustState` answers the stuck error with `driver.SetupTrustRecord`, a step of its own, and a Detail that names the command; `driver.PairRecordFile` and `driver.PairRecordFixCommand` build it from `ProgramData` (Windows only, where the record lives there). The client shows the Detail in a problem box under the trust step with "Copy the command", and the checkup words the step the same way, with Set up iPhone beside it, so `phone_handoff` and the person at the machine get the repair instead of advice that cannot work. Tests: `TestAskTrustAsksAgainWhenPhoneRefusesRecord` (a refused record is replaced by a pairing), `TestAskTrustKeepsAGoodRecord` (a phone that takes its record and refuses for another reason is not paired again), `TestAskTrustReportsARecordItCannotReplace` (the record does not change: `recordStuckError`, `driver.SetupTrustRecord`, the command in Detail, no second pairing past the 4 s window, and a pairing again once the record changes), `TestAskTrustLeavesStuckWhenReplugged` (a new attach pairs again), `TestTrustState`. Not reproduced in a test: the ACL itself — no pairing was made with a record written under SYSTEM on this machine after the repair. ## Dragging (`driver.Dragger`, `drag.go`, 2026-11-10) `phone_drag` (docs/notes/gateway.md, "Dragging") hands the driver a `driver.Drag`: press, hold, move through stops resting at each, let go. - **WebDriverAgent** (`WDADevice.Drag`, also Simulators through `simPhone`): one W3C touch pointer (`POST /actions`): move to the press point, down, a `pause` for the hold and for each rest, `pointerMove`s 40 ms apart along `Drag.Steps` (eased in and out), and up at the drop point, in WDA's points. XCTest plays the whole drag as one event, so its timing is the drag's own. - **Device Hub** (`Device.Drag`, iOS 27): `Drag.Steps` at 10 ms through `gestureSteps`, the loop `gesture` (swipes, long presses) now uses too: an HID report for each step that moves and none while the finger rests. Not tried on a phone: the owner's iPhone was not used. - **No helper** (`AXDevice`, below iOS 27): no input, so no `driver.Dragger`; the gateway answers `unsupported` before it asks. **Checked on an iOS 26.5 Simulator of its own** (WDA as for pinching; the other sessions' "Test iPhone" was left alone), through a temporary device test that ran `phone_drag` actions through the gateway (not committed): | What | Result | | --- | --- | | Hold 300 ms on an app icon, then move | no pickup: the page scrolled | | Hold 600, 800, 1000 ms | the icon lifts and the home screen goes into edit mode | | Hold 800, rest 1200 ms near the right edge, drop | two pages turned | | Hold 800, rest 700 ms near the right edge, drop | one page turned, the app on the next page; the same to the left brought it back | | Hold 800, rest 700 ms at the edge, then let go over a folder's icon on that page | the app went into the folder | The home screen stays in edit mode after a drag; Done (or the home key) ends it, which SKILL.md tells agents. The first run on a fresh Simulator lost WDA about 43 s in ("Test crashed with signal kill"; the cause was not found); a second run with logging went through. ## Settings › Developer › Enable UI Automation (2026-11-09) Device MCP's setup says taps and typing on iOS 27 need this switch on. Pinky does not ask for it, and nothing found on the Mac or in iOS 27.0.1's libraries reads it: - **What the switch is.** A `PSSwitchCell` in Settings' `DeveloperSettings.bundle` (`DTSettings.plist`, group `UI_AUTOMATION`): it writes `UIAutomationEnabled` in the preference domain `com.apple.UIAutomation`, has no default, and posts `com.apple.springboard.appIconVisibilityPreferencesChanged`. The bundle's code never mentions automation. Apple's names: Enable UI Automation, 启用UI自动化, UIオートメーションを有効. The same entry is in the iOS 18.5 and 26.5 simulators. - **Nothing reads it.** `UIAutomationEnabled` and `com.apple.UIAutomation` appear in no file but `DTSettings.plist` in: the iPhone 13 mini's iOS 27.0.1 libraries (Xcode's device support symbols, which hold SpringBoard.framework, SpringBoardFoundation, BackBoardServices, BackBoardHIDEventFoundation, AccessibilityUtilities and AXSpringBoardServerInstance), both volumes of Xcode's Developer Disk Image (27B52: `dtuhidd`, `testmanagerd`, XCTest), the iOS 18.5, 26.5 and 27.0 simulators, and Xcode.app and CoreDevice on the Mac. None of those SpringBoard and backboard frameworks mention UIAutomation at all, though SpringBoard.framework is among those listening for the notification. Not checked: the phone's executables outside the shared cache, as there is no IPSW here. - **Pinky's input on iOS 27 has no such gate.** Taps, swipes and keys go to `dtuhidd` (`com.apple.coredevice.hid.universalhidservice`), which by its manual page "receives HID events encoded using UniversalHID from the wire and posts them to the IOHIDEventSystem"; its binary has no check for a setting or for automation. - **What XCTest needs is Automation Mode, a different thing that iOS shows under the same name.** AutomationMode.framework keeps its own state (`/var/db/com.apple.dt.automationmode/automation-enabled`), and turning it on asks for the passcode in a prompt titled "Enable UI Automation" ("Authorize UI Automation Prompt" in the framework). `testmanagerd` and XCTAutomationSupport refuse automation sessions while it is off ("Rejecting automation session connection because Automation Mode is not enabled…"). That is the passcode request Pinky already handles on the helper's path (see "Choosing the connection"); the Device Hub path uses no XCTest and never brings it up. - **Pinky could not read the switch anyway.** CoreDevice has no feature that reads another domain's preferences, and lockdown has no domain for it. Not tried on a phone with the switch off: the owner decided the evidence above is enough. If a person ever reports taps refused on iOS 27, ask for the switch's state first; Pinky could only infer it from refused input. So Pinky has no setup step, Checkup line or agent error for the switch, and the README does not mention it. ## The on-screen keyboard while Pinky is attached (GitHub issue #65, 2026-11-10) Reported by oceandull: iPhone 12, iOS 27.2, USB, Device Hub, Pinky 2026.1110.5 to .15. In Xianyu (闲鱼) the search field showed no on-screen keyboard while Pinky was attached, also when the person tapped it by hand, and `phone_type` returned `settled` with nothing written. **Cause.** `ensureKeyboard` registered "Pinky virtual keyboard" on the touch connection at the first key and kept it until the session ended (`Close` only released its keys). While a hardware keyboard is attached, iOS hides its on-screen keyboard; an app whose search page waits for the keyboard to show before it takes focus or text (Xianyu's, as far as the report tells) then never gets there, and the person cannot type either. And `checkTyped` returned only a note, not an error, when no element had focus after typing or the field read empty or as before, so the gateway said `settled`. **What Pinky does now.** - The keyboard registers on a connection of its own to `com.apple.coredevice.hid.universalhidservice` (`hidSession.kb`), apart from touches, and is removed when the `Type`, `Key` or `Unlock` call that used it ends (`Device.releaseKeyboard`, `hidSession.dropKeyboard`): all keys up, then that connection closes, which is the only way known to remove a registered service (go-ios has no remove message either). The whole call keeps it, so the keyboard switch with Control-Space and `spendSwitchSpace` happen on one keyboard as before. After a removal a spare connection is dialed in the background (`dialSpare`), so the next registration saves the RemoteXPC handshake (about 0.5 s on the first request of a connection, as for the pasteboard). When a second connection cannot be dialed, the keyboard registers on the touch connection as before, and removing it closes that connection; `Device.live` dials a new one at the next input (not while a finger is down). - `checkTyped` tells the two nils of `focusedValue` apart (`focusedField`): when remoteAXService answers that nothing has focus, `Type` returns an error ("no field has focus…", `TypeStats.Checked` "unfocused"); a field that reads empty, or as before for appended text, is an error too ("unchanged"). The gateway maps both to `uncertain` with the reason, and Return is not pressed for `submit`. The note stays for a field that cannot be read (remoteAXService not connected, or the value read fails) and for one that hides its text. This matches Android's `notLanded`. - Before it says "unfocused", `checkTyped` reads focus again for up to `pasteConfirm` (1.5 s): with a third-party keyboard (Doubao Input, `com.bytedance.ios.doubaoime`) the app in front sometimes read as the keyboard's extension while its keyboard showed, and that has no focused field. - Tests: `TestJudgeTyped`. **On the phone** (iPhone 13 mini, iOS 27.2, over Wi-Fi, 2026-11-10, Settings' search and Calculator, a throwaway program next to `iosctl`): - A second universal HID connection registers the keyboard while the touch one is open, and its keys arrive: pasting ("display", "general", "wallpaper", "camera", all read back the same) and keys ("sound" arrived as "Sound", the Latin keyboard's capital as before). - The on-screen keyboard came back by itself about 1.5 s after each call. On the code before this change it never came back while Pinky was attached, not even when the field was tapped: the bug as reported. - Registering took 0.33–0.39 s with the spare connection ready (almost all of it the 300 ms wait) and 0.9–1.0 s without; Wi-Fi added outliers of 1.7–4 s. A paste call took 1.2–4.8 s over Wi-Fi, about as before (1.3–3.7 s). - In Calculator, which has no text field, `Type` returned the "no field has focus" error (`Checked` "unfocused") in about 5 s. - Not checked: Xianyu itself, and USB. ## An app using the camera or microphone (iOS 27, 2026-11-11) While an app on the iPhone uses the camera or microphone (a call or FaceTime, Camera, Voice Memos, a screen recording with the microphone, dictation, a Mac using it as Continuity Camera), the display service refuses the media stream with CoreDeviceError 9022, "The device microphone or camera is currently in use." (`errMediaInUse`; in the owner's log, from a new iPhone 15 Plus on iOS 27 on a cable). No other display helps (`otherDisplayMayStart`). Before, `Open` failed with that error, which was no `driver.SetupError`, so the gateway kept the phone as the provider listed it, online, and opened it again about every 2 s: a new tunnel and a warning in the log each time, and nothing that told the person why or what to do. **What Pinky does now** (`mediawait.go`): - `openCoreDevice` dials the tunnel (`dialDevice`) and starts the stream (`Device.start`). When the phone answers 9022 it keeps the session, its tunnel without a stream, as a `mediaWait`, and returns `driver.SetupError` with `driver.SetupMediaInUse` and `mediaInUseDetail`, so the gateway shows the step at once. - The wait starts the stream again every `mediaRetry` (2 s) on the same tunnel (`Device.retryMedia`, which is `restartMedia`): two connections inside the tunnel each time, no new tunnel and no new process. Meanwhile `List` (`mediaState`) reports the phone as `needs_setup` with `media_in_use`, so the gateway, which opens only phones listed online, does not open it again, and the status stays put. - Once a start succeeds, `List` says online, the gateway's next discovery opens the phone, and `openCoreDevice` takes the session over with its stream running (`Device.opened` starts what runs beside the stream). From the app letting go to the phone open takes up to one retry and one discovery, about 4 s. - The wait ends and closes its session when the phone is unplugged or attaches again (its `entryKey` changes), is no longer listed online over CoreDevice, is paused (`SetPaused`) or set to another connection (`SetConnection`), when the provider closes, when the stream fails for another reason (then `List` says online again and the next `Open` dials afresh, as without a wait), and when nobody opens it within `mediaTake` (30 s) of its stream starting. - The gateway shows it as "Camera or microphone in use" (`statusLocked`), and agents get `device_needs_setup` with `setup: media_in_use`, the detail and a hint to ask the person with `phone_handoff`, since they may be on a call. The client says it on the phone's page (its title, what to do, and that Pinky connects by itself, with a spinner instead of See what's wrong and Show the steps), in its sidebar row, tile and the menu bar flyout, in the checkup and in Add phone › iPhone (`setupWords`, the `mediaInUse*` messages). - A phone that refused one display for another reason before the 9022 does not also fail the checkup's screen item (`noteOpen`). - Tests: `TestProviderWaitsForCameraAndMicrophone`, `TestMediaWaitEnds` and `TestMediaInUseIsNoMissingPicture` (the provider, with a fake session), `TestMediaInUseStep` (the gateway: no flicker, one open, the checkup, an agent, online once free), `TestIPhoneMediaInUseStep` (the client in English, Chinese and Japanese). **Not done.** A phone that is already open, whose stream stopped after 30 s without a view or input (`idle.go`) and that an app then took the camera or microphone of, still fails the next view or input with the 9022 error, as before, without the step. Not tried on a phone yet: the error and its code come from the iPhone 15 Plus's log. ## WebDriverAgent builds with custom Xcode locations 2026-11-11, issue [#86](https://github.com/DIYgod/pinky/issues/86). Xcode's custom build locations can redirect products outside the directory passed with `-derivedDataPath`. Both the physical-device helper installer and the Simulator builder now pass `SYMROOT=
/Build/Products` and `OBJROOT=
/Build/Intermediates.noindex`, using their own temporary derived-data directory. The runner, `.xctestrun` lookup and Simulator build cache then use the same products directory for any user-selected build location. `TestHelperArgs` checks these arguments with a temporary path containing spaces. Local checks on macOS arm64 with Xcode 26.3 exercised `helperBuildArgs` through `-showBuildSettings` with the current custom settings, Unique locations, custom relative locations and custom absolute locations. All four resolved to the requested temporary directories. An unsigned `generic/platform=iOS` `build-for-testing` also produced both the runner app and `.xctestrun` there. The full Simulator build was attempted but stopped because this Xcode installation lacks `iPhoneSimulator.platform/Developer/usr/lib/lib_TestingInterop.dylib`. ## Resting an idle phone (GitHub issue #96, 2026-11-12) An iPhone 16 Pro on iOS 27.0.1 over Wi-Fi (Windows, remote agents over Tailscale) turned sluggish once an agent released it, and stayed so until the person paused Pinky's connection to it. The log showed the lease kept for the agent's next session (`ResumeGrace`, 2 min), then nothing. **What stayed up on an idle phone before.** Through CoreDevice (iOS 27): the tunnel; remoteAXService's session (while a client is connected the phone keeps its accessibility machinery up for it, and serves no other client); the inspector's session where it was used; the pasteboard, screenshot and display info services; and every 6 s a lockdown session for the lock state (`pollLocks`, a new TLS session over the network each time on Wi-Fi). The display stream and HID already stopped after 30 s without a view or input (`idle.go`). Through Pinky on the iPhone (WebDriverAgent, iOS 17.4–26, and on iOS 27 a task's helper connection): the XCTest runner, which keeps automation up, its tunnel and the WebDriver session. The inspector path (`AXDevice`) kept its tunnel and the inspector's session. Of these, the accessibility sessions and the runner are the likely cost to the phone; the tunnel itself carries nothing when unused. Not measured on a phone. **iOS 27 and the inspector path rest** (`rest.go`). Once a phone has had no agent working on it (`driver.WorkWatcher`: a lease that is not the person's, so a kept lease in its 2 min grace counts), no view of its picture (the mirror, a recording, the web page), no input and no call that needs its tunnel for `restIdle` (90 s), the device rests: it closes the remoteAXService and inspector sessions, the pasteboard, screenshot and display info services, app log followers and the tunnel. The device stays open in the gateway, which goes on showing the phone online. 90 s is longer than the stream's 30 s and the 30 s back from the helper, and lets an agent that releases and connects again within a minute find the phone awake. - **Waking.** The next call that needs the tunnel wakes the phone first: an agent's connect and first read, the mirror opening, input, the checkup, the app list, app logs. `Device.link` dials a new tunnel (`wakeUp`) and connects the display info, remoteAXService and pasteboard again in the background, as opening does; the stream starts as it does after `idle.go` stopped it. Concurrent callers wait for one wake. None of it needs the person: no passcode is asked. - **What a wake costs**, from the measurements above, not timed for this change: a tunnel takes about 1 s on a cable and 1–3 s over Wi-Fi, so the first call after a rest takes that much longer, plus remoteAXService's first walk (slow, as after opening). Never longer than opening the phone today. - **What does not wake it.** Lockdown reads (the lock state, battery, storage, locale) go through usbmuxd and work without the tunnel, so the gateway's lock polling and the tile's vitals never wake a resting phone. - **A wake that fails** (the phone left) reports `driver.ErrOffline`; the gateway drops the phone and opens it again the usual way. **The helper keeps its runner** (`rest_wda.go`). Starting the runner may make iOS ask for the passcode (the iPhone 13 mini asked after a restart, and again half an hour later; see "What ran where"), so a runner started again after every idle spell could put that prompt in front of the person on most agent connects, which is worse than the slowdown; Pause connection already ends the runner for a person who wants that. So an idle phone on the helper keeps its runner and tunnel, and after `restIdle` only ends its WebDriver session (the next request makes a new one on the same runner) and stops app log reads, on a runner Pinky started or an earlier Pinky left (never Xcode's, which may serve another tool, a Simulator's or one at a URL). **Unchanged.** Pause connection closes the device as before; a resting phone pauses and resumes the same way. Reconnecting over Wi-Fi (`wifi.go`) reopens the device, which starts awake. A task's helper connection and the helper fallback for rotate and pinch go through `WDADevice`, whose runner stays up; while an agent works nothing rests. **Lock polling over Wi-Fi.** The gateway now reads an idle phone's lock state every `idleLockEveryWiFi` (30 s) instead of 6 s when the phone is over Wi-Fi and nobody watches its picture (no stream hub for it); with a lease, or with the mirror open, it reads as before. This covers every phone over Wi-Fi, the helper's too. The All phones tile shows a lock on such a phone up to 30 s late; auto-lock (2026.1111.3) is unaffected, as the tile holds no stream, and agents' `device_locked` results read the lock on every tick while they hold the phone. Tests: `TestDeviceRestsAndWakes`, `TestDeviceWakeFails` (a fake tunnel and redial: no rest while working, watched or recently used; one dial for concurrent callers; a resting phone is not gone, a failed wake is), `TestWDADeviceIdleKeepsRunner` (a fake WebDriverAgent: the runner stays, the session ends once, the next request makes a new one), `TestWDADeviceTheirsNeverRests`, and the gateway's `TestIdleWiFiPhoneLockSlower`. **A phone must still confirm:** that resting removes the stutter the reporter saw on iOS 27; how long the first call after a rest takes over Wi-Fi; that the mirror opening on a resting phone shows a picture; and that a resting phone still locks by itself and pauses and resumes. Whether the helper's runner alone slows a phone below iOS 27 is not known; if it does, Pause connection is the way out.