# Pinky's iPhone app Started on 2026-11-11. Pinky's iPhone app, named **Pinky**, runs agent tasks on the iPhone itself, with no computer at run time, and is to replace Pinky Helper, the WebDriverAgent runner the desktop app installs on iOS 17.4–26. It grew out of the iOS prototype in [standalone-phone.md](standalone-phone.md#prototype-results-ios). The code is in [ios](../../ios/README.md), which also says how to build and run it. It is sideloaded only: it uses Apple's developer services, which App Review never allows. This file holds what the build found, the decisions, what is left, and what the owner has to clean up on the test phone. The app's screens were redesigned on 2026-11-13 to match the Android app's: [phone-apps-ux.md](phone-apps-ux.md) has the design, and how to look at them in the Simulator (`-pinkyDemo`). Since then the app's own words say Activity, not History, and it has no search. ## Milestones | | What | State | | --- | --- | --- | | M1 | The investigations below; the prototype renamed Pinky, with the real gateway (rules, approvals, history) on the phone; a task end to end on a real iPhone | Done on the iPhone 13 mini (iOS 27.0.1) with the fake model: a task ran end to end, with desktop Pinky's connection to the iPhone paused (runs 6 and 7 below). A real model was not tried. The helper probe is below too | | M2 | Onboarding (on-phone pairing on iOS 27), the app in three languages, desktop installing the app in place of Pinky Helper and the rename everywhere | Shipped in 2026.1112.16. On-phone pairing then worked on the iPhone 13 mini (iOS 27.0.1): [the run](#pairing-on-the-phone-run). The DDI mount from the phone and the rest of [the plan](#plan-for-m2-and-m3)'s owner items are open | | M3 | iOS 17.4–26 on Simulators, battery over long runs, polish | Not started: [the plan](#plan-for-m2-and-m3) | New here? Read [Pitfalls on a real phone](#pitfalls-on-a-real-phone) and [How M1 was signed](#how-m1-was-signed) before touching the test phone, and [docs/WORKING.md](../WORKING.md) for the rules about the real phones. ## What M1 built - **The gateway on the phone.** [onphone](../../ios/onphone) wraps `internal/standalone`'s core, the one Android's app uses: `gateway.New` with Pinky's rules and approvals, SQLite history, and the agent loop of `internal/pilot`, over one device, the iPhone itself through the loopback. Nothing in `internal/standalone` changed. - **The guard** sits between the gateway and the driver. It refuses input while Pinky's own app is in front (so an agent can never tap Allow or Stop), hides Pinky from the app list, refuses to launch it, and brings the task's app back after the person answers in Pinky. Pinky's app ids are its own, learned from the app, and any id under it (`.lite`, the extensions). For agents on a computer, `policy.IsPinkyScreen` blocks the app's screen as it blocks the Android app's, under the ids it knows: `cc.diygod.pinky` and any id under it, and `pinky.` alone or with more after a dot, the shape desktop Pinky gives what it builds with the person's team (`helperBundleID`). It leaves out XCTest runners (ids ending `.xctrunner`, Pinky Helper's among them), whose screens hold no approvals; `IsPinkyApp` still counts them, so agents cannot uninstall or stop them. It also takes no pictures of the screen: reads use the accessibility tree alone, without OCR, and screenshots are unsupported, since a screenshot with OCR does not fit in the extension's 50 MB (run 5 below). - **One JSON call** for the app's screens (`state`, `wait`, `start`, `decide`, `preset`, `stop`, `dismiss`, `ui`, `history`, `info`). `wait` is a long poll on the state's revision, so the app wakes only when something changed, not on a timer. - **The app**: the task and its last steps, approvals (Allow needs Face ID or the passcode) and handoffs, Stop, history with search, and settings for the model, its key and when Pinky asks first. The Live Activity shows the task, "Pinky waits for you" in orange while an approval waits, and Stop. - **The model**: bring your own key. Claude through the Anthropic API is the default; any OpenAI-compatible server with a base URL is the other choice. The key is kept per provider in the Keychain (`AfterFirstUnlockThisDeviceOnly`), never shown again, and stays in the core's memory only for the length of a task. Tests use `internal/standalone/cmd/fakemodel`. - **The bundle id is a build setting** (`PINKY_BUNDLE_ID`); every other name derives from it, so anyone can build Pinky under their own team. - **Two builds.** *Full*: Pinky's own packet tunnel, the core in the extension (paid developer account). *Lite* (`NO_TUNNEL`): no extension, LocalDevVPN does the loopback and the core runs in the app, which silent audio keeps alive during a task; a free personal team can sign it. ### Prototype follow-ups | Follow-up | Done? | | --- | --- | | The Live Activity that stays behind when a task ends while the app is suspended | **Fixed in two ways.** Silent audio keeps the app running while a task is busy and Pinky is not in front, so it sees the end and ends the activity (showing how it ended for 8 s). On every launch and foreground, an activity left over with no task running is ended. Stop on the activity ends it from the widget's process too | | Fewer CPU wakeups | **Done for the app**: the long-poll `wait` replaced the 1 s poll. The core's heartbeat is off unless asked for | | The display stream only around taps | **Done.** Opened over the tunnel (`OpenOverTunnel`), the driver starts the stream and its HID session with the first input or view (`live`), not at open, and stops them after 30 s without input or a viewer (`idle.go`) as before. Until 9765540 it started them at open, where they ran for 30 s at about 500 KB/s through the loopback with nobody watching | | `.shim.remote` services over the tunnel | Not done (M2) | | A fallback when `remoteAXService` fails | Not done (M2). On a computer the driver falls back to the inspector walk with OCR (`observe.go`). On the phone the inspector is out of reach (it goes through lockdown over usbmuxd; a `.shim.remote` twin was not tried), so a failed read says why instead. OCR itself builds for the phone (`internal/ocr`'s Vision binding is `darwin`, which GOOS=ios includes), but a screenshot with OCR does not fit in the extension's 50 MB (run 5 below), so the phone's reads take no pictures now | ## End-to-end run On the iPhone 13 mini (iOS 27.0.1), with the fake model on the Mac (`fakemodel -script calc`; its `-pinky`, the app the script tries to open, is Pinky by default) and the task "Work out 7 × 6 in Calculator" started from the debug launch arguments. The script opens Calculator, taps 7 × 6 =, checks that 42 shows, tries to open Pinky itself (the guard must refuse) and goes home. 1. **11 November: iOS killed the extension.** The app installed and launched; after the owner allowed the VPN configuration, the tunnel came up, the core in the extension pair-verified through the loopback and connected (remotepairingd's tunnel listener, on a port of its own each time), the app read its state by the long-poll `wait`, the debug task started once the phone was ready, and the Live Activity appeared ("Working · 0 steps"). About a second later iOS killed the extension: `memorystatus: killing_specific_process … [PinkyTunnel] (per-process-limit …) 51233KB`. The VPN dropped ("Plugin failed"), the fake model never got a request, and the app was left showing Turn on. Twice in a row. A memory kill writes no crash report; the kernel's line in the device's syslog is the only trace. The tunnel connection had brought in 9.3 MB in its 5.8 s: over the tunnel the driver listed apps through the installation proxy's shim with no `ReturnAttributes`, so the phone sent every key of every app, entitlements included, and Go decoded all of it into maps. 4b931cb asks only for the eight keys the app list reads (`shim.go`, `appAttributes`). 2. **12 November, 00:25, with 4b931cb: no kill, but the first read failed.** The extension stayed alive. The warm-up's `remoteAXService` dial got no answer within 5 s, the failure was remembered for 30 s, and the phone has no inspector to fall back to, so the task stalled at its first read. The display stream had started at open as well, about 500 KB/s through the same loopback with nobody watching. 9765540 starts the stream with the first input or view, waits 15 s for the first answer on the phone, asks again at every read there, and says why a read failed instead of trying the inspector. 3. **01:16, with 9765540: `remoteAXService` never answered.** No kill, no stream at open. The warm-up and the first read each waited their 15 s for an answer that never came, and the task ended at its first read: "Reading the screen failed: … remote accessibility: no answer within 15s". The fake model got no request. 4. **01:36, with desktop Pinky's connection to the iPhone paused: the task never started.** Desktop Pinky had the iPhone open during runs 1 to 3, and it opens a `remoteAXService` session as soon as it opens a phone (`device.go`, `run`) and keeps it. The prototype's reads on the phone, on the afternoon of 11 November, had all fallen while it had the iPhone paused (12:58 to 17:00 in its log). So the owner paused it (Pause connection) to see whether the phone answers only one remote accessibility client at a time. The run tested nothing, though: the harness relaunched Pinky with `devicectl device process launch --terminate-existing`, which ends every process of the app's bundle, the tunnel extension too. The new app found the VPN still connected and left it alone; 5 s later iOS gave up on the dead extension ("Plugin failed"), and the app, which connects only at launch, stayed off. From then on the harness ended only the app's process. (Run 3's extension had stayed up and connected for 20 minutes until then, the phone locked most of the time.) 5. **01:54, desktop Pinky's connection still paused: `remoteAXService` answered, and iOS killed the extension at the first read.** Pinky's app was ended by its process id this time, and the relaunched app started the tunnel again. The warm-up's `remoteAXService` dial connected in 467 ms, where in runs 2 and 3, with desktop Pinky connected, it never got an answer. That supports the guess that the phone serves one remote accessibility client at a time (not proven: one run each way). The task started, and 3 s later iOS killed the extension at 51,217 KB ("exceeded mem limit: ActiveHard 50 MB (fatal)"). The fake model got no request. The task's first read had asked for OCR, as every agent read does. On the loopback the driver takes no frame of the video (that is for Wi-Fi), so it took a screenshot: a PNG of 5 to 7 MB on this phone, which it decodes to 11 to 22 MB while Vision recognizes the same PNG with models of its own. Run 3 never got that far, because its reads had no answer to picture. The rest of the core is small. On a Mac with the demo phone, the same gateway, agent loop and history used 2.9 MB of Go heap when idle and 5.3 MB at most through a task's start (6.2 MB allocated in all, most of it decoding the model's JSON). So the guard on the phone now takes no pictures. Reads drop OCR and allow no picture, and screenshots are unsupported, which the gateway tells the agent ("This phone cannot take screenshots"). 6. **02:14, with the guard taking no pictures: the task ran to its end.** No kill this time. The extension lived through the whole task, 40 s and 10 steps: it opened Calculator, tapped 7, ×, 6 and =, each settled, tried to open Pinky itself, which the guard refused (`blocked_by_policy`), and went home. Between taps the script read the screen once more, since a tap's result lists only what changed and the next key was not among it. Calculator still held 11,111 from earlier use, so the display read 111,117×6 and then 666,702, and the script's check for "42" failed for that reason alone. The display's text came from the accessibility tree, with no OCR. One read was slow: 3 s, 1.8 s of it walking the tree. The memory heartbeat did not run. The tunnel had been started at 02:13:57 by an instance of the app launched without the heartbeat, and the run's own launch found the VPN on and kept that extension. 7. **02:22, the same build from a fresh tunnel: 42, and the memory measured.** This time the harness ended the extension by its process id as well as the app, so the launch started the tunnel again, with the heartbeat. The core connected in 296 ms (85 services) and `remoteAXService` in 330 ms. The task ran 35 s and 10 steps and ended done: Calculator showed 42 (run 6 had left a result on its display, and a digit after a result starts a new number), Pinky refused to open itself (`blocked_by_policy`), and the task went home. The extension's footprint was 12 MB once connected, 14 to 16 MB before the first read, 25 MB just after it, 20 to 22 MB through the taps, and 28 MB at most, at the read of the home screen at the end, where Go's heap peaked at 17 MB. That leaves about 22 MB of the 50. Those two reads were also the slow ones: 2.9 s and 2.2 s, most of it walking the tree (1.7 s and 2.0 s). - **Logging.** The core's and the app's own lines show in the device's syslog relay, as `PinkyTunnel[pid] : pinky time=… level=… msg=…` and `Pinky[pid] : pinky …`, next to other processes' lines about Pinky (runningboardd, neagent, the kernel's memory lines). The core logs the connection, each task's start and end (provider, model, steps, seconds, and the summary only when Pinky ended the task over an error), and failed or slow reads; never the task's words, the key or the screen. A debug build launched with `-pinkyHeartbeat ` has the core log its memory that often (the extension's footprint, Go's heap, goroutines), when that launch starts the tunnel. - **Desktop Pinky after these changes.** The driver's changes for the phone run on a computer too, where `shim.go` reaches each service through lockdown over usbmuxd. Checked on the iPhone 13 mini on 12 November. Locked: `TestDeviceShimDesktop` read the icon layout, the names on the home screen and the app list as go-ios's own clients read them (101 names, 109 apps), read the lock state, and got an answer from `remoteAXService`. Unlocked by the owner, at 12:06: `TestDeviceHomeAndObserve`, `TestDeviceApps` and `TestDeviceLocked` passed, in 23 s together. The home screen's read came from `remoteAXService` (17 elements, all placed) with 27 lines of OCR from a screenshot. It took 18 s, 15 s of it OCR: the first OCR in a process is that slow ([ios.md](ios.md) has 13.5 s). ## Pitfalls on a real phone What cost time in M1, with the runs above where it showed: - **50 MB for the extension.** iOS kills the packet tunnel extension past 50 MB and writes no crash report; the kernel's `memorystatus` line in the syslog is the only trace (runs 1 and 5). What went past it: the app list with every key of every app (run 1) and a screenshot with OCR (run 5). A whole task peaks at 28 MB now (run 7). Measure with `-pinkyHeartbeat 1` before the core takes on anything big. - **One remote accessibility client per phone.** While desktop Pinky had the iPhone open, `remoteAXService` never answered the core on the phone (runs 2 and 3); with it paused, it answered within half a second (runs 5 and 7). Before a task on the phone, choose Pause connection in the iPhone's menu in desktop Pinky; Resume connection afterwards. - **Relaunching the app.** `devicectl device process launch --terminate-existing` ends the tunnel extension too, and the new app, which connects only at launch, finds the VPN on and leaves it (run 4). End the app by its process id. To start the tunnel again, for the heartbeat say (it runs only when that launch starts the tunnel, run 6), end `PinkyTunnel` by its process id as well and launch about 8 s later. [ios/README.md](../../ios/README.md) has the commands. - **Calculator keeps its display** between runs (run 6), so a script that checks a result must not count on a clear display. - **The phone locks itself** after two minutes. Device tests that open a `Device` (`testDevice`) need it unlocked, and `TestMain` presses Home at the end once one was opened. Tests at the level of the link, such as `TestDeviceShimDesktop`, open no `Device`, send no input and run on a locked phone. A locked phone's Observe answers with the lock screen from lockdown, without walking the tree. - **First launches need the owner.** Each new build id asks for notifications, Local Network and adding the VPN configuration, which asks for the passcode. Only the owner answers them. - **Tests and the real phones.** Some go-ios tests act on whatever phone usbmuxd offers, the owner's iPad included: run `make check` and any other go-ios tests with `USBMUXD_SOCKET_ADDRESS=127.0.0.1:9`. Deliberate device tests go through `scripts/devlock.sh`, with `PINKY_DEVICE` set to the one phone and a `-run` for one or a few tests, and log counts, never app names or what is on the screen. - **Reading the logs.** The runs read the device's syslog relay with a throwaway Go program over go-ios's `ios/syslog`, filtered to Pinky's processes and the kernel's memory lines, and never committed. Keep such a stream outside `devlock`, which would hold the phone for as long as it runs. - **Ports change.** The RemotePairing port the phone advertises changes with each phone and reboot, and the tunnel listener remotepairingd opens gets a new port each time. The app reads the first from `_remotepairing._tcp` on `lo0` (49152 is only the fallback) and the core the second from remotepairingd's answer. ## Pinky Helper inside the app **An ordinary app can host WebDriverAgent**, so Pinky's iPhone app can take Pinky Helper's place: desktop Pinky would install one app and start WebDriverAgent in it the way it starts the helper today. The probe ran on 12 November on the iPhone 13 mini (iOS 27.0.1). It was a bare app (`cc.diygod.pinky.runnerprobe`, development-signed by the team with `get-task-allow`) with WebDriverAgent's `WebDriverAgentRunner.xctest`, the one Pinky Helper carries, copied into its `PlugIns`. `TestRunnerHostProbe` (`internal/driver/ios/runnerhost_probe_device_test.go`, build tag `device`) starts WebDriverAgent's test through testmanagerd as desktop Pinky starts the helper's (`startWDA`), with the probe as the test host, and waits for its status: "WebDriverAgent ready in 11.827s … Build 16.15.3, OS 27.0.1", PASS. The probe app was uninstalled afterwards. What the host app needs: - **A test mode.** testmanagerd launches the host with `XCTestSessionIdentifier` (or `XCTestBundlePath`, `XCTestConfigurationFilePath`) in its environment. The host then loads XCTest from the developer disk image (`dlopen` of `/System/Developer/Library/Frameworks/XCTest.framework/XCTest`) and, once launched, calls its `_XCTestMain` on the main run loop, as Xcode's runner apps do. XCTest asks testmanagerd for the test configuration, loads the xctest from `PlugIns`, takes a background assertion for the process and runs WebDriverAgent's test, which serves on port 8100 until it is stopped. The probe's own window showed as usual. Launched any other way, the app is just itself. - **The scene life cycle.** Apps built with the iOS 27 SDK need a `UIApplicationSceneManifest` and a scene delegate. The first probe had neither and died at launch: "UIScene life cycle is required for apps built with this SDK". Pinky's SwiftUI app has scenes already. - **One team.** WebDriverAgent's framework and the xctest are signed with the app's identity; the profile needs nothing special. The probe's Info.plist also copied `UIBackgroundModes` `continuous` and ATS's arbitrary loads from WebDriverAgent's runner; whether it needs either was not tested. What it did not show: - It ran on iOS 27.0.1 only, started from the Mac. The helper is for iOS 17.4–26, where this was not tried. That needs a phone on one of those versions (M3). - WebDriverAgent could not write `com.apple.keyboard.preferences`, where it turns off autocorrection, prediction and the prediction bar at start: "setting preferences outside an application's container requires user-preference-write or file-write-data sandbox access". Whether Pinky Helper's runner gets the same refusal, and whether typed text then gets autocorrected, was not checked. - XCTest's diagnostic screen recording did not start (error 10005). This configuration prefers screenshots anyway, so it does no harm. M2 built it: [Hosting WebDriverAgent](#hosting-webdriveragent). ## Hosting WebDriverAgent (2026-11-12, M2.) Pinky's iPhone app hosts WebDriverAgent for desktop Pinky in place of Pinky Helper. **The app in test mode** (`ios/app/App/TestHost.swift`). Launched with `XCTestSessionIdentifier`, `XCTestBundlePath` or `XCTestConfigurationFilePath` in its environment, the app starts no core, no VPN and asks for no notifications (`PinkyApp`, whose `MainView` holds the model). It shows a screen of its own, `HostingView`: "Pinky on your computer is using this iPhone", and that the person can switch to other apps while tasks on this iPhone wait until the computer is done. After its first frame it loads XCTest from the developer disk image and calls `_XCTestMain`, as in the probe. The app's runpath has the image's framework folders (`project.yml`, `LD_RUNPATH_SEARCH_PATHS`), which XCTest's own libraries need. When the computer ends the test it kills the app; launched again by the person, it is the ordinary app. **One phone, one remote accessibility client.** While it hosts WebDriverAgent the app posts the Darwin notification `.computer.on` every 5 s. Whichever process runs the core (the packet tunnel, or the app in the lite build) passes it on with `PinkyComputer`, and the core counts the computer as holding the phone until 15 s after the last one (`ios/onphone/claims.go`, standalone's `Claims`, so the same rules as Android's companion): - A task started on the phone meanwhile is refused with the code `computer`: "Pinky on your computer is using this iPhone. Start the task when it is done, or once you choose Pause connection for this iPhone there." The app's main screen says the same and turns Start off. - A task running on the phone when the computer takes it stops, as on Android when a computer takes the phone. - Stop on the phone cannot reach the computer (`stoppable` false): the person stops the computer's task there. - The computer holds the phone for as long as WebDriverAgent runs, idle or not: desktop Pinky keeps the runner up while the phone rests (`internal/driver/ios/rest_wda.go`, since starting it again may ask for the passcode). Pause connection on the computer ends the test; the notifications stop and the phone is free 15 s later. Whether a running WebDriverAgent that is not in use really keeps the phone's `remoteAXService` from the core was not tried; refusing is the safe side until it is. **Installing it** (`internal/driver/ios/appinstall.go`). Each release carries the iPhone app kit, `pinky-iphone-.tar.gz` (`ios/build.sh kit`, run by the release workflow): the Xcode project XcodeGen made, the Swift sources and the Go core built for phones, an empty pairing. Desktop Pinky cannot build the core itself (no Go toolchain for iOS on the person's Mac), so the kit brings it. `pinkyctl helper install` and Install Pinky for iPhone in Pinky's window, after building WebDriverAgent's runner as before: 1. download the kit of this Pinky's version into `/WebDriverAgent` once (`PINKY_IPHONE_KIT` names an archive or folder instead; a development build without it skips to the helper); 2. build the scheme `Pinky` with the person's team, bundle id `pinky.` (lower case), with automatic signing, first with the profiles on the Mac, then with provisioning updates as for the helper; a team that cannot sign Network Extensions (a personal team) falls back to `PinkyLite`, `pinky..lite`; both are ids the policy takes for Pinky's app; 3. copy WebDriverAgent's `WebDriverAgentRunner.xctest` from the runner into the app's `PlugIns`, mark the app (`PinkyWebDriverAgentHost` in its Info.plist) and add the background mode `continuous`, sign the test bundle and its frameworks with the app's certificate and then the app again (`resignApp`); 4. write an xctestrun from the runner's with the app as the test host and run `xcodebuild test-without-building` with it, which installs the app and starts WebDriverAgent in it; WebDriverAgent answering is success, as for the helper. Any step that fails says so in the install's output and the log and installs Pinky Helper as before; `--helper-only` skips the app. A helper already on a phone goes on working: Pinky picks the runner by `pickRunnerApp`, which prefers an app with the mark over a helper, then helpers as before, so with both installed it starts WebDriverAgent in Pinky's app. `ios/build.sh` builds the same from source with `PINKY_WDA=` ([ios/README.md](../../ios/README.md)). **Agents on a computer and Pinky's screens.** WebDriverAgent starting in Pinky's app brings the app to the front, unlike the helper, whose screen is not Pinky's. Agents can only press Home or open another app on Pinky's screens (`policy.Evaluate`), which is what they do first anyway. Not checked on a phone whether the app stays in front after WebDriverAgent starts. Unit-tested: the claims (`claims_test.go`), the runner choice (`TestPickWDARunner`), the kit, the build's arguments, the mark, the copy and the xctestrun (`appinstall_test.go`). Both schemes built from a kit with automatic signing for a generic iOS device: `PinkyLite` with the maintainer's wildcard profile, `Pinky` refused by the wildcard profile for lacking Network Extensions, as expected without provisioning updates. On the iPhone 13 mini (iOS 27.0.1, 2026-11-12), `pinky helper install` with a kit over an installed Pinky Helper: `Pinky` was refused by the wildcard profile (no Network Extensions), so it built and installed `PinkyLite`; `xcodebuild test-without-building` takes it as the host, WDA served in it (`/status`, then a session in `TestWDAStopDevice`, stopped in about 0.4 s with no runner left), and the device tests observing the home screen and tapping Calculator passed afterwards. That run found two faults in the test mode, both fixed: `_XCTestMain` takes the test configuration as its argument, and called without one it got whatever the register held and aborted (`-[OS_os_log testInteropMode]: unrecognized selector`), so it is called with nil, which makes XCTest read the configuration testmanagerd set; and started from a main-queue block, XCTest kept the main queue busy for good, so WDA answered `/status` but hung on `/session`, so it starts from a run loop block. `TestDeviceType` fails on this phone on `main` as on this branch (one case or another reads the field unchanged), so it says nothing about the host. Not run on a phone: a task through the desktop app with the app as host, the hosting screen, and the full `Pinky` build with a profile that has Network Extensions. ## Signing What the research found. [P] marks a primary source: Apple, or the project's own code or docs. - **A free personal team cannot sign a packet tunnel**, by any route. Apple's capability table does not tick Network Extensions (or Personal VPN, or Push) for free accounts, Xcode refuses it ("Personal development teams … do not support the Network Extensions capability"), and Apple DTS says there is no workaround on iOS ([capabilities](https://developer.apple.com/help/account/reference/supported-capabilities-ios) [P], [forum 825136](https://developer.apple.com/forums/thread/825136) [P]). - **It can sign the rest Pinky needs**: App Groups, Keychain sharing, Background modes (audio), and a Live Activity updated by the app (no capability is involved; updating it by push would need Push). - **Limits** ([compare memberships](https://developer.apple.com/support/compare-memberships) [P]): 10 App IDs per 7 days, 3 devices, 3 apps per device, profiles that expire after 7 days. Every extension is an App ID of its own: the full build costs 3, the lite build 2. Keep the bundle id fixed, or every rebuild spends new App IDs. - **After 7 days** the app and its extensions stop launching until they are rebuilt and reinstalled. The app should read the expiry from `embedded.mobileprovision` and warn ahead (M2). Done (2026-11-12, `App/Profile.swift`): two days before, the first screen says when the app stops opening, and a notification scheduled at launch says it too. **The fallback for free accounts is the lite build**: LocalDevVPN from the App Store provides the loopback (its address set to a /30, the phone 10.7.0.2 and the peer 10.7.0.1; its 1.3.0 default, 10.7.1.1/32, got no answer in the prototype), and the core runs in Pinky's app, kept alive by silent audio during a task, as StikDebug does. The prototype measured that for 15 minutes (17–39 MB, +160 mA against +90 mA in the extension); hours, a locked phone and memory pressure are untested. The other fallbacks were an XCUITest runner like iCTRL (heavier: the developer disk image and a passcode prompt, private XCTest API) and requiring a paid account; the lite build keeps both builds from one project. ### SideStore and AltStore - Both re-sign from the provisioning profile and keep only the entitlements the profile has (AltSign `ALTSigner.mm`, SideStore's SideSign `CodeSignerAPI.swift` [P]). A free team's profile never has Network Extensions, so the full build would install with a tunnel that cannot run. **Only the lite build works through them.** - AltStore appends the team id to App Group ids; Pinky must read its group from its entitlements at run time if it ever uses one (it does not today: the app and the extension talk through app messages and the Keychain). - SideStore refreshes apps on the phone before the 7 days run out, through LocalDevVPN and a pairing file; it takes one of the 3 app slots itself. A computer is needed once, to install SideStore and its pairing file. On iOS 26.4+ that file must be a RemotePairing one. - Pinky would need nothing special for them beyond the lite build, an `.ipa` and a bundle id the person's team can register. Not tried. ### How M1 was signed - **Three App IDs in the maintainer's paid team**: `cc.diygod.pinky.standalone` (the app), `.tunnel` (the packet tunnel) and `.activity` (the Live Activity), registered with the App Store Connect CLI, `asc`, which was already signed in on the Mac. The app and `.tunnel` have the Network Extensions capability. Their entitlements (`packet-tunnel-provider`) are in `ios/app/project.yml`, which XcodeGen writes to `ios/app/build/gen/.entitlements`. - **One iOS App Development profile per App ID**, listing the iPhone 13 mini and valid until 2027-11-11, made and installed with the `asc` commands in [ios/README.md](../../ios/README.md). `asc profiles local install` puts a profile where Xcode keeps them, `~/Library/Developer/Xcode/UserData/Provisioning Profiles`. - **`ios/build.sh` signs, not Xcode.** Xcode builds with `CODE_SIGNING_ALLOWED=NO`. The script then signs the extensions and the app, each with the newest unexpired profile for `.` (the team's wildcard profile when there is none) and with that profile's certificate whose private key is in the login keychain. It adds `application-identifier`, `com.apple.developer.team-identifier` and `get-task-allow` to the target's entitlements. So Xcode needs no signed-in account. `PINKY_TEAM` names the team; the default is the maintainer's, whose id stays out of docs, commits and reports. - **Another phone**: register it (`asc devices register --name --udid --platform IOS`), make the three profiles again listing it (a profile keeps the devices it was made with), install them and build again. - The helper probe (`cc.diygod.pinky.runnerprobe`) was signed with the team's wildcard profile and registered no App ID. ## Setup on the phone - **Pairing on iOS 27, with no computer.** iOS 27 lets the phone pair with a host on the network from Settings › Privacy & Security › Developer Mode › "Pair with …": the app advertises `_remotepairing-pairable-host._tcp` (with a Mac model, "Mac17,7", since iOS treats the host as a computer), the phone connects and runs pair-setup with the roles flipped, and the app shows a 6-digit PIN (StikDebug posts it as a notification, since the person is in Settings) that the person types on the phone. The result is the same RemotePairing identity `rppair pair` makes ([idevice responder.rs](https://github.com/jkcoxson/idevice/blob/master/idevice/src/remote_pairing/responder.rs) [P], StikDebug 3.2.0 [P]). It needs Local Network permission, Wi-Fi, the app alive while the person is in Settings (background audio), and Developer Mode on, whose switch appears only once something started pairing or a Mac paired with the phone. Built in M2, below. - **The developer disk image.** Pinky needs it on the phone too: taps go through the DDI's HID daemon. The test phone's came from a Mac. From 26.4 Xcode installs a Cryptex1 DDI through `cryptexd` with persistence 2, which survives reboots until an update or reset; from 17.0 to 26.3 it is a personalized image (a TSS ticket from Apple, so internet once) that every reboot unmounts. Both can be done from the phone over the loopback (StikDebug, iCTRL); the files come from Xcode or the community mirror (about 100 MB). - **What the app does for it (M2).** On iOS 27 an unpaired app shows Pair on its first screen. The core in the app (not the extension: pairing needs no tunnel) listens on a TCP port (`ios/core/pair.go`, `PinkyPairStart`); the app advertises it as `_remotepairing-pairable-host._tcp` through the system's Bonjour (`App/PhonePairing.swift`, `DNSServiceRegister`: Go's own multicast would need an entitlement), keeps itself running with the silent audio, and lists the three steps (Settings › Privacy & Security › Developer Mode, pair with "Pinky", type the code). `rppair.Respond` answers the phone: the handshake as a device that allows pair-setup, then M1–M6 with a fresh 6-digit PIN for each try, which the app shows and posts as a time-sensitive notification. A wrong PIN ends that try with error 2 and the app says so; the phone asks again with a new code. The identity that results goes to the Keychain as the pairing a Mac would have made, and the app starts the VPN with it, or restarts it when it was on (the extension's core reads the pairing at start); in the lite build the core in the app takes it on a second `PinkyStart`. Tested against our own client playing the phone (`respond_test.go`), and with the iPhone 13 mini on 2026-11-12: [the run](#pairing-on-the-phone-run). Each start makes a new identity and altIRK; a host that a phone already trusts is never advertised again, so the phone has no stale entry to verify against. - **The DDI is not mounted from the phone in M2.** On iOS 27 the Cryptex1 DDI survives reboots until the next system update, so a phone set up with Xcode or desktop Pinky keeps it, and the test phone has it. Mounting it from the phone means fetching about 100 MB (Xcode's files or the community mirror), a personalization ticket from Apple, and `cryptexd` over the loopback; Pinky has the mount code (`internal/driver/ios/cryptex`), but the extension that hosts the core has about 50 MB, so it would run in the app, and trying it means unmounting the test phone's DDI first, which is the owner's to do. Until then a phone without it fails its first read, and the app says only that Pinky cannot reach the phone. Follow-up: detect the missing DDI and say so, then mount it from the app (M3). - **iOS 17.4–26 needs a computer once**, to make the RemotePairing file; on-device pairing is iOS 27 only. It must be a RemotePairing file, not a lockdown pair record: from 26.4 the phone refuses lockdown over the loopback (seen by StikDebug, SideStore and iCTRL). Pinky's desktop can make it (`rppair pair`). On those versions Pinky's iPhone app also needs the DDI mounted for the accessibility and display services it uses, which below 26.4 means after every reboot. Not tried: M3. ### Pairing on the phone: run 2026-11-12, the iPhone 13 mini (iOS 27.0.1), a debug full build from main (2026.1112.16) plus the pairing logs, installed over the M1 build and launched once with `-pinkyForgetPairing YES` so that it started unpaired. The owner did every step on the phone; the run read the device's syslog only. - **Listed and paired.** The app listened (port 62597) and advertised itself; the owner found Pinky under Developer Mode › "Pair with …", picked it, and the phone asked for a code 42 s later. 7 s after that, pair-setup finished: the phone took our M6 (signature and info) without complaint. It sent a `remotepairing_udid`. First try, no wrong code. - **Connected with that pairing alone.** The app restarted the VPN with the new pairing; the extension's core pair-verified with remotepairingd through the loopback (`connected in=330ms services=85`) and `remoteAXService` answered in 14 ms, with desktop Pinky connected to the phone at the same time (resting, not paused). Pair-verify succeeding is the proof that iOS stored the identity M6 sent. - **A task with no computer in the phone's link.** The calc script of the fake model (on the Mac over Wi-Fi, standing in for the model's API) ran three times, each launched from the debug arguments: Calculator opened, 7 × 6 = tapped, 42 read from the accessibility tree, home pressed; 10 steps in 36 to 48 s, ended done, no memory kill. The first two tries to open Pinky got `not_found`: the phone also has the lite build, so two apps are called Pinky. Opening it by id (`-pinky cc.diygod.pinky.standalone`) was refused by the guard (`blocked_by_policy`, "Pinky's own apps"). - **What it took**: two app changes, both for tests. ### A real model on the phone: run 2026-11-12, 23:34, the iPhone 13 mini (iOS 27.0.1), the same debug full build as [the pairing run](#pairing-on-the-phone-run), with the pairing made on the phone. The lite build (`pinky..lite`, from desktop Pinky's install) was uninstalled first, so the phone has one app called Pinky. Desktop Pinky was connected to the phone and resting, not paused. - **The model**: the owner's OpenAI-compatible gateway on the home network (`http://mac.local:3425/v1`, model `group/deepseek`, a DeepSeek model), given by the debug launch arguments (`-provider openai -model group/deepseek -baseURL …`) with the key in the launch environment through devicectl's `DEVICECTL_CHILD_PINKY_MODEL_KEY`, so it was on no command line and in no file. The task: "Work out 12 × 34 in Calculator". - **What ran**: the app was ended by its process id and launched again, keeping the tunnel. The core in the extension started the task 5 s after the launch and ended it 24 s later, done. The model made two calls: `phone_launch` Calculator, then one `phone_batch` of seven taps (clear, 1, 2, ×, 3, 4, =), whose result read 408 from the accessibility tree; then `finish` with "在 Calculator 里依次输入 12 × 34 =，得到结果 408。" (it wrote the summary in Chinese although the task was English; the prompt asks for the task's language). 30,399 tokens in all, not priced (the gateway's model is unknown to the price list). The first read was slow, 2.9 s, 2.7 s of it walking the tree; no memory kill. - **No computer in the path**: the phone's core reached the gateway over Wi-Fi by itself; the Mac only launched the app and read the syslog, and the gateway is another machine on the network (the Mac's own port 3425 listens on loopback only, which the phone cannot reach). Steps, the summary and the screen reads came from the history database in the extension's container (`devicectl device copy from --domain-type appDataContainer --domain-identifier .tunnel`, `Library/Application Support/Pinky`), which a debug build allows; the copy was deleted afterwards. The app's own container could not be listed that way (CoreDevice error 3). The app now logs each pairing step to the syslog (`pinky pairing: …`, category `pairing`: listening, the phone asked for a code, a failed try with its error, paired and whether the phone sent a UDID; never the code). A debug build launched with `-pinkyForgetPairing YES` drops the pairing in its Keychain once, since that outlives reinstalling the app. ### Model services Since 2026-11-13 Settings offers desktop Pinky's model services ([model-services.md](model-services.md)), from `ios/app/App/ModelServices.json` (generated from `internal/pilot`, read by `Services.swift`). UserDefaults keeps `service`, and per service `model.` and `baseURL.` (empty follows the preset's address); settings of before (`provider`, `model`, `baseURL`) name the service and give the model and address to their API's entries. Keys stay in the Keychain as `key.`, the name being `pilot.KeyName`, so keys saved before still apply. Saving a key, model or address sends the core `check` (`AppModel.check`); Settings shows usage (today, 7 and 30 days, the last 7 days row by row) from the core's `usage`, and the task box shows today's. Checked: `ios/build.sh sim` and `ios/build.sh lite`, the string catalog in sync, and Settings and the task box in demo mode (`-pinkyDemo ready`) on a simulator. Not checked: the usage and check sections on screen, and a real phone (the simulator has no core). ## Plan for M2 and M3 **M2** makes the app something a person can set up and use: - **Onboarding on the phone**: pairing on iOS 27 and mounting the developer disk image from the phone ([Setup on the phone](#setup-on-the-phone)), then the VPN, the model and its key, in the app's first screens. Built (2026-11-12), except the DDI: the first screen leads through Pair (iOS 27; older systems still need a computer), Turn on for the VPN, then the model and key, one step at a time. Pairing worked on the iPhone 13 mini ([the run](#pairing-on-the-phone-run)); the VPN's first-time prompt and the model and key steps were not tried from a fresh app. - **Three languages**, as desktop Pinky: English, Simplified Chinese and Japanese, with the terms `cmd/pinky/locales` already uses (CLAUDE.md, "The app's languages"). - **The Live Activity with a real task**, an approval among its steps ("Pinky waits for you"), Stop from the activity, and how the activity ends. - **Pinky's app as WebDriverAgent's host** in place of Pinky Helper ([Pinky Helper inside the app](#pinky-helper-inside-the-app)): the test mode, WebDriverAgent's xctest and framework in `PlugIns` signed with the app, and desktop Pinky installing Pinky's app where `internal/driver/ios/helperinstall.go` installs the helper now, under `pinky.` ids, which the policy already takes for Pinky's app. Built; what the app shows meanwhile and what a task started on the phone does are in [Hosting WebDriverAgent](#hosting-webdriveragent). While Pinky's app is in front, agents on the computer can only press Home or open another app (`policy.Evaluate`). - **The rename**: Pinky Helper becomes Pinky for iPhone (「iPhone 版 Pinky」 in Chinese and Japanese, as Pinky for Android is 「Android 版 Pinky」) in desktop Pinky's screens and messages (three languages), both READMEs and the docs. Done (2026-11-12): the install button is Install Pinky for iPhone, Add phone's way is Use Pinky for iPhone, and the old name stays only where it means an older install (Pinky Helper keeps working) and in the IDs, `pinkyctl helper install` and `connection: helper`. The client still reads a gateway's older words ("Switching to the helper"). Helpers already installed keep working, and `IsPinkyApp` keeps counting every `.xctrunner`. - **The follow-ups below**, most of all desktop Pinky letting go of the phone's accessibility service while idle, the VPN reconnecting, and a retry for a failed read; and the warning before the profile expires ([Signing](#signing)). Done (2026-11-12), not tried on a phone. - **With the owner**: a real model, an approval with Face ID, a handoff and Stop during a task on the phone ([Needs the owner](#needs-the-owner)). **M3** widens what it runs on and how long: - **iOS 17.4–26**: the app's screens on Simulators of those versions (a Simulator runs no packet tunnel, so the loopback and the driver need a phone), and a phone on one of them for the rest: the pairing file from a computer, the DDI after every reboot below 26.4, and WebDriverAgent hosted by the app. - **Long runs**: battery and memory over hours, bigger trees (long lists, web pages), a phone that locks during a task, memory pressure in the extension. - **Other ways to install**: the lite build on a phone, signed by a free personal team, then SideStore and AltStore. - **Polish** of what M2 leaves rough. **Release notes.** M1 has none: the app is built from source only, and a real model, approvals and Stop on the phone are not verified yet. Its one change to what people get, that agents can no longer open or act on Pinky's iPhone app, rides along with the next release. When the app is ready for people, its notes say what it does and that change too. ## Quick entries Since 2026-11-13, after the Android app's (android-app.md, "Quick entries"): - **Voice input** in the task box (`App/Dictation.swift`): Apple's speech recognition, on the phone when its language allows (`requiresOnDeviceRecognition` when the recognizer supports it), otherwise through Apple as the keyboard's dictation does. It asks for speech recognition and the microphone the first time, listens until the person taps again or for 55 s (the recognizer takes about a minute), puts what it heard after what was typed, and never starts the task itself. It listens only while no task runs, so it never meets `SilentAudio`'s session. - **New Task in Pinky** (`App/QuickStart.swift`), an App Intent for Shortcuts and Siri ("New task in Pinky", "Ask Pinky"): it opens the app with the task in the box and, optionally, text or a link to go with it (the core's `shared`, as Android's Share to Pinky). A shortcut that takes the share sheet's input and runs it shares into Pinky. The person starts the task in the app: automations run without anyone looking, so a shortcut never starts one. - **No Share extension:** only the app can send the core in the tunnel a message (provider messages belong to the containing app), and with no App Group (AltStore appends the team id) an extension cannot hand the app what was shared. Pictures would also go through the tunnel extension's 50 MB, which a task already uses up to 28 MB of (run 7). - **No "ask about this screen":** a task starts from Pinky's own app, which is in front then, and opening the app from Shortcuts puts it in front too; there is no floating button over other apps on iOS. - **Checked:** the Simulator build (`ios/build.sh sim`), the App Intents metadata and both usage strings in the bundle, and the mic in the task box in the demo (`-pinkyDemo ready`, iOS 26.5 Simulator). Not checked: listening (the Simulator would use the Mac's microphone), the intent from Shortcuts, and both on a phone. ## Follow-ups - **Agents on a computer know Pinky's app only by its id**: `cc.diygod.pinky…` and `pinky.…`. A build under any other `PINKY_BUNDLE_ID` is guarded on the phone, where the guard learns the app's id, but agents on a computer could open it. [ios/README.md](../../ios/README.md) asks builders for `pinky.`; once desktop Pinky builds the app, it uses that shape anyway. - **The app connects the VPN only at launch.** When the extension dies while the app runs (a memory kill, a crash, or `--terminate-existing` as in run 4), the app shows Turn on and stays there. It could reconnect once by itself and say why the tunnel went down. Done (2026-11-12): when the tunnel reaches disconnected with an error iOS keeps (`fetchLastDisconnectError`) and nobody turned it off in the app, the app turns it on again, once in ten minutes, and says why; a second drop within that time only says so. A tunnel turned off in Settings › VPN is taken to come without an error and stays off; not checked on a phone. - **One failed read ends a task on the phone.** The gateway turns a failed read into an internal error ("Reading the screen failed", `pipeline.go`), and the agent loop stops on it before the model is asked. On a computer the inspector catches most of these; on the phone a retry, or the fallback in the table above, would have to. Done (2026-11-12) for the read that ended tasks: when `phone_connect` leases the phone but cannot read its first screen, Pinky's agent (`internal/pilot`) waits a second and reads once more with `phone_observe` before it gives up (`TestFirstReadRetried`). Later failed reads already reach the model as an error result, which it can answer by observing again. - **`raxReady` holds `svcMu` for its whole dial**, up to 15 s on the phone, so every other service waits behind a dial that hangs. A mutex of its own for the dial would fix that. Done (2026-11-12): the dial holds `raxDialMu`; `svcMu` guards only the session and the failure it remembers, and a session dialed while the device closed is closed at once. - **The phone's own reads need desktop Pinky to let go of the iPhone.** While desktop Pinky had the iPhone open, `remoteAXService` never answered the core on the phone (runs 2 and 3); with its connection paused, it answered at once (runs 5 and 6, and the prototype's reads on 11 November). So the phone most likely serves one remote accessibility client at a time. Desktop Pinky opens its session when it opens a phone and keeps it while no task runs (`device.go`, `run`). It could close the session while idle, and the phone's app could say that Pinky on a computer is using this iPhone instead of failing at the first read. Until then, a task on the phone needs Pause connection on the computer first. Done in 2026.1112.7: desktop Pinky lets an idle iPhone's sessions go (`internal/driver/ios/rest.go`). - **No pictures on the phone.** The core reads the accessibility tree alone (run 5), so text that only OCR finds (in images, canvases and some web views) is missing, and an agent that asks for a screenshot is told the phone cannot take one. A smaller picture might fit in the extension, and the lite build, whose core runs in the app, has more memory to spare; neither was tried. ## Not verified - Anything on iOS 17.4–26 (M3). - The lite build on a phone, and with a free personal team. - Hours-long runs, battery, a locked phone during a task, memory pressure in the extension. - On-phone pairing: one try on one phone (iOS 27.0.1, [the run](#pairing-on-the-phone-run)). Not tried: a wrong code, a phone where Developer Mode is still off, another iOS 27 version. The phone's SRP public key with a leading zero byte may still fail about one try in 256 as a wrong code; the person pairs again. - The app's VPN prompt and the model and key steps right after a fresh pairing: the test phone already had Pinky's VPN configuration. - The DDI mount from the phone (not built, see [Setup on the phone](#setup-on-the-phone)). - SideStore or AltStore installs. - A real model on the phone beyond one short task: one Calculator task with a DeepSeek model ran end to end ([the run](#a-real-model-on-the-phone-run)); Claude through the Anthropic API, longer tasks and the key typed in the app's Settings were not tried. - An approval, a handoff or Stop during a task on the phone: the calculator task needed none. The core's side of an approval is tested on a Mac with a fake phone (`TestApprovalInPinkysApp`); Allow behind Face ID, Stop on the Live Activity and how the activity ends were not tried on the phone. - Memory on other screens. The peak, 28 MB, came from reading the home screen, the biggest tree in the run; apps with bigger trees (long lists, web pages) were not tried. - Any WDA host on iOS 17.4–26 (Pinky's app as the host ran on the iPhone 13 mini, iOS 27; see [Hosting WebDriverAgent](#hosting-webdriveragent)). ## Needs the owner - Unlocking the phone and answering the first-launch prompts (notifications, Local Network, adding the VPN configuration) for each new build or bundle id. - A real-model run from the app's own screens: the owner types the key into Pinky's Settings on the phone (the [run](#a-real-model-on-the-phone-run) passed the owner's key in the launch environment instead). - Pairing on the phone (iOS 27): a debug build without `build/Pairing.json`, launched once with `-pinkyForgetPairing YES` (iOS may keep an app's Keychain items after the app is deleted, so reinstalling is not enough), then Pair, Settings › Privacy & Security › Developer Mode, and the code. Pairing touches a security setting, so only the owner does it. - A phone on iOS 17.4–26 to try the helper inside the app (M3). - A task from the computer with Pinky's app as WDA's host on the iPhone 13 mini (installed and its WDA checked on 2026-11-12), the app's hosting screen meanwhile, and a task started on the phone then, which should be refused. - Pause connection for the iPhone in desktop Pinky before a task on the phone, until desktop Pinky lets go of the phone's accessibility service by itself (Follow-ups). ## Cleanup list for the owner - Pinky (`cc.diygod.pinky.standalone`) on the iPhone 13 mini, and its VPN configuration ("Pinky", in Settings › VPN). Deleting the app removes its Keychain items (the pairing and any key) with it. - Three RemotePairing host identities on the iPhone 13 mini: the prototype's, EF52BBBF-C94D-495A-9BE5-09F369830DE7 from this build, and the one the app made when it paired on the phone on 2026-11-12, which lives only in the app's Keychain. iOS shows no list of them; Settings › General › Transfer or Reset iPhone › Reset › Reset Location & Privacy forgets trusted computers, and whether it forgets these too was not checked. A passcode change makes them useless. The Mac copy is `~/.pinky-standalone/rp_pairing.plist`; delete it when the app goes. The debug build carries another copy (`ios/app/build/Pairing.json`, and inside the built app), git-ignored in this worktree's `ios/app/build/`: it goes with the worktree. - The App IDs `cc.diygod.pinky.standalone`, `.tunnel` and `.activity` and their development profiles in the team (valid to 2027-11-11); keep them while the app is in use. - Desktop Pinky has had the iPhone paused since 12 November 01:36 (run 4 and later): choose Resume connection in the iPhone's menu. - The lite build desktop Pinky's install put on the iPhone 13 mini (`pinky..lite`) was uninstalled on 2026-11-12; WDA hosted in Pinky's app went with it, so desktop Pinky needs Pinky Helper or a new Install Pinky for iPhone to drive that phone through WDA. - The helper probe (`cc.diygod.pinky.runnerprobe`) is uninstalled. It was signed with the team's wildcard profile, so it registered no App ID.