# Engineering notes How the loop works and why it works that way. Almost every rule below exists because the simpler version failed on a real page first. ## Record, replay, heal A goal run can be saved as a recipe (`browse_goal saveAs`). `browse_replay` repeats it with **no model calls** — each element found again by several locators, strongest first: accessible role and name, label, the form field's own name, the id, a framework id's stable tail. When the page changes so that all of them miss, only that step is handed to Jev as a one-step instruction, and what Jev did is written back in its place. The next replay is free again. Measured on all six hard cases (`node bench/heal.mjs`): record, replay, then **redesign the page** — rename the control's id everywhere it is used and change its visible text, so the page still works but no recorded locator matches — and replay the old recipe. | case | record (Jev) | replay | old recipe after redesign | heal | repaired recipe | |---|---|---|---|---|---| | calendar | 10.0 s · $0.000506 | 3.0 s · **$0** | breaks | "Onayla" → "Tamam" · $0.000085 | 3.0 s · **$0** | | autocomplete | 8.8 s · $0.000202 | 1.3 s · **$0** | breaks | "Sehir" → "Konum" · $0.000043 | 1.3 s · **$0** | | dependent dropdown | 4.4 s · $0.00017 | 1.3 s · **$0** | breaks | "Ulke" → "Ulke secimi" · $0.000055 | 1.3 s · **$0** | | ordered form | 11.5 s · $0.000606 | 2.5 s · **$0** | breaks | two fields · $0.00017 | 2.9 s · **$0** | | delayed dialog | 5.8 s · $0.000166 | 2.6 s · **$0** | breaks | "Evet, devam" → "Onayliyorum" · $0.000045 | 2.6 s · **$0** | | form in an iframe | 9.2 s · $0.000531 | 2.3 s · **$0** | breaks | "Ad Soyad" → "Adiniz" · $0.000066 | 2.3 s · **$0** | Six of six on the latest run; across the day's runs the misses were a recording that failed on a gateway error and the iframe case's borderline decision at the *recording* stage — the replay and the heal did not fail once a recipe existed. That iframe case passes 28 of 28 when it runs alone; its earlier misses all happened while another benchmark was driving a second browser at the same time. Three things had to be right for this to work, and each was found by it failing first. The heal is asked in the imperative with the old label named as possibly changed ("Click the button that was labelled "Gonder"") — asked with the recorded past-tense sentence, Jev would not commit. The recorded value goes with it, because the text model will not invent a value for a field whose label no longer matches the goal. And the heal stops after one action: left to run, it finished the job — ticked the box as well — and the replay then ticked it again from its own recipe. ### On a real JSF application Recorded on a live JSF/PrimeFaces admin console, read-only (screen and field names anonymised): open the class-management screen, search the catalogue for "report", press Search. The recipe came out as three steps found by strong locators — `role link "Class Management"`, `name=searchInput_input`, `role link "Search"` — and replayed with no model in 4.4–4.9 s against 13–23 s to record. To prove the recipe does the work and not the server's memory, the session was first left holding a different search ("zzz", the matching rows not on screen); replayed from there three times, it reached the six matching rows and passed its check every time, at $0. Getting there took four fixes, each found on that console and none visible on the test pages: - **An unnamed field is never a place to type, and an unnamed control is never a guess to click.** Asked to open a screen and then search, Jev went straight for the only textbox on the first page — the header's global search, with no label, no placeholder and a generated id — and the text model filled it with "Class Management". With that removed, it clicked the header's unnamed search icon four times instead. Both are still reachable on purpose through `ref=`; they are just not offered as candidates while anything on the page has a name. - **A compound goal gets a second opinion as numbered steps.** "Open Class Management, type report into the search box, press Search" left the click target at "none" (0.51), because nothing on the first page advances the whole sentence; the same text as "1) … 2) … 3) …" chose the link at 0.97. An instruction to "do the first unfinished part" changed nothing. Planning every compound goal was tried and rejected — it folded a three-field registration into one step and broke a passing case — so the plan is written by the text model and consulted only when the sentence leaves Jev without a target. - **A field that is already filled is still recorded, in the order it must have been filled.** JSF keeps form state in the session, so a second run found "report" already in the box and went straight to Search; the recipe then had no typing step, and once the step was added it came after the click. It is now inserted before the steps taken on that page. - **A run that did not finish records no expected outcome.** The first attempt ended on a server error page, and its "new text on the final page" — an internal file path — became what a replay would be checked for. A replay with nothing to check now says `unverified` instead of passing; `expect` can be given on `browse_goal` or `browse_replay` once you have confirmed the page. A recipe can contain text from the pages it was recorded on. Treat recipe files like the pages themselves when the application is private. `browse_replay export=playwright` writes the same recipe as a plain `@playwright/test` spec. It has no fallbacks and no healing, so it is verified the hard way: run with plain Playwright, without this server, all six exported specs pass. ## Where the cost went The first working version spent 7,226 tokens per decision. It now spends about 1,500. Three measurements got it there: **The element list was being sent twice** — once in the state, once again as the target question's criteria — which was 56% of every request. Jev chooses from the criteria; it never needed the copy. **Removing it entirely broke the run.** With no idea what the page offered, the operation question answered correctly at p=0.51, below the bar, and the loop escalated on its first step. It needs the *shape* of what is available, not the list: a census (`{link: 42, button: 8, searchbox: 1}`) plus the handful of labels that share words with the goal, picked by set overlap in code. About fifty tokens. **Each element got shorter.** `{"id":"f1-e3","role":"link","label":"Donate","value":null,...}` is 146 characters; `link "Donate"` is 19 and answers the same question. Coarse position ("upper right") was dropped after it never once decided an answer. ## What the loop owns, and why These are settled in code rather than asked of the model, each because asking failed first. **Progress.** The loop scrolled twelve times in a row on a real page while the link it wanted sat in the element list. Jev was not wrong — it was shown an identical state each time. An action that changes nothing twice is withdrawn from the next decision. **Reaching a target.** On a 45-link index the target question named the right link at 0.91 on every step and the operation question chose `scroll` at 0.87–0.93, spending three steps travelling to it. The executor scrolls to whatever it is told to click, so a confident target now overrides a scroll. Five steps became two. Rewording the scroll instruction was tried first and moved the number by 0.02; longer instructions consistently made this model worse, not better. **Where the click lands.** Scrolling to a target moves it, so the click point is corrected by how far the page actually scrolled. Before that, every action on an element below the fold went to the coordinates the element used to occupy — usually empty space. Measured on a live portal: a plain `` in the lower half of the page could not be followed at all, and three runs in a row concluded the site was broken. One line of arithmetic turned a four-click failure into one click. **Waiting.** A fixed 700 ms pause is the same as no pause, because the pages that need waiting take longer than any constant you would pick. Settling samples the element count *and the page's text length* until both hold still — the count alone settles while a server-rendered app still has an empty content region — and a click that looks inert is given three more seconds to commit a navigation before it is written off. **Completion.** `DONE` is checked, never taken on the model's word. A separate `goalComplete` judgement must clear 0.7 and a `stillLoading` signal outranks both. Measured: goals objectively met answer 0.80, goals plainly not met answer 0.02–0.03, and the arguable ones land at 0.23–0.66 — which is why the middle band becomes an escalation with evidence instead of a verdict. That last check earned its place immediately: it refused a run that this project's own benchmark had been scoring as a pass. The goal said "open the card and see the items inside"; the URL was right, so the test was green. The account had zero selection rights, the card expanded to nothing, and there were no classes to see. The tool was right and the test was wrong. ## Embedded documents `page.evaluate` only ever reaches the main document, so a form served in an iframe — how most corporate portals deliver a legacy screen, and how every payment and consent widget arrives — was simply invisible. Every frame is now walked separately (a cross-origin frame has no reachable document at all, so `contentDocument` is not an option), each element's box is shifted by the position of the `