# Building for WebAssembly (Emscripten) The VM can be cross-built to WebAssembly with Emscripten. The build runs from the root of this repository, out of tree, by default in `build-wasm/`. It builds the StackVM, the interpreter (a JIT needs executable memory), for 64-bit images, with the FFI on libffi but no threads, and runs the stock Pharo 12 image by default. Pharo 13, 14 and 15 images run too: a site is built with one (`WASM_IMAGE_ZIP`), may offer several, which its pages let the user choose from (`WASM_EXTRA_IMAGES`), and the pages open any of them from disk; the Smalltalk of the pages works with all of them (see Pharo versions). The site that GitHub Actions builds offers Pharo 15 (the default), 14 and 13.1. Two VMs are linked from the same objects: - a command-line VM for node, with access to the host file system, environment and exit status; - a VM for a Web Worker, which a static web site runs: the Console, a Smalltalk REPL with Stop, Save, Download, Open, which starts an image of your own, and Images, which starts another image of the site or one saved in the browser, and its Notebook tab, cells of Smalltalk, which a second VM of their own runs, and of Markdown, which the page renders. The site also has the world page: the Morphic world of the image in a canvas, drawn through the WebDisplayPlugin. Both VMs link the libraries that the image calls through its FFI, built from pinned archives: FreeType 2.14.3 for the fonts of the world, and cairo 1.18.4 for Athens and Roassal, by default; and libgit2 1.4.4 for Iceberg (`WASM_LIBGIT2=ON`), on request. Requirements: - Emscripten 6.0.10 (emsdk). The port needs that version, and was tested only with it: the web driver replaces the stream operations of Emscripten's terminal device, which belong to the internals of its file system. Nothing checks the version: with another one, run `make wasm-check`, whose lanes 60 and 70 drive those operations in node. - GNU make 3.81 or later (tested with 3.81, 4.2.1, 4.3 and 4.4.1) and CMake 3.13 or later (tested with 3.13.5 and 3.28.3). - node, which stages the site, runs the tests and serves the pages (tested with 25.2.1). - A native C compiler: the sources are generated in a native CMake tree, `build-wasm/host`. - Network access for the first build (files.pharo.org and GitHub, and the sites of the archives of the libraries: github.com, download.savannah.gnu.org, www.cairographics.org and download.sourceforge.net), unless the offline settings below give its inputs. - A source tree whose path holds no blank. Engines, as Emscripten's feature matrix gives them for the default settings: - The node VM uses a 64-bit memory (memory64) and the exnref instructions of WebAssembly exception handling, for setjmp and longjmp. It needs node 24.15 or later. - The web VM has a 32-bit memory with 64-bit pointers (`WASM_WEB_MEMORY64=2`), so it needs only exnref: Chrome and Edge 137, Firefox 131 or Safari 18.4, or later. `WASM_SJLJ=wasm-legacy` builds it for engines without exnref (Chrome 95, Firefox 100, Safari 15.2). - The pages inflate what they download with DecompressionStream (Chrome 80, Firefox 113, Safari 16.4), and say so in a browser without it, so a `wasm-legacy` build needs Chrome 95, Firefox 113 or Safari 16.4, or later. - The pages were tested in Chromium 153 and Firefox 155. Safari is untested. - The world page also needs OffscreenCanvas: Chrome 69, Firefox 105, Safari 17. ## Building source /path/to/emsdk/emsdk_env.sh make wasm # everything, in build-wasm/ make wasm-check # the node test lanes make wasm-serve # serves build-wasm/web at http://localhost:8080/ `make wasm` first generates the sources of the StackVM in `build-wasm/host`. That native tree bootstraps a VMMaker image (a Pharo 13 image and VM from files.pharo.org, into which it loads the VMMaker packages of `smalltalksrc` and their dependencies from GitHub), and is configured with SSL and UUID off, since it builds nothing but the sources. Later, when a `.st` file of `smalltalksrc` changes, comes or goes, `scripts/refreshVMMaker.st` reloads the tree's packages into that image, and the sources are generated again. Then `build-wasm/cmake` is configured with `emcmake` and the initial cache `cmake/Emscripten.cache.cmake`, and built. It is configured again when a setting or that initial cache changes, and CMake configures it again by itself when a plugin of `src/emscripten/plugins` comes or goes. The build downloads the pinned Pharo 12 image (`Pharo12.0-SNAPSHOT.build.1599.sha.7d5f14cb47.arch.64bit.zip`, checked against its SHA256) and the pinned archives of the libraries (see The libraries), and stages `build-wasm/node` and `build-wasm/web`. The first `make wasm` took about 2.5 minutes from a clean clone with the downloads, and about 1.5 minutes offline, on the machine the port was made on: most of that time goes to the VMMaker image, which loads its packages and generates the sources in a single process. A new build directory with the sources generated already (`WASM_GENERATED`) and the archives of the libraries at hand (`WASM_DEPS_DIR`), with libgit2, took 54 s there, on 112 processors, with the three images of the site of GitHub Actions to prepare, and 56 s with the pinned Pharo 12. When nothing has changed, it takes under a second. To build offline, give the inputs that the first build would download: a VMMaker image, the Pharo VM that runs it, the image zip (from https://files.pharo.org/image/120/), and a directory holding the archives of the libraries. With a native build of this repository in `build/` that generated its sources, on Linux, and the zip and the archives in the source tree: make wasm WASM_VMMAKER_IMAGE=build/build/vmmaker/image/VMMaker.image \ WASM_VMMAKER_VM=build/build/vmmaker/vm/pharo \ WASM_IMAGE_ZIP=Pharo12.0-SNAPSHOT.build.1599.sha.7d5f14cb47.arch.64bit.zip \ WASM_DEPS_DIR=deps-archives An earlier `make wasm` has the same inputs in `build-wasm/host/build/vmmaker/` and `build-wasm/downloads/`. `WASM_GENERATED` instead takes sources generated already (see Settings). A site has a default image, the pinned Pharo 12 or the zip of `WASM_IMAGE_ZIP`, and may offer more images, each from a local zip, which its pages then let the user choose from (see The world page): `WASM_EXTRA_IMAGES` names them as `=` words. The site of GitHub Actions is built so, with the zips of files.pharo.org under their own names (from which the pages read the build of each image): make wasm WASM_IMAGE_ZIP=Pharo15.0-SNAPSHOT.build.50.sha.b85960fcc6.arch.64bit.zip WASM_IMAGE_ID=p15 \ WASM_EXTRA_IMAGES="p13=Pharo13.0-SNAPSHOT.build.749.sha.d7c6f761d5.arch.64bit.zip p14=Pharo14.0-SNAPSHOT.build.785.sha.74f3a1b398.arch.64bit.zip" \ WASM_EXTRA_SOURCES=Pharo15.0-64bit-4e572fe.sources WASM_LIBGIT2=ON Each extra image is unpacked into `build-wasm/image/stock-/`, prepared for the world as the default one is (into `build-wasm/image/web-/`, once, and again only when its zip or OSWindow-Web changes), and staged under `web/image//`; the default image keeps the URLs it had, `web/image/Pharo.image.gz` and the like. `WASM_EXTRA_SOURCES` stages `.sources` files that no image of the site has, under `web/image/sources/`: an image saved in a browser from an image that the site served before still finds the source of its methods there. `WASM_GIT_PROXY` is the site's default CORS proxy of git, `https://cors.isomorphic-git.org/` unless it is given (see libgit2). The site must be served over HTTP: browsers run no workers from `file://` URLs. It needs no special headers (no COOP or COEP, since nothing uses SharedArrayBuffer), so any static web server will do. ## This produces - `build-wasm/node/pharo`: the command-line VM, a shell script that runs `pharo.js` (and `pharo.wasm`) next to it with node. `NODE` selects the node binary, and `NODE_OPTIONS_WASM` gives it options. Next to it are `THIRD-PARTY-NOTICES.txt` and the empty placeholders of the libraries of the FFI (`libcairo.so.2`, and `libgit2.so.1.4.4` with libgit2; see The FFI). - `build-wasm/web/`: the static site. It holds the Console page (`index.html`), the web VM (`pharo-web.js`, the factory `createPharoVM`, and `pharo-web.wasm`), the scripts of the pages and of the workers (see The web pages), the default image, its `.changes` and its `.sources`, gzipped, in `image/`, those of each extra image in `image//`, the `WASM_EXTRA_SOURCES` in `image/sources/`, the Smalltalk scripts of the pages in `st/`, `THIRD-PARTY-NOTICES.txt`, which the pages link to, and `manifest.json`, which lists the files the worker loads and says what the build has (`ffi`, `fonts`, `git`, `gitHttp`, `gitProxyDefault`, `world`, `libraries`, `webPackage`), with the catalog of its images (`images`, `imageId`, `extraSources`; see The web pages). The pages load every file with the build id in its URL (`?v=`), so a new build is never mixed with a cached one. The browser downloads about 25 MB for the image it starts, and nothing of the others: an image of about 55 MB (15.4 MB gzipped), the 43 MB `.sources` (7.6 MB gzipped), and `pharo-web.wasm`, 2.1 MB in the default build (743 KB gzipped). - `build-wasm/web/world.html`: the world page. With the world (`WASM_WORLD=ON`, the default, and a native Pharo VM, `WASM_HOST_PHARO`), `web/` holds the world image, which is the stock image with the OSWindow-Web package, prepared at build time; both pages boot it. Without it, `web/` holds the stock image, and the world page says that the build has no world image. - `build-wasm/image/stock/`: the unpacked stock image (as `Pharo.image` and `Pharo.changes`, with its `.sources`); `build-wasm/image/stock.stamp` records the SHA256 of the zip it came from, and its path. - `build-wasm/image/web/`: `Pharo-web.image` and `.changes`, the world image, and `OSWindow-Web.st`, the package filed out for `web-bootstrap.st`. - `build-wasm/image/stock-/` and `build-wasm/image/web-/`: the same for each extra image. - `build-wasm/host/`: the native tree that generates the sources, in `host/generated/64/{vm,plugins}`, and its VMMaker image (in `host/build/vmmaker/image`, or in `host/vmmaker-image` for a copy of `WASM_VMMAKER_IMAGE`). - `build-wasm/cmake/`: the CMake tree of the WebAssembly build, with the unpacked archives of the libraries in `cmake/deps/` and `THIRD-PARTY-NOTICES.txt` in `cmake/wasm/`. - `build-wasm/downloads/` and `build-wasm/tests-run/`: the downloaded image zip and archives of the libraries, and the scratch directory of `make wasm-check`. - `build-wasm/config.make`, `build-wasm/config-host.make` and `build-wasm/.make-wasm`: the recorded settings, and the mark of a directory that `make wasm` builds in. The workflow `.github/workflows/build.yml` makes such a site and node VM on every push, with the images of Pharo 15 (build 50, the default), 14 (build 785) and 13.1 (build 749), which it downloads from files.pharo.org and checks against their SHA256, the `.sources` of Pharo 15 build 41 (the image of the site before), and `WASM_LIBGIT2=ON` (the command above), so they have the FFI, FreeType, cairo, libgit2 and the Notebook. It zips them as `PharoVM--WebAssembly-bin.zip`, with `serve.mjs` and the `README.md` and `node-try.sh` of `packaging/emscripten/dist` (the README says how to use them; `node-try.sh` runs the default image), and, for a push to `pharo-12`, publishes the zip with the release of the push. For a push to `pharo-12` it also deploys `web/` to GitHub Pages once it has checked that every file of the manifest is there and of its size. The node VM takes the usual arguments of the VM, for example on a copy of the stock image: mkdir -p build-wasm/try && cp build-wasm/image/stock/* build-wasm/try/ build-wasm/node/pharo --headless build-wasm/try/Pharo.image --no-default-preferences eval '3 + 4' build-wasm/node/pharo --headless build-wasm/try/Pharo.image --no-default-preferences st --quit script.st ## The Console page `index.html` runs the REPL of `st/web-repl.st` in the image, in a Web Worker. It has two tabs, Console and Notebook (see The Notebook), and opens the one last used. Each line typed is evaluated, and its value printed, or the error it raised, with up to five frames. A syntax error says where it is, as in `Error: CodeError Undeclared variable x (line 1, column 1)` (`OCCodeError` in Pharo 15). A Warning that nothing handles is reported (as `Error: Warning ...`), and the evaluation goes on. The REPL is also the error handler of the other processes: an error in a process that an evaluation forked is reported, and only that process ends. Output streams as the VM writes it, and the terminal keeps the last 20000 lines. - Enter evaluates, and Shift+Enter adds a line. Up and Down recall the history, which is kept in the browser's localStorage when there is one. - Stop (Esc, or Ctrl+C with nothing selected) interrupts the evaluation, also one that has not started yet. When nothing could be interrupted after the first prompt (the image registered no interrupt semaphore), or when the worker says nothing for 3 s after the Stop (a primitive that runs that long never yields), the worker is replaced and the VM starts again: the changes since the last Save are lost. - The browser keeps any number of images, the saved images, each under a name (see The saved images below). The page boots the one that its URL names, `?slot=` of a saved image or `?fresh=` of an image of the site, else the image saved or booted last, else the site's default image; once the VM is ready, the URL names the image that runs (`history.replaceState`), and so does the link to the world page. No other parameter of the URL is read: they only choose among the user's own images and the site's, so a link is safe to share. - Save evaluates `Smalltalk snapshot: true andQuit: false`. Whenever the image is saved, by Save or by the image itself, the worker keeps the image, its `.changes` and the files of its directory in the browser's IndexedDB: in the saved image that it booted, or in a new one, named after the image ("Pharo 15"), for an image started fresh or opened; the notice then says "The image is saved in this browser as ''", with Rename for a new one. The next visit boots it ("Started the image saved in this browser"). The `.changes` is stored again when it changed: about 0.5 s after the VM goes idle, at the latest 5 s after it was last stored, and when the VM exits. When the page goes away, the worker tries to store it too, but may not have the time. - Save as asks for a name and saves into a new saved image of that name, which the VM saves into from then on; the image it came from stays as it was saved. - The browser asks before the page goes while a save is being stored, or when code was changed since the image was last stored. - Download gives the image and its `.changes` as last saved. - Images opens the Images dialog: the images of the site to start fresh, each with its build and the size of its download, and the saved images, newest first, with their Pharo version, when they were saved, their size and the files they keep, to open, rename or delete, or to open on the world page; Open a file…; what the site uses of the browser's storage, and a button that asks the browser to keep the images from its cleanup (`navigator.storage.persist()`), until it does; and Delete all. Choosing an image restarts the VM on it, and the Notebook offers to restart on it too. - Open, or a drop of files on the page, starts an image of your own: a Pharo `.zip` as files.pharo.org gives them, even with the files in a directory, or an `.image` with its `.changes`, and its `.sources` when it has one of its own. - File in, or a drop of Smalltalk `.st` files alone on the page, writes them into `/pharo` and files them in with a line of the REPL (`CodeImporter evaluateFileNamed:`), which the terminal shows, as if typed: it waits for an evaluation that runs, and Stop interrupts it as any other; a control character in a name becomes `_`. The page unpacks the zip in the browser, with its progress, while the VM goes on, and checks the size and CRC-32 of every entry and that the image is a 64-bit Spur image. It refuses anything else, and says why (no image or several, not a whole zip, a damaged or encrypted entry, a method other than deflate, a split zip, a 32-bit image), and the session goes on as it was. The `.changes` and the `.sources` are those of the image's base name, or else the only ones of their kind. The image gets its own `.sources` when it has one, else the one of its name that the site serves (see Pharo versions). It becomes a new saved image, named after its file, once it has started, and never replaces one: one that crashes or quits before is not kept, and Restart boots the image of before again. Until it is kept, the URL and Images name no saved image as the one that runs. The page asks first only when something was evaluated since the last save ("What you did since the last save is lost"). - Delete deletes the saved image that runs, after asking ("Delete '' from this browser, and start again?"), and starts again on the saved image used last of those left, else on the site's default image; the other saved images stay. When the saved image itself does not start, the notice offers Delete next to Restart. - Restart starts the VM again from the image that runs, as last saved. Ctrl+D on an empty line ends the session (after asking), and Ctrl+L clears the terminal. - When two tabs run the same saved image, the last one to save wins. The other tab says that its image was replaced ("Another tab has replaced the image saved in this browser ('')"), and stops storing its `.changes` until it saves again; tabs on different saved images never interfere. - Settings, shown when the VM has the smart-HTTP transport of libgit2 (`WASM_LIBGIT2=ON`), holds the CORS proxy of git's requests: the site's default, none, or a proxy of your own (see libgit2). A note at the start says where git's requests go ("Git requests go through the proxy at (this site's default)"). - For screen readers, a live region next to the status pill says how the VM goes: loading, starting, ready, and how it ended. While the Notebook tab shows, Esc, Ctrl+C and Ctrl+L do nothing to the Console, and a drop of files goes to the Notebook. On a site built with the world, the Console links to the world page when the image can open it, and offers to save first when something was evaluated since the last save. An image that cannot yet (a stock image opened from disk, or one saved with an older version of OSWindow-Web) gets the offer "Prepare for the world", which files in the OSWindow-Web package (`CodeImporter evaluateFileNamed: '/pharo/st/web-bootstrap.st'`) and saves. ## The saved images The Console and the world page keep the images of the user in the browser's IndexedDB, in the database of the site's directory, each in a slot of `vm-storage.js` (see The web pages): its image, its `.changes`, its `.sources` when it is not one that the site serves, the files of its directory, and its name. Both pages, and the Notebook, see the same saved images, and the Images dialog of either page lists them. - A slot is made by the first save of an image started fresh (named after its label, "Pharo 14"), by an image opened from disk once it has started (named after its file), and by Save as (the name given); a name already taken gets " (2)", " (3)", and so on. Rename changes only the name; Delete removes the slot, and Delete all every slot. Deleting the image that runs restarts the page (Delete all says so when it is among them); an image started fresh or opened, not saved yet, goes on after Delete all, which deletes nothing of it. - A save stores, with the image, the files of `/pharo` that are neither the image's nor the site's: Iceberg's clones in `pharo-local` with their `.git`, the package cache, the Epicea sessions, `PharoDebug.log` and anything the user wrote there, as they were when the image saved, with their directories (empty ones too), symbolic links, modes and times. A boot of the slot writes them back before the image starts, so a clone made before a save is there after a reload, consistent with the image's Iceberg. The VM's `HOME` is among them, `/pharo/home`, which the worker makes at every boot: the global `.gitconfig` (see libgit2) and anything else the image keeps in its home are kept with the image, in each slot its own; a fresh image, an opened one and a slot saved before there was a home start with an empty one. (A slot saved before, whose files have a file or a link named `home` there, keeps it, and its VM the `HOME` of Emscripten, `/home/web_user`, which no save keeps.) At most 256 MiB and 50000 files are kept: beyond that, the save keeps the image but not its files, the slot keeps the files it had, and the notice says that the clones made since its last save would be lost on a reload. So does a save whose files the browser has no room for (they are written in the transaction of the image): the image is then saved again without them, so that the files never cost the image. - Each slot records which Pharo it is, from the name of its `.sources` (or, for a slot saved before there were several, from the bytes of the image, at its next boot: until then the list says "Pharo ?"), where it came from (a fresh image of the site, an opened file, a Save as), when it was made and saved, and its sizes. - The image saved before there were several (one per browser and site directory) is the slot `main`, as it was, named "Saved image": nothing is moved or copied, and the database keeps its version. No new slot is ever `main`: a page of the build of before, still open in a tab, saves into `main` without looking, and would overwrite an image made there since. - A saved image takes about 60 MB, more with its clones, uncompressed. The Images dialog says what the site uses of the browser's storage, and offers to ask the browser to keep it from its cleanup; a save that exceeds the quota keeps nothing and says so ("Saved, but ..."). ## The Notebook The Notebook tab of the Console page holds cells of Smalltalk, which a second VM evaluates one at a time, and of Markdown, which the page renders (`notebook-lib.js`); the VM is the kernel, `st/web-notebook.st`, in a worker of its own, driven by `nb-kernel.js`. The first visit to the tab starts it. It boots the image of the Console, at each start of the kernel: the saved image that the Console runs, read-only, with its files, or the image of the site that it runs fresh, and never saves: a snapshot in a cell ends ok and leaves the saved image as it was. When the Console saves, deletes or replaces its image, or starts another one, a notice offers to Restart the kernel on it. Its git proxy is the Console's, and follows Settings. In a page opened from `file://`, or in a browser without the memory of the build, the tab starts no worker and says why. - A code cell is parsed whole before anything runs: a syntax error runs nothing, and says its line and column. Then its statements are compiled and run one at a time, as one DoIt, so that a class can be defined and used in the same cell. The value of the last statement is shown, as its printString (cut at 64 K characters, and shown 30 lines high until it is opened); `^` ends the cell with its value. - A variable that a cell assigns, and nobody declared, is a variable of the notebook (a WorkspaceVariable), which the next cells see; the temporaries of a cell are its own. `Notebook variables` answers their names, and `Notebook forgetVariables` forgets them. - An error says its text, the statement, line and column of the cell, and the chain of frames, outermost first; `Notebook lastError` answers it. - Shift+Enter runs a cell and selects the next one, Ctrl+Enter runs it in place, and Alt+Enter runs it and inserts a cell below. Esc is command mode, where `?` lists every shortcut: a and b insert a cell, d d deletes it (z undoes it), m and y make it text or code, Alt+Up and Down move it, o hides its output, i i interrupts and 0 0 restarts the kernel. Cells run in their order of request, each once the cell before is done, and their counts say in which order they ran. - Run all runs every code cell from the top, and stops at the first one that does not end ok: an error, a syntax error or a Stop cancels the cells queued after it (a cell whose process terminates itself does not). The example notebook (More, Load example), 14 cells, stops at its error on purpose. The class `Notebook` shows rich output in the cell of the process that calls it: - `Notebook show: anObject` shows a display, a String as text, a Form, a Morph, a Roassal canvas or shape, or an AthensCairoSurface as an image, and anything else as its printString; `show:id:` replaces a display shown with the same id in the same cell, as a progress bar does. - `Notebook text:`, `html:`, `svg:`, `markdown:`, `table:` and `table:header:` make displays; a table's Strings, numbers and Characters are escaped. - `Notebook cairo: [ :canvas | ... ]` gives the block a new Roassal canvas to fill, and makes an SVG display of it, which cairo's SVG surface draws (`AthensCairoSVGSurface`, with `RSAthensRenderer`): the shapes fitted as `image:` fits them, on the canvas's color, and the labels as the paths of their glyphs, so it looks as the image draws it, without the page's fonts; the block may answer another canvas to draw instead, or a shape to add. It is vector output, and small: a tree of 15 nodes with labels is 26 KB of SVG, drawn in about 0.25 s. - `Notebook image:` makes the display of what an object looks like, as the raw pixels of a Form, which the page turns into a PNG (the VM has no ZipPlugin to encode one); `Notebook png:` takes the bytes of a PNG. - `Notebook clearOutput` clears the output of the cell so far. - A cell whose value is a Form, a Morph, a Roassal canvas or shape, or an Athens cairo surface shows it as an image. Output goes to the cell that produced it: - What a cell writes on the Transcript, stdout and stderr lies between its start and its end: the kernel flushes them before each of its events, which are JSON lines on an Emscripten character device of their own, `/dev/nbevents`, with binary attachments. - What a process that an earlier cell forked shows on the Transcript, or displays, goes to that cell, while it shows that run; an error that nothing handles in such a process is reported to that cell, and the process is terminated. - While a cell runs, its output keeps the first and the last 512 KB of its streams and its first and last 500 outputs, with a note of what was left out; a display over 4 MB comes with its size only, and an image has at most 16 MiB of pixel data (4 Mi pixels of 32 bits) and 16384 pixels a side. Stop (the button, i i, or Ctrl+C with nothing selected anywhere on the tab) signals the semaphore that the kernel registered with the WebHostPlugin: the cell that runs is terminated, and its ensure: blocks run, or the cell about to start does not run; the cells queued after it are cancelled. A Stop that comes while no cell runs does nothing. After a Stop, the page's watchdog gives the cell 3 s to end, more while it writes output, up to 10 s; after that the worker is terminated, the cell ends "killed", and a fresh kernel starts, without the variables. Stop ends neither the processes that cells forked nor the system's: More, Stop background processes, terminates those that cells forked at the priority of the cells or below, but not the processes of the system that a cell made start (the Morphic UI process, the event loops of the drivers). Restart starts a new kernel: the counts start at 1 again, and what ran before is marked stale; More, Restart & run all, then runs every cell. More also has Add text cell, Clear all outputs (of the cells not running or queued), New notebook (which asks before it replaces one that has content), and Theme, which goes from the system's to light and dark, for the whole page, as the Console's button does. The notebook is saved in this browser's localStorage, 0.6 s after a change and when the page is hidden or goes, under a key of the site's directory (`pharo-wasm.notebook:`), with the outputs of each cell capped at 64 KB (and without outputs when the whole would pass 2 MB); Ctrl+S saves at once. When another tab of the site saves its notebook, a notice offers "Load theirs" or "Keep mine". More, Export, gives it as `.st` (the chunk format that a fileIn reads, each cell a chunk marked by a `"%%"` comment, a Markdown cell's text in the comment), as `.json` (with the outputs) or as `.html` (below); Import takes a `.st` or a `.json` of at most 5 MB, 5000 cells and 1000 outputs per cell, and asks before it replaces a notebook that has content. A `.st` file of another kind is read as Pharo's chunk reader reads it: its DoIts are the cells, and its method definitions and class comments are left out, with a warning. More, Upload files, or a drop of files on the tab, writes them into the kernel's `/pharo`, now and into every new kernel of the page (the page keeps them in its memory, not across a reload): `'name.st' asFileReference fileIn` then files one in. File in, on the toolbar, or a drop on the tab, uploads the files the same way and files the `.st` ones in at once: the kernel's `{"op":"file-in"}` request, which runs as a cell does, in a process of its own (Stop interrupts it, and a cell run meanwhile waits for it), and answers a `filed-in` event with the error, if any; the toast says what was filed in and what was only uploaded. The page never inserts markup as given. The HTML and SVG of `html:` and `svg:`, and those of imported notebooks, are parsed inertly (DOMParser) and rebuilt element by element from an allowlist: the ids and the `url(#...)` references are rewritten, the styles filtered, links kept only to http, https, mailto and the fragments, images only as data: URLs of PNG, JPEG, GIF or WebP, filters charged against a budget of their painting cost, HTML elements inside SVG dropped, and markup nested deeper than the browsers' parsers take in linear time refused. Markdown, of the text cells and of `markdown:`, is built from elements and text nodes by a renderer of the page in time linear in its source. The Content-Security-Policy of `index.html` is a second line of defence behind them: `default-src 'self'; script-src 'self' 'wasm-unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; connect-src 'self'; worker-src 'self'; object-src 'none'; base-uri 'none'; form-action 'none'; frame-src 'none'`. The kernel is not a sandbox: a cell can call it and write events of its own, which the page checks like any other. Code cells are colored as they are typed, by the highlighter of `notebook-st.js`, which reads as Pharo's scanner does, in linear time, on any text, unterminated too. Its kinds are comment, string, character, number (radixes, exponents, scaled decimals), quote (symbols, and literal and byte arrays and what is inside them), constant (`nil`, `true`, `false`), special (`self`, `super`, `thisContext`, `^`, `:=`), keyword, paren (brackets, period, cascade, the bars of temporaries, the `<>` of pragmas) and global: | Kind | Light | Dark | |---|---|---| | comment | `#565d66` | `#9aa3ae` | | string, character | `#336316` | `#a8cc7a` | | number | `#962a5a` | `#f28db2` | | quote | `#255e86` | `#82b8e8` | | constant | `#7239ad` | `#c4a5ff` | | special | `#984208` | `#f5a524` | | keyword | `#0b645c` | `#5ccfbf` | | paren | `#49515a` | `#a3abb5` | | global | `#3445b4` | `#a3adff` | Each color has a contrast of at least 4.5:1 on the editor, on the selection tint and on the line of an error: the lowest measured is 4.79:1, of a comment on the line of an error in the light theme. The highlighter agrees with the image's own scanner on every token of every method of the images: 3,675,607 tokens of 138,191 methods on Pharo 12 (RBScanner), and 3,698,571 of 140,181 on Pharo 15 (OCScanner), in about 2 s each; lane 61 checks 5000 methods by default (`NB_SCAN_METHODS`). The colors are an inert overlay behind the editor's textarea, in its box and font, painted when a cell comes near the visible part of the notebook and repainted line by line on an edit, for cells of up to 32 KB; an input method composes in the textarea. Markdown cells are not colored, but their fenced code is when the fence names no language, or `st`, `smalltalk` or `pharo`, up to 32 KB per render. A cell of 32 KB (885 lines) painted first in 32 ms (Chromium) and 37 ms (Firefox), and an edit repainted it in 16 ms and 13 ms on average; the slowest of the adversarial 500 KB inputs of the spec took 157 ms and 139 ms. More, Export .html, downloads one page of what the notebook shows: the Markdown cells rendered (also those being edited, their Smalltalk fences colored), the code cells with their count, status bar and colored source (also cells not yet scrolled into view, up to 32 KB a cell and 1 MB in all, the rest plain), and their outputs as shown, without buttons: streams, values in full, errors with their bar and call history, displays and images as data: URLs, and hidden outputs in a closed `
`. Its title is the first heading, else the notebook's. It has no script and loads nothing: its Content-Security-Policy allows inline style and data: images and fonts only. It is built in a document of its own from trees that the page already trusts, serialized and read back before it is saved: an output whose markup would parse into another tree is left out with a note. It follows the light or dark preference, prints light, and fits 320 px. The example notebook, run, exported in 146 ms (Chromium) and 199 ms (Firefox) to 28 KB; 40 cells of 31 KB each, 1.2 MB of sources, in 1.8 s and 1.9 s to 10 MB. The kernel costs a second VM: on the machine the port was made on it was ready 1.9 s (Chromium) and 2.0 s (Firefox) after the first visit to the tab, about 0.9 s after the Console's prompt (1.0 s and 1.1 s after the page was opened); it reads 19.8 MB of the site's 42.8 MB `.sources` at every boot, which the REPL does not. In node, the kernel booted in 1.9 s against 0.8 s for the REPL on Pharo 12, and in 2.2 s against 1.0 s on Pharo 15 (27.7 MB of its 46.1 MB `.sources` read). Its worker has a linear memory of its own, as large as the Console's: 675 MiB once Pharo 12 has booted, 683 MiB for Pharo 15. It fetches the `.sources` once per boot, and the image only when the Console runs no saved image. ## The world page `world.html` boots an image with `--interactive`, and the world draws itself in the canvas. The image is the one its URL names (`?slot=` of a saved image, `?fresh=` of an image of the site, as in the Console), else the image saved or booted last, else the site's default image. On a first visit (nothing saved in this browser) to a site of several images, no VM starts: the Images dialog opens as the launcher, "Choose a Pharo image", with a button for each image of the site, its build and the size of its download, and the default one has the focus, so that Enter starts it; Escape does not close it. A site of one image starts it at once, as before. The canvas follows the size of the page, and the world has its size in CSS pixels. On a high-density display the world draws in device pixels: the page reports the canvas in device pixels too, from which Pharo's world renderer takes `devicePixelRatio` for its `canvasScaleFactor`, as the desktop VMs do on a Retina display, and draws a Form that much larger, which the page shows one pixel per device pixel; it follows the ratio when the window moves to another screen, or with the zoom of the page. The world's text is drawn with FreeType, in the stock fonts of the image (Source Sans Pro and Source Code Pro): text beyond Latin-1, emphasis such as the strike-out of deprecated classes, and exact font sizes are drawn as natively. With cairo, Athens draws too: Roassal, the Canvas view of the Inspector, Epicea's Code Changes (whose graph Hiedra draws) and the color picker of the Settings. - The mouse buttons, the wheel and the keyboard reach the world as the events of a desktop window would (numbered as SDL2, the driver of the desktop VMs, numbers them). Text typed with Shift, Option (macOS) or AltGr is text. On macOS Cmd is sent as Ctrl, Pharo's shortcut key. On Windows, Ctrl+Alt types the AltGr character of the key, when the key has one; elsewhere, Ctrl+Alt is a shortcut. - What the world copies (Ctrl/Cmd+C or X) goes to the system clipboard, and Ctrl/Cmd+V pastes the system clipboard, through the browser's paste event. When the browser refuses a copy to the system clipboard, Ctrl/Cmd+V pastes the world's own copy instead, until the page loses the focus to another window or something else on the page is copied. - The browser keeps some keys for itself, such as Ctrl/Cmd+W, T and N. Pharo's Ctrl+O, Ctrl+W would close the tab, so the page asks before it goes while the world has taken input since the last save. - F6 (or Shift+F6) moves the keyboard from the world to the toolbar, and Tab goes back into the world. - Stop (or Alt+. in the world) interrupts the busy process and opens a debugger on it. That is the UI process whenever it is ready to run, so that a loop evaluated in a Playground is interrupted even when another process of its priority is ready too: UserInterruptHandler alone may pick that one, and then no debugger opens, since the busy UI process would draw it. When the worker then says nothing for 3 s, it is replaced, as in the Console. - Save asks the image to save itself, and the worker keeps it as the Console does, with the files of its directory (Iceberg's clones): the next visit, here or in the Console, boots it. Save as asks for a name, and saves into a new saved image of that name. Download gives the image and its `.changes`. - Images, as in the Console: start an image of the site fresh, or a saved one, rename or delete one, or open it in the Console; the page asks first when the world has input that is not saved. Deleting the saved image that runs starts the saved image used last of those left, else the site's default image (the launcher, on a site of several images). - The canvas shows one OSWindow at a time: the newest one that has an event handler, such as the window of the Emergency Debugger of Pharo 12, takes the canvas and the input, and the world comes back when it closes. - The status pill says Busy while the VM has not slept for a second, or while it does not answer at all (a long primitive). - Open, or a drop of files on the page, starts an image of your own, as in the Console, after asking when the world has input that is not saved ("Open as a new image?"); it becomes a new saved image once it has started. - File in, or a drop of `.st` files alone, writes them into `/pharo`, lists their paths in a file of its own (`/pharo/.web-file-in-`), and sends the world a file-in request (type 12, OSWindow-Web 9): `OSWebDriver>>#fileInPending` takes every list at once, on its event loop, then files the paths in on a process of its own, whose errors open in the world. An image without the OSWindow-Web package, a stock image say, or with an older version of it (the site has version 8), is prepared first: the worker boots it with the REPL, files in the package and saves, the pill says Preparing meanwhile, and then the world boots from the saved image. The stock image of the build was unpacked, prepared and painted in 4.3 s (Chromium) and 5.0 s (Firefox) after it was chosen. - The Console button opens the Console page, on the image that runs. When the world has not drawn 30 s after the VM started, or the VM ended before it drew, the notice points to the Console, which can run the image or delete it. - Settings, as in the Console, when the VM has the smart-HTTP transport of libgit2: the git proxy (see libgit2), which this browser keeps for every page. The first git request of a VM through the site's public default says once that the proxy sees what is cloned, and a request that failed says why; both notices have a Settings button. The first frame came 1.4 s (Chromium) to 2.0 s (Firefox) after the page was first opened on localhost, nothing cached, and a key took 40 to 57 ms (median) to reach the canvas. ## Other targets `make wasm-check-browser` runs the Playwright specs of the pages (see Tests). Without Playwright (`PLAYWRIGHT_MODULE`, or a `playwright` package that node finds from `tests/wasm/lib`) it fails, and says so. It skips, and says why, `tests/wasm/world.spec.mjs` on a build without the world image, `ffi.spec.mjs` without the FFI, and `git.spec.mjs` without libgit2: `web/manifest.json` says what the build has. `make wasm-clean` removes the WebAssembly build (`build-wasm/cmake`, `node`, `web` and `tests-run`), and keeps the generated sources, the images and the downloads, the image zip and the archives of the libraries. `make wasm-distclean` also removes `build-wasm/host`, `build-wasm/image` and the recorded settings, and, with `WASM_CLEAN_DOWNLOADS=1`, `build-wasm/downloads`. GNU make reads `GNUmakefile` before `Makefile`. In an in-source CMake build, the default goal and every goal that `GNUmakefile` does not define go to that build's `Makefile`, as before. The WebAssembly tree can also be configured with CMake alone, from sources generated already: emcmake cmake -C cmake/Emscripten.cache.cmake -S . -B build-wasm-cmake -DGENERATED_SOURCE_DIR=$PWD/build-wasm/host cmake --build build-wasm-cmake It stages `node/` and `web/` in its build directory, or in `WASM_STAGE_DIR`, and downloads the archives of the libraries into `downloads/` there. Its cache variables are the settings `WASM_SJLJ`, `WASM_WEB_MEMORY64`, `WASM_STACK_SIZE`, `WASM_INITIAL_MEMORY`, `WASM_MAXIMUM_MEMORY`, `WASM_OLD_SPACE_BASE`, `WASM_SLICE_MS`, `WASM_WORLD`, `WASM_FFI`, `WASM_FREETYPE`, `WASM_CAIRO`, `WASM_CAIRO_PDF`, `WASM_LIBGIT2`, `WASM_LIBGIT2_HTTP`, `WASM_IMAGE_ZIP`, `WASM_IMAGE_ID`, `WASM_EXTRA_IMAGES` and `WASM_EXTRA_SOURCES` (CMake lists there, separated by `;`, of absolute paths), `WASM_GIT_PROXY`, `WASM_DEPS_DIR` and `WASM_HOST_PHARO` below, `CMAKE_BUILD_TYPE` (`Debug` for `WASM_DEBUG=1`), `WASM_STAGE_DIR`, and `NODE_JS_EXECUTABLE`, the node that stages `web/`. `WASM_HOST_PHARO` has no default there, so the world image is prepared only when it is given. Two more are options for developers of the FFI, which `make wasm` does not give: `WASM_FFI_LIBC_ALL=ON` lists every function of the C library in the registry, not only those the images call (about 100 KB more of gzipped wasm), and `WASM_FFI_TEST_LIBRARY=ON` links the library of the FFI tests into the web VM too, as it always is into the node VM. ## Settings Settings are given on the `make` command line and recorded in the build directory, in `config.make` (and the two VMMaker settings in `config-host.make`): changing one configures the build again, and repeating it changes nothing. They are recorded, not remembered: every `make wasm...` command for a build directory must repeat them, since a setting left out goes back to its default. To keep two builds, give each its own `WASM_BUILDDIR`: make wasm WASM_BUILDDIR=build-wasm-debug WASM_DEBUG=1 - `WASM_BUILDDIR`: the build directory (default `build-wasm`; not recorded). It must be one path, not empty, and neither it nor what its symbolic links resolve to may contain a blank or any of ``' " ` \ $ ; & | < > ( ) { } [ ] * ? % : #``; a relative one is taken from the current directory, whose path must then hold no blank either. It must not be `/`, `$HOME`, the source tree, a directory holding it, or a directory holding a `CMakeLists.txt` or a `.git`. And it must be new or empty, or marked by an earlier `make wasm` (with `.make-wasm`, or the `config.make` of an older version), or a `build-wasm*` directory of the source tree, where the inputs of an offline build can be put first. An empty `WASM_BUILDDIR=`, as an unset shell variable gives, is refused rather than taken for the default: leave the setting out instead. - `WASM_DEBUG`: `1` for a Debug build (default `0`): `-O0 -g`, the C asserts of the VM, Emscripten's `-sASSERTIONS=2`, and a checked stack (`-sSTACK_OVERFLOW_CHECK=2`). The libraries keep their optimisation, with `-g`. The VM is many times slower with its asserts, so the time limits of the lanes fail by design (see Troubleshooting). - `WASM_SJLJ`: how the VM's setjmp and longjmp are compiled (default `wasm`). `wasm` uses WebAssembly exception handling with exnref, `wasm-legacy` its legacy instructions (try, delegate and rethrow, for engines without exnref; Firefox logs them as deprecated), and `emscripten` calls out to JavaScript. The libraries are compiled in the same mode, since every object of a link must share it. - `WASM_WEB_MEMORY64`: the memory of the web VM (default `2`). `2` is a 32-bit memory with 64-bit pointers, which is 15 to 18% faster and runs in engines without memory64; `1` is memory64, as in the node VM, and the Console says so in a browser without it. The node VM always has memory64, so that a stray pointer traps rather than wrapping around. - `WASM_STACK_SIZE`: the C stack (default `8MB`); FreeType's autofitter needs more than Emscripten's default of 64 KB. - `WASM_INITIAL_MEMORY`, `WASM_MAXIMUM_MEMORY`: the linear memory, its initial size (default `32MB`) and its maximum (default `4GB`); it grows between them. - `WASM_OLD_SPACE_BASE`: where old space begins (default `0x20000000`, 512 MiB), a power of two above new space. Old space stays below twice that address, so it can grow to that size, 512 MiB by default, and twice it must fit in `WASM_MAXIMUM_MEMORY`. Past that window an allocation fails with an `OutOfMemory`, and an image that does not fit in it does not start. The environment variable `PHARO_WASM_OLD_SPACE_BASE` overrides it when the VM starts, for example `0x40000000` for 1 GiB. Changing the setting recompiles the VM core. - `WASM_SLICE_MS`: the length of a slice of the VM in milliseconds (default `20`); `PHARO_WASM_SLICE_MS` overrides it when the VM starts. - `WASM_WORLD`: `ON` (the default) prepares the world image with `WASM_HOST_PHARO`, on a build that has the world, and stages it in `web/`; `OFF` stages the stock image. - `WASM_FFI`: `ON` (the default) builds the FFI, libffi and the registry of the libraries; `OFF` builds the VMs without any of them, on the image's NullFFIBackend, and so without any library. - `WASM_FREETYPE`: `ON` (the default) builds FreeType, and the world image keeps the FreeType fonts of the image; `OFF` gives the world image bitmap fonts instead. - `WASM_CAIRO`: `AUTO` (the default) builds cairo when FreeType is built, `ON` stops the configuration without FreeType, and `OFF` leaves it out. - `WASM_CAIRO_PDF`: `ON` adds the PDF, PostScript and script surfaces of cairo, for AthensCairoPDFSurface (default `OFF`): 68 KB more of gzipped wasm, and cairo's configuration with zlib. - `WASM_LIBGIT2`: `ON` builds libgit2, for Iceberg (default `OFF`): 258.6 KiB more of gzipped wasm. - `WASM_LIBGIT2_HTTP`: with libgit2, `ON` (the default) gives it the smart-HTTP transport of `src/emscripten/gitSupport.c`; `OFF` leaves libgit2 with local repositories only. - There is no `WASM_SDL2` any more: the VMs have no SDL2, nor the site the page `sdl.html` that drew the world through the image's own OSSDL2Driver (the world page draws it through OSWebDriver). `make wasm WASM_SDL2=ON` (or with `WASM_SDL2` in the environment) is refused, and so is a configuration with CMake alone that gives it, or whose cache has it true; `WASM_SDL2=OFF` changes nothing, and `make wasm` drops it from the cache of a build directory configured with it, rebuilding without SDL2. - `WASM_DEPS_DIR`: a directory holding the pinned archives of the libraries, by their file names, instead of downloading them (see The libraries). - `WASM_HOST_PHARO`: a native Pharo VM. By default, it is the VM that runs VMMaker: `WASM_VMMAKER_VM`, or the one that the host tree downloads, `build-wasm/host/build/vmmaker/vm/pharo`. With `WASM_GENERATED`, which skips the host tree, that is only `WASM_VMMAKER_VM`, or a VM left there by an earlier build of the same directory: give it otherwise. The lanes use it to load an image that the WebAssembly VM saved (S14b, which is skipped without it), to compare FreeType, cairo and libgit2 with the native VM (a note, never a failure), and to check the highlighter of the Notebook against the image's scanner. It also prepares the world image (`packaging/emscripten/st/prepare-web-image.st`, in about a second), and lanes 80 and 82 run `tests/wasm/st/osweb-native.st` and `tests/wasm/st/osweb-windows.st` with it; without it, `web/` gets the stock image. - `WASM_IMAGE_ZIP`: a local image zip, instead of downloading the pinned Pharo 12 one into `build-wasm/downloads`: a copy of that one for an offline build, or a Pharo 13, 14 or 15 one (from https://files.pharo.org/image/150/ and its siblings), which the site then runs by default. Any zip holding one image, its `.changes` and a `.sources` is accepted; one that is not the pinned image gets a note with its SHA256. Under the name that files.pharo.org gives it (`...build..sha...`), the pages also show its build. - `WASM_IMAGE_ID`: the id of the default image in the catalog of the site, which `?fresh=` names (default: `p` of the name of its `.sources`, `p15` for Pharo 15, else `default`). An id is 1 to 16 lower-case letters, digits and dashes, not a dash first. - `WASM_EXTRA_IMAGES`: more images for the site, as words `=` separated by blanks (default none), for example `p13=Pharo13.zip p14=Pharo14.zip`; see Building. Each id is an id as above, given once, not the default's, and not `sources`; each zip must exist, and is accepted as given, without a pinned SHA256 (the workflow checks those it downloads). Paths with blanks are not supported. Each image is prepared for the world with `WASM_HOST_PHARO` as the default one is; two images whose `.sources` have the same name must have the same file. - `WASM_EXTRA_SOURCES`: `.sources` files, separated by blanks, that the site serves under `web/image/sources/` for the images saved in a browser from an image that it served before (default none); one that an image of the site has already is not staged twice, and one of the name of an image's `.sources` but another content is refused. - `WASM_GIT_PROXY`: the site's default CORS proxy of git's HTTP requests, which the pages use unless the user chose another or none in Settings (default `https://cors.isomorphic-git.org/`, the public proxy of isomorphic-git); `WASM_GIT_PROXY=` gives the site none. It must be an http:// or https:// URL without a user name, password or `#fragment`; a bare origin gets its `/`, and the manifest has it as the browser writes it. make checks it before it builds anything, also with `stage.mjs --check-git-proxy` when node runs, which refuses a port or a host that is none. - `WASM_GENERATED`: a directory holding `generated/64`, the generated StackVM sources, which skips the host tree. For example, `make wasm WASM_BUILDDIR=build-wasm-2 WASM_GENERATED=build-wasm/host WASM_HOST_PHARO=build-wasm/host/build/vmmaker/vm/pharo` builds a second directory with the sources and the native VM of the first. The sources must come from this tree's `smalltalksrc`: the configuration refuses sources without the Emscripten hooks. - `WASM_VMMAKER_IMAGE`, `WASM_VMMAKER_VM`: a VMMaker image, copied with its directory into `build-wasm/host/vmmaker-image` instead of bootstrapping one, and the Pharo VM that runs it (by default the one the host tree downloads, PharoVM 10.3.1 on Linux x86_64). The copy is refreshed from `smalltalksrc` the first time it is used, which takes about 12 s. These two are recorded in `config-host.make`: changing them configures the host tree again, and a new `WASM_VMMAKER_VM`, through the default `WASM_HOST_PHARO`, the WebAssembly tree too, unless `WASM_HOST_PHARO` is given. - `WASM_PORT`: the port of `make wasm-serve` (default `8080`; not recorded). - `WASM_JOBS`: the jobs of the CMake builds when `make` itself runs without `-j` (default: the number of processors; not recorded); with `-jN`, they share make's job slots. - `WASM_CLEAN_DOWNLOADS`: `1` makes `make wasm-distclean` remove `build-wasm/downloads` too (default `0`; not recorded). - `CMAKE`, `EMCMAKE`, `NODE`: the cmake, emcmake (default: from `$EMSDK`, else from the PATH) and node of the build, found on the PATH by the shell, and recorded. (make's own lookup would stop at the directories named `cmake` and `node` that `emsdk_env.sh` puts on the PATH.) The other `WASM_` names of `GNUmakefile` (`WASM_GOALS`, `WASM_REQUESTED`, `WASM_CMAKE_FLAGS`, `WASM_MARK`, `WASM_SPECIAL_CHARACTERS`, `WASM_BUILDDIR_CHECKS`, `WASM_BUILDDIR_CHECKED` and `WASM_CHECK_BUILDDIR`) are internal, not settings, and `WASM_DIR` is a variable of the lanes (see Tests). When the VM runs: - `NODE`, `NODE_OPTIONS_WASM`: the node that `build-wasm/node/pharo` runs (default: `node` on the PATH), and its options, for example `--stack-size=900` or `--liftoff-only`. - `PHARO_WASM_SLICE_MS`, `PHARO_WASM_OLD_SPACE_BASE`: override `WASM_SLICE_MS` and `WASM_OLD_SPACE_BASE`. An old-space base that is not a power of two above new space, or whose window does not fit in the memory, stops the VM with a message. - `PHARO_WASM_STATS`: `1` makes the node VM print its slices on stderr when it ends: `[pharo-wasm] slices=N busy=N sleeping=N`. - `PHARO_WASM_FFI_HIDE`, `PHARO_WASM_FFI_TRACE`, `PHARO_WASM_FFI_ADAPT`: the switches of the FFI (see The FFI). - `PHARO_WASM_GIT_PROXY`: the CORS proxy of libgit2's HTTP requests in the node VM (see libgit2). - `--maxFramesToLog`: the Smalltalk frames printed per process when the VM reports an error, 1000 by default on Emscripten (elsewhere, every frame); 0 prints every frame. A crash deep in a recursion would otherwise print hundreds of megabytes, which takes the VM minutes. The default is `PHARO_DEFAULT_MAX_FRAMES_TO_LOG`, in `include/pharovm/emscripten/sqPlatformSpecific.h`. The environment of the web VM is its defaults and what its worker sets, never the host's: the variables above are for the node VM. ## How it works ### Slices A browser worker, and node's event loop, need their thread back for input, timers and output, and the VM cannot run on a thread of its own: threads need SharedArrayBuffer, which needs cross-origin isolation, and JSPI is not available everywhere. So the VM runs in slices and returns to its host between them (`src/emscripten/emscriptenMain.c`). `main()` only gets the VM ready and returns with the runtime alive. Every slice is then a call of `vm_resume()`: the first one starts the VM, and the later ones call `interpret()` again, which carries on from the state the interpreter saved in its globals. A slice ends in `ioReturnToHostIfRequested()`, the last statement of `ioSynchronousCheckForEvents()` (`src/common/sqTicker.c`). There the interpreter has nothing left on the C stack but returns, so the driver can `longjmp` back to `vm_resume()`; it forces an interrupt check first, so that the next slice runs the rest of the event check. So the call must stay the last statement of the event check, and the callers of the event check must only return to the interpreter's loop: the lanes that run with 1 ms slices, and a check of Debug builds that every slice starts at the same depth of the C stack (`STACK DRIFT`), guard this. A slice never ends inside an FFI callback, whose interpreter runs on the C (and JavaScript) frames of the callout (see The FFI). `vm_resume()` answers BUSY (the slice was over: resume at once), SLEEPING (the image is idle: resume after `vm_wakeup_ms()`, or on input) or EXITED. When the VM exits, `exit()` flushes the C streams and unwinds out of `vm_resume()` as an `ExitStatus`, which carries the status. The node VM runs the slices from node's event loop (`packaging/emscripten/node/node-pump.js`), and the pages from their worker (`packaging/emscripten/web/vm-driver.js`). Entering `interpret()` again is also what makes the VM fast. V8 runs a function that is entered once in its baseline code (Liftoff), and uses its optimised code only when the function is called again: 20 loops of `inject:into:` over 100000 elements took 1217 ms in one activation, and 585 ms with the same objects re-entered every slice. ### The heartbeat There is no heartbeat thread either. Instead, `pollHeartbeat()` (the macro `PHARO_POLL_HEARTBEAT()`) counts down before the stack-limit check of every interpreted method activation, full block activation and backward jump: 14 sites in the generated StackVM, so that even a loop without sends polls. Every 2000 polls, `ioHeartbeatPoll()` reads the clock, beats every 2 ms as the thread would, and asks for the end of the slice once it is over (`src/common/heartbeat.c`). The clocks are brought up to date whenever they are read. When the image is idle, `ioRelinquishProcessorForMicroseconds()` delivers the input that is ready and, unless that woke something up, ends the slice as SLEEPING until the next wakeup. An idle image still wakes about every 16 ms, as it does natively: the Morphic UI process of the headless image waits on a Delay of at most 16 ms (`MinCycleLapse`) after each cycle. Slang emits the poll from `StackInterpreter>>pollHeartbeat`, and `include/pharovm/common/sq.h` defines it to nothing on every other platform. So the VM is built without threads: with `-pthread`, `include/pharovm/emscripten/sqPlatformSpecific.h` stops the build with an error. ### Memory Linear memory starts at 0 and grows only at the top, so the fixed addresses of the 64-bit memory map cannot be used. Under `__EMSCRIPTEN__` the Slang memory map puts the stack pages at 64 MiB and new space at 128 MiB. `src/emscripten/memoryEmscripten.c` puts old space at `WASM_OLD_SPACE_BASE` (512 MiB) instead of the 2^40 that the memory map asks for, and keeps its segments below twice that address, so that the VM still tells young objects from old ones by their address bits. A segment that would not fit there is refused, and Spur reports an `OutOfMemory`: "Cannot allocate N bytes of old space at P (...): old space must fit in [base, 2*base) (PHARO_WASM_OLD_SPACE_BASE moves it)". It manages that address space on top of `sbrk()`: the ranges it skips are holes that cost nothing until they are used, freed segments become holes again, and a hole at the top grows with the break. Everything stays below 4 GiB, so the same layout works with both memories. The VM has about 710 MB of linear memory once the image has booted (675 MiB for Pharo 12, 683 MiB for Pharo 15), which V8 on Linux commits only as it is touched. Perm space is not supported: WebAssembly memory cannot reach its address, 2^41. An image with perm space, or `--minPermSpaceSize`, does not start ("Cannot allocate N bytes of perm space at 0x20000000000: WebAssembly memory cannot reach it"), and the primitives that move objects there (90, 91 and 93) fail with `#'insufficient object memory'`. ### Primitives and plugins There is no `dlopen`. The VM core and every plugin are static libraries, and the plugins are built in: FilePlugin, NewFilePlugin, FileAttributesPlugin, FloatArrayPlugin, LargeIntegers, MiscPrimitivePlugin, LocalePlugin, SocketPlugin, BitBltPlugin, B2DPlugin, DSAPrims, JPEGReaderPlugin and JPEGReadWriter2Plugin, and the plugins written for this platform in `src/emscripten/plugins`, such as WebHostPlugin, which Stop uses. UUIDPlugin is built in as well: it uses the `uuid_generate` of Emscripten's JavaScript library (`crypto.getRandomValues`), and Pharo 15 makes its UUIDs only with it. So is SimdJSONPlugin, the JSON reader of `plugins/SimdJSONPlugin` (see `docs/SimdJSON.md`), with `FEATURE_PLUGIN_SIMDJSON` on, as it is by default: its C++ makes em++ link both VMs, with Emscripten's libc++ and libc++abi, and simdjson, which has no kernel for WebAssembly, runs its portable one (`fallback`). `make wasm` has no setting for it: `cmake -DFEATURE_PLUGIN_SIMDJSON=OFF build-wasm/cmake` leaves it out of that build directory. `cmake/plugins.cmake` leaves out UnixOSProcessPlugin (fork) and SurfacePlugin on Emscripten (the image's SDL2 driver blits through it, and the VMs have no SDL2; Athens goes without it, see cairo), and the initial cache SqueakSSL. The 20 primitives of `src/ffi` are in the builtin table of the VM's support code (`vmsupport_exports`, generated by `cmake/emscripten/genSupportTable.cmake`). WebAssembly checks the type of every indirect call. The interpreter calls every named primitive as `void (*)(void)`, while the plugins define many of them as `sqInt f(void)`, and such a call traps. So at build time `cmake/emscripten/genPrimitiveTable.cmake` reads the `_exports` table of the interpreter and of every plugin, and writes a translation unit that includes the unchanged source and adds a `void` trampoline for every primitive (every row with an accessor depth), in a `_primitives` table of the same rows; the named-primitive lookup of `src/common/sqNamedPrims.c` answers from those tables. No source of the interpreter or of a plugin changes, and natively nothing does. Emscripten's own answer, `-sEMULATE_FUNCTION_POINTER_CASTS`, made a VM without the trampolines 3.3 to 5.5 times slower. The links use `-sTABLE_BASE=1024`, so that no function pointer has the value of a quick primitive index (up to 519), which shares the primitive function slot. They also use `-Wl,--fatal-warnings`: wasm-ld links a stub that traps for a direct call whose prototype differs from its definition, and only warns. Lanes 10 and 50 check all this. ### The FFI The VMs have FEATURE_FFI with the same-thread runner of `src/ffi` (no worker threads), on upstream libffi 3.8.0, the release with a WebAssembly port for 64-bit pointers (`src/wasm`), built from its pinned archive by `cmake/emscripten/deps/libffi.cmake`. The image gets its TFFIBackend: callouts through uFFI, variadic ones with `fixedArgumentCount:`, structures by value, callbacks, and `Smalltalk os environment at:put:`. Every object of libffi is compiled once for 64-bit pointers and serves both links, the memory64 node VM and the web VM. The FFI adds 18.1 KiB of gzipped wasm, and 10000 callouts of `strlen` took 302 ms in node. There is no `dlopen`, so the libraries are linked in, and listed in a static registry, generated at build time (`cmake/emscripten/ffiRegistry.cmake`, `include/pharovm/emscripten/ffiRegistry.h`). `src/externalPrimitives.c`, under `__EMSCRIPTEN__`, reduces the name the image gives to a library of the registry: its base name, without a leading `lib`, without what follows its first `.`, and without a version after a `-` (`libc.so.6` is `c`, `libgit2.so.1.4.4` is `git2`, `/x/libfoo-2.0.so.1` is `foo`). Each library is a sorted table of functions, taken by name from a list, `cmake/emscripten/ffi/symbols/.txt`, every name of which must be declared by the library's public headers, or the build stops; its `# unavailable` sections list the names the images bind that the library of the VM does not define. The libraries of the registry (`cmake/emscripten/ffiLibraries.cmake`): | Library | The image's | Functions | Built | |---|---|---|---| | `c`, with the aliases `m`, `dl`, `pthread`, `rt` and `LibC` | `libc.so.6` | 144 | always | | `freetype` | `libfreetype.so.6` (FT2FFILibrary) | 17 | `WASM_FREETYPE` | | `cairo` | `libcairo.so.2` (CairoLibrary) | 120 | `WASM_CAIRO` | | `git2` | `libgit2.so.1.4.4` (LGitLibrary) | 239 | `WASM_LIBGIT2` | | `TestLibrary` | `libTestLibrary.so` | 122 | the node VM (and the web VM with `WASM_FFI_TEST_LIBRARY`) | The lists are what Pharo 12 (build 1599) and Pharo 15 (build 41) bind, test packages included, and, for the C library, some more for the callouts of the tests and of the image's users. `packaging/emscripten/tools/ffi-symbols.st` lists them: run on an image, natively or in the node VM, it prints a line for each function a method binds, with the library of the registry it is bound in and the method (on Pharo 12: 752 rows, 663 functions of 15 libraries). Lane 59 checks with it that the registry has every function of a library it registers. The library finders of the image look for the file of a library before they load it, in the VM's directory: the sites and the node VM have empty placeholders of those files (`libcairo.so.2`, `libgit2.so.1.4.4`), which the worker writes into `/pharo` from `manifest.libraries`, and which the VM never reads. libffi's `ffi_call` is JavaScript: it calls the function from JavaScript, which converts the arguments and the result by the types of the function's own WebAssembly signature. So a declaration that does not match the C function, harmless natively, throws a TypeError, or traps, and would end the VM. `emscriptenFFICall` (`src/emscripten/emscriptenSupport.c`) guards every callout: one that threw fails with `PrimErrFFIException`, and says why on stderr ("FFI callout failed, its declaration does not match function N: ..." or "FFI callout trapped in function N: ..."), and the VM goes on. libffi's own return of a 64-bit integer that comes back as a Number is widened, signed or unsigned, in a copy of its `src/wasm/ffi.c`. The declarations of the images do not always have the integer widths of C, and work natively, where an int passed for a long fills the register: `cairo_pattern_set_extend` takes an int that Pharo declares `ulong`, FreeType's `FT_Error` is an int that Pharo declares `long`, and `git_tree_entrycount` answers a `size_t` that Pharo declares `int`. So the registry also carries the C signature of every function, which `cmake/emscripten/ffiSignatures.mjs` reads from the headers, and `src/emscripten/ffiAdapt.c` adapts a callout whose integer arguments or result differ in width from the function's, as an x86-64 caller would, instead of failing it. It also calls a variadic function with its variable arguments as C takes them, also when the declaration has no `fixedArgumentCount:`; one with a float, or an integer narrower than an int, among its variable arguments fails, and says "FFI callout to a variadic function : declare it with fixedArgumentCount:". The adapter changes nothing else: a mismatch of an integer with a float, a struct, or more than 16 arguments goes to the guard. In the test runs it adapted `abs`, `bsearch`, `sprintf` and `strlen` (lane 58), `cairo_pattern_set_extend` and `cairo_scaled_font_extents` (lane 65), and `git_oid_nfmt` and `git_tree_entrycount` (lane 66). Three environment variables of the node VM switch the FFI for the tests, read at the first load or callout: - `PHARO_WASM_FFI_HIDE`: a comma-separated list of reduced library names that the registry does not answer, so that the image finds them missing (`PHARO_WASM_FFI_HIDE=freetype`). - `PHARO_WASM_FFI_TRACE=1`: a line on stderr for each function the first time a callout to it is adapted, `FFI callout adapted: (declared , C )`. - `PHARO_WASM_FFI_ADAPT=0`: adapt no callout. A callback runs a nested interpreter on top of the C and JavaScript frames of its callout, so a slice never ends inside one: `src/ffi/sameThread/sameThread.c` keeps the depth of callbacks (`emscriptenCallbackDepth`), and `ioReturnToHostIfRequested()` returns to the host only at depth 0. Callbacks are reentrant (lane 58 nests them 50 deep), but a callback that computes for long keeps the VM from its host meanwhile: no output, input or Stop reaches it until it returns, and a 2 s loop in a callback ended no slice (55 slices for the run, against 151 with the same loop outside a callback). A callback declared with another type than the C function pointer traps in the indirect call that reaches it, and its callout fails; a callout that fails inside a callback, whose interpreter the exception unwound, stops the VM ("an FFI callout failed inside a callback, which it unwound"). There are no threads: `callbackFromAnotherThread` of the test library fails, and TFWorker does not run. ### The libraries Every library is built by a file of `cmake/emscripten/deps` from its pinned release archive, and linked into both VMs as a static library: | Library | Archive | Built with | Optimisation | |---|---|---|---| | libffi 3.8.0 | `libffi-3.8.0.tar.gz` | `WASM_FFI` | `-O2` | | FreeType 2.14.3 | `freetype-2.14.3.tar.xz` | `WASM_FREETYPE` | `-O2` | | zlib 1.3.2 | `zlib-1.3.2.tar.gz` | cairo or libgit2 | `-O2` | | libpng 1.6.58 | `libpng-1.6.58.tar.gz` | cairo | `-O2` | | pixman 0.44.2 | `pixman-0.44.2.tar.gz` | cairo | `-O2` | | cairo 1.18.4 | `cairo-1.18.4.tar.xz` | `WASM_CAIRO` | `-O2` | | libgit2 1.4.4 | `libgit2-1.4.4.tar.gz` | `WASM_LIBGIT2` | `-Os` | | simdjson 5.0.5 | `simdjson-5.0.5.tar.gz` | `FEATURE_PLUGIN_SIMDJSON` | `-O3` | `cmake/emscripten/deps/fetch.cmake` looks for each archive in `WASM_DEPS_DIR`, then in `build-wasm/downloads`, where an earlier configure downloaded it, and otherwise downloads it there from its release URL. The archive must have its pinned SHA256: one in `build-wasm/downloads` that has not is removed and downloaded again, and one in `WASM_DEPS_DIR`, or a download, that has not stops the configuration, with a message that names the file, both hashes and `WASM_DEPS_DIR`. It is unpacked again only when the one unpacked had another hash. No configure step of a library runs: what their autotools, meson or CMake would make is committed next to the file that builds them (FreeType's module list and options; the config headers and source lists of pixman, cairo and libgit2, which `cmake/emscripten/deps/cairo/README` and `libgit2/sources.cmake` say how to make again), and each file checks the version of the archive it was made for. The libraries are compiled with flags of their own, not with the VM's warnings, definitions and Debug optimisation: `-m64`, the setjmp/longjmp flags of the VM, their optimisation and `-w`. The VM links one zlib, which libpng, cairo's PDF surface and libgit2 share. simdjson, the parser of the SimdJSONPlugin, is the exception: `cmake/simdjson.cmake` fetches it the same way, for the native VMs too, and compiles its amalgamation `singleheader/simdjson.cpp` in C++17 with `-O3 -fno-exceptions`. - FreeType: its 26 sources, with a module list of its own (TrueType, CFF, autofit, smooth and raster) and 12 options off: no zlib, png, bzip2, brotli or harfbuzz, no colour or variable fonts, no Mac fonts. The fonts are those the images embed: lane 64 draws text with FreeType and compares it with the golden files of Pharo 12 and Pharo 15, which are those of the native VM (with FreeType 2.12.1: the same 688 pixels, the same hash). It adds 123.6 KiB of gzipped wasm, and about 0.3 s to the first frame of the world. - cairo, with pixman, libpng and zlib, on the VM's FreeType (the build checks that cairo sees 2.14.3): its image surface only, no fontconfig, and no renderers of COLR v1 and OT-SVG glyphs, which that FreeType cannot feed. AthensCairoSurface, in OSWindow-Web, copies cairo's pixels into a Form instead of going through SurfacePlugin, which the VMs leave out. Lane 65 compares shapes, gradients, FreeType text, a Roassal canvas and a Mondrian view with the golden hashes of the native VM. It adds 318 KiB of gzipped wasm. - libgit2: see below. Some libraries were deliberately left out: - harfbuzz: FreeType is built without it, and no image binds it. - fontconfig: cairo is built without it, since the image makes its font faces from the FT_Face of its own fonts, and no image binds it. - OpenSSL: the SqueakSSL plugin is off, and the libraries get no TLS of their own; the browser does the TLS of what goes through it, such as the requests of libgit2's smart-HTTP transport. - libssh2: a browser opens no socket, so there is no SSH: libgit2 has no SSH transport, and OSWindow-Web makes Iceberg's remotes https:// ones. - bzip2: FreeType reads no compressed fonts here (the images embed theirs), and nothing else asks for it. ### libgit2 `make wasm WASM_LIBGIT2=ON` links libgit2 1.4.4, the version that the native VMs ship, whose structures Pharo 12's LGit bindings match exactly (`git_fetch_options` is 208 bytes in 1.4.4 and 216 in 1.9), built at `-Os` from the 169 sources that a configure of upstream's CMake lists: no threads, SSH, HTTPS, NTLM, GSSAPI or iconv, the C library's regcomp, SHA1DC, and the bundled http-parser. Iceberg then works on the file system of the VM, through the image's own bindings (65 FFI methods and 9 callback classes in lane 66): init, commit, status, diff, log, branches and file:// clones. libgit2's own transports would resolve names, and `getaddrinfo` fails with `EAI_FAIL` (it is wrapped: Emscripten's ends a memory64 runtime), so a git:// remote fails with "failed to resolve address for ", and the VM goes on. With `WASM_LIBGIT2_HTTP` (the default), `src/emscripten/gitSupport.c` registers a smart-HTTP transport for the http and https remotes instead: each request of git's smart HTTP protocol is a synchronous XMLHttpRequest of the worker, which blocks the VM until the response has come, and the browser does the TLS. It serves both services, git-upload-pack (clone, fetch, pull) and git-receive-pack (push), each an advertisement of the refs (`GET .../info/refs?service=`) then the POST of a pack, which libgit2 writes whole before the request goes: it is sent with its length, never chunked, and can be sent again as it is. Git servers, github.com among them, send no CORS headers, so a page of another origin reads them only through a CORS proxy: the proxy is a URL prefix put in place of the scheme of the remote's URL (`https://github.com/o/r.git` through `https://proxy.example/` is requested as `https://proxy.example/github.com/o/r.git/info/refs?...`). The site has a default proxy, `WASM_GIT_PROXY` (the manifest's `gitProxyDefault`): by default the public proxy of isomorphic-git, `https://cors.isomorphic-git.org/` (`@isomorphic-git/cors-proxy` 3.0.3), which forwards only the requests of git's smart HTTP (the advertisements and the POSTs of both services, with the content type of their service; anything else gets 403 with an empty body), passes the `Authorization` header on, and sees what is cloned and pushed through it. The Settings of the Console and of the world page choose among three: - Site default: the site's proxy, whatever the site makes it; the dialog names it (or says that the site has none). - None: the requests go to the git host itself, which works only for a host of the page's origin, or one that answers CORS. - My proxy: a URL of your own, an http:// or https:// one without a user name, password or `#fragment` (a self-hosted proxy, say); the dialog says why it refuses another. This browser keeps the choice in localStorage, under `pharo-wasm.gitProxy`: absent for the site's default, `none` for none, or the URL (anything else counts as none, as it did in the builds before the default). Every page of the site and the Notebook use it, and a change reaches their running VMs at once, also from another tab; without localStorage it holds for the page. It is never taken from a URL: the proxy sees the code and any credentials that pass through it, and a link must not choose where a user's git traffic goes; a `?gitProxy=` of the page's URL is ignored. The first git request of a VM that goes through the site's default says so once per proxy, in a notice with a Settings button: "Git requests (Iceberg clones, Metacello github://) go through the public CORS proxy , which sees what you clone." (`pharo-wasm.gitProxyNoted` remembers the proxy told). The Notebook's kernel says it, and why a request failed, in the Notebook's notice. In the node VM the proxy is `PHARO_WASM_GIT_PROXY`, and there is none by default. The transport follows redirects (the browser does, or curl in the node VM; for a request with credentials, the node VM follows them itself, up to 10, see Push and credentials), and takes the URL that the advertisement of the refs came from for the next requests of the remote, as libgit2's own HTTP transport does: GitLab and Codeberg redirect a remote named without its `.git` to it, and refuse the pack's POST to the old name with 422. It does not follow an https:// remote to http://: curl in the node VM is not let go there (`--proto-redir =https`), and the URL is never kept for the next requests. A request that fails says why, and what to do: - No answer without a proxy (CORS, the network): "git http: request failed (CORS or network): ... A web page cannot read git servers without a CORS proxy: choose one in Settings (Git HTTP proxy)", as `GIT_ERROR_HTTP`, which Iceberg does not retry: a CORS refusal fails the same every time, so the browser makes one request, not three. The page says "The browser blocked git's request to (CORS): git over HTTPS needs a proxy here", with Settings. - No answer through a proxy: "git http: request failed (CORS or network) through the proxy : the proxy is down or refuses this page, or the network is down; check Settings", as `GIT_ERROR_NET`, which Iceberg retries (a proxy may be down for a while); for a request with credentials it adds "or the proxy does not allow the Authorization header of the credentials" (its CORS preflight refused it); the page says "The git proxy did not answer". - 401, when no credentials were given (see Push and credentials): "git http: answered 401: the repository does not exist, or it is private, or it asks for a user name and a password or token, and none was given (store them for in Iceberg's credentials, then try again)", with `GIT_EAUTH` (github.com answers 401 to a push without credentials, and to a clone of a private or missing repository); when the host refused those it got: "git http: refused the user name and password or token (401)". Credentials that would go through the site's default proxy are refused before Iceberg is asked: "git http: asks for a user name and a password or token (401), which this page sends only through a git proxy that you chose: the site's default, the public proxy , would see them. Choose it, or another, as My proxy in Settings (Git HTTP proxy), then try again"; and through a proxy of plain http:// on another computer. A 401 of another host than the remote's, after a redirect: "git http: redirected the request to , which asks for a user name and a password or token (401): the credentials of are not sent to another host (use the URL of that host as the remote)", with `GIT_EAUTH`; and a request with credentials that the browser followed to another host through the proxy: "git http: redirected a request that carried the user name and password or token to , and the browser sent them there again, through the proxy (a web page cannot stop that): the request is stopped, and the credentials are forgotten. If is not one of yours, revoke that token", with `GIT_EAUTH` (see Push and credentials). - 403 from the proxy (whose 403 has no body, as the proxy of isomorphic-git answers it): "git http: the proxy refused the request to (403): it forwards only the requests of git"; the page says "The git proxy refused the request to ". 403 from the host: "git http: answered 403: access denied (a private repository, or a rate limit)", or, after credentials, "git http: answered 403: the user name and token may not do this (a push needs a token with the right to write to the repository)". - 404: "git http: no git repository at (404)", the URL that answered (the one a redirect went to, if any). - 407: "git http: the proxy asks for credentials (407), which this VM does not support", with `GIT_EAUTH`. - 413: "git http: the request is too large for [the proxy or for] (413): a push of a pack this large cannot go this way". - Any other status: "git http: unexpected http status code: "; a 200 that is not the answer of a smart HTTP server: "git http: invalid content-type: ...". - A push that the host answered 200, but whose report refuses the ref (`ng ` of report-status, in the side band when there is one: a pre-receive hook that declined it, a protected branch of github.com, a token that may not change workflows): "git http: refused the push of : ; says: ". libgit2 1.4.4 fails a push only when the pack was not unpacked, and gives a refused ref to a callback (`push_update_reference`) that Iceberg does not set: without this, `pushTo:` and the Push button would say nothing, and the commits would stay outgoing. A 401, 403 or other status whose body is plain text (`text/plain`, not a page of HTML, of a proxy or a CDN) adds the first line of it, as "; says: ", the one that answered: github.com explains there what it refused ("Invalid username or token. Password authentication is not supported for Git operations.", "No anonymous write access.", "Permission to o/r.git denied to ."). OSWindow-Web (version 4 and later) sets Iceberg's remotes to https, and version 7 sets its provider of credentials (see Push and credentials). In the browsers, a clone through the proxy of the spec took 573 ms (Chromium) and 478 ms (Firefox), with 2 requests. The clones live in the file system of the VM, in `/pharo/pharo-local/iceberg` say, and are kept with the image when it is saved in the browser (see The saved images): after a reload they are there as they were at the last save, consistent with the image's Iceberg; what was cloned or committed since is lost. A commit needs a user, which libgit2 reads from the configuration of the repository, else from the global one, `$HOME/.gitconfig`. In the browser `HOME` is `/pharo/home`, which is kept with the image like the clones, and has no `.gitconfig` at first: the first commit of an image shows Iceberg's dialog "Please provide Git missing properties", and with "Save properties as global?" checked it writes `/pharo/home/.gitconfig`, which every repository of that image then uses, also after a Save and a reload (unchecked, it writes the user into the repository's `.git/config`). Each saved image has its own home, so a new image asks once again; the node VM reads the `HOME` of its environment, as a native VM does. That dialog makes an LGitConfig, which frees itself when the garbage collector takes it. In Pharo 13 and later LGitConfig has no free function of its own and inherits LGitExternalObject's `git_object_free`, which, called on a `git_config`, damages the configuration that the repository keeps: the next commit failed with "FFI callout trapped ... memory access out of bounds" (Firefox, the node VM) or crashed the tab (Chromium), and a native VM may crash too. `packaging/emscripten/st/web-libgit.st`, which the site stages in `st/`, gives LGitConfig the `config_free:` and `freeFunctionSelector` of Pharo 12, which call `git_config_free`, when it still frees with `object_free:`; it leaves Pharo 12, a Pharo that frees configurations right, an image patched already and an image without LGit alone, and then compiles and logs nothing. It is the one copy of that patch, and every image that the site runs gets it: the Console and the Notebook's kernel evaluate it at every boot (`web-repl.st` and `web-notebook.st`), whatever the image, a stock one opened from disk or a saved one of any build too, and `OSWebDriver class>>setUpLibGit` (OSWindow-Web 6) evaluates it where an image is set up for the world: at build time for the images of the site (which `node-try.sh` runs too), and on the world page for an image opened there or saved with an older OSWindow-Web, which it prepares again; the world boots its images as they are, prepared. In an image with OSWindow-Web the two methods are extensions of that package, so that Monticello sees LibGit-Core unmodified and a load of LibGit-Core keeps them, and the `OSWindow-Web.st` of the site does not carry them. The page prepares an image for the world by booting it as the Console does, which puts the two methods in LibGit-Core before `web-bootstrap.st` files the package in; `setUpLibGit` then finds them there (their comment names `web-libgit.st`) and compiles them again as extensions, once. In an image without it, a stock one in the Console say, they go into the protocols of LibGit-Core that Pharo 12 has them in, rather than make a package OSWindow-Web of two methods, compiled without the announcements of the system, so that neither Monticello nor Iceberg sees LibGit-Core modified, nor Epicea a change (a load of another LibGit-Core removes them, and the next boot of the Console or of the Notebook compiles them again; the world does not evaluate `web-libgit.st` at its boots, but its images have the methods as extensions, which such a load keeps). Their source goes to the `.changes` when they are compiled, about 400 bytes, once for each boot of an image that is not saved since (uFFI keeps only the pointer to the source of a callout that it generates at the first call); an image saved patched logs nothing more. `web-libgit.st` also fixes `LGitConfig>>config_find_global_path:`, which every LibGit-Core from Pharo 12 to 15 declares `int git_config_find_global(String* out)`, where libgit2 1.4.4 takes a `git_buf *`, `{char *ptr; size_t reserved; size_t size}`, whose `ptr` it allocates and the caller disposes of. Its one sender, `LGitConfig>>globalConfigurationPath`, gives it a zeroed String of 256 characters and answers it: libgit2 writes the `git_buf` into its first 24 bytes, so it answered 256 characters of binary junk, never the path nor nil (also when `HOME` has no `.gitconfig`, `GIT_ENOTFOUND`), and leaked the 256 bytes of the path at each call, natively and in the web VMs alike; given a shorter String, the callout overruns it in the heap of the image, and given one that is not zeroed, libgit2 writes through the junk it takes for `ptr` (a native VM crashes, a web VM traps the callout). Nothing in Iceberg or LGit calls either method (Iceberg opens the global configuration with `git_config_open_global`, declared right): a user reaches them by evaluating them, `repository config globalConfigurationPath` say, in the Console, the Notebook or a Playground of the world. `web-libgit.st` compiles both again where the callout still gives `git_config_find_global` something else than an `LGitBuf` (the 24-byte struct of LibGit-Core, which `getString:ifPresent:ifAbsent:` gives `git_config_get_string_buf`): the callout takes an `LGitBuf`, and `globalConfigurationPath` answers the path, `/pharo/home/.gitconfig` in the browser once Iceberg's dialog wrote it, or nil, and frees the buffer (`LGitBuf>>free`, `git_buf_free`, which 1.4.4 makes `git_buf_dispose`). They go where `config_free:` goes, by the same rules: extensions of OSWindow-Web in an image that has it (OSWindow-Web 8, which takes them out of the snapshot of LibGit-Core, with Monticello's LibGit-Core still unmodified), else the protocols of LibGit-Core that the stock methods are in, without the announcements of the system; their source goes to the `.changes`, about 900 bytes, once for each boot of an image that is not saved since, as for `config_free:`; an image patched already compiles and logs nothing, and a load of LibGit-Core puts the stock methods back until the next boot of the Console or the Notebook, or the next preparation for the world. The other callouts of LGit that take a `git_buf *` (`git_config_get_string_buf`, `git_object_short_id`, `git_diff_stats_to_buf`, and `git_patch_to_buf` in Pharo 15) declare it an `LGitBuf *`, and no image binds another function that fills a `git_buf`. libgit2 adds 258.6 KiB of gzipped wasm; the default build is unchanged. #### Push and credentials A push, and a fetch or a clone of a private repository, needs credentials: on github.com, GitLab and Codeberg, HTTP Basic with a user name and a token as its password (SSH is out of reach of a browser). The transport sends each request without them first, and takes a 401 as a challenge of Basic, whatever its `WWW-Authenticate` (the proxy of isomorphic-git drops that header); it then finds a user name and a password, and makes the same request again with `Authorization: Basic base64(user:password)`, as libgit2's own HTTP transport does: 1. the user name and the password of the remote's URL (`https://user:token@host/o/r.git`), once; 2. then libgit2's credentials callback (`git_transport_smart_credentials`, offered the plaintext type only, since Iceberg answers any other with an SSH key), which Iceberg answers from its store, `IceCredentialStore` (the credentials of the remote's host), else from its provider, `IceCredentialsProvider class>>providerType` (see below); 3. until the callback gives none (Iceberg's provider gives up at its third try), or five rounds: the 401 then fails as above. Credentials that a host accepted go up front with the next requests of the same connection (the POST of the pack), never in a URL: every request goes to the URL without its user info, through a proxy or not. They are only for the host of the remote's URL (its host and port): a 401 of another host, the one a redirect went to, fails with the message above, without asking Iceberg, which would give those of the remote's host (it looks them up by the remote's URL); use that host's URL as the remote. A redirect of a request that carries them to another host: curl does not send them there (for such a request the node VM follows the redirects itself, and drops them for another host), nor does the browser across origins, so straight to the host they go nowhere else. Through a proxy, though, a redirect stays on the proxy's origin (the proxy of isomorphic-git makes any `Location` a path of its own, `//`), so the browser sends them again, and the proxy passes them on to that host: a synchronous `XMLHttpRequest` cannot refuse a redirect, nor say where it goes before it went. Then the transport stops the request (its response is not used), forgets the credentials, and says which host may have got them, and the page says " redirected a request with your user name and token to , and the git proxy sent them there: the request was stopped. Revoke that token if is not yours." (the worker's `failed`, credentials `redirected`). So use a proxy that you trust, and a token of the least rights (see below). The VM keeps them for one connection: it wipes them when libgit2 closes it (`git_remote_disconnect`, which a push and a fetch end with; libgit2 closes it when it connects again too), and when it frees the remote; `Module.onGitHttp` learns only whether a request carried some, never what. The browser sets the header with `setRequestHeader` (a user name and password in the URL, or given to `open()`, go nowhere across origins), and asks the proxy first whether it takes it (a CORS preflight, which the proxy of isomorphic-git and that of the tests answer with `Authorization` among their allowed headers; a proxy that does not fails the request through it); in the node VM curl reads it from a file of mode 0600 in a directory of its own in `TMPDIR` (`-H @file`), which is removed after the request, so that it is not on the command line. Iceberg's own provider, `IceTipInteractiveCredentialsProvider`, opens its dialog inside libgit2's callback, and waits: a web VM gets no input, and draws nothing, until a callback has returned (see The FFI), so that dialog would hang the page, and with nothing to answer it in the Console and the Notebook the VM spins until the page closes. So `packaging/emscripten/st/web-libgit.st` (which the Console and the Notebook's kernel evaluate at every boot) makes that provider `IceNonInteractiveCredentialsProvider`, which gives the stored credentials and else none: in the Console and the Notebook, store them before a push, IceCredentialStore current storeCredential: (IcePlaintextCredentials new username: 'me'; password: ''; yourself) forHostname: 'github.com' and the world's images have `OSWebCredentialsProvider` (OSWindow-Web 7, `OSWebDriver class>>setUpGit`): inside the callback it gives the stored credentials, or none, and asks once the callout has returned, in the next cycle of the world: Iceberg's dialog ("Please log in to ", Token for github.com) opens after the push failed, and what it is given is stored (its checkbox, checked by default) or kept for the session of the image (forgotten before a save), so that the next push takes it at once. Outside libgit2's callback, as when the Iceberg plugin of GitHub asks, it is Iceberg's own provider. A provider chosen otherwise is left alone. In the browsers, a push that has no credentials returned in 256 ms (Chromium) and 238 ms (Firefox), and the dialog opened after it. `IceCredentialStore` keeps what it stores in plain text, in the image and in `FileLocator localDirectory / 'iceberg' / 'credentials.fuel'`, `/pharo/pharo-local/iceberg/credentials.fuel` in the browser: a Save keeps both in the browser's IndexedDB, which every page of the same origin shares (every GitHub Pages site of an account), and a Download carries them. Use a token of its own for this, with the least rights (a fine-grained token of GitHub limited to one repository, say) and an expiry, and remove it from the store (or uncheck the store in the dialog) on a computer that is not yours. The credentials go only through a proxy that the user chose: through My proxy, or straight to the host with None (TLS goes all the way to it), never through the site's default, a public proxy that the user did not choose, which, and the services in front of it (Cloudflare, for the proxy of isomorphic-git), could read them. While the proxy is the site's default, the worker sets `Module.gitHttpCredentials` false, and the transport refuses credentials at the first 401, before Iceberg is asked (so no dialog asks for a token that would not be sent), with the message above; the page says " asks for a user name and token, which this page sends only through a git proxy you chose: the site's default, the public proxy , would see them. Choose it, or your own, then try again", with Settings and a button "Use for them" (the world page's notice has that button alone), which makes the site's proxy My proxy (`PharoGitProxy.useForCredentials`, for every page of the site, as Settings would): the user then pushes again, through the proxy that they chose with that button. The first request that carries credentials through a proxy says so once per proxy: "The user name and token that asked for go through the git proxy , which can read them." (`pharo-wasm.gitCredentialsNoted` remembers the proxy told). That choice, and that note, live in the `localStorage` of the origin, which every page of it can change, as it can read the store of credentials in IndexedDB: the origin is the boundary, not the page. A user name and password in a remote's URL follow the same rule, and go in the header too, only after a 401. But a token in a remote's URL does not stay with the transport: Iceberg keeps the URL in the repository's `.git/config` (under `/pharo`, saved in IndexedDB with the image, and in a Download), in the image (the remote), and prints it in `PharoDebug.log` and in the debugger's stacks; prefer Iceberg's store, or the world's dialog, and never share an image, a Download or a log that has such a URL. A proxy of plain http:// is refused for credentials unless it runs on this computer (localhost, [::1], or a numeric 127.x.y.z: not a name that starts with 127.), where nothing else sees them. The check is made again at each request that would carry credentials given before, with the proxy in effect then: through a proxy that may no longer have them, the request goes without them, and its 401 fails with the refusal. The node VM has no such page: its proxy, `PHARO_WASM_GIT_PROXY`, is the user's own, and `Module.gitHttpCredentials` is not set. The 401 that the transport answers with credentials is no failure for the page (`onGitHttp`'s `retried`), so it shows no notice for it. #### Pull A pull is a fetch (git-upload-pack) and Iceberg's own merge (`IceLocalBranch>>pullFrom:`, `IceMerge`), at the level of the definitions of the code, then a checkout of the result (`git_checkout_tree`): it needs nothing of the transport but the fetch, and credentials only for a private repository. A fast-forward loads the commits that the server got into the image and the working copy; with branches that diverged, a push first fails with `IceRemoteDesynchronized` (libgit2 refuses it after the advertisement, before any POST), the pull makes a merge commit of two parents, and the push of the merge lands. A merge that has conflicts asks how to resolve them in a dialog of Iceberg (`IceMergeResolutionRequest`, whose default action opens a modal `IceTipMergePresenter`, or `IceTipMergeMorph` in Pharo 12), which only the world shows: in the Console and the Notebook nothing would ever answer it, and the pull would never return. So `web-libgit.st` compiles that default action again, once (Iceberg's own kept as `#webWorldDefaultAction`, in the same protocol and package, without the announcements of the system, so that Monticello does not see Iceberg modified): in a world it is Iceberg's, without one (Morphic's UIManager does not draw) it signals `IceMergeAborted` at once, "Merging into : the merge has conflicts, which Iceberg resolves in a dialog that only the world shows (world.html): pull there, or merge by hand; nothing was merged", with the head, the working copy and the image as they were (the fetch has brought the commits of the server, which are incoming). Once: a default action whose source names `web-libgit.st` is left as it is. Before OSWindow-Web 8 the file looked for 'compiled by web-libgit.st', which its own default action did not say (it said 'Compiled by' and broke the line), so each evaluation after the first compiled both methods again, and `#webWorldDefaultAction` became a copy of the default action of the file, which in a world sends `#webWorldDefaultAction`, itself: a merge with conflicts in the world recursed without end. An image evaluated so twice and saved kept it, a stock image that the page prepared for the world above all (its boot as the Console, then `setUpLibGit`), and it cost about 1.3 KB of `.changes` at each later boot of the Console. Where `#webWorldDefaultAction` sends itself, `web-libgit.st` now compiles it again from Iceberg's own default action, which it finds through the `.changes`: the preamble of each source logged there points to the source it replaced (`prior:`), so the versions of `#webWorldDefaultAction`, then those of `#defaultAction`, lead to the first one that does not name `web-libgit.st` (the copy that the first evaluation logged, or Iceberg's in the `.sources`); where there is none (a condensed `.changes`), the method aborts the merge as without a world, rather than recurse. The world page prepares an image of version 7 again, booting it as the Console, so its saved images get it back at their next world boot, and the Console and the Notebook at their next boot. The callouts of a pull and a push are adapted as those of the other flows (no other function than those of `adapted-66.txt`). #### Clone from GitHub Iceberg's "Clone from github.com" (`IceTipGitHubRepositoryPanel`) clones over git, then asks GitHub's REST API whether the repository is a fork, to add its parent as a remote too (`getGitHubRepository`). That request goes through Zinc, which needs sockets and name resolution, which the web VMs do not have: it fails with `NameLookupFailure: cannot resolve 'api.github.com'`, which Iceberg does not catch, so the clone ended in that error and the repository it cloned was not registered. Since OSWindow-Web 10, `web-libgit.st` keeps Iceberg's method as `#webGetGitHubRepository` and compiles a `getGitHubRepository` that answers nil on a `NetworkError`, as Iceberg does for a private repository: the clone has its remote, without that of the parent of a fork. Once, in Iceberg's protocol and package, without the announcements of the system, as the default action of the merge above; the world page prepares an image of an older version again, so its saved images get it too. "Clone remote repository" with the `https://` URL never asked the API, and Metacello's `github://` repositories clone over git only. Iceberg's other uses of GitHub's API (its pull requests, the removal of remote branches) still fail so in the web VMs. ### What else differs - The Release build defines NDEBUG, as CMake's Release flags do: with the asserts, a 1M-deep recursion took 34 s instead of 1 s. - With `WASM_FFI=OFF`, the image uses its NullFFIBackend, and the support primitive that the image calls at every start to initialise its callbacks does nothing, silently. - `getaddrinfo` is disabled (`EAI_FAIL`): Emscripten's version fails in wasm64 with a JavaScript TypeError, which ends the VM. Name resolution fails instead with an error that the image can catch. - `aio` sets no `F_SETOWN` or `SIGIO`, and does not `fsync`. - The microsecond clock has a resolution of 1 ms (`Date.now()`), and `ioHighResClock()` answers monotonic nanoseconds. - The image attributes 1001, 1002 and 1003 are `unix`, `Emscripten` and `wasm64`, and the platform is `Unix64Platform`. - The path of the VM is its argv[0] when no file has that name: `/pharo/pharo` in the pages, so that `Smalltalk vm directory` is `/pharo`. ### The web pages The files of `web/`, from `packaging/emscripten/web`: - `index.html`, the Console page, and `page.js`, which drives its VM; `notebook.js`, its Notebook tab, with `notebook-lib.js` (the Markdown renderer, the sanitizer of markup, the `.json` format), `notebook-st.js` (what is Smalltalk: the brackets, the highlighter, the `.st` format, the example) and `nb-kernel.js` (the client of the kernel). - `world.html` and `world.js`, the world page, with `keymap.js` (the keys of the page as the keycodes, scancodes and modifiers of SDL2, which OSWindow numbers them as) and `display-worker.js` (the display of the WebDisplayPlugin, in the worker). - `vm-worker.js`, the worker of every VM, with `vm-driver.js` (the slices, the stdio devices and `/dev/nbevents`), `vm-storage.js` (the saved images in IndexedDB, which the pages read too) and `catalog.js` (the images of the site, from the manifest, for the worker and the pages); `open-image.js`, which reads the images of the user's own; `images.js`, the Images dialog, and `git-proxy.js`, the git proxy of Settings and its dialog, which the Console and the world page share; `pharo-logo.png`. - `pharo-web.js` and `pharo-web.wasm`, the VM; `manifest.json`; `THIRD-PARTY-NOTICES.txt`; `image/`, the images, their `.changes` and `.sources`, gzipped; `st/`, `web-repl.st`, `web-notebook.st`, `web-libgit.st`, `web-bootstrap.st` and `OSWindow-Web.st`. `manifest.json` (`packaging/emscripten/tools/stage.mjs` says every key) lists the files of the default image in `files`, and the catalog of the site's images in `images`, also when there is one: for each its `id`, `label` ("Pharo 13.1"), `pharo` ("13.1", from the name of its `.sources`), `pharoBuild` (from the name of its zip), `world` and `webPackage`, and its `files`; the default one has `default: true`, and `imageId` is its id. `extraSources` lists the `WASM_EXTRA_SOURCES`, and `gitProxyDefault` is the site's default git proxy (`""` for none). `catalog.js` is its only reader in the pages and the worker, and makes a catalog of one image of the manifest of an older build. The page compiles `pharo-web.wasm` once, and gives it to every worker it starts (`packaging/emscripten/web/vm-worker.js`). The worker fetches the files of the image it boots, as `manifest.json` lists them, inflates them as they come (DecompressionStream), writes them into `/pharo` in memory, after the files that a saved image keeps and the placeholders of the libraries, and boots the image in one of three modes (`PharoVMDriver.vmArgs`): `console`, the REPL, `--headless /pharo/Pharo.image --no-default-preferences st --no-source /pharo/st/web-repl.st`; `notebook`, the same with `web-notebook.st`; and `world`, `--headless /pharo/Pharo.image --no-default-preferences --interactive`, with a display. `--no-source` keeps the script, which is filed in at every boot, out of the `.changes`; it first evaluates `/pharo/st/web-libgit.st` (see libgit2). What the VM writes to stdout and stderr is posted at the end of every slice, and the worker pauses the VM while more than 1 MiB of it is not acknowledged, until less than 512 KiB is: the page acknowledges what it rendered, or, while it is hidden and gets no animation frames, what it received. Stdin is Emscripten's terminal device with its read and poll replaced: a read never blocks, and the REPL waits with `waitForData`, so the VM sleeps until the page sends a line. Stop calls `vm_interrupt()`, which signals the semaphore that the REPL, or the kernel, registered with the WebHostPlugin. When the image saves, the image file access handler tells the host (`Module.onPharoHost`), and after the slice the worker stores the image, its `.changes` and the files of `/pharo` in IndexedDB, in the slot that the VM is bound to: the one it booted, else a new one. The database belongs to the directory of the site: `pharo-wasm` for a site at the root of its origin, and `pharo-wasm:/dir/` for one in `/dir/`; it stays at version 1, with its one object store, `files`. A slot is `main` or an id of `newId()`; the keys of `main` are those of the single image of before (`meta`, `Pharo.image`, `Pharo.changes`, `Pharo.sources`), and those of another slot have the prefix `slot//`. Each slot has `meta` (its record: the id of its last save, its slot, its sizes and dates, the build and the version of OSWindow-Web it was saved with, the name of its `.sources`, its Pharo version, the catalog entry it came from, and its origin), `Pharo.image`, `Pharo.changes`, `Pharo.sources` (only when it has its own), `Pharo.tree` (its files: `{v: 1, entries, data}`, one record whose entries `[path, kind, mode, mtimeMs, offset, length, linkTarget]` index one Blob of their bytes) and `Pharo.name`, which only a rename or the creation of the slot writes, so that a save never undoes a rename. One more key, `last`, names the slot booted or saved last. A save writes the whole slot in one transaction; a new slot always gets a new id, never `main`, which a page of the build of before would overwrite without a guard. The `.changes` of a slot, stored on its own, is guarded by the per-save id of its slot, so that a tab never pairs its `.changes` with the image of another tab, and pages on different slots never interfere. In the Console, the worker names the file `/pharo/.pharo-web-world` in `PHARO_WEB_WORLD_FILE`, and the REPL writes there whether the image has OSWebDriver, and its version, when it starts and before it saves: so the page knows whether the image can open the world. A worker in `notebook` mode boots the image of the Console, a saved one too, but never stores anything: it ignores the VM's saves, `save`, uploads and preparations. The messages between a page and its worker: - page to worker: `init` (the compiled module, the manifest's URL, the build, the mode, the slice, whether to keep images, the image to boot: an upload, a saved slot or an image of the site, and the name of the slot it will make, a preparation, a display, the git proxy), `input`, `eof`, `interrupt`, `save`, `saveAs` (the name of a new slot for the next save), `ack`, `fs`, `download`, `resetStorage` (one slot, the one bound, or all), `flush`, `display`, and `gitProxy` (`null` for the site's default, `''` for none, or a URL); - worker to page: `progress`, `ready` (what was booted: the slot, the catalog entry, the Pharo version, and what was missing of what was asked; the proxy in effect and the site's; and what the VM has), `output` (text of fds 1 and 2, bytes of fd 3), `state`, `tick`, `interrupted`, `edited`, `storing`, `saved` (the slot, whether it was made, the files kept or why not), `prepared`, `superseded`, `reset`, `exit`, `crash`, `fs-result`, `file`, `error`, `display` and `git` (the first request of the VM, and a failed one, with its host, proxy and status, from `Module.onGitHttp` of the transport). The notebook kernel reads its requests on the VM's stdin, one JSON line each (`run {rid, name, code}`, `ping`, `stop-background`), and writes its events on `/dev/nbevents` (`hello`, `start`, `display`, `clear`, `stream`, `error`, `done`, `pong`, `stopped`, `bad-request`), which the worker posts as fd 3; the requests are checked strictly, and the page checks every event before it allocates anything for it. What the user opens is read by `open-image.js`, which the pages load: it reads the central directory of a zip, inflates its entries with `DecompressionStream('deflate-raw')` (and takes stored ones as they are), pairs the files, and checks the image's header before the VM is touched. The page gives the files to a new worker in `init.upload` (`{image, changes, sources}`). A `.sources` of the image's own goes to `/pharo/` instead of the site's, which is then not fetched, and the saved image keeps it in IndexedDB under the key `Pharo.sources` of its slot. A slot or an upload without one gets the `.sources` of its name that the site serves, of any of its images or of `extraSources`: the name its slot recorded, or the one that the bytes of the image hold (`Pharo-bit-`), else the default image's; when the site does not serve that name, the VM gets none, and the page says that the source of the system's methods shows decompiled. On the world page, `init.prepare` boots an image that has no OSWindow-Web, or an older version of it, with the REPL, which evaluates `CodeImporter evaluateFileNamed: '/pharo/st/web-bootstrap.st'. Smalltalk snapshot: true andQuit: false`; the worker answers `prepared {error, saved}`, and the page then boots the world from the saved image. ### The world The Pharo world draws itself into a Form, and the VM only blits it. SDL2's Emscripten driver expects the DOM of the main thread, and the image's SDL2 driver blits through SurfacePlugin; the world page needs neither, and the VMs have neither (`WASM_SDL2` is gone, see Settings). The world image has the OSWindow-Web package (`packaging/emscripten/st/OSWindow-Web`): OSWebDriver, a backend window and a Form renderer, which a startUp: hook picks whenever the page gives a display, and the extension of AthensCairoSurface that copies cairo's pixels into a Form. It keeps the FreeType fonts of the image, unless the build has no FreeType, and then gets bitmap fonts: `OSWebDriver>>setUpImage` takes the fonts the build gives from `webimage-fonts.txt`, and the manifest says which (`fonts`). The package is at version 8 (`OSWebDriver class>>packageVersion`, the manifest's `webPackage`): version 2 kept the FreeType fonts, 3 added the AthensCairoSurface extension, 4 makes Iceberg's remotes https:// ones, 5 draws the world in device pixels on high-density displays, 6 gives LGitConfig the free function of Pharo 12, 7 gives Iceberg `OSWebCredentialsProvider`, which asks for the credentials of a push outside libgit2's callback, and 8 gives `git_config_find_global` the `git_buf` it fills, and Iceberg's own dialog of the merge back in a world where an earlier `web-libgit.st` made it call itself (see libgit2, and Pull); an image saved with an older version is prepared again when the world page opens it. The native Pharo VM prepares it at build time, which is fast and keeps the world image independent of the VM being built; it is prepared again when a class of the package changes, comes or goes. `src/emscripten/plugins/WebDisplayPlugin.c` is a builtin plugin of 13 primitives. The image blits each damaged rectangle of its 32-bit Form into the frame, an RGBA copy, and presents the frame once per Morphic cycle. The dirty rectangles (up to 64, then their bounding box) go to the page, and the slice ends, so that the canvas shows them at once. The page's events come back as records of 8 integers, in a ring of 256 records, and each one signals the image's input semaphore. The title, the cursor (1-bit Forms, as RGBA), the clipboard and the focus go to the page as well. In the worker, `display-worker.js` paints the dirty rectangles, straight from the VM's memory, into the OffscreenCanvas that the page transferred to it. `world.js` sends it the size of the canvas, in CSS pixels and in device pixels, and the input, which `keymap.js` maps to the keycodes, scancodes and modifiers that OSWindow takes (SDL2's). A stock image cannot open the world page as it is: its world starts through OSSDL2Driver, the driver of the desktop VMs. The world page prepares the stock images that it opens, and the Console offers to ("Prepare for the world"): both run `packaging/emscripten/st/web-bootstrap.st`, which files in the package and sets up the fonts, Iceberg's remotes and provider of credentials, and LGitConfig, and then save. ### Pharo versions A local build runs the pinned Pharo 12 image (build 1599) by default. The site of GitHub Actions offers Pharo 15 (Pharo15.0-SNAPSHOT build 50, `b85960fcc6`), its default, Pharo 14 (Pharo14.0-SNAPSHOT build 785, `74f3a1b398`) and Pharo 13.1 (Pharo13.0-SNAPSHOT build 749, `d7c6f761d5`), each prepared for the world, and serves the `.sources` of Pharo 15 build 41 (`Pharo15.0-64bit-4e572fe.sources`), the image it had before, for the images that browsers saved from it. Lane 80 runs the world on every image of a site, and Pharo 12 and 15 images were also opened from disk on either page. The VM is the same; what differs is the Smalltalk of the pages (`packaging/emscripten/st`), which works with all of them; the differences below were found between Pharo 12 and 15: - Pharo 15's command line (Clap) takes a single file after `st`, with `--save` and `--quit` as its own options, and a file only after `st`: the world image is prepared with `st --save --quit prepare-web-image.st`, which finds its directories in `PHARO_WEB_ST_DIR` and `PHARO_WEB_IMAGE_DIR`. - Pharo 15's chunk reader takes one method per `methodsFor:` section, so `web-repl.st`, `web-notebook.st` and the `OSWindow-Web.st` that `prepare-web-image.st` writes (the same from both versions) give each method its section, and define every class before its methods; `web-notebook.st` is ASCII, since Pharo 15 mangles the other characters of a file-in. - Pharo 15 handles errors through `ErrorHandler default`, opens its world in `UIManager class>>startUp:` (so OSWebDriver registers for an earlier startup), and does not boot with an empty `.changes`: the pages give an image opened without one a `.changes` that holds only its version header, `"VERSION:1.0"!`. - The libraries of the registry are those both images bind: they bind the same functions of cairo and FreeType, and each binds one function of libgit2 that the other does not. - Pharo 13 writes a `pharo-local` directory next to the image when it is prepared at build time, which the build removes before the image is staged. ## Tests `make wasm-check` runs `tests/wasm/run-lanes.sh`, which runs every `tests/wasm/lanes/*.sh` in order; `sh tests/wasm/run-lanes.sh 20 60` runs only the lanes whose names start with 20 or 60. A lane that cannot run here is skipped, with the reason, and the run ends with the list of the lanes and of the checks that were skipped. `make wasm-check` gives the lanes `NODE`, `WASM_DIR` (the build directory), `GEN` (the generated sources), `HOST_PHARO` (`WASM_HOST_PHARO`), `SRCDIR` and `TEST_DIR` (`build-wasm/tests-run`). Run by hand, `run-lanes.sh` defaults them for `build-wasm`, except `HOST_PHARO`, which stays empty unless it is given: the checks that need a native Pharo VM are then skipped. The lanes are meant for Release builds. On the machine the port was made on, the 23 lanes passed in 13 minutes on the site of three images with libgit2 (lane 61 took 5 of them), and the 22 that a site of the pinned Pharo 12 with libgit2 runs (68-simdjson skips it) in 11 minutes; without libgit2, lane 66 is skipped. The times below are those of an earlier run, on a Pharo 12 build with libgit2 and `NOTEBOOK_IMAGES` of a Pharo 15 image (9 minutes of lane 61), but for lane 66, of the site of three images: | Lane | What it checks | Checks | Time | |---|---|---|---| | 05-build-rules | the rules of `GNUmakefile` and `cmake/emscripten/webimage.cmake`, in small trees where stand-ins of cmake, emcmake, the Pharo VMs and the specs log what they are asked to do: what is configured, generated or prepared again when a setting (the libraries', `WASM_DEPS_DIR` and those of the images and the git proxy too), the initial cache, a `.st` file or a class of OSWindow-Web or `web-libgit.st` changes, comes or goes, the refusals of bad images, ids, `.sources` and proxies, an extra image prepared once in a work directory of its own, the fonts of the world image, the patch of libffi, the guard of `WASM_BUILDDIR`, the refusal of `WASM_SDL2`, and which specs `make wasm-check-browser` runs; then `stage.mjs` on a catalog of images (the default at its URLs of before, an extra one under `image//`, the labels, builds and order of `manifest.images`, an extra `.sources`, the checks of the ids, of the `.sources` and of OSWindow-Web, and a build id that follows the build of an image); it needs GNU make and cmake, and builds no VM | every rule | 13 s | | 10-types | `tests/wasm/check-types.sh` compiles the generated interpreter with `sqVirtualMachine.c`, `client.c` and `sqExternalSemaphores.c` as one translation unit: every interpreterProxy slot and prototype must have the interpreter's type; both link commands of the CMake tree (Unix Makefiles or Ninja) carry `-Wl,--fatal-warnings`, and neither module has a signature mismatch stub | 0 errors | 1 s | | 20-smoke | `tests/wasm/wasm-smoke.sh`, S1 to S22 with the node VM: the exact output of `eval` and `st`, the platform and the FFI backend, files, the environment (`setenv` through the FFI), Delays, preemption, deep recursion, a growing heap, a snapshot reloaded in WebAssembly and natively, exit statuses, time zones, the builtin plugins, the slices and where old space is placed | 28 | 24 s | | 22-old-space | `tests/wasm/st/oldspace-window.st`, `oldspace-regrow.st` and `permspace.st` with the node VM: old space ends in an `OutOfMemory` at the end of its window, and the young objects stored at its top survive a scavenge; a freed segment at the top is used and grown again; the perm-space primitives fail | 3 | 12 s | | 25-session-id | `tests/wasm/session-id.mjs`: a snapshot booted again within the same second gets a session ID of its own and refuses the old session's file handles; the C streams are flushed on exit | 8 | 2 s | | 30-engines | S2, S8 and S9 again, in Liftoff code only (`--liftoff-only`) with a 900 KB stack, and in a worker thread with a 1 MB stack, the stack of a browser worker; S2t checks that V8 compiled nothing with TurboFan, and fails when the V8 flags of the lane do not reach node | 8 | 17 s | | 40-bench | `tests/wasm/bench.sh`: the measures below, next to reference numbers; it fails when 300k-deep recursion takes 1 s, when `eval '3+4'` takes 1.5 s, or when 2 s of computing run in fewer than 10 slices | 8 | 13 s | | 50-prim-audit | `tests/wasm/prim-audit.mjs` reads `pharo.wasm`: the named-primitive lookup answers the trampoline tables, every primitive is called through a `() -> ()` function, every function pointer is at least 1024, nothing goes through `-sEMULATE_FUNCTION_POINTER_CASTS`, the builtin modules are the expected ones (17 with the SimdJSONPlugin), and the 20 support primitives of `src/ffi` are there | every row (396) | < 1 s | | 55-webdisplay-unit | `tests/wasm/webdisplay-unit.c` on the frame code of the WebDisplayPlugin, then its primitives on bad arguments in the node VM | 235 + 34 | 2 s | | 56-memory-unit | `tests/wasm/memory-unit.c` drives `src/emscripten/memoryEmscripten.c` as Spur does, against a stub of `pharovm/pharo.h`: fixed spaces, segments that grow again where freed ones were and come back zeroed, old space that never leaves its window, and a first segment larger than the window | 55 + 2 | 1 s | | 58-ffi | `tests/wasm/st/ffi.st` with the node VM: callouts to the C library and to the FFI test library, scalars, a variadic function, structures by value, callbacks, reentrant ones 50 deep, results declared wider than the function's, the declarations that the adapter adapts (traced with `PHARO_WASM_FFI_TRACE=1`), and exactly two guarded failures and one of the adapter, each with its one line on stderr; a bench of 10000 `strlen` callouts (`WASM_FFI_BENCH_MS`, default 10000); then a 2 s loop in a callback must end no slice (`PHARO_WASM_STATS`) | 3 (26 in `ffi.st`) | 9 s | | 59-ffi-clean | the stock image and the world image boot, run `tests/wasm/st/ffi-session.st` (the image's own callouts: the environment, getpid, strerror, a qsort with a callback), save, and run it again on the saved image, then draw text with the image's default font, all without a line of the guard or anything on stderr; a negative control, one mismatched callout, shows that such a line would be seen; and `ffi-symbols.st` says that the registry has every function that the image binds in a library of the registry | 12 | 27 s | | 60-vm-harness | `tests/wasm/vm-harness.js`: the web VM and the REPL through `vm-driver.js`, in node, with the default engine and then in Liftoff code only | 32 + 32 | 33 s | | 61-notebook-harness | `tests/wasm/notebook-lib.test.mjs` on the libraries of the Notebook (the brackets, the highlighter and their fuzz, the `.st` and `.json` formats, the frames of `/dev/nbevents`, the PNGs, the sanitizer, in linear time) and, with `HOST_PHARO`, the highlighter against the image's own scanner; then `tests/wasm/notebook-harness.js`, the kernel through `vm-driver.js` (requests, values, errors, Stop, output credit, encodings, rich output, background processes, snapshots, the end of stdin; the FFI, cairo and FreeType of the build) and the page's client against the real `vm-worker.js` in worker threads (the queue, Stop and its watchdog, restarts, exits, floods, uploads, the saved image, the slot that the Console names), with the default engine and in Liftoff code only, on the stock image and on each image directory of `NOTEBOOK_IMAGES` | 28 + 65 + 65 per image | 548 s (2 images) | | 62-web-repl-regress | `tests/wasm/web-repl-regress.js`, through `vm-driver.js` as lane 60: a Stop of an evaluation that has not started yet, a Warning that nothing handles, where a syntax error is, and the file in which the REPL tells the page about its image | 4 | 3 s | | 64-freetype | `tests/wasm/st/ft-parity.st` on the stock and world images: FreeType 2.14.3 draws a line of the default font with the 688 pixels and the hash of the golden file of the image's version (`tests/wasm/golden/ft-p12.txt`, `ft-p15.txt`); with FreeType hidden (`PHARO_WASM_FFI_HIDE=freetype`) the image falls back to StrikeFont; a note of what FreeType costs a boot | 4 | 10 s | | 65-cairo | `tests/wasm/st/athens-parity.st` on the world image: shapes, FreeType text, a repeated gradient, a Roassal canvas, a Mondrian and a PNG drawn by cairo 1.18.4, against the golden hashes (`athens-p12.txt`, `athens-p15.txt`); the functions the adapter adapts (`adapted-65.txt`); with `PHARO_WASM_FFI_ADAPT=0` the gradient fails as a primitive; the stock image, without SurfacePlugin, fails cleanly; with cairo hidden, a SymbolNotFoundError | 5 | 20 s | | 66-libgit2 | `tests/wasm/st/iceflow.st` on the stock and world images: a repository, a package, two commits and a diff, the log, a branch, the status and a file:// clone through Iceberg, against the golden output of the native VM (`iceflow-p12.txt`, `iceflow-p15.txt`); `tests/wasm/st/git-identity.st` on each world image of the site, in an empty `HOME`: a first commit through Iceberg's dialog of the missing properties, global, then a full garbage collection and two more commits (`git-identity.txt`), LGitConfig frees with `config_free:`, whose callout is `git_config_free`, `globalConfigurationPath` answers nil before the dialog and `HOME/.gitconfig` after it, through extensions of OSWindow-Web, logged, that free the `git_buf`, the default action of `IceMergeResolutionRequest` is the one of `web-libgit.st`, with Iceberg's own in a world, a second set up compiles nothing, and a `#webWorldDefaultAction` that sends itself, as an earlier `web-libgit.st` left it, is Iceberg's own again after one more; the same flow on each stock image booted as the Console boots it (`web-repl.st`, a line of its REPL) and as the Notebook's kernel does (`web-notebook.st`, a cell), with `web-libgit.st` next to them, which their boot must have evaluated; each stock image booted as the page boots one that it prepares for the world (as the Console, then `web-bootstrap.st` with the site's `OSWindow-Web.st`), after which `config_free:`, `config_find_global_path:` and `globalConfigurationPath` must be extensions of OSWindow-Web, not in LibGit-Core, which Monticello sees unmodified, and `#webWorldDefaultAction` Iceberg's own, which does not send itself; and on each stock image the control, a commit of which fails once the LGitConfig is finalized when it frees with `object_free:`, and whose `globalConfigurationPath` answers 256 characters of a `git_buf`, not a path; the adapted functions (`adapted-66.txt`) and the control without the adapter; a git:// clone of an unresolvable host fails cleanly; with git2 hidden, a SymbolNotFoundError; with the smart-HTTP transport and a `git http-backend`, `tests/wasm/st/git-http-clone.st` clones over HTTP (curl) from `tests/wasm/lib/git-http.mjs` a remote named without its `.git`, which the server redirects, taking the redirect for the pack, and a missing repository fails with the 404 text; `tests/wasm/st/git-http-push.st` pushes to `git-http.mjs --push --auth`, which asks for HTTP Basic credentials as github.com does, from the stock image booted as the Console boots it (straight to the server) and from the world image (through the proxy of `git-http.mjs`): without credentials the push fails with the 401 of none given, with a wrong password stored with the refusal and what the server said, with the right one in Iceberg's store, with those of the URL of a remote, and with a provider that gives a wrong password then the right one, it pushes (a 401, then the same GET and the POST of the pack with an `Authorization`; two rounds for the latter), a second push through the same git_remote, once nothing is stored, fails with the 401 of none given (the close of the connection forgot the credentials), and the server's own git logs the commits, while no request line or log has the credentials, curl read them from a file (a wrapper of curl logs its arguments) and that file is gone; then a pull fast-forwards to a commit that the lane made on the server, and after a local commit and another on the server the push is refused, the pull merges (2 parents, both methods loaded) and the merge pushes, and a pull whose merge conflicts fails at once with `IceMergeAborted`, the repository as it was; `tests/wasm/st/git-http-hosts.st` (`git-http.mjs --mover`): credentials stored for a host that redirects (its advertisement, or the request that carries them, straight and through the proxy) never reach the git server, credentials never go through a proxy of plain http:// on another computer, and a push that a pre-receive hook declines fails with the transport's 'refused the push of refs/heads/master'; skipped without libgit2 | 13, and 5 for each image of the site (28 for three) | 111 s | | 68-simdjson | the SimdJSONPlugin (`docs/SimdJSON.md`) with the node VM: it is built in, `plugins/SimdJSONPlugin/tests/load-simdjson.st` loads the groups `Core` and `Tests` of the submodule `plugins/SimdJSONPlugin/simdjson.st` with Metacello (`SimdJSONReader` a subclass of `SimdJSONMapper`), the plugin answers the platform `emscripten`, the implementation `fallback` and simdjson 5.0.5; `SimdJSONWriter` writes a value that the plugin writes in part and the image in part (a Set, a Fraction); `tests/wasm/st/simdjson-lane.st` runs the package `SimdJSON-Tests` but `SimdJSONSlowTests` with no failure; a document of 17 MiB is refused under the default input cap, read without it and written back; then the same tests in pharo-web through `vm-driver.js` and the REPL, as lane 60; skipped without `FEATURE_PLUGIN_SIMDJSON` or on an image older than Pharo 15 | 7 | 15 s | | 70-worker-harness | `tests/wasm/storage.test.mjs` on `vm-storage.js` and `catalog.js` (the slots on the in-memory store: a new slot never `main`, which a page of before overwrites, a save, a load and a sync per slot, the guards per slot, rename, remove, `last`, a meta of before the slots, the tree and its paths; the catalog of a manifest with images and of one before them; `PharoGitProxy.parse`, a proxy written as the browser writes it, and `describe`, with the credentials refused through the site's default, those that went through a proxy, told once, and the 403 of the proxy, not of the host); then `tests/wasm/worker-harness.js`: `vm-worker.js` as staged, in worker threads behind a shim of the worker globals: the protocol, Stop, output credit, the downloads, persistence, two workers on one saved image, an image opened with a `.sources` of its own, the preparation of an image for the world and the versions of OSWindow-Web (from 5 to 6, LGitConfig then frees with `git_config_free`, which the set up of the preparation gives it, and a `config_free:` that `web-libgit.st` compiled in LibGit-Core becomes an extension of OSWindow-Web; from 6 to 7, Iceberg's provider of credentials then is `OSWebCredentialsProvider`; from 7 to 8, `config_find_global_path:` then gives `git_config_find_global` an `LGitBuf`, an extension of OSWindow-Web, which fails when the package is not raised to 8, and a `#webWorldDefaultAction` that sends itself is Iceberg's own again after the preparation, which boots the image as the Console), the placeholders of the libraries, and the notebook mode, which stores nothing; and, on sites of its own with a catalog of two images, `init.fresh` and `init.slot`, a slot per fresh image, Save as, two workers on two slots, `resetStorage` of a slot, a meta of before the slots, the files of `/pharo` kept with the image (and the image kept without them when the store has no room for them), the site's default git proxy (through which credentials may not go, `ready.gitCredentials`), the `.sources` a slot gets by name, and a save that never overwrites another image | 21 + 47 | 147 s | | 72-open-image | `tests/wasm/open-image.test.mjs` on `open-image.js` in node: zips (deflated and stored entries, data descriptors, UTF-8 names, directories and the files that the Finder adds, zip64), how the files are paired, every refusal, the image headers, drops, and the build's stock image zipped as files.pharo.org does and opened; `OPEN_ZIPS`, paths separated by colons, adds zips of your own | 11 | 3 s | | 80-world-harness | `tests/wasm/keymap.test.mjs`, then `tests/wasm/world-harness.mjs`, which plays the world page in node with a memory framebuffer, on the default image and then on every other world image of the catalog (`WORLD_IMAGE_ID`), and `tests/wasm/st/osweb-native.st` on the native VM | 16 + 16 per world image + 27 | 20 s (one image) | | 82-osweb-windows | `tests/wasm/st/osweb-windows.st` on the native VM: a second OSWindow without an event handler leaves the world shown, and the window of the Emergency Debugger (in Pharo 15, which has none, a window of the same kind) takes the canvas and the input, then gives them back | 11 | 10 s | The bench measured, on that machine with node 25.2.1, on a build with libgit2 (and SDL2, which the VMs had then): | Measure | This build | Native StackVM | |---|---|---| | `eval '3+4'`, wall time | 325 ms | 80 ms | | 300k-deep block recursion | 189 ms | | | 1M-deep block recursion | 934 ms | 766 ms | | tinyBenchmarks bytecodes/s | 154.5M | 941.0M | | tinyBenchmarks sends/s | 9.5M | 48.0M | | `26 benchFib` | 41 ms | 8 ms | | 20 x 100000 `inject:into:` | 568 ms | 120 ms | | GC stress, 3M arrays | 2510 ms | 1460 ms | The lanes take these settings: - `WASM_CHECK_TIMEOUT`: the seconds that a run of the VM, or of a unit test, may take (default 120); the harnesses that boot several images stop after 300 or 600 s. A run that is stopped then fails its check, or its lane, which says that it timed out. - `WASM_CHECK_TRIES`: the runs that a check that measures a time may take while that time is over its limit (default 3). The best run counts: a loaded machine slows a run down, never speeds one up. - `WASM_BENCH_RECURSION_MS` (1000), `WASM_BENCH_EVAL_MS` (1500) and `WASM_BENCH_SLICES` (10): the limits of the bench. `WASM_BENCH_BYTECODES` (100000000) and `WASM_BENCH_SENDS` (6000000) are those of the tinyBenchmarks warnings, which fail the bench only with `WASM_BENCH_STRICT=1`. - `WASM_FFI_BENCH_MS`: the limit of lane 58's bench of 10000 callouts, in ms (default 10000). - `NOTEBOOK_IMAGES`: image directories, separated by colons, on which lane 61 runs too, after the build's stock image: a Pharo 15 one on a Pharo 12 build, say. `NB_SCAN_METHODS`: the methods of the image whose tokens lane 61 compares with the highlighter (default 5000). `NB_CASES`: a regular expression; lane 61 then runs only the cases of `tests/wasm/notebook-harness.js` whose names match it. - `WASM_CC`: the compiler of lanes 10 and 56, instead of `emcc -m64` from the PATH or `$EMSDK`; an exported `CC` or `CFLAGS` is ignored. Lane 55 compiles with it too. Run by hand, `GEN=build-wasm/host/generated/64 sh tests/wasm/check-types.sh` takes the clang or emcc of `CC` (default `emcc -m64`), and exits 0 without errors, 1 for type errors or a failing compiler, and 2 when it cannot check (a `CC` that is not clang, or no generated interpreter in `GEN`). `run-lanes.sh` empties its `TEST_DIR` only when it is a scratch directory: the default `build-wasm/tests-run`, an empty directory, or one that it used before. One run at a time uses a `TEST_DIR`: a second `make wasm-check` of the same build waits for the first one. The runner exits 2 when it may not use its `TEST_DIR`, or cannot make its lock there. The browser specs drive the pages in real browsers through Playwright, which is not a build dependency, served by `packaging/emscripten/tools/serve.mjs` without COOP or COEP headers (`tests/wasm/lib/pw.mjs`). Each browser gets a context of its own, so its IndexedDB and localStorage start empty, and every spec ends with a check that no page or worker logged an error. No spec may send a request to the public git proxy that a site has by default: the server of `pw.mjs` serves `manifest.json` with the fields of `t.manifestOverride` merged in, by default a `gitProxyDefault` of a dead local proxy (`http://127.0.0.1:9/`), which a spec may change before it opens a page, and every request to `cors.isomorphic-git.org` is aborted and counts as an error. The specs that open images zip the build's stock image, `/../image/stock`, so the web directory they test must have that directory next to it. `make wasm-check-browser` runs, in order: - `tests/wasm/page.spec.mjs`, the Console: evaluation, the keys, history, Stop, Save, Download, the persistence, two tabs, Open with good and damaged zips (each a new saved image), drops, `?slot=`, Delete of one saved image, Images and Save as, Settings and the git proxy (the site's default, None, a proxy of one's own, the notes, and that a `?gitProxy=` is ignored), the tabs, and the Content-Security-Policy; `OPEN_ZIPS` (paths separated by colons, Pharo downloads say) makes it open those zips too. 46 checks per browser. - `tests/wasm/notebook.spec.mjs`, the Notebook tab: the lazy start, Run all on the example, rich outputs, the keys, Stop and its watchdog, the coloring, its contrast and its budget, the caps of the outputs, the sanitizer, the command keys, Upload, the layouts at 390, 360 and 320 px, Restart, Import and Export, autosave across tabs and reloads, Export .html opened from `file://`, the image of the Console, and no worker from `file://` or without memory64. `NB_OPEN_ZIP` names a Pharo zip (of the other version, say) that the Console opens last: the kernel then boots it, and Run all on the example passes on it too; and the kernel boots the Console's `?slot=` after a switch. 58 checks per browser. - `tests/wasm/world.spec.mjs`, the world page, on a build with the world image. It boots the world with `st /pharo/st/world-probe.st` too (`tests/wasm/st/world-probe.st`), which writes `/pharo/probe.json` (the menus, windows, Playground text, debuggers and fonts of the world), and reads it through `window.PharoWorld`, which the world page gives tests: `stats` (the workers started, the frames and when they came, the times from a key to the next frame), `state` (the status pill), `unsaved`, `choice`, `images`, `readFile(path)` and `fs(op, path, fields)`. It checks that FreeType draws the world, and, with cairo, that the Inspector's Roassal canvas, Code Changes and the color picker draw. On a site of several images it checks the launcher (Enter starts the default, Escape does not close it) and starts the other images fresh; then Save, `?slot=`, Save as, Images (open, rename, delete), the link to the Console, a file of `/pharo` kept across a Save and a reload, a `.gitconfig` of `/pharo/home` too, which is the `HOME` of the world (the probe says it), and Settings with the notes of git. 45 checks per browser on the site of three images, 43 on a site of one. - `tests/wasm/ffi.spec.mjs`, on a build with the FFI: a qsort callback, setenv, an adapted callout and a mismatched one in the Console. 6 checks per browser. - `tests/wasm/git.spec.mjs`, on a build with libgit2 and its transport, and a git with `git http-backend` (`tests/wasm/lib/git-http.mjs` serves the repositories, redirects a remote named without its `.git`, takes a push only with HTTP Basic credentials, and runs a CORS proxy that behaves as isomorphic-git's, all on 127.0.0.1, the proxy being the site's default of the spec): by default a clone goes through the site's proxy, and the one-time note says so once; a first commit asks for the user, which Iceberg's dialog of the missing properties (made but not opened) sets globally, and after a full garbage collection, which frees its LGitConfig with `git_config_free` and with no error in the terminal, the next commit works; that user is in `/pharo/home/.gitconfig` after a Save and a reload, and the next commit does not ask; the clone is kept with the saved image, and fetched again after a reload, without the note; with None a clone fails with "request failed (CORS or network)" after one request, and the notice points to Settings; a clone, a fetch and the clone of a redirected remote through the proxy of Settings, and a clone again after a reload; every request reached the git server through the proxy; with the site's default proxy a push fails at its first 401, before Iceberg is asked, the proxy gets no credentials, and the notice offers that proxy, whose button makes it My proxy; then a push without stored credentials fails with the 401 of none given, and with those of Iceberg's store it pushes (a 401, then the GET and the POST with an `Authorization`, through the proxy, no URL with the credentials, the server's git logs the commit, no `failed` for the 401 answered with credentials, and the credentials through the proxy are told once); a pull fast-forwards to a commit of the server, and after a local commit and another on the server the push is refused, the pull merges and the merge pushes; a pull whose merge conflicts fails at once with `IceMergeAborted`; credentials stored for the mover of `git-http.mjs` (localhost) never reach the git server after a redirect of the advertisement, and a request with them that the mover redirects through the proxy, which the browser follows with them, is stopped, with the worker's `redirected` and a notice that says to revoke the token; in a context of the browser without the routes of the tests, a push with credentials through the proxy makes the browser send a CORS preflight that asks for `Authorization`, and lands; in the world (booted with `tests/wasm/st/git-world.st`, whose evaluations run in its UI process), a push without credentials returns at once, Iceberg's dialog opens after it, and what it is given (Ok, made by the spec) is stored and pushes; the Settings of the world page; and, back in the Console, the stock image of the build opened from disk, whose boot gave LGitConfig `git_config_free` in a protocol of LibGit-Core (no package OSWindow-Web, LibGit-Core not modified for Monticello), with which the dialog, a full garbage collection and the next commit work, and the same in a cell of the Notebook, whose kernel boots that image as it was kept. 23 checks per browser, with the one that no page or worker logged an error; with the VM of before the credentials, the four of the push and the pull fail (and the Settings of the world, which then finds the site's default); with the transport of before the checks of the host, the redirected credentials are not stopped; with a proxy that does not allow `Authorization`, the push of the preflight check fails; with the `web-libgit.st` of before, the pull with a conflict never returns; with Iceberg's own provider of credentials in the world, its push never returns (the worker answers no request: the VM is stuck in a slice inside the callback); without the evaluation of `web-libgit.st` at the boot, the last two fail on a stock Pharo 13 or later (Chromium crashed the tab, Firefox failed the commit after the garbage collection). `tests/wasm/pages-regress.spec.mjs` holds regression checks of the pages (the live regions, input methods, output while the page is hidden, the questions before the page goes, Delete after a failed boot, two sites of one origin, a closed database connection, a browser without DecompressionStream, a notebook cell of 5 MB of output while hidden, the world's progress bar, F6, a refused clipboard write, a world that does not open, and the image saved by a build of before the saved images, which Images lists as "Saved image" and the world boots), and runs the same way; `make wasm-check-browser` does not run it. `PLAYWRIGHT_MODULE` names the Playwright package, and `BROWSERS` the browsers (default `chromium`); Playwright's own variables, such as `PLAYWRIGHT_BROWSERS_PATH`, apply: PLAYWRIGHT_MODULE=/path/to/node_modules/playwright BROWSERS=chromium,firefox make wasm-check-browser or for one spec, on any staged site: PLAYWRIGHT_MODULE=/path/to/node_modules/playwright BROWSERS=chromium,firefox node tests/wasm/page.spec.mjs build-wasm/web On the site of three images with libgit2, in Chromium and Firefox together, the Console spec passed its 92 checks, the Notebook spec its 116, the world spec its 90, `pages-regress.spec.mjs` its 37, the git spec its 46 in 88 s and the FFI spec its 12 in 7 s; on a site of the pinned Pharo 12 with libgit2, the same, but the world spec's 86. `SHOTS=1` saves screenshots of the pages, light and dark, at 1280 and 360 px, in `build-wasm/tests-run/shots`. ## Limitations - StackVM only: 5 (sends) to 6 (bytecodes) times slower than the native StackVM, and 14 times (bytecodes) to 27 times (sends) slower than the native JIT on tinyBenchmarks; on the machine the port was made on, the JIT ran 2181M bytecodes/s and 260M sends/s. - No `dlopen`, and no external plugins: every plugin is built in, and the FFI reaches only the libraries and functions of its registry (see The FFI); a library the image loads by another name, or a function that no list names, is not found. - A declaration that does not match its C function other than in the width of an integer fails as a primitive, where natively it may work; a callback declared with another type than C's traps, and fails its callout. - A callback keeps the VM from its host until it returns: no output, input or Stop meanwhile. - No name resolution (`getaddrinfo` is disabled), and sockets are of little use in browsers. - No `fork`, OSProcess, SSL, SSH, threads or TFWorker, no SDL2, and no external surfaces (SurfacePlugin is left out). - libgit2 reaches http:// and https:// remotes only, through a CORS proxy for a server of another origin (by default a public one, which sees what is cloned); credentials (a push, a private repository) go only through a proxy that the user chose, or none, and Iceberg asks for them in the world only after the operation failed (see libgit2). Through a proxy, a host that redirects a request with credentials to another host gets them sent there once by the browser before the transport stops (see Push and credentials); a merge with conflicts is resolved in the world only (the Console and the Notebook abort it). Not tried against github.com, GitLab or Codeberg with a real token: the tests push to a local `git http-backend` that asks for Basic credentials, and github.com was seen to answer a wrong token through the proxy of isomorphic-git. - A saved image keeps the files of its directory, its clones too, as they were at its last save: what was cloned or committed since is lost on a reload. At most 256 MiB and 50000 files are kept; beyond, the save keeps the image and the files of the save before, and says so. The files come back with their modes and modification times, but not their change times (ctime), which are those of the restore: git takes the files of a restored clone as possibly changed, and reads them again at a status (until its index is written again), which is slower but finds no change. - Old space is at most 512 MiB by default, and perm space is not supported (see `WASM_OLD_SPACE_BASE`). Spur asks for more old space than an object needs (512 MiB for a 256 MB ByteArray), so a single object of 256 MB already fails with the default base, in both VMs. Objects that the scavenger moves to a full old space get no `OutOfMemory`: the VM aborts, as a native one does when the system has no more memory to give. An endless recursion gets there in a second or two (about 2 million contexts a second, of some 230 bytes each, with node's default engine); in the Notebook the cell ends `crashed`, and the next run starts a new kernel. The VM has about 710 MB of linear memory once booted, and the Notebook's kernel as much again. - The SimdJSONPlugin parses no JSON value of more than 16 Mi characters or bytes (`SimdJSONReader maxInputUnits`): simdjson takes some 14 times the size of what it parses from the C heap, which grows into the window of old space and never gives it back. The cap applies to each value, before any parse, so a stream of small values (NDJSON) of any length is read by the plugin; a larger value signals a `SimdJSONResourceError`. For larger documents raise the cap, and old space with it (`PHARO_WASM_OLD_SPACE_BASE=0x40000000`); reading one blocks the page meanwhile (see `docs/SimdJSON.md`). - The clock has a resolution of 1 ms, and browsers clamp nested timers, which wake a sleeping VM, to 4 ms: 200 Delays of 1 ms took 1135 ms in Chromium and 1333 ms in Firefox. In Firefox, `performance.now()`, the monotonic clock of the VM and of its heartbeat, steps by 1 ms in a worker of a page that is not cross-origin isolated (the pages are not), against 0.1 ms in Chromium. - A long primitive (a huge `LargePositiveInteger` operation, say) does not yield: Stop then replaces the worker after 3 s, losing what was not saved. - Stop ends the evaluation, not the processes that it forked: one that never waits keeps the VM busy until Restart. - The saved images live in the browser of one site directory, shared by all its tabs and pages: two tabs that save the same saved image, the last save wins. They are not compressed (about 60 MB each, more with their clones), and the browser may clear them when the storage runs short, unless it was asked to keep them (Images). - A tab of a build of before the saved images, still open when the site is updated, deletes every saved image with its Reset, which knew of one only, until it is reloaded. - The Notebook: - its cells have no stdin: a cell that reads it says so, and waits until it is stopped; - a process that a cell forks and that waits on stdin in the background takes the bytes of the page's next request, which the kernel then refuses as a bad request: the cell sent never ends, until Stop and the watchdog, or Restart; - a block of an earlier cell that returns with `^` (a non-local return) has no cell to return to, and fails with BlockCannotReturn in the cell that calls it; an exception or a context kept from an earlier cell cannot be resumed; - what a background process writes on `Stdio stdout` or `Stdio stderr` (not on the Transcript) goes to the cell that runs then, or to the kernel's messages; - a cell can write events of its own on the kernel's device, a done of its own count say, which ends that cell early; the page checks every event, and the cells after it get their own results; - a huge output of right-to-left text takes long to lay out in Chromium: 500000 characters written in one line took 9.7 s (Hebrew) and 24.3 s (Arabic) to show, against 0.14 s for Latin text, and 1.0 s and 0.9 s in Firefox. - Safari is untested. - Input methods commit their text when the composition ends, touch gestures are not supported, and touch screens are untested. ## Licences `THIRD-PARTY-NOTICES.txt`, in `node/` and `web/` and linked from the pages, gives the licences of the software that the WebAssembly modules contain besides the VM, which is under the MIT License (`LICENSE`). `cmake/emscripten/stage.cmake` assembles it at configure time: `packaging/emscripten/THIRD-PARTY-NOTICES.head.txt`, then, for each library linked into the VM, its version, the URL of its archive and the SHA256 of that archive, with the full text of the licence files the library declares (`pharo_wasm_dep_fetch(... LICENSES ...)`), then the runtime of Emscripten: its JavaScript support code, the C library (musl) and the compiler runtime (compiler-rt), and, with the SimdJSONPlugin, the C++ library libc++ and libc++abi. It is 210374 bytes on a build with libgit2, 46441 of which the SimdJSONPlugin adds (simdjson, libc++ and libc++abi); a default build had 102448 bytes before the SimdJSONPlugin, and 148932 with it. Most libraries are under permissive licences: libffi and pixman under MIT licences, FreeType under the FreeType License, zlib under the zlib licence, libpng under its own, simdjson under the Apache License 2.0 or the MIT License, at your choice, libc++ and libc++abi under the Apache License 2.0 with LLVM Exceptions (their older code also under the University of Illinois/NCSA or the MIT licence), and the http-parser that libgit2 bundles under an MIT licence. Two components are under the LGPL or the MPL: - cairo, available under the GNU LGPL 2.1 or the MPL 1.1; the notices give its `COPYING`, `COPYING-LGPL-2.1` and `COPYING-MPL-1.1`. - LibXDiff, which libgit2 includes (its `src/xdiff`), under the LGPL 2.1 or later; libgit2 itself is under the GPL 2 with the linking exception of its `COPYING`, which also holds the text of the LGPL 2.1, and the notice of LibXDiff's sources is added to it (`xdiff-NOTICE.txt`). Their sources, as those of every library, are those of the pinned archives that the notices name, with their SHA256, built as `cmake/emscripten/deps` of this repository says, and are available on request from whoever distributed the build. ## Troubleshooting - `... lacks the Emscripten hooks of the Slang sources ... it was generated by a VMMaker image older than this tree`: the sources given (`WASM_GENERATED`, or `GENERATED_SOURCE_DIR` on the CMake-only route) predate this tree's `smalltalksrc`. Run `make wasm` without `WASM_GENERATED`, which refreshes the VMMaker image and generates the sources again. Sources generated by an older version of this tree may have the hooks but not every later change: lane 25-session-id, for one, fails on sources generated before the session-ID fix. - `emcmake (...) not found: source emsdk_env.sh of the Emscripten SDK ..., or set EMCMAKE`: the shell has not sourced `emsdk_env.sh`. - `CMake Error: The source directory ".../cmake/Emscripten.cache.cmake" is a file, not a directory.`, or `The source directory ".../host" does not exist.`: the CMake is older than 3.13, which `-S` and `-B` need. - `download.cmake: cannot download ...`, or a VMMaker bootstrap that cannot reach the network: build offline (see Building). - `Cannot download () from : ...; to build offline, put in a directory and give it as WASM_DEPS_DIR`: the archive of a library could not be downloaded; get it elsewhere, check its SHA256, and give its directory as `WASM_DEPS_DIR`. - ` is not (): its SHA256 is ..., not ...`: the file of that name in `WASM_DEPS_DIR`, or the one downloaded, is not the pinned archive; put the pinned one there, or remove it from `WASM_DEPS_DIR` to download it. - `WASM_DEPS_DIR: ... does not exist`: give an existing directory, or leave the setting out. - `WASM_CAIRO=ON needs FreeType, which WASM_FREETYPE=OFF leaves out`: give `WASM_CAIRO=AUTO` or `OFF`, or keep FreeType. - `WASM_EXTRA_IMAGES: '...' is not =`, `... the id of '...' is not 1 to 16 lower-case letters or digits or dashes (not first)`, `... is sources`, `... an id is given twice` or `... does not exist`, and `WASM_IMAGE_ID (...) is not ...`: give each extra image as `=`, with an id of its own, and a zip that exists at a path without blanks. `WASM_EXTRA_SOURCES: '...' does not exist or is no .sources file`: name `.sources` files that exist. `WASM_GIT_PROXY (...) is not an http:// or https:// URL without a user name or password or a #fragment`, or `WASM_GIT_PROXY (...) is refused: stage.mjs: ...` (a port or a host that is none): give such a URL, or `WASM_GIT_PROXY=` for none. - `stage.mjs: the images and have two different .sources`: two images of the site have `.sources` of one name that differ; `--extra-sources ... is not the of the site`: a `WASM_EXTRA_SOURCES` file has the name of an image's `.sources` but not its content; `... is not ...: the images of the world must be prepared with the same OSWindow-Web`: an extra world image was prepared by another tree (make it again); `--extra-image has the id of the default image`: give `WASM_IMAGE_ID` or the extra image another id. - `WASM_BUILDDIR (...) is not empty, and make wasm never built there`, and the other refusals of `WASM_BUILDDIR`: name a new or empty directory, or create `/.make-wasm` to build in a directory anyway. `WASM_BUILDDIR is empty`: leave the setting out for `build-wasm`. `WASM_BUILDDIR (...) holds a blank`, or `... is relative to the current directory, ..., whose path holds a blank`: name a path without blanks, or an absolute one. - `make wasm cannot find its source tree (...): the path of the tree must hold no blank`: move the source tree to a path without blanks. - `internal error: invalid --jobserver-auth string 'fifo:...'` from `make -jN wasm`: that make is not the one that CMake chose for the trees (their `CMAKE_MAKE_PROGRAM`), and the two do not share job slots in the same way (make 4.4 over trees configured with make 4.3, say); use that make, or run without `-j`. - The page says that it must be served over HTTP: it was opened from a `file://` URL. Run `make wasm-serve`, or serve `build-wasm/web` with any static web server. - The Console says that the browser has no 64-bit WebAssembly memory: the build has `WASM_WEB_MEMORY64=1`. Build with the default, `2`. - The page says that the browser cannot inflate the files of Pharo: the browser has no DecompressionStream (see the engines above). - Firefox warns that the WebAssembly `try` instruction is deprecated: the build has `WASM_SJLJ=wasm-legacy`. - The world page says that the build has no world image: the build had `WASM_WORLD=OFF`, or no `WASM_HOST_PHARO`, as with `WASM_GENERATED` alone (CMake said `WASM_HOST_PHARO is not set: the world image is not prepared, and web/ gets the stock image`). - `FFI callout failed, its declaration does not match function N: ...` or `FFI callout trapped in function N: ...` on stderr (the worker's console in a page), and a primitive failure: the image declares that function otherwise than C does, in a way that the adapter does not adapt; `PHARO_WASM_FFI_TRACE=1` in the node VM shows the calls it does adapt. - A `SymbolNotFoundError` for a function of a library: the function is in no list of the registry, or the build has not that library (`manifest.json` lists the libraries of the site, and the configuration prints `wasm deps: ffi ON, freetype ON, cairo ON (auto), ...`). - Iceberg says `There is no libgit2 available in your system!`: the build has no libgit2; build with `WASM_LIBGIT2=ON`. - A clone says `git http: request failed (CORS or network): ... choose one in Settings (Git HTTP proxy)`: no proxy is chosen (Settings says None, or the site has no default), and the remote sends no CORS headers; choose the site's default or a proxy of your own in Settings. `git http: request failed (CORS or network) through the proxy : ...`: the proxy did not answer, or did not let the page read its answer; try again later, or choose another proxy. `git http: the proxy refused the request to (403)`: the proxy forwards only git requests of public repositories, or not to that host; `git http: answered 401: ...` or `... 403: ...`: the repository is private, missing, or the host limits its rate; `git http: no git repository at (404)`: the URL of the remote is wrong. The page names the proxy that the requests go to in its notes and in Settings. - With `WASM_DEBUG=1`, the time checks of the lanes (S6b, S8 and the bench) and the checks of an empty stderr (S2, S13) fail: the VM is many times slower with its asserts (lanes 20, 30, 40 and 80 alone took 17 minutes, and failed so), and Emscripten's assertions print "Heap resize call ..." and "program exited (with status: 0), but keepRuntimeAlive() is set ..." when the image quits. In lane 80, the world harness fails its keystroke median, and its cases that wait for the world (Stop, the save) can time out. - `run-lanes: waiting for the run N ...`: another run uses that `TEST_DIR`; if it is gone, remove the lock that the message names. - An `OutOfMemory` for a large object, and the VM logs `Cannot allocate N bytes of old space at P ...: old space must fit in [base, 2*base)`: raise the old-space base, for example with `PHARO_WASM_OLD_SPACE_BASE=0x40000000` (1 GiB of old space). - `Cannot allocate N bytes of perm space at 0x20000000000: WebAssembly memory cannot reach it (perm space is not supported)`: the image has perm space, or the VM was given `--minPermSpaceSize`; neither works in WebAssembly. - A crash prints at most 1000 frames per process: pass `--maxFramesToLog=` for more, or `--maxFramesToLog=0` for all of them.