#+bibliography: bibliography.bib

#+BEGIN_EXAMPLE
         █     █░ ██▓  ██████  ██▓███
        ▓█░ █ ░█░▓██▒▒██    ▒ ▓██░  ██▒
        ▒█░ █ ░█ ▒██▒░ ▓██▄   ▓██░ ██▓▒
        ░█░ █ ░█ ░██░  ▒   ██▒▒██▄█▓▒ ▒
        ░░██▒██▓ ░██░▒██████▒▒▒██▒ ░  ░
        ░ ▓░▒ ▒  ░▓  ▒ ▒▓▒ ▒ ░▒▓▒░ ░  ░
          ▒ ░ ░   ▒ ░░ ░▒  ░ ░░▒ ░
          ░   ░   ▒ ░░  ░  ░  ░░
            ░     ░        ░
#+END_EXAMPLE

*Wisp* is a Lisp system for [[https://webassembly.org][WebAssembly]] and native execution.

It aims to offer an unusually pleasant developer experience
both while hacking and in production, though it's still /very
immature/ and far from ready for any serious use.

You can try the live browser-based IDE at [[https://wisp.town][Wisp.Town]].

Feel free to join the [[https://discord.gg/BeKtM3x687][Wisp Discord chat]] if you're interested
in following the development work, hacking on the system, or
chatting about Lisp, WebAssembly, and Zig.

Wisp has:

  - a browser-based IDE for live hacking and interactive
    debugging;

  - saving and loading of full system state images, including
    debugger state, also in browser local storage;

  - thoughtful integration with the browser environment, Node,
    Deno, and Unix in general;

  - first-class delimited continuations for advanced control
    flow;

  - interactive restartable conditions in the style of Common
    Lisp;

  - first-class stepwise execution for custom debuggers,
    schedulers, etc;

  - sandboxed execution with fine-grained capabilities via
    WebAssembly;

  - an implementation written in [[https://ziglang.org][Zig]];

  - and more!

* Building

Wisp currently requires Zig =0.16.0=.  The exact lower bound is
also recorded in =core/build.zig.zon=.

#+BEGIN_SRC sh
make
make test
#+END_SRC

* Segmented continuations

The evaluator keeps an ordinary frame segment in =run.way= and a
meta-continuation in =run.meta=. Prompts and dynamic bindings suspend
the current segment and begin another. Returning from a segment
restores the suspended one; only exhausting both registers finishes
the run.

Sending to a prompt searches dynamic boundaries, not ordinary
frames. Capture and composition share segments and copy only the
intervening meta spine. Captured frames and argument vectors are
protected by copy-on-write: capture freezes existing frames with an
allocation watermark, and execution copies a frozen frame only when
it needs to modify it. Ordinary uncaptured calls keep their mutable
argument-vector fast path. GC and image loading freeze restored
frames too.

The matching prompt is excluded from a capture; deep effect handlers
continue to reinstall it explicitly on resumption. Captured dynamic
bindings retain their values independently across resumptions, while
lexical assignment remains shared store. =get/cc= now returns a
stable snapshot rather than an alias to advancing control frames.
=KTX-*= and =RUN-WAY= expose a flattened debugger view, hiding the
internal composition boundaries.

The added run register changes the image layout to tape =v0.9.0=.
Older tapes are rejected explicitly; there is no image migration.

* Variable scope and evaluation

Function parameters and =let= bindings are always lexical. Reads and
=set!= first resolve the nearest captured lexical binding, then an
active dynamic binding when the symbol is declared dynamic, then the
global symbol value. Assignment to an otherwise unbound variable
still signals an error. =let= initializers run in the outer scope,
and closures share their captured mutable lexical store.

=defparameter= marks a symbol for dynamic lookup; it does not turn
same-named lexical binders into dynamic bindings. =binding= explicitly
installs dynamic values. Changing =set-symbol-dynamic!= cannot redirect
lexical reads or writes, but still changes free-variable lookup at
runtime. Free-variable classification is not frozen at expansion time.
Code that previously used a dynamic binding to override a lexical
local must remove or rename that local.

Public =eval= evaluates source without caller lexical bindings, while
retaining the current dynamic bindings, effect handlers and continuation.
It cannot read, assign or capture the caller's locals implicitly.
Macro-result evaluation is different: expansions still execute in the
lexical environment of the macro call site. =env=, =KTX-ENV= and =run=
remain explicit reflective operations; ordinary =eval= is not a debugger
"evaluate in this frame" operation.

Ordinary calls resolve the symbol's live function cell before evaluating
arguments. New calls observe redefinitions, but a saved function object
or an invocation already evaluating its arguments retains its callee,
including across continuation capture and multi-shot resumption.

* Macro expansion

=defun= pre-expands its body, including nested callbacks. =fn=
also pre-expands its body when the function is constructed, so
standalone lambdas do not repeatedly expand their macros on
every call. Quoted data, parameter lists, and function names
are not walked as expressions. =%fn= remains the low-level
constructor for an explicit function expression.

Pre-expansion shares a budget of =*macroexpand-limit*= macro
applications (256 by default) across the entire walk and any
reentrant expansion. This bounds macros that reproduce
themselves or branch recursively. On exhaustion, remaining
macro forms are left for normal runtime evaluation rather
than rejected. The limit can be dynamically bound when
calling =macroexpand-completely=; zero disables the walk.
There is no repeated fixed-point pass that restarts the budget.

Ordinary recursive helper functions used by macros still run
normally: the walker expands syntax, not function calls. The
budget does not interrupt a macro implementation that itself
never returns.

* Evaluator profiling

Run the diagnostic evaluator suite with semantic counters enabled:

#+BEGIN_SRC sh
make bench
#+END_SRC

The benchmark runner writes one JSON object per case.  A single
case and iteration count can be selected explicitly:

#+BEGIN_SRC sh
cd core
zig build bench -Doptimize=ReleaseFast -- call-16 25000
zig build bench -Doptimize=ReleaseFast -- --list
#+END_SRC

The counters cover evaluator steps, calls, argument-list scans and
reversals, lexical lookup depth and comparisons, continuation
searches, captures, and copied frames, logical heap allocations, GC
copies, and GC time.  The final bucket in the =call_arity= and
=lexical_depth= histograms means 16 or more.  Core loading,
benchmark setup, and parsing are excluded from the counters and
elapsed time.

The =effect-shallow= and =effect-deep= diagnostics repeatedly resume
a deep handler with zero or 64 pending arithmetic frames. The
=continuation_boundaries= counter reports search work; the retained
=continuation_frames= counter is zero with segmented capture.

The suite has two layers.  The =call-*= and =lookup-*=
diagnostics isolate evaluator mechanisms.  Four small
Gabriel-derived programs exercise them in recognizable Lisp
workloads:

- =tak= uses the historical =(18 12 6)= input and stresses
  non-tail calls and argument binding.
- =deriv= performs symbolic differentiation and allocates syntax
  trees.
- =diviter= and =divrec= divide a prebuilt list of 1000 =NIL=s,
  contrasting tail-recursive and non-tail-recursive traversal.

Four repository-derived cases exercise Wisp as it is used here:

- =stdlib-list= runs a map/append/reverse/remove pipeline using the
  boot core's list functions.
- =backquote= calls the boot core's actual
  =bq-completely-process= implementation on a nested unquote and
  unquote-splicing form.
- =router-hit= and =router-miss= preserve =web/http.wisp='s route
  matcher and nested continuation-prompt search, with fixed data in
  place of the Deno request and response bridge.

Each program checks its result after timing.  When no iteration
count is supplied, each case uses a workload-appropriate default;
an explicit count still overrides it:

#+BEGIN_SRC sh
cd core
zig build bench -Doptimize=ReleaseFast -- tak 1
zig build bench -Doptimize=ReleaseFast -- deriv 100
#+END_SRC

The JSON =input= field identifies the fixed program workload.
Sources are adapted from the
[[https://github.com/ecraven/r7rs-benchmarks][R7RS benchmark collection]]; the repetition count is the
harness parameter and is not part of that fixed input.

For an uncontaminated native CPU profile, compile the semantic
counters out and record a sufficiently long single case:

#+BEGIN_SRC sh
cd core
zig build bench-build \
  -Doptimize=ReleaseFast \
  -Dsemantic-profile=false
xcrun xctrace record \
  --template 'Time Profiler' \
  --output /tmp/wisp-call-16.trace \
  --launch -- \
  "$PWD/zig-out/bin/wisp-bench" call-16 5000000
#+END_SRC

The resulting trace opens in Instruments.  On Linux, the older
=etc/profile= script provides a starting point for recording with
=perf=.

** Cross-language sweep

Run the Gabriel workloads in Wisp, Python, Ruby, Tcl, JavaScript,
Wren, Chibi-Scheme, Racket, Common Lisp, and optimized C.  The
repository-derived Lisp cases run in Wisp, Racket, Chibi-Scheme, and
Common Lisp; the continuation router currently runs in Wisp and
Racket:

#+BEGIN_SRC sh
make bench-sweep
#+END_SRC

The sweep verifies results, takes five samples by default, prints
per-case rankings, a Wisp-focused comparison with the slowest other
runtime in each case, and a normalized geometric mean for runtimes
that support every selected case.  It saves raw JSONL samples outside
version control.  Unsupported cases are omitted rather than replaced
with unlike mechanisms.  Its pinned Wren and Chibi-Scheme runtimes
are built locally on first use, not installed system-wide.  See
[[file:benchmarks/cross-language/README.md][the cross-language benchmark notes]] for workload and
comparability details.

* Browser development

Start the Wisp development server and open
[[http://127.0.0.1:8765/]] in a browser:

#+BEGIN_SRC sh
./wisp-web start
#+END_SRC

Evaluate Wisp inside that live browser image from the shell:

#+BEGIN_SRC sh
./wisp-web '(+ 20 22)'
./wisp-web - < program.wisp
#+END_SRC

The result comes back as a Wisp form rendered by Wisp's own
pretty printer. Conditions remain suspended in the browser and
can be resumed from a later shell invocation:

#+BEGIN_SRC sh
./wisp-web '(oopsfail!)'
./wisp-web tickets
./wisp-web use 1 42
./wisp-web retry 1
./wisp-web abort 1
#+END_SRC

* GitHub Pages

Every push to =master= builds, tests, and publishes the browser
workbench at [[https://mbrock.github.io/wisp/]].  The workbench
uses only relative asset URLs, so the same files work at the
repository subpath and at a custom domain.

The listener's =download= button compacts the active Wisp system
and downloads its complete tape image as a =.core= file.  A
small service worker keeps the static workbench available after
it has been visited once.  Local named images can later use the
same snapshot bytes in IndexedDB without changing the Wisp tape
format.

* Arrow heap export

Export the native boot heap as one Apache Arrow IPC file:

#+BEGIN_SRC sh
./core/zig-out/bin/wisp arrow wisp.arrow
#+END_SRC

The file contains one nested heap record.  Its byte and word
arenas are Arrow arrays, and every Vat table is a list of
structs backed by the table's physical columns.  Zig presents
those existing buffers through the Arrow C Data Interface;
vendored nanoarrow 0.8.0 writes the IPC envelope.  The included
=core/arrow.sql= adds DuckDB views for the nested tables, Wisp
word decoding, strings, and symbols:

#+BEGIN_SRC sh
duckdb < core/arrow.sql
#+END_SRC

* Local Variables :noexport:
  Local Variables:
  fill-column: 62
  End: