✳   C R E A T U R E   L A B

Small creatures. Big questions.

A fruit-fly research playground for body physics, neural navigation, and learning through consequences.

Explore the showcase · Run locally · Read the methods · Credits

![Creature Lab home page in light mode](lab/assets/hub-light.png) ## Three ways to explore | Experiment | Inside the world | What you can do | | --- | --- | --- | | **01 · The walking world** | NeuroMechFly body geometry and recorded MuJoCo movement | Orbit, pan, zoom, replay, and design terrain. New physics runs need the optional simulation environment. | | **02 · A way through** | Measured FlyWire connectivity, simulated spikes, and a trained motor readout | Replay ten mazes; disconnect connections or silence descending output and compare the recorded outcomes. | | **03 · Practice makes progress** | A fruit fly in a classic Pac-Man-style maze with four distinct ghosts | Watch a trained policy, play with the keyboard locally, inspect rewards, and continue training on a configured server. | All three share a green visual language, persistent **light / dark** modes, and a home page. The maze and physics viewers support **middle-drag to pan**, orbit, zoom, and fullscreen. The **public showcase plays saved recordings**. It does not run a Python backend or launch training. Run the project locally for live games and server controls.
See the experiments ### 3D terrain lab ![3D terrain, physics replay and neural telemetry](lab/assets/terrain.png) ### Neural maze ![Recorded neural maze with causal controls](lab/assets/brain.png) ### Fly Pac-Man ![Classic arena with a fruit-fly player and reward telemetry](lab/assets/pacman.png) ### After dark ![Creature Lab in dark mode](lab/assets/hub-dark.png)
[Watch the 19-second visual tour](docs/creature-lab-tour.mp4) — interface stills combined with recorded 3D body replay. ## Does training help? In the saved `classic-more-v1` experiment, it did: | Same 200 fresh ghost seeds | Wins | Win rate | | --- | ---: | ---: | | Previous trained readout | 60 / 200 | **30.0%** | | Continued, selected readout | 97 / 200 | **48.5%** | | Difference | +37 wins | **+18.5 percentage points** | The continuation ran **20,000 additional training games**. A separate validation set selected the checkpoint after **5,000** additional games; the final comparison used seeds **90000–90199**, distinct from training and validation. The older 32% figure came from a different 50-game cohort and is not the matched baseline above. These results apply to this single simulated layout and rule set. They do not establish performance on unseen layouts or exact arcade emulation. Read [the saved result](lab/pacman-runs/classic-more-v1/result.json) and [method notes](docs/METHODS.md). ## Run it locally **Python 3.12 recommended. No npm install, GPU, or third-party Python packages are needed for the saved replays and live Pac-Man.** ```sh git clone https://github.com/RathaTart/creature-lab.git cd creature-lab python start.py ``` Open **http://127.0.0.1:8788/lab/**. On macOS/Linux, use `python3` if that is your Python command. Stop both services with `Ctrl+C`. | Route | Experience | | --- | --- | | `/lab/` | Creature Lab home | | `/demo/` | 3D terrain editor and physics replay | | `/lab/brain.html` | FlyWire neural maze | | `/lab/pacman.html` | Fly Pac-Man | The launcher starts the loopback gateway on **8788** and terrain service on **8766**. If 8766 is already occupied, it reuses that service; ensure it is the intended terrain server. `python start.py --no-terrain` starts only the gateway. `--port 8791` changes the gateway port. **Pac-Man controls:** choose *You play*, then arrows or WASD. Space pauses. Choose *Fly brain* to use the saved motor readout. *Start a fresh game* resets the arena with the selected ghost seed. Saved evaluation replays are separate recordings. ### Optional: calculate new body-physics runs Use a separate virtual environment and install the pinned FlyGym source revision: ```sh python -m venv .venv # Windows: .venv\Scripts\activate # macOS / Linux: source .venv/bin/activate python -m pip install -r requirements-physics.txt python start.py ``` FlyGym may download its model assets on first use. The **A* baseline** can calculate new terrain runs without FlyWire data. The neural controller additionally needs the data described below. Existing recordings work without these packages. The original simulation environment is recorded in [demo/requirements-lock.txt](demo/requirements-lock.txt); that lock contains platform-specific packages and is evidence, not a universal installer. ### Optional: use your tailnet training server Copy `local-config.example.json` to **`local-config.json`**, then set your SSH host, key path, remote workspace, and Python interpreter. This file is ignored by Git. The environment variables `CREATURE_HOST`, `CREATURE_SSH_KEY`, `CREATURE_REMOTE`, and `CREATURE_PYTHON` override those values. Without configuration, local play works and remote training requests return a clear setup message. On the Linux server, place a checkout in the configured workspace and install `requirements-training.txt` in its Python environment. Full brain simulations require CUDA-enabled PyTorch and enough GPU memory; the recorded experiment used an 8 GB GPU. Pac-Man continuation reuses cached neural responses and trains the readout on CPU. The workers use Linux `fcntl` locks, so remote workers are not native Windows programs. For fresh neural responses, obtain the FlyWire v783 data through the [FlyBrain data pipeline](https://github.com/snedea/flybrain), preserving the source data terms. Put `neurons.csv.gz`, `classification.csv.gz`, and `connectome.bin.gz` in `/data/`. The original source graph checksum is retained in the recorded manifests. Raw connectome downloads and machine-specific credentials are not included here. See [FlyWire's citation and data-use guidance](https://home.flywire.ai/guidelines). Restart the gateway after changing configuration. The UI can then train the maze readout or continue Pac-Man for 5,000, 10,000, or 20,000 games. Jobs save checkpoints remotely and the gateway downloads completed results. **Training changes the artificial motor readout; it never trains a living fly or rewires the measured connectome.** ## What is real, simulated, and learned? ```text Measured FlyWire connectivity ↓ Custom simulated spiking neurons ← engineered sensory cues ↓ Descending-neuron response ↓ Artificial motor readout ← imitation or numerical rewards ↓ Maze actions / arcade actions / body controller ``` - **Measured:** anatomical connectivity from the FlyWire adult female brain dataset. - **Simulated:** neuron dynamics, synthetic sensory stimulation, game worlds and body physics. - **Learned:** an artificial output readout. The anatomical graph stays fixed. - **Recorded:** the maze decisions and body replays shown in the viewers. Live Pac-Man uses a bank of 64 cached neural responses rather than simulating the whole brain every frame. - **Dopamine-inspired:** a numerical reward-prediction error during readout training. No biological dopamine circuit or injection is modeled. The 3D terrain experiment supports both an experimental neural controller and a separate A* baseline. Grid-maze legs are illustrative animation; terrain recordings contain calculated body transforms. These distinctions are shown in the interfaces and expanded in [METHODS.md](docs/METHODS.md). ## Project map ```text lab/ Hub, neural maze, arcade, gateway and training workers assets/ Actual interface screenshots brain-runs/ Published neural-maze recording and checkpoint pacman-runs/ Published paired evaluation and frozen policies demo/ Terrain editor, body viewer and physics adapter js/vendor/ Three.js and OrbitControls flynn/core/ Attributed A* planner dependency docs/ Methods, preview clip and license notices scripts/ Static showcase packaging and verification start.py Local launcher ``` ## Check the project ```sh python -m unittest discover -s lab -p test_pacman_classic.py python -m unittest discover -s lab -p test_pacman.py python scripts/check_project.py python scripts/build_showcase.py ``` The static build is written to `_site/` and contains an explicit set of viewer assets and recordings. The GitHub Actions workflow publishes that directory to Pages. It does not ship SSH configuration or Python services. ## Credits & reuse Created by **[Tart / RathaTart](https://github.com/RathaTart)**, building on the work of the FlyWire community, FlyBrain, Shiu and colleagues, NeuroMechFly/FlyGym, FLYNN, and Three.js. The fly character artwork and hub illustration are original code-based visuals. Original application code is MIT-licensed. **This does not relicense the upstream data or assets:** FlyWire data and derived research artifacts retain the applicable **CC BY-NC 4.0** conditions; NeuroMechFly body assets retain Apache-2.0. See [THIRD_PARTY.md](THIRD_PARTY.md) for attribution and scope. Pac-Man is an arcade-inspired research experiment, not an official Namco/Bandai Namco product. Contributions are welcome: include a reproducible example, keep the scientific claims tied to measured evidence, and run the relevant checks before opening a pull request.