# 🪰 DOOM-FlyWire: 139,248-Neuron Biological Connectome Autonomous Agent & 3D Real-Time Visualizer [![Connectome: FlyWire Nature 2024](https://img.shields.io/badge/Connectome-FlyWire%20(Nature%202024)-00f5d4.svg)](https://flywire.ai) [![Environment: ViZDoom](https://img.shields.io/badge/Environment-ViZDoom%20(ZDoom%20RL)-ff0055.svg)](https://vizdoom.farama.org) [![Scale: 100% Unreduced](https://img.shields.io/badge/Scale-139%2C248%20Neurons%20%7C%2015.1M%20Synapses-blueviolet.svg)](#system-architecture) [![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Live%20Demo-yellow.svg)](https://huggingface.co/spaces/emiraran/doom-flywire) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) A real-time, closed-loop biological connectome simulation running the complete, unreduced adult *Drosophila melanogaster* whole-brain connectome ([FlyWire Consortium, *Nature* 2024](https://www.nature.com/articles/s41586-024-07558-y): **139,248 neurons, 15,090,883 directed synapses**) as an autonomous agent playing **DOOM** ([ViZDoom](https://vizdoom.farama.org/)). The project features a real-time, browser-based **3D Connectome Neural Activity Visualizer** (Three.js WebGL) rendering all 139,248 biological neurons in 3D space with dynamic, calcium-fluorescence-style excitation halos as the fly perceives, processes, and navigates the 3D environment. --- ## ⚡ Key Highlights - **100% Complete Biological Wiring:** Zero neuron downsampling and zero synapse pruning. All 139,248 neurons and 15,090,883 synapses propagate physiological signals in real time. - **Dual-Tier Performance:** - **Biophysical Engine Throughput:** **>200 FPS** for pure sparse matrix-vector (SpMV) neural propagation across 15M synapses. - **Full Closed-Loop Streaming:** **50–60+ FPS** end-to-end (ViZDoom environment step + compound eye retinotopy + connectome biophysics + real-time WebSocket WebGL streaming). - **Compound Eye Retinotopy:** Translates ViZDoom game frames into optical ommatidia currents (32x32 array), stimulating photoreceptors and Elementary Motion Detectors (T4/T5 columns). - **Descending Motor Steering:** Decodes asymmetric population firing of 1,303 Descending Neurons (DNs) into continuous left/right steering, tracking, and attack decisions. - **Interactive 3D Connectome Visualizer:** WebGL point-cloud rendering of all 139,248 neurons with dynamic size and luminance modulation based on regional firing rates. - **Live Circuit Telemetry:** Real-time analog firing rate monitoring for Left/Right Optic Lobes, Central Complex (CX) heading compass, Mushroom Body (KC) memory circuits, Descending Motor pool, and Whole Brain. --- ## 🧠 System Architecture ```text [ViZDoom Game Engine] (320x240 RGB Screen) │ │ Raw Pixels ▼ ┌───────────────────────────────────────────────────┐ │ Compound Eye Retinotopy & Motion Encoder │ │ - 32x32 Ommatidia Array (1,024 optical facets) │ │ - Elementary Motion Detection (T4/T5 Columns) │ │ - Lobula Columnar (LC) Target Tracking │ └───────────────────────────────────────────────────┘ │ │ Injected Photoreceptor Current (I_sensory) ▼ ┌───────────────────────────────────────────────────┐ │ 100% FlyWire Whole-Brain Connectome │ │ - 139,248 Biological Neurons │ │ - 15,090,883 Directed Synapses (ACh, GABA, Glu) │ │ - Leaky Integrate-and-Fire Dynamic Integration │ └───────────────────────────────────────────────────┘ │ │ Population Firing of Descending Neurons ▼ ┌───────────────────────────────────────────────────┐ │ Motor Decoder │ │ - DN_Left vs DN_Right -> Continuous Steering │ │ - Bilateral Motor Vigor + Target Lock -> Attack │ └───────────────────────────────────────────────────┘ │ ▼ [Execute Action in DOOM] ``` --- ## 🔬 Biophysical Engine & Dynamics The simulator implements continuous-time leaky rate dynamics over sparse matrix-vector multiplication (SpMV): ```python I_syn[t] = W_syn @ act[t] I_tot[t] = I_syn[t] * gain + I_sensory[t] + noise V[t+1] = (1 - leak_rate) * V[t] + I_tot[t] act[t+1] = sigmoid((V[t+1] - firing_threshold) * 5.0) ``` - **Graph Memory:** Compressed to **~60.4 MB** for 15,090,883 sparse signed synapses using Compressed Sparse Row (CSR) format. - **Pure Tensor Propagation:** Zero heuristic if-else overrides in neural dynamics; motor commands are purely linear and softmax population readouts from descending motor neurons. - **Dynamic 3D Rendering Transfer:** Regional firing rates modulate point-cloud particle size and photon opacity via linear optical transfer functions, emulating biological calcium imaging (GCaMP). --- ## ☁️ Zero-Budget Cloud Optimization (Hugging Face ZeroGPU) Deploying a real-time, 15-million synapse simulation to the public web typically requires expensive dedicated GPU instances ($50–$200+/month). This project is engineered from the ground up for **$0 infrastructure cost**: - **Hugging Face ZeroGPU Integration:** Leverages Hugging Face's community tier powered by **Nvidia A10G** hardware with dynamic ZeroGPU quota allocation (`@spaces.GPU`). - **Memory Efficiency (60 MB Footprint):** By pre-indexing circuits into sparse CSR layout and zero-copy tensor slices, the entire connectome graph resides comfortably within basic cloud memory limits without swapping. - **Asynchronous WebSocket Streaming:** Decouples game execution from browser rendering via asynchronous WebSocket workers (`starlette.websockets`), preventing GPU timeout traps and ensuring smooth, stutter-free 50–60 FPS streaming for all concurrent visitors. --- ## 📁 Repository Structure ```text . ├── data/ │ └── connectome_cache.pt # Pre-built PyTorch sparse graph tensor (Git LFS) ├── src/ │ ├── __init__.py # Package exports │ ├── graph_loader.py # Fast FlyWire connectome loader │ ├── connectome_engine.py # 15M-synapse biophysical dynamics engine │ ├── retinotopy_encoder.py # Compound eye & motion optical flow encoder │ ├── motor_decoder.py # Descending neuron population motor decoder │ └── doom_agent.py # Closed-loop ViZDoom connectome agent ├── web/ │ ├── js/ # Three.js and OrbitControls │ ├── brain_139k_pos.bin # 139,248 neuron 3D coordinates (1.67 MB) │ ├── brain_139k_col.bin # 139,248 neuron anatomical colors (417 KB) │ ├── brain_anatomy.json # Regional circuit boundaries │ ├── brain_anatomy_3d.json # 3D spatial regions │ ├── synapses_top5k.bin # Top 5,000 anatomical synaptic tracts │ └── index.html # 3D Connectome and Live Action Dashboard ├── app.py # Hugging Face Space & Gradio entry point ├── .gitattributes # Git LFS binary tracking ├── .gitignore # Ignored virtual environments and artifacts ├── packages.txt # Linux system dependencies (ViZDoom runtime) ├── requirements.txt # Python dependencies ├── LICENSE # MIT License └── README.md ``` --- ## 🚀 Quickstart ### 1. Prerequisites & Environment Ensure you have **Python 3.10** (tested and recommended) and **Git LFS** installed: ```bash # Clone the repository git clone https://github.com/EmirMuhammetARAN/doom-flywire.git cd doom-flywire # Initialize Git LFS to pull the 15M-synapse cache tensor (~60 MB) git lfs install git lfs pull # Create and activate virtual environment (Python 3.10 recommended) python -m venv .venv # On Linux/macOS: source .venv/bin/activate # On Windows (PowerShell / Command Prompt): .venv\Scripts\activate # Install dependencies # (Note: Linux may require packages in packages.txt; Windows uses precompiled wheels) pip install -r requirements.txt ``` ### 2. Launch the Web Application Start the application server: ```bash python app.py ``` Open your browser at **http://localhost:7860**. You will see: - The live DOOM game viewport autonomously navigated by the 139,248-neuron connectome. - The interactive 3D WebGL connectome visualizer rendering active firing dynamics in real time. - Real-time biological circuit activity meters (Optic Lobes, Central Complex, Mushroom Body, Descending Motor pool, Whole Brain). --- ## 📚 References & Scientific Citations 1. **FlyWire Whole-Brain Connectome:** Dorkenwald, S., Matsliah, A., Sterling, P., et al. (2024). *Neuronal wiring diagram of an adult brain*. **Nature**, 634, 124–138. 2. **ViZDoom RL Platform:** Wydmuch, M., Kempka, M., & Jaśkowski, W. (2018). *ViZDoom Competitions: Playing Doom from Pixels*. **IEEE Transactions on Games**, 11(3), 248–258. 3. **Drosophila Motion Vision (T4/T5 Circuits):** Maisak, M. S., Haag, J., Ammer, G., et al. (2013). *A directional tuning map of Drosophila elementary motion detectors*. **Nature**, 500(7461), 212–216. 4. **Central Complex Navigation & Heading Compass:** Seelig, J. D., & Jayaraman, V. (2015). *Neural dynamics for landmark orientation and angular path integration*. **Nature**, 521(7551), 186–191.