# 🪰 FlyVector

> **Biological Locality-Sensitive Hashing (LSH) derived from the *Drosophila melanogaster* connectome.**  
> An experimental Python library that applies the fly brain's sparse coding architecture to vector compression. This is a research/educational project — see benchmark results below.

[![GitHub Stars](https://img.shields.io/github/stars/alieneseren/fly-vector?style=social)](https://github.com/alieneseren/fly-vector)
[![Python](https://img.shields.io/badge/Python-3.8+-3776AB?style=flat&logo=python&logoColor=white)](https://python.org)
[![License](https://img.shields.io/badge/License-MIT-emerald.svg)](LICENSE)
[![Princeton FlyWire](https://img.shields.io/badge/Connectome-Princeton_FlyWire_2024-purple.svg)](https://flywire.ai)

---

## ⚠️ Honest Benchmark Results

We ran real benchmarks on 5,000 vectors (1536-dim, OpenAI-equivalent). Here are the **actual** results:

| Parameters | Compression | Recall@10 | Notes |
| :--- | :--- | :--- | :--- |
| kenyon=2048, sparsity=5%, samp=8 | **24x** | 34.5% | Default config — good compression, poor recall |
| kenyon=4096, sparsity=20%, samp=32 | **12x** | 45.1% | Better recall, half compression |
| kenyon=8192, sparsity=25%, samp=48 | **6x** | 55.4% | Diminishing returns |
| kenyon=16384, sparsity=15%, samp=48 | **3x** | 60.2% | Best recall, minimal compression |

**Key finding:** There is a hard trade-off between compression ratio and recall accuracy in the current implementation. Production-grade systems like FAISS with Product Quantization achieve >90% Recall@10 at 32x compression, which this library does not yet match.

This project is **work in progress** and **not recommended for production use** until the recall issue is resolved.

---

## 💡 What This Is

FlyVector explores whether the fruit fly (*Drosophila melanogaster*) brain's sparse coding strategy can be applied to AI embedding compression.

In nature, the fly brain uses ~2,000 Kenyon cells in its Mushroom Body to encode complex odor spaces using:
1. **Sparse Random Synaptic Projections** — each Kenyon cell samples only ~8–10 input neurons
2. **APL Feedback Inhibition** — global normalization across the population
3. **Winner-Take-All (WTA)** — only the top ~5% highest-firing neurons remain active (*Science 2017, Dasgupta et al.*)

The 2,048-cell binary fingerprint packs into 256 bytes, enabling Hamming-distance search via `XOR + POPCNT`.

```
Dense Input: 1536-dim (6,144 bytes FP32)
         │
         ▼
Sparse Random Projection (W_proj)
         │
         ▼
APL Feedback Inhibition
         │
         ▼
Winner-Take-All (Top-5%)
         │
         ▼
uint8 Bitpacking → 256 bytes (24x smaller)
```

---

## 🚀 Installation & Usage

```bash
pip install -e .
```

```python
import numpy as np
from fly_vector import FlyVectorIndex

index = FlyVectorIndex(dim_in=1536, num_kenyon=2048, sparsity=0.05)

embeddings = np.random.normal(size=(1000, 1536)).astype(np.float32)
embeddings /= np.linalg.norm(embeddings, axis=1, keepdims=True)

index.add(embeddings, ids=[f"doc_{i}" for i in range(1000)])

results = index.search(embeddings[42], top_k=5)
for r in results:
    print(f"ID: {r['id']} | Hamming: {r['hamming_distance']}")
```

---

## 🔬 Scientific Foundations

1. **Dasgupta, S., Stevens, C. F., & Navlakha, S. (2017).**  
   *A neural algorithm for fundamental computing problems.*  
   **Science**, 358(6364), 793–796. [doi:10.1126/science.aam9868](https://doi.org/10.1126/science.aam9868)

2. **Dorkenwald, S., et al. (Princeton FlyWire Consortium, 2024).**  
   *Neuronal wiring diagram of an adult brain.*  
   **Nature**, 634, 124–138. [doi:10.1038/s41586-024-07558-y](https://doi.org/10.1038/s41586-024-07558-y)

---

## ⚖️ License

Released under the [MIT License](LICENSE). Free to use, modify, and contribute.