# FlyView FlyView draws one 3D scene two ways. The first way is the way a person sees it. The second way is the way a fruit fly sees it. The fly view is not an effect applied to a picture. Every viewing direction in it comes from one real neuron in the MaleCNS v1.0 connectome of the male fruit fly. A connectome is a map of all the neurons in a brain and of the connections between them. ![FlyView shows three panels. The top left panel holds the human view of a checkerboard floor with colored blocks. The top right panel holds the same view as a mosaic of large hexagons. The bottom panel holds the whole field of the fly as two discs, one for each eye.](docs/screenshot.png) ## Run the viewer You do not need the MaleCNS v1.0 download to run FlyView. The map of viewing directions is already built. It sits in this repository as `retina.json`, and it is 100 KB. The viewer needs Python and a browser, and nothing else. Start the server with one command: ```bash ./run.sh ``` Then open `http://127.0.0.1:8080/` in a browser. To use another port, set `PORT`: ```bash PORT=9000 ./run.sh ``` The viewer is one web page with no build step. It needs a web server, because a browser refuses to load module scripts from a file path. `run.sh` starts the web server that Python provides. Drag to turn the view. Scroll to move nearer or farther. The fly sits at the same point as the human camera and faces the same way, so the eye is the only difference between the two top panels. ## The three panels The top left panel is the human view, 60 degrees across. The top right panel is the fly view of the same 60 degrees, in the same projection. Nothing is bent or stretched here, so this panel shows the loss of sharpness on its own. The bottom panel is the whole field of the fly. The two eyes together cover about 92 percent of the sphere. A flat screen cannot hold that much without distortion, so the panel offers three ways to flatten it. Compare them to see where each one bends the picture. ## The fly does not see hexagons Each facet of the eye reports one number, and that number is a brightness. The brain of the fly receives 887 numbers per eye. It does not receive a picture with hexagons drawn on it. FlyView draws each facet as a hexagon so that you can see where the samples sit. A photograph works the same way. It is built from pixels, but the camera does not see squares. The hexagonal arrangement is real. Facets pack in a hexagonal grid, in the way that a honeycomb does. The coarseness is real too: - A fly takes one sample every 4.5 degrees. The center of a human eye resolves about 0.017 degrees. - One fly eye holds 887 samples, which is about the detail of a 30 by 30 thumbnail. - A fly cannot read text, and it loses the fine end of a striped pattern. A fly trades detail for speed and coverage. It resolves flicker up to about 200 hertz, where a person stops at about 60. It sees almost all directions at once and never has to turn its eyes. ## Where the map comes from The map anchors on the cell type Mi1. Every column of the medulla holds exactly one Mi1 cell. The medulla is the part of the fly brain that holds one column for each facet of the eye. The set of Mi1 cells is therefore the set of columns, and the data holds 886 of them for the left eye and 887 for the right. `build_retina.py` reads the shape of each Mi1 cell, fits a sphere to the positions, and turns each position into a viewing direction. The connectome does not give the angles. It gives the order of the columns. Measured about the center of curvature of the medulla, two neighboring columns sit 1.85 degrees apart, and the true angle in Drosophila is about 4.8 degrees. The medulla sits behind the retina and has a different shape, and the retina was never imaged. The script therefore keeps the order from the connectome and imposes the physical geometry on top of it. Three numbers come from published anatomy and not from the data: - `FIELD_HALF_ANGLE` is 90 degrees, so each eye covers a half sphere. - `EYE_DIVERGENCE` is 150 degrees between the two optical axes. This leaves a strip 30 degrees wide in front that both eyes see. - `RELAX_ITERS` is 20 passes of spreading, which covers 99 percent of the field and keeps 77 percent of the neighbors of each column. Every imposed number is an environment variable. Change one to see what the assumption costs. ## Rebuild the map This is the only task that needs the MaleCNS v1.0 download. The download gave the map its directions, but that work is done. The result is `retina.json`, and it is in this repository. Rebuild it only to change one of the assumptions above, or to derive the map again from the source. The rebuild reads the skeleton files and the body annotation table. The download is 11 GB, and Git ignores it. 1. Put the download at `malecns-v1.0/`, or set `MALECNS_DIR` to its path. 2. Install NumPy and pandas. 3. Delete `retina.json`. 4. Run `./run.sh`. The script rebuilds the map and then starts the server. To rebuild the map without starting the server, run `python3 build_retina.py`. It takes about 3 seconds. To change an assumption, set it on the same command line: ```bash EYE_DIVERGENCE=120 python3 build_retina.py ``` ## How the fly view is drawn The viewer renders the scene into six square views around the fly. Six views are the only way to capture a field wider than half the world. Each facet then takes 13 samples over the cone it accepts light from, and averages them into one value. The whole facet carries that one value, in the way that one facet of a real eye hands one number to its neuron. A fruit fly has almost no sensitivity to red. Turn off the red-blind setting to separate the loss of color from the loss of sharpness. ## Files | File | Purpose | |---|---| | `index.html` | The viewer. Plain JavaScript and three.js from a CDN. | | `retina.json` | 1773 viewing directions, 886 for the left eye and 887 for the right. Built already. | | `build_retina.py` | Rebuilds `retina.json` from the MaleCNS v1.0 download. | | `run.sh` | Starts the web server on port 8080. | ## Data `retina.json` comes from MaleCNS v1.0, the connectome of the central nervous system of the male fruit fly. `build_retina.py` reads two parts of that download: the skeleton files, which give the shape of each neuron, and the body annotation table, which gives the cell type and the body side of each neuron. The download is the source of the map. It is not something the viewer reads. Once `build_retina.py` writes `retina.json`, the 1773 directions stand on their own, and FlyView runs from that one file.