{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Connecting the Retina and the Lamina LPUs through Neurokernel API" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We demonstrate in this notebook how to plug in individual LPUs into a Neurokernel simulation, and connect these LPUs through the Neurokernel API. The two LPUs that we use here are the models for the retina and the lamina, the first two LPUs in the visual sytem of the fruit fly. According to the Neurokernel API, a successful integration of these LPUs requires each of them to expose named ports. We briefly summarize the ports exposed by the two LPUs first, and then detail the connection between them. A schematic diagram of the two connected LPUs is show in Fig. 1. This notebook is meant to be narrative, but not directly executable due to MPI limitation. For executable example, see [examples/retlam_demo/retlam_demo.py](https://github.com/neurokernel/retina-lamina/blob/master/examples/retlam_demo/retlam_demo.py) in the [neurokernel/retina-lamina repository](https://github.com/neurokernel/retina-lamina) for integrating the retina and lamina models." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "Figure 1. Schematic diagram of integrated simulation of the retina and the lamina LPUs under Neurokernel. The retina and the lamina both expose their ports to Neurokernel using the Neurokernel API, and Neurokernel handles the communication between the two LPUs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Configuration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before simulation we need to provide a configuration of various parameters of both LPUs. The configuration is assumed to have the form of a dictionary. The configuration combines those in the retina and the lamina modules. For easier manipulation this configuration can be read from a configuration file. Details of each parameter can be found in the configuration template in [examples/template_spec.cfg](https://github.com/neurokernel/retina-lamina/blob/master/examples/template_spec.cfg)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "config = {}\n", "config['General'] = {}\n", "config['General']['dt'] = 1e-4\n", "config['General']['steps'] = 1000\n", " \n", "config['Retina'] = {}\n", "config['Retina']['model'] = 'vision_model_template'\n", "config['Retina']['acceptance_factor'] = 1\n", "config['Retina']['screentype'] = 'Sphere'\n", "config['Retina']['filtermethod'] = 'gpu'\n", "config['Retina']['intype'] = 'Bar'\n", "config['Retina']['time_rep'] = 1\n", "config['Retina']['space_rep'] = 1\n", "\n", "config['Lamina'] = {}\n", "config['Lamina']['model'] = 'vision_model_template'\n", "config['Lamina']['relative_am'] = 'half'\n", "\n", "config['Screen'] = {}\n", "config['Screen']['SphereScreen'] = {}\n", "config['Screen']['SphereScreen']['parallels'] = 50\n", "config['Screen']['SphereScreen']['meridians'] = 100\n", "config['Screen']['SphereScreen']['radius'] = 10\n", "config['Screen']['SphereScreen']['half'] = False\n", "config['Screen']['SphereScreen']['image_map'] = 'AlbersProjectionMap'\n", "\n", "config['InputType'] = {}\n", "config['InputType']['shape'] = [100, 100]\n", "config['InputType']['infilename'] = ''\n", "config['InputType']['writefile'] = False\n", "\n", "config['InputType']['Bar'] = {}\n", "config['InputType']['Bar']['bar_width'] = 10\n", "config['InputType']['Bar']['direction'] = 'v'\n", "config['InputType']['Bar']['levels'] = [3e3, 3e4]\n", "config['InputType']['Bar']['speed'] = 1000\n", "config['InputType']['Bar']['double'] = False" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creation of Neurokernel Manager" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We first create the Neurokernel manager." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import neurokernel.core_gpu as core\n", "import neurokernel.mpi_relaunch\n", "\n", "manager = core.Manager()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Retina and its Interface" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We construct the retina LPU from the retina module and add it to the Neurokernel simulation. We follow closely the retina example in which the retina LPU is executed in isolation. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import networkx as nx\n", "import retina.geometry.hexagon as ret_hx\n", "import retina.retina as ret\n", "from retina.LPU import LPU as rLPU\n", "\n", "from retina.InputProcessors.RetinaInputProcessor import RetinaInputProcessor\n", "from neurokernel.LPU.OutputProcessors.FileOutputProcessor import FileOutputProcessor\n", "from retina.NDComponents.MembraneModels.PhotoreceptorModel import PhotoreceptorModel\n", "from retina.NDComponents.MembraneModels.BufferPhoton import BufferPhoton\n", "\n", "# create a hexagonal array for the retina\n", "# This describes the array of positions of neurons, their arrangement in space and a way to query for neighbors \n", "ret_hexagon = ret_hx.HexagonArray(num_rings=14, radius=1)\n", "# create a retina object that contains neuron and synapse information\n", "retina_array = ret.RetinaArray(ret_hexagon, config)\n", "\n", "# parameters from the configuration dictionary\n", "dt = config['General']['dt']\n", "\n", "output_file = 'retina_output.h5'\n", "gexf_file = 'retina.gexf.gz'\n", "\n", "input_processor = RetinaFileInputProcessor(config, retina)\n", "output_processor = FileOutputProcessor([('V',None)], output_file, sample_interval=1)\n", "\n", "G = retina_array.get_worker_nomaster_graph()\n", "# export the configuration of neurons and synapses to a GEXF file\n", "nx.write_gexf(G, gexf_file)\n", "# parse GEXF file\n", "(comp_dict, conns) = LPU.graph_to_dicts(G)\n", "retina_id = 'retina0'\n", "\n", "extra_comps = [PhotoreceptorModel, BufferPhoton]\n", "\n", "# add the retina LPU to Neurokernel manager\n", "manager.add(LPU, retina_id, dt, comp_dict, conns,\n", " device = retina_index, input_processors = [input_processor],\n", " output_processors = [output_processor],\n", " debug=debug, time_sync=time_sync, extra_comps = extra_comps)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Retina model ([Neurokernel RFC #3](http://neurokernel.github.io/docs.html)) exposes their outputs, the photoreceptors R1-R6, to the Neurokernel. The naming convention of the ports is as follows: A port associated with a photoreceptor is named as `/ret//`, where `omm_id` is a unique numeric identifier of the ommatidium that the photoreceptor resides, and `photor_name` the name of photoreceptor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Lamina and its Interface" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We construct the lamina LPU from the lamina module and add it to the Neurokernel simulation. We follow closely the lamina example in which the lamina LPU is executed in isolation." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import lamina.geometry.hexagon as lam_hx\n", "import lamina.lamina as lam\n", "from lamina.LPU import LPU as lLPU\n", "\n", "from retina.NDComponents.MembraneModels.BufferVoltage import BufferVoltage\n", "\n", "# create a hexagonal array for the lamina\n", "lam_hexagon = lam_hx.HexagonArray(num_rings=14, radius=1)\n", "# create a lamina object that contains neuron and synapse information\n", "lamina_array = lam.LaminaArray(lam_hexagon, config)\n", "\n", "# parameters from the configuration dictionary\n", "dt = config['General']['dt']\n", "\n", "output_file = 'lamina_output.h5'\n", "gexf_file = 'lamina.gexf.gz'\n", "G = lamina_array.get_graph()\n", "# export the configuration of neurons and synapses to a GEXF file\n", "nx.write_gexf(G, gexf_file)\n", "# parse GEXF file\n", "(comp_dict, conns) = LPU.graph_to_dicts(G)\n", "lamina_id = 'lamina0'\n", "extra_comps = [BufferVoltage]\n", "\n", "# add the lamina LPU to Neurokernel manager\n", "manager.add(LPU, lamina_id, dt, comp_dict, conns,\n", " output_processors = [output_processor],\n", " device=lamina_index+1, debug=debug, time_sync=time_sync,\n", " extra_comps = extra_comps)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Photoreceptors R1-R6 constitute the inputs to the lamina LPU from the retina LPU. The input ports in the lamina LPU follows the naming convention: a photoreceptor input port is named as `/lam//` where `cart_id` is the unique numeric identifier of the cartridge that the input port belongs to, and `photor_name` is the name of the photoreceptor. Note that retinotopy in the early visual system of the fruit fly is imposed by the hexagonal array of ommatidia in the retina and that of cartridges in the lamina. The two arrays are assumed to be compatible to each other, i.e., the ommatidia have a one-to-one correspondence to the cartridges." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Connection between the Retina and the Lamina" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The connections between retina and lamina follow the neural superposition rule of the fly's compound eye (see [Neurokernel RFC#2](http://neurokernel.github.io/docs.html)). The superposition rule is also illustrated in the Fig. 2." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "Figure 2. Neural superposition rule. The solid circles represent ommatidia in the retina, and dashed circles represent cartridges in the lamina. Individual photoreceptors R1-R6 are numbered and their relative positon highlighted in some of the ommatidia. On the left, cartridge A receives 6 photoreceptor inputs, each from a different ommatidium. On the right, 6 photoreceptors from a single ommatidium each projects to a different cartridge." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The two LPUs added to the Neurokernel have properly exposed their I/O to the Neurokernel. The connection between the two LPUs can then be configured through the Pattern provided by the Neurokernel API." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from neurokernel.pattern import Pattern\n", "\n", "retina_id = get_retina_id(index)\n", "lamina_id = get_lamina_id(index)\n", " \n", "retina_selectors = retina_array.get_all_selectors()\n", "lamina_selectors = ［］\n", "\n", "# obtain two lists of selectors,\n", "# each corresponding entry denotes\n", "# the selector of the outgoing port\n", "# and the selector of the incoming port\n", "from_list = []\n", "to_list = []\n", "\n", "rulemap = retina_array.rulemap\n", "for ret_sel in retina_selectors:\n", " if not ret_sel.endswith('agg'):\n", " # format should be '/ret//'\n", " _, lpu, ommid, n_name = ret_sel.split('/')\n", "\n", " # find neighbor of neural superposition\n", " neighborid = rulemap.neighbor_for_photor(int(ommid), n_name)\n", "\n", " # format should be '/lam//'\n", " lam_sel = lamina_array.get_selector(neighborid, n_name)\n", "\n", " # concatenate the selector to from and to lists\n", " from_list.append(ret_sel)\n", " to_list.append(lam_sel)\n", " \n", " # append aggregators\n", " from_list.append(lam_sel+'_agg')\n", " to_list.append(ret_sel+'_agg')\n", " lamina_selectors.append(lam_sel)\n", " lamina_selectors.append(lam_sel+'_agg')\n", "\n", "# create pattern from the two lists using from_concat method\n", "# This method is faster than creating pattern using __setitem__\n", "pattern = Pattern.from_concat(','.join(retina_selectors),\n", " ','.join(lamina_selectors),\n", " from_sel=','.join(from_list),\n", " to_sel=','.join(to_list),\n", " gpot_sel=','.join(from_list+to_list))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we connect the retina and the lamina LPUs using the pattern." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "manager.connect(retina_id, lamina_id, pattern)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Simulation of the connected retina and lamina LPUs can then be started." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "steps = config['General']['steps']\n", "manager.spawn()\n", "manager.start(steps=steps)\n", "manager.wait()" ] } ], "metadata": { "celltoolbar": "Edit Metadata", "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.12" } }, "nbformat": 4, "nbformat_minor": 0 }