{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastdot import *\n",
"from fastcore.all import *\n",
"from dataclasses import dataclass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# fastdot\n",
"\n",
"> A simple wrapper over `pydot` to make it more consistent, unsurprising, and pythonic"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Acknowledgement: `fastdot` is heavily influenced by work from [David Page](https://github.com/davidcpage/), who built a system for drawing graphs based on a highly flexible data structure he designed."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Install"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We suggest installing with conda: `conda install -c fastai fastdot`. You can alternatively install with pip: `pip install fastdot`; however, if you use this approach, you'll also need to install graphviz (e.g. using `apt`, `brew`, etc)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Synopsis"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Start with some data representing objects and connections between them (e.g. they wouldn't normally be just strings like in this example, but would be neural net layers, or users and products, or car trips, etc):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"layers1 = ['conv','conv','lin']\n",
"layers2 = ['conv','lin']\n",
"block1,block2 = ['block1','block2']\n",
"conns = ((block1, block2),\n",
" (block1, layers2[-1]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then map them directly to a visual respresentation:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"g = graph_items(seq_cluster(layers1, block1),\n",
" seq_cluster(layers2, block2))\n",
"g.add_items(*object_connections(conns))\n",
"g"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"See the *symbolic graphs* and *object graphs* sections below for a more complete example."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## fastdot overview"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`fastdot` is a thin wrapper over the excellent [pydot](https://github.com/pydot/pydot) program (which is in turn a thin wrapper over the absolutely wonderful [Graphviz software](https://www.graphviz.org/)), designed to make it more consistent, unsurprising, and pythonic. (An example of removing *surprise*: `pydot.Node('node')` gives an obscure compilation exception, since `node` is a keyword in the underlying `graphviz` program, whereas `fastdot.Node('node')` works just fine, due to auto-quoting.) In fact, you never need to provide names in `fastdot`; you can create edges directly between objects."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here's a quick example of some of the main functionality:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"g = Dot()\n",
"c = Cluster('cl', fillcolor='pink')\n",
"a1,a2,b = c.add_items('a', 'a', 'b')\n",
"c.add_items(a1.connect(a2), a2.connect(b))\n",
"g.add_item(Node('Check tooltip', tooltip=\"I have a tooltip!\"))\n",
"g.add_item(c)\n",
"g"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you see, graphs know how to show themselves in Jupyter notebooks directly and can be exported to HTML (it uses SVG behind the scenes). Tooltips appear in both notebooks and exported HTML pages. Nodes with the same label, by default, are set to the same color. Also, as shown above, you can just use `add_item` or `add_items`, regardless of the type of item."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Symbolic graphs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`fastdot` is particularly designed to make it easier to create graphs symbolically - for instance, for Python dictionaries, PyTorch/TensorFlow models, and so forth. Here's a simple example with some mock neural network layers and sequential models. First, let's define our mock classes:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"@dataclass(frozen=True)\n",
"class Layer: name:str; n_filters:int=1\n",
"class Linear(Layer): pass\n",
"class Conv2d(Layer): pass\n",
"\n",
"@dataclass(frozen=True)\n",
"class Sequential: layers:list; name:str"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here's our sequential blocks for our \"model\":"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"block1 = Sequential([Conv2d('conv', 5), Linear('lin', 3)], 'block1')\n",
"block2 = Sequential([Conv2d('conv1', 8), Conv2d('conv2', 2), Linear('lin')], 'block2')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`fastdot` can create all node properties directly from objects; you just have to define functions describing how to map the object's attributes to graph properties. These mappings go in the `node_defaults` and `cluster_defaults` dictionaries (although by default labels are set using `str()`, so we don't need any special cluster defaults in this case):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"node_defaults['fillcolor'] = lambda o: 'greenyellow' if isinstance(o,Linear) else 'pink'\n",
"cluster_defaults['label'] = node_defaults['label'] = attrgetter('name')\n",
"node_defaults['tooltip'] = str"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With that in place, we can directly create nodes from our objects, for instance using the convenient `seq_cluster` function:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"c1 = seq_cluster(block1.layers, block1)\n",
"c2 = seq_cluster(block2.layers, block2)\n",
"e1,e2 = c1.connect(c2),c1.connect(c2.last())\n",
"graph_items(c1,c2,e1,e2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that in this example we didn't even need to create the `Dot` object separately - `graph_items` creates it directly from the graph items provided."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using object graphs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the above example, we defined our edges directly between `fastdot` objects. In practice, however, you'll most likely have your edges defined directly between python objects, for instance like this:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"conns = (\n",
" (block1, block2),\n",
" (block1, block2.layers[-1]),\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this case, you'll want some way to connect your python objects to the `fastdot` graph items that represent them. A mapping is stored automatically by `fastdot`, and is made available through the `object2graph` function:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"g = graph_items(seq_cluster(block1.layers, block1), seq_cluster(block2.layers, block2))\n",
"object2graph(block1.layers[-1])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use this to graph your connections without needing access to the graph items:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"g.add_items(*[object2graph(a).connect(object2graph(b))\n",
" for a,b in conns])\n",
"g"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There's a helper function, `object_connections`, which creates these connections for you. So the above can be simplified to:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"g = graph_items(seq_cluster(block1.layers, block1), seq_cluster(block2.layers, block2))\n",
"g.add_items(*object_connections(conns))\n",
"g"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}