{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Getting Started with DaCe\n",
"\n",
"DaCe is a Python library that enables optimizing code with ease, from running on a single core to a full supercomputer. With the power of data-centric transformations, it can automatically map code for CPUs, GPUs, and FPGAs.\n",
"\n",
"Let's get started with DaCe by importing it:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"import dace"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A data-centric program can be generated from several general-purpose and domain-specific languages. Our main frontend, however, is Python/numpy. To define a program, we take an existing function on numpy arrays and decorate it with `@dace.program`:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"@dace.program\n",
"def getstarted(A):\n",
" return A + A"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Running our dace program, we will see several outputs and a prompt. These are the available transformations we can apply. For the first step, we opt to apply none (press Enter) and proceed with compilation and running:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.02638476, 0.15801766, 0.60640768],\n",
" [0.75281897, 0.02027034, 0.92066681]])"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"a = np.random.rand(2, 3)\n",
"a"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.05276951, 0.31603533, 1.21281536],\n",
" [1.50563794, 0.04054068, 1.84133362]])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"getstarted(a)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The results are, as expected, `2*A`.\n",
"\n",
"Now, let's inspect the intermediate representation of the data-centric program, its Stateful Dataflow Multigraph (SDFG):"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
""
],
"text/plain": [
"SDFG (getstarted)"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"getstarted.to_sdfg(a)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can drag the handle at the bottom right to make the SDFG frame larger.\n",
"\n",
"Notice the following four elements in the graph:\n",
"\n",
"1. **State** (blue region): This is the control flow part of the application, represented as a state machine. Since there is no control-flow in the data-centric representation of `A+A`, we see only one state encompassing the computation.\n",
"2. **Arrays** (circular nodes) and **Memlets** (arrows): These nodes represent disjoint N-dimensional memory regions (similar to numpy `ndarray`s), and the edges represent data that is moved throughout the state. Hovering over a memlet will show more information about the subset being moved.\n",
"3. **Tasklets** (octagon): This node represents the computational parts of the graph. Zooming into it will show the code (addition operation in this case). Tasklets act as pure functions that can only work with the data coming into/out of its **connectors** (cyan circles on the node).\n",
"4. **Maps** (trapezoid): Anything that is enclosed between these two nodes (the map *scope*) is replicated for the number of times specified on the node (in our case, `2*3` times). This creates parametric parallelism in the graph and can be nested in each other for efficient parallelization and distribution of work.\n",
"\n",
"Unfortunately (or fortunately in some cases), this graph is specialized for a specific size of array (as given to it), and will not work on other sizes. To compile a program that works with general sizes, we'll need to use symbolic sizes."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Symbols\n",
"\n",
"DaCe includes a symbolic math engine (extending SymPy) to support symbolic expressions for sizes, ranges, accesses, and more. \n",
"\n",
"Any number of symbols can be used throughout a computation. Defining a symbol is as easy as calling:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"N = dace.symbol('N')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"which we can now use for any computation and definitions. For example, annotating the types of our function from above will yield a version that works with any size:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"@dace.program\n",
"def getstarted_sym(A: dace.float64[N, 2*N]):\n",
" return A + A"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
""
],
"text/plain": [
"SDFG (getstarted_sym)"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"getstarted_sym.to_sdfg()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we compile this code, any array that can match a size of `Nx2N` will be automatically used to infer the value of `N` and invoke the function:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.98818461, 1.27933885, 0.2033508 , ..., 0.547033 , 0.4299565 ,\n",
" 0.24654365],\n",
" [1.91945996, 0.8587834 , 1.6074685 , ..., 0.60969216, 1.7881462 ,\n",
" 1.6251679 ],\n",
" [0.09656663, 0.86573612, 0.79912191, ..., 1.50199177, 0.14342504,\n",
" 0.77152323],\n",
" ...,\n",
" [1.86926975, 0.16524055, 0.57659078, ..., 0.06706506, 1.94858343,\n",
" 0.21332081],\n",
" [0.78987173, 0.32493361, 0.33111051, ..., 0.41438505, 1.6625166 ,\n",
" 1.4539469 ],\n",
" [0.32619914, 0.84155838, 0.85757214, ..., 0.93809 , 0.25236549,\n",
" 1.95588663]])"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"getstarted_sym(np.random.rand(100, 200))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Performance\n",
"\n",
"Given our symbolic SDFG, we would not like to recompile it every time. Thus, we can pre-compile the graph into an .so/.dll file:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"csdfg = getstarted_sym.compile()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A compiled SDFG, however, has to be invoked like an SDFG, with keyword arguments only:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"b = csdfg(A=np.random.rand(10,20), N=np.int32(10))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can now see the performance of the code on large arrays vs. numpy:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"tester = np.random.rand(2000, 4000)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"12 ms ± 258 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"%timeit tester + tester"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"5.1 ms ± 470 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"%timeit csdfg(A=tester, N=np.int32(2000))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Explicit Dataflow\n",
"\n",
"One can specify explicit dataflow in dace using `for i in dace.map[begin:end]:` syntax, as well as tasklets manually using `with dace.tasklet:`. Here is an example of a real-world example (Scattering Self-Energies) with an 8-dimensional parallel computation:"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
""
],
"text/plain": [
"SDFG (sse_sigma)"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Declaration of symbolic variables\n",
"Nkz, NE, Nqz, Nw, N3D, NA, NB, Norb = (\n",
" dace.symbol(name)\n",
" for name in ['Nkz', 'NE', 'Nqz', 'Nw', 'N3D', 'NA', 'NB', 'Norb'])\n",
"\n",
"\n",
"@dace.program\n",
"def sse_sigma(neigh_idx: dace.int32[NA, NB],\n",
" dH: dace.complex128[NA, NB, N3D, Norb, Norb],\n",
" G: dace.complex128[Nkz, NE, NA, Norb, Norb],\n",
" D: dace.complex128[Nqz, Nw, NA, NB, N3D, N3D],\n",
" Sigma: dace.complex128[Nkz, NE, NA, Norb, Norb]):\n",
"\n",
" # Declaration of Map scope\n",
" for k, E, q, w, i, j, a, b in dace.map[0:Nkz, 0:NE, 0:Nqz, 0:Nw, 0:N3D, 0:\n",
" N3D, 0:NA, 0:NB]:\n",
" dHG = G[k - q, E - w, neigh_idx[a, b]] @ dH[a, b, i]\n",
" dHD = dH[a, b, j] * D[q, w, a, b, i, j]\n",
" Sigma[k, E, a] += dHG @ dHD\n",
" \n",
"sse_sigma.to_sdfg()"
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 4
}