{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "In this notebook, we demonstrate the experimental dask-yt particle loader in `./dask_chunking/gadget_da.py` \n", "\n", "The dask approach here attempts to wrap the loading and filtering of individual chunks with the `dask.delayed` operator (to various degress of success...), resulting in a lazy load of a `gadget` particle dataset that is automatically parallelized when running a `dask.distributed.Client`. \n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from dask_chunking import gadget_da as gda\n", "from dask import compute, visualize\n", "import yt \n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, let's spin up a dask `Client`:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from dask.distributed import Client " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "c = Client(threads_per_worker=2,n_workers=4)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n",
"Client\n", "
| \n",
"\n",
"Cluster\n", "
| \n",
"