{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "hollow-genre", "metadata": {}, "outputs": [], "source": [ "import uproot\n", "import awkward as ak\n", "from coffea.nanoevents import NanoEventsFactory, NanoAODSchema, schemas\n", "from coffea import processor, hist" ] }, { "cell_type": "markdown", "id": "manual-disposition", "metadata": {}, "source": [ "## Define a Processor instance\n", "\n", "The processor implementation given below has been taken from [here](https://github.com/CoffeaTeam/coffea/blob/master/binder/nanoevents.ipynb)." ] }, { "cell_type": "code", "execution_count": 2, "id": "domestic-interim", "metadata": {}, "outputs": [], "source": [ "class MyZPeak(processor.ProcessorABC):\n", " def __init__(self):\n", " self._histo = hist.Hist(\n", " \"Events\",\n", " hist.Cat(\"dataset\", \"Dataset\"),\n", " hist.Bin(\"mass\", \"Z mass\", 60, 60, 120),\n", " )\n", " \n", " @property\n", " def accumulator(self):\n", " return self._histo\n", " \n", " # we will receive a NanoEvents instead of a coffea DataFrame\n", " def process(self, events):\n", " out = self.accumulator.identity()\n", " mmevents = events[\n", " (ak.num(events.Muon) == 2)\n", " & (ak.sum(events.Muon.charge, axis=1) == 0)\n", " ]\n", " zmm = mmevents.Muon[:, 0] + mmevents.Muon[:, 1]\n", " out.fill(\n", " dataset=events.metadata[\"dataset\"],\n", " mass=zmm.mass,\n", " )\n", " return out\n", " \n", " def postprocess(self, accumulator):\n", " return accumulator" ] }, { "cell_type": "markdown", "id": "together-assist", "metadata": {}, "source": [ "## Write some NanoEvents Parquet files to CephFS\n", "\n", "Here we populate the CephFS mounted directory with the parquet files created in the previous step. In this version, we need to make sure that the individual file sizes is under 4MB which is the default object size of Ceph to ensure one-to-one mapping of files to objects, which is a requirement in the multiple-file design that we have now." ] }, { "cell_type": "code", "execution_count": 3, "id": "muslim-retrieval", "metadata": {}, "outputs": [], "source": [ "import os, shutil\n", "\n", "if not os.path.isfile(\"nano_dy.parquet\"):\n", " ak.to_parquet(\n", " uproot.lazy(\"nano_dy.root:Events\"),\n", " \"nano_dy.parquet\",\n", " list_to32=True,\n", " use_dictionary=False,\n", " compression=\"GZIP\",\n", " compression_level=1,\n", " )\n", "\n", "if not os.path.isfile(\"nano_dimuon.parquet\"):\n", " ak.to_parquet(\n", " uproot.lazy(\"nano_dimuon.root:Events\"),\n", " \"nano_dimuon.parquet\",\n", " list_to32=True,\n", " use_dictionary=False,\n", " compression=\"GZIP\",\n", " compression_level=1,\n", " )\n", " \n", "if not os.path.isdir(\"/mnt/cephfs/nanoevents/ZJets\"):\n", " os.makedirs(\"/mnt/cephfs/nanoevents/ZJets\")\n", " for i in range(2):\n", " shutil.copyfile('nano_dy.parquet', '/mnt/cephfs/nanoevents/ZJets/nano_dy.{}.parquet'.format(i))\n", " \n", " \n", "if not os.path.isdir(\"/mnt/cephfs/nanoevents/Data\"):\n", " os.makedirs(\"/mnt/cephfs/nanoevents/Data\")\n", " for i in range(2):\n", " shutil.copyfile('nano_dimuon.parquet', '/mnt/cephfs/nanoevents/Data/nano_dimuon.{}.parquet'.format(i))" ] }, { "cell_type": "markdown", "id": "manufactured-america", "metadata": { "tags": [] }, "source": [ "## Running a job in parallel using Dask\n", "\n", "The `LocalCluster()` used below creates a process pool with worker count equal to the number of cores available to the Notebook where each worker is single-threaded. The `LocalCluster` can be replaced by other cluster resource managers provided by Dask Distributed like `KuberneresCluster`, `YarnCluster`, etc. Here, we create a `LocalCluster` and get a client handle to it." ] }, { "cell_type": "code", "execution_count": 4, "id": "e73bc893-3745-4048-b0cc-ea3f1c45093d", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "

Client

\n", "

Client-2c026108-c195-11ec-8676-2284b808c7e2

\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
Connection method: Direct
\n", " Dashboard: /user/oksana.shadura@cern.ch/proxy/8787/status\n", "
\n", "\n", " \n", "
\n", "

Scheduler Info

\n", "
\n", "
\n", "
\n", "
\n", "

Scheduler

\n", "

Scheduler-44ffa98d-cfbe-4b3c-a054-98c4c229546e

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", " Comm: tcp://127.0.0.1:34139\n", " \n", " Workers: 1\n", "
\n", " Dashboard: /user/oksana.shadura@cern.ch/proxy/8787/status\n", " \n", " Total threads: 4\n", "
\n", " Started: 42 minutes ago\n", " \n", " Total memory: 15.70 GiB\n", "
\n", "
\n", "
\n", "\n", "
\n", " \n", "

Workers

\n", " \n", "\n", " \n", "
\n", "
\n", "
\n", "
\n", " \n", "

Worker: 18

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\n", " Comm: tcp://127.0.0.1:34125\n", " \n", " Total threads: 4\n", "
\n", " Dashboard: /user/oksana.shadura@cern.ch/proxy/44441/status\n", " \n", " Memory: 15.70 GiB\n", "
\n", " Nanny: tcp://127.0.0.1:40603\n", "
\n", " Local directory: /home/cms-jovyan/dask-worker-space/worker-3ao_484a\n", "
\n", " Tasks executing: 0\n", " \n", " Tasks in memory: 0\n", "
\n", " Tasks ready: 0\n", " \n", " Tasks in flight: 0\n", "
\n", " CPU usage: 2.0%\n", " \n", " Last seen: Just now\n", "
\n", " Memory usage: 179.51 MiB\n", " \n", " Spilled bytes: 0 B\n", "
\n", " Read bytes: 2.36 kiB\n", " \n", " Write bytes: 2.36 kiB\n", "
\n", "
\n", "
\n", "
\n", " \n", "\n", "
\n", "
\n", "
\n", " \n", "\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask.distributed import Client\n", "\n", "client = Client(\"tcp://127.0.0.1:34139\")\n", "client" ] }, { "cell_type": "markdown", "id": "toxic-empty", "metadata": {}, "source": [ "We have added a new function called `run_parquet_job` to the executor API in coffea to run jobs on Parquet files using the Arrow Dataset API under the hood. \n", "This API takes an optional `ceph_config_path` parameter, which is basically the path to the configuration file of the Ceph cluster and instructs this function to read from RADOS using the `SkyhookFileFormat` (which allows pushdown) instead of the out of the box `ParquetFormat` API . This API also allows just passing a single directory path and the Datasets API does the dataset discovery task by itself. The calls to the Dataset API are launced in parallel and there will one Dataset API call per file." ] }, { "cell_type": "code", "execution_count": 5, "id": "altered-transition", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 645 ms, sys: 114 ms, total: 759 msCompleted | 7.4s\n", "Wall time: 8 s\n" ] } ], "source": [ "%%time\n", "\n", "run = processor.Runner(\n", " executor=processor.DaskExecutor(client=client),\n", " use_skyhook=True,\n", " format=\"parquet\",\n", " schema=schemas.NanoAODSchema,\n", ")\n", "\n", "hists = run(\n", " {\n", " \"ZJets\": \"/mnt/cephfs/nanoevents/ZJets/\",\n", " \"Data\": \"/mnt/cephfs/nanoevents/Data\",\n", " },\n", " \"Events\",\n", " processor_instance=MyZPeak(),\n", ")" ] }, { "cell_type": "markdown", "id": "statutory-worst", "metadata": {}, "source": [ "## Running iteratively using the `FuturesExecutor`\n", "\n", "Run the same job again, but now iteratively. The calls to the Dataset API will now be sequential." ] }, { "cell_type": "code", "execution_count": 6, "id": "aboriginal-immune", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "8067afe2a53542e89e376d22a18735bf", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "/opt/conda/lib/python3.8/site-packages/coffea/nanoevents/schemas/nanoaod.py:193: RuntimeWarning: Missing cross-reference index for FatJet_genJetAK8Idx => GenJetAK8\n", " warnings.warn(\n", "/opt/conda/lib/python3.8/site-packages/coffea/nanoevents/schemas/nanoaod.py:193: RuntimeWarning: Missing cross-reference index for FatJet_genJetAK8Idx => GenJetAK8\n", " warnings.warn(\n" ] }, { "data": { "text/html": [ "
\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
\n",
       "
\n"
      ],
      "text/plain": [
       "\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 496 ms, sys: 53.8 ms, total: 550 ms\n",
      "Wall time: 8.55 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "run = processor.Runner(\n",
    "    executor=processor.FuturesExecutor(compression=None),\n",
    "    use_skyhook=True,\n",
    "    format=\"parquet\",\n",
    "    schema=schemas.NanoAODSchema,\n",
    ")\n",
    "\n",
    "hists = run(\n",
    "    {\n",
    "        \"ZJets\": \"/mnt/cephfs/nanoevents/ZJets\",\n",
    "        \"Data\": \"/mnt/cephfs/nanoevents/Data\",\n",
    "    },\n",
    "    \"Events\",\n",
    "    processor_instance=MyZPeak(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ef6a420-8718-4be3-ab2a-b1a3685b8c30",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Running iteratively without Skyhook `FuturesExecutor`\n",
    "\n",
    "Run the same job again, but now iteratively without Skyhook. The calls to the Dataset API will now be sequential."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e4551f3e-1679-4bc8-baa2-49cb629be7f7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "05c8e9e40cb946399b3e0708129a09bf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
/opt/conda/lib/python3.8/site-packages/coffea/nanoevents/schemas/nanoaod.py:193: \n",
       "RuntimeWarning: Missing cross-reference index for FatJet_genJetAK8Idx => GenJetAK8\n",
       "  warnings.warn(\n",
       "
\n"
      ],
      "text/plain": [
       "/opt/conda/lib/python3.8/site-packages/coffea/nanoevents/schemas/nanoaod.py:193: \n",
       "RuntimeWarning: Missing cross-reference index for FatJet_genJetAK8Idx => GenJetAK8\n",
       "  warnings.warn(\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
/opt/conda/lib/python3.8/site-packages/coffea/nanoevents/schemas/nanoaod.py:193: \n",
       "RuntimeWarning: Missing cross-reference index for FatJet_genJetAK8Idx => GenJetAK8\n",
       "  warnings.warn(\n",
       "
\n"
      ],
      "text/plain": [
       "/opt/conda/lib/python3.8/site-packages/coffea/nanoevents/schemas/nanoaod.py:193: \n",
       "RuntimeWarning: Missing cross-reference index for FatJet_genJetAK8Idx => GenJetAK8\n",
       "  warnings.warn(\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
\n",
       "
\n"
      ],
      "text/plain": [
       "\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 6.43 s, sys: 174 ms, total: 6.61 s\n",
      "Wall time: 6.59 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "run = processor.Runner(\n",
    "    executor=processor.IterativeExecutor(compression=None),\n",
    "    use_skyhook=False,\n",
    "    format=\"parquet\",\n",
    "    schema=schemas.NanoAODSchema,\n",
    ")\n",
    "\n",
    "hists = run(\n",
    "    {\n",
    "        \"ZJets\": \"/mnt/cephfs/nanoevents/ZJets\",\n",
    "        \"Data\": \"/mnt/cephfs/nanoevents/Data\",\n",
    "    },\n",
    "    \"Events\",\n",
    "    processor_instance=MyZPeak(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "consolidated-panel",
   "metadata": {},
   "source": [
    "As expected, much slower than running using Dask."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "197b0470-3441-479d-b062-25733d207a24",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Running `DaskExecutor` without Skyhook\n",
    "\n",
    "Run the same job again, but with `DaskExecutor` abd without Skyhook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "58b40f75-c833-4ef8-b017-9325b8413b0c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 184 ms, sys: 23.5 ms, total: 208 msompleted |  4.4s\n",
      "Wall time: 4.49 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "run = processor.Runner(\n",
    "    executor=processor.DaskExecutor(client=client),\n",
    "    use_skyhook=False,\n",
    "    format=\"parquet\",\n",
    "    schema=schemas.NanoAODSchema,\n",
    ")\n",
    "\n",
    "hists = run(\n",
    "    {\n",
    "        \"ZJets\": \"/mnt/cephfs/nanoevents/ZJets\",\n",
    "        \"Data\": \"/mnt/cephfs/nanoevents/Data\",\n",
    "    },\n",
    "    \"Events\",\n",
    "    processor_instance=MyZPeak(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "surrounded-petroleum",
   "metadata": {},
   "source": [
    "## Plotting the results\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "collectible-applicant",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.8/site-packages/numpy/core/_methods.py:44: RuntimeWarning: invalid value encountered in reduce\n",
      "  return umr_minimum(a, axis, None, out, keepdims, initial, where)\n",
      "/opt/conda/lib/python3.8/site-packages/numpy/core/_methods.py:40: RuntimeWarning: invalid value encountered in reduce\n",
      "  return umr_maximum(a, axis, None, out, keepdims, initial, where)\n",
      "/opt/conda/lib/python3.8/site-packages/numpy/core/_methods.py:44: RuntimeWarning: invalid value encountered in reduce\n",
      "  return umr_minimum(a, axis, None, out, keepdims, initial, where)\n",
      "/opt/conda/lib/python3.8/site-packages/numpy/core/_methods.py:40: RuntimeWarning: invalid value encountered in reduce\n",
      "  return umr_maximum(a, axis, None, out, keepdims, initial, where)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       ""
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEGCAYAAABPdROvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAcVklEQVR4nO3df5xVdb3v8debYRAhNJNBUDTAB3kThIlGlEPyECsRM02rI+AxJZPUKL2d6w3zxkmOoR0JvWoH5KqQpuSxQj2CCnnFH6knRy6/UUHCHAEZqfAHys/P/WMvcDPsGTazZu89M7yfj8d+7PXju9b+fIV6s359lyICMzOzxmpT6gLMzKxlc5CYmVkqDhIzM0vFQWJmZqk4SMzMLJW2pS6gKXXu3Dl69OhR6jLM6rW69gMAelV0LHElZhkvv/zyOxFRkWYfrSpIevToQXV1danLMKvX+Xe8AMAD3x1U4krMMiS9kXYfPrVlZmapOEjMzCwVB4mZmaXSqq6RmJnla9u2bdTU1PDRRx+VupSiaN++Pd27d6e8vLzJ9+0gMbMDUk1NDZ06daJHjx5IKnU5BRURbNy4kZqaGnr27Nnk+/epLTM7IH300UccfvjhrT5EACRx+OGHF+zoy0FiZgesAyFEdilkXx0kZmaWiq+RmBXYdf+5jOVr3wVg+brM964HE48/8hD+5at9Slab5a+srIwTTjiBbdu20bZtWy666CKuuuoq2rSp/9/ja9as4fnnn2fUqFFNWsstt9zCmDFj6NChQ5Put7F8RGJWYMvXvrs7QPZYvu7d3QFjzd/BBx/MwoULWbZsGfPmzWPOnDlcd911DW6zZs0a7r///iav5ZZbbmHz5s1Nvt/GcpCYFcHx3Q7hge8O4vhuh+wxbS1Tly5dmDZtGrfffjsRwZo1azjllFMYMGAAAwYM4Pnnnwdg3LhxPPvss1RWVnLzzTfX227dunUMGTKEyspK+vbty7PPPgvA3LlzGTRoEAMGDOCb3/wm77//Prfeeitr165l6NChDB06tGT/DbIV7NSWpLuBs4ANEdE3WfYAcFzS5JPA3yOiMse2a4D3gB3A9oioKlSdZmaN0atXL3bu3MmGDRvo0qUL8+bNo3379qxcuZKRI0dSXV3NjTfeyKRJk3j00UcB2Lx5c852999/P8OGDePaa69lx44dbN68mXfeeYfrr7+eP/zhD3Ts2JGf//znTJ48mfHjxzN58mSeeuopOnfuXOL/ChmFvEYyA7gduGfXgog4f9e0pF8AmxrYfmhEvFOw6szMUooIIPNw49ixY1m4cCFlZWW89tprOdvX1+7EE0/k29/+Ntu2beNrX/salZWVPP300yxfvpzBgwcDsHXrVgYNap6DfRYsSCLiGUk9cq1T5j60fwROK9Tvm5kV0urVqykrK6NLly5cd911HHHEESxatIidO3fSvn37nNvcfPPNOdsNGTKEZ555htmzZ3PhhRdy9dVXc9hhh/HlL3+ZmTNnFrNbjVKqaySnAG9HxMp61gcwV9LLksY0tCNJYyRVS6qura1t8kLNzOqqra3lsssuY+zYsUhi06ZNdOvWjTZt2nDvvfeyY8cOADp16sR77723e7v62r3xxht06dKFSy+9lEsuuYQFCxZw8skn88c//pFVq1YBmdNiu45g6u631Ep1++9IoKGYHRwRayV1AeZJeiUinsnVMCKmAdMAqqqqoulLNTODDz/8kMrKyt23/1544YX88Ic/BOCKK67g61//Og8++CBDhw6lY8fMi8v69etH27Zt6d+/PxdffHG97ebPn89NN91EeXk5n/jEJ7jnnnuoqKhgxowZjBw5ki1btgBw/fXX85nPfIYxY8YwfPhwunXrxlNPPVWa/yBZtOscX0F2njm19eiui+3JsrbAW8DnI6Imj338FHg/Iibtq21VVVX4xVbW3GS/zKq+aSu+FStW8NnPfrbUZRRVrj5LejntDU2lOLX1JeCV+kJEUkdJnXZNA6cDS4tYn5mZ7YeCBYmkmcALwHGSaiRdkqwaQZ3TWpKOlDQnmT0CeE7SIuBPwOyIeLxQdZqZWTqFvGtrZD3LL86xbC1wZjK9GuhfqLrMzKxp+cl2MzNLxUFiZmapOEjMzPJ0/h0v7L7bzj7mIDEzK5GysjIqKyvp06cP/fv3Z/LkyezcubPBbQo1onAaDhIzsxJpTkPTp+EgMcvX9K9kPmYF0NRD0xeT35BoZge87LdYZqv7QrLNW7YDcMJPn9hjea53yzTm7ZdNOTR9MTlIzMyakaYamr6YHCRmdsDL98ih0OOjNeXQ9MXkayRmZs1AUw9NX0w+IjEzK5FCDk1fTA4SM7MSaejooXfv3ixevHj3/A033ABAeXk5Tz755B5tc7UrJgeJmVme/O6Y3HyNxMzMUnGQmJlZKg4SMzNLxUFiZmapOEjMzPLl8dZycpCYmZXIrFmzqKys3OPTpk0bpkyZQt++fRvctjmNAuwgMTMrkXPPPZeFCxfu/lxxxRWccsopDBs2bJ/bHhBBIuluSRskLc1a9lNJb0lamHzOrGfbMyS9KmmVpHGFqtHMrLl47bXXmDBhAvfeey9t2nz8f807duzg6quv5sQTT6Rfv37ccccdwN7DyS9btoyBAwdSWVlJv379WLlyZdFqL+QDiTOA24F76iy/OSIm1beRpDLgl8CXgRrgJUmPRMTyQhVqZge4x8bB+iV7L1+/eM/5rR9kvm84es/lXfvtvW3XE2D4jXn9/LZt2xg1ahSTJk3imGOOYc2aNbvX3XXXXRx66KG89NJLbNmyhcGDB3P66afvNZz897//fa688kouuOACtm7dWtQxtwoWJBHxjKQejdh0ILAqIlYDSPoNcA7gIDGzVuknP/kJffr0YcSIEXutmzt3LosXL+a3v/0tkBmkceXKlbRr126PdoMGDeJnP/sZNTU1nHfeefTu3bsotUNphkgZK+lbQDXwzxHxtzrrjwLezJqvAU6qb2eSxgBjAI455pgmLtXMDgh5HjnsvmNr9Owm++n58+fzu9/9jgULFuRcHxHcdttte103mT9//h7zo0aN4qSTTmL27NkMGzaMO++8k9NOO63J6mxIsS+2TwGOBSqBdcAvcrRRjmVR3w4jYlpEVEVEVUVFRZMUaWZWDH/7298YPXo099xzD506dcrZZtiwYUyZMoVt27YBmWspH3zwwV7Dya9evZpevXrxgx/8gLPPPnuPgRwLrahHJBHx9q5pSf8HeDRHsxog+wRkd2BtgUszMyu6qVOnsmHDBi6//PI9lo8cOZKDDjoIgO985zusWbOGAQMGEBFUVFTw0EMP7TWc/EcffcSvf/1rysvL6dq1K+PHjy9aP4oaJJK6RcS6ZPZcYGmOZi8BvSX1BN4CRgCjilSimVnRXHPNNVxzzTV7LX/44Yc59thjAWjTpg0TJ05k4sSJe7WrO5x8rn0VQ8GCRNJM4FSgs6Qa4F+AUyVVkjlVtQb4btL2SODOiDgzIrZLGgs8AZQBd0fEskLVaWaWtya8NlKf8ePH8/DDDzNjxoyC/1ZTKeRdWyNzLL6rnrZrgTOz5ucAcwpUmplZszVhwgQmTJhQ6jL2i59sN7MDVkS99/G0OoXsq4PEzA5I7du3Z+PGjQdEmEQEGzdupH379gXZv1+1a2YHpO7du1NTU0NtbW2pSymK9u3b071794Ls20FiZgek8vJyevbsWeoyWgWf2jIzs1QcJGZmloqDxMzMUnGQmJlZKr7YblZED3x3UKlLMGtyPiIxM7NUHCRmZpaKg8TMzFJxkJiZWSoOEjMzS8VBYmZmqThIzMwsFQeJmZml4iAxM7NUHCRmZpaKg8TMzFIpWJBIulvSBklLs5bdJOkVSYslzZL0yXq2XSNpiaSFkqoLVaOZmaVXyCOSGcAZdZbNA/pGRD/gNeCaBrYfGhGVEVFVoPrMzKwJFCxIIuIZ4K91ls2NiO3J7ItAYV4gbGZmRVPKayTfBh6rZ10AcyW9LGlMQzuRNEZStaTq2traJi/SzMwaVpIgkXQtsB24r54mgyNiADAc+J6kIfXtKyKmRURVRFRVVFQUoFozM2tI0YNE0kXAWcAFERG52kTE2uR7AzALGFi8Cs3MbH8UNUgknQH8CDg7IjbX06ajpE67poHTgaW52pqZWekV8vbfmcALwHGSaiRdAtwOdALmJbf2Tk3aHilpTrLpEcBzkhYBfwJmR8TjharTzMzSKdg72yNiZI7Fd9XTdi1wZjK9GuhfqLrM9stj42D9ksz0+sWZ7+lfyXx3PQGG31iausyaET/ZbtaQ9Us+DpJ8lpsdgBwkZvvS9QQYPRu69st8Rs/OLDMzwEFiZmYpOUjMzCwVB4mZmaXiIDEzs1QcJGZmloqDxMzMUnGQmJlZKg4SMzNLxUFiZmapOEjMzCwVB4mZmaXiIDEzs1QcJGZmlsp+B4mkwyT1K0QxZmbW8uQVJJLmSzpE0qeARcB0SZMLW5qZmbUE+R6RHBoR7wLnAdMj4vPAlwpXlpmZtRT5BklbSd2AfwQeLWA9ZmbWwuQbJNcBTwCrIuIlSb2AlYUry8zMWop8g2RdRPSLiCsAImI10OA1Ekl3S9ogaWnWsk9JmidpZfJ9WD3bniHpVUmrJI3LtzNmZlZ8+QbJbXkuyzYDOKPOsnHAkxHRG3gymd+DpDLgl8Bw4HhgpKTj86zTzMyKrG1DKyUNAv4BqJD0w6xVhwBlDW0bEc9I6lFn8TnAqcn0r4D5wI/qtBlI5hTa6qSG3yTbLW/o98zMrDT2dUTSDvgEmcDplPV5F/hGI37viIhYB5B8d8nR5ijgzaz5mmRZTpLGSKqWVF1bW9uIkszMLI0Gj0gi4mngaUkzIuKNItWkXKXU1zgipgHTAKqqquptZ2ZmhdFgkGQ5SNI0oEf2NhFx2n7+3tuSukXEuuR24g052tQAR2fNdwfW7ufvmJlZkeQbJA8CU4E7gR0pfu8R4CLgxuT74RxtXgJ6S+oJvAWMAEal+E0zMyugfINke0RM2Z8dS5pJ5sJ6Z0k1wL+QCZD/kHQJ8Bfgm0nbI4E7I+LMiNguaSyZ51bKgLsjYtn+/LaZmRVPvkHyn5KuAGYBW3YtjIi/1rdBRIysZ9UXc7RdC5yZNT8HmJNnbWbNzrKJXwCgz4+f23PF9K9kvkfPbtofLNR+zfKQb5BclHxfnbUsgF5NW46ZmbU0eQVJRPQsdCFmZtYy5TuMfAdJ/yu5cwtJvSWdVdjSzMysJch3iJTpwFYyT7lD5hbd6wtSkZmZtSj5BsmxEfFvwDaAiPiQ3A8OmpnZASbfINkq6WCSJ8wlHUvW3VtmZnbgyveurZ8CjwNHS7oPGAxcXKCazMysBcn3rq25kl4GTiZzSuvKiHinoJWZmVmLkFeQSHoEmAk8EhEfFLYkMzNrSfK9RvIL4BRguaQHJX1DUvsC1mVmZi1Evqe2dg0nXwacBlwK3E3mBVdmBwYPP2KWU74X20nu2voqcD4wgMwbDs3M7ACX7zWSB4CTyNy59UtgfkTsLGRhZmbWMuR7RDIdGBURad5FYmZmrVCDF9sl/U+AiHgcOK/OuokFrMvMzFqIfd21NSJr+po6685o4lrMzKwF2leQqJ7pXPNmZnYA2leQRD3TuebNzOwAtK+L7f0lvUvm6OPgZJpk3g8kmplZw0ckEVEWEYdERKeIaJtM75ovb8wPSjpO0sKsz7uSrqrT5lRJm7LajG/Mb5mZWeHl/UBiU4mIV4FKgORJ+beAWTmaPhsRfgujmVkzl+9YW4XyReD1iHijxHWYmVkjlTpIRpAZVTiXQZIWSXpMUp9iFmVmZvkrWZBIagecDTyYY/UC4NMR0R+4DXiogf2MkVQtqbq2trYgtZqZWf1KeUQyHFgQEW/XXRER70bE+8n0HKBcUudcO4mIaRFRFRFVFRUVha3YzMz2UsogGUk9p7UkdZWkZHogmTo3FrE2MzPLU9Hv2gKQ1AH4MvDdrGWXAUTEVOAbwOWStgMfAiMiwg9Ampk1QyUJkojYDBxeZ9nUrOnbgduLXZeZme2/Ut+1ZWZmLZyDxMzMUnGQmJlZKg4SMzNLxUFiZmapOEjMzCwVB4mZmaXiIDEzs1QcJGZmloqDxMzMUnGQmJlZKiUZa8usNXrx3y+l099XALDzo/cBWDbxC/yPrTvo0K4Mph8K6xdnGk//CuM3buKNtscCgxr3g4+Ng/VLMtNZ+wWg6wkw/MZG9sRs//iIxKyJdPr7Co7e+vpeyzu0K6Nju73/zdZj22o+vX3v9nlbv+TjIMlnuVmB+IjErAm92e5Y+vz4OZZN/AIAfX783J4Ndh0xjJ7NmqRNKl1PgNGz99jv7mmzIvERiZmZpeIgMTOzVBwkZmaWioPEzMxScZCYmVkqDhIzM0ulJEEiaY2kJZIWSqrOsV6SbpW0StJiSQNKUaeZme1bKZ8jGRoR79SzbjjQO/mcBExJvs3MrJlprqe2zgHuiYwXgU9K6lbqoszMbG+lCpIA5kp6WdKYHOuPAt7Mmq9Jlu1F0hhJ1ZKqa2trC1CqmZk1pFRBMjgiBpA5hfU9SUPqrFeObSLXjiJiWkRURURVRUVFU9dpZmb7UJIgiYi1yfcGYBYwsE6TGuDorPnuwNriVGdmZvuj6EEiqaOkTrumgdOBpXWaPQJ8K7l762RgU0SsK3KpZmaWh1LctXUEMEvSrt+/PyIel3QZQERMBeYAZwKrgM3A6BLUaWZmeSh6kETEaqB/juVTs6YD+F4x6zIzs8Zprrf/mplZC+EgMTOzVBwkZmaWioPEzMxScZCYmVkqDhIzM0vFQWJmZqk4SMzMLBUHiZmZpeIgMTOzVEr5hkSzVqvPj5/LvWL07ML8YKH2a5YHH5GYmVkqDhIzM0vFQWJmZqk4SMzMLBUHiZmZpeIgMTOzVBwkZmaWioPEzMxScZCYmVkqRQ8SSUdLekrSCknLJF2Zo82pkjZJWph8xhe7TjMzy08phkjZDvxzRCyQ1Al4WdK8iFhep92zEXFWCeozM7P9UPQjkohYFxELkun3gBXAUcWuw8zMmkZJr5FI6gF8DvivHKsHSVok6TFJfRrYxxhJ1ZKqa2trC1WqmZnVo2RBIukTwO+AqyLi3TqrFwCfjoj+wG3AQ/XtJyKmRURVRFRVVFQUrF4zM8utJEEiqZxMiNwXEb+vuz4i3o2I95PpOUC5pM5FLtPMzPJQiru2BNwFrIiIyfW06Zq0Q9JAMnVuLF6VZmaWr1LctTUYuBBYImlhsuzHwDEAETEV+AZwuaTtwIfAiIiIEtRqZmb7UPQgiYjnAO2jze3A7cWpyMzM0vCT7WZmloqDxMzMUnGQmJlZKg4SMzNLxUFiZmapOEjMzCwVB4mZmaXiIDEzs1QcJGZmloqDxMzMUnGQmJlZKg4SMzNLxUFiZmapOEjMzCwVB4mZmaXiIDEzs1QcJGZmloqDxMzMUnGQmJlZKg4SMzNLpSRBIukMSa9KWiVpXI71knRrsn6xpAGlqNPMzPat6EEiqQz4JTAcOB4YKen4Os2GA72TzxhgSlGLNDOzvLUtwW8OBFZFxGoASb8BzgGWZ7U5B7gnIgJ4UdInJXWLiHUN7fjDda+wbOIXClW3HYB6bFvNmvJeTLjjhT2Wj9+4KbMu6+/b0Vtf5812x+7X/o/e+nqj/87uT21mhVSKIDkKeDNrvgY4KY82RwF7BYmkMWSOWgC29L32j0ubrtRmpTPwTqmLKKBm3L91wD/sseQ/9liXbT1cq1w7aaB/64tUW0E14z+/JtGa+3dc2h2UIkhy/a8sGtEmszBiGjANQFJ1RFSlK695as19A/evpXP/Wi5J1Wn3UYqL7TXA0Vnz3YG1jWhjZmbNQCmC5CWgt6SektoBI4BH6rR5BPhWcvfWycCmfV0fMTOz0ij6qa2I2C5pLPAEUAbcHRHLJF2WrJ8KzAHOBFYBm4HRee5+WgFKbi5ac9/A/Wvp3L+WK3XflLkxyszMrHH8ZLuZmaXiIDEzs1RabJAkDyn+VtIrklZIGiTpU5LmSVqZfB9W6jobQ9JxkhZmfd6VdFUr6t9/l7RM0lJJMyW1by19A5B0ZdK3ZZKuSpa12P5JulvSBklLs5bV2x9J1yTDG70qaVhpqs5fPf37ZvLnt1NSVZ32raF/NyX/37lY0ixJn8xat9/9a7FBAvxv4PGI+G9Af2AFMA54MiJ6A08m8y1ORLwaEZURUQl8nswNB7NoBf2TdBTwA6AqIvqSueFiBK2gbwCS+gKXkhnBoT9wlqTetOz+zQDOqLMsZ3+S4Y5GAH2Sbf49GRapOZvB3v1bCpwHPJO9sBX1bx7QNyL6Aa8B10Dj+9cig0TSIcAQ4C6AiNgaEX8nM7TKr5JmvwK+Vor6mtgXgdcj4g1aT//aAgdLagt0IPOMUGvp22eBFyNic0RsB54GzqUF9y8ingH+Wmdxff05B/hNRGyJiD+TufNyYDHqbKxc/YuIFRHxao7mraV/c5O/nwAvknlWDxrZvxYZJEAvoBaYLun/SbpTUkfgiF3PmyTfXUpZZBMZAcxMplt8/yLiLWAS8BcyY3hsioi5tIK+JZYCQyQdLqkDmdvYj6b19G+X+vpT3/BGrUVr7N+3gceS6Ub1r6UGSVtgADAlIj4HfEDLOlWQl+SBzbOBB0tdS1NJzqWfA/QEjgQ6Svqn0lbVdCJiBfBzMqcOHgcWAdsb3Kh1yXt4oxaqVfVP0rVk/n7et2tRjmb77F9LDZIaoCYi/iuZ/y2ZYHlbUjeA5HtDieprKsOBBRHxdjLfGvr3JeDPEVEbEduA35MZdbA19A2AiLgrIgZExBAypxRW0or6l6ivP619eKNW0z9JFwFnARfExw8UNqp/LTJIImI98KakXaNWfpHMMPSPABclyy4CHi5BeU1pJB+f1oLW0b+/ACdL6iBJZP7sVtA6+gaApC7J9zFkLtjOpBX1L1Fffx4BRkg6SFJPMu8U+lMJ6iuUVtE/SWcAPwLOjojNWasa17+IaJEfoBKoBhYDDwGHAYeTuYNkZfL9qVLXmaJ/HYCNwKFZy1pF/4DrgFfIXE+4FziotfQt6d+zZP5hswj4Ykv/syMThOuAbWT+xXpJQ/0BrgVeB14Fhpe6/kb279xkegvwNvBEK+vfKjLXQhYmn6lp+uchUszMLJUWeWrLzMyaDweJmZml4iAxM7NUHCRmZpaKg8TMzFJxkJg1QNK5dUZiXpiMCDu81LWZNRe+/ddsP0gaA1wADI2InaWux6w58BGJWZ4kfQYYD1xYN0Qk9Uje73Bn8i6S+yR9SdIfk3d2DEzaDZT0fDLY6PO7RmeQ1EfSn5IjnsWSekvqKGm2pEXJPs8vfq/N9s1HJGZ5kFQOvABMiojf5Fjfg8zTwp8DlgEvkXmy/RIyA2+OjoivJa9A2BwR2yV9Cbg8Ir4u6TYyw8/flwzWWUZm5OAzIuLS5DcOjYhNBe+s2X5qW+oCzFqIfwWW5QqRLH+OiCUAkpaRefFTSFoC9EjaHAr8KnnZVQDlyfIXgGsldQd+HxErk+0mSfo58GhEPNv03TJLz6e2zPZB0qnA14Gx+2i6JWt6Z9b8Tj7+R9u/Ak9F5u2QXwXaA0TE/WSOXD4EnpB0WkS8RuYNmUuAGySNT90ZswLwEYlZA5L3p0wHRkXEe02wy0OBt5Lpi7N+pxewOiJuTab7SXoF+GtE/FrS+9ntzZoTB4lZwy4j8/a/KZlR73e7ISIeaMT+/o3Mqa0fAv83a/n5wD9J2gasByYAJwI3SdpJZuTWyxvxe2YF54vtZmaWiq+RmJlZKg4SMzNLxUFiZmapOEjMzCwVB4mZmaXiIDEzs1QcJGZmlsr/BxNCimJ1blkkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "
"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "hist.plot1d(hists)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}