{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Volumetric Benchmarks\n",
    "This tutorial demonstrates how to compute volumetric benchmarks in pyGSTi.  Volumetric benchmarks map a (*width*, *depth*) pair to a test suite of circuits with (at least approximately) the given width and depth, and define an overall success measure that lies between 0 and 1 (1 indicating better performance on the test suite).  Thus, by collecting the success measures for many (width, depth) pairs, one can explore, in addition to the overall processor performance, the tradeoff between a quantum processor's ability to perform deep vs. wide circuits.  For more information on the theory and motivation for volumetric benchmarks, see [this paper](https://arxiv.org/abs/1904.05546).\n",
    "\n",
    "We'll begin by importing pyGSTi as usual, and making `pp` a shorthand for `pygsti.protocols` since we'll be using it a lot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pygsti\n",
    "import pygsti.protocols as pp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the processor\n",
    "Next, we define the processor that we're benchmarking.  For this we use a `ProcessorSpec` object (see the [tutorial on processor specs](../objects/advanced/ProcessorSpec.ipynb)) to define a ring of 4 qubits.  Each qubit has 4 single-qubit gates: $X(\\pm\\pi/2)$ and $Y(\\pm\\pi/2)$, and CPHASE gates are allowed between neighbors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "nQubits = 4\n",
    "qubit_labels = ['Q0', 'Q1', 'Q2', 'Q3']\n",
    "gate_names = ['Gxpi2', 'Gxmpi2', 'Gypi2', 'Gympi2', 'Gcphase']\n",
    "availability = {'Gcphase': [('Q0', 'Q1'), ('Q1', 'Q2'), ('Q2', 'Q3'), ('Q3', 'Q0')]}\n",
    "\n",
    "pspec = pygsti.obj.ProcessorSpec(nQubits, gate_names, availability=availability, \n",
    "                                 qubit_labels=qubit_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: create an experiment design\n",
    "\n",
    "There are many types of volumetric benchmarks.  In this example, we'll use associate a suite of random circuits with various (width, depth) pairs by using direct-RB (DRB) circuits (a family of test suites for different *depths*) on different portions of the processor and therefore for different *widths*.\n",
    "\n",
    "We create `DirectRBDesign` experiment design objects, giving a different `qubit_labels` argument each time.  To each design we add a `VolumetricBenchmark` \"default protocol\" to make it easier to run the protocols later (see step 3).  We decide to save some time (perhaps at the expense of increased crosstalk error) by performing some of these experiment designs simultaneously.  This is achieved by using multiple \"sub-designs\" to construct a `SimultaneousExperimentDesign`.  We create two simultaneous designs (one which performs two 2-qubit DRB test suites at the same time, the other which performs four 1-qubit DRB suites), and combine these using a `CombinedExperimentDesign`.  (Note that if we didn't want to run any of the suites simultaneously, we could have just combined all six DRB designs under a `CombinedExperimentDesign`.)\n",
    "\n",
    "In this way, the cell below defines the entire experiment we want to perform, and all the circuits we need to run are nicely bundled into a single experiment design (`entire_design`), which we write to disk and await data collection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "depths = [0, 5, 10, 15, 20]\n",
    "circuits_per_depth = 30\n",
    "\n",
    "VB_design01 = pp.DirectRBDesign(pspec, depths, circuits_per_depth, qubit_labels=['Q0', 'Q1'])\n",
    "VB_design23 = pp.DirectRBDesign(pspec, depths, circuits_per_depth, qubit_labels=['Q2', 'Q3'])\n",
    "VB_design01.add_default_protocol(pp.VolumetricBenchmark())\n",
    "VB_design23.add_default_protocol(pp.VolumetricBenchmark())\n",
    "\n",
    "designS1 = pp.SimultaneousExperimentDesign([VB_design01, VB_design23], qubit_labels=qubit_labels)\n",
    "\n",
    "VB_design0 = pp.DirectRBDesign(pspec, depths, circuits_per_depth, qubit_labels=['Q0'])\n",
    "VB_design1 = pp.DirectRBDesign(pspec, depths, circuits_per_depth, qubit_labels=['Q1'])\n",
    "VB_design2 = pp.DirectRBDesign(pspec, depths, circuits_per_depth, qubit_labels=['Q2'])\n",
    "VB_design3 = pp.DirectRBDesign(pspec, depths, circuits_per_depth, qubit_labels=['Q3'])\n",
    "VB_design0.add_default_protocol(pp.VolumetricBenchmark())\n",
    "VB_design1.add_default_protocol(pp.VolumetricBenchmark())\n",
    "VB_design2.add_default_protocol(pp.VolumetricBenchmark())\n",
    "VB_design3.add_default_protocol(pp.VolumetricBenchmark())\n",
    "\n",
    "designS2 = pp.SimultaneousExperimentDesign([VB_design0, VB_design1, VB_design2, VB_design3], qubit_labels=qubit_labels)\n",
    "\n",
    "entire_design = pp.CombinedExperimentDesign({\"specA\": designS1, \"specB\": designS2})\n",
    "pygsti.io.write_empty_protocol_data(entire_design, \"../tutorial_files/vb_example\", clobber_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: collect data as specified by the experiment design\n",
    "Next, we just follow the instructions in the experiment design to collect data from the quantum processor.  In this example, we'll generate the data using a depolarizing noise model since we don't have a real quantum processor lying around.  The cell below simulates taking the data, and would be replaced with the user filling out the empty \"template\" data set file with real data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "mdl_datagen = pygsti.obj.ProcessorSpec(nQubits, gate_names, availability=availability,                                                                                                                       \n",
    "                                       qubit_labels=qubit_labels, construct_models=('TP',)).models['TP']\n",
    "for gate in mdl_datagen.operation_blks['gates'].values(): gate.depolarize(0.01)\n",
    "pygsti.io.fill_in_empty_dataset_with_fake_data(mdl_datagen, \"../tutorial_files/vb_example/data/dataset.txt\",\n",
    "                                               nSamples=1000, seed=2020)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the template file has been filled in (or just replaced with one containing data), we read it in from the same root directory we saved the data to above.  This loads in both the `dataset.txt` we simulated above and the experiment design (saved in the `.../vb_example/edesign` directory)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pygsti.io.load_data_from_dir('../tutorial_files/vb_example')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Run the volumetric benchmark protocol on each DRB experiment design\n",
    "Now that we have the data, we'd like to run the `VolumetricBenchmark` protocol on each of the DRB sub-designs defined above.  PyGSTi's protocol-object framework has kept track of all the nested experiment designs, and since we had the foresight to add a `VolumetricBenchmark` object as the default protocol to each DRB sub-design, we can easily run all of these protocols by calling `run_default_protocols`.  This function walks through the tree of experiment designs and runs any and all default protocols."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = pp.run_default_protocols(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Look at the results\n",
    "Volumetric benchmarks (VB) are fairly new to pyGSTi, and we don't have a nice built-in plot for displaying collections of VB data.  Below we demonstrate how the returned results object can be converted into a [Pandas](https://pandas.pydata.org) data-frame, which allows the user to easily slice the data and create plots using their favorite plotting tools.  We demonstrate how this is done with pyGSTi's preferred plotting library, [Plotly](https://plot.ly/python)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = results.as_dataframe()  # you'll need the 'pandas' python package for this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()  # The raw data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Filter the data\n",
    "df_relevant_cols = df.loc[:, ['value', 'Qty', 'Depth', 'Width']]\n",
    "df_vb = df_relevant_cols[ df_relevant_cols['Qty'] == 'volumetric_benchmarks' ].loc[:, ['value', 'Depth', 'Width']]\n",
    "df_vb.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Get the data to plot\n",
    "widths = sorted(df_vb.Width.unique())\n",
    "depths = sorted(df_vb.Depth.unique())\n",
    "vals = [ [ df_vb[(df_vb['Depth'] == d) & (df_vb['Width'] == w) ]['value'].mean()\n",
    "            for d in depths ]  for w in widths ]    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Make the plot (you'll need the 'plotly' python package for this)\n",
    "import plotly.graph_objects as go \n",
    "fig = go.Figure(data=go.Heatmap(z=vals, x=depths, y=widths, colorscale='Bluered_r'))\n",
    "fig.update_layout(title='Volumetric benchmarking example',\n",
    "                  xaxis={'title': 'Depth'}, yaxis={'title': 'Width'},\n",
    "                  height=300, width=400)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}