{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Different ways to run Gate Set Tomography"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `pygsti` package provides multiple ways to use its core Gate Set Tomography (GST) algorithms.  This  tutorial will show you how to work with pyGSTi's GST protocol objects to perform GST in different ways with a minimial amount of effort.  In order to run the GST protocol there are 3 essential ingredients: 1) an \"experiment design\" specifying the structure of the GST circuits and how the data should be collected, 2) the outcome counts for the circuits specified by the experiment design, and 3) a desired, or \"target\", `Model`.  The [GST overview tutorial](GST-Overview.ipynb), gave an end-to-end example of how to construct a GST experiment design, run GST, and generate a report.  This tutorial focuses on the first and second steps in more detail; related information about circuit construction and report generation can be found in the [GST circuits tutorial](../objects/advanced/GSTCircuitConstruction.ipynb) and [report generation tutorial](../reporting/ReportGeneration.ipynb).\n",
    "\n",
    "There are two different `Protocol` objects within pyGSTi for running GST:\n",
    "\n",
    "- [`GateSetTomography`](#gatesettomography) - runs a single model optimization based on a *given* initial model that can have any parameterization you like.  This protocol can be run on any `GateSetTomographyDesign` experiment design, which only needs a target model (to describe what gates occur in the circuits) and a list of circuit lists to specify the circuits used for each iteration of the model optimization.\n",
    "\n",
    "- [`StandardGST`](#standardgst) - runs multiple model optimizations based on an `ExplicitOpModel` target model by parameterizing this model in different ways.  The target model is expected to be a part of the experiment design, and only `StandardGSTDesign`-type experiment designs are allowed since the usual germs-and-fiducials structure of the GST circuits is expected.\n",
    "\n",
    "Overall, the `GateSetTomography` protocol is more flexible than the `StandardGST` protocol, but requires a little more work to get going because its inputs are more complicated.  Both protocols return a `ModelEstimateResults` object when they are run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pygsti"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup\n",
    "In the [DataSet tutorial](../objects/DataSet.ipynb) we simulate the circuits required by a GST experiment design and save the results.  In this tutorial, we'll be analyzing that data.  This illustrates a typical workflow where at some earlier time you setup an experiment (a \"GST experiment in this case) and save the experiment design to disk and at some later time (after the data has been collected) you want to analyze it.  Now *is* that later time, and we start by reading the the data we've collected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pygsti.io.read_data_from_dir(\"../tutorial_files/Example_GST_Data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gatesettomography\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `GateSetTomography`\n",
    "This protocol performs a single model optimization, and so computes a **single GST estimate** given a `DataSet`, a target `Model`, and other parameters.  (The returned `ModelEstimateResults` object may sometimes contain multiple related estimates in certain cases, but in these cases all the estimates are closely related.)  The experiment design provides all of the information about the GST circuits, in this case a *standard*  (*prep_fiducial + germ^power + meas_fiducial*) set, so the only thing needed by the protocol is an initial `Model` to optimize.  Thus, the `GateSetTomography` protocol is essentially just a model optimizer that you give an initial point.  Importantly, this initial point (a `Model`) also specifies the *parameterization*, i.e. the space of parameters that are optimized over.\n",
    "\n",
    "Minimally, when using `GateSetTomography` you should set the parameterization of the initial model.  This can be viewed as setting the constraints on the optimization.  For instance, when the gates in the model are parameterized as trace-preserving (TP) maps, the optimization will be constrained to trying gate sets with TP gates (because every set of parameters corresponds to a set of TP gates).  In the cell below, we constrain the optimization to TP gate sets by using `.target_model(\"full TP\")`, which returns a version of the target model where all the gates are TP-parameterized, the state preparation has trace = 1, and the POVM effects always add to the identity.  This could also be done by calling `set_all_parameterizations(\"TP\")` on the fully-parameterized target model returned by `.target_model()`.  See the [tutorial on explicit models](../objects/ExplicitModel.ipynb) for more information on setting a model's parameterization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from pygsti.modelpacks import smq1Q_XYI\n",
    "target_model_TP = smq1Q_XYI.target_model(\"full TP\")\n",
    "proto = pygsti.protocols.GateSetTomography(target_model_TP)\n",
    "results_TP = proto.run(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A summary of what's inside a Results object is obtained by printing it\n",
    "(for more examples of how to use a Results object, see the [Results tutorial](../objects/advanced/Results.ipynb))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(results_TP)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gauge optimization parameters\n",
    "The `gaugeopt_suite` argument specifies a set of gauge optimizations to be performed on the final GST estimate.  It is a dictionary whose keys are gauge-optimization names (these can be whatever you want) and whose values are dictionaries of arguments ultimately to be passed to the `gaugeopt_to_target` function (which provides full documentation).  (For example, by specifying `item_weights` we can set the ratio of the state preparation and measurement (SPAM) weighting to the gate weighting when performing a gauge optimization.)  In lieu of a dictionary of `gaugeopt_to_target` arguments, the elements of `gaugeopt_suite` may also be strings which name a built-in set of gauge optimizations (e.g. `\"stdgaugeopt\"` is the name of the standard gauge optimization).\n",
    "\n",
    "If `gaugeopt_suite` is set to a string, this is the same as passing a dictionary with a single key-value pair where both key and value are equal to the string.  Thus, the default `\"stdgaugeopt\"` is equivalent to specifying the dictionary `{\"stdgaugeopt\": \"stdgagueopt\"}`.\n",
    "\n",
    "The example below performs a customized gauge-optimization where the gate parameters are weighted 1000 times more relative to the SPAM parameters.  Mathematically this corresponds to a multiplicative factor of 0.001 preceding the sum-of-squared-difference terms corresponding to SPAM elements in the model.   Typically it is good to weight the gates parameters more heavily since GST amplifies gate parameter errors via long operation sequences but cannot amplify SPAM parameter errors.  For more details on the arguments of `gaugeopt_to_target`, see the previous tutorial on low-level algorithms.  For more infomation, see the [gauge optimization tutorial](advanced/GaugeOpt.ipynb).\n",
    "\n",
    "The cell below also illustrates how you can create a TP target model by calling `set_all_parameterizations` explicitly instead of using the equivalent and more condensed `.target_model(\"TP\")`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_model_TP2 = smq1Q_XYI.target_model() # a \"fully parameterized\" (unconstrained) model\n",
    "target_model_TP2.set_all_parameterizations(\"full TP\") # change parameterization to TP gates\n",
    "\n",
    "proto = pygsti.protocols.GateSetTomography(\n",
    "    target_model_TP2, name=\"GSTwithMyGO\",\n",
    "    gaugeopt_suite={'my_gauge_opt': {'item_weights': {'gates': 1.0, 'spam': 0.001}}}\n",
    "    )\n",
    "results_TP2 = proto.run(data, disable_checkpointing=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(results_TP2.estimates['GSTwithMyGO'].goparameters.keys())  # names of all the gauge opts that were done\n",
    "custom_gauge_opt_model = results_TP2.estimates['GSTwithMyGO'].models['my_gauge_opt']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wildcard parameters\n",
    "\n",
    "TODO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proto = pygsti.protocols.GateSetTomography(\n",
    "    target_model_TP, name=\"GSTwithPerGateWildcard\",\n",
    "    badfit_options={'actions': ['wildcard']}\n",
    "    )\n",
    "\n",
    "# Artifically unset threshold so that wildcard runs. YOU WOULD NOT DO THIS IN PRODUCTION RUNS\n",
    "proto.badfit_options.threshold = None\n",
    "\n",
    "results_pergate_wildcard = proto.run(data, disable_checkpointing=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The wildcard can be retrieved by looking at unmodeled_error in the estimates\n",
    "results_pergate_wildcard.estimates['GSTwithPerGateWildcard'].parameters['unmodeled_error']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another common form of wildcard is to have one parameter for SPAM and one for all the other gates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "op_label_dict = {k:0 for k in target_model_TP.operations} # Assign all gates to value 0\n",
    "op_label_dict['SPAM'] = 1 # Assign SPAM to value 1\n",
    "\n",
    "proto = pygsti.protocols.GateSetTomography(\n",
    "    target_model_TP, name=\"GSTwithPerGateWildcard\",\n",
    "    badfit_options={'actions': ['wildcard'], 'wildcard_primitive_op_labels': op_label_dict}\n",
    "    )\n",
    "\n",
    "# Artifically unset threshold so that wildcard runs. YOU WOULD NOT DO THIS IN PRODUCTION RUNS\n",
    "proto.badfit_options.threshold = None\n",
    "\n",
    "results_globalgate_wildcard = proto.run(data, disable_checkpointing=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately both of these wildcard strategies have the same problem. They are not unique, i.e. it is possible to \"slosh\" wildcard strength from one parameter to another to get another valid wildcard solution. This makes it difficult to make any quantitative statements about relative wildcard strengths.\n",
    "\n",
    "In order to avoid this, we have also introduced a 1D wildcard solution. This takes some reference weighting for the model operations and scales a single wildcard parameter ($\\alpha$) up until the model fits the data. Since there is only one parameter, this does not have any of the ambiguity of the above wildcard strategies. Currently, the reference weighting used is the diamond distance from the noisy model to the target model, with the intuition that \"noisier\" operations are more likely to contribute to model violation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proto = pygsti.protocols.GateSetTomography(\n",
    "    target_model_TP, name=\"GSTwithPerGateWildcard\",\n",
    "    badfit_options={'actions': ['wildcard1d'], 'wildcard1d_reference': 'diamond distance'}\n",
    "    )\n",
    "\n",
    "# Artifically unset threshold so that wildcard runs. YOU WOULD NOT DO THIS IN PRODUCTION RUNS\n",
    "proto.badfit_options.threshold = None\n",
    "\n",
    "results_1d_wildcard = proto.run(data, disable_checkpointing=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### running GST using a custom set of circuits\n",
    "So far we've giving the `GateSetTomography.run` method an \"standard\" experiment design containing circuits chosen to amplify all of a standard TP (or CPTP) model's parameters (see the `StandardGSTExpermentDesign` used in the [DataSet tutorial](../objects/DataSet.ipynb)).  A `GateSetTomography` protocol can be run on more general experiment designs, namely those that specify the circuits to use as either a list of lists of `Circuit` objects or a list of or single `CircuitStructure` object(s).  A `CircuitStructure` is preferable as it allows the structured plotting of the sequences in report figures.  In this example, we'll just generate a standard set of circuit structures, but with some of the sequences randomly dropped (see the [tutorial on GST circuit reduction](advanced/GST-FiducialPairReduction.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Create the same sequences but drop 50% of them randomly for each repeated-germ block.\n",
    "# and only go out to a max-length of 8\n",
    "pspec = target_model_TP2.create_processor_spec() # ProcessorSpec based on the target model\n",
    "orig_design = data.edesign  # the original StandardGSTDesign\n",
    "custom_maxlengths = [1, 2, 4, 8]  # a subset of orig_design.maxlengths\n",
    "circuit_structs = pygsti.circuits.create_lsgst_circuit_lists(\n",
    "    target_model_TP2, orig_design.prep_fiducials, orig_design.meas_fiducials,\n",
    "    orig_design.germs, custom_maxlengths, keep_fraction=0.5, keep_seed=2020)\n",
    "reduced_exp_design = pygsti.protocols.GateSetTomographyDesign(pspec, circuit_structs)\n",
    "reduced_data = pygsti.protocols.ProtocolData(reduced_exp_design, data.dataset)\n",
    "\n",
    "\n",
    "proto = pygsti.protocols.GateSetTomography(target_model_TP2, name=\"GSTwithReducedData\")\n",
    "results_reduced = proto.run(reduced_data, disable_checkpointing=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"standardgst\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `StandardGST`\n",
    "The protocol embodies a standard *set* of GST protocols to be run on a set of data.  It essentially runs multiple `GateSetTomography` protocols on the given data which use different parameterizations of an `ExplicitOpModel`  (the `StandardGST` protocol doesn't work with other types of `Model` objects, e.g. *implicit* models, which don't implement `set_all_parameterizations`).  The `modes` argument is a list strings corresponding to the parameterization types that should be run (e.g. `[\"full TP\",\"CPTPLND\"]` will compute a Trace-Preserving estimate *and* a Completely-Positive & Trace-Preserving estimate). The currently available modes are:\n",
    " - \"full\" : unconstrained gates (fully parameterized)                                                                 \n",
    " - \"TP\"   : TP-constrained gates and state preparations\n",
    " - \"CPTP\" : CPTP-constrained gates and TP-constrained state preparations               \n",
    " - \"H+S\"  : Only Hamiltonian and Pauli stochastic errors allowed (CPTP)                                             \n",
    " - \"S\"    : Only Pauli-stochastic errors allowed (CPTP)                                                           \n",
    " - \"Target\" : use the target (ideal) gates as the estimate     \n",
    "\n",
    "Gauge optimization(s) are controlled by the `gaugeopt_suite` and `gaugeopt_target` arguments, jsut as in `GateSetTomography`.  The `gaugeopt_target` argument may be set to a `Model` that is used as the target for gauge optimization, overriding the (typically ideal) target gates given by within the experiment design."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_stdprac = pygsti.protocols.StandardGST().run(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Estimates: \", \", \".join(results_stdprac.estimates.keys()))\n",
    "print(\"TP Estimate's gauge optimized models: \", \", \".join(results_stdprac.estimates[\"full TP\"].goparameters.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll perform the same analysis but with a **non-default standard suite of gauge optimizations** - this one toggles the SPAM penalty in addition to varying the spam weight (the default suite just varies the spam weight without any SPAM penalty).  See the [gauge optimization tutorial](advanced/GaugeOpt.ipynb) for more details on gauge optmization \"suites\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proto = pygsti.protocols.StandardGST(gaugeopt_suite=\"varySpam\", name=\"StdGST_varySpam\")\n",
    "results_stdprac_nondefaultgo = proto.run(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Estimates: \", \", \".join(results_stdprac_nondefaultgo.estimates.keys()))\n",
    "print(\"TP Estimate's gauge optimized models: \", \", \".join(results_stdprac_nondefaultgo.estimates[\"full TP\"].goparameters.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we'll demonstrate how to specify a fully custom set of gauge optimization parameters and how to use a **separately-specified target model for gauge optimization**.  You can get a more intuitive gauge-optimized `Model` when by placing as much expected noise as possible into the gauge-optimization target, as this essentially tells the algorithm \"this is what I think the estimated model should look like\".  If you just use the perfect or ideal model for this (the default), then the gauge optimizer may make tradeoffs which don't reflect the expected physics (remember, all gauge-equivalent models product the same observables!).  For example, it may spread error across all your gate operations when you expect just the 2-qubit operations are noisy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_goparams = { 'item_weights': {'gates': 1.0, 'spam': 0.001} }\n",
    "my_gaugeOptTarget = smq1Q_XYI.target_model().depolarize(\n",
    "    op_noise=0.005, spam_noise=0.01) # a guess at what estimate should be\n",
    "my_gaugeopt_suite = pygsti.protocols.GSTGaugeOptSuite(gaugeopt_argument_dicts={'myGO': my_goparams},\n",
    "                                                      gaugeopt_target=my_gaugeOptTarget)\n",
    "\n",
    "proto = pygsti.protocols.StandardGST(gaugeopt_suite=my_gaugeopt_suite,\n",
    "                                     name=\"StdGST_myGO\")\n",
    "results_stdprac_nondefaultgo = proto.run(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Estimates: \", \", \".join(results_stdprac_nondefaultgo.estimates.keys()))\n",
    "print(\"TP Estimate's gauge optimized models: \", \", \".join(results_stdprac_nondefaultgo.estimates[\"full TP\"].goparameters.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To finish up, we'll write the results for processing in other tutorials.  We do this by calling `.write` on the results objects, optionally specifying the root diretory under which the results should be written.  This is the *same* root directory that the experiment design and data are written to, as subdirectories beneath this directory separate these quantities.\n",
    "\n",
    "Two remarks are in order:\n",
    "1. When results are from running a protocol on data that was loaded with the `load_data_from_dir` method (see the beginning of this notebook), then knowledge of this directory is remembered and you don't need to give a directory to `write` (this is the case for all except `results_reduced`, which created a new experiment design containing less experiments).\n",
    "\n",
    "2. Notice how the `name=` arguments given to protocols above are used as sub-directory names, e.g. under the \"tutorial_files/Example_GST_Data/results\" parent directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_TP.write()  # uses \"../tutorial_files/Example_GST_Data\" (where data was loaded from)\n",
    "results_TP2.write() # ditto\n",
    "results_stdprac.write() # ditto\n",
    "results_reduced.write(\"../tutorial_files/Example_Reduced_GST_Data\") # choose a different dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While it is also possible to **pickle** a results object, this method of serialization is **not recommended** for long-term storage since pickle files are relatively fragile to changes in pyGSTi or other python libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Not recommended:\n",
    "# import pickle\n",
    "# pickle.dump(results_TP, open('../tutorial_files/exampleResults_TP.pkl',\"wb\"))\n",
    "# pickle.dump(results_TP2, open('../tutorial_files/exampleResults_TP2.pkl',\"wb\"))\n",
    "# pickle.dump(results_reduced, open('../tutorial_files/exampleResults_reduced.pkl',\"wb\"))\n",
    "# pickle.dump(results_stdprac, open('../tutorial_files/exampleResults_stdprac.pkl',\"wb\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checkpointing/Warmstarting\n",
    "\n",
    "The `GateSetTomography` and `StandardGST` protocols both support checkpointing to enable resuming GST analysis after an unexpected failure, such as an out-of-memory error, or an unexpected timeout in resource limited compute environments (clusters etc.), or for whatever other reason. Checkpointing is enabled by default, so no additional changes are needed in order to have these generated. \n",
    "\n",
    "Each protocol has a corresponding checkpoint object, `GateSetTomographyCheckpoint` and `StandardGSTCheckpoint`, which are saved to disk over the course of an iterative fit in serialized json format. By default checkpoint files associated with a `GateSetTomographyCheckpoint` object are saved to a new directory located in whichever current working directory the protocol is being run from named 'gst_checkpoints'. A new file is written to disk after each iteration with default naming of the form `GateSetTomography_iteration_{i}.json` where i is the index of the completed GST iteration associated with that checkpoint. Similarly, for a `StandardGSTCheckpoint` object the checkpoints are by default saved to a directory named 'standard_gst_checkpoints' with default file names of the form `StandardGST_{mode}_iteration_{i}` where mode corresponds to the current parameterized fit or model test associated with that file (including checkpoint information for all previously completed modes prior to the currently running one) and i is the index of the completed iteration within that current mode.\n",
    "\n",
    "Below we repeat our first example of the notebook, but this time with checkpointing enabled (as is the default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pygsti.modelpacks import smq1Q_XYI\n",
    "target_model_TP = smq1Q_XYI.target_model(\"full TP\")\n",
    "proto = pygsti.protocols.GateSetTomography(target_model_TP)\n",
    "results_TP = proto.run(data, checkpoint_path = '../tutorial_files/gst_checkpoints/GateSetTomography')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in the example above we have specified a value for an additional kwarg called `checkpoint_path`. This allows for overriding the default behavior for the save location and naming of checkpoint files. The expected format is `{path}/{name}` where path is the directory to save the checkpoint files to (with that directory being created is required) and where name is the stem of the checkpoint file names `{name}_iteration_{i}.json`. Inspecting the contents of the directory we just specified, we can see that it is now populated by 8 new checkpoint files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.listdir('../tutorial_files/gst_checkpoints/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose hypothetically that a GST fit had failed at iteration 5 and we wanted to restart from that point without redoing all of the previous iterations from scratch again. We'll call this warmstarting. We can do so by reading in the appropriate serialized checkpoint object using the `read` class method of `GateSetTomographyCheckpoint` and passing that now loaded checkpoint object in for the `checkpoint` kwarg of `run`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pygsti.protocols import GateSetTomographyCheckpoint\n",
    "gst_iter_5_checkpoint = GateSetTomographyCheckpoint.read('../tutorial_files/gst_checkpoints/GateSetTomography_iteration_5.json')\n",
    "results_TP_from_iter_5= proto.run(data, checkpoint= gst_iter_5_checkpoint, checkpoint_path = '../tutorial_files/gst_checkpoints/GateSetTomography')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see from the output that we indeed started from iteration 6 (note the output log indexes from 1 instead of 0). Moreover we can see that we've indeed produced the same output as before without warmstarting, as we would expect/hope:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all(results_TP.estimates['GateSetTomography'].models['final iteration estimate'].to_vector() == \\\n",
    "results_TP_from_iter_5.estimates['GateSetTomography'].models['final iteration estimate'].to_vector())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The checkpoint object itself contains information that could be useful for diagnostics or debugging, including the current list of models associated each iterative fit, the last completed iteration it is associated with, and the list of circuits for the last completed iteration it is associated with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checkpointing with the StandardGST protocol works similarly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "proto_standard_gst = pygsti.protocols.StandardGST(modes=['full TP', 'CPTPLND', 'Target'], verbosity=3)\n",
    "results_stdprac = proto_standard_gst.run(data, checkpoint_path = '../tutorial_files/standard_gst_checkpoints/StandardGST')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Except this time we have significantly more files saved, as during the course of the StandardGST protocol we're actually running three subprotocols:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.listdir('../tutorial_files/standard_gst_checkpoints/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the StandardGST protocol runs the subprotocols in the order listed in the `modes` argument, and checkpoint objects labeled with a given model label additionally contain the checkpointing information for the final iterations of any preceding modes which have been completed. i.e. the CPTPLND checkpoint objects contain the information required for full TP. Likewise, checkpoints for Target contain the information required for the full TP and CPTPLND modes. As before, imagine that our fitting failed for whatever reason during iteration 5 of CPTPLND, we can warmstart the protocol by loading in the checkpoint object associated with iteration 4 as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from pygsti.protocols import StandardGSTCheckpoint\n",
    "standard_gst_checkpoint = StandardGSTCheckpoint.read('../tutorial_files/standard_gst_checkpoints/StandardGST_CPTPLND_iteration_4.json')\n",
    "results_stdprac_warmstart= proto_standard_gst.run(data, checkpoint= standard_gst_checkpoint, checkpoint_path = '../tutorial_files/standard_gst_checkpoints/StandardGST')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that we've indeed skipped past the previously completed full TP mode and jumped straight to the 6th iteration of the CPTPLND fit as expected. \n",
    "\n",
    "As for the GateSetTomographyCheckpoint object described above, the `StandardGSTCheckpoint` can often be useful to inspect as a debugging/diagnostic tool. `StandardGSTCheckpoints` are essentially structured as container object that hold a set of child `GateSetTomographyCheckpoint` and `ModelTestCheckpoint` (more on these in the ModelTest tutorial) objects for each of the modes being run (and potentially more types of chile checkpoints in the future as we add additional functionality). These children can be accessed using the `children` attribute of a `StandardGSTCheckpoint` instance which is a dictionary with keys given by the mode names contained therein."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(standard_gst_checkpoint.children['CPTPLND'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pygsti",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}