{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true, "deletable": true, "editable": true }, "source": [ "# Results Object Tutorial\n", "This tutorial explains the structure and usage of the `pygsti.report.Results` objects which are returned from the high-level driver functions (see prior tutorial).\n", "\n", "## `pygsti.report.Results`\n", "A `Results` object is used to store estimated `GateSet`s for a *single `DataSet`*, along with the associated input parameters which led to each estimate. As a concrete example, we'll explore one of the `Results` objects generated by the high-level algorithms tutorial." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "from __future__ import print_function\n", "import pygsti\n", "import pickle" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----------------------------------------------------------\n", "---------------- pyGSTi Results Object -------------------\n", "----------------------------------------------------------\n", "\n", "How to access my contents:\n", "\n", " .dataset -- the DataSet used to generate these results\n", "\n", " .gatestring_lists -- a dict of GateString lists w/keys:\n", " ---------------------------------------------------------\n", " iteration\n", " final\n", " all\n", " iteration delta\n", " prep fiducials\n", " effect fiducials\n", " germs\n", "\n", " .gatestring_structs -- a dict of GatestringStructures w/keys:\n", " ---------------------------------------------------------\n", " iteration\n", " final\n", "\n", " .estimates -- a dictionary of Estimate objects:\n", " ---------------------------------------------------------\n", " default\n", "\n", "\n" ] } ], "source": [ "results = pickle.load(open('tutorial_files/exampleResults.pkl','rb'))\n", "print(results)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "As you can see, printing a `Results` object gives you a summary of its structure and what you can do with it. The *single* `DataSet` can be accessed via the `.dataset` member, and the estimated `GateSet` objects can be found within the `pygsti.report.Estimate` objects contained within the `.estimates` member. As the summary states, `.estimates` is a *dictionary* of `Estimate` objects, and **can contain as multiple estimates of the data *with the caveat* that all of these estimates must use the same gate sequences**, that is, the same `GateString` lists and/or `GateStringStructure`s for each algorithm iteration, and the same number of iterations. The reasoning behind this limitation is that when you alter the gate sequences which are used, this alters which data is used, which is similar to altering the data set itself - and we've already drawn the line that each `Results` object holds information for just a single data set.\n", "\n", "## `pygsti.report.Estimate`\n", "The `Estimate` objects represent different *gauge-unfixed* or *\"up-to-gauge\"* estimates, and each holds one or more `GateSet` and associated `ConfidenceRegion` objects, and dictionaries containing the parameters used to generate the estimate. They also may contain **multiple gauge-optimized \"versions\" of their single gauge-unfixed estimate**. They can be printed to display a summary of their contents:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "odict_keys(['_gaugeGroupEl', 'gateset', 'itemWeights', 'returnAll', 'targetGateset'])\n" ] } ], "source": [ "print(results.estimates['default'].goparameters['go0'].keys())" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "`Estimate` objects do *not* store the gate sequences - rather, since these must be the same for all the estimates of a `Results` object, the `Results` object holds them separately in its `.gatestring_lists` and `.gatestring_structs` members (both of which are dictionaries like `.estimates`). Furthermore, since varying the gauge optimization parameters is such a common variation, a single `Estimate` may hold *multiple* dictionaries of gauge-optimization parameters as the elements of its `.goparameters` dictionary. **The keys of `goparameters` will and must correspond to keys within the `.gatesets` member** (the `GateSet` estimate of that gauge optimization). The `.confidence_regions` dictionary (empty in the above example and so not included in the summary) holds `ConfidenceRegion` objects, each associated with 1) one the `GateSet`s in `.gatesets`, 2) one of the `GateString` lists in the parent `Results` object's `.gatestring_lists`, and 3) a confidence level. The keys of `.confidence_regions` are complicated because they must include all three of these associations, and so the `.get_confidence_region(...)` method is preferred to directly accessing `.confidence_regions`. Different `Estimate` objects typically hold estimates for different gate set parameterizations and/or algorithm parameters.\n", "\n", "In our example, `results` contains only a single `Estimate` called `default` (the default estimate label created by `do_long_sequence_gst`). This `Estimate` contains the raw un-gauge-optimized `GateSet` labeled `\"final iteration estimate\"`, as well as a single gauge-optimized `GateSet` labeled `go0` (again, the default created by `do_long_sequence_gst`).\n", "\n", "## Rolling your own...\n", "\n", "Creating your own `Results` object is as simple as\n", "1. calling `pygsti.report.Results()` to create an empty object\n", "2. initializing the data set and gate sequences via calls to `init_dataset` and `init_gatestrings`\n", "3. creating contained `Estimate` objects by calling `add_estimate` with the essential components of a new gauge-unfixed estimate (the target gate set, the starting gate set, the estimated gate sets by iteration, and the parameter dictionary).\n", "\n", "This is demonstrated with dummy parameters below." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----------------------------------------------------------\n", "---------------- pyGSTi Results Object -------------------\n", "----------------------------------------------------------\n", "\n", "How to access my contents:\n", "\n", " .dataset -- the DataSet used to generate these results\n", "\n", " .gatestring_lists -- a dict of GateString lists w/keys:\n", " ---------------------------------------------------------\n", " iteration\n", " final\n", " all\n", " iteration delta\n", " prep fiducials\n", " effect fiducials\n", " germs\n", "\n", " .gatestring_structs -- a dict of GatestringStructures w/keys:\n", " ---------------------------------------------------------\n", " iteration\n", " final\n", "\n", " .estimates -- a dictionary of Estimate objects:\n", " ---------------------------------------------------------\n", " myTestEstimate\n", "\n", "\n" ] } ], "source": [ "my_results = pygsti.objects.Results()\n", "my_results.init_dataset( results.dataset ) #use the same data and strings as our results\n", "my_results.init_gatestrings( results.gatestring_structs['iteration'] )\n", "\n", "gs_target = results.estimates['default'].gatesets['target']\n", "gs_initial = gs_target.copy()\n", "my_estimate = gs_target.depolarize(gate_noise=0.01, spam_noise=0.1)\n", "my_estimate_by_iter = [my_estimate]*len(my_results.gatestring_structs['iteration'])\n", "my_parameters = {'someParam': 1.0 }\n", "my_results.add_estimate(gs_target, gs_initial, my_estimate_by_iter, my_parameters, estimate_key=\"myTestEstimate\")\n", "\n", "print(my_results)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Creating a new gauge-optimized estimate\n", "In many circumstances, one may want to perform a new gauge optimization on an existing gauge-unfixed `Estimate`, `est`, creating a new gauge-optimized `GateSet` to be stored in `est`. This is accomplished using the `est.add_gaugeoptimized` which lightly wraps a call to `pygsti.gaugeopt_to_target`. You specify the arguments to `gaugeopt_to_target` as a dictionary to `add_gaugeoptimized`, but you're allowed to leave out the first two: the `GateSet` to be optimized, (`gateset`, taken to be the `est.gatesets['final iteration estimate']`) and the gate set to optimize toward (`targetGateset`, taken to be the `est.gatesets['target']`). Note that these arguments *can* still be specified to override their defaults. In particular, **setting `targetGateset` in the dictionary of parameters allows one to independently specify the gate set to optimize toward** (*and this need not be a perfect, ideal gate set!*).\n", "\n", "The optional `label` argument of `add_gaugeoptimized` specifies the key within `est.goparameters` and `est.gatesets` where the gauge optimization argument dictionary and resulting gauge-optimized `GateSet` will be stored. If the label given already exists, that gauge-optimized estimate is replaced with the new one. If `label` is left as `None`, then \"go*X*\" is used as the label, where *X* is the next available integer.\n", "\n", "If the `gateset` argument of `add_gaugeoptimized` is supplied, then this is taken to be the result of the described gauge optimization and no call to `gaugeopt_to_target` is made. (In this case, one could simply pass an empty dictionary of as `goparams`.)\n", "\n", "Below we demonstrate how to add gauge-optimized gate sets to an `Estimate` in several ways. Please refer to the previous tutorial on low-level algorithms for an explanation of the various arguments to `gaugeopt_to_target`." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----------------------------------------------------------\n", "---------------- pyGSTi Estimate Object ------------------\n", "----------------------------------------------------------\n", "\n", "How to access my contents:\n", "\n", " .gatesets -- a dictionary of GateSet objects w/keys:\n", " ---------------------------------------------------------\n", " target\n", " seed\n", " iteration estimates\n", " final iteration estimate\n", " go0\n", " equal_footing\n", " Gx_heavy\n", " imperfect gopt target\n", "\n", " .parameters -- a dictionary of simulation parameters:\n", " ---------------------------------------------------------\n", " objective\n", " memLimit\n", " starting point\n", " profiler\n", " minProbClip\n", " minProbClipForWeighting\n", " probClipInterval\n", " radius\n", " weights\n", " cptpPenaltyFactor\n", " spamPenaltyFactor\n", " distributeMethod\n", " depolarizeStart\n", " contractStartToCPTP\n", " tolerance\n", " maxIterations\n", " useFreqWeightedChiSq\n", " nestedGateStringLists\n", " profile\n", " check\n", " truncScheme\n", " gateLabelAliases\n", " includeLGST\n", " max length list\n", "\n", " .goparameters -- a dictionary of gauge-optimization parameter dictionaries:\n", " ---------------------------------------------------------\n", " go0\n", " equal_footing\n", " Gx_heavy\n", " imperfect gopt target\n", "\n", "\n" ] } ], "source": [ "est = results.estimates['default']\n", "est.add_gaugeoptimized({'itemWeights': {'gates': 1.0, 'spam': 1.0}}, label=\"equal_footing\")\n", "est.add_gaugeoptimized({'itemWeights': {'gates': 1.0, 'spam': 1.0, 'Gx': 10.0}}, label=\"Gx_heavy\")\n", "\n", "gs_guess = est.gatesets['target'].depolarize(gate_noise=0.05, spam_noise=0.02) # a guess at what gates should be...\n", "est.add_gaugeoptimized({'targetGateset': gs_guess, 'itemWeights': {'spam': 0.01}}, label=\"imperfect gopt target\")\n", "\n", "print(est)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Confidence Region Factories\n", "In order to compute confidence regions and intervals within reports (see later tutorials), an `Estimate` object must be equipped with one or more \"confidence region factory\" objects. These factories are instances of `pygsti.objects.ConfidenceRegionFactory` (suprise, suprise). Their purpose is to generate confidence regions and intervals (for *any* confidence level) for quantities computed from a particular `GateSet` that in turn resulted from optimizaing the likelihood function corresponding to a particular set of gate sequences. Thus, a confidence region factory has associated with it three things: 1) a `GateSet`, 2) a list of `GateString`s, and 3) a `DataSet`. A dictionary of factories is held as the `.confidence_region_factories` member of an `Estimate` object. Each factory within this dictionary is associated with the one-and-only `DataSet` of the `Estimate`'s parent `Results` object, and the associated `GateSet` and `GateString` list are given by the keys of `.confidence_region_factories` (*gateset-key*, *gatestring-list-key* tuples). Here *gateset-key* is the key of a `GateSet` within the `Estimate`'s `.gatesets` member and *gatestring-list key* is the key of a list within the parent `Results` object's `.gatestring_lists` member.\n", "\n", "Thankfully, you won't usually need to deal with the `.confidence_region_factories` member directly. To create a new factory for a given `GateSet`, `GateString`-list pair you can simply call the `add_confidence_region_factory` with the appropriate key labels. Once a factory is created, it must be initialized for computing confidence regions. The only non-experimental way to do this currently is to compute the Hessian of the log-likelihood (often computationally intensive) and then projecting the inverse of this Hessian onto the non-gauge space of the gate set. These two steps are performed via the `compute_hessian` and `project_hessian` member functions of a `ConfidenceRegionFactory` object. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "gateset_label = \"go0\"\n", "gslist_label = \"final\"\n", "crfactory = results.estimates['default'].add_confidence_region_factory(gateset_label, gslist_label)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Note that there are different ways of projecting the Hessian which have different strengths and weakenesses. The \"optimal gate CIs\" method is the most robust method for giving the smallest error bars possible, but it takes significant computation time. The \"intrinsic error\" method is fast and usually reliable, but may not always give the smallest possible error bars." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " \n", "--- Hessian Projector Optimization from separate SPAM and Gate weighting ---\n", " Resulting intrinsic errors: 0.00381267 (gates), 0.00134802 (spam)\n", " Resulting sqrt(mean(gateCIs**2)): 0.00454592\n", " Resulting sqrt(mean(spamCIs**2)): 0.00303855\n" ] } ], "source": [ "crfactory.compute_hessian(comm=None) #could use lots of processor here...\n", "inv_proj_H = crfactory.project_hessian('intrinsic error')" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "**Alternate way**: In the special case of constructing factories for `GateSet`s which are gauge-equivalent to one another, one can skip the `compute_hessian` step for all but the first `GateSet`, so long as the gauge optimization parameters *and* the final gauge-tranformation element are stored in the `Estimate`s `.goparameters` dictionary (automatically populated when adding a gauge optimization via `add_gaugeoptimized`). Instead, one must *gauge-propagate* the Hessian from the first `GateSet` to the others using the `gauge_propagate_confidence_region_factory` method of the `Estimate` object.\n", "\n", "Below, we show how this might usually be done: first a confidence region factory for the \"final iteration estimate\" `GateSet` and 'final' gate string list (the defaults) is created and a Hessian is computed. Then, when a factory is needed for the gauge-equivalent `GateSet` \"go0\", the Hessian is propagated from the \"final iteration estimate\" `GateSet`. Note that the propagated Hessian must still be projected for the \"go0\" gate set." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " *** Propagating Hessian from 'final iteration estimate' to 'go0' ***\n", "Column: [##################################################] 100.0% \n", " Successfully transported Hessian and ConfidenceRegionFactory.\n", " \n", "--- Hessian Projector Optimization from separate SPAM and Gate weighting ---\n", " Resulting intrinsic errors: 0.00381267 (gates), 0.00134802 (spam)\n", " Resulting sqrt(mean(gateCIs**2)): 0.00454592\n", " Resulting sqrt(mean(spamCIs**2)): 0.00303855\n" ] } ], "source": [ "crfact_final = results.estimates['default'].add_confidence_region_factory() #default 'final iteration estimate'\n", "crfact_final.compute_hessian(comm=None)\n", "\n", "results.estimates['default'].gauge_propagate_confidence_region_factory('go0', verbosity=1) #instead of computing one\n", "crfact_go0 = results.estimates['default'].get_confidence_region_factory('go0')\n", "inv_proj_H = crfact_go0.project_hessian('intrinsic error')" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Summary\n", "\n", "In summary, when thinking about `Results` and `Estimate` objects, remember:\n", "- each **`Results`** object represents the **results for a single set of data** (or \"effective\" set of data as defined by the sequences used).\n", "- each contained **`Estimate`** object represents a **single *gauge-unfixed* estimate** based on the data. An `Estimate` may also contain **one or more *gauge-optimized* versions** of the gauge-invariant estimate. \n", "- an `Estimate` can construct confidence intervals only after a **`ConfidenceRegionFactory`** object is created and initialized using a multi-step process. Because it may be computationally expensive, these steps are *not* performed automatically when reports are generated.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 2 }