{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Report Generation Tutorial\n", "\n", "PyGSTi is able to construct polished report documents, which provide high-level summaries as well as detailed analyses of results (Gate Set Tomography (GST) and model-testing results in particular). Reports are intended to be quick and easy way of analyzing `Model`-type estimates, and pyGSTi's report generation functions are specifically designed to interact with the `Results` object (producted by several high-level algorithm functions - see, for example, the [GST overview tutorial](../algorithms/GST-Overview.ipynb) and [GST functions tutorial](../algorithms/GST-Drivers.ipynb).). The report generation functions in pyGSTi takes one or more results (often `Results`-type) objects as input and produces an HTML file as output. The HTML format allows the reports to include **interactive plots** and **switches** (see the [workspace switchboard tutorial](advanced/WorkspaceSwitchboards.ipynb), making it easy to compare different types of analysis or data sets. \n", "\n", "PyGSTi's reports are stand-alone HTML documents which cannot run Python. Thus, all the results displayed in a report must be pre-computed (in Python). If you find yourself wanting to fiddle with things and feel that these reports are too static, please consider using a `Workspace` object (see the [Workspace tutorial](Workspace.ipynb)) within a Jupyter notebook, where you can intermix report tables/plots and Python. Internally, functions like `create_standard_report` (see below) are just canned routines which use a `WorkSpace` object to generate various tables and plots and then insert them into a HTML template. \n", "\n", "**Note to veteran users:** PyGSTi has for some time now transitioned to producing HTML (rather than LaTeX/PDF) reports. The way to generate such reports is largely unchanged, with one important exception. Previously, the `Results` object had various report-generation methods included within it. We've found this is too restrictive, as we'd sometimes like to generate a report which utilizes the results from multiple runs of GST (to compare them, for instance). Thus, the `Results` class is now just a container for a `DataSet` and its related `Model`s, `CircuitStructure`s, etc. All of the report-generation capability is now housed in within separate report functions, which we now demonstrate.\n", "\n", "\n", "### Get some `Results`\n", "We start by performing GST using `do_long_sequence_gst`, as usual, to create a `Results` object (we could also have just loaded one from file). See the [GST functions tutorial](../algorithms/GST-Drivers.ipynb) for more details." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loading from cache file: ../tutorial_files/Example_Dataset.txt.cache\n", "--- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", "--- LGST ---\n", " Singular values of I_tilde (truncating to first 4 of 6) = \n", " 4.243730350963286\n", " 1.1796261581655645\n", " 0.9627515645786063\n", " 0.9424890722054706\n", " 0.033826151547621315\n", " 0.01692336936843073\n", " \n", " Singular values of target I_tilde (truncating to first 4 of 6) = \n", " 4.242640687119286\n", " 1.414213562373096\n", " 1.414213562373096\n", " 1.4142135623730954\n", " 2.484037189058858e-16\n", " 1.506337939585075e-16\n", " \n", " Resulting model:\n", " \n", " rho0 = TPSPAMVec with dimension 4\n", " 0.71-0.02 0.03 0.75\n", " \n", " \n", " Mdefault = TPPOVM with effect vectors:\n", " 0: FullSPAMVec with dimension 4\n", " 0.73 0 0 0.65\n", " \n", " 1: ComplementSPAMVec with dimension 4\n", " 0.69 0 0-0.65\n", " \n", " \n", " \n", " Gi = \n", " TPDenseOp with shape (4, 4)\n", " 1.00 0 0 0\n", " 0.01 0.92-0.03 0.02\n", " 0.01-0.01 0.90 0.02\n", " -0.01 0 0 0.91\n", " \n", " \n", " Gx = \n", " TPDenseOp with shape (4, 4)\n", " 1.00 0 0 0\n", " 0 0.91-0.01 0\n", " -0.02-0.02-0.04-0.99\n", " -0.05 0.03 0.81 0\n", " \n", " \n", " Gy = \n", " TPDenseOp with shape (4, 4)\n", " 1.00 0 0 0\n", " 0.05 0 0 0.98\n", " 0.01 0 0.89-0.03\n", " -0.06-0.82 0 0\n", " \n", " \n", " \n", " \n", "--- Iterative MLGST: Iter 1 of 5 92 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (92 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 87.7156, mu=0, |J|=1139.8\n", " --- Outer Iter 1: norm_f = 72.5388, mu=331.735, |J|=4004.6\n", " --- Outer Iter 2: norm_f = 49.7369, mu=110.578, |J|=4000.74\n", " --- Outer Iter 3: norm_f = 49.7313, mu=36.8595, |J|=4000.76\n", " --- Outer Iter 4: norm_f = 49.7312, mu=12.2865, |J|=4000.76\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 49.7312 (92 data params - 31 model params = expected mean of 61; p-value = 0.848365)\n", " Completed in 0.2s\n", " 2*Delta(log(L)) = 49.9289\n", " Iteration 1 took 0.2s\n", " \n", "--- Iterative MLGST: Iter 2 of 5 168 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (168 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 151.528, mu=0, |J|=4116.86\n", " --- Outer Iter 1: norm_f = 116.544, mu=1805.29, |J|=4114.08\n", " --- Outer Iter 2: norm_f = 111.925, mu=601.763, |J|=4113.57\n", " --- Outer Iter 3: norm_f = 111.481, mu=200.588, |J|=4113.47\n", " --- Outer Iter 4: norm_f = 111.47, mu=66.8625, |J|=4113.46\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 111.47 (168 data params - 31 model params = expected mean of 137; p-value = 0.94621)\n", " Completed in 0.2s\n", " 2*Delta(log(L)) = 111.83\n", " Iteration 2 took 0.2s\n", " \n", "--- Iterative MLGST: Iter 3 of 5 450 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (450 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 496.301, mu=0, |J|=4503.89\n", " --- Outer Iter 1: norm_f = 422.607, mu=2013.07, |J|=4503.59\n", " --- Outer Iter 2: norm_f = 421.667, mu=671.023, |J|=4503.46\n", " --- Outer Iter 3: norm_f = 421.662, mu=223.674, |J|=4503.46\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 421.662 (450 data params - 31 model params = expected mean of 419; p-value = 0.454312)\n", " Completed in 0.4s\n", " 2*Delta(log(L)) = 422.134\n", " Iteration 3 took 0.5s\n", " \n", "--- Iterative MLGST: Iter 4 of 5 862 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (862 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 854.092, mu=0, |J|=5091.17\n", " --- Outer Iter 1: norm_f = 813.479, mu=2302.96, |J|=5076.94\n", " --- Outer Iter 2: norm_f = 813.094, mu=767.654, |J|=5076.79\n", " --- Outer Iter 3: norm_f = 813.093, mu=255.885, |J|=5076.81\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 813.093 (862 data params - 31 model params = expected mean of 831; p-value = 0.664967)\n", " Completed in 0.7s\n", " 2*Delta(log(L)) = 814.492\n", " Iteration 4 took 0.8s\n", " \n", "--- Iterative MLGST: Iter 5 of 5 1282 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (1282 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 1263.73, mu=0, |J|=5733.35\n", " --- Outer Iter 1: norm_f = 1250.68, mu=2582.97, |J|=5732.82\n", " --- Outer Iter 2: norm_f = 1250.62, mu=860.99, |J|=5733.21\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 1250.62 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.497713)\n", " Completed in 1.0s\n", " 2*Delta(log(L)) = 1252.41\n", " Iteration 5 took 1.1s\n", " \n", " Switching to ML objective (last iteration)\n", " --- MLGST ---\n", " --- Outer Iter 0: norm_f = 626.205, mu=0, |J|=3678.46\n", " --- Outer Iter 1: norm_f = 626.2, mu=3871.69, |J|=3391.16\n", " --- Outer Iter 2: norm_f = 626.195, mu=4.22891e+07, |J|=3314.59\n", " --- Outer Iter 3: norm_f = 626.186, mu=2.22294e+07, |J|=3102.02\n", " --- Outer Iter 4: norm_f = 626.178, mu=2.05642e+07, |J|=3014.46\n", " --- Outer Iter 5: norm_f = 626.177, mu=2.02336e+07, |J|=3121.67\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Maximum log(L) = 626.177 below upper bound of -2.13594e+06\n", " 2*Delta(log(L)) = 1252.35 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.483889)\n", " Completed in 1.1s\n", " 2*Delta(log(L)) = 1252.35\n", " Final MLGST took 1.1s\n", " \n", "Iterative MLGST Total Time: 4.1s\n", " -- Adding Gauge Optimized (go0) --\n" ] } ], "source": [ "import pygsti\n", "from pygsti.construction import std1Q_XYI\n", "\n", "target_model = std1Q_XYI.target_model()\n", "fiducials = std1Q_XYI.fiducials\n", "germs = std1Q_XYI.germs\n", "maxLengths = [1,2,4,8,16]\n", "ds = pygsti.io.load_dataset(\"../tutorial_files/Example_Dataset.txt\", cache=True)\n", "\n", "#Run GST\n", "target_model.set_all_parameterizations(\"TP\") #TP-constrained\n", "results = pygsti.do_long_sequence_gst(ds, target_model, fiducials, fiducials, germs,\n", " maxLengths, verbosity=3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Make a report\n", "Now that we have `results`, we use the `create_standard_report` method within `pygsti.report` to generate a report. \n", "`pygsti.report.create_standard_report` is the most commonly used report generation function in pyGSTi, as it is appropriate for smaller models (1- and 2-qubit) which have *operations that are or can be represeted as dense matrices and/or vectors*. \n", "\n", "If the given filename ends in \"`.pdf`\" then a PDF-format report is generated; otherwise the file name specifies a folder that will be filled with HTML pages. To open a HTML-format report, you open the `main.html` file directly inside the report's folder. Setting `auto_open=True` makes the finished report open in your web browser automatically. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", "*** Generating plots ***\n", "*** Merging into template file ***\n", "Output written to ../tutorial_files/exampleReport directory\n", "Opening ../tutorial_files/exampleReport/main.html...\n", "*** Report Generation Complete! Total time 25.961s ***\n", "\n", "\n", "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", "*** Generating plots ***\n", "*** Merging into template file ***\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/usr/local/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:424: MatplotlibDeprecationWarning:\n", "\n", "\n", "Passing one of 'on', 'true', 'off', 'false' as a boolean is deprecated; use an actual boolean (True/False) instead.\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Latex file(s) successfully generated. Attempting to compile with pdflatex...\n", "Opening ../tutorial_files/exampleReport.pdf...\n", "*** Report Generation Complete! Total time 77.0165s ***\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "ERROR: pdflatex returned code 1 Check exampleReport.log to see details.\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#HTML\n", "pygsti.report.create_standard_report(results, \"../tutorial_files/exampleReport\", \n", " title=\"GST Example Report\", verbosity=1, auto_open=True)\n", "\n", "print(\"\\n\")\n", "\n", "#PDF\n", "pygsti.report.create_standard_report(results, \"../tutorial_files/exampleReport.pdf\", \n", " title=\"GST Example Report\", verbosity=1, auto_open=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are several remarks about these reports worth noting:\n", "1. The **HTML reports are the primary report type in pyGSTi**, and are much more flexible. The PDF reports are more limited (they can only display a *single* estimate and gauge optimization), and essentially contain a subset of the information and descriptive text of a HTML report. So, if you can, use the HTML reports. The PDF report's strength is its portability: PDFs are easily displayed by many devices, and they embed all that they need neatly into a single file. **If you need to generate a PDF report** from `Results` objects that have multiple estimates and/or gauge optimizations, consider using the `Results` object's `view` method to single out the estimate and gauge optimization you're after.\n", "2. It's best to use **Firefox** when opening the HTML reports. (If there's a problem with your brower's capabilities it will be shown on the screen when you try to load the report.)\n", "3. You'll need **`pdflatex`** on your system to compile PDF reports.\n", "4. To familiarize yourself with the layout of an HTML report, click on the gray **\"Help\" link** on the black sidebar." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Multiple estimates in a single report\n", "Next, let's analyze the same data two different ways: with and without the TP-constraint (i.e. whether the gates *must* be trace-preserving) and furthermore gauge optmimize each case using several different SPAM-weights. In each case we'll call `do_long_sequence_gst` with `gaugeOptParams=False`, so that no gauge optimization is done, and then perform several gauge optimizations separately and add these to the `Results` object via its `add_gaugeoptimized` function." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- Circuit Creation ---\n", "--- LGST ---\n", "--- Iterative MLGST: [##################################################] 100.0% 1282 operation sequences ---\n", "Iterative MLGST Total Time: 3.8s\n" ] } ], "source": [ "#Case1: TP-constrained GST\n", "tpTarget = target_model.copy()\n", "tpTarget.set_all_parameterizations(\"TP\")\n", "results_tp = pygsti.do_long_sequence_gst(ds, tpTarget, fiducials, fiducials, germs,\n", " maxLengths, gaugeOptParams=False, verbosity=1)\n", "\n", "#Gauge optimize\n", "est = results_tp.estimates['default']\n", "mdlFinal = est.models['final iteration estimate']\n", "mdlTarget = est.models['target']\n", "for spamWt in [1e-4,1e-2,1.0]:\n", " mdl = pygsti.gaugeopt_to_target(mdlFinal,mdlTarget,{'gates':1, 'spam':spamWt})\n", " est.add_gaugeoptimized({'itemWeights': {'gates':1, 'spam':spamWt}}, mdl, \"Spam %g\" % spamWt)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- Circuit Creation ---\n", "--- LGST ---\n", "--- Iterative MLGST: [##################################################] 100.0% 1282 operation sequences ---\n", "Iterative MLGST Total Time: 3.9s\n" ] } ], "source": [ "#Case2: \"Full\" GST\n", "fullTarget = target_model.copy()\n", "fullTarget.set_all_parameterizations(\"full\")\n", "results_full = pygsti.do_long_sequence_gst(ds, fullTarget, fiducials, fiducials, germs,\n", " maxLengths, gaugeOptParams=False, verbosity=1)\n", "\n", "#Gauge optimize\n", "est = results_full.estimates['default']\n", "mdlFinal = est.models['final iteration estimate']\n", "mdlTarget = est.models['target']\n", "for spamWt in [1e-4,1e-2,1.0]:\n", " mdl = pygsti.gaugeopt_to_target(mdlFinal,mdlTarget,{'gates':1, 'spam':spamWt})\n", " est.add_gaugeoptimized({'itemWeights': {'gates':1, 'spam':spamWt}}, mdl, \"Spam %g\" % spamWt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll now call the *same* `create_standard_report` function but this time instead of passing a single `Results` object as the first argument we'll pass a *dictionary* of them. This will result in a **HTML report that includes switches** to select which case (\"TP\" or \"Full\") as well as which gauge optimization to display output quantities for. PDF reports cannot support this interactivity, and so **if you try to generate a PDF report you'll get an error**." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", " targetSpamBriefTable took 0.201891 seconds\n", " targetGatesBoxTable took 0.268738 seconds\n", " datasetOverviewTable took 0.599083 seconds\n", " bestGatesetSpamParametersTable took 0.001285 seconds\n", " bestGatesetSpamBriefTable took 2.048558 seconds\n", " bestGatesetSpamVsTargetTable took 0.33649 seconds\n", " bestGatesetGaugeOptParamsTable took 0.000436 seconds\n", " bestGatesetGatesBoxTable took 1.570876 seconds\n", " bestGatesetChoiEvalTable took 3.876884 seconds\n", " bestGatesetDecompTable took 1.6544 seconds\n", " bestGatesetEvalTable took 0.00531 seconds\n", " bestGermsEvalTable took 0.040365 seconds\n", " bestGatesetVsTargetTable took 0.187189 seconds\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/Users/enielse/research/pyGSTi/packages/pygsti/extras/rb/theory.py:204: UserWarning:\n", "\n", "Output may be unreliable because the model is not approximately trace-preserving.\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " bestGatesVsTargetTable_gv took 0.925095 seconds\n", " bestGatesVsTargetTable_gvgerms took 0.317397 seconds\n", " bestGatesVsTargetTable_gi took 0.014471 seconds\n", " bestGatesVsTargetTable_gigerms took 0.054638 seconds\n", " bestGatesVsTargetTable_sum took 0.818716 seconds\n", " bestGatesetErrGenBoxTable took 7.480171 seconds\n", " metadataTable took 0.001853 seconds\n", " stdoutBlock took 0.000206 seconds\n", " profilerTable took 0.000871 seconds\n", " softwareEnvTable took 0.000413 seconds\n", " exampleTable took 0.086893 seconds\n", " singleMetricTable_gv took 0.838561 seconds\n", " singleMetricTable_gi took 0.04354 seconds\n", " fiducialListTable took 0.000611 seconds\n", " prepStrListTable took 0.000239 seconds\n", " effectStrListTable took 0.0003 seconds\n", " colorBoxPlotKeyPlot took 0.095837 seconds\n", " germList2ColTable took 0.000381 seconds\n", " progressTable took 2.919536 seconds\n", "*** Generating plots ***\n", " gramBarPlot took 0.217321 seconds\n", " progressBarPlot took 1.468046 seconds\n", " progressBarPlot_sum took 0.000597 seconds\n", " finalFitComparePlot took 0.711186 seconds\n", " bestEstimateColorBoxPlot took 12.230943 seconds\n", " bestEstimateTVDColorBoxPlot took 4.842788 seconds\n", " bestEstimateColorScatterPlot took 4.917573 seconds\n", " bestEstimateColorHistogram took 1.924395 seconds\n", " progressTable_scl took 7.8e-05 seconds\n", " progressBarPlot_scl took 7e-05 seconds\n", " bestEstimateColorBoxPlot_scl took 0.000162 seconds\n", " bestEstimateColorScatterPlot_scl took 0.000156 seconds\n", " bestEstimateColorHistogram_scl took 0.000157 seconds\n", " progressTable_ume took 7.1e-05 seconds\n", " progressBarPlot_ume took 6.9e-05 seconds\n", " bestEstimateColorBoxPlot_ume took 0.000194 seconds\n", " bestEstimateColorScatterPlot_ume took 0.000191 seconds\n", " bestEstimateColorHistogram_ume took 0.000188 seconds\n", " dataScalingColorBoxPlot took 6.6e-05 seconds\n", " unmodeledErrorBudgetTable took 0.000104 seconds\n", "Statistical hypothesis tests did NOT find inconsistency between the datasets at 5.00% significance.\n", "Statistical hypothesis tests did NOT find inconsistency between the datasets at 5.00% significance.\n", "Statistical hypothesis tests did NOT find inconsistency between the datasets at 5.00% significance.\n", "Statistical hypothesis tests did NOT find inconsistency between the datasets at 5.00% significance.\n", " dsComparisonSummary took 0.22732 seconds\n", " dsComparisonHistogram took 0.848321 seconds\n", " dsComparisonBoxPlot took 1.279172 seconds\n", "*** Merging into template file ***\n", " Rendering topSwitchboard took 0.000104 seconds\n", " Rendering maxLSwitchboard1 took 7.7e-05 seconds\n", " Rendering targetSpamBriefTable took 0.344551 seconds\n", " Rendering targetGatesBoxTable took 0.325133 seconds\n", " Rendering datasetOverviewTable took 0.001299 seconds\n", " Rendering bestGatesetSpamParametersTable took 0.007272 seconds\n", " Rendering bestGatesetSpamBriefTable took 2.667283 seconds\n", " Rendering bestGatesetSpamVsTargetTable took 0.007938 seconds\n", " Rendering bestGatesetGaugeOptParamsTable took 0.003673 seconds\n", " Rendering bestGatesetGatesBoxTable took 1.972593 seconds\n", " Rendering bestGatesetChoiEvalTable took 2.811332 seconds\n", " Rendering bestGatesetDecompTable took 0.976653 seconds\n", " Rendering bestGatesetEvalTable took 0.025112 seconds\n", " Rendering bestGermsEvalTable took 0.08723 seconds\n", " Rendering bestGatesetVsTargetTable took 0.004299 seconds\n", " Rendering bestGatesVsTargetTable_gv took 0.012343 seconds\n", " Rendering bestGatesVsTargetTable_gvgerms took 0.018309 seconds\n", " Rendering bestGatesVsTargetTable_gi took 0.004365 seconds\n", " Rendering bestGatesVsTargetTable_gigerms took 0.005716 seconds\n", " Rendering bestGatesVsTargetTable_sum took 0.012835 seconds\n", " Rendering bestGatesetErrGenBoxTable took 3.893303 seconds\n", " Rendering metadataTable took 0.01158 seconds\n", " Rendering stdoutBlock took 0.001182 seconds\n", " Rendering profilerTable took 0.002821 seconds\n", " Rendering softwareEnvTable took 0.002449 seconds\n", " Rendering exampleTable took 0.943825 seconds\n", " Rendering metricSwitchboard_gv took 5.4e-05 seconds\n", " Rendering metricSwitchboard_gi took 4.1e-05 seconds\n", " Rendering singleMetricTable_gv took 0.034174 seconds\n", " Rendering singleMetricTable_gi took 0.027452 seconds\n", " Rendering fiducialListTable took 0.005128 seconds\n", " Rendering prepStrListTable took 0.004599 seconds\n", " Rendering effectStrListTable took 0.006621 seconds\n", " Rendering colorBoxPlotKeyPlot took 0.067882 seconds\n", " Rendering germList2ColTable took 0.007428 seconds\n", " Rendering progressTable took 0.007178 seconds\n", " Rendering gramBarPlot took 0.125275 seconds\n", " Rendering progressBarPlot took 0.105399 seconds\n", " Rendering progressBarPlot_sum took 0.10687 seconds\n", " Rendering finalFitComparePlot took 0.054535 seconds\n", " Rendering bestEstimateColorBoxPlot took 1.040426 seconds\n", " Rendering bestEstimateTVDColorBoxPlot took 1.042014 seconds\n", " Rendering bestEstimateColorScatterPlot took 1.502723 seconds\n", " Rendering bestEstimateColorHistogram took 0.783763 seconds\n", " Rendering progressTable_scl took 0.000847 seconds\n", " Rendering progressBarPlot_scl took 0.001318 seconds\n", " Rendering bestEstimateColorBoxPlot_scl took 0.001162 seconds\n", " Rendering bestEstimateColorScatterPlot_scl took 0.000931 seconds\n", " Rendering bestEstimateColorHistogram_scl took 0.000939 seconds\n", " Rendering progressTable_ume took 0.000822 seconds\n", " Rendering progressBarPlot_ume took 0.000633 seconds\n", " Rendering bestEstimateColorBoxPlot_ume took 0.000813 seconds\n", " Rendering bestEstimateColorScatterPlot_ume took 0.000613 seconds\n", " Rendering bestEstimateColorHistogram_ume took 0.000916 seconds\n", " Rendering dataScalingColorBoxPlot took 0.000834 seconds\n", " Rendering unmodeledErrorBudgetTable took 0.0008 seconds\n", " Rendering dscmpSwitchboard took 4.3e-05 seconds\n", " Rendering dsComparisonSummary took 0.068067 seconds\n", " Rendering dsComparisonHistogram took 0.421229 seconds\n", " Rendering dsComparisonBoxPlot took 0.618521 seconds\n", "Output written to ../tutorial_files/exampleMultiEstimateReport directory\n", "Opening ../tutorial_files/exampleMultiEstimateReport/main.html...\n", "*** Report Generation Complete! Total time 82.3834s ***\n" ] } ], "source": [ "ws = pygsti.report.create_standard_report({'TP': results_tp, \"Full\": results_full},\n", " \"../tutorial_files/exampleMultiEstimateReport\",\n", " title=\"Example Multi-Estimate Report\", \n", " verbosity=2, auto_open=True)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "In the above call we capture the return value in the variable `ws` - a `Workspace` object. PyGSTi's `Workspace` objects function as both a factory for figures and tables as well as a smart cache for computed values. Within `create_standard_report` a `Workspace` object is created and used to create all the figures in the report. As an intended side effect, each of these figures is cached, along with some of the intermediate results used to create it. As we'll see below, a `Workspace` can also be specified as input to `create_standard_report`, allowing it to utilize previously cached quantities.\n", "\n", "**Another way**: Because both `results_tp` and `results_full` above used the same dataset and operation sequences, we could have combined them as two estimates in a single `Results` object (see the previous tutorial on pyGSTi's `Results` object). This can be done by renaming at least one of the `\"default\"`-named estimates in `results_tp` or `results_full` (below we rename both) and then adding the estimate within `results_full` to the estimates already contained in `results_tp`: " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "results_tp.rename_estimate('default','TP')\n", "results_full.rename_estimate('default','Full')\n", "results_both = results_tp.copy() #copy just for neatness\n", "results_both.add_estimates(results_full, estimatesToAdd=['Full'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Creating a report using `results_both` will result in the same report we just generated. We'll demonstrate this anyway, but in addition we'll supply `create_standard_report` a `ws` argument, which tells it to use any cached values contained in a given *input* `Workspace` to expedite report generation. Since our workspace object has the exact quantities we need cached in it, you'll notice a significant speedup. Finally, note that even though there's just a single `Results` object, you **still can't generate a PDF report** from it because it contains multiple estimates." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", " targetSpamBriefTable took 0.00044 seconds\n", " targetGatesBoxTable took 0.000298 seconds\n", " datasetOverviewTable took 0.00022 seconds\n", " bestGatesetSpamParametersTable took 0.001013 seconds\n", " bestGatesetSpamBriefTable took 0.001373 seconds\n", " bestGatesetSpamVsTargetTable took 0.000931 seconds\n", " bestGatesetGaugeOptParamsTable took 0.000709 seconds\n", " bestGatesetGatesBoxTable took 0.001309 seconds\n", " bestGatesetChoiEvalTable took 0.001177 seconds\n", " bestGatesetDecompTable took 0.00092 seconds\n", " bestGatesetEvalTable took 0.000392 seconds\n", " bestGermsEvalTable took 0.050954 seconds\n", " bestGatesetVsTargetTable took 0.006293 seconds\n", " bestGatesVsTargetTable_gv took 0.000973 seconds\n", " bestGatesVsTargetTable_gvgerms took 0.002265 seconds\n", " bestGatesVsTargetTable_gi took 0.000398 seconds\n", " bestGatesVsTargetTable_gigerms took 0.066392 seconds\n", " bestGatesVsTargetTable_sum took 0.000719 seconds\n", " bestGatesetErrGenBoxTable took 0.000676 seconds\n", " metadataTable took 0.003255 seconds\n", " stdoutBlock took 0.000167 seconds\n", " profilerTable took 0.000897 seconds\n", " softwareEnvTable took 0.000129 seconds\n", " exampleTable took 0.000102 seconds\n", " singleMetricTable_gv took 0.963833 seconds\n", " singleMetricTable_gi took 0.045002 seconds\n", " fiducialListTable took 0.000195 seconds\n", " prepStrListTable took 0.000207 seconds\n", " effectStrListTable took 0.000191 seconds\n", " colorBoxPlotKeyPlot took 0.000375 seconds\n", " germList2ColTable took 0.0015 seconds\n", " progressTable took 0.707788 seconds\n", "*** Generating plots ***\n", " gramBarPlot took 0.000454 seconds\n", " progressBarPlot took 0.760048 seconds\n", " progressBarPlot_sum took 0.000572 seconds\n", " finalFitComparePlot took 0.088992 seconds\n", " bestEstimateColorBoxPlot took 5.80522 seconds\n", " bestEstimateTVDColorBoxPlot took 2.450461 seconds\n", " bestEstimateColorScatterPlot took 2.432255 seconds\n", " bestEstimateColorHistogram took 1.908432 seconds\n", " progressTable_scl took 0.000801 seconds\n", " progressBarPlot_scl took 6e-05 seconds\n", " bestEstimateColorBoxPlot_scl took 0.000156 seconds\n", " bestEstimateColorScatterPlot_scl took 0.000152 seconds\n", " bestEstimateColorHistogram_scl took 0.000142 seconds\n", " progressTable_ume took 7.9e-05 seconds\n", " progressBarPlot_ume took 6.5e-05 seconds\n", " bestEstimateColorBoxPlot_ume took 0.000201 seconds\n", " bestEstimateColorScatterPlot_ume took 0.000191 seconds\n", " bestEstimateColorHistogram_ume took 0.000183 seconds\n", " dataScalingColorBoxPlot took 0.000117 seconds\n", " unmodeledErrorBudgetTable took 0.000191 seconds\n", "*** Merging into template file ***\n", " Rendering topSwitchboard took 9.9e-05 seconds\n", " Rendering maxLSwitchboard1 took 8.2e-05 seconds\n", " Rendering targetSpamBriefTable took 0.333739 seconds\n", " Rendering targetGatesBoxTable took 0.323861 seconds\n", " Rendering datasetOverviewTable took 0.001096 seconds\n", " Rendering bestGatesetSpamParametersTable took 0.006244 seconds\n", " Rendering bestGatesetSpamBriefTable took 1.933245 seconds\n", " Rendering bestGatesetSpamVsTargetTable took 0.007183 seconds\n", " Rendering bestGatesetGaugeOptParamsTable took 0.003619 seconds\n", " Rendering bestGatesetGatesBoxTable took 1.909 seconds\n", " Rendering bestGatesetChoiEvalTable took 2.84683 seconds\n", " Rendering bestGatesetDecompTable took 0.981645 seconds\n", " Rendering bestGatesetEvalTable took 0.023699 seconds\n", " Rendering bestGermsEvalTable took 0.088832 seconds\n", " Rendering bestGatesetVsTargetTable took 0.004196 seconds\n", " Rendering bestGatesVsTargetTable_gv took 0.012065 seconds\n", " Rendering bestGatesVsTargetTable_gvgerms took 0.019041 seconds\n", " Rendering bestGatesVsTargetTable_gi took 0.004192 seconds\n", " Rendering bestGatesVsTargetTable_gigerms took 0.005039 seconds\n", " Rendering bestGatesVsTargetTable_sum took 0.010589 seconds\n", " Rendering bestGatesetErrGenBoxTable took 4.822777 seconds\n", " Rendering metadataTable took 0.011822 seconds\n", " Rendering stdoutBlock took 0.001504 seconds\n", " Rendering profilerTable took 0.002887 seconds\n", " Rendering softwareEnvTable took 0.002497 seconds\n", " Rendering exampleTable took 0.056633 seconds\n", " Rendering metricSwitchboard_gv took 3.9e-05 seconds\n", " Rendering metricSwitchboard_gi took 2.9e-05 seconds\n", " Rendering singleMetricTable_gv took 0.019223 seconds\n", " Rendering singleMetricTable_gi took 0.015186 seconds\n", " Rendering fiducialListTable took 0.002964 seconds\n", " Rendering prepStrListTable took 0.002168 seconds\n", " Rendering effectStrListTable took 0.002232 seconds\n", " Rendering colorBoxPlotKeyPlot took 0.061218 seconds\n", " Rendering germList2ColTable took 0.004123 seconds\n", " Rendering progressTable took 0.007311 seconds\n", " Rendering gramBarPlot took 0.109358 seconds\n", " Rendering progressBarPlot took 0.106966 seconds\n", " Rendering progressBarPlot_sum took 0.112411 seconds\n", " Rendering finalFitComparePlot took 0.052846 seconds\n", " Rendering bestEstimateColorBoxPlot took 1.025283 seconds\n", " Rendering bestEstimateTVDColorBoxPlot took 1.063756 seconds\n", " Rendering bestEstimateColorScatterPlot took 1.556072 seconds\n", " Rendering bestEstimateColorHistogram took 0.787144 seconds\n", " Rendering progressTable_scl took 0.000644 seconds\n", " Rendering progressBarPlot_scl took 0.000733 seconds\n", " Rendering bestEstimateColorBoxPlot_scl took 0.001121 seconds\n", " Rendering bestEstimateColorScatterPlot_scl took 0.000887 seconds\n", " Rendering bestEstimateColorHistogram_scl took 0.000945 seconds\n", " Rendering progressTable_ume took 0.001037 seconds\n", " Rendering progressBarPlot_ume took 0.000782 seconds\n", " Rendering bestEstimateColorBoxPlot_ume took 0.000873 seconds\n", " Rendering bestEstimateColorScatterPlot_ume took 0.000761 seconds\n", " Rendering bestEstimateColorHistogram_ume took 0.000577 seconds\n", " Rendering dataScalingColorBoxPlot took 0.000746 seconds\n", " Rendering unmodeledErrorBudgetTable took 0.00082 seconds\n", "Output written to ../tutorial_files/exampleMultiEstimateReport2 directory\n", "Opening ../tutorial_files/exampleMultiEstimateReport2/main.html...\n", "*** Report Generation Complete! Total time 34.166s ***\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pygsti.report.create_standard_report(results_both,\n", " \"../tutorial_files/exampleMultiEstimateReport2\",\n", " title=\"Example Multi-Estimate Report (v2)\", \n", " verbosity=2, auto_open=True, ws=ws)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Multiple estimates and `do_stdpractice_gst`\n", "It's no coincidence that a `Results` object containing multiple estimates using the same data is precisely what's returned from `do_stdpractice_gst` (see docstring for information on its arguments, and see the [GST functions tutorial](../algorithms/GST-Drivers.ipynb)). This allows one to run GST multiple times, creating several different \"standard\" estimates and gauge optimizations, and plot them all in a single (HTML) report. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-- Std Practice: Iter 1 of 3 (TP) --: \n", " --- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", " --- LGST ---\n", " Singular values of I_tilde (truncating to first 4 of 6) = \n", " 4.243730350963286\n", " 1.1796261581655645\n", " 0.9627515645786063\n", " 0.9424890722054706\n", " 0.033826151547621315\n", " 0.01692336936843073\n", " \n", " Singular values of target I_tilde (truncating to first 4 of 6) = \n", " 4.242640687119286\n", " 1.414213562373096\n", " 1.414213562373096\n", " 1.4142135623730954\n", " 2.484037189058858e-16\n", " 1.506337939585075e-16\n", " \n", " Resulting model:\n", " \n", " rho0 = TPSPAMVec with dimension 4\n", " 0.71-0.02 0.03 0.75\n", " \n", " \n", " Mdefault = TPPOVM with effect vectors:\n", " 0: FullSPAMVec with dimension 4\n", " 0.73 0 0 0.65\n", " \n", " 1: ComplementSPAMVec with dimension 4\n", " 0.69 0 0-0.65\n", " \n", " \n", " \n", " Gi = \n", " TPDenseOp with shape (4, 4)\n", " 1.00 0 0 0\n", " 0.01 0.92-0.03 0.02\n", " 0.01-0.01 0.90 0.02\n", " -0.01 0 0 0.91\n", " \n", " \n", " Gx = \n", " TPDenseOp with shape (4, 4)\n", " 1.00 0 0 0\n", " 0 0.91-0.01 0\n", " -0.02-0.02-0.04-0.99\n", " -0.05 0.03 0.81 0\n", " \n", " \n", " Gy = \n", " TPDenseOp with shape (4, 4)\n", " 1.00 0 0 0\n", " 0.05 0 0 0.98\n", " 0.01 0 0.89-0.03\n", " -0.06-0.82 0 0\n", " \n", " \n", " \n", " \n", " --- Iterative MLGST: Iter 1 of 5 92 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (92 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 87.7156, mu=0, |J|=1139.8\n", " --- Outer Iter 1: norm_f = 72.5388, mu=331.735, |J|=4004.6\n", " --- Outer Iter 2: norm_f = 49.7369, mu=110.578, |J|=4000.74\n", " --- Outer Iter 3: norm_f = 49.7313, mu=36.8595, |J|=4000.76\n", " --- Outer Iter 4: norm_f = 49.7312, mu=12.2865, |J|=4000.76\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 49.7312 (92 data params - 31 model params = expected mean of 61; p-value = 0.848365)\n", " Completed in 0.2s\n", " 2*Delta(log(L)) = 49.9289\n", " Iteration 1 took 0.2s\n", " \n", " --- Iterative MLGST: Iter 2 of 5 168 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (168 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 151.528, mu=0, |J|=4116.86\n", " --- Outer Iter 1: norm_f = 116.544, mu=1805.29, |J|=4114.08\n", " --- Outer Iter 2: norm_f = 111.925, mu=601.763, |J|=4113.57\n", " --- Outer Iter 3: norm_f = 111.481, mu=200.588, |J|=4113.47\n", " --- Outer Iter 4: norm_f = 111.47, mu=66.8625, |J|=4113.46\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 111.47 (168 data params - 31 model params = expected mean of 137; p-value = 0.94621)\n", " Completed in 0.2s\n", " 2*Delta(log(L)) = 111.83\n", " Iteration 2 took 0.2s\n", " \n", " --- Iterative MLGST: Iter 3 of 5 450 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (450 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 496.301, mu=0, |J|=4503.89\n", " --- Outer Iter 1: norm_f = 422.607, mu=2013.07, |J|=4503.59\n", " --- Outer Iter 2: norm_f = 421.667, mu=671.023, |J|=4503.46\n", " --- Outer Iter 3: norm_f = 421.662, mu=223.674, |J|=4503.46\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 421.662 (450 data params - 31 model params = expected mean of 419; p-value = 0.454312)\n", " Completed in 0.4s\n", " 2*Delta(log(L)) = 422.134\n", " Iteration 3 took 0.5s\n", " \n", " --- Iterative MLGST: Iter 4 of 5 862 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (862 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 854.092, mu=0, |J|=5091.17\n", " --- Outer Iter 1: norm_f = 813.479, mu=2302.96, |J|=5076.94\n", " --- Outer Iter 2: norm_f = 813.094, mu=767.654, |J|=5076.79\n", " --- Outer Iter 3: norm_f = 813.093, mu=255.885, |J|=5076.81\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 813.093 (862 data params - 31 model params = expected mean of 831; p-value = 0.664967)\n", " Completed in 0.7s\n", " 2*Delta(log(L)) = 814.492\n", " Iteration 4 took 0.8s\n", " \n", " --- Iterative MLGST: Iter 5 of 5 1282 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (1282 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params).\n", " --- Outer Iter 0: norm_f = 1263.73, mu=0, |J|=5733.35\n", " --- Outer Iter 1: norm_f = 1250.68, mu=2582.97, |J|=5732.82\n", " --- Outer Iter 2: norm_f = 1250.62, mu=860.99, |J|=5733.21\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 1250.62 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.497713)\n", " Completed in 1.0s\n", " 2*Delta(log(L)) = 1252.41\n", " Iteration 5 took 1.1s\n", " \n", " Switching to ML objective (last iteration)\n", " --- MLGST ---\n", " --- Outer Iter 0: norm_f = 626.205, mu=0, |J|=3678.46\n", " --- Outer Iter 1: norm_f = 626.2, mu=3871.69, |J|=3391.16\n", " --- Outer Iter 2: norm_f = 626.195, mu=4.22891e+07, |J|=3314.59\n", " --- Outer Iter 3: norm_f = 626.186, mu=2.22294e+07, |J|=3102.02\n", " --- Outer Iter 4: norm_f = 626.178, mu=2.05642e+07, |J|=3014.46\n", " --- Outer Iter 5: norm_f = 626.177, mu=2.02336e+07, |J|=3121.67\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Maximum log(L) = 626.177 below upper bound of -2.13594e+06\n", " 2*Delta(log(L)) = 1252.35 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.483889)\n", " Completed in 1.1s\n", " 2*Delta(log(L)) = 1252.35\n", " Final MLGST took 1.1s\n", " \n", " Iterative MLGST Total Time: 3.9s\n", " -- Performing 'single' gauge optimization on TP estimate --\n", " -- Adding Gauge Optimized (single) --\n", " -- Performing 'Spam 0.001' gauge optimization on TP estimate --\n", " -- Adding Gauge Optimized (Spam 0.001) --\n", " -- Performing 'Spam 0.001+v' gauge optimization on TP estimate --\n", " -- Adding Gauge Optimized (Spam 0.001+v) --\n", "-- Std Practice: Iter 2 of 3 (CPTP) --: \n", " --- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", " --- Iterative MLGST: Iter 1 of 5 92 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (92 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 60 params (taken as 1 param groups of ~60 params).\n", " --- Outer Iter 0: norm_f = 1.10824e+07, mu=0, |J|=776.628\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " --- Outer Iter 1: norm_f = 13800.3, mu=76.022, |J|=1599.57\n", " --- Outer Iter 2: norm_f = 763.11, mu=25.3407, |J|=816.023\n", " --- Outer Iter 3: norm_f = 255.531, mu=23.2393, |J|=642.003\n", " --- Outer Iter 4: norm_f = 56.0615, mu=7.74643, |J|=683.416\n", " --- Outer Iter 5: norm_f = 50.4927, mu=4.60597, |J|=690.695\n", " --- Outer Iter 6: norm_f = 50.2238, mu=302.543, |J|=691.859\n", " --- Outer Iter 7: norm_f = 50.08, mu=625.953, |J|=692.087\n", " --- Outer Iter 8: norm_f = 50.0063, mu=1252.23, |J|=692.272\n", " --- Outer Iter 9: norm_f = 49.994, mu=1254.18, |J|=692.395\n", " --- Outer Iter 10: norm_f = 49.9873, mu=1254.18, |J|=692.445\n", " --- Outer Iter 11: norm_f = 49.9826, mu=1245.43, |J|=692.488\n", " --- Outer Iter 12: norm_f = 49.9788, mu=1164.85, |J|=692.528\n", " --- Outer Iter 13: norm_f = 49.9754, mu=960.324, |J|=692.568\n", " --- Outer Iter 14: norm_f = 49.9717, mu=788.918, |J|=692.612\n", " --- Outer Iter 15: norm_f = 49.968, mu=778.874, |J|=692.664\n", " --- Outer Iter 16: norm_f = 49.9657, mu=870.975, |J|=692.716\n", " --- Outer Iter 17: norm_f = 49.9647, mu=1483.87, |J|=692.763\n", " --- Outer Iter 18: norm_f = 49.9579, mu=1500.05, |J|=692.766\n", " --- Outer Iter 19: norm_f = 49.9549, mu=1482.36, |J|=692.793\n", " --- Outer Iter 20: norm_f = 49.9525, mu=889.486, |J|=692.816\n", " --- Outer Iter 21: norm_f = 49.9487, mu=296.495, |J|=692.85\n", " --- Outer Iter 22: norm_f = 49.9404, mu=279.821, |J|=692.941\n", " --- Outer Iter 23: norm_f = 49.936, mu=2197.07, |J|=692.96\n", " --- Outer Iter 24: norm_f = 49.9345, mu=732.357, |J|=692.977\n", " --- Outer Iter 25: norm_f = 49.9299, mu=244.119, |J|=693.015\n", " --- Outer Iter 26: norm_f = 49.9165, mu=81.373, |J|=693.097\n", " --- Outer Iter 27: norm_f = 49.8855, mu=48.3574, |J|=693.193\n", " --- Outer Iter 28: norm_f = 49.8755, mu=343.197, |J|=693.547\n", " --- Outer Iter 29: norm_f = 49.8723, mu=2564.54, |J|=693.554\n", " --- Outer Iter 30: norm_f = 49.8711, mu=875.858, |J|=693.568\n", " --- Outer Iter 31: norm_f = 49.8681, mu=291.953, |J|=693.602\n", " --- Outer Iter 32: norm_f = 49.8599, mu=97.3176, |J|=693.686\n", " --- Outer Iter 33: norm_f = 49.8568, mu=719.379, |J|=694.836\n", " --- Outer Iter 34: norm_f = 49.8484, mu=1438.71, |J|=694.773\n", " --- Outer Iter 35: norm_f = 49.846, mu=1953.37, |J|=694.828\n", " --- Outer Iter 36: norm_f = 49.8408, mu=2078.72, |J|=694.759\n", " --- Outer Iter 37: norm_f = 49.837, mu=2085.7, |J|=694.723\n", " --- Outer Iter 38: norm_f = 49.8343, mu=2069.11, |J|=694.717\n", " --- Outer Iter 39: norm_f = 49.832, mu=1700.46, |J|=694.721\n", " --- Outer Iter 40: norm_f = 49.8295, mu=942.418, |J|=694.734\n", " --- Outer Iter 41: norm_f = 49.8257, mu=695.968, |J|=694.767\n", " --- Outer Iter 42: norm_f = 49.8248, mu=1014.39, |J|=694.948\n", " --- Outer Iter 43: norm_f = 49.8195, mu=2058.22, |J|=694.832\n", " --- Outer Iter 44: norm_f = 49.8172, mu=2052.16, |J|=694.821\n", " --- Outer Iter 45: norm_f = 49.8155, mu=1503.29, |J|=694.826\n", " --- Outer Iter 46: norm_f = 49.8134, mu=530.976, |J|=694.84\n", " --- Outer Iter 47: norm_f = 49.8085, mu=393.107, |J|=694.898\n", " --- Outer Iter 48: norm_f = 49.8069, mu=2812.7, |J|=694.886\n", " --- Outer Iter 49: norm_f = 49.8059, mu=937.568, |J|=694.894\n", " --- Outer Iter 50: norm_f = 49.8031, mu=312.523, |J|=694.916\n", " --- Outer Iter 51: norm_f = 49.7955, mu=104.174, |J|=694.971\n", " --- Outer Iter 52: norm_f = 49.7797, mu=44.277, |J|=695.102\n", " --- Outer Iter 53: norm_f = 49.7774, mu=367.359, |J|=695.394\n", " --- Outer Iter 54: norm_f = 49.774, mu=2818.86, |J|=695.263\n", " --- Outer Iter 55: norm_f = 49.7733, mu=1867.59, |J|=695.261\n", " --- Outer Iter 56: norm_f = 49.7726, mu=622.531, |J|=695.272\n", " --- Outer Iter 57: norm_f = 49.7704, mu=207.51, |J|=695.303\n", " --- Outer Iter 58: norm_f = 49.765, mu=69.1701, |J|=695.393\n", " --- Outer Iter 59: norm_f = 49.7637, mu=553.511, |J|=695.481\n", " --- Outer Iter 60: norm_f = 49.7625, mu=1122.46, |J|=695.477\n", " --- Outer Iter 61: norm_f = 49.7614, mu=1122.43, |J|=695.485\n", " --- Outer Iter 62: norm_f = 49.7605, mu=1076.81, |J|=695.496\n", " --- Outer Iter 63: norm_f = 49.7596, mu=840.997, |J|=695.513\n", " --- Outer Iter 64: norm_f = 49.7586, mu=555.013, |J|=695.536\n", " --- Outer Iter 65: norm_f = 49.7573, mu=516.685, |J|=695.584\n", " --- Outer Iter 66: norm_f = 49.7569, mu=718.533, |J|=695.684\n", " --- Outer Iter 67: norm_f = 49.7564, mu=1151.92, |J|=695.741\n", " --- Outer Iter 68: norm_f = 49.7543, mu=1159.8, |J|=695.66\n", " --- Outer Iter 69: norm_f = 49.7536, mu=1142.95, |J|=695.664\n", " --- Outer Iter 70: norm_f = 49.7529, mu=688.757, |J|=695.678\n", " --- Outer Iter 71: norm_f = 49.752, mu=229.586, |J|=695.705\n", " --- Outer Iter 72: norm_f = 49.7501, mu=206.313, |J|=695.813\n", " --- Outer Iter 73: norm_f = 49.7493, mu=1588.09, |J|=695.796\n", " --- Outer Iter 74: norm_f = 49.7489, mu=529.363, |J|=695.807\n", " --- Outer Iter 75: norm_f = 49.748, mu=176.454, |J|=695.84\n", " --- Outer Iter 76: norm_f = 49.7456, mu=58.8181, |J|=695.929\n", " --- Outer Iter 77: norm_f = 49.7415, mu=39.4064, |J|=696.175\n", " --- Outer Iter 78: norm_f = 49.7409, mu=1162.82, |J|=696.202\n", " --- Outer Iter 79: norm_f = 49.7406, mu=974.639, |J|=696.207\n", " --- Outer Iter 80: norm_f = 49.7404, mu=324.88, |J|=696.222\n", " --- Outer Iter 81: norm_f = 49.7398, mu=108.293, |J|=696.264\n", " --- Outer Iter 82: norm_f = 49.7383, mu=36.0977, |J|=696.377\n", " --- Outer Iter 83: norm_f = 49.7361, mu=29.9593, |J|=696.675\n", " --- Outer Iter 84: norm_f = 49.7356, mu=856.051, |J|=696.696\n", " --- Outer Iter 85: norm_f = 49.7354, mu=793.038, |J|=696.696\n", " --- Outer Iter 86: norm_f = 49.7354, mu=264.346, |J|=696.708\n", " --- Outer Iter 87: norm_f = 49.7351, mu=88.1154, |J|=696.744\n", " --- Outer Iter 88: norm_f = 49.7345, mu=29.3718, |J|=696.843\n", " --- Outer Iter 89: norm_f = 49.7335, mu=18.2704, |J|=697.084\n", " --- Outer Iter 90: norm_f = 49.7334, mu=467.768, |J|=697.106\n", " --- Outer Iter 91: norm_f = 49.7333, mu=369.005, |J|=697.118\n", " --- Outer Iter 92: norm_f = 49.7332, mu=123.002, |J|=697.136\n", " --- Outer Iter 93: norm_f = 49.7331, mu=41.0005, |J|=697.188\n", " --- Outer Iter 94: norm_f = 49.7328, mu=35.6311, |J|=697.337\n", " --- Outer Iter 95: norm_f = 49.7327, mu=288.264, |J|=697.348\n", " --- Outer Iter 96: norm_f = 49.7326, mu=287.805, |J|=697.363\n", " --- Outer Iter 97: norm_f = 49.7326, mu=240.026, |J|=697.38\n", " --- Outer Iter 98: norm_f = 49.7325, mu=119.89, |J|=697.4\n", " --- Outer Iter 99: norm_f = 49.7324, mu=76.7863, |J|=697.442\n", " --- Outer Iter 100: norm_f = 49.7324, mu=123.618, |J|=697.524\n", " --- Outer Iter 101: norm_f = 49.7323, mu=253.711, |J|=697.519\n", " --- Outer Iter 102: norm_f = 49.7323, mu=253.72, |J|=697.532\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 49.7323 (92 data params - 31 model params = expected mean of 61; p-value = 0.848339)\n", " Completed in 5.7s\n", " 2*Delta(log(L)) = 49.9301\n", " Iteration 1 took 5.7s\n", " \n", " --- Iterative MLGST: Iter 2 of 5 168 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (168 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 60 params (taken as 1 param groups of ~60 params).\n", " --- Outer Iter 0: norm_f = 151.52, mu=0, |J|=967.039\n", " --- Outer Iter 1: norm_f = 130.967, mu=116.389, |J|=928.618\n", " --- Outer Iter 2: norm_f = 111.594, mu=38.7965, |J|=947.135\n", " --- Outer Iter 3: norm_f = 111.489, mu=25.4472, |J|=946.409\n", " --- Outer Iter 4: norm_f = 111.485, mu=56.0514, |J|=946.049\n", " --- Outer Iter 5: norm_f = 111.48, mu=59.7501, |J|=946.009\n", " --- Outer Iter 6: norm_f = 111.477, mu=478.572, |J|=945.79\n", " --- Outer Iter 7: norm_f = 111.476, mu=499.083, |J|=945.722\n", " --- Outer Iter 8: norm_f = 111.476, mu=509.545, |J|=945.664\n", " --- Outer Iter 9: norm_f = 111.476, mu=509.992, |J|=945.611\n", " --- Outer Iter 10: norm_f = 111.475, mu=508.735, |J|=945.566\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " --- Outer Iter 11: norm_f = 111.475, mu=479.54, |J|=945.524\n", " --- Outer Iter 12: norm_f = 111.475, mu=380.228, |J|=945.484\n", " --- Outer Iter 13: norm_f = 111.475, mu=276.25, |J|=945.438\n", " --- Outer Iter 14: norm_f = 111.474, mu=261.037, |J|=945.382\n", " --- Outer Iter 15: norm_f = 111.474, mu=284.527, |J|=945.338\n", " --- Outer Iter 16: norm_f = 111.474, mu=510.574, |J|=945.312\n", " --- Outer Iter 17: norm_f = 111.474, mu=512.86, |J|=945.255\n", " --- Outer Iter 18: norm_f = 111.473, mu=500.515, |J|=945.225\n", " --- Outer Iter 19: norm_f = 111.473, mu=277.122, |J|=945.197\n", " --- Outer Iter 20: norm_f = 111.473, mu=92.3741, |J|=945.15\n", " --- Outer Iter 21: norm_f = 111.473, mu=89.0899, |J|=945.033\n", " --- Outer Iter 22: norm_f = 111.473, mu=696.009, |J|=945.01\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 111.473 (168 data params - 31 model params = expected mean of 137; p-value = 0.946189)\n", " Completed in 1.3s\n", " 2*Delta(log(L)) = 111.833\n", " Iteration 2 took 1.4s\n", " \n", " --- Iterative MLGST: Iter 3 of 5 450 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (450 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 60 params (taken as 1 param groups of ~60 params).\n", " --- Outer Iter 0: norm_f = 496.32, mu=0, |J|=1561.66\n", " --- Outer Iter 1: norm_f = 426.477, mu=86.5337, |J|=1540.16\n", " --- Outer Iter 2: norm_f = 421.67, mu=28.8446, |J|=1549.6\n", " --- Outer Iter 3: norm_f = 421.67, mu=56.7229, |J|=1549.64\n", " --- Outer Iter 4: norm_f = 421.663, mu=41.9593, |J|=1549.24\n", " --- Outer Iter 5: norm_f = 421.663, mu=325.909, |J|=1549.16\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 421.663 (450 data params - 31 model params = expected mean of 419; p-value = 0.454296)\n", " Completed in 0.7s\n", " 2*Delta(log(L)) = 422.138\n", " Iteration 3 took 0.8s\n", " \n", " --- Iterative MLGST: Iter 4 of 5 862 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (862 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 60 params (taken as 1 param groups of ~60 params).\n", " --- Outer Iter 0: norm_f = 854.094, mu=0, |J|=2193.25\n", " --- Outer Iter 1: norm_f = 823.991, mu=391.282, |J|=2165.95\n", " --- Outer Iter 2: norm_f = 813.8, mu=130.427, |J|=2181.12\n", " --- Outer Iter 3: norm_f = 813.166, mu=64.052, |J|=2186.38\n", " --- Outer Iter 4: norm_f = 813.111, mu=55.6099, |J|=2185.8\n", " --- Outer Iter 5: norm_f = 813.109, mu=110.423, |J|=2184.67\n", " --- Outer Iter 6: norm_f = 813.107, mu=853.18, |J|=2184.58\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 813.107 (862 data params - 31 model params = expected mean of 831; p-value = 0.664836)\n", " Completed in 1.3s\n", " 2*Delta(log(L)) = 814.503\n", " Iteration 4 took 1.4s\n", " \n", " --- Iterative MLGST: Iter 5 of 5 1282 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " bulk_evaltree: created initial tree (1282 strs) in 0s\n", " bulk_evaltree: split tree (1 subtrees) in 0s\n", " Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing)\n", " groups of ~1 procs each, to distribute over 60 params (taken as 1 param groups of ~60 params).\n", " --- Outer Iter 0: norm_f = 1263.74, mu=0, |J|=2688.77\n", " --- Outer Iter 1: norm_f = 1250.78, mu=223.791, |J|=2680.2\n", " --- Outer Iter 2: norm_f = 1250.64, mu=74.5969, |J|=2682.22\n", " --- Outer Iter 3: norm_f = 1250.64, mu=1192.45, |J|=2682.15\n", " --- Outer Iter 4: norm_f = 1250.64, mu=1193, |J|=2682.06\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Sum of Chi^2 = 1250.64 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.497589)\n", " Completed in 1.5s\n", " 2*Delta(log(L)) = 1252.42\n", " Iteration 5 took 1.7s\n", " \n", " Switching to ML objective (last iteration)\n", " --- MLGST ---\n", " --- Outer Iter 0: norm_f = 626.208, mu=0, |J|=1896.56\n", " --- Outer Iter 1: norm_f = 626.182, mu=111.219, |J|=1897.22\n", " Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06\n", " Maximum log(L) = 626.182 below upper bound of -2.13594e+06\n", " 2*Delta(log(L)) = 1252.36 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.483808)\n", " Completed in 0.7s\n", " 2*Delta(log(L)) = 1252.36\n", " Final MLGST took 0.7s\n", " \n", " Iterative MLGST Total Time: 11.7s\n", " -- Performing 'single' gauge optimization on CPTP estimate --\n", " -- Adding Gauge Optimized (single) --\n", " -- Performing 'Spam 0.001' gauge optimization on CPTP estimate --\n", " -- Adding Gauge Optimized (Spam 0.001) --\n", " -- Performing 'Spam 0.001+v' gauge optimization on CPTP estimate --\n", " -- Adding Gauge Optimized (Spam 0.001+v) --\n", "-- Std Practice: Iter 3 of 3 (Target) --: \n", " --- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", " -- Performing 'single' gauge optimization on Target estimate --\n", " -- Adding Gauge Optimized (single) --\n", " -- Performing 'Spam 0.001' gauge optimization on Target estimate --\n", " -- Adding Gauge Optimized (Spam 0.001) --\n", " -- Performing 'Spam 0.001+v' gauge optimization on Target estimate --\n", " -- Adding Gauge Optimized (Spam 0.001+v) --\n", "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", "*** Generating plots ***\n", "*** Merging into template file ***\n", "Output written to ../tutorial_files/exampleStdReport directory\n", "Opening ../tutorial_files/exampleStdReport/main.html...\n", "*** Report Generation Complete! Total time 97.2188s ***\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results_std = pygsti.do_stdpractice_gst(ds, target_model, fiducials, fiducials, germs,\n", " maxLengths, verbosity=4, modes=\"TP,CPTP,Target\",\n", " gaugeOptSuite=('single','toggleValidSpam'))\n", "\n", "# Generate a report with \"TP\", \"CPTP\", and \"Target\" estimates\n", "pygsti.report.create_standard_report(results_std, \"../tutorial_files/exampleStdReport\", \n", " title=\"Post StdPractice Report\", auto_open=True,\n", " verbosity=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reports with confidence regions\n", "To display confidence intervals for reported quantities, you must do two things:\n", "\n", "1. you must specify the `confidenceLevel` argument to `create_standard_report`.\n", "2. the estimate(s) being reported must have a valid confidence-region-factory.\n", "\n", "Constructing a factory often means computing a Hessian, which can be time consuming, and so this is *not* done automatically. Here we demonstrate how to construct a valid factory for the \"Spam 0.001\" gauge-optimization of the \"CPTP\" estimate by computing and then projecting the Hessian of the likelihood function. " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " \n", "--- Hessian Projector Optimization from separate SPAM and Gate weighting ---\n", " Resulting intrinsic errors: 0.00716691 (gates), 0.00379068 (spam)\n", " Resulting sqrt(mean(operationCIs**2)): 0.00983088\n", " Resulting sqrt(mean(spamCIs**2)): 0.0077282\n", "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", "*** Generating plots ***\n", "*** Merging into template file ***\n", "Output written to ../tutorial_files/exampleStdReport2 directory\n", "Opening ../tutorial_files/exampleStdReport2/main.html...\n", "*** Report Generation Complete! Total time 112.6s ***\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#Construct and initialize a \"confidence region factory\" for the CPTP estimate\n", "crfact = results_std.estimates[\"CPTP\"].add_confidence_region_factory('Spam 0.001', 'final')\n", "crfact.compute_hessian(comm=None) #we could use more processors\n", "crfact.project_hessian('intrinsic error')\n", "\n", "pygsti.report.create_standard_report(results_std, \"../tutorial_files/exampleStdReport2\", \n", " title=\"Post StdPractice Report (w/CIs on CPTP)\",\n", " confidenceLevel=95, auto_open=True, verbosity=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reports with multiple *different* data sets\n", "We've already seen above that `create_standard_report` can be given a dictionary of `Results` objects instead of a single one. This allows the creation of reports containing estimates for different `DataSet`s (each `Results` object only holds estimates for a single `DataSet`). Furthermore, when the data sets have the same operation sequences, they will be compared within a tab of the HTML report.\n", "\n", "Below, we generate a new data set with the same sequences as the one loaded at the beginning of this tutorial, proceed to run standard-practice GST on that dataset, and create a report of the results along with those of the original dataset. Look at the **\"Data Comparison\" tab** within the gauge-invariant error metrics category." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-- Std Practice: Iter 1 of 3 (TP) --: \n", " --- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", " --- LGST ---\n", " Singular values of I_tilde (truncating to first 4 of 6) = \n", " 4.244829997162508\n", " 1.1936677889884049\n", " 0.9868539533169907\n", " 0.932197724091589\n", " 0.04714742318656945\n", " 0.012700520808584604\n", " \n", " Singular values of target I_tilde (truncating to first 4 of 6) = \n", " 4.242640687119286\n", " 1.414213562373096\n", " 1.414213562373096\n", " 1.4142135623730954\n", " 2.484037189058858e-16\n", " 1.506337939585075e-16\n", " \n", " --- Iterative MLGST: Iter 1 of 5 92 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 50.2568 (92 data params - 31 model params = expected mean of 61; p-value = 0.835246)\n", " Completed in 0.2s\n", " 2*Delta(log(L)) = 50.4026\n", " Iteration 1 took 0.2s\n", " \n", " --- Iterative MLGST: Iter 2 of 5 168 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 112.85 (168 data params - 31 model params = expected mean of 137; p-value = 0.934965)\n", " Completed in 0.2s\n", " 2*Delta(log(L)) = 112.943\n", " Iteration 2 took 0.2s\n", " \n", " --- Iterative MLGST: Iter 3 of 5 450 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 409.836 (450 data params - 31 model params = expected mean of 419; p-value = 0.616314)\n", " Completed in 0.3s\n", " 2*Delta(log(L)) = 410.099\n", " Iteration 3 took 0.4s\n", " \n", " --- Iterative MLGST: Iter 4 of 5 862 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 833.69 (862 data params - 31 model params = expected mean of 831; p-value = 0.467224)\n", " Completed in 0.5s\n", " 2*Delta(log(L)) = 834.058\n", " Iteration 4 took 0.6s\n", " \n", " --- Iterative MLGST: Iter 5 of 5 1282 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 1262.38 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.405135)\n", " Completed in 0.8s\n", " 2*Delta(log(L)) = 1263.06\n", " Iteration 5 took 1.0s\n", " \n", " Switching to ML objective (last iteration)\n", " --- MLGST ---\n", " Maximum log(L) = 631.509 below upper bound of -2.13633e+06\n", " 2*Delta(log(L)) = 1263.02 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.400201)\n", " Completed in 1.2s\n", " 2*Delta(log(L)) = 1263.02\n", " Final MLGST took 1.2s\n", " \n", " Iterative MLGST Total Time: 3.5s\n", " -- Performing 'single' gauge optimization on TP estimate --\n", " -- Performing 'Spam 0.001' gauge optimization on TP estimate --\n", " -- Performing 'Spam 0.001+v' gauge optimization on TP estimate --\n", "-- Std Practice: Iter 2 of 3 (CPTP) --: \n", " --- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", " --- Iterative MLGST: Iter 1 of 5 92 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 50.2612 (92 data params - 31 model params = expected mean of 61; p-value = 0.835132)\n", " Completed in 3.8s\n", " 2*Delta(log(L)) = 50.4048\n", " Iteration 1 took 3.9s\n", " \n", " --- Iterative MLGST: Iter 2 of 5 168 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 112.852 (168 data params - 31 model params = expected mean of 137; p-value = 0.934949)\n", " Completed in 1.2s\n", " 2*Delta(log(L)) = 112.944\n", " Iteration 2 took 1.2s\n", " \n", " --- Iterative MLGST: Iter 3 of 5 450 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 409.843 (450 data params - 31 model params = expected mean of 419; p-value = 0.616221)\n", " Completed in 3.8s\n", " 2*Delta(log(L)) = 410.108\n", " Iteration 3 took 3.9s\n", " \n", " --- Iterative MLGST: Iter 4 of 5 862 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 833.69 (862 data params - 31 model params = expected mean of 831; p-value = 0.467216)\n", " Completed in 1.0s\n", " 2*Delta(log(L)) = 834.062\n", " Iteration 4 took 1.1s\n", " \n", " --- Iterative MLGST: Iter 5 of 5 1282 operation sequences ---: \n", " --- Minimum Chi^2 GST ---\n", " Sum of Chi^2 = 1262.38 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.405131)\n", " Completed in 1.1s\n", " 2*Delta(log(L)) = 1263.06\n", " Iteration 5 took 1.2s\n", " \n", " Switching to ML objective (last iteration)\n", " --- MLGST ---\n", " Maximum log(L) = 631.5 below upper bound of -2.13633e+06\n", " 2*Delta(log(L)) = 1263 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.400337)\n", " Completed in 0.6s\n", " 2*Delta(log(L)) = 1263\n", " Final MLGST took 0.6s\n", " \n", " Iterative MLGST Total Time: 11.9s\n", " -- Performing 'single' gauge optimization on CPTP estimate --\n", " -- Performing 'Spam 0.001' gauge optimization on CPTP estimate --\n", " -- Performing 'Spam 0.001+v' gauge optimization on CPTP estimate --\n", "-- Std Practice: Iter 3 of 3 (Target) --: \n", " --- Circuit Creation ---\n", " 1282 sequences created\n", " Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing\n", " -- Performing 'single' gauge optimization on Target estimate --\n", " -- Performing 'Spam 0.001' gauge optimization on Target estimate --\n", " -- Performing 'Spam 0.001+v' gauge optimization on Target estimate --\n", "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", "*** Generating plots ***\n", "Statistical hypothesis tests did NOT find inconsistency between the datasets at 5.00% significance.\n", "The datasets are INCONSISTENT at 5.00% significance.\n", " - Details:\n", " - The aggregate log-likelihood ratio test is significant at 20.33 standard deviations.\n", " - The aggregate log-likelihood ratio test standard deviations signficance threshold is 1.98\n", " - The number of sequences with data that is inconsistent is 14\n", " - The maximum SSTVD over all sequences is 0.15\n", " - The maximum SSTVD was observed for Qubit * ---|Gx|-|Gi|-|Gi|-|Gi|-|Gi|---\n", "\n", "The datasets are INCONSISTENT at 5.00% significance.\n", " - Details:\n", " - The aggregate log-likelihood ratio test is significant at 20.33 standard deviations.\n", " - The aggregate log-likelihood ratio test standard deviations signficance threshold is 1.98\n", " - The number of sequences with data that is inconsistent is 14\n", " - The maximum SSTVD over all sequences is 0.15\n", " - The maximum SSTVD was observed for Qubit * ---|Gx|-|Gi|-|Gi|-|Gi|-|Gi|---\n", "\n", "Statistical hypothesis tests did NOT find inconsistency between the datasets at 5.00% significance.\n", "*** Merging into template file ***\n", "Output written to ../tutorial_files/exampleMultiDataSetReport directory\n", "Opening ../tutorial_files/exampleMultiDataSetReport/main.html...\n", "*** Report Generation Complete! Total time 202.698s ***\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#Make another dataset & estimates\n", "depol_gateset = target_model.depolarize(op_noise=0.1)\n", "datagen_gateset = depol_gateset.rotate((0.05,0,0.03))\n", "\n", "#Compute the sequences needed to perform Long Sequence GST on \n", "# this Model with sequences up to lenth 512\n", "circuit_list = pygsti.construction.make_lsgst_experiment_list(\n", " std1Q_XYI.target_model(), std1Q_XYI.prepStrs, std1Q_XYI.effectStrs,\n", " std1Q_XYI.germs, [1,2,4,8,16,32,64,128,256,512])\n", "ds2 = pygsti.construction.generate_fake_data(datagen_gateset, circuit_list, nSamples=1000,\n", " sampleError='binomial', seed=2018)\n", "results_std2 = pygsti.do_stdpractice_gst(ds2, target_model, fiducials, fiducials, germs,\n", " maxLengths, verbosity=3, modes=\"TP,CPTP,Target\",\n", " gaugeOptSuite=('single','toggleValidSpam'))\n", "\n", "pygsti.report.create_standard_report({'DS1': results_std, 'DS2': results_std2},\n", " \"../tutorial_files/exampleMultiDataSetReport\", \n", " title=\"Example Multi-Dataset Report\", \n", " auto_open=True, verbosity=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other cool `create_standard_report` options\n", "Finally, let us highlight a few of the additional arguments one can supply to `create_standard_report` that allows further control over what gets reported.\n", "\n", "- Setting the `link_to` argument to a tuple of `'pkl'`, `'tex'`, and/or `'pdf'` will create hyperlinks within the plots or below the tables of the HTML linking to Python pickle, LaTeX source, and PDF versions of the content, respectively. The Python pickle files for tables contain pickled pandas `DataFrame` objects, wheras those of plots contain ordinary Python dictionaries of the data that is plotted. Applies to HTML reports only.\n", "\n", "- Setting the `brevity` argument to an integer higher than $0$ (the default) will reduce the amount of information included in the report (for details on what is included for each value, see the doc string). Using `brevity > 0` will reduce the time required to create, and later load, the report, as well as the output file/folder size. This applies to both HTML and PDF reports.\n", "\n", "Below, we demonstrate both of these options in very brief (`brevity=4`) report with links to pickle and PDF files. Note that to generate the PDF files you must have `pdflatex` installed." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "*** Creating workspace ***\n", "*** Generating switchboard ***\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "Found standard clifford compilation from std1Q_XYI\n", "*** Generating tables ***\n", "*** Generating plots ***\n", "*** Merging into template file ***\n", "Output written to ../tutorial_files/exampleBriefReport directory\n", "Opening ../tutorial_files/exampleBriefReport/main.html...\n", "*** Report Generation Complete! Total time 144.395s ***\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pygsti.report.create_standard_report(results_std,\n", " \"../tutorial_files/exampleBriefReport\", \n", " title=\"Example Brief Report\", \n", " auto_open=True, verbosity=1,\n", " brevity=4, link_to=('pkl','pdf'))" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## Advanced Reports: `create_report_notebook`\n", "In addition to the standard HTML-page reports demonstrated above, pyGSTi is able to generate a Jupyter notebook containing the Python commands to create the figures and tables within a general report. This is facilitated\n", "by `Workspace` objects, which are factories for figures and tables (see previous tutorials). By calling `create_report_notebook`, all of the relevant `Workspace` initialization and calls are dumped to a new notebook file, which can be run (either fully or partially) by the user at their convenience. Creating such \"report notebooks\" has the advantage that the user may insert Python code amidst the figure and table generation calls to inspect or modify what is display in a highly customizable fashion. The chief disadvantages of report notebooks is that they require the user to 1) have a Jupyter server up and running and 2) to run the notebook before any figures are displayed.\n", "\n", "The line below demonstrates how to create a report notebook using `create_report_notebook`. Note that the argument list is very similar to `create_general_report`." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Report Notebook created as ../tutorial_files/exampleReport.ipynb\n" ] } ], "source": [ "pygsti.report.create_report_notebook(results, \"../tutorial_files/exampleReport.ipynb\", \n", " title=\"GST Example Report Notebook\", confidenceLevel=None,\n", " auto_open=True, connected=False, verbosity=3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Multi-qubit reports\n", "The dimension of the density matrix space with with more than 2 qubits starts to become quite large, and Models for 3+ qubits rarely allow every element of the operation process matrices to vary independently. As such, many of the figures generated by `create_standard_report` are both too unwieldy (displaying a $64 \\times 64$ grid of colored boxes for each operation) and not very helpful (you don't often care about what each element of an operation matrix is). For this purpose, we are developing a report that doesn't just dump out and analyze operation matrices as a whole, but looks at a `Model`'s structure to determine how best to report quantities. This \"n-qubit report\" is invoked using `pygsti.report.create_nqnoise_report`, and has similar arguments to `create_standard_report`. It is, however still under development, and while you're welcome to try it out, it may crash or not work in other weird ways." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" } }, "nbformat": 4, "nbformat_minor": 2 }