{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Reconstruction of a positive-real wavefunction\n", "\n", "In this tutorial, a walkthrough of how to reconstruct a **positive-real** wavefunction via training a *Restricted Boltzmann Machine* (RBM), the neural network behind qucumber, will be presented. The data used for training will be $\\sigma^{z}$ measurements from a one-dimensional transverse-field Ising model (TFIM) with 10 sites at its critical point.\n", "\n", "## Transverse-field Ising model\n", "The example dataset, located in *tfim1d_data.txt*, comprises of 10,000 $\\sigma^{z}$ measurements from a one-dimensional transverse-field Ising model (TFIM) with 10 sites at its critical point. The Hamiltonian for the transverse-field Ising model (TFIM) is given by\n", "\n", "\\begin{equation}\n", "\t\\mathcal{H} = -J\\sum_i \\sigma^z_i \\sigma^z_{i+1} - h \\sum_i\n", "\\sigma^x_i\n", "\\end{equation}\n", "\n", "where $\\sigma^{z}_i$ is the conventional spin-1/2 Pauli operator on site $i$. At the critical point, $J=h=1$. As per convention, spins are represented in binary notation with zero and one denoting spin-down and spin-up, respectively.\n", "\n", "## Using qucumber to reconstruct the wavefunction\n", "\n", "### Imports\n", "To begin the tutorial, first import the required Python packages." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "attributes": { "classes": [], "id": "", "n": "14" } }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "from qucumber.nn_states import PositiveWaveFunction\n", "from qucumber.callbacks import MetricEvaluator\n", "\n", "import qucumber.utils.training_statistics as ts\n", "import qucumber.utils.data as data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Python class *PositiveWaveFunction* contains generic properties of a RBM meant to reconstruct a positive-real wavefunction, the most notable one being the gradient function required for stochastic gradient descent.\n", "\n", "To instantiate a *PositiveWaveFunction* object, one needs to specify the number of visible and hidden units in the RBM. The number of visible units, *num_visible*, is given by the size of the physical system, i.e. the number of spins or qubits (10 in this case), while the number of hidden units, *num_hidden*, can be varied to change the expressiveness of the neural network.\n", "\n", "**Note:** The optimal *num_hidden* : *num_visible* ratio will depend on the system. For the TFIM, having this ratio be equal to 1 leads to good results with reasonable computational effort.\n", "\n", "### Training\n", "To evaluate the training in real time, the fidelity between the true ground-state wavefunction of the system and the wavefunction that qucumber reconstructs, $\\vert\\langle\\psi\\vert\\psi_{RBM}\\rangle\\vert^2$, will be calculated along with the Kullback-Leibler (KL) divergence (the RBM's cost function). It will also be shown that any custom function can be used to evaluate the training.\n", "\n", "First, the training data and the true wavefunction of this system must be loaded using the *data* utility." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "attributes": { "classes": [], "id": "", "n": "2" } }, "outputs": [], "source": [ "psi_path = \"tfim1d_psi.txt\"\n", "train_path = \"tfim1d_data.txt\"\n", "train_data, true_psi = data.load_data(train_path, psi_path)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As previously mentioned, to instantiate a *PositiveWaveFunction* object, one needs to specify the number of visible and hidden units in the RBM. These two quantities equal will be kept equal." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "nv = train_data.shape[-1]\n", "nh = nv\n", "\n", "nn_state = PositiveWaveFunction(num_visible=nv, num_hidden=nh)\n", "# nn_state = PositiveWaveFunction(num_visible=nv, num_hidden=nh, gpu = False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By default, qucumber will attempt to run on a GPU if one is available (if one is not available, qucumber will default to CPU). If one wishes to run qucumber on a CPU, add the flag \"gpu = False\" in the *PositiveWaveFunction* object instantiation (i.e. uncomment the line above). \n", "\n", "Now the hyperparameters of the training process can be specified.\n", "\n", "1. **epochs**: the total number of training cycles that will be performed (default = 100)\n", "2. **pos_batch_size**: the number of data points used in the positive phase of the gradient (default = 100)\n", "3. **neg_batch_size**: the number of data points used in the negative phase of the gradient (default = *pos_batch_size*)\n", "4. **k**: the number of contrastive divergence steps (default = 1)\n", "5. **lr**: the learning rate (default = 0.001)\n", "\n", " **Note:** For more information on the hyperparameters above, it is strongly encouraged that the user to read through the brief, but thorough theory document on RBMs located in the qucumber documentation. One does not have to specify these hyperparameters, as their default values will be used without the user overwriting them. It is recommended to keep with the default values until the user has a stronger grasp on what these hyperparameters mean. The quality and the computational efficiency of the training will highly depend on the choice of hyperparameters. As such, playing around with the hyperparameters is almost always necessary. \n", " \n", "For the TFIM with 10 sites, the following hyperparameters give excellent results." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "epochs = 500\n", "pbs = 100 # pos_batch_size\n", "nbs = 200 # neg_batch_size\n", "lr = 0.01\n", "k = 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For evaluating the training in real time, the *MetricEvaluator* will be called in order to calculate the training evaluators every 100 epochs. The *MetricEvaluator* requires the following arguments.\n", "\n", "1. **log_every**: the frequency of the training evaluators being calculated is controlled by the *log_every* argument (e.g. *log_every* = 200 means that the *MetricEvaluator* will update the user every 200 epochs)\n", "2. A dictionary of functions you would like to reference to evaluate the training (arguments required for these functions are keyword arguments placed after the dictionary)\n", "\n", "The following additional arguments are needed to calculate the fidelity and KL divergence in the *training_statistics* utility.\n", "\n", "- **target_psi**: the true wavefunction of the system\n", "- **space**: the hilbert space of the system\n", "\n", "The training evaluators can be printed out via the *verbose=True* statement.\n", "\n", "Although the fidelity and KL divergence are excellent training evaluators, they are not practical to calculate in most cases; the user may not have access to the target wavefunction of the system, nor may generating the hilbert space of the system be computationally feasible. However, evaluating the training in real time is extremely convenient. \n", "\n", "Any custom function that the user would like to use to evaluate the training can be given to the *MetricEvaluator*, thus avoiding having to calculate fidelity and/or KL divergence. Any custom function given to *MetricEvaluator* must take the neural-network state (in this case, the *PositiveWaveFunction* object) and keyword arguments. As an example, the function to be passed to the *MetricEvaluator* will be the fifth coefficient of the reconstructed wavefunction multiplied by a parameter, *A*." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def psi_coefficient(nn_state, space, A, **kwargs):\n", " norm = nn_state.compute_normalization(space).sqrt_()\n", " return A * nn_state.psi(space)[0][4] / norm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now the hilbert space of the system can be generated for the fidelity and KL divergence and the dictionary of functions the user would like to compute every \"*log_every*\" epochs can be given to the *MetricEvaluator*." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "attributes": { "classes": [], "id": "", "n": "2" } }, "outputs": [], "source": [ "log_every = 10\n", "space = nn_state.generate_hilbert_space(nv)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now the training can begin. The *PositiveWaveFunction* object has a property called *fit* which takes care of this. *MetricEvaluator* must be passed to the *fit* function in a list (*callbacks*)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch: 10\tFidelity = 0.524441\tKL = 1.311481\tA_Ψrbm_5 = 0.102333\n", "Epoch: 20\tFidelity = 0.627167\tKL = 0.887134\tA_Ψrbm_5 = 0.151670\n", "Epoch: 30\tFidelity = 0.733927\tKL = 0.582645\tA_Ψrbm_5 = 0.194329\n", "Epoch: 40\tFidelity = 0.794879\tKL = 0.445741\tA_Ψrbm_5 = 0.221883\n", "Epoch: 50\tFidelity = 0.829248\tKL = 0.363647\tA_Ψrbm_5 = 0.232239\n", "Epoch: 60\tFidelity = 0.860589\tKL = 0.287518\tA_Ψrbm_5 = 0.241004\n", "Epoch: 70\tFidelity = 0.886160\tKL = 0.231527\tA_Ψrbm_5 = 0.244122\n", "Epoch: 80\tFidelity = 0.902777\tKL = 0.196992\tA_Ψrbm_5 = 0.234641\n", "Epoch: 90\tFidelity = 0.914448\tKL = 0.174226\tA_Ψrbm_5 = 0.231594\n", "Epoch: 100\tFidelity = 0.923648\tKL = 0.156510\tA_Ψrbm_5 = 0.234137\n", "Epoch: 110\tFidelity = 0.929855\tKL = 0.142626\tA_Ψrbm_5 = 0.220506\n", "Epoch: 120\tFidelity = 0.937082\tKL = 0.127953\tA_Ψrbm_5 = 0.228048\n", "Epoch: 130\tFidelity = 0.943320\tKL = 0.114683\tA_Ψrbm_5 = 0.225533\n", "Epoch: 140\tFidelity = 0.948913\tKL = 0.102805\tA_Ψrbm_5 = 0.220003\n", "Epoch: 150\tFidelity = 0.953720\tKL = 0.092966\tA_Ψrbm_5 = 0.219529\n", "Epoch: 160\tFidelity = 0.957696\tKL = 0.085269\tA_Ψrbm_5 = 0.219721\n", "Epoch: 170\tFidelity = 0.960716\tKL = 0.079273\tA_Ψrbm_5 = 0.215919\n", "Epoch: 180\tFidelity = 0.963032\tKL = 0.075418\tA_Ψrbm_5 = 0.219223\n", "Epoch: 190\tFidelity = 0.965285\tKL = 0.071062\tA_Ψrbm_5 = 0.217072\n", "Epoch: 200\tFidelity = 0.966294\tKL = 0.069517\tA_Ψrbm_5 = 0.218791\n", "Epoch: 210\tFidelity = 0.968279\tKL = 0.065436\tA_Ψrbm_5 = 0.214237\n", "Epoch: 220\tFidelity = 0.969002\tKL = 0.063958\tA_Ψrbm_5 = 0.208316\n", "Epoch: 230\tFidelity = 0.970735\tKL = 0.060499\tA_Ψrbm_5 = 0.211827\n", "Epoch: 240\tFidelity = 0.971954\tKL = 0.058173\tA_Ψrbm_5 = 0.213458\n", "Epoch: 250\tFidelity = 0.972797\tKL = 0.056356\tA_Ψrbm_5 = 0.216414\n", "Epoch: 260\tFidelity = 0.973940\tKL = 0.054098\tA_Ψrbm_5 = 0.219072\n", "Epoch: 270\tFidelity = 0.975173\tKL = 0.051311\tA_Ψrbm_5 = 0.213439\n", "Epoch: 280\tFidelity = 0.976146\tKL = 0.049353\tA_Ψrbm_5 = 0.214791\n", "Epoch: 290\tFidelity = 0.977626\tKL = 0.046184\tA_Ψrbm_5 = 0.215294\n", "Epoch: 300\tFidelity = 0.978880\tKL = 0.043539\tA_Ψrbm_5 = 0.215247\n", "Epoch: 310\tFidelity = 0.979931\tKL = 0.041293\tA_Ψrbm_5 = 0.211467\n", "Epoch: 320\tFidelity = 0.981140\tKL = 0.038849\tA_Ψrbm_5 = 0.213601\n", "Epoch: 330\tFidelity = 0.982012\tKL = 0.036976\tA_Ψrbm_5 = 0.216033\n", "Epoch: 340\tFidelity = 0.982764\tKL = 0.035460\tA_Ψrbm_5 = 0.217036\n", "Epoch: 350\tFidelity = 0.983499\tKL = 0.033983\tA_Ψrbm_5 = 0.208566\n", "Epoch: 360\tFidelity = 0.984789\tKL = 0.031407\tA_Ψrbm_5 = 0.218186\n", "Epoch: 370\tFidelity = 0.985142\tKL = 0.030643\tA_Ψrbm_5 = 0.215245\n", "Epoch: 380\tFidelity = 0.985985\tKL = 0.028931\tA_Ψrbm_5 = 0.217562\n", "Epoch: 390\tFidelity = 0.986345\tKL = 0.028262\tA_Ψrbm_5 = 0.217989\n", "Epoch: 400\tFidelity = 0.986798\tKL = 0.027449\tA_Ψrbm_5 = 0.215068\n", "Epoch: 410\tFidelity = 0.987459\tKL = 0.026076\tA_Ψrbm_5 = 0.220650\n", "Epoch: 420\tFidelity = 0.987785\tKL = 0.025427\tA_Ψrbm_5 = 0.220902\n", "Epoch: 430\tFidelity = 0.988085\tKL = 0.024916\tA_Ψrbm_5 = 0.217657\n", "Epoch: 440\tFidelity = 0.988270\tKL = 0.024565\tA_Ψrbm_5 = 0.218701\n", "Epoch: 450\tFidelity = 0.988164\tKL = 0.024811\tA_Ψrbm_5 = 0.222711\n", "Epoch: 460\tFidelity = 0.988564\tKL = 0.024018\tA_Ψrbm_5 = 0.212042\n", "Epoch: 470\tFidelity = 0.988859\tKL = 0.023432\tA_Ψrbm_5 = 0.221610\n", "Epoch: 480\tFidelity = 0.989148\tKL = 0.022804\tA_Ψrbm_5 = 0.224286\n", "Epoch: 490\tFidelity = 0.989477\tKL = 0.022194\tA_Ψrbm_5 = 0.223508\n", "Epoch: 500\tFidelity = 0.989738\tKL = 0.021626\tA_Ψrbm_5 = 0.223838\n" ] } ], "source": [ "callbacks = [\n", " MetricEvaluator(\n", " log_every,\n", " {\"Fidelity\": ts.fidelity, \"KL\": ts.KL, \"A_Ψrbm_5\": psi_coefficient},\n", " target_psi=true_psi,\n", " verbose=True,\n", " space=space,\n", " A=3.,\n", " )\n", "]\n", "\n", "nn_state.fit(\n", " train_data,\n", " epochs=epochs,\n", " pos_batch_size=pbs,\n", " neg_batch_size=nbs,\n", " lr=lr,\n", " k=k,\n", " callbacks=callbacks,\n", ")\n", "# nn_state.fit(train_data, callbacks=callbacks)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All of these training evaluators can be accessed after the training has completed, as well. The code below shows this, along with plots of each training evaluator versus the training cycle number (epoch)." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "fidelities = callbacks[0].Fidelity\n", "KLs = callbacks[0].KL\n", "coeffs = callbacks[0].A_Ψrbm_5\n", "# Please note that the key given to the *MetricEvaluator* must be what comes after callbacks[0].\n", "epoch = np.arange(log_every, epochs + 1, log_every)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "# Some parameters to make the plots look nice\n", "params = {'text.usetex': True,\n", " 'font.family': 'serif',\n", " 'legend.fontsize': 14,\n", " 'figure.figsize': (10, 3),\n", " 'axes.labelsize': 16,\n", " 'xtick.labelsize':14,\n", " 'ytick.labelsize':14,\n", " 'lines.linewidth':2,\n", " 'lines.markeredgewidth': 0.8,\n", " 'lines.markersize': 5,\n", " 'lines.marker': \"o\",\n", " \"patch.edgecolor\": \"black\"\n", " }\n", "plt.rcParams.update(params)\n", "plt.style.use('seaborn-deep')" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+gAAADQCAYAAACUYku5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3X+QHHWa3/nPwzD2bgc/Ggk0O2uvkFqsZ1h6Laq6h1iLG4RRS2N27H+EBJ6InX9u1ZKGMYHtWCQw3nCcfbMgNCaYOKM5tVrYxn+BxDrC4b2ImRacBAEbAa0qtNMs47PVzOzas2uxalqIbfvC533uj8xqZWdnVWVVV1VmVr1fEQqpsrKyv1lV/VU++X2e79fcXQAAAAAAIFvXZd0AAAAAAABAgA4AAAAAQC4QoAMAAAAAkAME6AAAAAAA5AABOgAAAAAAOUCADgAAAABADhCgAwAAAACQAwToAAAAAADkAAE6AAAACs3MRszskJlNhH8PN9i3bGb7w/1OmdlI5LkjZuZm9omZzUSfA4BeuD7rBgAAAABrdMrdxyTJzGYlnZC0N75TGLiPu/tU+HhC0oykLeEu77m79abJALBaZgG6mc24+84m+4xI2iOpIqksacrdF5sd+9Zbb/VNmzZ1pJ0A+tv58+f/zN1vy7odWaPfBJBW3vpNMytLWqg9dvfFMPBOMiLpsKSp8PGspBEzG05zjRlH3wkgrbR9Z88D9LDDHJFUr+OMSnU3NG7Tpk2anZ1dUzsBDAYz+2nWbcgD+k0AaeWw3xyRFA+uF8xsxN3noxvdvWJmY5FN45IWI8H5iJntCY+3U9Iz8cDdzPZL2i9JGzdupO8EkEravrPnAbq7n5GCGp9G+7V4NxQAAACDaV2d7Yl16LGA+4Ckycjj5WxNM1uQdEpBoB59/ZTCEfjx8XFvs80AkCjPk8TVvRuatHM42cesmc1+/PHH3W8dgExVq1WdPHlS1Wq14TZ0B+81gBxZ0OpgvF7QviwcCX/F3U/XtkWDd3evKF3GZ67RXwPFkudJ4lq9G8rdTCCmWq2qUqmoXC6rVCr1zbZ9Bx7V2XcuaGjDqJZeeEn3b9sqSau2TR8/1vX3eBBNPrpf56pv68Y71+vq9PPaXrpXJ45NNX8hAHTHfNLGMMBOFGZlztcyO8NtZUknauWV/eDb+/Zp7tybGhu6QS8vfabR7ffpxenprJsFoIE8B+ht3Q0FiiyLILZo20b/2i9q7v/5mb688wmZmdwf1A/+3T/R9Z//Od314FPL287OHFW1Wl1+f9AZ1WpV56pva+vjXw3e612uc997i/caQGbCuvLla8Qw2/JM7PFCJHW9HD6uhI/3hKPo85KOR143IWl5dL1oqtWq5s69qe/cOSoz0253PX3uTfprIOfyHKC3fDcUyIM8BNRpg9gibnvnld/SL3z5AZkFq+CYmfzzw7rhi19asW1ow6gqlQoXIR1WqVR0453rV7zXN965nvcaQNYmzeyQrq38E60rP6JgKbWpMFh/XdJwrR9TcM15OpzvaD5MfZeCpdeix8m96LVFpVLR2NANK/rrsaEb6K+BnMtVgB69w9nsbijQTb0OsjsdUKcNYou47ea/slWf/HFFG7f+eni+Lvsfi/rsTy7I7762benSnMrlg135fgyycrmsq9PPy3f58nt99cPLKj9WzrppAAZYOIBTG8Q5E3tub+Tf85JuaXCcwl5rxtPZv3DXr+i/Ln2m3X6tvz6/9Jm+Waa/BvIsi2XWygom3BgOZ3KfiXSGy3c4w8eN7oYCDRUpyO50QJ02iC3iNv9vf6qv/PU79AczR4P3/tKcvrbj3uDziGy7f9tWRgi6oFQqaXvpXp373ltBDfqHl7W9dC/vNQBkKDGd/YM5/eKvjurpH81pbOgGnQ9r0OmvgXzLYpm12h3O5xKe21tnX4nR84E0KEF2pwPqtEFsEbfVJn+79pkfTPgeHOQCpItOHJu69l4/Vua9BoCM1Utnv+vrX1f5t39blUpF3yzTXwNFkKsUd/QnguzeB9StBrFF3FYqlVZdaCRtQ3fwXgNAfpTLZb1cJ52d/hooFnPvvxXJxsfHfXZ2NutmDKSGQXYYOEpqum11kO26sCqgTt72bhhk337316+16/96Xrd88UvaVGq87cdvvaz/vvhTbf3b/7jln/vjmaP661/6Jf3Bf/jjls93dUDdmeXJ0JyZnXf38azbkTX6TQBp0W9ek6e+M1qDfp4l1YDcSdt3MoKOtjUb8R60kexOjFqnHRVm9BgAAES9OD29fG3RLJ2dm/pAfhGgI5V2gnGC7GsIqAEAQLelubaIz/beaKSdQB7oPQJ0rNKpYJwgGwAAID8SZ3s/96aq1eqq65lWAnkAnUOAPuC6GYwTZAMAAHRHO6Pb9WZ7r1Qqy8+Xw3XS0wbyADqLAH2AdTsYJ8gGAADovHZHt+vN9v4nv/d7evnZI8vHu+4XvlA3kOd6C+guAvQBUwuCh4aGdPadCz0JxgmyAQAAOqOVNPW4Uqmk0e336enIbO9fuOtX9LMfza043uPvz+rdz12fuGwbgO4iQB8g0RHzP/3x69rwpb/Zk2AcAAAAndEoTT3NNVh8tvdKpaIPPvqjFcfbfsutmhu+SU9/OLdi2Tau8YDuI0DvY9HaJEkrRsxv/uKX9OG5f6lNd3+dYBwAAKAg6qWptzK6Hb+GSzreC//qpCSxbBvQYwTofSpeX/7FdX9JQxtGl++ODn9hi6773PW68O//d93yS2ME4wAAAAWQlKa+ltHtZsfr5LJtAJojQO8TjUbL3R/UhX/3T3T959+Xb31w+e7oTTcO6Z/9o7+npaUlgnEAAICCiKepr/W6rd3jNaqHl8SoOtAGAvQ+0Gy03Mx0y+2/plv+Yl4/jqWvf+Mb38i49QAAAGhVpwdU2jlevXr4w3//H+j//dnPGFVH4eShXIMAvYDaGS1fujSnf/3yv5CkVenrAHrHzG6SNCFp0d3fMLO73f39rNsFAECrkurhz33yZ/r5Txf17K/ezRrqKJR65RpJQXs3A3kC9IJZy2h52loiAN1hZjskHZdUkfSupDckfWRmu939dzNtHAAgt/IwqpckqX79pttvV+nTP2cNdWSmnd+XeuUa39i9Wz/70dyKoF1SV+ddIEDPOUbLgb5Sdvc7pOVgXe5+xWpXMQAAxOR9ErZ4/bok/f2HH2ENdaxJuzelWvl9if6MpHKNX7Lr9J9mZ/Xdu8eXg/bHf/hD/fznru9qhggBeo4xWg70nYt1tntPWwEAKIRGk7BldY2XFDjF69frzQqf10wAZCv+vWj3plQrvy/xn/GFu35F/zVWrvHulU/0t3/hF1fEXuvddNeNN3U1Q4QAPaeq1Sqj5UD/ucfMzrv7TxUG5Wa2SdI9kkhxBwCsUG8StqzSxdMGTkmzwnejvrfT+3UaNySaSwyUP/jDtm5Kpf19SQzkP5jTL/7qqJ4O09nPL32mu//G39D5D/5wRdB+2VzvXf1UD3UxQ4QAPacqlQqj5UD/eUbSeTNzSYuRzPYd2TUJAJBXSZOwZZUu3upofnRUvRv1vWlvFmRVIpD30oS01nKTodlrk74X/+vvv71q1LrRTanoz0j7+1IvkL/r619X+bd/e9WNpWg2yK/t2iVJiRkinUKAniPxL9jSCy/JndFyoF+4+xVJd5jZHkmbJVXc/fWMmwUAyKmkSdg6HQyktZbR/E7X96Zdf11SJiUCeSxNiEsTeLdbz502TT3pe3HPzbfonU8ua/ftm1cF2Wl+Rprfl0aBfLxcIykbJHq+3+xCdgQBek7E683v37ZV92/bqrOMlgN9JVxW7XTsMcusAQAS1QsQem0to/lJr221vle6FninXX/9ul/4Qs9KBJpNOJZlaUI7wfOa67lTpKknfS/+2P9Cm8fHV6Saj26/T9MvvpjqZ7zw6ivSt7/d8Pel1Rtf8aC93rZOIUDPgaR687MzR3Xq5X+hbx9ktBzoF2b2kKQpMxtz95+Em6+Y2QPu/kaGTQMA5Fg3g4FW2tDuaH7Sa1up7/2T3/s9vfzskYYTeiWtv/74+7N693PXd71EIM2EY/V+btoa/E7Nap42eG50k6H2fL0shUZp6tHX1vtO1VsZIO3P+M3f/M2m71FebnwlIUDPgaR686ENo6m/YAAKY9Hd10c3uPtHZsYvOQAg99YS1NSbOK5Zfe8X7voV/exHc00n9Epaf337LbdqbvgmPf3h3KqbCp2qrZYS0ugT2pd0MyNpJLt2vHbX3W7WtrQ13vUyJuI3S5KyFOqlqcdfmxSMJ60McPLkyZZS4dPKw42vJAToGUlTb14uH8y6mQA66+Y629f1tBUAALRpLUFNO/W9lUpFH3z0R00n9JKS119/4V+dlKRUM8qnEX9tvTT6pAnHouc2NDS0KnhOqsFvZd3tNG1LG9gmjW4n3SxJylJISlNPvNESOY9G36lWUuHzGHC3igA9A63WmwPoG/eY2Xy05tzMHpA0Jql4U7sCALBGaep7007o1SgFv9mM8vUmcGs2It0ojT7evmgA/e//9L/o17/wxaY1+Gnr8tO2rZXANn4DJelmSb0shTSvTVuXnzYVPo9L7bWDAL3HqDcHBpe7P2lmPzSzMUnzkkbCv1Mvs2ZmM+6+s8k+I5L2SKpIKkuacvfF9lsOAEA2Wql9TxOwtTKBW5oR6UZp9FHxGwOjw8P6P/7wR9qzaaRhDX7auvxW2tZKYJvmZklSlkLa16ZNSU+TCl9P0Za8I0DvMerNgcHm7rvMrKxg1HzW3atpXmdmEwoC+okUu59y97HwdbOSTkja22aTAQDI1FoCyri0s9EnjbTXGy2vF6BGxW8MfGn4Fn3ertNvvT+rbbfcWrcGP21dfqtta6dUodnNkmafy1qXDGynzUVY8i6OAL3HqDcH4O4VBaPbkiQz2xSZ1b3ea86E+x5ptF8Y/C9EXrcYBvcAABRWpyb0ahQoNlsqrdloeas3Bn7+5pt18J/+b1paWmq6xna7qeatLNGcJg280xMFdlvelrxLgwC9x0qlEvXmwIAzs5timw5L+laHDj8iKZ7OvmBmI+4+36GfAQBAYdWbUT7NUmlpRsuT1Lsx8I1vfCNx33bq8tttm9RaGngnJwrstrQZE3lCgJ6B6ePHIneoqDcHBoWZTUo6LskVBNEmaTh83KkAvd6M8MMJ7dkvab8kbdy4sUM/HgCA/IsGiolp0E2WSmvn+r1TI8hrSTVPUsQ08LQ6kVrfawToGcnrunsAumqLu18X32hmz3bwZyxodTCeGLS7+5SkKUkaHx/3DrYBAIDCqJcGXW+ptLXoVAzQyXTxIqaBtyKL1Pq1IEDvgSJN6w+gq2bqbH+mgz8jMY09rHsHAAAxjdKg8zyo1qm2FTENvFV5/hzjCNC7LGnN8+njx7JuFoBseJ0J4SYlfbfdg4bLqi24+6K7V8xsXey5M+0eGwA6wcx2K5gjox3z7v67nWwPEFXENOhOGvTzzxsC9C6qt+Z5P9RzAGjLQUmlMIWsNtK9XlJJTQL0cHb2CUnD4UzuM7WZ3SUdUTA6PxU+njSzQ7q2DvpkJ08CANqwU8GEmNbGa5+VRICOripaGnSnDfr55wkBehc1WvOcLz0wkEYkPamVs6ybpEPNXhhZmu25hOf21tlXYvQcQD5U3P3Tdl5oZpTooCeKlAbdDYN+/nlBgN5FrHkOIGbS3avxjWZ2OYvGAECvuPuJLF4LAEVDgN5FrHkOIMrdq+Ea6BOSFt39DTO7OyloB4B+kzQHR7j85GFJmxVk/vyOu//bNo49ImmPrpX2TLn7Yp19y5LGFax48RVJh919vtXjAEA3EKB3GWueA6gxsx0K1kGvSHpX0huSPjKz3UyABGAA7DWzi7X+zsyekLRFQYC+qKAM6FtmZm30iafcfSw87qykE5L2xncys2FJ4+EykzKzCQVzeGxp5TgA0C0E6D1APQeAUNnd75CWg3W5+xWrTVRRYCwnCSCFE5Iu6tqEb4vuHq37e13SCTN7Ri1MCheOiC/UHrv7Yhh4JxlRcEOgNqnmrKSRMHAfSXMcM9svab8kbdy4MW0zASCV69p5UdhxAgBac7HOdu9pKzps8tH9euTR39Cxt/+1Hnn0NzT56P6smwQgn25RsBLFpvBxvfk35utsr2dEKyfflKSFMF19hXASzbHIpnEFNwpqI/hNj+PuU+4+7u7jt912W4tNBYDG2grQJR0ws31hLSUiqtWqTp48qWqVklIAq9xjZreH/3YpqMmUdE9WDVqrarWqc9W3tfXxr2rL1+7S1se/qnPVt+kDASTZE/5dG6VeV7uWjF1TDrd43HV1ticeJ1ZTfkDXlqJs6TgA0A3tprjvdffXzWxHmBJ00d3f72TDimjfgUd19p0LwYRwL7yk+7dt1fTxY1k3C0B+PCPpvJm5pMVIZvuO7Jq0NpVKRTfeuX7FcpI33rme5SQBJJmXdFDSw5HSnnWSPlXQJ9aWnJxKenEDC1odRNcLtpeFqeqvuPvptRyn0ygZAgZbWwG6u78e/dvM7jazHyiYWGO6g+0rjGq1qrPvXNCXdz4RLqn2oM7OHFW1WqVzBSApqDeXdIeZ7VE4Y3GtHy2qcrmsq9PPy3f58nKSVz+8rPJj5aybBiBn3P21Bk/fEu5zpY1DJ6bEh+nsicLa8nl3P7OW43Tat/ft09y5NzU2dINeXvpMo9vv04vTA3lpDQysdmvQ7679bWavKpyJWMHI0ENmtruDbSyESqWioQ2jK0aRhjaMqlLpWZ8OIOfMbJ8kuftpdz8aDc7NbFOtby2SUqmk7aV7deF7b+niDz7Qhe+9pe2le7kxCWCFZvMXufuVesG5mX2/yWsriox0hzXjZ6KPw4zP2uOypIVacB7eNG16nG6rVquaO/emvnPnqB7atFnfuXNUc+fepGQIGDDtprifDlM0Jek5d3848lxVCi5EB2k0vVwua+mFl+T+4PIo0tKlOZXLB5u/GMCgSJyt3cwekvSUpPfMbKZoS66dODZ1LSXzMVIyASTaUrtJ2SKTtD7FfpNhinxt/fLJyHNHFCylNhUG3a8rmKyu9vy8pNMpjtNVlUpFY0M3rBjsGRu6gZIhYMCsZZm1g/VSM83sgTUct5BKpZLu37ZVZ2eOBjXol+Z0/7atdKgAVjCzHyqYQfhVd/9WuHm/pH3u/n64LnDhsJwkgCYm1X4996vNdghHv2tpi2diz+2N/HteYTp9q8fptnK5rJeXPtNuv1YydH7pM32zTMkQMEjaDdAPN6mbHJN0vs1jF9b08WORiT0OcrEKIG6LgpGceUllM9sdjpZv0bXax/gSPwBQeGH6ejv15QOjVCppdPt9ejqsQT8f1qBzPQkMlnYniVsxyYeZ3R2dxd3dj661YUXFKBKABt6N3Nz8KDJfh7v7p7V/Z9AuAEAOvDg9vTzY801mcQcGUlsBekJ9+RUz2yFp8yDVnQNAi+4xs/OSPpE0oWvpnuvN7EZ3vyrW2wUwQMxsk7v/JOt25AmDPcBga2sWd8UuIN39o1aWCgpn0zxkZhPh33UvSM2sbGb7zWxPuO9Im20GgKwdl/SaggD9EUmfmNmzkg5L+kfhiDopoAAGyWEzuzHrRgBAXqQeQTezSQW15bcoqJ38SmyXEUmzktKMoJ9y97HwuLOSTkjaW2ffCXd/LtKO45IOpG13N12rNycFCUBz7v6RpPHY5tckycxKksbd/UTPGwYA2dki6Y1wsKaiYLb1M+7+k3gJJQAMgtQBenjReMLM/k8FS6nFZ9Scd/emCzXW1p6MHHfRzCYavOSAmU25e8OJk8xsv4KZkLVx48ZmzVizfQce1dl3LgQztr/wku7ftlXTx491/ecCKK5whHw+6YIz7D9Z7BbAQHH3XeENykUFGZoTurYcmkv65SzbBwC91k4N+mEFo9qvNd0z2YhWz1K8YGYj4dIXcUcUTKZ0OPLzV3H3KUlTkjQ+Pt7VSZaq1arOvnNBX975RLgMxoM6O3NU1WqVkXQAjfxdSb8T32hmN0UmiQOAgeLuVTO7WUGAPlWbbDgsAQKAgdJyDbq7X6kXnJvZvhSHqLcGZmIdehh4P6Mgrf1Ag9f3TKVS0dCGUZmZJMnMNLRhVJVKpckrAQy4V3RtObWo/b1uCADkSXh9WZW0zszuDjc/k2WbACALqQJ0M/t+rbM0s5KZ/SDhzw8VjHY3s6DVwXjdoNvMDrn7c2HN+nEFtUmZKpfLWro0J/dgoN7dtXRpTuVyOeOWAci5nZLOh33mK+GfVyU9lXXDACAPwrk6rpjZb0nanHV7AKDX0qa4W+zxFQXBcnyfQymOlTR6JHdfNfwc1qZXIvtMmdkWMysn7d8rpVJJ92/bqrMzR4Ma9Etzun/bVtLbATQzLuk5RebhCLG0GoCBFAbiWxSUQK5XUHc+L+mjcBuTxAEYKKkCdHc/GHk4L+lweIdzBTO7nOJYFTNbHjEPJwE5E3u8EE4Kt6BgspAz8WOkaXc3TR8/FpnF/SDBOYA0DictSWlmiTcuAWAA/F0F13lPpplsGAD6XcuTxLn7FdVfp3eH0s1CPGlmhxSMjpclTUaeO6IgjX0qDOZHwhnapWCU6ZVW29wtpVKJwBxAau7+upndpODG46K7vxEuI8RFKYBBNRlOErfZzB6IbP9E0sPuTgkQgIGSKkAP10Df02w3Beukf7fZ8cIR8NooeHx0fG/s8ek0bQSAvDOzHQrKgyqS3pX0hoJVKna7++9m2jgAyEDtBmWYmbmcnRnO6j6WVbsAICtpR9C3SDqtOvXjobQ16AAwqMrufoe0HKzL3a9YbUkIAICk5b7xQNbtAIBeSxugv5ImBTNNDToADLCLdbZ7T1sBADliZg+4+xvx7UnzHQFAv0u1zFpScG5mN5nZ7lq9EHWUANDUPWZ2e/hvlyQz2yTpnqwaBABZCpeafNLMfhA+fiJcgnJ3xk0DgEy0PEmcRB0lALTpWUmzZuaSFiOZ7TuyaxIAZOo9dz8aTgr8jIJyylclHTQzcV0JYNC0FaCLOkoAaFm4fOQdZrZH0mZJlaRl1wBggFyUJHefN7NZd38t3P6amT0riQAdwEBpN0CnjhIAWmRm33f3b7E6BQAsG4n8O359+V4vGwIAeZCqBj0BdZQA0LpHzGxfuBY6ACBIZf+PZvaKgj5yU+Q5Bn4ADJx2R9CfkXSeOkoAaMled3/dzHaY2Yiki0kzFwPAANnr7lUz2yxpQsGEcROSPpG0IFLcAQyYtgJ0d7+iAaujrFarqlQqKpfLKpVKWTcHQAHV+snw79fNbHM4c/Epd5/OtnUA0Hu1FYDCJdVOhH9kZsMKJtYEgIHS7gi6JCleR2lmN7n7p2trUv7sO/Cozr5zQUMbRrX0wku6f9tWTR8/lnWzABRMuBzl+2Z2t6SDkh6WdEbSbLYtA4Ds1Ll+dEnfz6I9AJCldmvQ6znR4eNlrlqt6uw7F/TlnU/o9rt/XV/e+YTOvnNB1SpLvgNo2Wkze0/SKUn/SdJmd3/Y3d/PuF1dUa1WdfLkSfpLAHWZ2aSkn4R16Mvzc4TZmluyaxkAZKPpCLqZ3aygDmggJ+qoVCoa2jCqWp29mWlow6gqlQqp7gDa8WS/lwRJ0uSj+3Wu+rZuvHO9rk4/r+2le3Xi2FTWzQKQP8Puvi6sO3/KzC5K2itpMXyeGnQAA6XpCHp4B/O0u3+u9kfSLkl3xLZ9RUG6Zl8pl8taujQn9+D+hLtr6dKcyuVyxi0DUECHByE4r1arOld9W1sf/6q2fO0ubX38qzpXfZuRdABJFiXJ3c9IukVByc+TCsp/nsywXQCQibQ16JOxxzeHk3ksc/dKOKFHXymVSrp/21adnTka1KBfmtP927Yyeg6gZe7+WtJ2M9vXT5PEVSoV3Xjn+hWZRzfeuZ7MIwBJbo78+1Sk5Ic7egAGUqoAPRxFT6PvAnRJmj5+LDKL+0EuMAGkYmbfl3Q8nBiupOQZiU3SmKS+CdDL5bKuTj8v3+UyM7m7rn54WeXHyDwCsMpBMzsgqSJp3swuuvtPJMnMNtX+DQCDot1Z3LfUZiOubQhnJb5HfVorVCqVCMwBtMpij69IOp6wz6HeNKc3SqWStpfu1bnvvRXUoH94WdtL99KHAkgSXwf9ufCG5hUF8x99JdPWAUCPtbsO+lEz+2HYmS5KGpE0L2lHJxsHAEXm7gcjD+cV1KB/FN/PzC73rlW9ceLY1LXMo8fKBOcAErEOOgCs1PY66O6+y8zKClIzZ2sdLABgtbBUKLFcqF/7TzKPALTL3RfN7EjW7QCAXlvTOujuXnH3E7WLSzPb15lmAUD/MLNNZvZKuM7v/zSzy+HjrVm3DQCyYmavmtkPzGy3mT0Qfz4p4wgA+l2qEfRBnegIANbKzCYV1J2fljSloCxoWNIWSf+3mX3H3f95hk0EgKy8IulMbTJiM3tI0nkmhgMwyNKmuA/kREcAsBZmtkPBBEj1spUOhiNID7j7GymONyJpj4LZjsuSptx9sc6+RxT0yYsK1hU+4O7z7ZwHAHRC2Cderk0yHF960t1fM7MdZrbg7p9m0kgAyFjaZdYGdqIjAFiDPe6+q9EO7v5wmKXUNEBXsEbwmCSZ2ayCyZT21tn3PXeP31wFgMy4++tmtjkcKfdwcyU6Yh7us1t9uioQADSTNsV9k4KZ2oclzUeXV4vq14mOAKBNiZPCtbNfOCnnQu1xOIHSRLsNC4+5X9J+Sdq4ceNaDgUAqYQDPMuDPJGAXQqC9komDcvI8moXZVa7ABBIm+J+UUF6JPXlAJBe2qyiNPuNKEhXj1ows5E6qesjZrYnfM1OSc/E0+HdfUpBXbzGx8d99SEAoLuSAnZJO82WE4DOpEl3b6UEKPKaGXffGdvWs/Kgb+/bp7lzb2ps6Aa9vPSZRrffpxenudQGBl3aAP01gnMAaFnaoDfNfuvqbB+us3354tTMFiSdUhCoA0AuhCPnD0uakfSqu3/q7h+Z2RV3/91wn5KkNBmaqUuAwuyjEUlJWUg9KQ+qVquaO/emvnPnqMxMu9319Lk3Va1WGUkHBlzaAP292j9qkx5J2iHpjIIOMU3tJAAMmkfCUaBGozjDkh6R9N0mx1rQ6mAH2WlxAAAUnklEQVS8XtCu6MiRu1fWmg7fKaRzAohYJ+lVBX3gc2Z2UUHgPFnbIU35ZKslQO5+JnxdZuusVyoVjQ3doFqmgJlpbOgGVSoV+kZgwKUN0Je5++uSXjezV939W9HnzGwTS2MAwLIxSbeoeYC+OcWxElMs3X1VvWZ4sXqiNpqUF5OP7te56tu68c71ujr9vLaX7tWJY1NZNwtAdhYkXXT3h6Xl0fL52rJrLWi1BKjhsZqVB3VCuVzWy0ufabe7zEzurvNLn+mb5XKnfxSAgkkboCelX76bsG2Pmo8CAcCgeM7dn2y2k5k922yfcBR8ecQ8rLc8E3u8EF5IziuyFGY4knS6xbZ3VLVa1bnq29r6+FeDi9FdrnPfe4t0TmCAhcuqlWoDPGuYbLjVEqBGmpYHdWKCzVKppNHt9+npsAb9fFiDTn8IIG2AXkvTjNqZsO2ACNABoOZ4811a2m/SzA7p2iRIk5Hnjiio45wK0zvnw4tISdoS27fnKpWKbrxz/Yp0zhvvXE86JzDgkoJyM7upxXXQWyoBatKepuVBnZpg88Xp6eWyn29S9gMglDZAr6Vpxm2JPU6TpgkAAyGcnbiT+1V0bQmiM7Hn9sYer3g+a+VyWVenn5fvupbOefXDyyo/RjongFWeCv+klboEqJEsyoNKpRKBOYAV0gboHUvTBAAMnlKppO2le3Xue28FNegfXtb20r1cmAIDxsxuUjC6PaJg1HtEwYBPbVtte+oAvcUSoEZyVx4EYPCkDdA7naYJABgwJ45NXZvF/THSOYFBY2Y/VLAKkCSZgonY5hWkqK+T9IqkKwrmNGpVqhKgsB1lBUusDYczuc+4+5k8lgcBGDypAvROp2kCAAYT6ZzA4HL3XWEQvS6emWlmk+5+Ivz3xTaO3UoJUG3f5xKOk6vyIACD57qsGwAAGGzValUnT55UtdruBM4AisLdn5N03MyeMbMHok9F9nm99y0DgHxoeR10AAA6hbXRgcETZlw+ZWYPmdn3JR3Ouk0AkBcE6ACQATN7SMFkSDsl/YW7/62Mm9RzrI0ODDZ3f03Sa+EkwyVJ0xk3CQAyR4o7AGTA3V9z96OSDkq6J+v2ZKHR2ugABkdYj/6kmX3fzDaZ2e6s2wQAWSFAB4AMufu8YhMaDYpyuayrH16We1B6urw2epm10YFB4+5Vd/+WpL2STmXdHgDICinuANAlZrbJ3X+SYtf3ut2WPGJtdABx7n7UzL6SdTsAICsE6AmW1+kts04vgDXZI+m7Kfbz5rv0J9ZGB5CASeMADCwC9Jh9Bx7V2XcuaGjDqJZeeEn3b9uq6ePHsm4WgGI6YGYuyZrs94jSBfJ9ibXRAUSFs7wDwEAiQI+oVqs6+84FfXnnE8GMwv6gzs4cZUZhAO3aIulbKfbb3O2GFA2ZTAAAYBARoEdUKhUNbRhdMaPw0IZRVSoVLhABtOO5cHbihsIlhhBibXQAADComMU9olwua+nS3IoZhZcuzTGjMIB2He/wfn0vujb6lq/dpa2Pf1Xnqm+rWq1m3TQAAICuYwQ9olQq6f5tW3V25mhQg35pTvdv28roOYB2XU65381dbUWBNFobnb4YAAD0OwL0mOnjxyK1jwe5IASwFkeUrgb9KQUTxQ28crmsq9PPy3d5OBdIuDb6Y2QyAQCA/keAnoAZhQF0yAEze9bdf5r0pJndJGla0kO9bVZ+sTY6AAAYZAToANA9JyTtNbPT7v6T6BNmtjt8/iNJFFhH1FsbnZndAQBAv8skQDezEUl7JFUklSVNuftig/33RB+7++nuthAA1s7dD0iSmU2a2Svu/mk4an5a0g5JB939hJmxzFpMPJOJmd0BAMAgyGoE/ZS7j0mSmc0qHGVK2tHMDkmad/fTZjYs6XUFF7cAUAhhED5pZjcrqEs/I2mdu18Jn/8o0wbmXHRmdzOT73Kd+95bqlarjKQDAIC+0vNl1sysLGmh9jgcOZ9o8JKnaiPm7r5YC+wTjrvfzGbNbPbjjz/uaJsBYK3c/YSkWyU96e5fqwXnaK7RzO4AAAD9JIt10EckxdPZF8K09xXMbELSvJntMbMJMzuUtJ8kufuUu4+7+/htt93WhWYDQGvM7Leij939SUmfhGnudffDSuVyWVc/vCx3l6RrM7uXy6pWqzp58iTrpAMAgL6QRYr7ujrbhxO2jUgq10bQw3T485K2dKltANBJB8zMJVls+ykzm4nuJ+m7vWtWsdSb2f3Yie9Tlw4AAPpKFgH6glYH4/WC9vnwj6Qgxd3MRsxsxN3n67wGAPJii+qvgx690cgkcU3EZ3aXpEce/Y3EunRJzPYOAAAKKYsAPTGwdvekYsKkfevO9g4AOfNcmNbekJk90YvGFF10ZveTJ08m1qX/gyf+oX7255cYVQcAAIXU8wDd3StmtjxiHtaUn4k9XggnhJs3s8XIc8MKZnRn9BxAERxPuR8rU7SoXC7r6vTz8l0ejKC76+PZ/6zFn/u8yv/wbzKqDgAACimrZdYmw+XTauugT0aeOyJpRlJtyGOvmR2RdFFBSmjicmwAkDdpl09jmbXWJdWlb1r/V/Xnm5V6VH05ZZ6gHQAA5EQmAXqYzl5LaT8Te25v7PG8pMM9ahoAoCDq1aWnGVXf/cgezf3Rh6TCAwCAXMlqBB0AgDWL1qVLSjWq/rkNf1nn/2NV9xzaSSo8AADIFQJ0AEDfSDOqvjD3p/qr920hFR4AAOQOAToAoK80G1XfVv41ffDj/yD/GqnwAAAgXwjQAQB9LT6qXiqVNPnoflLhAQBA7hCgAwD6XnxUvVep8KTHAwCAVhCgAwAGUrdT4SXpXPVtAnkAAJAaAToAAOpsKvwPfueH+nyPAnmCewAA+gcBOgAAoU6lwttN1+vmX97Q9UA+adtag3sCfgAAskOADgBAA+2kwvun/5+ufHBpxbZOB/LdCO6TtjGaDwBA7xCgAwDQgjSp8F/7XyYkqauBfKeD+6KM5nMTAADQzwjQAQBoUbNU+FXBZBcC+U4H90UYzU/a1qubAPFt3CgAAHTDwAfo/AcLAOiEeNCetK2TgXyng/u8j+ZneRMgvu3z/930P37Oc3GjoBvb0B281wDSGOgAfd+BR3X2nQsa2jCqpRde0v3btmr6+LGsmwUA6GOdDOSTtrUb3Od9ND+rmwDxbQu//LH+4N/8vrYf/juZ3yjoVkYCOu/b+/Zp7tybGhu6QS8vfabR7ffpxenprJsFIIcGNkCvVqs6+84FfXnnE+HFxIM6O3NU1WqVu5oAgMylCeSTtq0luE/alpfR/KxuAsS3XfnJn+mv3LM58xsF3cpIKOp1kJmNSNojqSKpLGnK3RebvGbG3Xeu9TjNVKtVzZ17U9+5c1Rmpt3uevrcm4V9rwF018AG6JVKRUMbRlf8Bzu0YVSVSoXOEkAutXLh2I2LTBRHu8F90ra8jOZndRMgvu3mTbfqD/7N7+uOvzWa6Y2Cbmy78c71Rb4OOuXuY5JkZrOSTkjam7SjmU1IGpE0sZbjpFWpVDQ2dMOK93ps6IYiv9cAumhgA/RyuaylF16S+4PL/8EuXZpTuXww66YBQD2tXDh2/CITgysvo/lJ27p9EyBp221/+RZdyPhGQTe2Xf3wssqPlTv3xekRMytLWqg9dvfFMAhP5O5nwtcdWctx0iqXy3p56TPt9mvv9fmlz/TNcvHeawDdZ+6edRs6bnx83GdnZ5vut6IG/dIcNejAADKz8+4+nnU7mgkvHI9E0zHN7BN3v6Xdfc1sv6T9krRx48axn/70p11rP9BNeZjFffLR/dfquT+8vLrGuyDb0tSg563fNLM9kh5x972RbRcl7XT3+QavW9Evpj1OO31ntAb9PDXowEBK23cOdIAuMaMmMOjydqFZTysXoO1crLbSbwJIlqeZ2Ls5i3ve+s0wYN6Z0OftdfdKg9fFA/SWj8M1J4C0CNC50ASQQt4uNOtp5cKx2xeZAAZb3vrN8KbkgXjWkKQdLQboLR+HvhNAWmn7zut60RgAwJotSBqObVvXgX0BoOgSM4MaBeddPg4AtI0AHQCKoZULRy4yAQyMsG9bvgkZrmJxJvrYzOI3LVs+DgD0Ql+muJvZx5Lqzdhxq6Q/62FzuqUfzqMfzkHiPPKm1fO43d1v61ZjOilMjarNzD6iYCK4vZHHC7Wl1BrtW+fYjfpNqT++H/1wDhLnkSf9cA5SH/Sb4eSYE0pYWtLMTkmacfep2L5HJD0XPnem2XHq/FyuOYujH86jH85BGtzzSNV39mWA3oiZzeapbqpd/XAe/XAOEueRN/1yHknavADtyDro/fC+9sM5SJxHnvTDOUj9cx550y/vK+eRH/1wDhLn0czAroMOAEUTpl/W0tTPxJ7bm3ZfAAAA5BM16AAAAAAA5MAgBuhTWTegQ/rhPPrhHCTOI2/65Tzyph/e1344B4nzyJN+OAepf84jb/rlfeU88qMfzkHiPBoauBp0AAAAAADyaBBH0AEAAAAAyB0CdAAAAAAAcoAAHQAAAACAHCBAzyEzm0nYNmJmh8xsIvx7OM1zQJyZHY895ruFvkDfiW6h30Q/o+9Et9B3tsnd+/6PpBFJhyRNhH8PZ92mOu2ckLQ/+FhWPXc+8u9hSafSPJfhuZTDczkk6ZSkkTSfR94+q/A8JiTtkXS8qOcRO59PivjdknREkkv6RNJM0T+LIvwpyntH35m/z6qf+s4i95thG+g7e/+eF+K965e+k34zX+cROx/6zjY+j8w/vB69wbn6wFO0N/5lLkuaSdqn0XMZtn9Y0v7I4wlJF9N8Hnn7rMJfyuHw3/uj73WRziPSjhWdZZG+W5L2NHiuUJ9FUf4U7b2j78zPZ9UvfWfR+83w59N39v49L9R7V+S+k34zf+cRaQd9Z5ufR9+nuJtZWdJC7bG7Lyr45S2SEUmLsW0LZjbS5LmsjEg6HHk8K2nEzIYbfR45/aw2h+2Qgratkxq3NafnIUkT7l6JbSvad2uVgn4Wudcn713Rvt/0nfk7j77sN6VCfhaF0CfvXZG+4/Sb+TsPib5z1XOt6PsAXQX6wBtYV2f7cJPnMhH+Qo5FNo1LWgy/pIX65Yx0lJJ0QNf+EyjUeZjZhKQzCU8V6bs1YmZ7wtqkI5HapEJ9FgXSD+9dkb7f9J05O48+6Tcl+s5e64f3rjDfcfrN/J0HfefaP4/r22tvoeTxA2/Vgla3d12K5zKT0MlMhv8u2i+nwl+qPQpSb2odTmHOI+xQFmKfSU2RvltTtXMwswUFdWY7G7Qpd59FwfTDe1ek77ck+s4Gz/VUH/WbEn1nr/XDe1eo7zj9Zt3neo6+szOfxyAE6Hn8wFs1n7TR3StmlviChLSSTJjZfkmvuPvpcFPRfjnl7vOSnjOz/WY24+47VazzqKXd1O7eDYefyxkV6LsV7ezD9tVShor0WRRJP7x3hfl+x9F3rnqu1/qi3wx/Nn1nb/XDe1eo73gN/eaq57JA39mBz2MQAvS6X4ZeN6Rd4Zdi+cMNv/Rnmj2XtfCLPB+5AygV6JezdhfT3Z8LN70q6Xi4vTDnEfmPSpJkZnL3qcjj3H+3wpqeE+4+lvB0YT6LgqHvzAh95+rnutLQBvqh3wx/Pn1n79F3ZoB+c/VzXWloE/Sdnfk8+j5Az9sH3kj4ZZhQcLfpiFamt0ya2SFJFQUzHU5GXtrouUzUJkmofSHNbI+7ny5Ypz8iaX3s8WJ4d7MwnUykHcMKZgVV+H05HZ5LEb5b8wqWHJG0/B/xaamYFxJFUKT3jr4zd59V3/SdBe83JfrOnivSe9cvfSf9Zq7Oo9YO+k61/3mYB1PA97VIB1T7wJdrCtB54ZfxvFameMy7+5bw+bqfR94+KzPbo2upKTslPRP5D6Aw59EPws6xljK1RcFn0fT95rNoH+9db9F35u88+gF9Z+/x3vUO/Wb+zqNfZNl3DkSADgAAAABA3g3CMmsAAAAAAOQeAToAAAAAADlAgA4AAAAAQA4QoAMAAAAAkAME6AAAAAAA5AABOgAAAAAAOUCADgAAAABADlyfdQOAODMbkXRA0iFJFUmvRJ7eImm/pAPuPtXldpQlPSLpYrd/FgCsFX0nALSGfhN5ZO6edRuARGZ2UdJxd38utr0saSK+vUttOCRpp7vv7PbPAoBOoO8EgNbQbyJPSHFH4bh7RdJij35cpUc/BwC6ir4TAFpDv4ksEKCjMMxsIvJwPrOGAECB0HcCQGvoN5ElatBRJDslnZEkdz8jLXegRxR0njOSFiSNSFqM1vCY2bCCOqJaJzuSkMZ0RNJ7Cu6UDrv76chztY56p6gPAlAs9J0A0Br6TWSGGnTkVlgPVOsEt0ja7+6WsN8eSUfcfUtk2ylJM7VOzczOS9rh7ovh4wlJe939QPh4RtLhMJVJZvaJpB2S1kk6JWnM3efD5zypHQCQB/SdANAa+k3kCSPoyLuZ2l1HM2tUAxSv2zmuoJObqt2JrHWU4b/PmNmMmR1WcPdzvNZRhsbcfT587UKto6wxs+Ho8QAgZ+g7AaA19JvIBWrQUSTLS1+Y2USYQlTPvKTa82UFaUhxi5LGwz8rOsNY5xivPaKTBFAk9J0A0Br6TWSGAB2FEbvbWG5yN3FE1zq1eQVpQ3HDkmbDPyMdaSQA5Ax9JwC0hn4TWSJAR+GEdzG/Ettcjj0+IOmwJNUm3jCz5Q4xrCE67e6LYSdcSy2qPT8Srn0pre5oG91FBYBcou8EgNbQbyIL1KAjd8JO7YCCO4yPmK2YG2OLpIclvRp72XzYAS4q6DiXJ+sI7ZD0lJm9p6DzG3b3vbHnj4Q/eyE8zoKCDrdsZockTUl6Ktz/iJkdidcJAUBW6DsBoDX0m8gjZnFH4YWd5COxzg8A0AB9JwC0hn4TvUCKO/oFKUAA0Dr6TgBoDf0muooAHYUW1uwckDRuZvuzbg8AFAF9JwC0hn4TvUKKOwAAAAAAOcAIOgAAAAAAOUCADgAAAABADhCgAwAAAACQAwToAAAAAADkAAE6AAAAAAA5QIAOAAAAAEAOEKADAAAAAJAD/z/TiJ3c8gnCaAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plotting\n", "fig, axs = plt.subplots(nrows=1, ncols=3, figsize=(14, 3))\n", "ax = axs[0]\n", "ax.plot(epoch, fidelities, \"o\", color = \"C0\", markeredgecolor=\"black\")\n", "ax.set_ylabel(r'Fidelity')\n", "ax.set_xlabel(r'Epoch')\n", "\n", "ax = axs[1]\n", "ax.plot(epoch, KLs, \"o\", color = \"C1\", markeredgecolor=\"black\")\n", "ax.set_ylabel(r'KL Divergence')\n", "ax.set_xlabel(r'Epoch')\n", "\n", "ax = axs[2]\n", "ax.plot(epoch, coeffs, \"o\", color = \"C2\", markeredgecolor=\"black\")\n", "ax.set_ylabel(r'$A\\psi_{RBM}[5]$')\n", "ax.set_xlabel(r'Epoch')\n", "\n", "plt.tight_layout()\n", "plt.savefig(\"fid_KL.pdf\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It should be noted that one could have just ran *nn_state.fit(train_samples)* and just used the default hyperparameters and no training evaluators.\n", "\n", "To demonstrate how important it is to find the optimal hyperparameters for a certain system, restart this notebook and comment out the original *fit* statement and uncomment the one below. The default hyperparameters will be used instead. Using the non-default hyperparameters yielded a fidelity of approximately 0.994, while the default hyperparameters yielded a fidelity of approximately 0.523!\n", "\n", "The RBM's parameters will also be saved for future use in other tutorials. They can be saved to a pickle file with the name \"saved_params.pt\" with the code below." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "nn_state.save(\"saved_params.pt\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This saves the weights, visible biases and hidden biases as torch tensors with the following keys: \"weights\", \"visible_bias\", \"hidden_bias\"." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }