{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reconstruction of a complex wavefunction\n",
    "\n",
    "In this tutorial, a walkthrough of how to reconstruct a **complex** wavefunction via training a *Restricted Boltzmann Machine* (RBM), the neural network behind QuCumber, will be presented.\n",
    "\n",
    "\n",
    "## The wavefunction to be reconstructed\n",
    "The simple wavefunction below describing two qubits (coefficients stored in `qubits_psi.txt`) will be reconstructed.\n",
    "\n",
    "\\begin{equation}\n",
    "            \\vert\\psi \\rangle = \\alpha \\vert00\\rangle + \\beta \\vert 01\\rangle + \\gamma \\vert10\\rangle + \\delta \\vert11\\rangle\n",
    "\\end{equation}\n",
    "\n",
    "where the exact values of $\\alpha, \\beta, \\gamma$ and $\\delta$ used for this tutorial are \n",
    "\n",
    "\\begin{align}\n",
    "\\alpha & = 0.2861  + 0.0539 i \\\\\n",
    "\\beta &  = 0.3687 - 0.3023 i \\\\\n",
    "\\gamma & = -0.1672 - 0.3529 i \\\\\n",
    "\\delta & = -0.5659 - 0.4639 i.\n",
    "\\end{align}\n",
    "\n",
    "The example dataset, `qubits_train.txt`, comprises of 500 $\\sigma$ measurements made in various bases (X, Y and Z). A corresponding file containing the bases for each data point in `qubits_train.txt`, `qubits_train_bases.txt`, is also required. As per convention, spins are represented in binary notation with zero and one denoting spin-down and spin-up, respectively.\n",
    "\n",
    "## Using qucumber to reconstruct the wavefunction\n",
    "\n",
    "### Imports\n",
    "To begin the tutorial, first import the required Python packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from qucumber.nn_states import ComplexWaveFunction\n",
    "\n",
    "from qucumber.callbacks import MetricEvaluator\n",
    "\n",
    "import qucumber.utils.unitaries as unitaries\n",
    "import qucumber.utils.cplx as cplx\n",
    "\n",
    "import qucumber.utils.training_statistics as ts\n",
    "import qucumber.utils.data as data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Python class `ComplexWaveFunction` contains generic properties of a RBM meant to reconstruct a complex wavefunction, the most notable one being the gradient function required for stochastic gradient descent.\n",
    "\n",
    "To instantiate a `ComplexWaveFunction` object, one needs to specify the number of visible and hidden units in the RBM. The number of visible units, `num_visible`, is given by the size of the physical system, i.e. the number of spins or qubits (2 in this case), while the number of hidden units, `num_hidden`, can be varied to change the expressiveness of the neural network.\n",
    "\n",
    "**Note:** The optimal `num_hidden` : `num_visible` ratio will depend on the system. For the two-qubit wavefunction described above, good results are yielded when this ratio is 1.\n",
    "\n",
    "On top of needing the number of visible and hidden units, a `ComplexWaveFunction` object requires the user to input a dictionary containing the unitary operators (2x2) that will be used to rotate the qubits in and out of the computational basis, Z, during the training process. The `unitaries` utility will take care of creating this dictionary.\n",
    "\n",
    "The `MetricEvaluator` class and `training_statistics` utility are built-in amenities that will allow the user to evaluate the training in real time. \n",
    "\n",
    "Lastly, the `cplx` utility allows QuCumber to be able to handle complex numbers. Currently, PyTorch does not support complex numbers.\n",
    "\n",
    "\n",
    "### Training\n",
    "To evaluate the training in real time, the fidelity between the true wavefunction of the system and the wavefunction that QuCumber reconstructs, $\\vert\\langle\\psi\\vert\\psi_{RBM}\\rangle\\vert^2$, will be calculated along with the Kullback-Leibler (KL) divergence (the RBM's cost function). First, the training data and the true wavefunction of this system need to be loaded using the `data` utility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_path = \"qubits_train.txt\"\n",
    "train_bases_path = \"qubits_train_bases.txt\"\n",
    "psi_path = \"qubits_psi.txt\"\n",
    "bases_path = \"qubits_bases.txt\"\n",
    "\n",
    "train_samples, true_psi, train_bases, bases = data.load_data(\n",
    "    train_path, psi_path, train_bases_path, bases_path\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The file `qubits_bases.txt` contains every unique basis in the `qubits_train_bases.txt` file. Calculation of the full KL divergence in every basis requires the user to specify each unique basis.\n",
    "\n",
    "As previously mentioned, a `ComplexWaveFunction` object requires a dictionary that contains the unitary operators that will be used to rotate the qubits in and out of the computational basis, Z, during the training process. In the case of the provided dataset, the unitaries required are the well-known $H$, and $K$ gates. The dictionary needed can be created with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "unitary_dict = unitaries.create_dict()\n",
    "# unitary_dict = unitaries.create_dict(unitary_name=torch.tensor([[real part],\n",
    "#                                                                 [imaginary part]],\n",
    "#                                                                 dtype=torch.double)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the user wishes to add their own unitary operators from their experiment to `unitary_dict`, uncomment the block above. When `unitaries.create_dict()` is called, it will contain the identity and the $H$ and $K$ gates by default under the keys \"Z\", \"X\" and \"Y\", respectively.\n",
    "\n",
    "The number of visible units in the RBM is equal to the number of qubits. The number of hidden units will also be taken to be the number of visible units."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "nv = train_samples.shape[-1]\n",
    "nh = nv\n",
    "\n",
    "nn_state = ComplexWaveFunction(\n",
    "    num_visible=nv, num_hidden=nh, unitary_dict=unitary_dict, gpu=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, QuCumber will attempt to run on a GPU if one is available (if one is not available, QuCumber will fall back to CPU). If one wishes to guarantee that QuCumber runs on the CPU, add the flag `gpu=False` in the `ComplexWaveFunction` object instantiation. Set `gpu=True` in the line above to run this tutorial on a GPU.\n",
    "\n",
    "Now the hyperparameters of the training process can be specified. \n",
    "\n",
    "1. `epochs`: the total number of training cycles that will be performed (default = 100)\n",
    "2. `pos_batch_size`: the number of data points used in the positive phase of the gradient (default = 100)\n",
    "3. `neg_batch_size`: the number of data points used in the negative phase of the gradient (default = `pos_batch_size`)\n",
    "4. `k`: the number of contrastive divergence steps (default = 1)\n",
    "5. `lr`: the learning rate (default = 0.001)\n",
    "\n",
    "    **Note:** For more information on the hyperparameters above, it is strongly encouraged that the user to read through the brief, but thorough theory document on RBMs. One does not have to specify these hyperparameters, as their default values will be used without the user overwriting them. It is recommended to keep with the default values until the user has a stronger grasp on what these hyperparameters mean. The quality and the computational efficiency of the training will highly depend on the choice of hyperparameters. As such, playing around with the hyperparameters is almost always necessary. \n",
    "    \n",
    "The two-qubit example in this tutorial should be extremely easy to train, regardless of the choice of hyperparameters. However, the hyperparameters below will be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 100\n",
    "pbs = 50  # pos_batch_size\n",
    "nbs = 50  # neg_batch_size\n",
    "lr = 0.1\n",
    "k = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For evaluating the training in real time, the `MetricEvaluator` will be called to calculate the training evaluators every 10 epochs. The `MetricEvaluator` requires the following arguments.\n",
    "\n",
    "1. `period`: the frequency of the training evaluators being calculated (e.g. `period=200` means that the `MetricEvaluator` will compute the desired metrics every 200 epochs)\n",
    "2. A dictionary of functions you would like to reference to evaluate the training (arguments required for these functions are keyword arguments placed after the dictionary)\n",
    "\n",
    "The following additional arguments are needed to calculate the fidelity and KL divergence in the `training_statistics` utility.\n",
    "\n",
    "- `target_psi` (the true wavefunction of the system)\n",
    "- `space` (the entire Hilbert space of the system)\n",
    "\n",
    "The training evaluators can be printed out via the `verbose=True` statement.\n",
    "\n",
    "Although the fidelity and KL divergence are excellent training evaluators, they are not practical to calculate in most cases; the user may not have access to the target wavefunction of the system, nor may generating the Hilbert space of the system be computationally feasible. However, evaluating the training in real time is extremely convenient. \n",
    "\n",
    "Any custom function that the user would like to use to evaluate the training can be given to the `MetricEvaluator`, thus avoiding having to calculate fidelity and/or KL divergence. As an example, functions that calculate the the norm of each of the reconstructed wavefunction's coefficients are presented. Any custom function given to `MetricEvaluator` must take the neural-network state (in this case, the `ComplexWaveFunction` object) and keyword arguments. Although the given example requires the Hilbert space to be computed, the scope of the `MetricEvaluator`'s ability to be able to handle any function should still be evident."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def alpha(nn_state, space, **kwargs):\n",
    "    rbm_psi = nn_state.psi(space)\n",
    "    normalization = nn_state.compute_normalization(space).sqrt_()\n",
    "    alpha_ = cplx.norm(\n",
    "        torch.tensor([rbm_psi[0][0], rbm_psi[1][0]], device=nn_state.device)\n",
    "        / normalization\n",
    "    )\n",
    "\n",
    "    return alpha_\n",
    "\n",
    "\n",
    "def beta(nn_state, space, **kwargs):\n",
    "    rbm_psi = nn_state.psi(space)\n",
    "    normalization = nn_state.compute_normalization(space).sqrt_()\n",
    "    beta_ = cplx.norm(\n",
    "        torch.tensor([rbm_psi[0][1], rbm_psi[1][1]], device=nn_state.device)\n",
    "        / normalization\n",
    "    )\n",
    "\n",
    "    return beta_\n",
    "\n",
    "\n",
    "def gamma(nn_state, space, **kwargs):\n",
    "    rbm_psi = nn_state.psi(space)\n",
    "    normalization = nn_state.compute_normalization(space).sqrt_()\n",
    "    gamma_ = cplx.norm(\n",
    "        torch.tensor([rbm_psi[0][2], rbm_psi[1][2]], device=nn_state.device)\n",
    "        / normalization\n",
    "    )\n",
    "\n",
    "    return gamma_\n",
    "\n",
    "\n",
    "def delta(nn_state, space, **kwargs):\n",
    "    rbm_psi = nn_state.psi(space)\n",
    "    normalization = nn_state.compute_normalization(space).sqrt_()\n",
    "    delta_ = cplx.norm(\n",
    "        torch.tensor([rbm_psi[0][3], rbm_psi[1][3]], device=nn_state.device)\n",
    "        / normalization\n",
    "    )\n",
    "\n",
    "    return delta_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the Hilbert space of the system must be generated for the fidelity and KL divergence and the dictionary of functions the user would like to compute every `period` epochs must be given to the `MetricEvaluator`. Note that some of the coefficients aren't being evaluated as they are commented out. This is simply to avoid cluttering the output, and may be uncommented by the user."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "period = 2\n",
    "space = nn_state.generate_hilbert_space(nv)\n",
    "\n",
    "callbacks = [\n",
    "    MetricEvaluator(\n",
    "        period,\n",
    "        {\n",
    "            \"Fidelity\": ts.fidelity,\n",
    "            \"KL\": ts.KL,\n",
    "            \"normα\": alpha,\n",
    "            # \"normβ\": beta,\n",
    "            # \"normγ\": gamma,\n",
    "            # \"normδ\": delta,\n",
    "        },\n",
    "        target_psi=true_psi,\n",
    "        bases=bases,\n",
    "        verbose=True,\n",
    "        space=space,\n",
    "    )\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the training can begin. The `ComplexWaveFunction` object has a function called `fit` which takes care of this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 2\tFidelity = 0.623747\tKL = 0.226386\tnormα = 0.272518\n",
      "Epoch: 4\tFidelity = 0.744691\tKL = 0.142639\tnormα = 0.248872\n",
      "Epoch: 6\tFidelity = 0.818254\tKL = 0.094584\tnormα = 0.263589\n",
      "Epoch: 8\tFidelity = 0.867098\tKL = 0.067506\tnormα = 0.278453\n",
      "Epoch: 10\tFidelity = 0.900217\tKL = 0.051592\tnormα = 0.281094\n",
      "Epoch: 12\tFidelity = 0.922993\tKL = 0.041311\tnormα = 0.276052\n",
      "Epoch: 14\tFidelity = 0.937807\tKL = 0.034972\tnormα = 0.274676\n",
      "Epoch: 16\tFidelity = 0.947232\tKL = 0.030543\tnormα = 0.283873\n",
      "Epoch: 18\tFidelity = 0.955277\tKL = 0.027313\tnormα = 0.278906\n",
      "Epoch: 20\tFidelity = 0.959930\tKL = 0.025034\tnormα = 0.290271\n",
      "Epoch: 22\tFidelity = 0.963333\tKL = 0.023719\tnormα = 0.296183\n",
      "Epoch: 24\tFidelity = 0.969419\tKL = 0.021086\tnormα = 0.276108\n",
      "Epoch: 26\tFidelity = 0.972300\tKL = 0.020200\tnormα = 0.290305\n",
      "Epoch: 28\tFidelity = 0.974777\tKL = 0.018635\tnormα = 0.284231\n",
      "Epoch: 30\tFidelity = 0.976208\tKL = 0.017865\tnormα = 0.282036\n",
      "Epoch: 32\tFidelity = 0.978382\tKL = 0.016862\tnormα = 0.282498\n",
      "Epoch: 34\tFidelity = 0.980578\tKL = 0.015977\tnormα = 0.279435\n",
      "Epoch: 36\tFidelity = 0.980983\tKL = 0.015545\tnormα = 0.277835\n",
      "Epoch: 38\tFidelity = 0.982651\tKL = 0.014751\tnormα = 0.280070\n",
      "Epoch: 40\tFidelity = 0.983155\tKL = 0.014353\tnormα = 0.276912\n",
      "Epoch: 42\tFidelity = 0.983996\tKL = 0.013827\tnormα = 0.278844\n",
      "Epoch: 44\tFidelity = 0.982731\tKL = 0.015100\tnormα = 0.305219\n",
      "Epoch: 46\tFidelity = 0.984791\tKL = 0.013417\tnormα = 0.293674\n",
      "Epoch: 48\tFidelity = 0.985395\tKL = 0.012845\tnormα = 0.280658\n",
      "Epoch: 50\tFidelity = 0.986767\tKL = 0.012093\tnormα = 0.277599\n",
      "Epoch: 52\tFidelity = 0.987795\tKL = 0.011650\tnormα = 0.278886\n",
      "Epoch: 54\tFidelity = 0.987057\tKL = 0.011843\tnormα = 0.271735\n",
      "Epoch: 56\tFidelity = 0.987125\tKL = 0.011552\tnormα = 0.280304\n",
      "Epoch: 58\tFidelity = 0.987295\tKL = 0.011382\tnormα = 0.288229\n",
      "Epoch: 60\tFidelity = 0.988201\tKL = 0.011201\tnormα = 0.266736\n",
      "Epoch: 62\tFidelity = 0.989181\tKL = 0.010504\tnormα = 0.288520\n",
      "Epoch: 64\tFidelity = 0.989308\tKL = 0.010293\tnormα = 0.292218\n",
      "Epoch: 66\tFidelity = 0.989321\tKL = 0.009901\tnormα = 0.282069\n",
      "Epoch: 68\tFidelity = 0.989347\tKL = 0.009836\tnormα = 0.275723\n",
      "Epoch: 70\tFidelity = 0.989494\tKL = 0.009838\tnormα = 0.293840\n",
      "Epoch: 72\tFidelity = 0.990115\tKL = 0.009225\tnormα = 0.282556\n",
      "Epoch: 74\tFidelity = 0.990199\tKL = 0.009095\tnormα = 0.278911\n",
      "Epoch: 76\tFidelity = 0.989979\tKL = 0.009214\tnormα = 0.273241\n",
      "Epoch: 78\tFidelity = 0.989633\tKL = 0.009275\tnormα = 0.274384\n",
      "Epoch: 80\tFidelity = 0.989972\tKL = 0.008976\tnormα = 0.275430\n",
      "Epoch: 82\tFidelity = 0.989920\tKL = 0.008871\tnormα = 0.285605\n",
      "Epoch: 84\tFidelity = 0.991177\tKL = 0.008183\tnormα = 0.282607\n",
      "Epoch: 86\tFidelity = 0.991249\tKL = 0.008095\tnormα = 0.276934\n",
      "Epoch: 88\tFidelity = 0.990857\tKL = 0.008273\tnormα = 0.272151\n",
      "Epoch: 90\tFidelity = 0.990802\tKL = 0.008071\tnormα = 0.280823\n",
      "Epoch: 92\tFidelity = 0.991090\tKL = 0.007838\tnormα = 0.279963\n",
      "Epoch: 94\tFidelity = 0.990995\tKL = 0.007861\tnormα = 0.275772\n",
      "Epoch: 96\tFidelity = 0.990326\tKL = 0.008202\tnormα = 0.289882\n",
      "Epoch: 98\tFidelity = 0.991012\tKL = 0.007690\tnormα = 0.277037\n",
      "Epoch: 100\tFidelity = 0.991736\tKL = 0.007292\tnormα = 0.275516\n"
     ]
    }
   ],
   "source": [
    "nn_state.fit(\n",
    "    train_samples,\n",
    "    epochs=epochs,\n",
    "    pos_batch_size=pbs,\n",
    "    neg_batch_size=nbs,\n",
    "    lr=lr,\n",
    "    k=k,\n",
    "    input_bases=train_bases,\n",
    "    callbacks=callbacks,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of these training evaluators can be accessed after the training has completed, as well. The code below shows this, along with plots of each training evaluator versus the training cycle number (epoch)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note that the key given to the *MetricEvaluator* must be\n",
    "# what comes after callbacks[0].\n",
    "fidelities = callbacks[0].Fidelity\n",
    "\n",
    "# Alternatively, we may use the usual dictionary/list subscripting\n",
    "# syntax. This is useful in cases where the name of the metric\n",
    "# may contain special characters or spaces.\n",
    "KLs = callbacks[0][\"KL\"]\n",
    "coeffs = callbacks[0][\"normα\"]\n",
    "epoch = np.arange(period, epochs + 1, period)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Some parameters to make the plots look nice\n",
    "params = {\n",
    "    \"text.usetex\": True,\n",
    "    \"font.family\": \"serif\",\n",
    "    \"legend.fontsize\": 14,\n",
    "    \"figure.figsize\": (10, 3),\n",
    "    \"axes.labelsize\": 16,\n",
    "    \"xtick.labelsize\": 14,\n",
    "    \"ytick.labelsize\": 14,\n",
    "    \"lines.linewidth\": 2,\n",
    "    \"lines.markeredgewidth\": 0.8,\n",
    "    \"lines.markersize\": 5,\n",
    "    \"lines.marker\": \"o\",\n",
    "    \"patch.edgecolor\": \"black\",\n",
    "}\n",
    "plt.rcParams.update(params)\n",
    "plt.style.use(\"seaborn-deep\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+gAAADQCAYAAACUYku5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3W9sHPd95/HPN02uPcGyaSlW0jSnSFR6iRq18g4Zo7AvkS+inLrtAYUiWX3QPDmLpKLASFrUUny+4u6JG8vKGTZQKxElG1f3nthWckCBtmhI+yQbdg8xuRu3zCm9wlSSFmkrRzQdubwUbfO9BzNLDpe73Nm/82ffL8CwdmZ29jfL4Zfz/f01dxcAAAAAAEjXO9IuAAAAAAAAIEEHAAAAACATSNABAAAAAMgAEnQAAAAAADKABB0AAAAAgAwgQQcAAAAAIANI0AEAAAAAyAASdAAAAAAAMoAEHQAAAACADHhnWh9sZtPufqDJMcOSDkkqSwokTbn7UrNzv/vd7/YdO3Z0pZwAim1ubu4H7n5L2uVIG3ETQFLEzVXETgBJJY2dfU/QzWxM0rCksQSHP+fuI9H7ZiWdk3S42Zt27Nih2dnZjsoJYDCY2XfTLkMWEDcBJEXcXEXsBJBU0tjZ9y7u7j7j7lOSNmwJN7NA0mLsfUtKltQDAAAAAJA7WR6DPqz1Sfxi1O0dAAAAAIBCyXKCvqXB9qF6G81swsxmzWz2jTfe6GGxgP6qVCp68sknValU2NbGNvQG3zUAoGj424YsMHdP54PN3nT3mzfYf0jSZHwiOTN7U9J+dy9vdO7R0VFnPBAqlYrK5bKCIFCpVMrltqOTx3Xxlde0adseLV+d152375UktiXcdv7smab3iZnNufto0wMLrpW4OX58QpcqL2vz7q26fvma9pXu0LkzUz0uIYCsIG6u4pmzOD579KjmL72okU03aG75be3Z93E9cf582sVCgSSNnVlO0ANJp2oT9I3eU0WwLLZBSWz3/Nv3af7/fl8fPnC/zEzurtf+8L/one/6KX3k7gfY1mTbt6dP67mnf2/lHmmEB81Q0rhZqVR05PhvaO/nPrb6/T/+kp458z+aftcAioG4uYpnzmKoVCr6/D1H9NDuPSt/2x68PK/Hnn2Gv23omqSxM7Vl1uqJxpcvuvuSu5fNbEvNvpn0Sod29axF+bGn1ie2jz1VJ7G9W3+6LoHL/rZXnvltvffDn5CZSVK4/V1DuuGnP8S2BNs2bdujcrnMH9YuK5fL2rx765rvevPurXzXAIDcKpfLGtl0w5q/bSObbuBvG1KRxjJrgcLZ2IfM7JSkaXevJt6nJE1LqvaVHDezE1pdB3283+VFY91KqJNuS5p4FyWxveln9urNvy5r+95fXqnNtX9a0tt/+5r8VrY127Z8dV5BcKxXt//ACoJA188/Kr/LV77r65evKbgvSLtoAAC0JQgCPb38tg766t+2ueW39emAv23ov74n6NH48bKkR+rsO9zgWInW877Jakt20sS7KImt/7+/00d/4YP68+nTK93eP7n/jvB7ZlvTbXfevpda7x4olUraV7pDlx5/ac0YdL5rAEBelUol7dn3cT1YMwadv21IQ2pj0HuJ8UDJtTN2O+nY6G9ECfUHbv2V1c/740d1809/SDtKrW/79ktP60dL39XeX/3PTcce/8KH/o3+/C//OvVx5N2Y5CwrE9bldVszjKUMtRo32/muARQDcXMVz5zFwt829FLS2EmCPmDigeeJr5zraeKdNKFuZdKvpIl3kRJb9BYPmiHiJoCkiJuriJ0AkiJBJ1hu2Dr+5nf/t/75x6aRX/uvPUu8e9GS3UriDSTBg2aIuAkgKeLmKmIngKSSxs5MzeKO7qkdI17bOq6feJd+/E8/amvsdtKx0esT6mN1EurWt5VKpXVJeL1tAAAAAJAnJOgFEW9BlsKW5/jkbLUTrA29Z5cuv/j72lH61Z4n3kkTahJvAAAAAIOMBD2Hms2o/tNb/pU2bduzYev4jbfs1L/8aEnf+pMvavP7bu1p4g0AAAAAaI4EPWeadV13vzsaD/5N+d67N2wdP/hrv6zPHhsn8QYAAACADCBBz7hWu66bmW7+wC/q5h8v6NsJu6WTeAMAgDwzs2FJhySVJQWSptx9qcGxgaQtkoYkHZB0yt0XWj0PAPQCCXqGtdN13d21fHVev//070lSotZxAACAnHvO3UckycxmJZ2TdLjBsc9L2unuS2a2RdJZhYl6q+cBgK4jQc+oSqWyrrU8adf1O2/fS+s4AAAYCFGL+GL1dZR4j23wlp2xVvFFha3pic9jZhOSJiRp+/btnV8AAMSQoGdIvDt7uVxe11reatd1APnTRjfNUYXdND8q6STdNAEMoGFJtfFt0cyGqzExriYWTko62cp53H1K0pQUroPeaeEBII4EPSPqTf62fPX7cr+bruvAYEnUvdLMhiSNRg+Kilp5piXtauU8AFAAWxpsH2r0hlgl5rS7z7R7HgDoNhL0DKjXnX1++rR+4UP/hq7rwABpsZvmsMJWn6no9ayk4ShxH27hPACQd4tan0Q3SrYlSVGL+CNmNmFm0+5+oJ3zAEC3kaBnQL3u7Ju27dGv3H2bfufBgK7rwOBI3E3T3ctmNhLbNCppKUrGE52HcZQACmJdN3YpjJO126ot5+7+SLTpWUlno+2JzwMAvUKCnpL4ePMgCLT82FPrurNXk3IScyBfzOxGSWMKE+YXzOxWd/9mgre21L2yzjjK8VbOwzhKAEUQVViuxL0o2Z6peb0YxcxhSVtjbx9WGKur83c0PA8A9AMJegpqx5vfefte3Xn7Xl1s0J0dQH6Y2X6FS/aUJX1D0guSrpjZQXf/WpO3t9W9MmoJf8bdL3RyHgDIsXEzO6HViTHHY/tOKZyjY8rdZ8xsKIqbUri82v6E54HWNjLxrAp0Hwl6n9Ubb35x+rSee/r39Nlj6yd/A5A7gbt/UFpJ1uXub1l1DMvGWu5eGY0tX4hNctTWeQAgz6L4Vo1xMzX7Dte8vhB7OVWzr+F5IH326FHNX3pRI5tu0NPLb2vPvo/rifPn0y4WUCgk6H3WaLx5uVzWvffeS2IO5N/rDbY37ULeYjfNlUnlqom3mR1y9wvNzgMAQKsqlYrmL72oh3aHz7EH3fXgpRdVqVR4fgW66B1pF2DQBEGg5avzcg+f1VfHmwcplwxAl9xmZh+I/u2SZGY7JN2W8P3jZnYiahk/pPXdNO+Jzjks6XlJc2bmZubR/iTnAQCgJeVyWSObbljTyDSy6QaVy3TOArqJFvQ+qB2rw3hzoNC+qChplrQU69m+v/FbViXtphlNaHRzO+cBAKBVQRDo6eW3ddB9ZVLjueW39WkamYCuIkHvsXoTwp0/eyaWtDPeHCgSd39L0gfN7JCknZLK7v58ysUCAKAjpVJJe/Z9XA9GY9DnojHoPMcC3UWC3kONJoSrjtUhoAHFFC2rdqHmdZJl1gAAyKwnzp9faWT69IDM4j5Is9YP0rVmGQl6D200IRw3PVBMZvYpSVNmNuLu34k2v2Vmn3D3F1IsGgAAHRukRqZBmrV+kK4165gkroeYEA4YSEvuvjWWnMvdr2j9uuQAACCj4rPWf2rHTj20e4/mo1nri2aQrjUPSNB7qDoh3LenT+u73/xjfXv6NBPCAcV3U4PtWxpsBwAAGTNIs9YP0rXmAV3ce4wJ4YCBc5uZLcTHnJvZJySNSKKvGAAAOZDXWevbGUee12stKhL0PhiksTrAoHP3L5jZ181sRNKCpOHo/4mWWQMAAOnL46z17Y4jz+O1FhkJepcx+yEAd7/LzAKFreaz7s4gLgAAciZPs9bHx5GbmQ6668FoHHmScufpWosulQTdzIYlHZJUlhRImnL3pQbHBpLGtNoS1fDYtDVa8xzA4HH3ssIYJ0kysx3xieMAAED25aUn7EbjyJOWPy/XWnRptaA/5+4jkmRms5LOSTpce5CZDUXH7opeD0s6JWmyj2VNpNma5wAGi5ndWLPppKTPpFEWAAD6jV6l/cU48uLoe4IetYgvVl+7+5KZjTU4vNpyXj12wcwmVCdBj7ZPSNL27du7WuYkWPMcgCSZ2biks5Jc0pIkU7jEmosEHQCQI+0m2ayp3X+MIy+ONFrQhxU+tMYtmtmwuy/UbF9SnaWJ6h3r7lOSpiRpdHTUu1jeRIIg0PJjT8n97pVaq3DN82P9LgqAdO1y93VLWJrZw2kUBgCAdrSbZHc6FhrtYxx557LQ8yONBL3RWsBDtRvcfcbMZGZDUUt70OjYtFXXPL84fTocg351njXPgcE03WD7F/taCgAA2tRJkt2NsdBoH+PI25eVnh9pJOiLWp9gN0ra5e4jZnbIzBa02vJe29KeCax5DkCSN5gQblzSl1IoDwAALekkyWYsNPIoSz0/0kjQ6ybX0YzHdbn7BWllkriFrM7iLlFrBUDHJJWih5pqvNsqqSQSdABADnSSZDMWunNJu1lnoTt2UWSp50ffE3R3L5vZSot5lHTP1LxerCbhZvamu98c7Z5UOBMyAGTVsKQvaO1cGybpRDrFAQCgNZ0m2YyFbl/SbtZZ6Y5dFFnq+ZHWMmvjZnZCq+ugj8f2nVI4hnMqen3SzA4p7Ab/arU1HQAyatzdK7UbzexaGoUBAKAdnSbZ9CptXdJu1lnqjl0UWer5kUqCHnVnr3Zpn6nZd7jm9ZQAICfcvRKtgT4macndXzCzW+sl7QAAZBlJdn8l7Wadpe7YRZKVnh/rlgICALTPzPYrrID8dYU9hCTpipkdTK9UAAAg64Ig0Nzy23IPV4yudrMOarpZJz0OrSuVSrr33ntTrehoqwXdzL7o7g90uzAAUACBu39QWknW5e5vWbWaGwAAoI6k3ayz1B0b3dduF/dJM3td0rPu/sNuFihPmDkRQB2vN9juSd4cTZR5SKtzdEw1W7nCzKbd/UDNtlMKJ6ZbkjQradLdM7lEJQAACCXtZp2V7tjovnYT9MPu/ryZ7TezIUmvu/s3u1mwrDs6eVwXX3lNm7bt0fJjT+nO2/fq/NkzaRcLQPpuM7M5d/+uoqTczHZIuk3S1xK8/zl3H4neNyvpnKTD9Q40szGFs8aP1dn9qrvTag8AQM4kHftflDkCaPRcq60E3d2fj//fzG41sz9V+GBZ+Pn9K5WKLr7ymj584P5oGv67dXH6NDMnApCkL0qaMzOXtBTr2b6/2RvNLJC0WH3t7ktREl6Xu89E7zvVUYkBAABSwHJx67U1SZyZ3Vr9v5k9K+kFSVcUPpR+quiTIZXLZW3atmfNzImbtu1RuVxu8k4ARefub0Vj0B+Q9KykL7j7RxMOBxrW2vXTJWkx6vbeqmEzO2RmY2Z2KurtBAAAkAnx5eI+tWOnHtq9R/PRcnGDrN1Z3C+Y2V9Jek7StLtvcfdj7l5x96+6+9fM7GgXy5kpQRBo+er8mpkTl6/OM3MiAFVjn7tfcPfT1Z5G0b4d1QrOBrY02N5Ocj0VlWFG0jMK43VtWSfMbNbMZt944402PgIAgOQqlYqefPLJgU/AENpoubhB1sk66MfiD55xZvaJDs6beaVSSXfevlcXp0+HY9CvzuvO2/fSvR2AJNUd921mn1LYqv5qNKlbvfHoi1qfjDdK2jcUn1jO3cv1usq7+5SkKUkaHR1NNIldI4wfA4D8SCNm05UZtYIg0NPLb+ugezRsOFwu7tMD3ujZboJ+slFyHhmRNNfmuXPh/NkzseB2jAdSACvM7OsK4+Cz7v6ZaPOEpKPu/k0zu7/BW+vOsu7uLVUlR2PZz1Unm+u18eMTulR5WZt3b9X1849qX+kOnTsz1Y+PBgC0KI1EOd6V2cx00F0PRl2ZeYYeXCwXV1+7k8R9Nf7azG6Nz+Lu7qc7LVgeFGXmRABdtUvSKYXJdmBmB6PW8l1aTcDrLpsWtXSvtJhHY89nal4vNlt2Lfqcs7H3jUm60Ma1NFWpVHSp8rL2fu5jYe33Xa5Lj7/EQxcAZFBaifJGXZn5WzHYWC5uvXYniasdX/5WtORaYcedA0BC33D35939Sk1lpscmituoO/m4mZ2IkupDksZj+05Juqf6wswCMzshaSiaCG4s+qAlSQvRGPMJSQdqztM15XJZm3dvXfPQtXn31oEfPwYAWZTWmN8gCDS3/Paa+Zvmlt9uOH8TY9Wzox8/i1KppHvvvXdNcj7I90C7XdzXjJF09yuSrpCgA0C4DrqkNxWuT15tEd9qZpvd/bo2mPQt6s5efVKaqdl3uMGxj9Q5z0zttl4IgkDXzz8qv2t1/Nj1y9cU3DfY48cAIIvSGvO7UVfm2vHwjFXvvnbnHEjrZzHo90DiBN3MxhWOqbxZYbfNj9YcMixpVtLgfHsAsN5ZSV+VVFLYrfxZM3tY0klJ/8nMXpX0Vorl66pSqaR9pTt06fGXwjHol69pX+kOuqgBQAalOea3Xlfm2kTsPR/5Of39t/4PY9W7qN1kN63hEMxX0EKC7u7nJJ0zs69Iqihc3zduwd0Hrw8CAMREPYpGazZ/VZLMrCRpNIqnhXHuzNRq7fx9jB8DgCxLc8xvfP6meonYf/yzl/Wr730fY9W7pJNkN615A5ivoL0u7icljdVOFAcAkMzsoMIKy2/W7osqMQtZkcmkmQDQX50slZaFmF0vEbvtppv1ypvXdPADO1l2qws6SXbTGg7Rjc/N+9KvLSfo7v6WotagWmZ21N3p4g5gkP26pN+t3WhmN8YmiQMAoG1FGKNbLxH7a/+xdo6O6sG/mM/0slt5SQA7SXbTGg7R6ecW4XcjUYJuZl+WdDZav7ck6eF6hykco56vbwAAuusZ1V/PfELSl/pcFgBAwRRljG6jRKzTLvi9Tp67nQD2srydJrtpDYdo93M3+t2QlItKFSl5C7rVvH5LsTV2Y8ec6LhEAJBvByQ9bGYLWl3v3CTtFwk6APSEmQ0rXJqyLCmQNBUtOVnv2EDhXCFDkj4q6aS7L9TsW1Q4AfKF6r6sKNIY3UaJWLtd8HvdetrtypF+tPZ2mmSnNRyinc9t9Ltx8vO/qX/8/vdz06qeKEF392OxlwsKA9mV2uPM7Fq3CgYAOTWqcNmzxZrtDZdWAwB07Dl3H5EkM5uVdE7S4dqDzGxI4WSdU9HrMUnTknZFh4y5+yOx489Kmuxx2VuS1tjgXkmaiDVrae5Hz4JuVo70sydEFuYc6Id6vxuX3vyB/vUPl/Twz9+amx4n7Y5Bb7RE0H4VdAKkvIw1AZC6k+7+fO3GqEUdANBlUav3SqWouy9FiXc9wwonPJ6KXs9KGjazoajFfdLMGra+R583oXDYkrZv396NS2hJr8YGZ/lZN0lLc6vJczvX22rlyEafkeeeEFm9V+r9btz4gQ+o9MN/yNX3nHQM+rjCbkMbHqZwDHrhunAenTyui6+8pk3b9mj5sad05+17df7smbSLBSCD3P15M7tR0pikJXd/wcxuZRlKAOiZYa0OKapaNLPh2u7p7l42s5HYplGFsbr6/lOSrpjZyej1SdWIWt+nJGl0dNS7cQHN1CZE3R4bnOWJtZK2NLeSPLd7va1UjjT7jLz2hMjyvSKt79IvSZ+/50iuvuekLei7JF1Q/YmPqgo5Br1SqejiK6/pwwfuj36od+vi9OlMd4sAkB4z269wjo6ypG9IekHhw95Bd/9aqoUDgAwws/slbWnz7dfcvbYxqNG56g4tqmkdn5Q0Hts3FXWDr3Zrn9H65L+vGiVE3eq2nPVJ5zZqaa7ur1ZcJEmeO73eJJUjST4jrVnSO5H1e6Wq9ncjb99z0gT9mSStP0Ucg14ul7Vp2541QWHTtj2Z7hYBIFWBu39QWknW5e5vWTWIAACmmh/SkkWtT8abVgBEXdWfcfcLsW0nojHoj0T74+PT+y5v46p7oVFL89/+0R/p6YdPrau4aJY8d+N661WOxHs5JP2MtGZJb1fW75VG8vY9J50kbl1yPihdOIMg0PJjT8n97pWgsHx1XkFwrPmbAQyi1xts70s3SADIumg+o26q28PT3cuN3hCNUV9w95mabSvviVrTd5lZsNG5eqkfCVHWu1rXa2l+z0d+Tt//i/mGFRcbfTe9uN7aXg7v+cjP6e8TfkaeJnDL+r2ykU6+536PuW95kjhpsLpwlkol3Xn7Xl2cPh2OQb86rztv35ubXyQAfXebmc25+3cVJeVmtkPSbZIKFR8BoJuixp5vtvq+aFz5Sot5tOTaTM3rxWrX9uqkctWk28wORa3oiwobn2Zqz9/O9XRDPxKiPHS1rm0BLZfL+taV77U9IVw3r7duL4dvzet9P79HD/7FfGa/03akfa+kMTldGmPu20rQNWBdOM+fPRO7IY7l/pcLQE89LGnWzFzSUiws7k+vSACQC2+Z2VFJs9VE3cyOunuSp+FxMzuh1XXQx2P7Tinsqj4VJevPSxqKxecFheudl81sOOraLoXd5p/p+Ko60K+EKA9dgGtbQDudEK5b19uol8NHfuVXFPzO72T6O21HWvdKGolyWmPu203QO+rCGQXHQ1oNog2Xs4hqOUcV1moOKwygfV+uKE/dTwCkJ4plHzSzQ5J2SirXW3YNALDONUlXJB0xsyMKnzdHJDV9Co9auast3bUt4Idj/16QdPMG57nQaF9aWkmIOmlhzFMX4G5NCNeNsm7Uy6Go+UO/r6tXiXKz+zatMfftJuidduF8zt1HovfNSjon6XCDY8eiyToUHX9WqzNrAkCmmNmX3f0zWXzIA4Csig2ffF1SdYx6IJ75JCVLiNJa/iqtz+3XhHDNpN3tu11prWXezuf24ueY5L7dqPKll99fuwn6FyXNtdOFszrup/ra3ZeiSTkamTSzhi3sAJAxR8xsTtKz7v7DtAsDADmxMnyyysx2qsFSaVgrra64aS+7lcaEcPXkYYhAXN4qc7r9c0x63zaqfDn/xBM9/f7aStCj2Tfb7cI5rPXrSS6a2XCDruunFE5AdzJ6fbLOMdWlMiYkafv27QmLAgBdd9jdnzez/dFwntfd/YW0CwUAGbdu+KS7XzGzg5Janjhu0LSyVni/PjcLSWo/W7fz0p09j5U53f45tnLf1la+SNLn7znS0++v3RZ0SevH6ZjZjQlajBqtS1m3hjRa4mJIq12cZrQ+wZe7TylaV3N0dJTljACkolpZGf3/eTPbaWZ/qnBoT++rpwEgn3Y1eI6kB2UCra4V3uvPzdKyW3lr3e61flaqtLM2fCPd/Dm2et/GK1+efPLJnn9/HSXodZyTdKTJMYtan4w3StplZieiMeiPRK3k05J2dVRKAOiR6jJBZnarpGOS7lFYsTibbskAINOmJL1gZg+5+/+MbR9WuJwvNtDOWuG9+twsjr/OS+t2P/SrUqWTteEb6dbPsZP7th/fX9ME3cxukvSmEs7QnkDdGdjrrTEZjU0vx46ZMrNdZhakuSYlAGzggpm9qbAi8qykndGwIABAA9FyvfdIetbMLih8/htSg6GNWK/TtcK79bkkwtnWq0qVeGu5pMyvDd/ufduPSqmmCXoUMC+4+z3VbdFMmwvufiW2LVA4Hr3Z+cpmttJiHo3RnKl5vRhNCrcoaUzrl8sgOQeQZV9od2m1VpahjL1n2t0PdHoeAEhTNBfRaBS/SgrnOLrS5G2IaXet8G5/LrKt25Uqta3l73jve3KxNny7922vK6WSdnEfr3l9U23AjBLvpDNtjpvZCa0+OMbPf0phN/ap6JzDUdd2KaxJfSbhZwBAGk52uO554mUoo15GwworMts+DwBkSZSo1+1xmWdZXSscg6lblSr1Jn/73Ddn9Y2feGeh14bv5XUkStBb6J6ZKEGPWsCrreC1reOHa16zljCA3HD3r9bbbmZHm00S1+oylO4+E73vVCfnAYB+MrP7tcH8Q01cc/cvdbM8/ZDltcKBTtSb/G3fze/W/NCNevBy77uzp7Weey+1O0ncrupESNUN0YRIt0n6WldKBgA5YGZflnQ2mhiuJOnheodJGpHU7Gms1WUoe30eAOiFqbQL0E9ZXysc/VW0hLLRpGmP/fcnJamnlUNpVXz1WrvroJ82s6+b2U6FD4HDCrsi7e9m4QAgB6zm9VsKJ4erPeZEgnO1tAxlp+eJhg9NSNL27dtb/IjmivYQAqA7Bm3izKyvFY7+KWJC2WwoRa/u8bQrvnqp7WXW3P2uqBvliKRZd690r1gAkA/ufiz2ckHhGPR1kxqZ2bUEp2tpGcpOz+PuU4paskZHR7u1Uockafz4hC5VXtbm3Vt1/fyj2le6Q+fODFSjGQBIysda4ei9IieUaQylKHLFV0froNeMJU80xhIAiipqFarbMpSwEjPxMpR9Ok9bKpWKLlVe1t7PfSx8GL3LdenxlwrxEAIArWKyNkjFTiil/g+lKHLFV6IEvctjLAGgcMxsh8JVKAKtjgGfkfS77v5aknO0uAxl2+fptXK5rM27t655CNm8e2thHkIAoFVM1oYiJ5RpKHLFV9IW9G6OsQSAQjGzcYUx8YLCLuNLCruY75L0v8zsIXf/bwlPl2gZyuhzA4VLrA1FM7lPV2d2b3KengqCQNfPPyq/a/Uh5Prlawru4yEEwOBisrbBVuSEMi1Frfgy99aGHZrZTZK2NBhjWcrCWPTR0VGfnZ1NuxgAcsDM5tx9tIP371c47vyuDY55VtJX3P2Fdj+n17odN9eMQb98jTHoQIF0GjeLhGdOtIoJVAdX0tiZtIv7DoVdNockLcSXV4vLQnIOAH12aKPkXJLc/Z5oqFBmE/RuO3dmavUh5D4eQgAAkOhJgeaSdnF/XdIkE8ABwDpJlwsaqGWFJB5CAAAAWvWOhMd9leQcAOpKsnxaK8cBAABgQCVN0F+t/sPM9pvZV8zsr8zsy2b2iR6VDQDyIOlEHl1dZxwAAADF0/I66O7+vKTnzexZd/9MfJ+Z7XD373SrcGlh8gYALTgSLSe20dJnQ5KOSPpSX0oEAACAXEqaoNdr+flGnW2HlPMH0KOTx3Xxlde0adseLT/2lO68fa/Onz2TdrEAZNeIpJvVPEHf2Z/iAACAoqEBcXAkTdCrLURxB+psm1SOE/RKpaKLr7ymDx8m/WEDAAAUv0lEQVS4P1q7925dnD6tSqXCLwKARh5x9y80O8jMHu5HYQAAQLF89uhRzUfrpz8drZ/+xHmmByuqpAl6tYWo1q6a17luISqXy9q0bY+qFQ9mpk3b9qhcLpOgA2jkbJePAwAAkBQ2IM5felEP7Q5zlIPuevDSizQgFljSSeIecfcPNvtP0uleFrbXgiDQ8tV5uYc9+t1dy1fnFQRByiUDkFXufqWbxwEAAFSVy2WNbLphTQPiyKYbVC6XUy4ZeiVpC/pAtBCVSiXdefteXZw+HY5BvzqvO2/fS+0UAAAAgL4LgkBPL7+tg+7REFzX3PLb+jQNiIWVKEEfpBai82fPxCZhOEZyDgAAACAVpVJJe/Z9XA9GY9DnojHo5CjF1fIya4OgVCpx0wNADzALLQAArXni/PmVv5+f5u9n4ZGgAwD6Yvz4hC5VXtbm3Vt1/fyj2le6Q+fOTKVdLAAAMo8GxMFBgg4A6LlKpaJLlZe193MfC8fQ3eW69PhLzEILADXoaQQMNhJ0AOgRM/uUpGFJByT92N1/KeUipaZcLmvz7q1rZqHdvHsry1gCQAzrXQNIuswaAKBF7v5Vdz8t6Zik29IuT5qCIND1y9fWLGN5/fI1lrEEgEh8vetP7diph3bv0Xy03jWAwUELOgD0mLsvmNlM2uVIU6lU0r7SHbr0+EvhGPTL17SvdAet5wAQ2Wi9a2IlMDhI0AGgA2a2w92/k+DQV3tdlqw7d2ZqdWzlfYytBIA41rsGIJGgA0CnDkn6UoLjvNcFyQNmoQWA+ljvGoBEgg4AnZo0M5dkTY47omSJPABgQLHeNYBUEnQzG1bY6lSWFEiacvelBse+KWkoerkU/fukuz/Sj7ICQBO7JH0mwXE7e10QAED+0dMIGGxptaA/5+4jkmRms5LOSTpce5CZDUk67O4zsW0T7j7Vt5ICwMYecfcvNDvIzB7uR2HyiDV/AQAAQn1P0M0skLRYfe3uS2Y21uj4muT8kKSBngkZQOac7eZxLfYwanismZ2SdEJhz6NZSZPuvpCwrH0zfnxClyovhzO7n39U+0p36NwZ6mABtKbF2BlIGlXYK/OjCntmLsT2H4of7+4XelVuAKiVRgv6sMIHxrhFMxuufXiMB9aoNX1LowdMM5uQNCFJ27dv726JAaCxawmPuynhcYl6GCU49lV3bzYuPlWVSkWXKi9r7+c+Fs5YfJfr0uMvqVKp0JIOoFWt9M4crfbGjBqJphUOV5KZnZC04O4XomOfl0SCDqBv3pHCZ25psH2owfaqByQ922inu0+5+6i7j95yyy1tFw4AWnQq4XEPNDugXg8jSXV7GLVybFaVy2Vt3r11zZq/m3dvVblcTrlkAPKkxXg4LOlk7PWspOEoGZekB6ot5u6+VE36az5vwsxmzWz2jTfe6Mo1AEBVGgn6otYn442S9rixRl2VACBFk2b2gUY7zexGM3tWYdfLZhr2MGrj2GEzO2RmY2Z2KvbwGS9bqg+ZQRDo+uVrcg9XoHN3Xb98TQFr/gJoTeLY6e5lSfGke1TSUmzI5UIsdp5ocA4ahQD0TBpd3Ot2UY8CZl1RwFxstB8AUnRO0mEzu+Du34nvMLOD0f4rkioJztVKD6Nmx8bHoy9Kek7SgfiBURfPKUkaHR3t+zrtpVJJ+0p36NLjL4Vj0C9f077SHXRvB9Cqlnpn1jT4TEoaj/49LCmotqBHXeXnFHV/B4B+6HuC7u5lM1sJpFHN5EzN68Wa4Blofc0oAKTO3SclyczGzewZd/+hmd2ocMzifknH3P2cmSVZZq2VHkYbHhuPoVHczWT393NnplZncb9vdRZ3ZnYH0IK2emdG8xc9E5sEbkGxhqSoVX243jxJANAraS2zNh5NwlGdaXM8tu+Uwsk64tP4LqlByzsAZEGUhI+b2U0K49iMwokt34r2X0lwmlZ6GDU8NhqPea7e2Mksql3zl5ndAbSo3d6ZC/HVghqchwYiAH2VSoIeBcxq0Jyp2bduxk3WPQeQB1GS/rCkL7j76Tben7iHUZNjFxRb1i16EM3FLMTM7A6gVa32zqxOKldN4M3skLtfcPcFM6tdQWiB1nMA/ZTGJHEAUBhm9tvx1+7+BUlvRt3cGx63gfFoYqIxhRPL1fYwuqfZsdFD6EI0CdyEwrHn8fNkFjO7A2hTotgZJevPS5ozMzcz19rVOA5HE2tOKFx9o9EylwDQE2l1cQeAopiMHvBq1xx/zsym48dJ+lKzk7XSw6jJsWte50UQBLp+/lH5XR62oFdndr8vYFw6gIaSxs6oNfzmDc6zoLXLsAFAX5GgA0Bndkn6zAb7qpJMEjfwGs3sfubclxmXDgAACo8EHQA680jUrX1DZnZ/PwpTBLUzu0vSkeO/wbh0AABQeCToANCZs80PkZSTSdqyIj6z+5NPPrnhuHS6vQMAgKJgkjgA6EDC5dMSH4f1giDQ9cvX5O6StDIu/Y++/ic6cvw3dObl39eR47+h8eMTKZcUAACgM7SgAwAyrd649I+8/0Oa/97lut3eJVrVAQBAPpGgAwAyr3Zcerlc1vde/sG6bu+/ef9v6fv/cHXdZHLMAA8AAPKABB0AkAvxcemS1i3H9sbs32jpp96l4Lf+/ZpW9YNHDmn+e5dJ2gEAQOaRoAMAcqdet/cdW9+vf9ipNa3qP7HtJzX3VxXdduJAoqQdAAAgTSToAIBcarQcW7xVfXH+7/T+j+9KlLQ3Gr9OSzsAAOgXEnQAQG7VdnuvbVW/PfhFfevbfyn/5MZJe6Px65J0qfJyou7xJPIAAKBTJOgAgMKobVUvlUoaPz7RNGmvN379T3/363pXwjHtUvJEHgAAoBESdABAodS2qidJ2uuNX7cb36mbfnZb0+7xrSTyrbS+d5LcUzEAAEA+kaADAAqvWdIurR+/7j/8Z731ratNu8cnTeRbaX2vty1pcj9+fILWfAAAcooEHQAwkJqNX//kvxuTpKbd45Mm8klb3zvpWv+R939I3/qbv9Tez32s5635jMMHAKD7SNABAFD9rvCSmnaPT5rIJ21976Rr/YsP/mHiWes7ac1Puq1flQBUFgAAioIEHQCASG2rer1t7SbySVvfO+lav2XPe/WD176v4U9+pGet+d1u9e/Xtm5XFnS7oqDbFQpUUABAPg18gs4fMABAq9pN5JO0vnfStf5frv6jRn62pNd62Jrf7Vb/fmzrdmXBu35k+qef8q5VFLQyb0C35yHgOShdfP8Aag10gn508rguvvKaNm3bo+XHntKdt+/V+bNn0i4WAKAg2m19r7ctSXK/LhHrQWt+t1v9+7Gtm5UFiz/7hv78D/5M+07+h65UFLQyb4DUvBKg0/OdOzPVi18F1PHZo0c1f+lFjWy6QU8vv609+z6uJ86fT7tYAFI2sAl6pVLRxVde04cP3B89TNyti9OnValUqMEEAPRMktb3ettaSe572Zrf7Vb/fmzrZmXBW9/5gX7mtp1dqyhIOm9A0kqATs536fGXeA7qk0qlovlLL+qh3XtkZjrorgcvvcj3D2BwE/RyuaxN2/as+QO2adselctlAiMAIJOSJvdJ3ttJa37Sbb2uBEijsuCmHe/Wn//Bn+mDv7SnKxUFSecNSFoJ0Mn5Nu/eynNQn5TLZY1sumHN9z+y6Qa+fwCDm6AHQaDlx56S+90rf8CWr84rCI6lXTQAAPqi3db8pNv6UQmQdFs3Kwtu+cmb14zz76SiIOm8AUkrATo53/XL1xTcF7R2E6EtQRDo6eW3ddBXv/+55bf16YDvHxh05u5pl6HrRkdHfXZ2tulxa8agX51nDDowgMxszt1H0y5HlZkNSzokqSwpkDTl7kutHtvKeaTkcRPIsyzM4r5mArd68wZscJykRNs6OV+SMehZi5tp6iR2xsegzzEGHSi8pLFzoBN0idkzgUGXtQfNqDwj0b+HJJ1z98OtHtvKeSQSdKCfOll6rdtLubXzHJS1uJmmTmMnz6HA4CBB50ETQAJZetA0s0DSKXc/ENv2prvf3MqxrZynirgJIKksxc20ETsBJJU0dr6jH4UBACQyLKm2G/pi1F29lWMTncfMJsxs1sxm33jjjU7KDQAAgC4gQQeA7NjSYPtQi8cmOo+7T7n7qLuP3nLLLQmLCAAAgF4p5Czuc3NzPzCz7zbY/W5JP+hneXqkCNdRhGuQuI6safU6PtCrgrRhUeuT8UbJ9kbHtnIeSU3jplSM+6MI1yBxHVlShGuQ8h03U8UzZ64U4TqKcA3S4F5HothZyATd3Rs2BZnZbBHGTRXhOopwDRLXkTU5v46FehvdvdzKsdV1dROep7pvwyb0nH+vkopxDRLXkSVFuAapONeRBp4586MI11GEa5C4jmbo4g4AGREl0Cst3dGY8Zn462hG9g2PbXYeAAAAZFMhW9ABIMfGzeyEVtcvH4/tOyVpWtJUgmM32gcAAIAMGsQEfar5IblQhOsowjVIXEfW5Po6otbvalf0mZp9h1s4tuG+NuX6e40U4RokriNLinANUnGuI2uK8r1yHdlRhGuQuI4NFXIddAAAAAAA8oYx6AAAAAAAZAAJOgAAAAAAGUCCDgAAAABABpCgo+/M7GzN62EzO2FmY9H/h9IqG7LNzKbrbGt4/3BvoSiIm+gEsRODitiJTqQVOwdikrhoDeBDWl1uaMrdl9ItVTJmFkgalTQk6aOSTrr7QrQvd9cVXc/z7n5zbNucu49E/x6SdK52tuqsMLND8dfufiHanpufReyeWpQ0LOlC1u8pMxtTWNaz7m41+xreP3m6t7Ioq/dDM8TN7CF2poPYmY6s3g/NEDuzJ++xM49xU8pA7HT3wv8naS727yFJz6VdpoTlHpI0EXs9Jun1vF5XVMZA0puxbYGk6Zrj3uxnuVoo/wlJh2LXEv/+c/OzkHSi5vXZvFxH7b2x0f2Tp3srq/9l/X5oUGbiZsb+I3am/x+xs+/fd6bvhwZlJnZm7L8ixM48x81690e/Ymfhu7hHNTeL1dce1syMpVeilgxLOhl7PStp2MyGcnpdYx6uzRw3LKm2tmwxqlXLmgc8qrl09yVfrSHL289isl6Xmxxeh7Tx/ZOneytzcno/SMTNLCJ2Zg+xs0dyej9IxM4sKkLsLFLclPoUOwufoCt/v4wrosAyEts0KmkpuolzdV1RV5GZOru2NHhLpsYEReVfMLNDsXEl1e86Vz8LSackXTGzCTOb0Oof5Lxdh7Tx/ZOLeyvD8ng/EDczhtiZWcTO3snj/UDszJgCxc4ixU2pT7Hzna2+IYdy88tYj68dizEpaTz6d26uK6o5W/T640oWtb7Mja4tTcOSAl8d+zMraU7SLuXoZyFJ7j4V/Uwmo00zCoNkrq4jstH9k5d7K6vyeD9IIm5mDLEzm4idvZPH+0ESsTNjChE7CxY3pT7FzkFI0PP0y9hQVOv0TPUXVfm6rjFpZTIISRqKrmdG0kK9N9TplpS2BcXK6u5L0UyNw8rXz0JmdsLdH5H0SPRzmFYY8HN1HZGG94+Z1duVxXsrq/J4P6xB3MwEYmc2ETt7J4/3wxrEzkwoROwsWNyU+hQ7ByFBz9MvY13Vbi7uHu+uk5vrigV4SZKZyd2nYq+3xP49rPrdktJW7/te2mBfJn8W0b20Uq6oZnNXNBYoN9dRFQXEuvfPRvuQSO7uhzjiZmYQOzOI2NlTubsf4oidmZH72Fm0uCn1L3YWPkHP+x+a6iQK1RvWzA65+4U8XlfUxWUi+vcJrS61MB69ri6zMN74LOlw9wUzW+kuFV3Lgq8uFZGXn8WiwtrlNeWL3V+ZvI7o92BMYU34KYWzZFbLttH9k/l7K6vyGGOqiJvZQexMF7Gz//IYZ6qIndlRkNiZy7gppR87B2Ud9OqXnLl19jYS3axzWtsFZMHdd0X7c3ldeRX9PCYlva6we87ZWKDMzc/CwjU1q0FxSNJMLFjm5jrQe3m8H4ib2UPsxKDJ4/1A7MyeIsRO4mZ7BiJBBwAAAAAg6wZhmTUAAAAAADKPBB0AAAAAgAwgQQcAAAAAIANI0AEAAAAAyAASdAAAAAAAMoAEHQAAAACADCBBBwAAAAAgA96ZdgGAWmY2LGlS0glJZUnPxHbvkjQhadLdp3pcjkDSEUmv9/qzAKBTxE4AaA1xE1lk7p52GYC6zOx1SWfd/ZGa7YGksdrtPSrDCUkH3P1Arz8LALqB2AkArSFuIkvo4o7ccfeypKU+fVy5T58DAD1F7ASA1hA3kQYSdOSGmY3FXi6kVhAAyBFiJwC0hriJNDEGHXlyQNKMJLn7jLQSQE8pDJ7TkhYlDUtaio/hMbMhheOIqkF2uE43plOSXlVYUzrk7hdi+6qB+oAYHwQgX4idANAa4iZSwxh0ZFY0HqgaBHdJmnB3q3PcIUmn3H1XbNtzkqarQc3M5iTtd/el6PWYpMPuPhm9npZ0MurKJDN7U9J+SVskPSdpxN0Xon1erxwAkAXETgBoDXETWUILOrJuulrraGYbjQGqHbdzVmGQm6rWRFYDZfTvGTObNrOTCms/R6uBMjLi7gvRexergbLKzIbi5wOAjCF2AkBriJvIBMagI09Wlr4ws7GoC1EjC5Kq+wOF3ZBqLUkajf5bEwxrgmPt2COCJIA8IXYCQGuIm0gNCTpyo6a2MWhSmzis1aC2oLDbUK0hSbPRf8NdKSQAZAyxEwBaQ9xEmkjQkTtRLeZHazYHNa8nJZ2UpOrEG2a2EhCjMUQX3H0pCsLVrkXV/cPR2pfS+kC7US0qAGQSsRMAWkPcRBoYg47MiYLapMIaxiNma+bG2CXpHknP1rxtIQqASwoD58pkHZH9kh4ws1cVBr8hdz9cs/9U9NmL0XkWFQbcwMxOSJqS9EB0/CkzO1U7TggA0kLsBIDWEDeRRczijtyLguSRmuAHANgAsRMAWkPcRD/QxR1FQRcgAGgdsRMAWkPcRE+RoCPXojE7k5JGzWwi7fIAQB4QOwGgNcRN9Atd3AEAAAAAyABa0AEAAAAAyAASdAAAAAAAMoAEHQAAAACADCBBBwAAAAAgA0jQAQAAAADIABJ0AAAAAAAygAQdAAAAAIAM+P+Dglem0YapTAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1008x216 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axs = plt.subplots(nrows=1, ncols=3, figsize=(14, 3))\n",
    "ax = axs[0]\n",
    "ax.plot(epoch, fidelities, \"o\", color=\"C0\", markeredgecolor=\"black\")\n",
    "ax.set_ylabel(r\"Fidelity\")\n",
    "ax.set_xlabel(r\"Epoch\")\n",
    "\n",
    "ax = axs[1]\n",
    "ax.plot(epoch, KLs, \"o\", color=\"C1\", markeredgecolor=\"black\")\n",
    "ax.set_ylabel(r\"KL Divergence\")\n",
    "ax.set_xlabel(r\"Epoch\")\n",
    "\n",
    "ax = axs[2]\n",
    "ax.plot(epoch, coeffs, \"o\", color=\"C2\", markeredgecolor=\"black\")\n",
    "ax.set_ylabel(r\"$\\vert\\alpha\\vert$\")\n",
    "ax.set_xlabel(r\"Epoch\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"complex_fid_KL.pdf\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It should be noted that one could have just ran `nn_state.fit(train_samples)` and just used the default hyperparameters and no training evaluators.\n",
    "\n",
    "At the end of the training process, the network parameters (the weights, visible biases and hidden biases) are stored in the `ComplexWaveFunction` object. One can save them to a pickle file, which will be called `saved_params.pt`, with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "nn_state.save(\"saved_params.pt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This saves the weights, visible biases and hidden biases as torch tensors with the following keys: \"weights\", \"visible_bias\", \"hidden_bias\"."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}