{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Running EasyVVUQ on HPC resources with QCG-PilotJob" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Author**: Bartosz Bosak, PSNC (bbosak@man.poznan.pl)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If this is your first Jupyter Notebook - you can execute code cells by selecting them and pressing ```Shift+Enter```. Just have in mind that the order of execution might matter (if later cells depend on things done in earlier ones)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As defined in the [VECMA glossary](https://wiki.vecma.eu/glossary) , uncertainty quantification UQ is a “discipline, which seeks to estimate the \n", "uncertainty in the model input and output parameters, to analyse the sources of these uncertainties, \n", "and to reduce their quantities.” However, this process can quickly become cumbersome because just \n", "a few uncertain inputs could require hundreds or even thousands of samples. If a single sample is a demanding simulation, \n", "such a number of tasks cannot be performed effectively without (1) adequate \n", "computational resources, (2) a dedicated approach and (3) specialised programming solutions. \n", "\n", "EasyVVUQ has been designed as a modular software that can benefit from the solutions providing\n", "advanced capabilities for execution of demanding operations on computational resources. \n", "One of such solutions is [QCG-PilotJob](https://qcg-pilotjob.readthedocs.io), which allows to efficiently run a number of tasks inside \n", "a single large allocation on a Slurm cluster.\n", "\n", "In this tutorial, based on the scenario presented in the basic tutorial, we demonstrate how EasyVVUQ workflows \n", "can be adapted to enable their executions with QCG-PilotJob on HPC machines. \n", "As it will be shown the adaptation is quite easy. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisities\n", "You need to have EasyVVUQ installed in your environment. There is no need to install QCG-PilotJob's packages separatetly since they are installed as EasyVVUQ's dependencies. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting git+https://github.com/UCL-CCS/EasyVVUQ.git@qcgpj-executor\n", " Cloning https://github.com/UCL-CCS/EasyVVUQ.git (to revision qcgpj-executor) to /tmp/pip-req-build-1pet2s_e\n", " Running command git clone -q https://github.com/UCL-CCS/EasyVVUQ.git /tmp/pip-req-build-1pet2s_e\n", " Running command git checkout -b qcgpj-executor --track origin/qcgpj-executor\n", " Switched to a new branch 'qcgpj-executor'\n", " Branch 'qcgpj-executor' set up to track remote branch 'qcgpj-executor' from 'origin'.\n", "Collecting numpy\n", " Using cached numpy-1.20.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.3 MB)\n", "Collecting pandas\n", " Using cached pandas-1.2.4-cp37-cp37m-manylinux1_x86_64.whl (9.9 MB)\n", "Collecting scipy\n", " Using cached scipy-1.6.3-cp37-cp37m-manylinux1_x86_64.whl (27.4 MB)\n", "Requirement already satisfied: wheel in /opt/conda/lib/python3.7/site-packages (from easyvvuq==0.9.3+130.g4fb85615) (0.34.1)\n", "Collecting chaospy==4.3.2\n", " Using cached chaospy-4.3.2-py3-none-any.whl (247 kB)\n", "Processing /home/jovyan/.cache/pip/wheels/a1/31/12/e9aa7dc4ac7d4bae86f75769474a0f514d2339221e92e7187b/SALib-1.3.13-py2.py3-none-any.whl\n", "Collecting pytest\n", " Using cached pytest-6.2.4-py3-none-any.whl (280 kB)\n", "Processing /home/jovyan/.cache/pip/wheels/b1/f5/ee/9c072cedde5286a5f4b27e07269ad83a67878ff249794c8c97/pytest_pep8-1.0.6-py3-none-any.whl\n", "Collecting pytest-benchmark\n", " Using cached pytest_benchmark-3.4.1-py2.py3-none-any.whl (50 kB)\n", "Processing /home/jovyan/.cache/pip/wheels/7e/72/eb/c96a0b4b22f42d092914ba8fe7b4c639443ef02b529dbbefcf/pytest_dependency-0.5.1-py3-none-any.whl\n", "Requirement already satisfied: SQLAlchemy in /opt/conda/lib/python3.7/site-packages (from easyvvuq==0.9.3+130.g4fb85615) (1.3.13)\n", "Processing /home/jovyan/.cache/pip/wheels/6c/bd/4b/a3b7dcd0bc718e601a9ba7cba5f7c1d73fd777b6dc48d6eaa0/Cerberus-1.3.4-py3-none-any.whl\n", "Collecting dask[complete]\n", " Using cached dask-2021.6.0-py3-none-any.whl (965 kB)\n", "Collecting dask_jobqueue\n", " Using cached dask_jobqueue-0.7.2-py2.py3-none-any.whl (39 kB)\n", "Collecting cloudpickle\n", " Using cached cloudpickle-1.6.0-py3-none-any.whl (23 kB)\n", "Collecting scikit-learn\n", " Using cached scikit_learn-0.24.2-cp37-cp37m-manylinux2010_x86_64.whl (22.3 MB)\n", "Requirement already satisfied: jinja2 in /opt/conda/lib/python3.7/site-packages (from easyvvuq==0.9.3+130.g4fb85615) (2.11.0)\n", "Collecting kubernetes\n", " Using cached kubernetes-17.17.0-py3-none-any.whl (1.8 MB)\n", "Collecting autopep8\n", " Using cached autopep8-1.5.7-py2.py3-none-any.whl (45 kB)\n", "Collecting squarify\n", " Using cached squarify-0.4.3-py3-none-any.whl (4.3 kB)\n", "Collecting dill\n", " Using cached dill-0.3.3-py2.py3-none-any.whl (81 kB)\n", "Requirement already satisfied: tqdm in /opt/conda/lib/python3.7/site-packages (from easyvvuq==0.9.3+130.g4fb85615) (4.42.0)\n", "Collecting qcg-pilotjob@ git+https://github.com/vecma-project/QCG-PilotJob.git@namespaces#subdirectory=components/core&egg=qcg-pilotjob\n", " Cloning https://github.com/vecma-project/QCG-PilotJob.git (to revision namespaces) to /tmp/pip-install-ifzs9czc/qcg-pilotjob\n", " Running command git clone -q https://github.com/vecma-project/QCG-PilotJob.git /tmp/pip-install-ifzs9czc/qcg-pilotjob\n", " Running command git checkout -b namespaces --track origin/namespaces\n", " Switched to a new branch 'namespaces'\n", " Branch 'namespaces' set up to track remote branch 'namespaces' from 'origin'.\n", " Running command git submodule update --init --recursive -q\n", "Collecting qcg-pilotjob-executor-api@ git+https://github.com/vecma-project/QCG-PilotJob.git@namespaces#subdirectory=components/executor_api&egg=qcg-pilotjob-executor-api\n", " Cloning https://github.com/vecma-project/QCG-PilotJob.git (to revision namespaces) to /tmp/pip-install-ifzs9czc/qcg-pilotjob-executor-api\n", " Running command git clone -q https://github.com/vecma-project/QCG-PilotJob.git /tmp/pip-install-ifzs9czc/qcg-pilotjob-executor-api\n", " Running command git checkout -b namespaces --track origin/namespaces\n", " Switched to a new branch 'namespaces'\n", " Branch 'namespaces' set up to track remote branch 'namespaces' from 'origin'.\n", " Running command git submodule update --init --recursive -q\n", "Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from pandas->easyvvuq==0.9.3+130.g4fb85615) (2.8.1)\n", "Collecting pytz>=2017.3\n", " Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)\n", "Collecting numpoly<2.0.0,>=1.1.2\n", " Using cached numpoly-1.2.3-py3-none-any.whl (147 kB)\n", "Collecting matplotlib\n", " Using cached matplotlib-3.4.2-cp37-cp37m-manylinux1_x86_64.whl (10.3 MB)\n", "Collecting iniconfig\n", " Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)\n", "Collecting packaging\n", " Using cached packaging-20.9-py2.py3-none-any.whl (40 kB)\n", "Collecting toml\n", " Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)\n", "Requirement already satisfied: attrs>=19.2.0 in /opt/conda/lib/python3.7/site-packages (from pytest->easyvvuq==0.9.3+130.g4fb85615) (19.3.0)\n", "Requirement already satisfied: importlib-metadata>=0.12; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from pytest->easyvvuq==0.9.3+130.g4fb85615) (1.5.0)\n", "Collecting pluggy<1.0.0a1,>=0.12\n", " Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)\n", "Collecting py>=1.8.2\n", " Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)\n", "Collecting pep8>=1.3\n", " Using cached pep8-1.7.1-py2.py3-none-any.whl (41 kB)\n", "Processing /home/jovyan/.cache/pip/wheels/76/fb/36/4304dce3f49d3aecf92d63f079db516641a8061a83a0b5a292/pytest_cache-1.0-py3-none-any.whl\n", "Processing /home/jovyan/.cache/pip/wheels/d2/f1/1f/041add21dc9c4220157f1bd2bd6afe1f1a49524c3396b94401/py_cpuinfo-8.0.0-py3-none-any.whl\n", "Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from cerberus->easyvvuq==0.9.3+130.g4fb85615) (45.1.0.post20200119)\n", "Collecting toolz>=0.8.2\n", " Using cached toolz-0.11.1-py3-none-any.whl (55 kB)\n", "Collecting pyyaml\n", " Using cached PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636 kB)\n", "Collecting fsspec>=0.6.0\n", " Using cached fsspec-2021.6.0-py3-none-any.whl (114 kB)\n", "Collecting partd>=0.3.10\n", " Using cached partd-1.2.0-py3-none-any.whl (19 kB)\n", "Processing /home/jovyan/.cache/pip/wheels/71/96/e1/80f7dbc16a958ed861b3c3ccd00b4c3b11951ff004856c398a/bokeh-2.3.2-py3-none-any.whl\n", "Collecting distributed==2021.06.0; extra == \"complete\"\n", " Using cached distributed-2021.6.0-py3-none-any.whl (715 kB)\n", "Collecting threadpoolctl>=2.0.0\n", " Using cached threadpoolctl-2.1.0-py3-none-any.whl (12 kB)\n", "Collecting joblib>=0.11\n", " Using cached joblib-1.0.1-py3-none-any.whl (303 kB)\n", "Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from jinja2->easyvvuq==0.9.3+130.g4fb85615) (1.1.1)\n", "Requirement already satisfied: urllib3>=1.24.2 in /opt/conda/lib/python3.7/site-packages (from kubernetes->easyvvuq==0.9.3+130.g4fb85615) (1.25.7)\n", "Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from kubernetes->easyvvuq==0.9.3+130.g4fb85615) (2.22.0)\n", "Collecting google-auth>=1.0.1\n", " Downloading google_auth-1.31.0-py2.py3-none-any.whl (147 kB)\n", "\u001b[K |████████████████████████████████| 147 kB 36 kB/s eta 0:00:01\n", "\u001b[?25hRequirement already satisfied: certifi>=14.05.14 in /opt/conda/lib/python3.7/site-packages (from kubernetes->easyvvuq==0.9.3+130.g4fb85615) (2019.11.28)\n", "Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0\n", " Using cached websocket_client-1.1.0-py2.py3-none-any.whl (68 kB)\n", "Requirement already satisfied: six>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from kubernetes->easyvvuq==0.9.3+130.g4fb85615) (1.14.0)\n", "Collecting requests-oauthlib\n", " Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)\n", "Collecting pycodestyle>=2.7.0\n", " Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)\n", "Processing /home/jovyan/.cache/pip/wheels/3d/19/38/5312d02df169de6d79ffa36d2096b3affd5cf8e44682acd749/zmq-0.0.0-py3-none-any.whl\n", "Collecting click\n", " Using cached click-8.0.1-py3-none-any.whl (97 kB)\n", "Collecting psutil\n", " Using cached psutil-5.8.0-cp37-cp37m-manylinux2010_x86_64.whl (296 kB)\n", "Collecting pillow>=6.2.0\n", " Using cached Pillow-8.2.0-cp37-cp37m-manylinux1_x86_64.whl (3.0 MB)\n", "Collecting cycler>=0.10\n", " Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)\n", "Collecting kiwisolver>=1.0.1\n", " Using cached kiwisolver-1.3.1-cp37-cp37m-manylinux1_x86_64.whl (1.1 MB)\n", "Collecting pyparsing>=2.2.1\n", " Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)\n", "Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata>=0.12; python_version < \"3.8\"->pytest->easyvvuq==0.9.3+130.g4fb85615) (2.1.0)\n", "Collecting execnet>=1.1.dev1\n", " Using cached execnet-1.8.1-py2.py3-none-any.whl (39 kB)\n", "Collecting locket\n", " Using cached locket-0.2.1-py2.py3-none-any.whl (4.1 kB)\n", "Requirement already satisfied: tornado>=5.1 in /opt/conda/lib/python3.7/site-packages (from bokeh!=2.0.0,>=1.0.0; extra == \"complete\"->dask[complete]->easyvvuq==0.9.3+130.g4fb85615) (6.0.3)\n", "Collecting typing-extensions>=3.7.4\n", " Using cached typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)\n", "Collecting sortedcontainers!=2.0.0,!=2.0.1\n", " Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)\n", "Collecting zict>=0.1.3\n", " Using cached zict-2.0.0-py3-none-any.whl (10 kB)\n", "Collecting msgpack>=0.6.0\n", " Using cached msgpack-1.0.2-cp37-cp37m-manylinux1_x86_64.whl (273 kB)\n", "Collecting tblib>=1.6.0\n", " Using cached tblib-1.7.0-py2.py3-none-any.whl (12 kB)\n", "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->kubernetes->easyvvuq==0.9.3+130.g4fb85615) (2.8)\n", "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->kubernetes->easyvvuq==0.9.3+130.g4fb85615) (3.0.4)\n", "Collecting rsa<5,>=3.1.4; python_version >= \"3.6\"\n", " Using cached rsa-4.7.2-py3-none-any.whl (34 kB)\n", "Collecting cachetools<5.0,>=2.0.0\n", " Using cached cachetools-4.2.2-py3-none-any.whl (11 kB)\n", "Collecting pyasn1-modules>=0.2.1\n", " Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)\n", "Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib->kubernetes->easyvvuq==0.9.3+130.g4fb85615) (3.0.1)\n", "Requirement already satisfied: pyzmq in /opt/conda/lib/python3.7/site-packages (from zmq->qcg-pilotjob@ git+https://github.com/vecma-project/QCG-PilotJob.git@namespaces#subdirectory=components/core&egg=qcg-pilotjob->easyvvuq==0.9.3+130.g4fb85615) (18.1.1)\n", "Collecting apipkg>=1.4\n", " Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)\n", "Collecting heapdict\n", " Using cached HeapDict-1.0.1-py3-none-any.whl (3.9 kB)\n", "Collecting pyasn1>=0.1.3\n", " Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)\n", "Building wheels for collected packages: easyvvuq, qcg-pilotjob, qcg-pilotjob-executor-api\n", " Building wheel for easyvvuq (setup.py) ... \u001b[?25ldone\n", "\u001b[?25h Created wheel for easyvvuq: filename=easyvvuq-0.9.3+130.g4fb85615-py3-none-any.whl size=198906 sha256=150ed31a9d236fad8cda2025d952ea5e894cf3d3198d252b53f7ee3668e35b41\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-h1eso0k0/wheels/ce/19/20/1358337eadfe29295d3cfe175f1dc8bac71f3dca2dd042f3dc\n", " Building wheel for qcg-pilotjob (setup.py) ... \u001b[?25ldone\n", "\u001b[?25h Created wheel for qcg-pilotjob: filename=qcg_pilotjob-0.10.0+145.g69df110-py3-none-any.whl size=128217 sha256=9f63fa317c3b8e96cc4de81c5a72d6a4b29b65354330f7b1b4bb878ed91938eb\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-h1eso0k0/wheels/cb/92/4a/ced119bd031bbd10a6ea12f8c5edc4d5195419b8075b936573\n", " Building wheel for qcg-pilotjob-executor-api (setup.py) ... \u001b[?25ldone\n", "\u001b[?25h Created wheel for qcg-pilotjob-executor-api: filename=qcg_pilotjob_executor_api-0.10.0+145.g69df110-py3-none-any.whl size=7178 sha256=4f3103c323265aebc6f596d28222aa1142699885aedab3f6bf63171e7adfde4f\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-h1eso0k0/wheels/fb/af/e1/b95264465e7c8b72eaa853b0838aabf5b99dd335d6b3e28550\n", "Successfully built easyvvuq qcg-pilotjob qcg-pilotjob-executor-api\n", "Installing collected packages: numpy, pytz, pandas, scipy, numpoly, chaospy, pillow, cycler, kiwisolver, pyparsing, matplotlib, SALib, iniconfig, packaging, toml, pluggy, py, pytest, pep8, apipkg, execnet, pytest-cache, pytest-pep8, py-cpuinfo, pytest-benchmark, pytest-dependency, cerberus, toolz, cloudpickle, pyyaml, fsspec, locket, partd, typing-extensions, bokeh, click, sortedcontainers, heapdict, zict, psutil, msgpack, tblib, distributed, dask, dask-jobqueue, threadpoolctl, joblib, scikit-learn, pyasn1, rsa, cachetools, pyasn1-modules, google-auth, websocket-client, requests-oauthlib, kubernetes, pycodestyle, autopep8, squarify, dill, zmq, qcg-pilotjob, qcg-pilotjob-executor-api, easyvvuq\n", "Successfully installed SALib-1.3.13 apipkg-1.5 autopep8-1.5.7 bokeh-2.3.2 cachetools-4.2.2 cerberus-1.3.4 chaospy-4.3.2 click-8.0.1 cloudpickle-1.6.0 cycler-0.10.0 dask-2021.6.0 dask-jobqueue-0.7.2 dill-0.3.3 distributed-2021.6.0 easyvvuq-0.9.3+130.g4fb85615 execnet-1.8.1 fsspec-2021.6.0 google-auth-1.31.0 heapdict-1.0.1 iniconfig-1.1.1 joblib-1.0.1 kiwisolver-1.3.1 kubernetes-17.17.0 locket-0.2.1 matplotlib-3.4.2 msgpack-1.0.2 numpoly-1.2.3 numpy-1.20.3 packaging-20.9 pandas-1.2.4 partd-1.2.0 pep8-1.7.1 pillow-8.2.0 pluggy-0.13.1 psutil-5.8.0 py-1.10.0 py-cpuinfo-8.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycodestyle-2.7.0 pyparsing-2.4.7 pytest-6.2.4 pytest-benchmark-3.4.1 pytest-cache-1.0 pytest-dependency-0.5.1 pytest-pep8-1.0.6 pytz-2021.1 pyyaml-5.4.1 qcg-pilotjob-0.10.0+145.g69df110 qcg-pilotjob-executor-api-0.10.0+145.g69df110 requests-oauthlib-1.3.0 rsa-4.7.2 scikit-learn-0.24.2 scipy-1.6.3 sortedcontainers-2.4.0 squarify-0.4.3 tblib-1.7.0 threadpoolctl-2.1.0 toml-0.10.2 toolz-0.11.1 typing-extensions-3.10.0.0 websocket-client-1.1.0 zict-2.0.0 zmq-0.0.0\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "pip install git+https://github.com/UCL-CCS/EasyVVUQ.git@qcgpj-executor" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Application scenario\n", "Let's remind the basic use-case. It is a simulation of a vertical deflection of a round metal \n", "tube suspended on each end in response to a force applied at certain point ```a``` along its length. \n", "Our goal is to determine the influence of the input parameters on the vertical deflection at point ```a```." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The usage of the application is:\n", "\n", "```beam ```\n", "\n", "It outputs calculated displacements to a file called `output.json`. Its content will look like \n", "\n", "```{'g1': x, 'g2': y, 'g3': y}```\n", "\n", "In order to produce statistically significant results, EasyVVUQ needs to run a number of model evaluations\n", " appropriately selecting input arguments from a given sample parameter space. \n", " Once selected, input parameters need to be transformed into a format understandable by the application. \n", "Our application takes a single file as an input and the transformation may be based on a single template file,\n", "called `beam.template`, with the following content:\n", "\n", "```{\"outfile\": \"$outfile\", \"F\": $F, \"L\": $L, \"a\": $a, \"D\": $D, \"d\": $d, \"E\": $E}```\n", "\n", "The template will be used to generate files called `input.json` that will be the input to each run of beam.\n", "All placeholders (signified by the $ delimeter) will be replaced by concrete values from the sample parameter space. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So, for example (commands preceded by an exclamation mark are treated as shell commands):" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/jovyan/tutorials/VECMAtk/BEAM\n" ] } ], "source": [ "!pwd\n", "!echo \"{\\\"outfile\\\": \\\"output.json\\\", \\\"F\\\": 1.0, \\\"L\\\": 1.5, \\\"a\\\": 1.0, \\\"D\\\": 0.8, \\\"d\\\": 0.1, \\\"E\\\": 200000}\" > input.json" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "!./beam input.json" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"g1\": -6.909453505549654e-06, \"g2\": -1.3818907011099308e-05, \"g3\": 1.7273633763874136e-05}" ] } ], "source": [ "!cat output.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this tutorial, in the similar fashion as in the basic one, \n", "we will demonstrate how to use of EasyVVUQ to do variance based sensitivity analysis of `beam` application using stochastic collocation. \n", "\n", "Nevertheless, the way of usage of QCG-PilotJob in EasyVVUQ \n", "is generic and will look the same also for other applications and other supported by EasyVVUQ methods." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Campaign" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to use EasyVVUQ, we need to configure the EasyVVUQ Campaign object. We do this in almost the same way as in case of basic use-case. \n", "Firstly we import the same set of libraries as in the original example:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "import os\n", "import easyvvuq as uq\n", "import chaospy as cp\n", "import matplotlib.pyplot as plt\n", "from easyvvuq.actions import CreateRunDirectory, Encode, Decode, CleanUp, ExecuteLocal, Actions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We only extend this set of imports with two modules for QCG-PilotJob:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "from easyvvuq.actions import QCGPJPool, ExecuteQCGPJ" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, we can continue the code from the basic workflow. For the validation purposes, we describe a set of parameters used by the application:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "params = {\n", " \"F\": {\"type\": \"float\", \"default\": 1.0}, \n", " \"L\": {\"type\": \"float\", \"default\": 1.5}, \n", " \"a\": {\"type\": \"float\", \"min\": 0.7, \"max\": 1.2, \"default\": 1.0}, \n", " \"D\": {\"type\": \"float\", \"min\": 0.75, \"max\": 0.85, \"default\": 0.8},\n", " \"d\": {\"type\": \"float\", \"default\": 0.1},\n", " \"E\": {\"type\": \"float\", \"default\": 200000},\n", " \"outfile\": {\"type\": \"string\", \"default\": \"output.json\"}\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and, by specification of encoder and decoder, define how EasyVVUQ should convert data between its internal and the application logic:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "encoder = uq.encoders.GenericEncoder(template_fname='beam.template', delimiter='$', target_filename='input.json')\n", "decoder = uq.decoders.JSONDecoder(target_filename='output.json', output_columns=['g1'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hence our application takes and produces very simple data structures, we use build-in Encoder and Decoder classes, but you are able to provide custom implementations of encoders and decoders that will fit your own use-case. \n", "\n", "The next step in the original tutorial was a definition of an execute action that will be used to run the `beam` application with a prepared input file. \n", "We could use the basic form of this action, but then the potential of QCG-PilotJob wouldn't be fully exploited. Instead, we wrap the original action in a ExecuteQCGPJ decorator that configures the action to be properly executed within a QCG-PilotJob's task:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "execute = ExecuteQCGPJ(\n", " ExecuteLocal('{}/beam input.json'.format(os.getcwd()))\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are allowed to push all actions we want to execute for samples into the Actions object:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "actions = Actions(CreateRunDirectory('/tmp'), \n", " Encode(encoder), execute, Decode(decoder))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and finally create EasyVVUQ Campaign:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "campaign = uq.Campaign(name='beam', params=params, actions=actions)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The remaining steps to configure the campaign object are to define distributions for the input parameters and initialise a sampler. This code is the same regardeless we use or not QCG-PilotJob:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "vary = {\n", " \"F\": cp.Normal(1, 0.1),\n", " \"L\": cp.Normal(1.5, 0.01),\n", " \"a\": cp.Uniform(0.7, 1.2),\n", " \"D\": cp.Triangle(0.75, 0.8, 0.85)\n", "}\n", "campaign.set_sampler(uq.sampling.SCSampler(vary=vary, polynomial_order=1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are able to execute all evaluations. However, in contrast to the basic tutorial, where the code was prepared to be run on a local machine, our target here is to demonstrate how to use QCG-PilotJob to execute evaulations on an HPC cluster. \n", "\n", "To this end, we need to create a QCGPJPool object and provide it to the campaign's execute method. In the most simplistic configuration, we can initialise `QCGPjPool` within the `with` statement and don't provide any arguments to the constructor:" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "with QCGPJPool() as qcgpj:\n", " campaign.execute(pool=qcgpj).collate()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Empty list of QCGPJPool constructor's parameter lead to the default settings of the pool and execution of all evaluations with a default task template. This may be sufficient for basic use-cases, but in order to support more advanced execution scenarios, \n", "several parameters may need to be provided to the constructor:\n", "* `qcgpj-executor` - allows to setup specific parameters of the QCG-PilotJob service by creation of a custom QCGPJExecutor instance. For example, if we skip this parameter, QCG-PilotJob will automatically set up to execute on all available resources, which is perfectly fine if we run the code on HPC resources, since it will take care of proper and efficient scheduling of tasks to the resources available in the allocation. However, if we would like to make some tests on a local machine, it may be more optimal to define virtual resources, and this may be defined with the `resources` parameter to the `QCGPJExecutor` constructor.\n", "* `template` and `template_params` - by default the tasks use a predefined template (`EasyVVUQBasicTemplate)` that leads to execution of QCG-PilotJob's tasks in a default mode, only on a single core. It could be altered by providing custom task `template` and `template_params`. \n", "* `polling_interval` - allows to change the default interval between queries asking about the status of tasks to the QCG-PilotJob Manager service.\n", "\n", "Let us show how to modify this example to demonstrate usage of these more advanced options. \n", "\n", "Firstly, we would change the `template` and `template_params` to enable execution of tasks on many resources. Thus, instead of a default template class, we will employ `EasyVVUQParallelTemplate` that allows us also to set `numCores` and `numNodes` parameters. \n", "\n", "In order to demonstrate it, we have to have a clean situation and therefore we need to initialise a new campaign:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "campaign = uq.Campaign(name='beam', params=params, actions=actions)\n", "campaign.set_sampler(uq.sampling.SCSampler(vary=vary, polynomial_order=1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are able to execute campaign actions once again. This time it looks as follows: " ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "from easyvvuq.actions.execute_qcgpj import EasyVVUQParallelTemplate\n", "\n", "with QCGPJPool(\n", " template=EasyVVUQParallelTemplate(), \n", " template_params={'numCores': 4}) as qcgpj:\n", " campaign.execute(pool=qcgpj).collate()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have set `numCores` to 4, which is one of parameters supported by `EasyVVUQParallelTemplate`. It results in assigning 4 cores for each task. Please note that this setting is not optimal for our example `beam` code, which is not parallel - in case of such codes single core tasks are perfectly fine. Nevertheless, possibility to define `numCores` and `numNodes` is essential for the proper execution of MPI or OpenMP applications. \n", "\n", "If you want to get information what parameters a given template supports, you can print its content and check the keys in `${}` expressions. " ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " {\n", " 'name': '${name}',\n", " 'execution': {\n", " 'exec': '${exec}',\n", " 'args': ${args},\n", " 'stdout': '${stdout}',\n", " 'stderr': '${stderr}',\n", " 'venv': '${venv}',\n", " 'model': '${model}',\n", " 'model_opts': ${model_opts}\n", " },\n", " 'resources': {\n", " 'numCores': {\n", " 'exact': ${numCores}\n", " },\n", " 'numNodes': {\n", " 'exact': ${numNodes}\n", " }\n", " }\n", " }\n", " \n" ] } ], "source": [ "print(EasyVVUQParallelTemplate().template()[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also easily get information about default values for the keys:" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'args': [],\n", " 'stdout': 'stdout',\n", " 'stderr': 'stderr',\n", " 'venv': '',\n", " 'model': 'default',\n", " 'model_opts': {},\n", " 'numCores': 1,\n", " 'numNodes': 1}" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "EasyVVUQParallelTemplate().template()[1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* **Note 1:** If the functionality of built-in templates is not sufficient, you can always create a new one by extensions of existings.\n", "* **Note 2:** The keys `name`, `stdout` and `stderr` are necessary for the code to work properly, so newly created templates must define these keys in an analogous way as they are defined in the existing templates. It is also not possible to set these keys to custom values, because they are substituted automatically by the internal software logic.\n", "\n", "Now let's get back to the `EasyVVUQParallelTemplate` which should be sufficient for many scenarios and try to set some larger number of nodes:" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Request failed - Not enough resources for job 1\n" ] } ], "source": [ "campaign = uq.Campaign(name='beam', params=params, actions=actions)\n", "campaign.set_sampler(uq.sampling.SCSampler(vary=vary, polynomial_order=1))\n", "\n", "try:\n", " with QCGPJPool(\n", " template=EasyVVUQParallelTemplate(), \n", " template_params={'numNodes': 2, 'numCores': 4}) as qcgpj:\n", " campaign.execute(pool=qcgpj).collate()\n", "except Exception as e:\n", " print(e)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you see that the exception saying that there is not enough resources has been cought, it is not a suprise. Well, we don't have 2 nodes to use and QCG-PilotJob reports that our task is too big. \n", "\n", "But what if we want to prepare and test the workflow on a local machine before it will be transfered to the HPC environment? QCG-PilotJob has a solution for this which is called *Local Mode* and allows to define virtual resources. Let's modify our example a bit: " ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "campaign = uq.Campaign(name='beam', params=params, actions=actions)\n", "campaign.set_sampler(uq.sampling.SCSampler(vary=vary, polynomial_order=1))\n", "\n", "from qcg.pilotjob.executor_api.qcgpj_executor import QCGPJExecutor\n", "\n", "with QCGPJPool(\n", " qcgpj_executor=QCGPJExecutor(resources=\"node1:4,node2:4\"),\n", " template=EasyVVUQParallelTemplate(), \n", " template_params={'numNodes': 2, 'numCores': 4}) as qcgpj:\n", " campaign.execute(pool=qcgpj).collate()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As it can be seen, we added `qcpj_executor` parameter to the `QCGPJPool` constructor. The parameter is set to the customised `QCGPJExecutor` instance, which has been created\n", "with parameter `resources` set to `node1:4,node2:4`. In this way we have defined two virtual nodes, each with 4 cores. In a result this example can been executed successfully. \n", "\n", "At this moment we should have our evaluations ready and their results collated and stored in the campaing's database. Since we have used `QCGPJPool` inside the `with` statement it has been already cleaned up and we don't need any other code specific for QCG-PilotJob's execution. Thus the remaining part of the tutorial can be no different from its basic version. In other words, starting from now, we have all data needed to perform analysis in a typical for EasyVVUQ way.\n", "\n", "At the beginning we can display the collattion results:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
run_iditerationFLaDdEoutfileg1
0000000000
0100.91.490.8056620.7795880.1200000output.json-0.000008
1200.91.490.8056620.8204120.1200000output.json-0.000007
2300.91.491.0943380.7795880.1200000output.json-0.000005
3400.91.491.0943380.8204120.1200000output.json-0.000004
4500.91.510.8056620.7795880.1200000output.json-0.000009
5600.91.510.8056620.8204120.1200000output.json-0.000007
6700.91.511.0943380.7795880.1200000output.json-0.000006
7800.91.511.0943380.8204120.1200000output.json-0.000005
8901.11.490.8056620.7795880.1200000output.json-0.000010
91001.11.490.8056620.8204120.1200000output.json-0.000008
101101.11.491.0943380.7795880.1200000output.json-0.000006
111201.11.491.0943380.8204120.1200000output.json-0.000005
121301.11.510.8056620.7795880.1200000output.json-0.000011
131401.11.510.8056620.8204120.1200000output.json-0.000009
141501.11.511.0943380.7795880.1200000output.json-0.000007
151601.11.511.0943380.8204120.1200000output.json-0.000006
\n", "
" ], "text/plain": [ " run_id iteration F L a D d E outfile \\\n", " 0 0 0 0 0 0 0 0 0 \n", "0 1 0 0.9 1.49 0.805662 0.779588 0.1 200000 output.json \n", "1 2 0 0.9 1.49 0.805662 0.820412 0.1 200000 output.json \n", "2 3 0 0.9 1.49 1.094338 0.779588 0.1 200000 output.json \n", "3 4 0 0.9 1.49 1.094338 0.820412 0.1 200000 output.json \n", "4 5 0 0.9 1.51 0.805662 0.779588 0.1 200000 output.json \n", "5 6 0 0.9 1.51 0.805662 0.820412 0.1 200000 output.json \n", "6 7 0 0.9 1.51 1.094338 0.779588 0.1 200000 output.json \n", "7 8 0 0.9 1.51 1.094338 0.820412 0.1 200000 output.json \n", "8 9 0 1.1 1.49 0.805662 0.779588 0.1 200000 output.json \n", "9 10 0 1.1 1.49 0.805662 0.820412 0.1 200000 output.json \n", "10 11 0 1.1 1.49 1.094338 0.779588 0.1 200000 output.json \n", "11 12 0 1.1 1.49 1.094338 0.820412 0.1 200000 output.json \n", "12 13 0 1.1 1.51 0.805662 0.779588 0.1 200000 output.json \n", "13 14 0 1.1 1.51 0.805662 0.820412 0.1 200000 output.json \n", "14 15 0 1.1 1.51 1.094338 0.779588 0.1 200000 output.json \n", "15 16 0 1.1 1.51 1.094338 0.820412 0.1 200000 output.json \n", "\n", " g1 \n", " 0 \n", "0 -0.000008 \n", "1 -0.000007 \n", "2 -0.000005 \n", "3 -0.000004 \n", "4 -0.000009 \n", "5 -0.000007 \n", "6 -0.000006 \n", "7 -0.000005 \n", "8 -0.000010 \n", "9 -0.000008 \n", "10 -0.000006 \n", "11 -0.000005 \n", "12 -0.000011 \n", "13 -0.000009 \n", "14 -0.000007 \n", "15 -0.000006 " ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "campaign.get_collation_result()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We then call the analyse method whose functionality will depend on the sampling method used. It returns an [AnalysisResults]() object which can be used to retrieve numerical values or plot the results. In this case Sobols indices." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "results = campaign.analyse(qoi_cols=['g1'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can plot the results in a treemap format. Each square representing the relative influence of that parameter to the variance of the output variable (vertical displacement at point ```a```). A square labeled ```higher orders``` represent the influence of the interactions between the input parameters." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAI+CAYAAACym37DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAptElEQVR4nO3dd5iU5b248fu7laWzlKWDgKCgKChGVCxYY8VYMCbGE6PmJBr1RI9Gc6LGxBZLjsnRGJOYorHGxP6LLbGjiBULICLSQXpnd2ee3x8zrLsIuCDuCtyf6/LSnbfM877gzr3v++xMpJSQJEna2hU09gAkSZK+DIwiSZIkjCJJkiTAKJIkSQKMIkmSJMAokiRJAowiSbVExEUR8fv1LP9GRDzekGPKP++eEfF+RCyNiBEN/fwbKyK658dc2NhjkfTZwvcpkuqKiMlABVANZIB3gb8At6SUso04tAYVET2BD4HilFJ1I4/lKeDBlNING7FtCXAHsCvQA9gvpfT0ph2hpC2BV4qktTsipdSC3IvoVcAFwB8ad0hbtR7AO59j++eBbwKzNs1wPltEFDXUc0naNIwiaT1SSotSSg8CI4GTI2IHgIgojYhrI2JKRMyOiJsjomz1dhFxVES8ERGLI+KDiDgk/3jniHgwIuZHxMSIOK3WNpdGxL0RcXtELImIsRHRNyIujIg5ETE1Ig6qtf7TEXFlRIzOP88DEVFea/mREfFORCzMr7t9rWUXRMT0/POMj4j9a43h9vxqz+b/vTB/C2hoRPxHRDxfaz97RMQrEbEo/+891hjfzyLihfzzPB4R7dZ1riPitPw5mZ8/R53zj38A9AIeyo+jdC3bDo6I1/PPc29E3B0RP8//GVamlP43pfQ8uSt/6xQRIyNizBqP/VdEPJj/78Pyz7M4/+dxaa31ekZEiojvRMQU4F+1HivKr/PtiHgvP85JEfHdWtvvGxHTIuLc/J/3zIj4dq3lZRFxXUR8lD/fz6/+OxcRu0fEi/k/6zcjYt/1HaektTOKpHpIKY0GpgHD8g9dBfQFdgb6AF2AiwEiYjdyt9v+G2gN7A1Mzm93V34/nYFjgSsiYnitpzoCuA1oA7wOPEbu/9MuwGXAb9cY2reAU4BO5G73/So/hr7AncA5QHvgUXJRURIR/YAzgSH5q2EH1xpfbXvn/906pdQ8pTSq9sJ8gD2Sf862wPXAIxHRttZqJwLfBjoAJcB5a3ke8ufgSuD4/LF8lD9XpJR6A1PIXb1rnlJatca2JcA/gD8B5fnjPnptz1MPDwH9ImLbNY7hjvx/LyN3zlsDhwHfi0/PcdoH2J7ceV3THOBwoCW58/LLiBhca3lHoBW5P+/vADdGRJv8smuBXYA9yB3n+UA2IrqQ+3P4ef7x84D7IqL9hhy4JKNI2hAzgPKICOB04L9SSvNTSkuAK4AT8ut9B7g1pfRESimbUpqeUhoXEd2APYELUkorU0pvAL8n9yK72nMppcfyc3juJRc0V6WUqshFQs+IaF1r/dtSSm+nlJYBPwGOj9yk3pHAI/kxVJF7QS0j94KaAUqB/hFRnFKanFL6YCPOx2HA+yml21JK1SmlO4Fx5MJutT+mlCaklFYA95CLyLX5Rv6cvZaPnguBoZGb1/RZdgeKgF+llKpSSn8HRm/E8ZBSWg48AHwdIB9H2wEP5pc/nVIam/9zfYtcgO2zxm4uTSktyx/zmvt/JKX0Qcp5BnicT0IboAq4LH8cjwJLyUVaAbn4PTv/9ymTUnoxf66+CTyaUno0P64ngDHAoRtzDqStmVEk1V8XYD65UGkKvJq/XbEQ+Gf+cYBuwNoiozOwOqJW+yi/39Vm1/rvFcDclFKm1tcAzWutM3WNfRUD7fLP9dHqBfkJ4lOBLimlieSuIF0KzImIu1bfqtpAdZ5jHcdTew7P8jXGvs59pZSWAvPW2Nf6xjE91f2tkanrWrke7iAfReSuEt2fjyUi4isR8e+I+DgiFgH/Se5817bO546Ir0bES/lbhAvJhUvt7eetMal99TlrBzRh7X+vegDHrf67mN/vXuSuuEnaAEaRVA8RMYTcC/TzwFxygTIgpdQ6/0+rlNLqF/ypQO+17Gb1laYWtR7rDkz/HEPrtsa+qvLjm0HuxXL1+CO/7nSAlNIdKaW98usk4Oq17PuzfjW1znPUGsPGHM+a421G7pZcffY1E+iSP8bVuq1r5Xp4AmgfETuTi6M7ai27g9xVo24ppVbAzUCssf1az1t+LtR95K7aVaSUWpO7rbnm9mszF1jJ2v9eTSV3xbB1rX+apZSuqsd+JdViFEnrEREtI+Jwcreubl996wT4Hbn5IB3y63WJiNVzSP4AfDsi9o+Igvyy7VJKU4EXgSsjoklEDCR3q+32Tz9zvX0zIvpHRFNyc47+lr+ydA9wWH4MxcC5wCrgxYjoFxHD8y/SK8kF3treauDj/OO91vHcjwJ9I+LEiCiKiJFAf+DhjTiOO8mds53z47oCeDmlNLke244id0vwzPw4jgJ2q71C5CbGN8l/WZI//2uNkfztxnuBa8jN0Xmi1uIW5K72rczPHTux/odICbnblh8D1RHxVeCg9W9SM6YscCtwfeQm6xdGbuJ7Kbm/P0dExMH5x5vkJ2133YCxScIoktbloYhYQu6n8B+Tm0T87VrLLwAmAi9FxGLgSaAf1EzK/jbwS2AR8AyfXAX5OtCT3JWRfwCXpJSe/BzjvI3cBONZ5G6vnJUfw3hyc01+Te4qwxHkJipXknthvir/+Cxyk6AvXHPH+VtGlwMv5G/L7L7G8nnkJg2fS+5W1/nA4SmluRt6EPlz8BNyV1JmkrsicsJ6N/pk20rga+QCcyG5436YXASuNp5c/HUhN3l9BZ++ylXbHcABwL1r3M76PnBZ/u/GxeTis17yt03Pym+zgFxQPVjf7clNoB4LvELuNu7VQEE+to8CLiIXXFPJTfL3+7u0gXzzRmkzFRFPk7t6tc53oN5aRcTLwM0ppT829lgkbT78SULSZi8i9omIjvnbZycDA8lNfpekevMdVyVtCfqRuy3VDJgEHJtSmtm4Q5K0ufH2mSRJEt4+kyRJAowiSZIkwCiSJEkCjCJJkiTAKJIkSQKMIkmSJMAokiRJAowiSZIkYCPf0XrkP07wHR/X4e6j71rrJ2+vdV2KPY+SpAY3kqp6v1Z9f/yIL+Vr1U397q/3MdSXV4okSZIwiiRJkgCjSJIkCTCKJEmSAKNIkiQJ2Myj6O8n3tfYQ5AkSVuIzTqKJEmSNhWjSJIkCaNIkiQJMIokSZIAo0iSJAkwiiRJkoCN/EDYL4vqVdU8dOqDNV/3PbIf/Y7s14gjkiRJm6vNOoqOv29kYw9BkiRtIbx9JkmShFEkSZIEGEWSJElAI80pmvnaTN649XVSNrHNAb3Y/mvb11meqcow+oaXWTBpASUtShh67h4069CMVUtW8eI1L7Jg4nx67teTwaftAuQmXI+65kWWzl5KFASdd+3MwJN2AmD8g+P58MlJRGFQ2rKUIWfsRrMOzQB49rJnmDdhHu22b8ewH+/dsCdBkiR9qTR4FGUzWV773avsc8m+lLUt48nzn6DzkM606taqZp0Pn5xEcfMSDr3pMKY8P4W3/vImQ8/bg8LiQnb4+g4smrKIxVMW1dlvv6P60WHHCjJVGZ659GlmvjaTToM70Wab1vS+5kCKSouY+M+JNfsC6DdiOzKrqvng8Q8a9BxIkqQvnwa/fTZ/4nyad2pB847NKSwupPte3Zkxenqddaa/MoOe+/UEoOvQrsweO5uUEkVNimi/fXsKiwvrrF9UWkSHHSsAKCwupE2vNiyftxyADjtWUFSaa7+2fdvWPA5QMbCCorLiL+pQJUnSZqTBo2jFvBU0bVtW83VZ26asmL9ijXWW07RtUwAKCgsoblpM5ZLKeu2/clklM8bMoCIfSbV9+NQkOg3u9DlGL0mStlRb1ETrbCbLS9ePYttDt6V5x+Z1ln30zGTmT5xPvxHbNdLoJEnSl1mDR1FZ2zKWz/vkytCKecspKy9bY52mNbe5spksVcurKGlR8pn7HvObMTTv1IK+R9R9V+vZb87i3b+9y14XDvvUrTdJkiRohCgq71PO0plLWDp7KZmqDFOen0LnIV3qrNN5SGcm/3syANNGTaPDjhVExHr3O/aOsVQtr2LQKYPqPL5g0gLG3DyGvS4cRpPWTTbpsUiSpC1Hg//2WUFhAYNPHcyzlz2T+5X8/XvRqnsr3r5zLG16l9Nlty702r8XL9/wEo9+/xFKmpew+w+H1mz/8HcfonpFNdnqLNNfns7el+xDcVkx7/3tXVp0acET5z0OQJ+v9qHXgb158y9vUr2ymlHXvghA03ZN2euiYQD868dPsWT6EqpX5j5DbcgZQ+g4yDlHkiRtjSKltMEbjfzHCRu+0Vbi7qPvWv8lrdrrUux5lCQ1uJFU1fu16vvjR3wpX6tu6nd/vY+hvraoidaSJEkbyyiSJEnCKJIkSQKMIkmSJMAokiRJAowiSZIkwCiStBn5emE1F+xczXkDqjl/p2oevi5LNvul/G1h6UtlzuTEeTtUr3VZRFwWEQesb/tXfj2ON/4w8QsZW30tnracu4741xf6HBv15o0b8l48krSplJTB1W/kvm0tmpP49YlZVixOHPdTP75H2lgppYu/6OfIZhIFhRuWDtnqLAVFG3/tJiKKUkprL8F1aPB3tJakTaFVh+C0Wwr48ZAMx16aPvOjgKStXTYDt5yWYcKLiRPejceBo1JKKyLiT8DDKaW/RcShwPXAMuAFoNf3xh0FwIIPlvDASc+zZOYKBn6rFwO/1RuACQ9OZextk8hUZakY2IZhl+xEQWHwu8EPM+D4nkwb9THDLh5Ip13a1oxl7nuLeObSN6lekaFV96bsd/kgSluV8MBJz9N2+1bMenUefQ7rSpfd2vHvH78OQNc9O9Q6lkREXAPsC5QCN6aUfhsR+wI/AxYA20XEIOAeoCtQCPwspXT3us6Rt88kbbYqegXZDCya09gjkb78Zr0PB51RwLXvFAEsBI6pvTwimgC/Bb6aUtoFaF97+YJJSzj8D0M55t69GXPjeDJVWRZ8sISJj05nxB3DOP7+/YjC4P2HpgJQvTxDh53acPwD+9UJIoCnLniNoef2Z+SD+1HetyWv3Di+Zlm2Ksux9+3Lzqf04V8Xvc5e/7Mjxz+wX53tx/3tI4BFKaUhwBDgtIjYJr94MHB2SqkvcAgwI6W0U0ppB+Cf6ztHRpEkSVuBDttAz51rrqi+CvRcY5XtgEkppQ/zX99Ze2GPfSsoLCmkrE0pZW1LWTFvFdNGfczH7yzkvuOe4Z4R/2baqI9ZPHU5AFEY9Dqo86fGsWpJFZVLqui8WzsA+o3ozswx82qW9/lq7kPiVy3Orzckv95RXWvWmfrCHIBvRcQbwMtAW2Db/OLRtY5hLHBgRFwdEcNSSovWd468fSZpszV7UqKgEFp1+Ox1pa1dUWmdLzNA2YZsX1j8ydy9KAyy1QlSLmp2P7f/p9cvLdjgeUQARWWfPUcw/7GtP0gpPVb78fzts2WfrJcmRMRg4FDg5xHxVErpsnXt1ytFkjZLiz9O/P4/sxx8ZjifSNo0xgO9IqJn/uuRn7VBl6HtmPT4DJbPWwXAyoWVLJm+fL3blLYoprRlMTPyV4cmPDCVzkPafnq9lsWUtChm5qv59R6aVrOs+14dAL4XEcUAEdE3IpqtuY+I6AwsTyndDlxD7tbaOnmlSNJmo3IFXLBzNZkqKCiCYScVcNgPDSJpU8hPuv4+8M+IWAa88lnblPdpyW5nb8/D33mRlIWComDYxQNp0aXpercbftXgmonWLbs1ZfgVg9a+3hWDchOtA7rVmmi9/XE9eOaSN98FXovcT0UfAyPWsosdgWsiIgtUAd9b37giJd/jo7HcTbEnX5LU4EZStdafJiKieUppaT40bgTe/964o65v2NHVz0397t/kPxF5+0ySJK12Wn7y8jtAK3K/jbbVMIokSRIAKaVfppR2Tin1Tyl9I6W0/glCWxijSJIkiY2caH3fcxOcC7MOxwzr66xPaR3e+GeWP5+dJZuB4acWcNSP6v5c9t6ziT+fk2HKW3DWXQXsfmxu+ccfJa47OkPKQqYKDv5BAQf+Z27Zi3dnuf/y3D4HHR584+rCmm1uPiXLko8TzcrhzNsLadv1k/89ly9OnNc/w64jglP+z48JkdZlynOzef7ysaQsbH9sdwaf3rfO8kxlhqcueI2P31lEk9bFHHj9EFp2bcrKBZU8dvYrzHl7AduN6M6wiwfW2ibLcz97ixmj5xIFwW7nbE/vgzuvc19TX5jDS9e9S7YqS0FxAUPPHwD9Nv2x+ttnkhpENpO49YwsP36ikLZd4aIhGXY5Muja/5NQadsdvvenQh6+Nltn2zad4GejCikuDVYuTZy3Q27b4lL4639nufLVQlq2D246OcPYp7LsuH8Bt5+XZe9vBfucXMjb/8py54VZzrztk/i55ydZttvbn2Gk9YmIwpbdmnLErXvQrKKM+457hp7DO1Lep2XNOu/9bQqlLUv4xuMH8P4j03jpunc46JdDKCwtYLezt2P++4uZP2FJnf2+evMEytqWcuJjB5CyiZWLKte7ryZtSjj0N1+hWUUZ8yYs5pFTR8HJm/54vX0mqUFMHA0d+wQVvYKikmCPEwoY80Ddi84degY9BgaxxnemopKguDQXMFWrIOWbac4k6Lht0LJ9btkOBwSj78vtc/q7iQHDc48P2C94tdZzTXo1sWg2DDzIKJI+w26tujejZbdmFJYU0OfQLkx+aladFSY/NZN+I7oB0PvgzkwfNZeUEsVNi+i0S1sKSz59JXbc3z9i8Om5N6COgqCsTel699W+f2uaVeTea7J82xZUr8oQEaWf2vHnZBRJahDzpyfadvvk6/Kuucfqa+7UxPkDqzmjW4YjLyigvHNQ0Qdmjk/MmZzIVCfG3J+Yl/vYJbrvFIz+e27/r/wjsWIJLJmXyGYTt52b4ZvX+u1PqocuzTp98sbXzTqWsWz2yjorLJ2zkub5dQqKCihpUcTKhZXr3OGqxVUAjL5hHPd+7WkeO/sVls9dWe99TXpsJu36tyKltOrzH15dfleQtFlo1y34xVtF/O/EQp79c5aFsxPN2wTf+U0BN4zMcOmwDO17QkH+h9JvXlvAe88kfjSomnefSZR3yS17/KbEoEML6swvktRwspksy2atpOOgco77+7503LkNo37xTr22nf/+Yl667h32+enOX8jYnFMkqUGUdwnmTf3kytD8abnHNng/nYNuOwTjnkvsfmywyxEF7HJE7ue7J2/JUlCYatY79++5Qlq5NDH6vgzNWgfvj8oy7rnE4zdlWbUUqiuhSfMMJ17lZGtpLaYvm7mi5otls1bQrKJJnRWad2jC0pkraN6xjGx1lsol1TRpXbLOHTZpXUJRWSG9DuoEQO9DuvDefVM+c19LZ63gn2eOZvjVg2nV/VOf6LFJeKVIUoPoPQRmvZ+Y82GiujLx4l1ZdjmyflE0b1qickUudpYuSIx7PtG5X27bRXM+efyJm7Lsd2ru29riublbZQD3X5ll31Ny6//gr4XcOKWI/5tcxDeuLWDYt8IgktbtlYUfLWPxtGVkKrNMfHQ6PYd3rLNCz+EdGX9/7r71B4/NoMvu7db7eYQRQc/9OjJ99FwApo36mDa9W6x3X6sWV/Hod19i93P702nwpz8nbVPxSpGkBlFYFHz7/wq44uAM2Qzsd0oB3QYE91ycodeuwa5HFvDBK7lfvV+2AF57KPG3S7Jc+04R099L3H5uFgJIcPh5BXTfMfdN989nZ/nozVz8HHNxAZ3z74rx7tOJuy7MbbP93sEpN/ozoLShUkrVh90ylIe/M4qUTWx3THfKt23J6F+9R/sdWrPN8E5sd2wPnjr/Nf560JM0aVXMgdfvWrP97cMfp3JZNZmqLB8+NZPD/zCU8j4t2f3c/jx1wWu8cMXblJWXsF/+s8/Wta+3/zqJRVOWMeam8Yy5aTwAvxkXHVJKczbl8W7UZ5/5PkXrtiHvU+Rnn0mSGsO6Pvtsbb4/fsSX8rXKzz6TJEn6ghhFkiRJGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgRAUWMPYGs2kqpo7DFIkrQ+N/W7f6t5rfJKkSRJEkaRJEkSYBRJkiQBRpEkSRJgFEmSJAFGkSRJEmAUSZIkAUaRJEkSYBRJkiQBRpEkSRLgx3w0qoduOTs19hgkSdocHXH6DZv840e8UiRJkoRRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEQFFjD2BjHb/v9nTv1bfm6/Mvv5EOnbo24ogkSdLmbLONopLSJlx76wONPQxJkrSF8PaZJEkSm/GVospVKznvlKMA6NCpK+dffmMjj0iSJG3ONtso8vaZJEnalLx9JkmShFEkSZIEGEWSJEnAZhxFtz/2emMPQZIkbUE22yiSJEnalIwiSZIkGulX8l9/+Vn++KvLyWaz7H/YcRz9zdPrLH/3jVf446+v4KNJ4/mvS65n6L6HAPDxrOn84sdnklKW6upqvnrMNzn4qK8DcPFZJ7Fw3hxKSpsA8JPrbqVVm7br3BfU/aiQdh068aOrbm6Iw5ckSV9CDR5FmUyG3//yMi6+/o+Ut6/gR6cfy657Dadbzz4167Sr6MQZF13Jg3fdWmfb1m3bc8Vv7qa4pIQVy5fxw/84giF7Dqe8XQUAZ/3kWvpst2Odbda1L/C9jiRJ0icaPIomvvcWHbv0oKJzNwD23P8wXnn+qTpRtPqDXQui7t294uKSmv+urqokZbOf+Xzr2pckSVJtDR5F8+fOpl2HjjVft21fwfvvvlXv7efOnskVF5zOrOlTOOl759dcJQK46cqLKCgs4Cv7HMSx3/o+EbHefVVWruL8075GYWERR3/jdHYbdsCGH5AkSdoibHYf89GuohPX/+kh5s+dzS8uOoOh+x5M6/J2nP2Ta2nbvoIVy5dyzf+cxTOPPcC+h4xY775+c8+/adu+gtkzpnLpOSfTvVdfOnbp3jAHIkmSvlQa/J5SebsK5s6ZVfP1vI9nU96+Yj1brHs/3Xpty3tvjQFyV5wAypo2Z9iBhzPxvc+++rR6m4rO3Riw8258+P67GzwOSZK0ZWjwKOqz3Y7MnDaZ2TOmUlVVyQtPPcKQPYfXa9t5c2axatVKAJYuWcS4t16jc7dtyFRXs3jhfACqq6t49cWn6dZr2/Xua+mSRVRVVgKweOF8xo19ja615jVJkqStS4PfPissKuLUcy7m5+edSjabYfihx9Btm2256w830LvfDgzZa38mvvcWv/ifM1m2ZDFjXvw3d9/6a/73L48w7aMP+PONVxERpJQ48oRT6NG7HytXLOfn551KdXUV2WyWgbsM5YDDjwdY974mf8At115CFAQpmzj6G6fVmewtSZK2LpFS2uCN7ntuwoZvtJU4Zljf9c/uruWhW872PEqStBGOOP2Ger/e1pe/py5JkoRRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJQCN8zIckfdldftPDzF2whMqqDEcM35lD9t6hsYckqQEYRZK0hrNOPoAWzZqwqrKac6+8iz0G96Zl87LGHpakL5hRJElreOhfb/DSG5MAmDt/KTPmLDSKpK2AUSRJtYwdP40335vKNRccR2lJMRdddx9VVZnGHpakBuBEa0mqZdmKVTRvWkppSTHTZs1n/KRZjT0kSQ3EKJKkWnYZ0INMNvH9S27jz39/kX69Ojb2kCQ1EG+fSVItxcVFXHrWUY09DEmNwCtFkiRJGEWSJEmAUSRJkgQ4p0jSFuTVtyfz+3ueJZNNHLTXAI49ZNdPrfP8mAnc+fDLQLBN13acd+ohAPzpvhcY8/aHAIw8dDeGDekLQEqJ2x8YxQuvTqSgIPjqPjtyxPCdgdyv7//+nmepzmRp2bwJV553LAAPPPk6jz//DhHQo0s7zj75AEqK/XYrfdn5f6mkLUImm+W3dz7NZeccTds2zTn3yrvZbeA2dO/ctmadGbMXcu8/x3D1fx9H82ZNWLh4OQCvjP2QD6bO4Yb/OZGq6gwXXXcfu+zQg6ZlpTz14nvMXbCUm356EgUFUbPN0uWruPnOf3PpWSNoX96i5vF5C5by0L/e5MZLv0lpSRFX3/Ioz70ygf336N/wJ0XSBvH2maQtwvsfzqZTh9Z0bN+K4qJChu26LS+/OanOOo89/zaH7TuQ5s2aANC6ZVMAps6Yz4Btu1BYWECT0mJ6dm3Ha+98BMD/e3YsIw/bjYKCqLPNs6PHM3TnPrQvb1HncYBsNktlVTWZTJZVldWUt272xR68pE3CK0WStgjzFi6lXZvmNV+3a9Oc8R/OrrPOjNkLATj/F/eSzWb5+uFfYZcderJNt3bc+fBojj5wEKsqqxk7fhrdOpUDMOvjRTw/5n1eev0DWrYo4/SR+9C5ojXTZy8kk8ldVVqxspIjhu/M8KHb07ZNc0YcOJjvXPhHSooLGdS/B4P692iw8yBp43mlSNJWI5PNMnPOQq4492ucd+oh3Hj7v1i6fBWD+vdg1x16cv7V93LN7//Jdr06URC5K0NV1RmKiwu5/scncNBeA/jVX56s2dfEKXO4+Mwj+enZI7j70dFMn72ApctW8vKbk/jd5Sfzp198h5Wrqvj3S+Ma87Al1ZNRJGmL0LZ1c+YuWFrz9dwFS2m7xm2rdm2as9vAXhQVFtKxXSs6d2jNzDkLATj+0CHc8JMT+dk5R5NSoktFm5r9Dh3UG4Chg3ozedrcmn0N7t+DJqXFtGxexoBtu/DhtLm8MW4qFe1a0qpFU4oKCxk6qDfjJs1sgDMg6fMyiiRtEbbtWcGMOQuZNXcRVdUZnhvzPl/ZqVeddb6yUy/GTpgGwOKlK5gxZyEV7VqSyWZZvHQFAB9Om8vk6XMZ1L87ALvv3Iux43PbvD1hOp0rWtfs692JM/LzhqqY8OEsunUsp315C8ZPmsWqyipSSrw5birdOpY30FmQ9Hk4p0jSFqGwsIDvnrAvl97wANlslgP2HED3zm3564Mv0adHB76yUy8GD+jBG+9O4YxLb6MgCviPY/aiZfMyKququfDavwFQ1qSEH55yMIWFuZ8ZjzlkV67/w2M8+OQbNCkt5gcn7Q9At07lDB7Qg7N+9lciggP3HECPLrnfdNtzcB/O+fldFBYGvbq15+BhAxrnpEjaIJFS2uCN7ntuwoZvtJU4ZljfqO+6D91ytudRkqSNcMTpN9T79ba+vH0mSZKEUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgRApJQaewySJEmNzitFkiRJGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJgFEkSZIEGEWSJEmAUSRJkgQYRZIkSYBRJEmSBBhFkiRJAPx/3+fi55Gy6FoAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "results.plot_sobols_treemap('g1', figsize=(10, 10))\n", "plt.axis('off');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Alternatively you can get the Sobol index values using the method call below." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'F': array([0.13515478]),\n", " 'L': array([0.01220653]),\n", " 'a': array([0.69667914]),\n", " 'D': array([0.13994264])}" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results.sobols_first('g1')" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" }, "pycharm": { "stem_cell": { "cell_type": "raw", "metadata": { "collapsed": false }, "source": [] } } }, "nbformat": 4, "nbformat_minor": 5 }