{ "cells": [ { "cell_type": "markdown", "id": "acoustic-monkey", "metadata": {}, "source": [ "

\n", " \n", "

\n", "\n", "
\n", "
\n", "\n", "\n", "\n", "# Tutorial for Hyperactive (v4)\n", " \n", " \n", "This is a tutorial to introduce you to the basic functionalities of Hyperactive and provide some interesting applications. It will also give an introduction to some optimization techniques. Hyperactive is a package that can optimize any python function and collect its search data. \n", "\n", "### [You can learn more about Hyperactive on Github](https://github.com/SimonBlanke/Hyperactive)\n", " \n", "## Table of contents:\n", "* [Introduction](#intro)\n", " \n", "* Optimization basics:\n", " * [Convex Optimization](#convex)\n", " * [Non-convex Optimization](#non_convex)\n", " \n", "* Advanced Usage:\n", " * [Hyperactive memory](#memory)\n", " * [Machine Learning Optimization](#machine_learning)\n", " * [Continuing the search](#continuing-search)\n", " * [Parallel Computing](#parallel-comp)\n", " * [Data collection](#data_collect)\n", " * [Multiple objectives](#multi_objectives)\n", " * [Non-numerical search space](#search_space)\n", " * [Deep Learning Optimization](#deep_learning)\n", " \n" ] }, { "cell_type": "markdown", "id": "8beda744", "metadata": {}, "source": [ "\n", "\n", "This tutorial is made for version 4 of Hyperactive" ] }, { "cell_type": "code", "execution_count": 1, "id": "technical-logging", "metadata": {}, "outputs": [], "source": [ "def warn(*args, **kwargs):\n", " pass\n", "import warnings\n", "warnings.warn = warn\n", "\n", "import time\n", "import numpy as np\n", "import plotly.express as px\n", "import plotly.graph_objects as go\n", "from sklearn.preprocessing import Normalizer, MinMaxScaler\n", "from sklearn.model_selection import cross_val_score\n", "from sklearn.ensemble import GradientBoostingRegressor\n", "from sklearn.neural_network import MLPClassifier\n", "from sklearn.gaussian_process.kernels import Matern, WhiteKernel, RBF, ConstantKernel\n", "\n", "from sklearn.tree import DecisionTreeRegressor\n", "from sklearn.datasets import load_wine, load_iris\n", "\n", "from hyperactive import Hyperactive\n", "from hyperactive.optimizers import BayesianOptimizer, HillClimbingOptimizer\n", "from surfaces.visualize import plotly_surface, plotly_heatmap\n", "\n", "color_scale = px.colors.sequential.Jet" ] }, { "cell_type": "code", "execution_count": 2, "id": "smart-edwards", "metadata": {}, "outputs": [], "source": [ "def _create_grid(objective_function, search_space):\n", " def objective_function_np(*args):\n", " para = {}\n", " for arg, key in zip(args, search_space.keys()):\n", " para[key] = arg\n", "\n", " return objective_function(para)\n", "\n", " (x_all, y_all) = search_space.values()\n", " xi, yi = np.meshgrid(x_all, y_all)\n", " zi = objective_function_np(xi, yi)\n", "\n", " return xi, yi, zi\n", "\n", "import plotly.graph_objects as go\n", "from plotly.subplots import make_subplots\n", "\n", "def compare_objective_functions(objective_function1, objective_function2):\n", " search_space_plot = {\n", " \"x\": list(np.arange(-5, 5, 0.2)),\n", " \"y\": list(np.arange(-5, 5, 0.2)),\n", " }\n", "\n", " xi_c, yi_c, zi_c = _create_grid(objective_function1, search_space_plot)\n", " xi_a, yi_a, zi_a = _create_grid(objective_function2, search_space_plot)\n", "\n", " fig1 = go.Surface(x=xi_c, y=yi_c, z=zi_c, colorscale=color_scale)\n", " fig2 = go.Surface(x=xi_a, y=yi_a, z=zi_a, colorscale=color_scale)\n", "\n", " fig = make_subplots(rows=1, cols=2,\n", " specs=[[{'is_3d': True}, {'is_3d': True}]],\n", " subplot_titles=['Convex Function', 'Non-convex Function'],\n", " )\n", "\n", " fig.add_trace(fig1, 1, 1)\n", " fig.add_trace(fig2, 1, 2)\n", " fig.update_layout(title_text=\"Objective Function Surface\")\n", " fig.show()" ] }, { "cell_type": "code", "execution_count": 3, "id": "blessed-faith", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-01-02 08:46:37.569045: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n", "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", "2024-01-02 08:46:37.650657: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:37.650673: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", "2024-01-02 08:46:38.153488: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.153539: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.153543: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Device mapping: no known devices.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2024-01-02 08:46:38.609600: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n", "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", "2024-01-02 08:46:38.695715: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n", "2024-01-02 08:46:38.695926: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.695976: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.696017: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublasLt.so.11'; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.696055: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcufft.so.10'; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.696094: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcurand.so.10'; dlerror: libcurand.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.696132: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.696170: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusparse.so.11'; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:\n", "2024-01-02 08:46:38.696273: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n", "Skipping registering GPU devices...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /tmp/ipykernel_1406914/2148136278.py:15: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.\n", "\n" ] } ], "source": [ "from tensorflow.keras.datasets import mnist\n", "from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dropout, Dense, Flatten\n", "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.optimizers import SGD\n", "from keras.utils import np_utils\n", "from tensorflow import keras\n", "\n", "import tensorflow as tf\n", "\n", "config = tf.compat.v1.ConfigProto()\n", "config.gpu_options.allow_growth = True\n", "config.log_device_placement = True\n", "\n", "sess = tf.compat.v1.Session(config=config)\n", "tf.compat.v1.keras.backend.set_session(sess)" ] }, { "cell_type": "code", "execution_count": 4, "id": "moving-adapter", "metadata": {}, "outputs": [], "source": [ "# load dataset\n", "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n", "\n", "img_width = 28\n", "img_height = 28\n", "\n", "x_train = x_train.astype(\"float32\")\n", "x_train /= 255.0\n", "x_test = x_test.astype(\"float32\")\n", "x_test /= 255.0\n", "\n", "# reshape input data\n", "x_train = x_train.reshape(x_train.shape[0], img_width, img_height, 1)\n", "x_test = x_test.reshape(x_test.shape[0], img_width, img_height, 1)\n", "\n", "# one hot encode outputs\n", "y_train = np_utils.to_categorical(y_train)\n", "y_test = np_utils.to_categorical(y_test)\n", "num_classes = y_test.shape[1]\n", "\n", "\n", "data = load_wine()\n", "X_boston, y_boston = data.data, data.target\n", "\n", "data = load_iris()\n", "X_iris, y_iris = data.data, data.target" ] }, { "cell_type": "markdown", "id": "loaded-engagement", "metadata": {}, "source": [ "\n", " \n", "## Introduction \n", " \n", "There are two things you need to define before starting your first optimization run:\n", " \n", " - the objective function: \n", " Contains some kind of model. It always returns a score that will be maximized during\n", " - a search space: \n", " Defines the parameter space in which the optimizer searches for the best parameter set\n", " \n", "In this notebook you will see several different examples for objective functions. \n", " " ] }, { "cell_type": "code", "execution_count": 5, "id": "provincial-canvas", "metadata": {}, "outputs": [], "source": [ "def objective_function(para):\n", " loss = para[\"x\"]*para[\"x\"]\n", " # -x*x is an inverted parabola \n", " return -loss\n", "\n", "# We have only one dimension here\n", "search_space = {\n", " \"x\": list(np.arange(-5, 5, 0.01)),\n", "}" ] }, { "cell_type": "markdown", "id": "similar-questionnaire", "metadata": {}, "source": [ "\n", "\n", "In the next step we will start the optimization run.\n", "\n", "You only need the objective_function, search_space and the number of iterations. Each iteration will evaluate the objective function. This will generate a score, that the optimization algorithm uses to determine which position in the search space to look next. All of the calculations will be done by Hyperactive in the background. You will receive the results of the optimization run when all iterations are done." ] }, { "cell_type": "code", "execution_count": 6, "id": "genetic-lotus", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " x score\n", "0 0.22 -0.0484\n", "1 -1.46 -2.1316\n", "2 4.99 -24.9001\n", "3 -5.00 -25.0000\n", "4 4.06 -16.4836\n", ".. ... ...\n", "65 -4.73 -22.3729\n", "66 -2.99 -8.9401\n", "67 4.65 -21.6225\n", "68 4.01 -16.0801\n", "69 -0.78 -0.6084\n", "\n", "[70 rows x 2 columns] \n", "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
xscore
00.22-0.0484
1-1.46-2.1316
24.99-24.9001
3-5.00-25.0000
44.06-16.4836
.........
65-4.73-22.3729
66-2.99-8.9401
674.65-21.6225
684.01-16.0801
69-0.78-0.6084
\n", "

70 rows × 2 columns

\n", "
" ], "text/plain": [ " x score\n", "0 0.22 -0.0484\n", "1 -1.46 -2.1316\n", "2 4.99 -24.9001\n", "3 -5.00 -25.0000\n", "4 4.06 -16.4836\n", ".. ... ...\n", "65 -4.73 -22.3729\n", "66 -2.99 -8.9401\n", "67 4.65 -21.6225\n", "68 4.01 -16.0801\n", "69 -0.78 -0.6084\n", "\n", "[70 rows x 2 columns]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hyper_0 = Hyperactive(verbosity=False)\n", "hyper_0.add_search(objective_function, search_space, n_iter=70, initialize={\"random\": 2, \"vertices\": 2})\n", "hyper_0.run()\n", "\n", "search_data_0 = hyper_0.search_data(objective_function)\n", "search_data_0[[\"x\", \"score\"]]" ] }, { "cell_type": "markdown", "id": "connected-nerve", "metadata": {}, "source": [ "\n", "\n", "In the table above you can see the 70 iterations performed during the run. This is called the **search data**. In each row you can see the parameter ```x``` and the corresponding score. As we previously discussed the optimization algorithm determines which position to select next based on the score from the evaluated objective function. \n", "\n", "When Hyperactive starts the optimization the **first iterations are initializations** from the ```initialize```-dictionary. In the example above there are 4 initializations (2 random and 2 vertices). They determine the initial positions in the search space that are used to evaluate the objective funtion. As you can see in the search data the 2. and 3. iteration are the vertices (edge points) of the search space. The 0. and 1.\n", "The first rows of the search data are randomly selected. After those few initialization steps the optimization algorithm will select the next positions in the search space based on the score of the previous position(s).\n", " \n", "The default algorithm for the optimization is the **random-search**. You can see the random pattern in the last few iterations of the search data. We can also see the random pattern if we plot the search data:" ] }, { "cell_type": "markdown", "id": "fancy-terrain", "metadata": {}, "source": [ "\n", "\n", "### Random Search Optimizer\n", " \n", "The random search optimizer is a very simple algorithm. It randomly selects the position in each iteration without adapting to the optimization problem (exploitation). One the other hand it is very useful to initialy explore the search space or find new regions with optima (exploration).\n", " \n", "The following two gifs show how random search explores the search space for two different objective functions:" ] }, { "cell_type": "markdown", "id": "sweet-enough", "metadata": {}, "source": [ "\n", "\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 7, "id": "annoying-keyboard", "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "x=%{x}
score=%{y}", "legendgroup": "", "marker": { "color": "#636efa", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.21999999999988873, -1.4600000000000755, 4.989999999999787, -5, 4.059999999999807, 3.679999999999815, 4.279999999999802, 3.7099999999998143, 2.699999999999836, -0.5800000000000942, 1.4099999999998634, -2.230000000000059, -3.7500000000000266, -0.4300000000000974, -4.920000000000002, 3.959999999999809, -3.4400000000000333, -3.550000000000031, -3.540000000000031, -0.6300000000000932, 2.789999999999834, -4.910000000000002, -0.4400000000000972, -4.280000000000015, 1.1799999999998683, -1.200000000000081, -4.750000000000005, -0.7500000000000906, -2.570000000000052, -3.3400000000000354, 1.5799999999998597, 0.14999999999989022, -3.59000000000003, -2.7700000000000475, -1.4200000000000763, 1.3399999999998649, 2.5399999999998393, -4.660000000000007, -3.260000000000037, 4.129999999999805, -0.40000000000009805, 4.799999999999791, 4.699999999999793, 3.679999999999815, 3.399999999999821, 1.1199999999998695, -0.07000000000010509, -4.260000000000016, 0.9199999999998738, 2.079999999999849, 4.569999999999796, -4.390000000000013, -1.2400000000000801, 0.5699999999998813, 4.769999999999792, -1.1300000000000825, 2.5699999999998386, -4.830000000000004, -2.3100000000000573, 3.869999999999811, 3.389999999999821, 0.34999999999988596, -0.8800000000000878, 4.879999999999789, -2.5800000000000516, -4.730000000000006, -2.990000000000043, 4.649999999999794, 4.009999999999808, -0.78000000000009 ], "xaxis": "x", "y": [ -0.04839999999995104, -2.1316000000002204, -24.900099999997874, -25, -16.483599999998432, -13.542399999998638, -18.318399999998306, -13.764099999998622, -7.289999999999114, -0.3364000000001093, -1.9880999999996147, -4.972900000000263, -14.0625000000002, -0.18490000000008378, -24.206400000000016, -15.681599999998488, -11.833600000000228, -12.60250000000022, -12.531600000000221, -0.3969000000001174, -7.784099999999073, -24.108100000000018, -0.19360000000008554, -18.318400000000132, -1.392399999999689, -1.4400000000001945, -22.56250000000005, -0.5625000000001359, -6.604900000000266, -11.155600000000236, -2.4963999999995568, -0.022499999999967067, -12.888100000000216, -7.672900000000263, -2.0164000000002167, -1.7955999999996377, -6.451599999999184, -21.715600000000066, -10.627600000000243, -17.056899999998393, -0.16000000000007844, -23.039999999997995, -22.089999999998057, -13.542399999998638, -11.559999999998782, -1.2543999999997077, -0.004900000000014713, -18.147600000000136, -0.8463999999997678, -4.3263999999993725, -20.884899999998137, -19.272100000000115, -1.5376000000001988, -0.32489999999986463, -22.752899999998014, -1.2769000000001864, -6.60489999999917, -23.328900000000036, -5.336100000000265, -14.976899999998537, -11.492099999998787, -0.12249999999992017, -0.7744000000001545, -23.814399999997946, -6.656400000000266, -22.372900000000055, -8.940100000000257, -21.622499999998087, -16.08009999999846, -0.6084000000001404 ], "yaxis": "y" } ], "layout": { "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "x" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "score" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_0, x=\"x\", y=\"score\")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "quantitative-closing", "metadata": {}, "source": [ "\n", " \n", "The plot above shows the score of each parameter set (in this case just one parameter \"x\"). The random search explores the search space very well, so that we can see the inverse parabola." ] }, { "cell_type": "markdown", "id": "loose-alexandria", "metadata": {}, "source": [ "\n", " \n", "## Objective Functions \n", " \n", "The shape of the objective function in the search space is a very important part of the optimization problem, because it heavily impacts the behaviour of the algorithm. Each optimization algorithm is well equiped to solve some kind of objective functions and perform poorly for others. \n", " \n", "A basic classifications of optimization problems is into convex- and nonconvex-problems. Convex optimization problems have a shape, where the score improves continuously as the position gets closer to the global optimum. So it does not have any local optima in its shape. Nonconvex-problems do have these local optima." ] }, { "cell_type": "markdown", "id": "furnished-mapping", "metadata": {}, "source": [ "\n", " \n", "### Convex Optimization \n", " \n", "Lets take a closer look at the convex optimization problem and try out different optimization algorithms:" ] }, { "cell_type": "code", "execution_count": 8, "id": "portuguese-mounting", "metadata": {}, "outputs": [], "source": [ "def convex_function(para):\n", " loss = (para[\"x\"]*para[\"x\"] + para[\"y\"]*para[\"y\"])\n", " return -loss\n", "\n", "\n", "search_space = {\n", " \"x\": list(np.arange(-5, 5, 0.01)),\n", " \"y\": list(np.arange(-5, 5, 0.01)),\n", "}" ] }, { "cell_type": "code", "execution_count": 9, "id": "aware-official", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " x y score\n", "0 -4.92 4.14 -41.3460\n", "1 3.81 -2.28 -19.7145\n", "2 -1.67 -1.67 -5.5778\n", "3 -1.67 1.66 -5.5445\n", "4 1.66 -1.67 -5.5445\n", "... ... ... ...\n", "1995 0.53 0.48 -0.5113\n", "1996 -3.74 -2.94 -22.6312\n", "1997 -3.21 2.37 -15.9210\n", "1998 1.99 -3.61 -16.9922\n", "1999 0.12 0.68 -0.4768\n", "\n", "[2000 rows x 3 columns] \n", "\n" ] } ], "source": [ "hyper_convex_0 = Hyperactive(verbosity=False)\n", "hyper_convex_0.add_search(convex_function, search_space, n_iter=2000)\n", "hyper_convex_0.run()" ] }, { "cell_type": "code", "execution_count": 10, "id": "abandoned-tribute", "metadata": {}, "outputs": [], "source": [ "search_data_convex_0 = hyper_convex_0.search_data(convex_function)" ] }, { "cell_type": "code", "execution_count": 11, "id": "armed-anthropology", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "x=%{x}
y=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ -41.345999999998405, -19.714499999998836, -5.577800000000474, -5.544499999999766, -5.544499999999766, -5.511199999999057, -49.90009999999788, -49.80019999999575, -49.90009999999788, -50, -19.15039999999788, -14.67609999999831, -2.2695999999995538, -32.635300000000285, -38.526099999996504, -5.367699999999398, -13.590899999998326, -29.624199999998204, -20.80519999999805, -13.148499999999348, -28.74169999999942, -20.34490000000046, -30.083299999999284, -5.274099999999393, -19.80979999999801, -2.8368000000002627, -7.862500000000111, -12.309799999999262, -24.55450000000013, -7.68999999999995, -16.937000000000015, -19.62649999999791, -30.13939999999907, -18.23759999999876, -15.425799999998711, -12.706599999999103, -20.3875999999994, -10.707700000000493, -0.4516999999999871, -7.997299999999955, -28.359999999997214, -27.895299999998308, -2.482600000000312, -21.04569999999832, -9.005299999999968, -14.768800000000095, -22.425800000000102, -6.583399999999438, -20.8835999999982, -26.462899999998932, -18.38019999999994, -17.923299999998605, -22.909299999999547, -21.591199999997734, -24.72370000000024, -8.538499999999544, -31.20209999999699, -18.012100000000256, -5.0659999999994385, -18.33769999999793, -23.82879999999792, -21.931399999998643, -31.172499999998852, -10.30450000000042, -17.004799999998713, -20.983300000000217, -12.65449999999948, -3.6913999999999962, -23.72289999999762, -8.51530000000015, -2.1784999999997727, -35.03529999999863, -15.202899999998301, -23.60970000000046, -25.104999999999166, -3.4037000000000575, -28.515999999999007, -24.198500000000447, -11.149000000000104, -17.22250000000015, -21.360799999999934, -29.03169999999715, -47.15330000000006, -22.921300000000098, -22.852099999998284, -15.18249999999823, -17.22499999999803, -3.1203999999994023, -36.634000000000256, -2.859399999999605, -19.888999999999896, -1.307299999999601, -15.752500000000502, -23.144199999997564, -22.944999999997584, -4.313700000000168, -46.368899999995975, -41.89139999999805, -29.005000000000383, -15.835299999998995, -10.63010000000053, -1.8793999999996385, -0.40729999999979294, -25.37049999999855, -7.400000000000494, -7.952499999998917, -22.50979999999896, -1.1187999999997946, -7.832499999999285, -2.5009999999995993, -15.130399999999458, -36.57329999999662, -23.67080000000025, -12.049300000000487, -17.64819999999914, -15.879699999998289, -11.674899999998528, -10.901699999999249, -9.314499999999653, -19.641699999998128, -3.8404999999992455, -25.778599999999063, -8.946500000000158, -17.43999999999907, -15.350599999998458, -22.14609999999769, -10.852199999998554, -10.19299999999939, -28.841600000000255, -31.7872999999984, -15.197799999998427, -19.96249999999884, -3.397700000000224, -7.255300000000204, -9.260499999998746, -31.879999999998205, -12.692499999999699, -6.0085000000001765, -19.54980000000037, -10.594900000000045, -28.06249999999723, -4.900999999999348, -0.9000000000000895, -3.15009999999942, -27.420500000000263, -18.642100000000042, -15.710499999999985, -17.997199999998784, -24.659999999997517, -20.864499999998642, -13.816900000000354, -4.983200000000044, -19.966500000000433, -15.509000000000485, -4.904499999999409, -10.51449999999859, -23.052500000000464, -7.264099999999667, -22.11050000000042, -2.7199999999998328, -14.049800000000275, -37.98019999999653, -6.723999999999014, -28.742799999998372, -21.1680999999981, -3.978399999999259, -1.1708999999999565, -19.459599999998716, -36.944199999998155, -12.164000000000017, -16.48169999999961, -11.562500000000414, -25.6490000000003, -3.3205000000002656, -12.603699999999261, -6.040899999999395, -8.295299999999306, -4.371700000000247, -1.5784000000002567, -5.661199999999994, -19.909299999999142, -3.8424999999998595, -7.7444999999988315, -0.865800000000219, -17.398899999998463, -14.73939999999944, -15.362100000000476, -21.591199999999652, -21.244100000000014, -0.3842000000000689, -23.188999999998877, -8.55050000000013, -12.883399999998836, -0.033699999999983646, -0.2105000000000571, -10.042000000000513, -38.632099999996484, -12.658499999999927, -27.960499999999215, -10.448799999999292, -31.21279999999705, -24.855400000000266, -25.10930000000025, -19.234099999999096, -33.828499999998954, -7.912999999999744, -20.306899999997967, -14.475399999998427, -24.236899999997505, -8.405199999998768, -7.964500000000285, -7.229699999998893, -7.312399999998874, -22.142499999999607, -21.222499999997734, -19.894100000000094, -24.743299999998275, -38.636899999996494, -15.682499999999745, -28.611699999998255, -21.631699999997892, -0.9050000000000168, -44.78129999999608, -14.297299999998728, -18.678799999998756, -1.8207999999997007, -32.64339999999689, -20.359399999999106, -16.035399999999832, -34.13799999999678, -9.62649999999876, -2.1040999999996095, -29.30769999999714, -25.078500000000204, -24.47459999999984, -22.65770000000037, -4.41649999999932, -21.952400000000466, -6.160500000000191, -1.955300000000149, -15.967299999998115, -17.4980000000004, -40.666100000000185, -21.68079999999797, -8.609599999999897, -12.056500000000533, -9.85360000000044, -6.924999999998916, -24.200499999997565, -10.766500000000523, -33.42529999999817, -14.260499999999974, -7.260500000000481, -1.6666000000001933, -34.63449999999892, -24.305299999997633, -12.608900000000203, -12.796899999999917, -7.313300000000104, -9.948500000000413, -12.672499999999172, -21.072500000000144, -20.622399999998496, -24.479299999999395, -29.628099999999097, -29.006899999998215, -22.413700000000315, -24.473000000000095, -19.442899999998026, -19.750900000000485, -22.30369999999861, -14.861600000000466, -20.743400000000484, -12.867999999999775, -19.698499999998408, -15.946900000000443, -6.534899999999139, -4.03239999999944, -21.33199999999783, -35.50900000000026, -1.3213000000000479, -20.60499999999923, -34.033699999996784, -43.57699999999616, -21.23209999999786, -7.920100000000367, -20.44329999999888, -18.34759999999861, -8.451999999999376, -14.548000000000497, -33.66249999999836, -14.329299999998364, -27.69639999999724, -23.11700000000038, -21.141800000000337, -37.41049999999858, -24.724899999998982, -38.424999999998505, -5.140000000000462, -20.917799999998277, -18.49040000000051, -40.61519999999837, -7.096899999999584, -35.5279999999967, -1.5384999999998727, -6.8772999999992575, -18.434599999998177, -8.459600000000504, -2.3105000000003555, -17.52009999999938, -38.23699999999651, -7.255299999998912, -36.632499999998586, -5.050899999999109, -25.874599999999422, -17.798799999999822, -0.57849999999976, -17.236900000000517, -15.462500000000457, -7.996499999999717, -15.742100000000526, -40.05699999999845, -6.868099999999961, -19.785799999999913, -11.40879999999861, -7.769000000000193, -29.430899999997138, -42.45249999999833, -17.832999999998055, -20.712399999999626, -0.9297999999999818, -33.095199999996886, -4.782499999999253, -3.3679999999996264, -21.36039999999939, -20.70279999999935, -14.68729999999993, -18.76009999999805, -12.858400000000032, -0.04770000000002994, -29.46099999999907, -6.92570000000038, -5.781799999999741, -23.53219999999938, -7.707199999998845, -6.7471999999992605, -22.495299999997638, -21.016999999997786, -10.368499999999424, -35.14099999999845, -12.360099999998683, -21.141700000000476, -3.6580999999994583, -26.29890000000042, -24.20689999999986, -41.405799999998244, -43.62340000000013, -8.12889999999885, -12.356899999998465, -20.73799999999797, -12.782499999998626, -29.46259999999844, -17.350899999999694, -30.764499999997092, -38.007399999998555, -30.759999999997095, -15.209600000000425, -24.430499999997622, -0.9305000000000861, -24.86649999999752, -6.288499999999598, -5.562499999999416, -15.036500000000427, -28.681299999997297, -24.5545000000004, -19.896999999998762, -8.493699999998984, -15.347599999999332, -12.559699999999864, -27.522100000000275, -16.89969999999957, -27.586599999997283, -10.896999999999167, -8.467300000000149, -6.314500000000113, -7.785700000000271, -10.069700000000184, -23.704200000000434, -10.086099999998762, -3.6991999999993306, -28.58659999999875, -24.068899999997573, -14.215400000000209, -7.526499999999336, -3.0780999999999947, -10.48049999999915, -26.269299999997354, -18.644499999999756, -8.84499999999949, -36.47249999999825, -8.94609999999953, -6.73999999999903, -6.358600000000462, -35.705000000000275, -36.22810000000027, -7.418899999999434, -13.68249999999975, -26.038899999998492, -33.13699999999814, -12.796899999999745, -32.94259999999687, -14.02419999999872, -10.366599999998925, -15.299999999998272, -25.36199999999952, -6.119999999998997, -1.1258000000002504, -1.2769000000001864, -11.79459999999857, -12.251699999999484, -11.243299999999405, -0.19539999999985952, -17.65009999999961, -3.807699999999469, -22.801299999998058, -15.789799999998824, -39.300999999998396, -4.880200000000197, -5.328200000000322, -15.688000000000516, -28.58579999999718, -28.71129999999853, -38.60199999999648, -10.985299999998553, -10.17609999999959, -22.823999999998232, -18.155300000000416, -4.964200000000342, -17.413700000000443, -28.248100000000395, -17.8407999999983, -9.174799999999374, -4.969000000000359, -7.564099999999083, -6.061699999999641, -38.32520000000022, -23.539399999997933, -0.7824999999998153, -5.514499999999387, -13.925000000000407, -32.29490000000033, -27.800999999999313, -21.680200000000376, -22.620200000000203, -9.178000000000022, -0.09699999999999159, -6.174499999999052, -23.29489999999998, -28.13479999999831, -20.097999999999146, -35.72419999999884, -31.162900000000306, -4.119699999999213, -6.574099999999104, -37.58529999999655, -8.214099999998794, -17.472499999999282, -15.366499999999263, -13.93939999999828, -39.57300000000021, -26.16639999999861, -28.98900000000035, -26.177599999997426, -18.55609999999853, -14.82739999999901, -19.584999999999464, -4.58899999999919, -0.238500000000073, -32.19890000000032, -10.95119999999915, -0.07240000000007792, -3.1281999999993335, -4.6961999999995445, -28.351699999998182, -37.534600000000225, -8.932499999999767, -36.04129999999847, -4.188499999999985, -20.548099999997955, -47.65759999999802, -40.80039999999805, -18.590800000000463, -2.5649000000002466, -30.536500000000295, -26.051300000000413, -24.973699999998008, -25.46899999999858, -7.90329999999897, -17.457699999998052, -30.58009999999855, -12.377299999998609, -11.767400000000483, -18.355999999998474, -8.860900000000214, -24.05449999999783, -5.174099999999824, -15.619599999999492, -3.9064999999995713, -10.098899999998808, -18.1114000000005, -25.56199999999899, -14.239999999999137, -12.540999999999176, -0.8856999999999728, -1.6145000000003085, -13.09690000000013, -19.57809999999955, -6.826000000000421, -4.1904999999992185, -14.896800000000004, -18.31209999999937, -25.110499999997476, -6.656800000000262, -13.410499999998336, -6.894500000000432, -2.95489999999939, -7.020100000000147, -0.8004999999998871, -22.505599999997813, -16.08020000000034, -0.6656000000001763, -7.388199999999029, -3.152000000000326, -5.575699999999709, -20.49939999999945, -37.34729999999657, -33.06399999999686, -0.241999999999849, -15.82579999999921, -5.6899999999990545, -2.4624999999995985, -0.25360000000010857, -5.392000000000162, -36.84129999999811, -25.948900000000414, -4.195299999999235, -17.462600000000243, -35.208999999996735, -26.214499999999198, -28.124199999999064, -2.8001000000001173, -3.808899999999354, -26.443299999998093, -28.112499999997343, -23.372099999997577, -31.1544999999988, -4.243999999999375, -19.7341000000004, -32.17450000000032, -6.530000000000204, -23.164999999997814, -3.141199999999333, -20.933300000000127, -6.8560000000004155, -25.330000000000304, -15.870499999999367, -23.31249999999789, -26.335299999998547, -18.725199999997926, -2.8097000000003858, -12.371599999999248, -1.1712999999997349, -4.72899999999996, -40.67649999999811, -21.718599999998016, -11.554599999998732, -16.839400000000502, -19.624900000000107, -16.567299999998728, -10.108100000000318, -12.137600000000369, -6.820200000000246, -6.5544999999998215, -26.615700000000395, -38.67939999999648, -6.896199999998922, -33.42369999999827, -19.502599999999113, -5.566499999999718, -45.998499999998074, -17.448399999998056, -17.034099999999533, -18.975399999998874, -25.42369999999765, -24.492499999999502, -21.26439999999974, -24.220999999998426, -6.185000000000201, -11.972499999999627, -2.8396999999995867, -11.251999999999034, -19.22440000000016, -6.113599999999134, -7.045000000000485, -6.823999999999274, -26.384200000000412, -27.58479999999725, -17.502499999998072, -19.7729999999988, -5.575999999999089, -28.383699999998527, -43.117699999998045, -21.46499999999993, -21.096499999999978, -37.569999999998316, -25.33700000000043, -18.737000000000467, -17.96890000000035, -0.4645000000000591, -9.41199999999872, -25.72820000000016, -22.270899999997628, -23.120000000000175, -22.480899999997938, -1.640500000000316, -26.496500000000218, -21.938499999999983, -18.84169999999791, -9.114499999999337, -17.912199999999896, -14.945599999999029, -11.942799999998924, -3.2368999999999195, -0.02610000000004365, -2.889999999999463, -0.2664999999998373, -1.4629000000000163, -20.576799999998396, -34.05249999999868, -0.49540000000017503, -5.847299999999143, -10.365799999998696, -31.501000000000325, -30.760899999997044, -5.70049999999929, -9.33140000000029, -12.780199999998384, -20.012499999998752, -14.854600000000357, -14.164900000000127, -0.5729000000002037, -20.9537000000004, -3.3204999999993188, -10.913799999998938, -33.861200000000295, -35.35539999999672, -4.146499999999357, -23.375300000000422, -24.94900000000044, -22.000400000000475, -46.02049999999798, -4.081999999999404, -34.20799999999679, -11.096099999998522, -19.5024999999995, -17.407299999998003, -15.660999999998241, -39.46000000000021, -26.281999999997428, -18.43369999999792, -19.2921999999991, -23.68159999999753, -0.5018000000000724, -13.396999999998986, -17.248499999998312, -21.012499999998088, -5.121699999999132, -11.29320000000022, -21.415999999998547, -0.7785000000002291, -17.738499999999693, -2.045000000000339, -27.159700000000413, -17.78319999999862, -14.472999999999123, -25.40879999999769, -10.023699999998628, -17.593000000000366, -30.606499999998412, -7.992199999998994, -15.80740000000046, -9.68000000000015, -28.49450000000038, -26.96839999999817, -17.349299999998717, -2.328500000000176, -15.423300000000527, -10.164499999998636, -22.441599999999333, -18.09649999999817, -13.583299999999696, -22.896100000000306, -16.39839999999936, -1.3956999999996316, -6.958399999999763, -21.985299999997977, -13.225999999998937, -35.339600000000274, -0.21169999999989078, -39.76999999999852, -40.09010000000019, -26.42000000000023, -17.576199999998888, -18.243700000000064, -13.149799999998374, -10.339600000000527, -42.34809999999809, -24.906599999998285, -29.313999999999215, -49.10650000000002, -26.373799999997328, -22.59139999999806, -32.01530000000033, -4.57539999999917, -27.58969999999954, -30.312499999998867, -27.291399999999538, -12.57380000000036, -22.71399999999763, -3.4292000000000202, -29.204499999998376, -3.2449000000003583, -5.875300000000477, -5.075600000000453, -28.695199999998522, -36.9624999999985, -24.57999999999904, -15.383299999998536, -2.5479999999994734, -0.4149000000000399, -27.976999999998796, -22.513299999999028, -40.656400000000176, -20.200999999997855, -19.49139999999859, -19.187999999998695, -6.011199999999889, -15.352999999999778, -20.721999999998932, -14.896099999999723, -7.812999999999927, -2.858000000000164, -19.516099999998573, -12.658499999999838, -2.6248999999996387, -4.435599999999525, -28.138099999997227, -16.78240000000045, -12.227199999999309, -17.590600000000347, -17.406100000000432, -2.100100000000015, -1.8169000000003213, -6.830600000000336, -5.379999999999958, -23.78979999999789, -21.014499999997888, -5.428800000000357, -5.017999999999697, -12.936399999999274, -6.234500000000325, -4.82649999999954, -36.996200000000236, -4.841999999999576, -13.044200000000531, -22.543299999998986, -22.782499999998485, -30.914599999999076, -16.43239999999972, -21.05379999999798, -0.9117000000002489, -19.6744999999992, -18.37599999999793, -13.552199999998438, -39.8554000000002, -22.974499999999196, -23.344399999999094, -28.21299999999824, -0.30919999999982056, -31.273599999996996, -15.383300000000206, -24.245000000000317, -10.164099999999479, -23.122599999999174, -2.00249999999964, -6.884000000000482, -27.28999999999891, -5.511400000000473, -14.292999999999783, -12.083399999999843, -16.056399999999694, -19.93209999999828, -16.620499999998348, -3.23719999999967, -3.3055999999998082, -34.262499999996784, -11.632400000000438, -27.893199999998473, -1.7992000000002217, -20.054799999999815, -27.7157000000003, -0.8650000000001634, -45.6324999999981, -26.748499999999396, -17.588799999999516, -9.499399999999717, -9.451300000000133, -8.660799999998769, -9.823399999998648, -27.0425000000004, -19.670899999999126, -8.328500000000478, -10.254399999999963, -11.43459999999961, -33.77049999999848, -13.087400000000418, -22.792099999998648, -21.73939999999769, -16.135299999998903, -16.44879999999998, -17.80579999999809, -8.302100000000182, -3.962599999999396, -20.36660000000046, -2.4797000000000797, -22.4441999999977, -6.664499999999185, -1.2289999999998156, -5.734100000000392, -23.029699999999632, -28.434499999999062, -9.472999999999114, -1.0933000000000153, -32.7065000000003, -4.692500000000088, -12.15250000000011, -2.07609999999976, -12.032099999998675, -25.122499999997576, -43.93490000000013, -14.307999999998255, -16.06129999999988, -45.70959999999809, -12.368500000000267, -23.832499999998674, -14.855299999999385, -27.836500000000388, -20.937999999998215, -25.402400000000213, -1.1904999999998065, -24.852199999997637, -11.368999999999627, -20.165000000000347, -14.000499999998748, -3.122599999999752, -20.21849999999867, -2.200900000000177, -27.378499999997338, -23.804099999999224, -11.603699999998577, -6.598600000000196, -4.304900000000186, -45.03849999999805, -2.662600000000326, -0.28529999999982797, -5.188999999999336, -0.33049999999989427, -30.50649999999924, -11.696199999999585, -3.6159999999995556, -14.105799999999803, -20.856999999997836, -20.82409999999843, -5.82579999999939, -12.035299999998646, -29.945799999999032, -20.179299999998257, -15.232400000000334, -17.18689999999888, -43.14739999999618, -19.766799999998113, -17.88039999999924, -28.15879999999843, -7.753999999998847, -37.78659999999823, -22.292499999998142, -12.972999999999072, -16.427300000000354, -7.896200000000265, -25.71929999999969, -10.795699999998757, -18.80730000000038, -21.508099999997977, -31.694499999996957, -29.036499999997243, -24.435999999997584, -5.457699999999496, -7.969000000000488, -11.381199999998623, -2.084000000000158, -25.138099999999266, -5.631400000000353, -26.916499999999328, -3.9320999999993016, -14.027399999999313, -16.635600000000455, -2.0563999999996523, -1.0908999999998745, -21.77370000000023, -17.086599999999727, -12.087699999998472, -22.517199999998958, -11.884899999998646, -16.06249999999813, -24.868000000000055, -0.40050000000010444, -6.772900000000386, -12.947599999999333, -19.155199999999507, -18.324999999998024, -4.836099999999796, -11.336200000000455, -20.903199999999664, -4.0948000000004345, -7.873000000000001, -24.514899999997468, -26.464099999997337, -14.400999999998358, -7.276099999999325, -5.262100000000456, -36.687999999996606, -36.06099999999856, -9.83050000000018, -16.570899999998257, -12.736099999999118, -1.3905999999998042, -5.908900000000479, -6.162499999999407, -3.930100000000406, -0.9602000000000613, -15.682499999998495, -6.791300000000468, -3.5521000000001663, -18.91969999999821, -19.321299999998164, -10.60660000000035, -22.272999999998568, -9.183399999998972, -18.938499999999625, -20.11690000000045, -0.1439999999999427, -7.494499999999704, -38.10890000000023, -23.971999999999824, -19.50440000000019, -1.8708999999995088, -44.072499999996126, -5.906500000000275, -22.78369999999759, -35.02529999999881, -9.184499999998975, -24.411399999998324, -11.504799999998529, -10.033699999998646, -3.9233000000004274, -3.0370000000001944, -14.410600000000061, -1.206900000000006, -11.528499999999289, -21.985299999999967, -5.497999999999088, -28.672999999998304, -14.224999999998344, -5.6889999999993695, -9.007699999999344, -22.018000000000292, -21.352399999997722, -22.83250000000037, -17.26599999999904, -9.672199999998654, -4.7008999999998995, -6.713600000000174, -3.190500000000382, -4.6889999999997745, -15.400399999998324, -2.478499999999952, -12.658899999999488, -35.13699999999837, -8.138600000000023, -3.049999999999363, -18.555299999999242, -26.36289999999755, -19.483700000000482, -1.2136999999996392, -10.74959999999857, -25.907299999998287, -1.926799999999854, -19.316799999999084, -9.088999999999276, -4.247199999999393, -17.186500000000485, -9.787599999998658, -19.48679999999949, -22.642099999997605, -0.6586000000000657, -14.102500000000527, -16.24569999999895, -7.452800000000143, -39.024399999998124, -28.704399999997246, -27.518600000000326, -37.05649999999659, -15.371299999998525, -29.2729999999985, -6.63650000000035, -36.72499999999661, -19.79690000000049, -4.452999999999179, -10.17699999999888, -11.409799999998576, -23.8180999999983, -20.57680000000049, -15.976999999998663, -10.976400000000517, -39.656499999998196, -14.436499999998768, -28.209699999998488, -22.895299999997725, -10.972999999999635, -6.4524999999998425, -22.88799999999791, -6.073699999999087, -7.129800000000259, -24.696999999999775, -16.16530000000028, -14.107399999999522, -17.108799999999995, -10.910800000000503, -19.014500000000133, -4.520499999999984, -7.276900000000378, -11.336899999998638, -9.220900000000297, -8.724999999999799, -29.916199999997165, -15.780499999998511, -22.416999999999142, -23.72839999999826, -5.882499999999164, -3.467299999999296, -19.097999999999967, -33.89089999999683, -35.25799999999838, -24.248499999999886, -23.516099999997635, -0.5124999999997757, -18.429199999998552, -25.212499999998165, -26.162899999998835, -18.885800000000465, -20.61800000000049, -6.375199999999856, -40.15359999999825, -31.42120000000033, -12.924199999998644, -29.010499999998835, -13.980799999999062, -1.6207999999999223, -14.548499999998466, -33.2077000000003, -9.684200000000226, -35.79889999999832, -3.90979999999933, -28.057000000000393, -42.15620000000016, -0.6153999999997777, -6.589599999999471, -2.462500000000136, -1.681000000000022, -12.785000000000437, -22.75540000000006, -7.07200000000033, -6.522500000000466, -21.241799999999607, -3.6460999999993993, -1.3625000000002296, -20.90499999999861, -4.456400000000019, -23.124799999999365, -12.54250000000037, -0.16899999999993737, -38.38330000000022, -16.791699999999704, -1.791399999999813, -0.7309000000002097, -26.82099999999736, -10.733599999999107, -17.167999999999576, -23.142099999997612, -28.44609999999719, -33.9775999999968, -13.942400000000532, -0.06759999999994169, -14.05219999999827, -13.792400000000525, -20.18929999999898, -19.154499999998926, -5.5655999999992005, -32.91080000000032, -33.71540000000031, -23.633800000000377, -7.998500000000441, -19.416400000000497, -22.889699999999312, -16.141000000000062, -18.43849999999859, -37.212499999998165, -28.996900000000274, -1.4050000000000593, -15.981799999998373, -28.558400000000344, -19.216000000000427, -18.749300000000126, -13.999299999998403, -23.162499999997564, -22.418599999999735, -23.854899999999276, -5.216499999999225, -16.60729999999921, -25.992499999997438, -17.67329999999984, -23.23279999999786, -28.211299999997305, -2.190799999999587, -24.204199999997854, -30.672499999997022, -7.171700000000195, -4.230500000000435, -14.336500000000438, -6.38170000000048, -20.695700000000148, -12.37370000000031, -20.830899999999705, -5.813299999999997, -35.098899999998835, -12.175699999998985, -0.036500000000056, -38.540499999998445, -9.659199999999801, -6.28489999999944, -23.97969999999799, -7.766600000000385, -24.59249999999907, -25.508500000000282, -15.192199999999609, -17.806099999998132, -18.33769999999793, -3.5715999999999033, -16.89220000000015, -32.70339999999899, -22.008999999999155, -7.537700000000163, -5.595300000000113, -18.620199999997922, -34.072999999996796, -22.18099999999793, -33.4322000000003, -3.6161000000001464, -35.352199999998575, -3.429700000000201, -2.5449999999994226, -11.740900000000487, -8.232999999999198, -2.1650000000003295, -10.385299999999802, -10.003999999999326, -5.828500000000159, -13.368999999998397, -37.9047999999981, -26.412200000000283, -19.007299999998093, -20.751700000000383, -31.162899999997038, -1.4450000000001089, -10.679299999999373, -9.426499999999857, -10.726599999998719, -0.1809999999999454, -4.8304999999991995, -10.345000000000368, -22.262400000000177, -28.79539999999856, -17.114499999999676, -3.7025000000003856, -2.7557000000003624, -19.658000000000353, -11.858499999999829, -34.13799999999681, -23.162499999997692, -0.5200000000001399, -10.260199999998603, -1.6783999999996129, -6.008499999999401, -25.252999999997506, -13.196499999998633, -26.45199999999837, -4.920499999999575, -26.519200000000424, -1.791399999999813, -41.57320000000017, -5.932999999999436, -0.7624999999997436, -0.7113999999998247, -23.689000000000416, -6.324099999999003, -11.592499999999772, -4.116499999999229, -18.352800000000496, -7.788499999998855, -13.422399999998515, -24.384999999999685, -26.291600000000212, -4.113999999999871, -34.05649999999682, -19.814799999998662, -16.092499999998694, -17.52479999999887, -22.848999999998075, -22.990899999999453, -11.42799999999935, -26.934799999999214, -8.308899999999596, -11.348899999999638, -26.18119999999956, -24.638499999998114, -13.424499999998611, -16.052900000000395, -7.998500000000488, -37.01519999999868, -24.542599999999158, -20.418499999998225, -10.961000000000224, -10.676799999999451, -22.343399999999725, -6.139299999999429, -9.332500000000444, -2.1124999999996734, -10.442899999998888, -27.37730000000035, -10.526499999998897, -26.16369999999823, -8.262499999999335, -8.033000000000335, -5.861299999999219, -5.268499999999436, -15.770499999998755, -16.78729999999809, -26.78849999999949, -8.576199999999293, -26.06439999999753, -20.82280000000032, -18.22969999999802, -23.32839999999755, -21.752099999999988, -5.457700000000038, -11.695699999999217, -2.6119999999996244, -17.937999999998258, -11.107699999999362, -24.265799999998762, -15.07090000000047, -3.292999999999719, -10.40449999999975, -13.49649999999981, -17.176899999998483, -5.385999999999852, -17.43409999999855, -14.482100000000361, -41.06169999999632, -22.921999999998473, -25.4439999999974, -19.038499999999424, -14.290599999999618, -19.095700000000427, -15.68339999999814, -14.700799999999798, -28.85059999999861, -30.384100000000327, -24.519999999998095, -5.154499999999532, -8.366499999999109, -14.672800000000311, -5.1857000000003435, -23.58009999999765, -23.615999999997662, -17.368099999999306, -3.173299999999641, -0.13250000000000928, -10.256899999998609, -29.468899999998673, -18.87879999999978, -5.000500000000365, -26.188199999999206, -11.0289999999991, -12.592400000000524, -12.862600000000317, -5.081000000000376, -20.97319999999838, -26.450499999997405, -10.857600000000357, -35.12479999999837, -4.289000000000163, -28.005299999999032, -17.176899999998067, -0.1930000000001239, -19.337000000000376, -9.276500000000366, -31.40089999999919, -13.764499999998627, -9.091999999999075, -20.367699999999722, -6.350499999999834, -1.386399999999796, -38.01920000000024, -20.540899999997787, -22.395399999998002, -5.645300000000059, -24.132499999998657, -8.883699999998889, -12.873999999999608, -27.965599999997224, -0.20180000000005535, -32.41940000000029, -28.664099999998157, -4.774399999999558, -12.13640000000051, -37.24739999999812, -11.133700000000474, -2.8617999999995796, -2.1425000000002603, -17.852500000000322, -0.9768999999998667, -5.448499999999067, -6.196100000000164, -20.41849999999805, -6.939699999999464, -21.759700000000365, -12.243400000000122, -19.27849999999787, -30.425799999998276, -1.4929999999995824, -0.5296999999997919, -12.893199999998508, -2.573599999999413, -24.17860000000045, -6.016399999999868, -7.990599999998836, -26.17920000000042, -19.682000000000265, -19.45969999999805, -27.228999999998262, -0.7421000000001411, -28.664499999997272, -20.625799999999238, -17.895399999998347, -12.167299999999344, -12.982399999999036, -18.603700000000345, -40.66609999999835, -5.849300000000465, -12.255700000000427, -16.041999999999064, -34.43049999999861, -18.843700000000105, -15.103300000000518, -23.0290000000002, -1.9604000000000827, -15.558500000000013, -33.67999999999826, -16.100899999998106, -28.9401999999994, -3.2124999999994386, -30.8017999999971, -13.666500000000383, -19.475600000000266, -38.79089999999648, -28.381699999998844, -10.589000000000405, -23.076499999999292, -4.00689999999929, -5.373800000000462, -26.189299999997434, -28.102099999998224, -12.680199999999545, -26.86849999999869, -10.48719999999998, -39.69529999999832, -23.991199999997527, -5.572999999999068, -8.093300000000463, -1.3649000000002893, -40.812500000000185, -14.03619999999987, -12.51370000000012, -19.786499999999265, -23.761999999997595, -16.757000000000193, -6.05199999999952, -9.645799999999184, -13.56339999999995, -26.088499999999556, -3.5594000000003985, -15.173000000000018, -11.945600000000464, -14.089000000000503, -22.421299999998695, -20.189599999997824, -20.448499999998475, -10.49920000000049, -2.3849999999997706, -15.129999999999335, -11.832999999999846, -17.887399999998127, -16.285699999998098, -4.414600000000442, -24.198500000000447, -19.501199999998978, -20.097799999997832, -24.992499999997545, -43.6569999999982, -25.509200000000426, -8.304499999999278, -24.761599999998722, -40.62440000000018, -26.700999999997332, -14.498099999998443, -39.18650000000021, -11.764000000000044, -24.382099999997557, -3.0850000000002096, -6.126799999998992, -33.302599999998606, -17.91080000000007, -11.074000000000453, -1.0121000000002573, -22.48849999999763, -27.147599999997293, -4.0544999999994245, -15.657299999998278, -15.480399999999037, -15.13060000000045, -16.02249999999924, -31.486499999996987, -2.9856999999997296, -18.412999999999936, -8.804899999998927, -1.9141000000003, -9.306400000000481, -21.003699999998556, -4.589000000000418, -27.408800000000408, -0.8698000000001462, -16.79530000000005, -14.482099999998404, -12.456099999998404, -15.55450000000041, -12.825799999998942, -10.80369999999972, -39.512499999998184, -16.69769999999945, -4.260499999999867, -22.908099999998832, -10.256499999999207, -22.90010000000033, -21.491300000000265, -10.481299999998818, -8.762399999999218, -21.672500000000387, -20.756499999998365, -17.484999999999644, -13.080399999998424, -18.062500000000302, -17.270499999999966, -7.720399999999326, -24.141799999997676, -19.98399999999831, -0.41050000000011255, -2.4904999999998747, -34.97539999999826, -1.8559999999996482, -23.744499999998656, -10.249700000000065, -18.729999999999897, -24.431399999997705, -29.104999999998462, -17.67679999999809, -2.3041999999994753, -22.815699999997893, -10.892500000000227, -22.33359999999889, -3.054499999999851, -26.270499999998457, -10.624999999999142, -12.18599999999985, -12.098599999999323, -28.711999999998667, -11.075599999999667, -14.632400000000489, -9.792100000000454, -13.645599999998531, -11.289999999998628, -11.650000000000505, -4.142500000000184, -7.576899999998873, -25.761699999999657, -20.81049999999811, -9.472999999999114, -19.46569999999814, -11.058500000000196, -9.31929999999955, -24.507199999998146, -6.92180000000028, -12.84100000000028, -25.36199999999952, -24.30580000000001, -23.821200000000456, -23.66409999999985, -1.7929000000000876, -20.328999999998054, -17.660899999997994, -40.77889999999634, -13.470099999999231, -4.494099999999257, -5.366599999999209, -5.236199999999099, -3.0325000000001414, -3.1449999999996017, -26.413699999998357, -3.7744999999992572, -3.9463999999995503, -14.619199999999436, -31.296199999998407, -25.126099999998058, -7.719400000000498, -4.936999999999269, -1.4066000000001233, -26.41640000000032, -47.44999999999801, -5.059399999999767, -14.592499999999964, -14.952500000000525, -34.3444999999968, -25.19449999999818, -10.549699999999685, -8.654600000000364, -19.850499999998643, -22.958499999999837, -4.5704000000004195, -2.756499999999433, -9.781700000000466, -11.064999999998753, -25.559300000000412, -23.26479999999951, -17.806099999998622, -24.99369999999767, -21.360799999999934, -33.40090000000032, -34.171599999996786, -6.292899999999376, -6.535400000000463, -17.424999999998704, -9.522499999999669, -3.7104999999996267, -2.6643999999994983, -2.0725000000000033, -27.32889999999738, -19.80010000000035, -13.470499999999259, -10.448800000000356, -8.333000000000323, -9.616900000000449, -44.00959999999826, -20.022499999999752, -19.909300000000357, -24.968500000000404, -2.7753999999994807, -2.578899999999905, -4.371999999999498, -15.50260000000052, -37.67779999999852, -14.928999999999679, -4.594899999999448, -36.826399999998145, -25.968999999999667, -2.8051999999996036, -20.62249999999831, -7.63300000000045, -9.549799999999529, -17.398899999998346, -27.710800000000376, -18.35529999999817, -2.8344999999999834, -3.186499999999807, -16.298499999999205, -16.414599999998412, -17.287399999999494, -12.455199999998515, -3.090600000000393, -44.554899999996096, -8.98289999999956, -16.249000000000148, -2.386599999999992, -23.96879999999834, -10.573599999998663, -31.4592999999983, -18.95330000000032, -2.0873000000000412, -24.132199999998186, -8.547999999999448, -24.19360000000041, -8.512400000000186, -12.65449999999839, -31.698499999998877, -12.888899999999392, -9.759999999998886, -17.846899999997966, -6.626599999999363, -25.135699999998465, -0.04249999999996621, -14.375300000000326, -28.611200000000316, -5.6320999999991015, -13.2544999999993, -31.648999999999123, -24.040999999997553, -18.348200000000283, -6.720499999999503, -1.9337000000001456, -23.408200000000367, -45.222500000000096, -0.9178000000002081, -23.488899999997884, -0.4137999999998203, -24.966099999998043, -15.63920000000034, -4.663700000000196, -15.208100000000483, -27.433799999997294, -2.297800000000137, -18.9466000000005, -2.6224999999994214, -20.952499999998416, -5.886500000000322, -7.906600000000183, -8.822500000000414, -27.460099999999276, -12.686600000000432, -17.450500000000282, -4.579700000000264, -17.82889999999797, -1.8441999999999812, -2.9000000000002175, -26.14600000000032, -22.798899999997765, -1.3699999999996304, -38.90770000000021, -11.07619999999853, -6.253200000000462, -42.06249999999817, -20.382099999997855, -6.712399999999326, -30.350499999997083, -9.782599999999315, -11.978099999998447, -20.174500000000457, -25.437599999998582, -24.773200000000344, -4.458500000000443, -1.512999999999586, -7.892899999999115, -24.420499999999937, -16.321999999998447, -2.9236999999995703, -7.92849999999943, -3.626000000000114, -9.075400000000462, -17.76770000000004, -13.06450000000012, -11.913700000000437, -5.408000000000435, -44.462899999998115, -13.663399999998441, -8.98930000000051, -10.318099999999708, -25.228899999999218, -4.298499999999841, -10.239400000000527, -4.849699999999659, -25.76329999999836, -22.446899999998283, -7.932199999999471, -23.582599999997644, -7.548999999998877, -4.748499999999246, -0.5608000000001978, -41.559399999998405, -31.20009999999839, -11.436999999998678, -34.782499999996766, -5.451399999999849, -1.3025000000002749, -21.47409999999889, -24.92319999999746, -26.407999999999223, -3.2224999999993575, -23.300099999997876, -0.369999999999835, -0.8067999999998163, -22.513299999999028, -22.19859999999774, -18.021699999998027, -16.804999999998962, -27.235299999999278, -2.234000000000148, -14.660500000000269, -6.90439999999919, -19.065799999998877, -6.75609999999988, -24.38280000000038, -17.01290000000041, -37.25649999999657, -31.81249999999701, -11.856200000000518, -18.19459999999804, -24.82649999999831, -21.35199999999833, -26.764999999999556, -9.902499999998629, -19.25689999999816, -21.693200000000264, -8.94979999999881, -21.082500000000298, -5.707699999999938, -21.725600000000362, -16.355700000000294, -28.012899999999355, -27.15939999999955, -7.150400000000437, -14.952499999998201, -23.744499999997558, -20.98000000000007, -2.8969999999994247, -19.65799999999939, -7.988500000000361, -19.808899999998225, -3.1024999999999636, -40.42449999999824, -11.193800000000426, -11.405000000000364, -17.021299999999062, -9.880899999998634, -24.693299999997574, -3.341300000000209, -26.900899999998913, -14.012099999998334, -8.503399999998845, -1.8100000000001488, -7.338499999999046, -14.870499999998216, -10.732500000000021, -23.822800000000413, -20.155699999998006, -30.357199999997135, -25.60499999999901, -1.1452999999997273, -2.2237000000000866, -5.596999999999454, -8.405800000000516, -26.179700000000388, -12.220100000000267, -3.7023999999992623, -32.76409999999899, -15.055300000000443, -14.989600000000248, -2.503699999999452, -17.740999999999552, -12.032500000000525, -16.77799999999945, -13.624199999998652, -29.66209999999837, -7.712899999999815, -19.75770000000015, -12.70119999999971, -1.9924000000001751, -28.245200000000267, -27.816399999997362, -0.4348999999999964, -19.164799999999747, -8.526499999999515, -6.461299999998989, -27.084999999998843, -48.7124999999979, -7.920100000000367, -23.34599999999761, -22.573299999998962, -20.55649999999924, -34.15129999999865, -32.265999999998726, -1.9215999999999096, -13.270099999999966, -22.722500000000128, -6.041800000000339, -12.73459999999842, -0.1489000000001066, -36.311300000000266, -20.752399999998726, -0.6524999999997355, -23.622099999999314, -6.088899999999661, -2.3722000000000056, -29.479299999999245, -15.47010000000028, -13.52649999999963, -8.456500000000409, -0.41049999999994624, -11.969800000000493, -16.439199999999214, -37.55559999999847, -24.914499999997492, -30.20899999999714, -28.35049999999879, -14.822599999998928, -18.539299999999063, -22.604900000000413, -8.190800000000388, -2.30599999999994, -12.342600000000253, -4.666599999999388, -1.7497999999995757, -15.87219999999919, -4.937000000000212, -4.940800000000437, -14.638500000000482, -18.285800000000386, -24.165200000000183, -22.728099999997664, -4.589599999999327, -34.043399999998705, -21.45159999999822, -33.38890000000032, -22.297699999999416, -1.6337000000001287, -21.556899999999366, -1.828099999999515, -44.712499999998165, -25.00129999999762, -25.758899999997375, -1.8250000000001994, -12.65450000000012, -14.383699999998608, -16.629699999999026, -0.5625000000001998, -2.4905000000003694, -0.311200000000072, -35.02529999999675, -17.38969999999831, -19.660500000000496, -8.260199999999775, -14.986899999998515, -32.57689999999689, -5.9305999999990355, -2.0449000000002178, -9.30079999999917, -21.92449999999772, -31.09250000000035, -6.340000000000463, -36.00999999999824, -32.874399999996875, -5.016999999999266, -6.827600000000319, -11.334500000000505, -15.140199999999998, -27.44649999999894, -20.81319999999796, -29.72809999999709, -13.60250000000039, -31.73859999999913, -5.918799999999052, -42.65770000000015, -4.2403999999994735, -6.106900000000394, -19.51700000000041, -18.98729999999911, -14.004499999998409, -4.455400000000433, -6.705800000000451, -23.979599999999888, -14.692500000000097, -2.2945000000001086, -26.15649999999822, -17.8489999999981, -2.5329999999994146, -7.6144999999988885, -6.196500000000139, -13.243599999998356, -14.0424999999989, -2.76019999999978, -16.71009999999814, -4.016500000000006, -30.54009999999876, -1.229599999999615, -19.135699999999282, -3.366500000000012, -14.03169999999856, -11.093199999999293, -17.24409999999869, -15.548200000000062, -1.1665000000001826, -0.08080000000000509, -30.80899999999912, -5.849600000000467, -0.311400000000153, -23.39600000000018, -20.457999999998492, -20.643300000000444, -0.7207999999999778, -22.432399999999262, -23.77959999999755, -27.939399999998365, -35.19849999999887, -16.05249999999975, -23.120499999998533, -5.989000000000201, -46.87119999999804, -42.35649999999836, -6.012500000000394, -20.004100000000296, -44.20000000000012, -18.363599999999202, -18.772999999998632, -7.300999999998883, -24.43089999999763, -14.99890000000027, -20.36289999999955, -17.116200000000482, -36.54159999999873, -18.52959999999937, -11.858599999999381, -25.491699999997415, -36.01329999999666, -17.822500000000375, -22.262499999999513, -23.582899999999768, -21.219999999997775, -20.189599999999473, -11.048899999999918, -16.147699999998306, -20.73999999999852, -13.689999999999207, -6.884199999999646, -8.540800000000232, -10.946000000000058, -27.724499999999516, -18.449799999998543, -11.963300000000404, -11.00689999999863, -5.309200000000461, -10.702899999998868, -2.520199999999466, -15.587999999998159, -2.39719999999945, -12.252999999998675, -19.694799999999614, -2.411600000000268, -15.95620000000021, -20.928199999998938, -12.142600000000321, -32.90449999999823, -19.908499999998337, -8.400199999998891, -17.14100000000021, -21.271400000000483, -33.46250000000026, -15.491599999999455, -1.1986000000002388, -0.5112999999997629, -22.63120000000046, -15.9209999999995, -16.99219999999962, -0.47679999999980915 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "showlegend": false, "type": "scattergl", "x": [ -4.920000000000002, 3.809999999999812, -1.670000000000071, -1.670000000000071, 1.659999999999858, 1.659999999999858, -5, 4.989999999999787, 4.989999999999787, -5, 3.5199999999998184, 3.5499999999998177, 0.13999999999989043, -4.880000000000003, 4.899999999999789, 2.2599999999998452, 3.029999999999829, 4.889999999999789, 4.539999999999797, 2.7699999999998344, -4.99, -3.8000000000000256, -4.970000000000001, -0.4600000000000968, 1.0299999999998715, -1.6800000000000708, 0.6399999999998798, -1.9300000000000654, -4.920000000000002, 1.1999999999998678, -4.07000000000002, 1.8399999999998542, -4.630000000000008, 3.8999999999998103, -0.870000000000088, -1.5500000000000735, 2.49999999999984, -1.5900000000000727, 0.4599999999998836, 1.1799999999998683, 2.9999999999998295, -2.370000000000056, -1.5100000000000744, -0.8400000000000887, -2.7800000000000473, -3.820000000000025, -0.23000000000010168, -1.0300000000000846, -0.3000000000001002, 3.499999999999819, -4.210000000000017, 4.0799999999998064, 1.9299999999998523, 2.2599999999998452, -4.810000000000004, -1.8400000000000674, 3.7499999999998135, -0.6100000000000936, -0.5600000000000946, 2.889999999999832, 0.11999999999989086, 4.149999999999805, 3.499999999999819, -0.9700000000000859, 3.91999999999981, -4.53000000000001, -2.560000000000052, -1.670000000000071, 1.9799999999998512, -2.880000000000045, -0.6800000000000921, -4.470000000000011, 3.6999999999998145, -3.5100000000000318, 2.9499999999998305, -1.6900000000000706, 3.219999999999825, -3.160000000000039, 0.5699999999998813, -1.0658141036401503e-13, -4.580000000000009, 3.259999999999824, -4.780000000000005, -4.780000000000005, 4.6399999999997945, 3.43999999999982, 3.349999999999822, 0.4799999999998832, -4.560000000000009, 1.6499999999998582, 0.8899999999998744, 0.7299999999998779, -3.2900000000000365, 3.2899999999998233, 3.7099999999998143, 0.3899999999998851, 4.799999999999791, 4.949999999999788, -4.05000000000002, -1.9300000000000654, -2.3500000000000565, -0.050000000000105516, 0.36999999999988553, -2.5900000000000514, -2.2000000000000597, 2.7099999999998357, 3.529999999999818, -0.2800000000001006, -0.870000000000088, -0.19000000000010253, 2.49999999999984, 4.3799999999998, -4.720000000000006, -3.070000000000041, 3.209999999999825, 3.859999999999811, 3.069999999999828, -1.5600000000000733, -2.270000000000058, 0.43999999999988404, 1.429999999999863, -3.9500000000000224, -2.9600000000000435, 3.359999999999822, 0.2499999999998881, 4.099999999999806, 2.609999999999838, 2.6299999999998374, -5, 4.519999999999797, 0.4699999999998834, 3.7999999999998124, -1.8400000000000674, -2.6800000000000495, 2.7699999999998344, -2.840000000000046, 1.8899999999998531, 0.3899999999998851, -1.4700000000000752, -3.1500000000000394, 4.3099999999998015, 2.2099999999998463, 0.299999999999887, 1.7399999999998563, -1.8200000000000678, 0.3899999999998851, 0.8099999999998762, -1.760000000000069, 4.3799999999998, 4.139999999999805, -0.8000000000000895, -2.0600000000000627, -3.960000000000022, -3.410000000000034, -0.39000000000009827, 1.8399999999998542, -3.410000000000034, 1.9599999999998516, -4.090000000000019, 1.2799999999998661, -0.3700000000000987, 4.089999999999806, 0.8199999999998759, 4.619999999999795, 4.599999999999795, 0.8999999999998742, -0.78000000000009, 3.999999999999808, -3.6100000000000296, -3.3800000000000345, 2.03999999999985, -1.0000000000000853, -4.730000000000006, -0.09000000000010466, -1.9900000000000642, -0.7200000000000912, 2.6699999999998365, -2.090000000000062, -1.2200000000000806, -2.140000000000061, 3.1799999999998256, -1.4400000000000759, 1.8599999999998538, -0.33000000000009955, -0.42000000000009763, 2.549999999999839, -3.450000000000033, -4.340000000000014, 0.28999999999988724, -0.590000000000094, 3.7099999999998143, 0.5599999999998815, -0.6500000000000927, -0.09000000000010466, -0.4400000000000972, -1.8400000000000674, 4.3899999999998, 1.1699999999998685, 2.6599999999998367, -1.580000000000073, 4.879999999999789, -4.770000000000005, -1.3700000000000774, 3.2899999999998233, -4.940000000000001, -2.1700000000000603, 4.3699999999998, 3.729999999999814, 3.999999999999808, 2.03999999999985, -2.8200000000000465, 2.1899999999998467, 1.8199999999998546, 1.7699999999998557, 2.559999999999839, -4.4600000000000115, 4.719999999999793, 4.879999999999789, -3.60000000000003, 4.809999999999791, 1.009999999999872, -0.7900000000000897, 4.469999999999798, 3.729999999999814, 3.91999999999981, -0.2800000000001006, 4.0299999999998075, 3.249999999999824, -3.770000000000026, 4.239999999999803, 0.9299999999998736, -0.04000000000010573, 4.409999999999799, -4.890000000000002, -4.890000000000002, -1.9100000000000659, 2.089999999999849, -3.680000000000028, 0.3299999999998864, -1.3700000000000774, 2.879999999999832, -1.4600000000000755, -4.310000000000015, 4.619999999999795, 1.3599999999998644, -2.4900000000000535, -2.940000000000044, 2.0499999999998497, 2.179999999999847, -2.040000000000063, 4.929999999999788, -3.6600000000000286, -2.2900000000000578, -1.290000000000079, -4.99, 4.669999999999794, 0.07999999999989171, -3.3700000000000347, 0.6699999999998791, -0.9100000000000872, 3.0999999999998273, -4.580000000000009, 4.319999999999801, -4.370000000000013, 2.9099999999998314, 4.86999999999979, -1.4400000000000759, -0.41000000000009784, 4.269999999999802, -2.700000000000049, 4.209999999999804, -1.7000000000000703, -2.9500000000000437, -3.180000000000039, 4.3599999999998005, -3.6500000000000288, 0.17999999999988958, -0.18000000000010274, 1.5799999999998597, -4.830000000000004, 0.5299999999998821, -3.450000000000033, 4.159999999999805, 4.929999999999788, 4.3899999999998, -0.5500000000000949, 3.719999999999814, 4.099999999999806, 2.5799999999998384, -3.260000000000037, 4.549999999999796, 1.2299999999998672, 3.999999999999808, -4.330000000000014, -4.370000000000013, 3.8899999999998105, 3.4299999999998203, -4.710000000000006, -1.7000000000000703, 4.529999999999797, -3.1000000000000405, 4.259999999999803, -1.6000000000000725, 3.639999999999816, -0.7200000000000912, -0.47000000000009656, 0.6099999999998804, -2.3600000000000563, -0.920000000000087, 2.649999999999837, 4.0299999999998075, 1.2799999999998661, 3.90999999999981, 1.6999999999998572, 2.1499999999998476, -4.020000000000021, 0.6899999999998787, -2.8700000000000454, -3.5600000000000307, -2.130000000000061, -2.8600000000000456, 4.089999999999806, -2.3500000000000565, 0.8299999999998757, 3.219999999999825, -2.7700000000000475, 3.029999999999829, -4.9300000000000015, 3.869999999999811, 1.799999999999855, -0.730000000000091, 4.739999999999792, 2.1499999999998476, -0.6400000000000929, -3.9000000000000234, 2.6199999999998376, 1.0699999999998706, 1.239999999999867, 0.7799999999998768, 0.05999999999989214, -4.53000000000001, -2.560000000000052, -1.670000000000071, -4.210000000000017, 2.2399999999998457, 2.559999999999839, 3.91999999999981, 4.069999999999807, -1.9400000000000652, 4.3099999999998015, 3.5099999999998186, -2.9100000000000446, -0.10000000000010445, -3.9000000000000234, -4.870000000000003, -4.570000000000009, -4.970000000000001, 1.1699999999998685, 1.5999999999998593, 4.459999999999798, 3.5599999999998175, -3.0500000000000416, -3.780000000000026, 2.5799999999998384, 3.9299999999998096, 4.9199999999997885, -1.4000000000000767, 4.679999999999794, -0.9100000000000872, 2.1899999999998467, 2.0599999999998495, -0.6400000000000929, -3.6200000000000294, 4.979999999999787, -2.4800000000000537, -2.1100000000000616, 0.15999999999989, 2.7999999999998337, 1.3899999999998638, -1.900000000000066, 2.139999999999848, 2.789999999999834, -1.3100000000000787, -2.8700000000000454, -2.430000000000055, -2.790000000000047, -3.160000000000039, -3.9900000000000215, 0.6899999999998787, 1.8399999999998542, -3.7100000000000275, 4.3999999999998, -0.050000000000105516, 2.5699999999998386, 0.909999999999874, -1.1400000000000823, 2.97999999999983, -4.06000000000002, 2.3899999999998425, 4.709999999999793, -1.900000000000066, 0.6999999999998785, -1.25000000000008, -4.430000000000012, -4.350000000000014, -1.25000000000008, -3.2800000000000367, 4.419999999999799, -2.8700000000000454, -3.12000000000004, 4.509999999999797, 3.7099999999998143, 3.209999999999825, 1.379999999999864, 1.8599999999998538, 1.9199999999998525, -0.9700000000000859, -1.1300000000000825, 1.1099999999998698, 2.459999999999841, -2.0800000000000622, 0.26999999999988766, 1.989999999999851, -0.2100000000001021, 4.769999999999792, 3.729999999999814, -4.590000000000009, 0.28999999999988724, -2.2900000000000578, -2.4400000000000546, 4.229999999999803, 4.319999999999801, 4.559999999999796, 2.7699999999998344, -2.3100000000000573, -0.9600000000000861, -3.920000000000023, -0.41000000000009784, -1.810000000000068, -3.9100000000000232, 4.2199999999998035, 2.6199999999998376, -2.1700000000000603, 2.749999999999835, -1.4900000000000748, -4.840000000000003, 4.84999999999979, 0.8699999999998749, -0.5200000000000955, -3.550000000000031, -3.8500000000000245, 2.369999999999843, -4.290000000000015, -0.7900000000000897, -2.880000000000045, -0.2100000000001021, 0.8899999999998744, -4.820000000000004, 4.719999999999793, -3.180000000000039, 3.2899999999998233, -4.770000000000005, 1.4599999999998623, 0.34999999999988596, 4.619999999999795, 2.2999999999998444, -3.020000000000042, 2.9599999999998303, 2.649999999999837, -4.710000000000006, 4.199999999999804, -2.9100000000000446, 4.599999999999795, -1.0000000000000853, -1.7500000000000693, 2.3499999999998433, 0.9699999999998727, -0.48000000000009635, -4.580000000000009, 3.059999999999828, -0.18000000000010274, 1.289999999999866, 2.0099999999998506, -2.0600000000000627, -4.950000000000001, -2.430000000000055, -4.270000000000016, 1.009999999999872, 1.0899999999998702, -5, -4.000000000000021, -3.7200000000000273, -0.07000000000010509, -2.6300000000000505, -3.080000000000041, -0.6100000000000936, 4.269999999999802, 2.7699999999998344, 1.9599999999998516, -3.550000000000031, 3.4699999999998195, -1.5700000000000731, -0.7400000000000908, 0.19999999999988916, 4.879999999999789, 1.4999999999998614, 2.3999999999998423, 1.8699999999998536, 3.1499999999998263, -2.65000000000005, 3.3799999999998214, 3.199999999999825, -1.7300000000000697, 0.6399999999998798, -0.730000000000091, 0.36999999999988553, 2.089999999999849, -2.460000000000054, 1.7199999999998568, -3.780000000000026, -3.360000000000035, 2.4699999999998408, -2.5800000000000516, 2.2099999999998463, -2.4400000000000546, 1.56999999999986, -2.600000000000051, 0.7899999999998766, 4.599999999999795, -0.8900000000000876, -0.16000000000010317, 2.689999999999836, -1.72000000000007, 1.7899999999998553, 2.3499999999998433, 4.679999999999794, 4.279999999999802, 0.21999999999988873, -2.5300000000000527, 1.2999999999998657, -0.17000000000010296, -0.0600000000001053, -2.280000000000058, 4.979999999999787, -4.08000000000002, 1.8299999999998544, -4.150000000000018, 3.389999999999821, -4.280000000000015, -4.310000000000015, 0.489999999999883, 0.34999999999988596, -1.3200000000000784, 4.949999999999788, 4.049999999999807, -4.240000000000016, 0.019999999999892992, -4.100000000000019, -4.570000000000009, -2.5400000000000524, 0.8899999999998744, 1.1599999999998687, -4.570000000000009, -2.4800000000000537, -4.700000000000006, -2.9200000000000443, 4.809999999999791, -2.7700000000000475, 3.639999999999816, -1.1600000000000819, 2.9599999999998303, -0.07000000000010509, 1.1099999999998698, 4.85999999999979, 0.3099999999998868, 3.389999999999821, -2.370000000000056, -1.0658141036401503e-13, 3.8799999999998107, -3.160000000000039, -3.400000000000034, -2.610000000000051, -2.0300000000000633, -4.260000000000016, 4.129999999999805, 2.089999999999849, -3.310000000000036, -2.990000000000043, 1.7699999999998557, 4.709999999999793, 1.899999999999853, -3.460000000000033, 3.729999999999814, 1.009999999999872, 1.9699999999998514, -4.400000000000013, -1.9700000000000646, 0.28999999999988724, -2.750000000000048, -0.2900000000001004, 3.219999999999825, -4.380000000000013, 2.4399999999998414, -2.2100000000000595, -0.5200000000000955, -3.210000000000038, 3.1799999999998256, 3.7999999999998124, 3.869999999999811, 2.1199999999998482, 4.339999999999801, 4.909999999999789, -4.590000000000009, 0.4599999999998836, 4.519999999999797, -3.9100000000000232, -3.7100000000000275, -4.08000000000002, -0.6300000000000932, 1.379999999999864, -4.99, 3.219999999999825, -4.760000000000005, 4.719999999999793, -0.870000000000088, -1.340000000000078, -4.670000000000007, 3.589999999999817, -1.390000000000077, 1.009999999999872, 3.399999999999821, -0.7200000000000912, 1.1299999999998693, -0.0600000000001053, 1.6799999999998576, 0.26999999999988766, 0.6499999999998796, -1.0200000000000848, -4.420000000000012, -0.6500000000000927, 2.369999999999843, 0.9699999999998727, -4.590000000000009, 4.529999999999797, -0.19000000000010253, -0.17000000000010296, 2.8099999999998335, -2.100000000000062, -3.7500000000000266, -3.7500000000000266, -0.5200000000000955, -4.160000000000018, 1.5099999999998612, 3.2699999999998237, -3.640000000000029, 4.769999999999792, 2.02999999999985, -2.5300000000000527, -3.8100000000000254, -3.1000000000000405, -4.660000000000007, -0.04000000000010573, 3.7599999999998133, 2.309999999999844, -3.8000000000000256, 3.119999999999827, 3.6899999999998148, -4.860000000000003, 4.6399999999997945, 2.9599999999998303, -2.9100000000000446, 3.0399999999998286, 0.22999999999988852, -1.3300000000000782, 0.3299999999998864, 0.18999999999988937, 1.0399999999998712, -3.360000000000035, 4.279999999999802, -0.5100000000000957, -3.8400000000000247, -1.150000000000082, -3.6600000000000286, 4.059999999999807, 3.2299999999998246, 4.979999999999787, 2.5399999999998393, -4.010000000000021, -3.1900000000000386, 0.3099999999998868, -3.5700000000000305, 0.43999999999988404, -3.1900000000000386, -1.72000000000007, -1.3800000000000772, 0.21999999999988873, -2.6700000000000497, 2.739999999999835, 2.599999999999838, 0.7899999999998766, -3.170000000000039, -1.4500000000000757, -3.0000000000000426, 0.3099999999998868, -2.000000000000064, 4.669999999999794, 3.4599999999998197, -3.640000000000029, 0.009999999999893205, 3.91999999999981, -4.010000000000021, -1.4200000000000763, 3.6899999999998148, -4.260000000000016, 1.9299999999998523, -2.140000000000061, 4.83999999999979, 4.709999999999793, -4.760000000000005, -4.99, 3.529999999999818, -0.17000000000010296, -3.5200000000000315, 1.7499999999998561, -4.99, 3.499999999999819, -4.950000000000001, -0.730000000000091, 4.019999999999808, 0.8599999999998751, 4.609999999999795, -1.6800000000000708, -1.8800000000000665, -1.3000000000000789, -3.1400000000000396, 4.109999999999806, 3.299999999999823, -0.1300000000001038, 1.5399999999998606, 0.299999999999887, 3.769999999999813, 3.3799999999998214, -4.08000000000002, 1.989999999999851, 4.1699999999998045, -1.7400000000000695, -2.040000000000063, -3.59000000000003, -2.7600000000000477, -3.4400000000000333, -2.4900000000000535, 0.3199999999998866, -1.4000000000000767, 1.469999999999862, -0.40000000000009805, 1.9999999999998508, 4.339999999999801, -1.8000000000000682, -2.040000000000063, -4.05000000000002, -3.8100000000000254, -1.2400000000000801, -1.1200000000000827, -0.3500000000000991, -2.0200000000000635, 4.86999999999979, 1.3199999999998653, -2.280000000000058, 1.7799999999998555, -2.100000000000062, -2.4800000000000537, -0.8400000000000887, -4.910000000000002, 1.9799999999998512, -2.410000000000055, -3.2300000000000377, -1.8500000000000671, 2.889999999999832, -3.680000000000028, 0.7299999999998779, -0.6600000000000925, -3.2300000000000377, 2.7599999999998346, 3.5099999999998186, -4.270000000000016, -3.770000000000026, 3.199999999999825, -2.2100000000000595, 0.43999999999988404, 3.399999999999821, -3.920000000000023, -1.7300000000000697, 2.449999999999841, -3.7500000000000266, -0.12000000000010402, -1.4600000000000755, 3.5199999999998184, -1.6500000000000714, -3.4300000000000335, 1.469999999999862, 1.799999999999855, -0.3600000000000989, 0.36999999999988553, 1.6399999999998585, 1.3999999999998636, 4.559999999999796, -1.1800000000000814, 4.439999999999799, -1.340000000000078, 1.1799999999998683, -4.810000000000004, -0.010000000000106368, 4.649999999999794, -4.690000000000007, 2.279999999999845, -2.4500000000000544, -3.030000000000042, 1.519999999999861, 1.849999999999854, -4.190000000000017, -3.0500000000000416, -1.3700000000000774, 1.1199999999998695, -2.610000000000051, -3.920000000000023, -3.4300000000000335, 4.149999999999805, 3.8499999999998114, -1.72000000000007, -3.9800000000000217, 1.429999999999863, -2.8600000000000456, 1.989999999999851, -2.4500000000000544, -1.4600000000000755, 2.0099999999998506, 2.5799999999998384, 1.0299999999998715, -2.2900000000000578, 1.6399999999998585, 3.069999999999828, -0.730000000000091, -0.870000000000088, -4.760000000000005, 0.6999999999998785, -3.450000000000033, 1.3099999999998655, 3.44999999999982, 4.749999999999792, -4.55000000000001, 2.3799999999998427, -3.830000000000025, 4.699999999999793, -3.5100000000000318, -2.65000000000005, -2.7700000000000475, -4.210000000000017, -0.3800000000000985, -4.900000000000002, 1.0299999999998715, 4.809999999999791, -2.6300000000000505, -4.270000000000016, -0.6200000000000934, -0.9500000000000863, -1.8900000000000663, -1.4700000000000752, 2.319999999999844, -3.960000000000022, 1.1399999999998691, -2.5500000000000522, -2.050000000000063, -4.680000000000007, -1.5500000000000735, 0.3299999999998864, -0.19000000000010253, 0.5599999999998815, -4.960000000000001, -2.610000000000051, 1.8399999999998542, -3.4300000000000335, 1.7699999999998557, 4.3999999999998, -0.6300000000000932, 0.5199999999998823, -4.53000000000001, 4.479999999999798, -0.8000000000000895, 3.6999999999998145, 4.429999999999799, 0.4799999999998832, 2.97999999999983, -2.7800000000000473, 2.319999999999844, 4.709999999999793, 4.699999999999793, 3.2699999999998237, -3.920000000000023, -0.010000000000106368, 1.2299999999998672, 0.4099999999998847, -4.08000000000002, 0.5899999999998808, 3.6099999999998165, 4.909999999999789, 1.8599999999998538, -0.8900000000000876, -2.410000000000055, 0.9399999999998734, -1.4200000000000763, 2.6599999999998367, -2.330000000000057, 2.3799999999998427, 0.5999999999998806, -2.430000000000055, -3.6600000000000286, -0.20000000000010232, -0.5300000000000953, -0.8400000000000887, -3.790000000000026, 1.7399999999998563, -3.160000000000039, 3.399999999999821, 2.1999999999998465, -0.26000000000010104, 0.05999999999989214, -0.6500000000000927, -2.2600000000000584, 2.2399999999998457, 3.949999999999809, -1.5600000000000733, -1.290000000000079, 1.659999999999858, -1.480000000000075, -2.610000000000051, 3.819999999999812, 3.0399999999998286, 3.589999999999817, 2.559999999999839, -1.3000000000000789, 4.439999999999799, -4.470000000000011, -3.12000000000004, 0.8999999999998742, 3.1899999999998254, -0.450000000000097, -1.6000000000000725, -0.8000000000000895, -0.9500000000000863, -0.8900000000000876, 3.959999999999809, -1.2800000000000793, 0.35999999999988574, 4.339999999999801, 0.36999999999988553, -0.5500000000000949, -2.0100000000000637, 3.029999999999829, -3.920000000000023, -2.2500000000000586, 0.35999999999988574, -1.9900000000000642, -4.400000000000013, -4.820000000000004, -4.400000000000013, 0.8999999999998742, 4.3899999999998, -2.430000000000055, 3.4099999999998207, 3.4199999999998205, -0.0600000000001053, -1.670000000000071, 2.97999999999983, 1.6399999999998585, -1.5200000000000742, -1.7300000000000697, -3.7500000000000266, -0.9000000000000874, 2.8599999999998325, 0.41999999999988447, 2.0599999999998495, 4.729999999999793, 1.429999999999863, -0.5100000000000957, -1.390000000000077, -1.2600000000000797, 3.819999999999812, -4.380000000000013, -2.3600000000000563, 2.3899999999998425, 1.2799999999998661, -2.560000000000052, -0.870000000000088, -1.4700000000000752, 0.9799999999998725, -1.2400000000000801, -2.5800000000000516, -3.870000000000024, -2.6900000000000492, 1.519999999999861, 2.96999999999983, 4.979999999999787, -2.5400000000000524, 0.43999999999988404, 2.699999999999836, 4.729999999999793, 1.1199999999998695, -2.880000000000045, 2.7699999999998344, 2.0599999999998495, -2.19000000000006, 1.7399999999998563, -3.780000000000026, 3.6099999999998165, -0.7500000000000906, -2.410000000000055, 3.5599999999998175, -2.6800000000000495, 4.879999999999789, 2.3799999999998427, -4.690000000000007, 4.579999999999796, -0.07000000000010509, 4.3699999999998, -0.4300000000000974, 4.629999999999795, -2.8700000000000454, 1.289999999999866, 0.02999999999989278, 1.2699999999998663, -1.4500000000000757, -3.4200000000000337, -0.8300000000000889, -1.9200000000000657, -4.1800000000000175, -0.8200000000000891, 4.409999999999799, 4.519999999999797, -2.570000000000052, -2.050000000000063, 0.4799999999998832, 2.359999999999843, -2.6700000000000497, -4.870000000000003, -0.5700000000000944, -2.930000000000044, 0.6799999999998789, -1.72000000000007, -0.07000000000010509, 1.0299999999998715, -2.6300000000000505, 3.249999999999824, -3.030000000000042, 1.6499999999998582, 4.909999999999789, 3.969999999999809, -3.5700000000000305, -1.2800000000000793, 2.3999999999998423, 1.1299999999998693, 0.6599999999998793, 4.84999999999979, 4.459999999999798, 0.579999999999881, 1.9499999999998519, 0.6499999999998796, 4.159999999999805, -1.3000000000000789, -3.500000000000032, -2.230000000000059, -3.1400000000000396, 1.4599999999998623, -4.400000000000013, -3.260000000000037, 0.18999999999988937, -3.970000000000022, 3.3199999999998226, 0.9199999999998738, 3.779999999999813, -3.3900000000000343, 0.10999999999989107, 4.579999999999796, 1.9299999999998523, -3.3900000000000343, -4.690000000000007, 0.14999999999989022, -1.1400000000000823, -1.5200000000000742, 0.6699999999998791, -1.25000000000008, -4.770000000000005, -0.32000000000009976, -2.2100000000000595, -4.230000000000016, 1.899999999999853, -0.20000000000010232, -1.8300000000000676, 0.9199999999998738, -4.120000000000019, -3.450000000000033, 0.3899999999998851, -3.880000000000024, 1.7399999999998563, -0.6500000000000927, -0.78000000000009, 2.50999999999984, -1.060000000000084, 2.1199999999998482, 4.139999999999805, 4.189999999999804, 3.999999999999808, -2.6800000000000495, 0.2599999999998879, 2.589999999999838, -2.320000000000057, 3.529999999999818, 3.639999999999816, 0.299999999999887, -4.280000000000015, -3.730000000000027, -2.130000000000061, -2.6300000000000505, -3.3000000000000362, 2.639999999999837, 0.4699999999998834, -1.2100000000000808, 4.84999999999979, -4.950000000000001, 0.5099999999998825, 0.4699999999998834, -4.6000000000000085, -1.8800000000000665, -4.330000000000014, 3.5699999999998173, 3.629999999999816, 1.3099999999998655, 2.699999999999836, 0.3299999999998864, -2.6800000000000495, 4.569999999999796, 1.2299999999998672, 0.6199999999998802, 2.1199999999998482, 0.019999999999892992, 0.3099999999998868, 3.819999999999812, 0.3099999999998868, -1.2800000000000793, -1.4200000000000763, -1.4900000000000748, -4.54000000000001, -0.4400000000000972, -4.300000000000015, 1.0299999999998715, 3.3299999999998224, 3.3399999999998222, -0.1400000000001036, 4.179999999999804, 1.6399999999998585, 2.319999999999844, 4.889999999999789, -0.6500000000000927, 3.209999999999825, -1.6600000000000712, 2.089999999999849, 4.059999999999807, 2.889999999999832, -1.4600000000000755, -4.110000000000019, 3.029999999999829, 3.0899999999998276, -2.710000000000049, -2.280000000000058, 3.5099999999998186, 3.729999999999814, 0.6099999999998804, -4.610000000000008, -1.8500000000000671, 4.009999999999808, 0.20999999999988894, 1.3499999999998646, -1.6000000000000725, -0.6700000000000923, -0.6700000000000923, 1.6299999999998587, -1.580000000000073, 0.4599999999998836, 1.5099999999998612, 4.979999999999787, -1.810000000000068, 0.8799999999998747, -4.210000000000017, 2.8999999999998316, -1.150000000000082, -1.8700000000000667, 1.4799999999998619, 3.209999999999825, -0.17000000000010296, 0.7099999999998783, -3.1500000000000394, -4.680000000000007, -3.2500000000000373, 1.8099999999998548, -1.7500000000000693, -1.4600000000000755, -1.3600000000000776, -3.0900000000000407, 4.7799999999997915, 4.509999999999797, -0.7200000000000912, 2.50999999999984, 0.19999999999988916, -0.730000000000091, 4.569999999999796, 0.13999999999989043, -2.2600000000000584, -0.9800000000000857, -3.740000000000027, -0.6500000000000927, -4.840000000000003, -0.8300000000000889, 0.19999999999988916, -0.15000000000010338, -2.510000000000053, 2.2099999999998463, -2.9500000000000437, 1.7299999999998565, -2.5800000000000516, 2.419999999999842, 0.6799999999998789, 1.3499999999998646, -4.960000000000001, 1.3199999999998653, 3.2899999999998233, 4.0799999999998064, -0.9800000000000857, -1.9200000000000657, 4.769999999999792, -4.250000000000016, 2.739999999999835, 2.7199999999998354, -1.9200000000000657, -2.65000000000005, 1.659999999999858, 4.86999999999979, 3.6599999999998154, -1.2700000000000795, -1.480000000000075, 3.6599999999998154, 2.9899999999998297, 4.509999999999797, 0.06999999999989193, 2.5199999999998397, -4.53000000000001, 2.319999999999844, -2.850000000000046, -0.3100000000001, 3.2299999999998246, -2.5300000000000527, -0.17000000000010296, 4.809999999999791, 2.6299999999998374, -0.3700000000000987, 2.419999999999842, 2.2099999999998463, -1.1200000000000827, 3.529999999999818, 1.8599999999998538, 2.7099999999998357, 4.879999999999789, -1.2800000000000793, 3.8899999999998105, 3.5999999999998167, -4.6500000000000075, 0.8899999999998744, -1.6600000000000712, 1.5799999999998597, 0.35999999999988574, 2.7099999999998357, -2.9700000000000433, -1.7800000000000686, 1.56999999999986, -2.6900000000000492, -3.3400000000000354, 4.119999999999806, -1.8200000000000678, 4.099999999999806, -0.8900000000000876, 4.289999999999802, 4.419999999999799, 3.3399999999998222, 2.4799999999998406, 2.089999999999849, -1.8900000000000663, 2.549999999999839, 1.519999999999861, 4.149999999999805, -4.6500000000000075, 4.879999999999789, 2.079999999999849, 2.8699999999998322, -0.6200000000000934, -0.41000000000009784, 1.799999999999855, 4.559999999999796, -3.0500000000000416, -0.6200000000000934, 0.21999999999988873, 2.599999999999838, 3.999999999999808, 1.379999999999864, -2.1700000000000603, 2.789999999999834, -1.110000000000083, -2.18000000000006, -0.5100000000000957, -0.6100000000000936, 4.459999999999798, 4.609999999999795, -0.6000000000000938, 4.479999999999798, -2.0300000000000633, 3.1799999999998256, 1.9999999999998508, -0.33000000000009955, -4.1300000000000185, -0.5800000000000942, 2.5299999999998395, -0.020000000000106155, -0.4600000000000968, 1.4899999999998617, -2.0100000000000637, 1.09999999999987, -4.360000000000014, 2.4699999999998408, 4.729999999999793, -2.230000000000059, 4.129999999999805, 0.489999999999883, -2.880000000000045, 4.099999999999806, -0.4300000000000974, -2.9500000000000437, -2.040000000000063, -0.8800000000000878, -2.9200000000000443, 4.949999999999788, -3.0100000000000424, -0.2700000000001008, -0.20000000000010232, -4.110000000000019, 0.8799999999998747, 1.7799999999998555, -2.4500000000000544, 4.479999999999798, -1.2100000000000808, -4.340000000000014, 0.4499999999998838, 2.5699999999998386, -2.7700000000000475, 1.0699999999998706, 0.15999999999989, 3.4599999999998197, 0.9399999999998734, -3.2500000000000373, -2.000000000000064, 2.449999999999841, -3.2400000000000375, -0.8200000000000891, 0.9399999999998734, -2.0700000000000625, 0.04999999999989235, 2.179999999999847, 2.929999999999831, -0.050000000000105516, -2.1200000000000614, 3.3199999999998226, -1.2100000000000808, -4.700000000000006, -2.0200000000000635, -3.310000000000036, -2.18000000000006, -4.330000000000014, -4.340000000000014, -3.080000000000041, -4.730000000000006, 0.5199999999998823, -3.880000000000024, 4.759999999999792, 2.7199999999998354, -4.99, 1.7799999999998555, 4.9699999999997875, -0.9600000000000861, -0.8000000000000895, 4.949999999999788, -3.8900000000000237, -0.8900000000000876, 2.689999999999836, 0.62999999999988, -1.3700000000000774, 4.629999999999795, 4.84999999999979, 2.309999999999844, 4.0299999999998075, -3.0600000000000414, 4.429999999999799, 2.739999999999835, 1.989999999999851, -1.2200000000000806, -0.6800000000000921, -4.370000000000013, 1.3099999999998655, 0.43999999999988404, 2.879999999999832, 4.3599999999998005, -0.17000000000010296, 2.179999999999847, 2.929999999999831, -3.5300000000000313, 1.6899999999998574, -1.6300000000000718, 0.7099999999998783, -1.4000000000000767, -2.0100000000000637, 4.069999999999807, 3.859999999999811, -1.2700000000000795, -2.840000000000046, -0.7500000000000906, -2.700000000000049, 1.469999999999862, 4.069999999999807, 3.159999999999826, -1.6100000000000723, -3.770000000000026, -2.6400000000000503, 3.8299999999998118, 1.9099999999998527, -4.810000000000004, -3.9400000000000226, -0.9800000000000857, -2.9600000000000435, -4.100000000000019, 4.269999999999802, 3.7499999999998135, -4.840000000000003, -3.3400000000000354, 4.449999999999799, 0.14999999999989022, 1.7799999999998555, -4.150000000000018, -4.220000000000017, -3.080000000000041, -0.8000000000000895, 3.819999999999812, 4.259999999999803, -0.12000000000010402, 1.1699999999998685, -1.9800000000000644, -3.550000000000031, 2.9999999999998295, 4.559999999999796, 1.5099999999998612, -4.210000000000017, 0.3199999999998866, -0.5000000000000959, -2.700000000000049, 4.259999999999803, -0.9700000000000859, -3.820000000000025, 0.06999999999989193, 0.4799999999998832, 0.8899999999998744, 2.549999999999839, -3.7200000000000273, 3.4299999999998203, -2.710000000000049, -4.140000000000018, -3.2400000000000375, 1.3399999999998649, -2.9100000000000446, -1.2700000000000795, -4.5100000000000104, -1.0300000000000846, 3.2299999999998246, 2.8199999999998333, -1.900000000000066, -0.9300000000000868, 1.8899999999998531, 3.299999999999823, -4.160000000000018, 0.7899999999998766, -0.9800000000000857, 4.729999999999793, 4.439999999999799, 0.02999999999989278, -1.0700000000000838, -3.550000000000031, 1.3599999999998644, -2.570000000000052, -3.1100000000000403, -4.220000000000017, 1.049999999999871, -3.0500000000000416, 3.91999999999981, 0.6099999999998804, 4.739999999999792, 0.04999999999989235, -2.940000000000044, 1.2799999999998661, 4.479999999999798, -1.150000000000082, -3.180000000000039, 2.8099999999998335, 4.039999999999807, -2.66000000000005, -1.900000000000066, -2.890000000000045, 3.6599999999998154, 3.219999999999825, -2.9000000000000448, -2.0100000000000637, 2.369999999999843, 1.3599999999998644, 0.12999999999989065, -0.730000000000091, 0.43999999999988404, -3.320000000000036, 2.279999999999845, -1.0400000000000844, -0.07000000000010509, -3.5700000000000305, -4.680000000000007, 0.02999999999989278, -3.360000000000035, 0.7899999999998766, -1.25000000000008, 0.5899999999998808, 3.399999999999821, 4.579999999999796, 3.0099999999998293, 0.5399999999998819, 0.34999999999988596, 1.289999999999866, -1.6900000000000706, 1.709999999999857, -2.2100000000000595, 1.2099999999998676, -0.5800000000000942, 2.559999999999839, 4.509999999999797, -0.8500000000000885, -2.2500000000000586, 2.2099999999998463, -1.150000000000082, -2.0800000000000622, -4.960000000000001, -1.5500000000000735, -3.7000000000000277, -2.5000000000000533, 3.259999999999824, -1.3300000000000782, 1.9599999999998516, -0.5500000000000949, -1.72000000000007, -4.710000000000006, -0.9800000000000857, 1.56999999999986, -2.840000000000046, 0.3299999999998864, -2.930000000000044, 2.0199999999998504, 4.059999999999807, 1.0399999999998712, -4.580000000000009, -4.200000000000017, 3.959999999999809, -0.730000000000091, -2.230000000000059, -1.3500000000000778, 1.9999999999998508, 1.7699999999998557, 0.19999999999988916, -1.2100000000000808, 1.9999999999998508, -1.3500000000000778, -2.1700000000000603, -0.5800000000000942, -2.8700000000000454, -1.150000000000082, -5, 1.3999999999998636, -1.4200000000000763, -2.66000000000005, 1.6499999999998582, 1.09999999999987, -0.5400000000000951, -2.4900000000000535, 4.0299999999998075, 1.8899999999998531, -0.4300000000000974, 4.899999999999789, 1.289999999999866, -0.34000000000009933, 4.489999999999798, -2.5300000000000527, 2.3299999999998438, 4.1699999999998045, -2.9200000000000443, 4.229999999999803, -1.4100000000000765, 1.3899999999998638, 3.0799999999998278, 4.049999999999807, 2.3499999999998433, 3.3399999999998222, -1.0500000000000842, 4.929999999999788, 2.2499999999998455, 0.0899999999998915, -1.290000000000079, 4.619999999999795, 3.059999999999828, -3.030000000000042, -4.220000000000017, -1.2800000000000793, -1.0900000000000833, 2.459999999999841, -2.560000000000052, -2.9000000000000448, 2.4699999999998408, -4.480000000000011, 2.6699999999998365, 3.119999999999827, 3.0999999999998273, -0.7900000000000897, 4.459999999999798, -0.050000000000105516, -3.730000000000027, -2.4400000000000546, 2.1999999999998465, -2.240000000000059, -4.9300000000000015, 2.409999999999842, -0.7900000000000897, -1.2700000000000795, 0.28999999999988724, -4.410000000000013, -4.7900000000000045, -0.23000000000010168, 4.8299999999997905, 0.62999999999988, -0.7500000000000906, -3.8600000000000243, -2.140000000000061, -1.9100000000000659, 4.469999999999798, 0.36999999999988553, -2.850000000000046, 1.429999999999863, 4.419999999999799, -2.410000000000055, 0.34999999999988596, -2.840000000000046, 2.50999999999984, -3.350000000000035, -4.120000000000019, -2.140000000000061, 3.0499999999998284, -1.0900000000000833, -1.7000000000000703, -4.680000000000007, 1.3499999999998646, 1.1199999999998695, -4.860000000000003, 2.589999999999838, -2.1600000000000605, -4.490000000000011, 4.099999999999806, -0.6800000000000921, 4.6399999999997945, 2.749999999999835, 2.549999999999839, -2.330000000000057, 4.259999999999803, -4.54000000000001, -1.6400000000000716, 1.1099999999998698, -0.23000000000010168, -4.9300000000000015, -0.020000000000106155, -0.26000000000010104, 2.459999999999841, -1.8200000000000678, -2.750000000000048, 0.4599999999998836, 0.41999999999988447, -3.2400000000000375, -2.0800000000000622, -4.730000000000006, 1.0299999999998715, -2.420000000000055, -2.5900000000000514, 2.7999999999998337, 1.4099999999998634, -2.370000000000056, -1.2100000000000808, -2.0800000000000622, -1.0500000000000842, 2.3899999999998425, 4.489999999999798, 2.369999999999843, 2.129999999999848, -0.6200000000000934, -4.950000000000001, 4.549999999999796, 0.5899999999998808, 3.3699999999998216, -1.8300000000000676, -1.0000000000000853, -2.790000000000047, 2.7599999999998346, 2.7199999999998354, 1.6399999999998585, 4.799999999999791, 0.09999999999989129, -0.18000000000010274, 3.3799999999998214, 4.349999999999801, 3.8399999999998116, -2.050000000000063, 2.5199999999998397, -1.4600000000000755, -3.8100000000000254, 2.6199999999998376, -2.270000000000058, -2.19000000000006, -4.380000000000013, -3.8500000000000245, 4.2199999999998035, 2.739999999999835, -1.9900000000000642, 3.949999999999809, -1.7100000000000701, -0.9600000000000861, -4.910000000000002, 2.2499999999998455, 0.3999999999998849, -1.0400000000000844, 0.9699999999998727, -1.1700000000000816, 1.2099999999998676, -4.340000000000014, -3.9900000000000215, -4.800000000000004, 1.6299999999998587, -2.4800000000000537, 2.9499999999998305, 2.5699999999998386, 0.05999999999989214, 1.6299999999998587, -3.6200000000000294, -0.5100000000000957, -0.08000000000010488, 0.9999999999998721, -4.410000000000013, -3.170000000000039, -0.6700000000000923, 3.3699999999998216, 2.3999999999998423, 4.619999999999795, 0.1699999999998898, -3.8000000000000256, 3.359999999999822, 0.9699999999998727, 0.2599999999998879, 0.3199999999998866, 2.279999999999845, 0.8999999999998742, -4.1800000000000175, 0.9399999999998734, 2.4399999999998414, -3.8100000000000254, -0.020000000000106155, 0.5399999999998819, -0.7900000000000897, -2.0300000000000633, -4.340000000000014, -0.20000000000010232, 1.3999999999998636, 3.0399999999998286, -1.5200000000000742, -3.8600000000000243, 0.6099999999998804, 2.169999999999847, -2.140000000000061, -3.260000000000037, 3.6899999999998148, -2.9000000000000448, 1.5999999999998593, -0.2100000000001021, 1.8599999999998538, 0.17999999999988958, -1.9600000000000648, 4.9199999999997885, 0.42999999999988425, -4.120000000000019, 2.3299999999998438, 2.229999999999846, 3.6899999999998148, -4.890000000000002, -0.5500000000000949, 4.259999999999803, -3.180000000000039, -3.460000000000033, 3.8799999999998107, -4.240000000000016, 0.9999999999998721, 1.009999999999872, -4.750000000000005, -2.430000000000055, 3.109999999999827, -0.20000000000010232, -4.030000000000021, -2.18000000000006, 0.6899999999998787, 2.609999999999838, -1.5500000000000735, 0.8099999999998762, -4.830000000000004, -3.9000000000000234, -3.040000000000042, -0.8200000000000891, -0.3600000000000989, -1.670000000000071, 3.059999999999828, -4.500000000000011, 2.4299999999998416, 4.909999999999789, 3.769999999999813, -1.4900000000000748, -2.6800000000000495, -4.150000000000018, -2.7800000000000473, -1.1600000000000819, -0.15000000000010338, -0.2100000000001021, 1.2699999999998663, 3.109999999999827, -2.2100000000000595, -1.9200000000000657, -1.8300000000000676, -1.430000000000076, -4.840000000000003, 4.249999999999803, 2.139999999999848, -4.470000000000011, 4.599999999999795, -4.170000000000018, -4.090000000000019, -1.2400000000000801, 2.549999999999839, 0.909999999999874, -4.900000000000002, 1.329999999999865, 3.4199999999998205, -1.3500000000000778, 0.43999999999988404, -0.1400000000001036, 3.43999999999982, -0.450000000000097, -1.1600000000000819, -0.5400000000000951, 3.4199999999998205, 4.159999999999805, -3.0900000000000407, -2.3100000000000573, 0.09999999999989129, 4.119999999999806, 2.089999999999849, -1.0658141036401503e-13, 2.919999999999831, 2.1099999999998484, -3.550000000000031, -2.18000000000006, 4.739999999999792, 3.6999999999998145, 2.229999999999846, -2.600000000000051, -1.760000000000069, 0.7899999999998766, 3.4299999999998203, 4.459999999999798, 3.90999999999981, -3.550000000000031, 2.689999999999836, 2.179999999999847, -4.440000000000012, -0.40000000000009805, -0.7000000000000917, -1.810000000000068, 3.2699999999998237, 1.0899999999998702, -1.150000000000082, -2.330000000000057, 0.5999999999998806, -3.8100000000000254, -1.4400000000000759, 4.819999999999791, 1.329999999999865, 1.009999999999872, 1.2099999999998676, 0.5399999999998819, 2.9999999999998295, -1.200000000000081, 1.3899999999998638, 3.7399999999998137, 0.9399999999998734, -4.05000000000002, 0.6999999999998785, -3.310000000000036, -1.6100000000000723, 3.7399999999998137, 2.839999999999833, 3.959999999999809, -3.9100000000000232, -0.010000000000106368, -0.2200000000001019, -4.810000000000004, -2.000000000000064, -0.33000000000009955, -4.780000000000005, -1.340000000000078, -2.280000000000058, -0.6200000000000934, 2.7999999999998337, 4.099999999999806, 4.629999999999795, -4.970000000000001, 1.6299999999998587, -2.0700000000000625, -2.430000000000055, 4.739999999999792, -4.98, -0.7000000000000917, -4.350000000000014, -4.6000000000000085, -3.0000000000000426, -1.430000000000076, 1.609999999999859, 4.699999999999793, -0.42000000000009763, -4.020000000000021, -3.5100000000000318, -4.900000000000002, 2.639999999999837, -2.1500000000000608, 2.889999999999832, 3.8299999999998118, -1.3200000000000784, -4.240000000000016, 1.09999999999987, 4.099999999999806, 2.2999999999998444, -3.080000000000041, 0.7899999999998766, 4.299999999999802, -2.0800000000000622, -1.7100000000000701, 0.11999999999989086, -3.220000000000038, -4.98, -1.0300000000000846, -3.320000000000036, 1.09999999999987, -1.860000000000067, 3.2699999999998237, 0.489999999999883, 3.1799999999998256, 0.7399999999998776, 3.489999999999819, 1.8799999999998533, -0.20000000000010232, -3.9900000000000215, -2.8100000000000467, -3.450000000000033, 4.819999999999791, -0.6100000000000936, 2.8099999999998335, -4.1300000000000185, -3.350000000000035, -4.970000000000001, 2.49999999999984, -0.3100000000001, 0.5299999999998821, -3.740000000000027, -3.210000000000038, 1.989999999999851, 0.11999999999989086 ], "xaxis": "x", "y": [ 4.139999999999805, -2.280000000000058, -1.670000000000071, 1.659999999999858, -1.670000000000071, 1.659999999999858, 4.989999999999787, 4.989999999999787, -5, -5, 2.599999999999838, 1.4399999999998627, 1.4999999999998614, -2.9700000000000433, 3.809999999999812, -0.5100000000000957, 2.0999999999998487, -2.3900000000000556, 0.43999999999988404, -2.3400000000000567, 1.9599999999998516, -2.430000000000055, 2.319999999999844, 2.2499999999998455, 4.329999999999801, -0.12000000000010402, -2.7300000000000484, 2.929999999999831, -0.590000000000094, -2.5000000000000533, 0.6099999999998804, 4.0299999999998075, 2.9499999999998305, -1.7400000000000695, 3.8299999999998118, 3.209999999999825, -3.7600000000000264, -2.8600000000000456, -0.49000000000009614, -2.570000000000052, 4.3999999999998, 4.719999999999793, -0.450000000000097, 4.509999999999797, 1.1299999999998693, 0.41999999999988447, -4.730000000000006, 2.3499999999998433, 4.559999999999796, -3.770000000000026, 0.8099999999998762, -1.1300000000000825, -4.380000000000013, 4.059999999999807, -1.2600000000000797, 2.269999999999845, 4.139999999999805, -4.200000000000017, 2.179999999999847, 3.159999999999826, 4.879999999999789, -2.1700000000000603, -4.350000000000014, -3.0600000000000414, -1.2800000000000793, -0.6800000000000921, 2.4699999999998408, 0.9499999999998732, 4.449999999999799, 0.4699999999998834, 1.3099999999998655, 3.8799999999998107, 1.2299999999998672, -3.360000000000035, -4.05000000000002, 0.7399999999998776, -4.260000000000016, -3.770000000000026, -3.2900000000000365, -4.150000000000018, 0.6199999999998802, 4.289999999999802, -4.9300000000000015, -0.2700000000001008, -1.150000000000082, 1.8299999999998544, 2.449999999999841, 1.6999999999998572, -3.9800000000000217, -0.3700000000000987, -4.370000000000013, 0.8799999999998747, -2.2200000000000593, 3.5099999999998186, 3.029999999999829, -2.040000000000063, 4.8299999999997905, -4.170000000000018, -3.550000000000031, 3.4799999999998192, -2.2600000000000584, 1.3699999999998642, 0.5199999999998823, 4.319999999999801, -1.6000000000000725, 0.7799999999998768, -3.170000000000039, 1.0199999999998717, 2.6599999999998367, 1.56999999999986, -2.980000000000043, 4.1699999999998045, -1.1800000000000814, -1.620000000000072, -2.710000000000049, 0.9899999999998723, 1.4999999999998614, 2.9099999999998314, 2.03999999999985, 4.409999999999799, 1.3399999999998649, 3.1899999999998254, 0.42999999999988425, -2.4800000000000537, 3.90999999999981, 2.309999999999844, 2.0099999999998506, -1.810000000000068, -1.9600000000000648, -3.3700000000000347, 3.869999999999811, -2.3500000000000565, 0.10999999999989107, 0.26999999999988766, 1.2599999999998666, 4.879999999999789, -3.020000000000042, -2.420000000000055, -4.170000000000018, 0.8199999999998759, 3.0799999999998278, -0.1300000000001038, -0.9000000000000874, 0.34999999999988596, -4.910000000000002, -4.300000000000015, -3.880000000000024, 3.859999999999811, 2.3399999999998435, -1.9300000000000654, -3.630000000000029, 0.8599999999998751, -2.0700000000000625, -1.9700000000000646, 2.179999999999847, 2.6699999999998365, -3.3800000000000345, -1.8500000000000671, -2.320000000000057, -1.0400000000000844, -3.730000000000027, 4.609999999999795, 2.459999999999841, -2.7200000000000486, 0.0899999999998915, 1.7799999999998555, 0.7499999999998774, -1.860000000000067, 4.889999999999789, 0.8599999999998751, -3.5100000000000318, -3.2500000000000373, -1.810000000000068, -1.8200000000000678, 2.9399999999998307, 2.3499999999998433, -1.0800000000000836, 0.05999999999989214, -0.3000000000001002, 1.0399999999998712, -3.13000000000004, 1.329999999999865, 2.0699999999998493, -0.870000000000088, 4.149999999999805, -2.8700000000000454, -1.860000000000067, 1.659999999999858, -4.6000000000000085, 0.18999999999988937, -3.070000000000041, -2.8700000000000454, 3.529999999999818, 0.15999999999989, 0.12999999999989065, -2.5800000000000516, 4.3999999999998, -3.360000000000035, -4.570000000000009, 2.8199999999998333, 2.7199999999998354, -1.4500000000000757, -4.820000000000004, -2.9000000000000448, 3.069999999999828, 1.7899999999998553, 1.09999999999987, 0.7499999999998774, 2.8699999999998322, 2.0599999999998495, -0.11000000000010424, 1.5599999999998602, 1.9999999999998508, -4.360000000000014, 3.8299999999998118, 0.04999999999989235, -1.5700000000000731, 3.8499999999998114, 1.6499999999998582, -2.3400000000000567, 4.539999999999797, 0.5299999999998821, 4.979999999999787, -0.6200000000000934, -1.8200000000000678, 1.3199999999998653, 4.049999999999807, -3.13000000000004, 1.3499999999998646, 4.019999999999808, 2.9599999999998303, 1.4499999999998625, 3.1399999999998265, -1.0800000000000836, 0.7499999999998774, -4.360000000000014, 0.21999999999988873, -2.9000000000000448, -2.460000000000054, 0.27999999999988745, 2.7699999999998344, -3.920000000000023, -4.700000000000006, 0.579999999999881, -2.600000000000051, -2.420000000000055, -1.1000000000000831, 1.6499999999998582, 4.409999999999799, -2.570000000000052, -3.020000000000042, 0.9299999999998736, -1.4200000000000763, 0.04999999999989235, 3.119999999999827, 1.5799999999998597, -3.550000000000031, 1.1999999999998678, -2.6200000000000507, -3.020000000000042, -1.7500000000000693, -0.3100000000001, -1.4000000000000767, 2.319999999999844, -4.6000000000000085, -2.3000000000000576, -4.5100000000000104, -4.9300000000000015, 1.09999999999987, -3.5300000000000313, -2.140000000000061, -3.460000000000033, -3.4700000000000326, 1.659999999999858, -0.8300000000000889, -1.620000000000072, 2.549999999999839, 1.9999999999998508, 4.339999999999801, -3.490000000000032, -1.0200000000000848, 2.9499999999998305, 4.089999999999806, 4.3899999999998, 1.3999999999998636, -2.7600000000000477, -2.570000000000052, -1.2400000000000801, -1.340000000000078, -1.9800000000000644, -3.60000000000003, 3.579999999999817, 3.4199999999998205, -2.090000000000062, -1.430000000000076, -4.720000000000006, -3.60000000000003, 4.0299999999998075, -1.5000000000000746, -0.6300000000000932, -2.980000000000043, -4.7400000000000055, 2.129999999999848, 4.719999999999793, 1.009999999999872, 2.5799999999998384, 4.249999999999803, -1.7000000000000703, -1.2100000000000808, -3.2400000000000375, 4.6899999999997934, 2.369999999999843, -4.620000000000008, 1.469999999999862, -4.610000000000008, 1.2799999999998661, 0.3199999999998866, -3.0000000000000426, -1.670000000000071, 1.8599999999998538, -2.750000000000048, -4.830000000000004, 1.1599999999998687, -4.370000000000013, 1.0199999999998717, 0.3099999999998868, 4.4999999999997975, 4.259999999999803, 1.6899999999998574, -4.1800000000000175, 0.62999999999988, 3.259999999999824, 0.3999999999998849, 1.7199999999998568, 2.4799999999998406, -3.7200000000000273, -3.680000000000028, 4.149999999999805, -3.500000000000032, -0.2100000000001021, 2.9899999999998297, -0.6100000000000936, 1.7299999999998565, 2.409999999999842, 1.6399999999998585, -0.4400000000000972, 2.6699999999998365, 2.1099999999998484, 2.5699999999998386, -4.07000000000002, 0.19999999999988916, -3.5600000000000307, 1.9099999999998527, -3.3300000000000356, 0.6999999999998785, 4.529999999999797, -4.350000000000014, 2.599999999999838, 3.1299999999998267, 0.9199999999998738, 0.3299999999998864, 4.489999999999798, 1.7499999999998561, 4.909999999999789, -4.750000000000005, 2.559999999999839, -3.640000000000029, 1.5899999999998595, 0.3199999999998866, 4.479999999999798, -1.430000000000076, 2.269999999999845, -1.390000000000077, 1.9699999999998514, -4.290000000000015, 3.9299999999998096, 2.9099999999998314, -2.740000000000048, -3.260000000000037, -4.890000000000002, -3.5100000000000318, 4.449999999999799, 3.029999999999829, 0.4799999999998832, 0.6399999999998798, -0.04000000000010573, 0.28999999999988724, -2.790000000000047, 3.0999999999998273, 0.5599999999998815, 3.8499999999998114, 2.169999999999847, -3.770000000000026, -0.9600000000000861, -1.5000000000000746, 3.029999999999829, 4.1699999999998045, 1.469999999999862, -1.7700000000000689, -3.780000000000026, 2.309999999999844, 2.49999999999984, -2.19000000000006, -4.010000000000021, -4.160000000000018, 2.419999999999842, 1.709999999999857, -2.5500000000000522, 4.989999999999787, 1.7499999999998561, 3.5499999999998177, -0.5100000000000957, -0.25000000000010125, 3.6599999999998154, -4.680000000000007, 1.5599999999998602, -0.4300000000000974, -1.0658141036401503e-13, 3.249999999999824, -2.4900000000000535, 2.6299999999998374, 0.34999999999988596, -3.7000000000000277, 1.939999999999852, -0.2200000000001019, -1.3700000000000774, 4.269999999999802, -2.19000000000006, -0.2900000000001004, -3.12000000000004, 3.2699999999998237, -3.170000000000039, 4.2199999999998035, 1.8199999999998546, 2.1999999999998465, 4.679999999999794, -1.670000000000071, -2.19000000000006, -3.7600000000000264, -3.60000000000003, 0.17999999999988958, -1.5200000000000742, -0.5100000000000957, 0.039999999999892566, 1.9599999999998516, -3.8600000000000243, 0.12999999999989065, -0.16000000000010317, 2.2899999999998446, -1.150000000000082, -4.1800000000000175, -4.710000000000006, -1.810000000000068, -4.690000000000007, 0.9399999999998734, 0.22999999999988852, 2.319999999999844, 0.2499999999998881, -2.420000000000055, 3.159999999999826, -4.99, -2.9000000000000448, 1.4099999999998634, 2.5399999999998393, 4.0299999999998075, 1.709999999999857, 2.889999999999832, -2.570000000000052, 2.6299999999998374, -4.170000000000018, -2.9200000000000443, -4.53000000000001, 2.2399999999998457, 4.189999999999804, 3.4299999999998203, -3.7500000000000266, 1.9099999999998527, 0.0899999999998915, -3.350000000000035, -1.2600000000000797, -0.20000000000010232, 1.2099999999998676, -0.8100000000000893, 4.909999999999789, -3.6100000000000296, 1.7399999999998563, 4.2199999999998035, -1.7800000000000686, 4.3999999999998, 4.759999999999792, 4.979999999999787, -2.18000000000006, -1.6000000000000725, -4.860000000000003, -4.07000000000002, 4.959999999999788, -2.6900000000000492, 0.4799999999998832, 3.6899999999998148, 4.239999999999803, 0.579999999999881, -3.0500000000000416, 4.2199999999998035, -2.9700000000000433, 0.489999999999883, -1.7100000000000701, -3.1400000000000396, -0.6400000000000929, 0.41999999999988447, -3.3300000000000356, -3.7600000000000264, -2.000000000000064, 3.0899999999998276, -0.6900000000000919, -1.0400000000000844, -3.60000000000003, -3.9000000000000234, -0.8800000000000878, 1.1099999999998698, 0.7799999999998768, 2.649999999999837, 4.3599999999998005, 0.019999999999892992, 2.919999999999831, -0.9700000000000859, 0.6999999999998785, 0.5099999999998825, -0.42000000000009763, 1.1599999999998687, -3.9100000000000232, -0.8000000000000895, 0.3899999999998851, -0.4400000000000972, -1.5400000000000738, -3.870000000000024, 3.9299999999998096, 3.8399999999998116, 0.43999999999988404, 3.069999999999828, 1.9999999999998508, 1.5599999999998602, -0.5000000000000959, 0.43999999999988404, -3.4700000000000326, -3.0500000000000416, 0.9199999999998738, -0.49000000000009614, 4.86999999999979, 2.8099999999998335, 3.0899999999998276, -1.6000000000000725, 1.9199999999998525, 4.9699999999997875, 1.899999999999853, 2.639999999999837, 3.629999999999816, 2.0599999999998495, -1.7100000000000701, -3.360000000000035, 0.27999999999988745, 4.729999999999793, 1.3399999999998649, -0.2200000000001019, -0.8400000000000887, -1.8000000000000682, 2.7099999999998357, 0.41999999999988447, 4.319999999999801, 2.3399999999998435, -1.2100000000000808, -1.900000000000066, 1.0799999999998704, -1.8700000000000667, -4.1300000000000185, 4.649999999999794, 0.2499999999998881, -3.350000000000035, -4.430000000000012, -1.2300000000000804, -0.3500000000000991, -0.7600000000000904, 0.0899999999998915, 1.5599999999998602, -2.9100000000000446, 4.649999999999794, 1.5899999999998595, 4.739999999999792, 3.249999999999824, -1.5600000000000733, -4.880000000000003, 3.719999999999814, 2.2499999999998455, -2.2500000000000586, 4.939999999999788, -4.54000000000001, 1.379999999999864, 4.509999999999797, -2.470000000000054, 2.0999999999998487, 1.659999999999858, -0.9400000000000865, -0.20000000000010232, 0.3999999999998849, -1.4700000000000752, 2.559999999999839, -4.010000000000021, 4.179999999999804, 1.7499999999998561, -2.19000000000006, 1.0399999999998712, -3.0900000000000407, -4.360000000000014, 0.62999999999988, -4.570000000000009, -4.140000000000018, -3.170000000000039, -2.230000000000059, -1.150000000000082, 0.2599999999998879, 2.739999999999835, -0.9100000000000872, 3.44999999999982, -0.6800000000000921, 0.4499999999998838, -0.9400000000000865, -4.970000000000001, 0.35999999999988574, 2.4399999999998414, 2.6799999999998363, -4.110000000000019, -1.8400000000000674, 3.3799999999998214, -1.4000000000000767, -0.15000000000010338, 0.2599999999998879, 0.43999999999988404, -1.0200000000000848, 4.419999999999799, 3.809999999999812, -0.2700000000001008, 0.4799999999998832, 3.069999999999828, -3.2300000000000377, 3.199999999999825, 2.3799999999998427, -3.0500000000000416, 2.2099999999998463, 3.949999999999809, -0.8900000000000876, 0.3199999999998866, -0.5500000000000949, -1.9100000000000659, 1.0199999999998717, -0.47000000000009656, -4.54000000000001, 3.5499999999998177, 0.15999999999989, -4.120000000000019, -3.2300000000000377, -3.5200000000000315, 4.929999999999788, 2.0199999999998504, 4.479999999999798, 2.3999999999998423, 2.2499999999998455, 2.7699999999998344, 1.429999999999863, -3.9800000000000217, 2.179999999999847, 3.109999999999827, 3.2899999999998233, 3.7999999999998124, -0.6700000000000923, 3.4099999999998207, 4.139999999999805, 4.579999999999796, 2.0099999999998506, 0.05999999999989214, -1.760000000000069, -0.7200000000000912, 1.7299999999998565, -0.8500000000000885, -3.7100000000000275, -1.1400000000000823, -2.0100000000000637, 0.7799999999998768, 1.8899999999998531, -1.2300000000000804, 4.519999999999797, 2.8099999999998335, -1.7500000000000693, -3.080000000000041, -4.280000000000015, 4.899999999999789, 3.9299999999998096, -1.5100000000000744, -2.880000000000045, 1.6299999999998587, -3.960000000000022, 4.179999999999804, 1.8799999999998533, -4.560000000000009, 2.7199999999998354, 1.1399999999998691, 1.7199999999998568, 0.41999999999988447, -1.1200000000000827, -4.700000000000006, 0.4599999999998836, -4.940000000000001, -4.900000000000002, -4.940000000000001, -1.9900000000000642, 0.3099999999998868, 3.069999999999828, -2.4000000000000554, -4.350000000000014, -1.6500000000000714, 2.5799999999998384, -4.920000000000002, 3.729999999999814, 4.749999999999792, -4.430000000000012, 1.2299999999998672, 1.6399999999998585, -4.250000000000016, 1.6699999999998578, -3.4700000000000326, 2.559999999999839, -1.6400000000000716, -2.8200000000000465, -0.6500000000000927, -1.530000000000074, -1.8400000000000674, 4.339999999999801, -4.480000000000011, -3.7000000000000277, 3.91999999999981, 0.41999999999988447, -0.5700000000000944, -3.7100000000000275, -3.3300000000000356, -4.900000000000002, 4.0299999999998075, -1.4500000000000757, 4.019999999999808, 1.3599999999998644, 1.56999999999986, 3.6199999999998163, 1.7499999999998561, 1.2699999999998663, -1.6600000000000712, 4.189999999999804, -3.2400000000000375, 1.56999999999986, -0.6600000000000925, 3.0499999999998284, -3.680000000000028, 2.839999999999833, -1.0900000000000833, -1.7000000000000703, 0.7499999999998774, -0.7500000000000906, -2.5900000000000514, 1.1399999999998691, 0.26999999999988766, 4.3899999999998, -0.48000000000009635, -1.3600000000000776, 2.919999999999831, -0.2900000000001004, 2.02999999999985, -3.59000000000003, -0.9600000000000861, -2.6900000000000492, 3.4799999999998192, 4.3999999999998, -4.750000000000005, 1.6999999999998572, 4.529999999999797, -0.6900000000000919, 3.0399999999998286, 3.2799999999998235, 1.1099999999998698, -4.6500000000000075, 2.9599999999998303, -3.6200000000000294, 4.8299999999997905, 0.33999999999988617, 4.439999999999799, -0.1300000000001038, -4.610000000000008, -2.040000000000063, 3.0099999999998293, 1.4099999999998634, -2.18000000000006, -3.8600000000000243, -1.670000000000071, 1.5899999999998595, -3.1500000000000394, -3.5800000000000303, 4.449999999999799, 4.059999999999807, -0.7400000000000908, -1.1600000000000819, 3.669999999999815, -3.2000000000000384, -2.8600000000000456, -0.0600000000001053, -4.3200000000000145, -2.140000000000061, -0.9300000000000868, -4.900000000000002, 2.179999999999847, -3.5200000000000315, 1.8699999999998536, 0.5199999999998823, 2.5199999999998397, 2.5299999999998395, -3.080000000000041, 3.219999999999825, -2.5400000000000524, -3.0000000000000426, 2.1499999999998476, 4.289999999999802, -1.150000000000082, -2.3600000000000563, 2.6299999999998374, 3.629999999999816, 0.7799999999998768, 3.969999999999809, 0.34999999999988596, 0.04999999999989235, -3.790000000000026, 0.5899999999998808, 4.289999999999802, -0.09000000000010466, -0.41000000000009784, -0.7000000000000917, -4.5100000000000104, -4.360000000000014, 2.9899999999998297, 0.579999999999881, -3.170000000000039, -2.050000000000063, 0.49999999999988276, -0.6000000000000938, 0.35999999999988574, 1.5999999999998593, -4.820000000000004, 2.9399999999998307, 1.1799999999998683, -4.860000000000003, -0.2200000000001019, 4.099999999999806, 2.6799999999998363, -3.180000000000039, 4.559999999999796, -1.1800000000000814, -0.3600000000000989, 1.3099999999998655, 2.1099999999998484, -1.390000000000077, 3.6899999999998148, 1.4899999999998617, 4.0799999999998064, 0.19999999999988916, 4.6899999999997934, 2.8499999999998327, 3.209999999999825, 0.3099999999998868, 0.3199999999998866, 4.809999999999791, -0.5100000000000957, 0.41999999999988447, 2.269999999999845, -0.1300000000001038, 2.4299999999998416, 2.2099999999998463, -0.48000000000009635, 1.5299999999998608, 4.209999999999804, -1.2100000000000808, 2.3299999999998438, 3.4299999999998203, 3.069999999999828, -0.33000000000009955, -3.820000000000025, -1.8700000000000667, 4.84999999999979, 4.419999999999799, -3.0000000000000426, 4.519999999999797, 1.5399999999998606, -3.9500000000000224, -0.450000000000097, -1.5100000000000744, -1.0300000000000846, -2.8100000000000467, -4.920000000000002, 3.259999999999824, -1.4700000000000752, 4.599999999999795, 4.319999999999801, 2.219999999999846, 4.579999999999796, 2.1599999999998474, -1.4700000000000752, 3.2399999999998244, 0.2599999999998879, -4.250000000000016, -0.450000000000097, -4.610000000000008, 1.8899999999998531, 2.8499999999998327, -1.8000000000000682, 1.4199999999998631, 0.8999999999998742, -4.590000000000009, 1.6499999999998582, 3.0099999999998293, 3.539999999999818, 0.5699999999998813, 3.349999999999822, -4.98, -0.6300000000000932, -2.520000000000053, 2.7999999999998337, -3.7600000000000264, 1.6499999999998582, 1.5499999999998604, -3.1100000000000403, -4.260000000000016, -1.3800000000000772, 1.0299999999998715, 3.1499999999998263, 4.149999999999805, 1.2299999999998672, -0.8500000000000885, -1.8900000000000663, 4.119999999999806, 4.009999999999808, 0.3099999999998868, 3.969999999999809, -1.6000000000000725, 1.0899999999998702, -1.8300000000000676, 2.3499999999998433, -1.7400000000000695, 0.4099999999998847, -0.030000000000105942, -2.270000000000058, -1.8500000000000671, 0.28999999999988724, 4.3799999999998, -3.210000000000038, 4.269999999999802, -0.050000000000105516, 1.8899999999998531, -3.880000000000024, -0.12000000000010402, 1.8799999999998533, -4.330000000000014, 0.8599999999998751, -0.3800000000000985, 1.0299999999998715, 4.979999999999787, -0.04000000000010573, 3.3399999999998222, -4.830000000000004, 3.029999999999829, 4.649999999999794, 1.6199999999998589, 2.7099999999998357, -1.2700000000000795, 0.20999999999988894, 0.5899999999998808, 0.62999999999988, -1.8300000000000676, -4.670000000000007, 1.1199999999998695, -2.510000000000053, 3.489999999999819, 2.3299999999998438, 2.6599999999998367, -4.52000000000001, 2.599999999999838, -1.9100000000000659, 3.4199999999998205, 1.989999999999851, -1.7500000000000693, 0.3999999999998849, -1.5600000000000733, 1.5899999999998595, 3.7999999999998124, 0.9699999999998727, 2.449999999999841, 4.489999999999798, 0.9499999999998732, 0.8599999999998751, -3.12000000000004, 1.2499999999998668, -3.6100000000000296, 1.009999999999872, 1.8599999999998538, -1.8800000000000665, -0.8200000000000891, 3.3199999999998226, -1.1900000000000812, -0.0600000000001053, -3.5200000000000315, 2.599999999999838, 2.279999999999845, 3.0999999999998273, 0.3099999999998868, -2.880000000000045, -1.8900000000000663, 0.5199999999998823, -3.9000000000000234, 4.799999999999791, -2.3500000000000565, 4.009999999999808, 3.91999999999981, -3.1900000000000386, -2.5400000000000524, 3.90999999999981, -3.400000000000034, 1.6699999999998578, 3.1899999999998254, 3.1299999999998267, 4.659999999999794, -2.980000000000043, 3.90999999999981, -2.700000000000049, 4.709999999999793, 3.7099999999998143, -2.9600000000000435, 1.56999999999986, 2.089999999999849, 1.4999999999998614, 4.759999999999792, 0.7099999999998783, 0.02999999999989278, 0.9899999999998723, -3.9800000000000217, 2.3499999999998433, -4.08000000000002, -2.8200000000000465, -4.360000000000014, -1.860000000000067, -0.6000000000000938, 0.8799999999998747, -0.20000000000010232, -2.4500000000000544, 2.409999999999842, -0.1400000000001036, 3.109999999999827, 4.699999999999793, 0.34999999999988596, 1.4799999999998619, -4.3200000000000145, 3.219999999999825, -3.920000000000023, -4.890000000000002, 4.439999999999799, 0.299999999999887, -1.060000000000084, 4.84999999999979, 3.729999999999814, -3.730000000000027, -3.2800000000000367, -2.0600000000000627, 4.559999999999796, -4.560000000000009, 3.589999999999817, 3.639999999999816, -1.72000000000007, -0.8800000000000878, 0.5099999999998825, -4.660000000000007, -3.1100000000000403, -3.8500000000000245, 0.42999999999988425, -4.07000000000002, -4.490000000000011, 0.769999999999877, 2.2999999999998444, 0.3899999999998851, -1.110000000000083, -3.350000000000035, -0.050000000000105516, -2.6400000000000503, -1.2800000000000793, 1.8299999999998544, 0.18999999999988937, -1.150000000000082, 4.189999999999804, -1.900000000000066, 2.4799999999998406, -0.8000000000000895, -0.1300000000001038, -4.830000000000004, -3.7100000000000275, 1.1699999999998685, -0.3500000000000991, 4.529999999999797, 3.0999999999998273, -3.5600000000000307, 2.449999999999841, 3.299999999999823, 4.239999999999803, -2.600000000000051, -1.0658141036401503e-13, 2.7099999999998357, -2.9000000000000448, -2.7800000000000473, -2.430000000000055, 2.3399999999998435, -3.820000000000025, -4.450000000000012, -4.370000000000013, -1.0400000000000844, -2.9200000000000443, -3.9900000000000215, -3.9900000000000215, 4.119999999999806, -3.7000000000000277, -2.1200000000000614, -1.0700000000000838, 3.969999999999809, -2.7200000000000486, -3.960000000000022, -0.020000000000106155, 1.1199999999998695, 3.159999999999826, -4.55000000000001, -4.07000000000002, 2.2599999999998452, 3.069999999999828, 2.2599999999998452, -4.020000000000021, 4.7799999999997915, 4.86999999999979, 1.4799999999998619, 4.909999999999789, 4.009999999999808, -2.66000000000005, -1.6100000000000723, -3.5100000000000318, -2.040000000000063, -0.2900000000001004, -3.490000000000032, 1.5299999999998608, -2.18000000000006, -4.900000000000002, -1.010000000000085, -0.1300000000001038, -4.590000000000009, -2.6400000000000503, -0.9500000000000863, -0.26000000000010104, -2.710000000000049, -3.780000000000026, -4.770000000000005, -3.2900000000000365, 1.149999999999869, 3.159999999999826, 1.1999999999998678, 0.009999999999893205, -4.850000000000003, -3.5300000000000313, 0.43999999999988404, 0.62999999999988, 2.50999999999984, 4.489999999999798, 4.669999999999794, -3.490000000000032, 0.43999999999988404, -4.390000000000013, -1.8400000000000674, 0.8499999999998753, -3.030000000000042, 2.789999999999834, -1.3100000000000787, -2.7800000000000473, 2.739999999999835, -2.370000000000056, 3.3299999999998224, -3.6200000000000294, -4.810000000000004, 4.269999999999802, -1.7400000000000695, 4.769999999999792, 0.34999999999988596, 2.6799999999998363, -2.6900000000000492, 0.6499999999998796, 0.3899999999998851, 2.079999999999849, -0.6500000000000927, -0.6000000000000938, 4.269999999999802, -3.7200000000000273, -0.8000000000000895, -0.7900000000000897, -4.220000000000017, 1.519999999999861, 3.359999999999822, 1.6799999999998576, -0.04000000000010573, 1.989999999999851, 1.2799999999998661, 2.3399999999998435, 2.089999999999849, 3.629999999999816, 4.619999999999795, 1.989999999999851, -3.540000000000031, 1.1699999999998685, -4.260000000000016, 2.2899999999998446, 0.8499999999998753, 0.8299999999998757, -4.170000000000018, 1.1999999999998678, 1.6999999999998572, 1.0599999999998708, -3.4200000000000337, 1.3899999999998638, 3.5999999999998167, -4.750000000000005, -1.3000000000000789, -1.5400000000000738, 4.819999999999791, -1.7800000000000686, 3.8899999999998105, 3.719999999999814, -0.3100000000001, 2.219999999999846, -1.9800000000000644, -4.420000000000012, 2.1499999999998476, 2.079999999999849, -4.840000000000003, -0.9600000000000861, 0.1699999999998898, -3.8000000000000256, -2.410000000000055, -4.860000000000003, -3.9500000000000224, -0.2800000000001006, -3.310000000000036, -2.0800000000000622, 1.3499999999998646, -0.870000000000088, -1.1000000000000831, 1.4199999999998631, -0.10000000000010445, -4.580000000000009, 3.2399999999998244, -1.7400000000000695, -1.1600000000000819, -2.8100000000000467, 0.06999999999989193, -0.6200000000000934, 3.809999999999812, 2.079999999999849, -4.830000000000004, -1.110000000000083, 1.4999999999998614, -4.380000000000013, 1.759999999999856, 3.219999999999825, 0.35999999999988574, -2.1600000000000605, 2.9899999999998297, -0.34000000000009933, 4.2199999999998035, -1.9400000000000652, 3.9299999999998096, -3.450000000000033, -0.9100000000000872, 1.7799999999998555, 1.5299999999998608, -0.450000000000097, 1.4399999999998627, -0.7900000000000897, -3.7000000000000277, 4.759999999999792, -1.8400000000000674, 3.779999999999813, -3.59000000000003, -3.1500000000000394, -3.9400000000000226, 3.029999999999829, -3.5200000000000315, -3.410000000000034, -2.9600000000000435, -0.8400000000000887, -0.9100000000000872, -0.3600000000000989, -3.780000000000026, -2.240000000000059, 4.509999999999797, 1.6799999999998576, 2.839999999999833, 1.6699999999998578, -0.2900000000001004, 1.8699999999998536, -3.6700000000000284, -4.120000000000019, -0.5400000000000951, -4.290000000000015, 3.1299999999998267, -2.800000000000047, -3.550000000000031, -2.1700000000000603, -1.0400000000000844, 2.279999999999845, -3.2400000000000375, -3.880000000000024, 0.4099999999998847, -4.230000000000016, 3.629999999999816, -0.2900000000001004, -1.5100000000000744, -2.990000000000043, -5, 3.7099999999998143, 2.97999999999983, -4.260000000000016, 1.519999999999861, -0.42000000000009763, -4.360000000000014, 3.7999999999998124, 0.14999999999989022, 0.8199999999998759, -2.66000000000005, 2.9399999999998307, 2.139999999999848, 3.3399999999998222, 0.12999999999989065, -4.870000000000003, 4.949999999999788, 1.9999999999998508, -1.900000000000066, -3.5700000000000305, -1.4400000000000759, 1.6699999999998578, -1.4500000000000757, -0.9800000000000857, -0.450000000000097, 1.5099999999998612, 0.43999999999988404, 0.5899999999998808, 2.3399999999998435, -1.7100000000000701, -3.4700000000000326, 3.5599999999998175, 4.769999999999792, 0.5899999999998808, 0.7099999999998783, 0.959999999999873, 1.2999999999998657, -3.690000000000028, 1.4199999999998631, 1.4099999999998634, -3.960000000000022, -4.360000000000014, 4.3099999999998015, 4.789999999999791, -0.8600000000000882, 4.889999999999789, -3.4700000000000326, 4.229999999999803, 2.7699999999998344, -1.4000000000000767, -4.140000000000018, 4.3099999999998015, -1.3300000000000782, -1.1400000000000823, 3.359999999999822, 3.959999999999809, 0.0899999999998915, -2.370000000000056, -0.8100000000000893, -1.3000000000000789, 0.7099999999998783, -3.320000000000036, 2.9499999999998305, 2.0099999999998506, 0.20999999999988894, 2.4699999999998408, -3.5700000000000305, -4.340000000000014, 3.779999999999813, 3.639999999999816, -3.13000000000004, -3.9800000000000217, 1.899999999999853, -1.8700000000000667, 2.179999999999847, -2.140000000000061, -2.710000000000049, -3.260000000000037, 1.0599999999998708, -4.480000000000011, 4.059999999999807, 1.2699999999998663, -2.570000000000052, -0.9500000000000863, -4.660000000000007, -3.5100000000000318, -3.5100000000000318, -3.3900000000000343, 2.179999999999847, -4.090000000000019, -1.1400000000000823, -1.0300000000000846, 1.049999999999871, -4.820000000000004, -0.9500000000000863, -3.830000000000025, -3.160000000000039, -3.170000000000039, -2.420000000000055, 2.2999999999998444, 4.339999999999801, -1.5600000000000733, 1.3499999999998646, 2.7999999999998337, -3.1100000000000403, 1.149999999999869, 2.50999999999984, -1.3500000000000778, -3.160000000000039, 3.539999999999818, 2.3299999999998438, 4.619999999999795, 4.529999999999797, -3.160000000000039, 2.7099999999998357, 3.999999999999808, -4.880000000000003, 2.9099999999998314, 0.6599999999998793, -3.970000000000022, 0.7799999999998768, 2.139999999999848, -1.7500000000000693, 1.7199999999998568, 4.009999999999808, 0.3199999999998866, -1.2600000000000797, -0.6100000000000936, 2.8099999999998335, 2.9999999999998295, 2.0099999999998506, 3.779999999999813, 3.399999999999821, -1.5900000000000727, -2.65000000000005, 3.2699999999998237, -0.8400000000000887, 0.8299999999998757, 2.9499999999998305, -1.290000000000079, -1.4200000000000763, -1.6900000000000706, -1.9100000000000659, -3.5800000000000303, -0.9300000000000868, -4.07000000000002, 3.6999999999998145, 2.4399999999998414, -1.3100000000000787, -1.0300000000000846, 1.8599999999998538, 4.729999999999793, 2.4899999999998403, -1.5700000000000731, 3.7999999999998124, 2.9399999999998307, -1.6000000000000725, -4.52000000000001, 0.21999999999988873, -0.9000000000000874, -4.250000000000016, 4.459999999999798, -3.730000000000027, 1.4799999999998619, -0.870000000000088, -4.08000000000002, 2.599999999999838, 1.329999999999865, -0.5200000000000955, -0.6400000000000929, 1.1599999999998687, 4.729999999999793, -0.08000000000010488, 4.139999999999805, 0.7599999999998772, 0.959999999999873, 4.8299999999997905, 4.449999999999799, 1.519999999999861, 1.3899999999998638, 0.5899999999998808, -3.3000000000000362, 3.6999999999998145, -1.1900000000000812, -2.4900000000000535, 3.0499999999998284, 1.4399999999998627, -2.050000000000063, -3.5200000000000315, 1.9999999999998508, -3.320000000000036, -1.200000000000081, 0.49999999999988276, 0.959999999999873, -1.8000000000000682, 0.3199999999998866, 1.3999999999998636, -4.890000000000002, 4.559999999999796, 2.9899999999998297, 4.3899999999998, 0.18999999999988937, -2.0300000000000633, 4.83999999999979, -2.6300000000000505, -0.3100000000001, 1.8599999999998538, -4.9300000000000015, -3.540000000000031, -4.800000000000004, 0.4799999999998832, 4.469999999999798, 2.4699999999998408, 4.449999999999799, -2.100000000000062, 2.0499999999998497, 2.2899999999998446, 1.8899999999998531, 0.41999999999988447, -0.47000000000009656, 4.6399999999997945, 1.519999999999861, 1.899999999999853, -2.840000000000046, -3.310000000000036, 4.939999999999788, -1.6300000000000718, 0.22999999999988852, 0.28999999999988724, -4.700000000000006, 4.7799999999997915, 1.6299999999998587, 0.9499999999998732, -2.9500000000000437, 4.86999999999979, 4.83999999999979, -2.5900000000000514, -2.890000000000045, 4.109999999999806, 0.8799999999998747, -1.900000000000066, 0.5399999999998819, -1.3100000000000787, 3.309999999999823, -4.120000000000019, -4.380000000000013, -1.150000000000082, 4.889999999999789, 0.6199999999998802, -3.970000000000022, 4.299999999999802, 2.3999999999998423, -1.25000000000008, 3.949999999999809, -2.3500000000000565, -0.7600000000000904, 1.6199999999998589, 0.7799999999998768, 4.8299999999997905, -4.240000000000016, 2.9599999999998303, -3.180000000000039, -0.3100000000001, -2.880000000000045, 4.3599999999998005, -4.250000000000016, -4.230000000000016, -4.230000000000016, 0.22999999999988852, -1.1700000000000816, 2.0199999999998504, -3.0500000000000416, -4.630000000000008, -3.3700000000000347, 2.0999999999998487, -3.5800000000000303, -4.9300000000000015, 1.6399999999998585, -0.6800000000000921, -1.110000000000083, -2.0300000000000633, 0.09999999999989129, -4.380000000000013, 0.6799999999998789, 0.9199999999998738, -1.1200000000000827, -2.610000000000051, 0.10999999999989107, -3.4300000000000335, 1.1399999999998691, -1.4100000000000765, 4.4999999999997975, -1.9800000000000644, -4.030000000000021, 0.8499999999998753, -1.620000000000072, 1.09999999999987, 4.719999999999793, -1.0700000000000838, 0.6699999999998791, 4.789999999999791, -1.580000000000073, -4.200000000000017, 0.3199999999998866, 2.559999999999839, 3.4099999999998207, -2.4000000000000554, 0.15999999999989, 2.8699999999998322, 2.449999999999841, -2.2900000000000578, 0.19999999999988916, -0.6800000000000921, -4.760000000000005, 0.8899999999998744, 2.8699999999998322, 2.7099999999998357, 4.269999999999802, -4.210000000000017, 2.2599999999998452, -1.3600000000000776, -1.9900000000000642, -4.720000000000006, -0.9300000000000868, 0.3999999999998849, 0.12999999999989065, 4.939999999999788, -0.8600000000000882, 0.28999999999988724, -3.400000000000034, 2.7299999999998352, -1.4700000000000752, -3.2900000000000365, 0.7599999999998772, -1.1900000000000812, -0.2800000000001006, -2.790000000000047, -0.870000000000088, -4.6000000000000085, -1.2100000000000808, -0.6900000000000919, -0.010000000000106368, 2.919999999999831, 0.8099999999998762, 0.09999999999989129, -2.0600000000000627, 4.579999999999796, 0.33999999999988617, -3.9100000000000232, 2.089999999999849, -1.2600000000000797, 4.679999999999794, 1.8899999999998531, 2.49999999999984, 2.96999999999983, -1.4900000000000748, 2.3399999999998435, -3.8400000000000247, -2.700000000000049, -2.040000000000063, -1.3300000000000782, 0.5299999999998821, 2.7999999999998337, 0.33999999999988617, 4.039999999999807, 1.6899999999998574, -1.3700000000000774, 0.5599999999998815, -1.2300000000000804, -4.190000000000017, -3.59000000000003, -1.1900000000000812, -1.0400000000000844, 4.699999999999793, 3.5499999999998177, -1.7700000000000689, 1.899999999999853, -4.170000000000018, -1.5200000000000742, -2.1500000000000608, 1.8399999999998542, 4.629999999999795, 4.619999999999795, -1.4900000000000748, 1.849999999999854, 1.3899999999998638, 0.4599999999998836, -0.42000000000009763, 4.129999999999805, -3.2400000000000375, 3.3299999999998224, 4.83999999999979, 1.4499999999998625, -0.5500000000000949, 3.6999999999998145, 4.159999999999805, -4.360000000000014, 0.7299999999998779, 0.5099999999998825, 0.5999999999998806, 0.8799999999998747, -3.3300000000000356, 1.8099999999998548, 1.8099999999998548, 3.5499999999998177, -4.570000000000009, 0.3199999999998866, -0.3800000000000985, -0.20000000000010232, 3.729999999999814, 1.3999999999998636, -2.280000000000058, -1.480000000000075, 4.409999999999799, 4.929999999999788, -2.8100000000000467, 1.609999999999859, 4.679999999999794, 4.519999999999797, 1.6299999999998587, 2.1999999999998465, 4.3699999999998, -4.54000000000001, 2.829999999999833, -4.440000000000012, -2.0600000000000627, -1.7000000000000703, -0.6600000000000925, 2.229999999999846, -4.950000000000001, -1.0000000000000853, 2.49999999999984, 4.139999999999805, -4.580000000000009, 0.489999999999883, 2.559999999999839, -2.7800000000000473, 4.449999999999799, -1.4500000000000757, 4.579999999999796, -1.0700000000000838, -3.310000000000036, -2.380000000000056, 2.02999999999985, 1.8299999999998544, -1.8200000000000678, 3.529999999999818, 1.6499999999998582, 2.749999999999835, -1.3200000000000784, 2.689999999999836, 3.109999999999827, -3.1500000000000394, -2.520000000000053, 4.3899999999998, 4.939999999999788, 3.3299999999998224, 1.0699999999998706, -1.390000000000077, 2.229999999999846, -2.0700000000000625, -2.710000000000049, -3.490000000000032, 1.3199999999998653, -4.850000000000003, -3.5700000000000305, -0.3000000000001002, 1.4599999999998623, -3.6100000000000296, -2.7300000000000484, 2.4799999999998406, -0.09000000000010466, 4.609999999999795, -2.270000000000058, -4.440000000000012, -3.040000000000042, -1.4000000000000767, -4.940000000000001, 1.899999999999853, -0.5000000000000959, 1.4799999999998619, -1.760000000000069, 1.2199999999998674, -3.6700000000000284, 4.979999999999787, -2.7600000000000477, 2.279999999999845, 3.529999999999818, 2.929999999999831, -4.370000000000013, 3.779999999999813, -0.9600000000000861, -3.500000000000032, -0.40000000000009805, -0.3700000000000987, 1.7499999999998561, -0.33000000000009955, -4.480000000000011, 3.999999999999808, 0.41999999999988447, -4.100000000000019, 1.9199999999998525, -1.3100000000000787, 2.4799999999998406, -0.5100000000000957, 2.0699999999998493, -2.790000000000047, 0.5299999999998821, -3.030000000000042, -2.66000000000005, 4.159999999999805, 4.3599999999998005, 2.4699999999998408, -3.7600000000000264, 3.5499999999998177, 3.3699999999998216, -2.320000000000057, -0.6800000000000921, 0.9799999999998725, -3.5100000000000318, 2.1499999999998476, 0.36999999999988553, -2.4900000000000535, 0.22999999999988852, -1.1200000000000827, -3.360000000000035, -4.030000000000021, -0.8600000000000882, 2.1599999999998474, 0.09999999999989129, 3.7499999999998135, -0.5400000000000951, -4.000000000000021, 2.359999999999843, 0.3099999999998868, -3.880000000000024, 0.9999999999998721, 4.549999999999796, 4.819999999999791, 3.7499999999998135, 0.04999999999989235, -3.5300000000000313, 3.7899999999998126, -2.19000000000006, -0.6000000000000938, -1.0700000000000838, 0.13999999999989043, 4.8299999999997905, 0.28999999999988724, -3.180000000000039, 1.709999999999857, 3.869999999999811, 3.949999999999809, 1.2499999999998668, -1.430000000000076, -0.8800000000000878, 4.179999999999804, -4.300000000000015, -1.2600000000000797, -3.680000000000028, 4.3799999999998, 0.20999999999988894, -0.26000000000010104, -2.8700000000000454, -3.8100000000000254, -3.960000000000022, 0.959999999999873, 3.7999999999998124, -1.0000000000000853, -4.950000000000001, 1.0799999999998704, -4.7900000000000045, 2.0199999999998504, -2.370000000000056, -4.030000000000021, -2.880000000000045, 3.579999999999817, -1.7700000000000689, -1.1300000000000825, -4.860000000000003, 0.41999999999988447, 0.4699999999998834, -1.7100000000000701, 4.009999999999808, 1.2299999999998672, 2.4799999999998406, -2.430000000000055, 2.0599999999998495, 3.5499999999998177, -0.9100000000000872, 1.6499999999998582, -1.7700000000000689, 3.7599999999998133, 0.8599999999998751, 2.8599999999998325, 0.8799999999998747, 0.20999999999988894, -1.7400000000000695, -1.25000000000008, 0.5099999999998825, -1.0800000000000836, 0.17999999999988958, 2.7699999999998344, -1.3600000000000776, -0.450000000000097, -0.7400000000000908, 4.319999999999801, -3.930000000000023, 0.579999999999881, -3.820000000000025, 2.639999999999837, -2.5500000000000522, 3.2399999999998244, -3.6600000000000286, 4.339999999999801, 0.28999999999988724, -4.940000000000001, 4.189999999999804, -2.3500000000000565, -1.0400000000000844, -4.800000000000004, 3.059999999999828, 4.089999999999806, 2.169999999999847, 1.5299999999998608, -3.8500000000000245, 2.0499999999998497, -2.19000000000006, 3.539999999999818, -3.400000000000034, 2.689999999999836, 4.139999999999805, 4.619999999999795, -4.010000000000021, 2.0699999999998493, -4.730000000000006, 2.0999999999998487, -3.8600000000000243, 1.2499999999998668, 3.9399999999998094, -1.5000000000000746, 3.059999999999828, 1.989999999999851, -2.9200000000000443, 0.7599999999998772, 1.709999999999857, 4.1699999999998045, -0.9700000000000859, 3.1299999999998267, -1.3600000000000776, -0.10000000000010445, 1.5099999999998612, 2.3399999999998435, 1.3599999999998644, 0.26999999999988766, -4.020000000000021, -1.5400000000000738, -0.19000000000010253, 3.6099999999998165, -0.49000000000009614, -3.1100000000000403, 4.419999999999799, 0.7099999999998783, -0.2900000000001004, -3.170000000000039, -2.9600000000000435, -3.040000000000042, -1.0500000000000842, 0.4799999999998832, -2.940000000000044, 2.369999999999843, -3.6100000000000296, 0.6799999999998789 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "x" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "y" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_convex_0, x=\"x\", y=\"y\", color=\"score\", color_continuous_scale=color_scale)\n", "fig.update_layout(width=900, height=800, xaxis_range=[-5, 5], yaxis_range=[-5, 5])\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "confused-declaration", "metadata": {}, "source": [ "\n", " \n", "The plot above shows the samples from the search data acquired from the convex-function in a 2-dimensional search space. The score is shown by the color of each point in the scatter plot. " ] }, { "cell_type": "markdown", "id": "stainless-subcommittee", "metadata": {}, "source": [ "\n", "\n", "We were able to see, that random search is a good optimization technique to explore the search space. But the goal is often to quickly find position in the search space with a high score. Therefore we should consider other optimization techniques like the hill climbing algorithm." ] }, { "cell_type": "markdown", "id": "legislative-future", "metadata": {}, "source": [ "\n", "\n", "### Hill Climbing Optimizer\n", " \n", "The hill climbing optimization algorithm works by finding a random neighbour position close to the current position. If the score of the new position is better than the current one the algorithm makes a step to the new position and returns to finding the next position. This behaviour is like someone who tries to find the highest (highest score) position in a space by only moving up and never moves down. \n", " \n", "The hill climbing algorithm works very well with convex optimization problems, because the score continuously improves towards a direction. Hill climbing can find this direction by exploring the scores of its neighbours.\n", "Hill climbing does not work of there are local optima. It tends to get \"stuck\" in certain regions, where the current position is surrounded by positions with worse scores. The algorithm would need to first \"go down\" and later \"go up\" again to find other (even better) positions in the search space.\n" ] }, { "cell_type": "markdown", "id": "spectacular-terrorist", "metadata": {}, "source": [ "\n", "\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 12, "id": "separate-updating", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " x y score\n", "0 4.99 -5.000000e+00 -49.9001\n", "1 4.99 -4.470000e+00 -44.8810\n", "2 4.82 -4.910000e+00 -47.3405\n", "3 4.82 -4.790000e+00 -46.1765\n", "4 4.99 -5.000000e+00 -49.9001\n", ".. ... ... ...\n", "85 -0.28 4.000000e-02 -0.0800\n", "86 0.80 -1.065814e-13 -0.6400\n", "87 0.02 -2.500000e-01 -0.0629\n", "88 0.43 -1.065814e-13 -0.1849\n", "89 0.23 6.000000e-02 -0.0565\n", "\n", "[90 rows x 3 columns] \n", "\n" ] } ], "source": [ "optimizer = HillClimbingOptimizer(rand_rest_p=0)\n", "\n", "hyper_convex_1 = Hyperactive(verbosity=False)\n", "hyper_convex_1.add_search(convex_function, search_space, n_iter=90, optimizer=optimizer, initialize={\"vertices\":1})\n", "hyper_convex_1.run()\n", "\n", "search_data_convex_1 = hyper_convex_1.search_data(convex_function)" ] }, { "cell_type": "code", "execution_count": 13, "id": "controlled-coalition", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "x=%{x}
y=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ -49.90009999999788, -44.880999999997975, -47.340499999998, -46.17649999999802, -49.90009999999788, -43.809799999998106, -40.82409999999807, -37.76929999999835, -38.22259999999808, -43.562499999997996, -47.752899999998014, -44.71359999999821, -40.43209999999832, -37.23969999999841, -32.40249999999861, -42.480499999998116, -31.478899999998603, -34.65619999999846, -36.22449999999849, -31.372999999998633, -28.80529999999878, -33.09049999999853, -26.428899999998887, -26.006499999998926, -29.339599999998736, -26.532999999998925, -24.70499999999887, -25.68489999999906, -25.132499999998767, -23.934499999998845, -21.062499999999087, -26.588899999998745, -19.351399999999153, -22.147699999999, -18.303699999999235, -22.56649999999911, -24.332499999999023, -20.91049999999896, -18.45369999999916, -18.972499999999265, -14.270599999999359, -22.526499999999004, -16.16199999999913, -16.254799999999335, -15.469699999999262, -14.534499999999353, -11.774899999999597, -10.78689999999962, -14.018499999999452, -13.234499999999354, -11.940999999999434, -9.704499999999666, -9.14559999999973, -7.656199999999742, -8.582599999999669, -7.86689999999985, -5.347699999999762, -9.616899999999669, -5.0569999999997925, -5.312499999999579, -7.736899999999707, -3.149199999999819, -7.486899999999901, -4.147999999999815, -3.1257999999998756, -1.915699999999882, -3.5728999999998754, -1.6593999999999465, -4.099599999999795, -1.6713999999997966, -0.6400000000000068, -2.3258000000000503, -1.7356999999998515, -0.2465999999999383, -1.0530000000001085, -0.6124999999998992, -0.989299999999798, -0.6569999999999464, -1.3024999999996694, -0.03609999999995796, -0.19329999999987452, -0.02890000000001369, -0.08730000000002826, -0.3677000000001399, -0.06609999999995668, -0.08000000000004774, -0.6399999999998022, -0.06290000000004635, -0.18489999999990045, -0.056499999999935775 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 4.989999999999787, 4.989999999999787, 4.819999999999791, 4.819999999999791, 4.989999999999787, 4.729999999999793, 4.949999999999788, 4.419999999999799, 4.989999999999787, 4.989999999999787, 4.769999999999792, 4.439999999999799, 4.3899999999998, 4.339999999999801, 4.049999999999807, 4.789999999999791, 4.099999999999806, 4.3099999999998015, 4.1699999999998045, 4.0299999999998075, 3.769999999999813, 4.209999999999804, 3.5999999999998167, 3.5199999999998184, 3.859999999999811, 3.5099999999998186, 3.6899999999998148, 3.199999999999825, 3.8999999999998103, 3.7599999999998133, 3.2799999999998235, 3.91999999999981, 3.169999999999826, 3.4599999999998197, 2.9899999999998297, 3.1899999999998254, 3.3399999999998222, 3.5599999999998175, 3.159999999999826, 2.8999999999998316, 2.749999999999835, 3.43999999999982, 3.2399999999998244, 2.779999999999834, 2.9599999999998303, 2.7599999999998346, 2.179999999999847, 2.129999999999848, 2.5299999999998395, 2.7599999999998346, 2.5699999999998386, 2.0099999999998506, 1.8399999999998542, 1.7899999999998553, 1.989999999999851, 1.4999999999998614, 1.659999999999858, 1.9999999999998508, 1.56999999999986, 2.0199999999998504, 1.8799999999998533, 1.3599999999998644, 1.3499999999998646, 1.4599999999998623, 1.2299999999998672, 1.0599999999998708, 1.2699999999998663, 0.8699999999998749, 1.4999999999998614, 1.1699999999998685, 0.4799999999998832, 0.6699999999998791, 1.0899999999998702, 0.4499999999998838, 0.26999999999988766, 0.6999999999998785, 0.9699999999998727, 0.62999999999988, 1.1299999999998693, -1.0658141036401503e-13, 0.41999999999988447, 0.07999999999989171, 0.11999999999989086, -0.590000000000094, 0.2499999999998881, -0.2800000000001006, 0.7999999999998764, 0.019999999999892992, 0.42999999999988425, 0.22999999999988852 ], "xaxis": "x", "y": [ -5, -4.470000000000011, -4.910000000000002, -4.7900000000000045, -5, -4.630000000000008, -4.0400000000000205, -4.270000000000016, -3.6500000000000288, -4.3200000000000145, -5, -5, -4.6000000000000085, -4.290000000000015, -4.000000000000021, -4.420000000000012, -3.830000000000025, -4.010000000000021, -4.340000000000014, -3.8900000000000237, -3.820000000000025, -3.920000000000023, -3.6700000000000284, -3.690000000000028, -3.8000000000000256, -3.770000000000026, -3.3300000000000356, -3.930000000000023, -3.1500000000000394, -3.13000000000004, -3.210000000000038, -3.350000000000035, -3.0500000000000416, -3.1900000000000386, -3.0600000000000414, -3.5200000000000315, -3.630000000000029, -2.8700000000000454, -2.9100000000000446, -3.2500000000000373, -2.5900000000000514, -3.270000000000037, -2.380000000000056, -2.9200000000000443, -2.5900000000000514, -2.6300000000000505, -2.65000000000005, -2.5000000000000533, -2.7600000000000477, -2.370000000000056, -2.3100000000000573, -2.380000000000056, -2.4000000000000554, -2.1100000000000616, -2.1500000000000608, -2.370000000000056, -1.6100000000000723, -2.370000000000056, -1.6100000000000723, -1.110000000000083, -2.050000000000063, -1.1400000000000823, -2.380000000000056, -1.4200000000000763, -1.2700000000000795, -0.8900000000000876, -1.4000000000000767, -0.9500000000000863, -1.3600000000000776, -0.5500000000000949, -0.6400000000000929, -1.3700000000000774, -0.7400000000000908, -0.2100000000001021, -0.9900000000000855, -0.3500000000000991, -0.2200000000001019, -0.5100000000000957, 0.15999999999989, 0.18999999999988937, 0.12999999999989065, -0.15000000000010338, -0.2700000000001008, -0.1400000000001036, -0.0600000000001053, 0.039999999999892566, -1.0658141036401503e-13, -0.25000000000010125, -1.0658141036401503e-13, 0.05999999999989214 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "x" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "y" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_convex_1, x=\"x\", y=\"y\", color=\"score\", color_continuous_scale=color_scale)\n", "fig.update_layout(width=900, height=800, xaxis_range=[-5, 5], yaxis_range=[-5, 5])\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "boxed-processor", "metadata": {}, "source": [ "\n", " \n", "The 2D-scatter plot above shows that the hill climbing algorithm convertes quickly to the optimum of the objective function in the search space. Hill climbing is specialized to find the optimum of convex functions quickly. It was able to find a good score in less than 100 iterations, while the random search used much more for a similar maximum score. " ] }, { "cell_type": "markdown", "id": "sexual-compatibility", "metadata": {}, "source": [ "\n", " \n", "## Non-convex Optimization \n", " \n", "Next we explore the performance of different optimization algorithms on the ackley-function, which is a nonconvex objective function:" ] }, { "cell_type": "code", "execution_count": 14, "id": "alert-plenty", "metadata": {}, "outputs": [], "source": [ "def ackley_function(para):\n", " x, y = para[\"x\"], para[\"y\"]\n", "\n", " loss = (\n", " -20 * np.exp(-0.2 * np.sqrt(0.5 * (x * x + y * y)))\n", " - np.exp(0.5 * (np.cos(2 * np.pi * x) + np.cos(2 * np.pi * y)))\n", " + np.exp(1)\n", " + 20\n", " )\n", "\n", " return -loss\n", "\n", "\n", "search_space = {\n", " \"x\": list(np.arange(-5, 5, 0.01)),\n", " \"y\": list(np.arange(-5, 5, 0.01)),\n", "}" ] }, { "cell_type": "markdown", "id": "guilty-harvest", "metadata": {}, "source": [ "\n", "\n", "The ackley function is a non convex function that has a lot of local optima. This is because of the cos-terms that create wave-like patterns in both dimensions.\n", " \n", "In the following 3D-surface plots you can see an example for the sphere-function and ackley-function. Both plots are interactive, so you can take a closer look at the shape of those objective functions:" ] }, { "cell_type": "code", "execution_count": 15, "id": "municipal-parliament", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ], "scene": "scene", "type": "surface", "x": [ [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ] ], "y": [ [ -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5 ], [ -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8 ], [ -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6 ], [ -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995 ], [ -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999 ], [ -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999 ], [ -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999 ], [ -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988 ], [ -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986 ], [ -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984 ], [ -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982 ], [ -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998 ], [ -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998 ], [ -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977 ], [ -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975 ], [ -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973 ], [ -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972 ], [ -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997 ], [ -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968 ], [ -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966 ], [ -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964 ], [ -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963 ], [ -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961 ], [ -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959 ], [ -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574 ], [ 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15 ], [ 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462 ], [ 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048 ], [ 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005 ], [ 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052 ], [ 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053 ], [ 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055 ], [ 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057 ], [ 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059 ], [ 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006 ], [ 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006 ], [ 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064 ], [ 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066 ], [ 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068 ], [ 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007 ], [ 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007 ], [ 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064 ], [ 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075 ], [ 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085 ], [ 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008 ], [ 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007 ], [ 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008 ], [ 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009 ], [ 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085 ], [ 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008 ] ], "z": [ [ -50, -48.04, -46.16, -44.36, -42.63999999999999, -40.99999999999999, -39.43999999999999, -37.959999999999994, -36.55999999999999, -35.23999999999999, -33.999999999999986, -32.83999999999999, -31.75999999999999, -30.75999999999999, -29.83999999999999, -28.99999999999999, -28.239999999999988, -27.55999999999999, -26.95999999999999, -26.43999999999999, -25.999999999999993, -25.639999999999993, -25.359999999999996, -25.159999999999997, -25.04, -25, -25.040000000000003, -25.160000000000004, -25.360000000000007, -25.640000000000008, -26.00000000000001, -26.440000000000012, -26.960000000000015, -27.56000000000002, -28.240000000000023, -29.000000000000025, -29.84000000000003, -30.760000000000034, -31.760000000000034, -32.84000000000004, -34.00000000000004, -35.24000000000004, -36.56000000000005, -37.960000000000065, -39.44000000000006, -41.00000000000006, -42.64000000000007, -44.360000000000085, -46.16000000000008, -48.04000000000008 ], [ -48.04, -46.08, -44.199999999999996, -42.39999999999999, -40.67999999999999, -39.03999999999999, -37.47999999999999, -35.999999999999986, -34.59999999999999, -33.27999999999999, -32.03999999999999, -30.87999999999999, -29.79999999999999, -28.79999999999999, -27.87999999999999, -27.03999999999999, -26.279999999999987, -25.59999999999999, -24.99999999999999, -24.47999999999999, -24.039999999999992, -23.679999999999993, -23.399999999999995, -23.199999999999996, -23.08, -23.04, -23.080000000000002, -23.200000000000003, -23.400000000000006, -23.680000000000007, -24.04000000000001, -24.48000000000001, -25.000000000000014, -25.60000000000002, -26.280000000000022, -27.040000000000024, -27.880000000000027, -28.800000000000033, -29.800000000000033, -30.880000000000038, -32.04000000000004, -33.280000000000044, -34.60000000000005, -36.00000000000006, -37.48000000000006, -39.040000000000056, -40.680000000000064, -42.40000000000008, -44.200000000000074, -46.08000000000007 ], [ -46.16, -44.199999999999996, -42.31999999999999, -40.519999999999996, -38.79999999999999, -37.15999999999999, -35.59999999999999, -34.11999999999999, -32.719999999999985, -31.399999999999984, -30.159999999999986, -28.999999999999986, -27.919999999999987, -26.919999999999987, -25.999999999999986, -25.159999999999986, -24.399999999999984, -23.719999999999988, -23.119999999999987, -22.599999999999987, -22.15999999999999, -21.79999999999999, -21.519999999999992, -21.319999999999993, -21.199999999999996, -21.159999999999997, -21.2, -21.32, -21.520000000000003, -21.800000000000004, -22.160000000000007, -22.60000000000001, -23.12000000000001, -23.720000000000017, -24.40000000000002, -25.16000000000002, -26.000000000000025, -26.92000000000003, -27.92000000000003, -29.000000000000036, -30.16000000000004, -31.400000000000038, -32.72000000000005, -34.12000000000006, -35.60000000000006, -37.16000000000005, -38.80000000000007, -40.52000000000008, -42.32000000000008, -44.200000000000074 ], [ -44.36, -42.39999999999999, -40.519999999999996, -38.71999999999999, -36.999999999999986, -35.359999999999985, -33.79999999999999, -32.319999999999986, -30.919999999999987, -29.599999999999987, -28.359999999999985, -27.199999999999985, -26.119999999999983, -25.119999999999983, -24.199999999999985, -23.359999999999985, -22.599999999999987, -21.919999999999987, -21.319999999999986, -20.799999999999986, -20.35999999999999, -19.99999999999999, -19.71999999999999, -19.519999999999992, -19.399999999999995, -19.359999999999996, -19.4, -19.52, -19.720000000000002, -20.000000000000004, -20.360000000000007, -20.800000000000008, -21.32000000000001, -21.920000000000016, -22.600000000000016, -23.36000000000002, -24.200000000000024, -25.120000000000026, -26.120000000000033, -27.200000000000035, -28.36000000000004, -29.600000000000037, -30.920000000000044, -32.32000000000006, -33.800000000000054, -35.360000000000056, -37.000000000000064, -38.72000000000008, -40.520000000000074, -42.40000000000007 ], [ -42.63999999999999, -40.67999999999999, -38.79999999999999, -36.999999999999986, -35.27999999999999, -33.639999999999986, -32.079999999999984, -30.599999999999984, -29.19999999999998, -27.87999999999998, -26.639999999999983, -25.479999999999983, -24.399999999999984, -23.399999999999984, -22.479999999999983, -21.639999999999983, -20.87999999999998, -20.199999999999985, -19.599999999999984, -19.079999999999984, -18.639999999999986, -18.279999999999987, -17.99999999999999, -17.79999999999999, -17.679999999999993, -17.639999999999993, -17.679999999999996, -17.799999999999997, -18, -18.28, -18.640000000000004, -19.080000000000005, -19.60000000000001, -20.200000000000014, -20.880000000000017, -21.64000000000002, -22.48000000000002, -23.400000000000027, -24.400000000000027, -25.480000000000032, -26.640000000000036, -27.880000000000035, -29.200000000000045, -30.600000000000055, -32.080000000000055, -33.64000000000005, -35.28000000000006, -37.00000000000007, -38.80000000000007, -40.680000000000064 ], [ -40.99999999999999, -39.03999999999999, -37.15999999999999, -35.359999999999985, -33.639999999999986, -31.999999999999986, -30.439999999999984, -28.959999999999983, -27.55999999999998, -26.23999999999998, -24.999999999999982, -23.839999999999982, -22.759999999999984, -21.759999999999984, -20.839999999999982, -19.999999999999982, -19.23999999999998, -18.559999999999985, -17.959999999999983, -17.439999999999984, -16.999999999999986, -16.639999999999986, -16.35999999999999, -16.15999999999999, -16.039999999999992, -15.999999999999993, -16.039999999999996, -16.159999999999997, -16.36, -16.64, -17.000000000000004, -17.440000000000005, -17.960000000000008, -18.560000000000013, -19.240000000000016, -20.000000000000018, -20.84000000000002, -21.760000000000026, -22.760000000000026, -23.840000000000032, -25.000000000000036, -26.240000000000034, -27.560000000000045, -28.960000000000054, -30.440000000000055, -32.00000000000005, -33.64000000000006, -35.36000000000007, -37.16000000000007, -39.04000000000006 ], [ -39.43999999999999, -37.47999999999999, -35.59999999999999, -33.79999999999999, -32.079999999999984, -30.439999999999984, -28.879999999999985, -27.399999999999984, -25.999999999999982, -24.679999999999982, -23.439999999999984, -22.27999999999998, -21.19999999999998, -20.19999999999998, -19.27999999999998, -18.439999999999984, -17.679999999999982, -16.999999999999982, -16.399999999999984, -15.879999999999985, -15.439999999999985, -15.079999999999986, -14.799999999999988, -14.599999999999989, -14.479999999999992, -14.439999999999992, -14.479999999999995, -14.599999999999996, -14.799999999999999, -15.08, -15.440000000000003, -15.880000000000006, -16.40000000000001, -17.00000000000001, -17.680000000000014, -18.44000000000002, -19.280000000000022, -20.200000000000024, -21.200000000000028, -22.28000000000003, -23.440000000000033, -24.680000000000035, -26.000000000000043, -27.400000000000055, -28.880000000000052, -30.440000000000047, -32.08000000000006, -33.800000000000075, -35.60000000000007, -37.48000000000007 ], [ -37.959999999999994, -35.999999999999986, -34.11999999999999, -32.319999999999986, -30.599999999999984, -28.959999999999983, -27.399999999999984, -25.91999999999998, -24.519999999999982, -23.19999999999998, -21.95999999999998, -20.79999999999998, -19.719999999999978, -18.719999999999978, -17.79999999999998, -16.95999999999998, -16.19999999999998, -15.51999999999998, -14.91999999999998, -14.399999999999983, -13.959999999999983, -13.599999999999984, -13.319999999999986, -13.119999999999987, -12.99999999999999, -12.95999999999999, -12.999999999999993, -13.119999999999994, -13.319999999999997, -13.599999999999998, -13.96, -14.400000000000004, -14.920000000000005, -15.520000000000008, -16.20000000000001, -16.960000000000015, -17.80000000000002, -18.72000000000002, -19.720000000000027, -20.80000000000003, -21.960000000000033, -23.20000000000003, -24.52000000000004, -25.92000000000005, -27.40000000000005, -28.960000000000047, -30.60000000000006, -32.32000000000007, -34.12000000000007, -36.000000000000064 ], [ -36.55999999999999, -34.59999999999999, -32.719999999999985, -30.919999999999987, -29.19999999999998, -27.55999999999998, -25.999999999999982, -24.519999999999982, -23.11999999999998, -21.79999999999998, -20.55999999999998, -19.399999999999977, -18.31999999999998, -17.31999999999998, -16.399999999999977, -15.55999999999998, -14.79999999999998, -14.11999999999998, -13.519999999999982, -12.999999999999982, -12.559999999999983, -12.199999999999983, -11.919999999999986, -11.719999999999986, -11.599999999999989, -11.55999999999999, -11.599999999999993, -11.719999999999994, -11.919999999999996, -12.199999999999998, -12.56, -13.000000000000004, -13.520000000000007, -14.120000000000008, -14.800000000000011, -15.560000000000015, -16.40000000000002, -17.32000000000002, -18.320000000000025, -19.400000000000027, -20.56000000000003, -21.800000000000033, -23.12000000000004, -24.520000000000053, -26.00000000000005, -27.560000000000045, -29.20000000000006, -30.920000000000073, -32.72000000000007, -34.600000000000065 ], [ -35.23999999999999, -33.27999999999999, -31.399999999999984, -29.599999999999987, -27.87999999999998, -26.23999999999998, -24.679999999999982, -23.19999999999998, -21.79999999999998, -20.47999999999998, -19.23999999999998, -18.079999999999977, -16.99999999999998, -15.999999999999979, -15.079999999999979, -14.239999999999979, -13.479999999999979, -12.79999999999998, -12.199999999999982, -11.679999999999982, -11.239999999999982, -10.879999999999983, -10.599999999999985, -10.399999999999986, -10.279999999999989, -10.23999999999999, -10.279999999999992, -10.399999999999993, -10.599999999999996, -10.879999999999997, -11.24, -11.680000000000003, -12.200000000000006, -12.800000000000008, -13.480000000000011, -14.240000000000014, -15.080000000000018, -16.00000000000002, -17.000000000000025, -18.080000000000027, -19.24000000000003, -20.480000000000032, -21.80000000000004, -23.200000000000053, -24.68000000000005, -26.240000000000045, -27.88000000000006, -29.600000000000072, -31.40000000000007, -33.280000000000065 ], [ -33.999999999999986, -32.03999999999999, -30.159999999999986, -28.359999999999985, -26.639999999999983, -24.999999999999982, -23.439999999999984, -21.95999999999998, -20.55999999999998, -19.23999999999998, -17.99999999999998, -16.83999999999998, -15.759999999999978, -14.759999999999978, -13.839999999999979, -12.999999999999979, -12.239999999999979, -11.55999999999998, -10.95999999999998, -10.439999999999982, -9.999999999999982, -9.639999999999983, -9.359999999999985, -9.159999999999986, -9.039999999999988, -8.99999999999999, -9.039999999999992, -9.159999999999993, -9.359999999999996, -9.639999999999997, -10, -10.440000000000003, -10.960000000000004, -11.560000000000008, -12.24000000000001, -13.000000000000014, -13.840000000000018, -14.760000000000021, -15.760000000000025, -16.84000000000003, -18.000000000000032, -19.24000000000003, -20.560000000000038, -21.96000000000005, -23.440000000000047, -25.000000000000046, -26.640000000000057, -28.36000000000007, -30.160000000000068, -32.04000000000006 ], [ -32.83999999999999, -30.87999999999999, -28.999999999999986, -27.199999999999985, -25.479999999999983, -23.839999999999982, -22.27999999999998, -20.79999999999998, -19.399999999999977, -18.079999999999977, -16.83999999999998, -15.679999999999978, -14.599999999999978, -13.599999999999978, -12.679999999999978, -11.839999999999979, -11.079999999999979, -10.399999999999979, -9.79999999999998, -9.279999999999982, -8.839999999999982, -8.479999999999983, -8.199999999999985, -7.999999999999986, -7.8799999999999875, -7.839999999999989, -7.879999999999991, -7.999999999999993, -8.199999999999996, -8.479999999999997, -8.84, -9.280000000000003, -9.800000000000004, -10.400000000000007, -11.08000000000001, -11.840000000000014, -12.680000000000017, -13.600000000000021, -14.600000000000025, -15.680000000000028, -16.840000000000032, -18.08000000000003, -19.40000000000004, -20.80000000000005, -22.28000000000005, -23.840000000000046, -25.480000000000057, -27.20000000000007, -29.000000000000068, -30.880000000000063 ], [ -31.75999999999999, -29.79999999999999, -27.919999999999987, -26.119999999999983, -24.399999999999984, -22.759999999999984, -21.19999999999998, -19.719999999999978, -18.31999999999998, -16.99999999999998, -15.759999999999978, -14.599999999999978, -13.519999999999978, -12.519999999999978, -11.599999999999978, -10.759999999999978, -9.999999999999979, -9.319999999999979, -8.719999999999981, -8.199999999999982, -7.759999999999982, -7.3999999999999835, -7.119999999999984, -6.919999999999986, -6.799999999999987, -6.759999999999989, -6.799999999999991, -6.919999999999993, -7.119999999999995, -7.399999999999998, -7.76, -8.200000000000003, -8.720000000000006, -9.320000000000007, -10.00000000000001, -10.760000000000014, -11.600000000000017, -12.520000000000021, -13.520000000000024, -14.600000000000028, -15.760000000000032, -17.00000000000003, -18.32000000000004, -19.72000000000005, -21.20000000000005, -22.760000000000048, -24.400000000000055, -26.12000000000007, -27.920000000000066, -29.80000000000006 ], [ -30.75999999999999, -28.79999999999999, -26.919999999999987, -25.119999999999983, -23.399999999999984, -21.759999999999984, -20.19999999999998, -18.719999999999978, -17.31999999999998, -15.999999999999979, -14.759999999999978, -13.599999999999978, -12.519999999999978, -11.519999999999978, -10.599999999999978, -9.759999999999978, -8.999999999999979, -8.319999999999979, -7.71999999999998, -7.1999999999999815, -6.759999999999982, -6.3999999999999835, -6.119999999999984, -5.919999999999986, -5.799999999999987, -5.759999999999989, -5.799999999999991, -5.919999999999993, -6.119999999999995, -6.399999999999998, -6.76, -7.200000000000003, -7.720000000000005, -8.320000000000007, -9.00000000000001, -9.760000000000014, -10.600000000000017, -11.520000000000021, -12.520000000000024, -13.600000000000028, -14.760000000000032, -16.00000000000003, -17.32000000000004, -18.72000000000005, -20.20000000000005, -21.760000000000048, -23.400000000000055, -25.12000000000007, -26.920000000000066, -28.80000000000006 ], [ -29.83999999999999, -27.87999999999999, -25.999999999999986, -24.199999999999985, -22.479999999999983, -20.839999999999982, -19.27999999999998, -17.79999999999998, -16.399999999999977, -15.079999999999979, -13.839999999999979, -12.679999999999978, -11.599999999999978, -10.599999999999978, -9.679999999999978, -8.839999999999979, -8.079999999999979, -7.399999999999979, -6.79999999999998, -6.279999999999982, -5.839999999999982, -5.4799999999999836, -5.199999999999984, -4.999999999999986, -4.8799999999999875, -4.839999999999989, -4.879999999999991, -4.999999999999993, -5.199999999999995, -5.479999999999998, -5.84, -6.280000000000003, -6.800000000000005, -7.4000000000000075, -8.08000000000001, -8.840000000000014, -9.680000000000017, -10.600000000000021, -11.600000000000025, -12.680000000000028, -13.840000000000032, -15.08000000000003, -16.40000000000004, -17.80000000000005, -19.28000000000005, -20.840000000000046, -22.480000000000057, -24.20000000000007, -26.000000000000068, -27.880000000000063 ], [ -28.99999999999999, -27.03999999999999, -25.159999999999986, -23.359999999999985, -21.639999999999983, -19.999999999999982, -18.439999999999984, -16.95999999999998, -15.55999999999998, -14.239999999999979, -12.999999999999979, -11.839999999999979, -10.759999999999978, -9.759999999999978, -8.839999999999979, -7.999999999999979, -7.239999999999979, -6.559999999999979, -5.95999999999998, -5.439999999999982, -4.999999999999982, -4.639999999999984, -4.359999999999984, -4.159999999999986, -4.039999999999988, -3.9999999999999893, -4.039999999999991, -4.159999999999993, -4.359999999999995, -4.639999999999998, -5, -5.440000000000003, -5.960000000000005, -6.560000000000008, -7.240000000000011, -8.000000000000014, -8.840000000000018, -9.760000000000021, -10.760000000000025, -11.840000000000028, -13.000000000000032, -14.24000000000003, -15.56000000000004, -16.96000000000005, -18.440000000000047, -20.000000000000046, -21.640000000000057, -23.36000000000007, -25.160000000000068, -27.040000000000063 ], [ -28.239999999999988, -26.279999999999987, -24.399999999999984, -22.599999999999987, -20.87999999999998, -19.23999999999998, -17.679999999999982, -16.19999999999998, -14.79999999999998, -13.479999999999979, -12.239999999999979, -11.079999999999979, -9.999999999999979, -8.999999999999979, -8.079999999999979, -7.239999999999979, -6.479999999999979, -5.799999999999979, -5.199999999999981, -4.679999999999982, -4.2399999999999824, -3.8799999999999835, -3.599999999999985, -3.399999999999986, -3.279999999999988, -3.2399999999999896, -3.2799999999999914, -3.3999999999999932, -3.5999999999999956, -3.8799999999999977, -4.24, -4.680000000000003, -5.2000000000000055, -5.800000000000008, -6.480000000000011, -7.240000000000014, -8.080000000000018, -9.000000000000021, -10.000000000000025, -11.080000000000028, -12.240000000000032, -13.48000000000003, -14.80000000000004, -16.200000000000053, -17.68000000000005, -19.240000000000045, -20.88000000000006, -22.600000000000072, -24.40000000000007, -26.280000000000065 ], [ -27.55999999999999, -25.59999999999999, -23.719999999999988, -21.919999999999987, -20.199999999999985, -18.559999999999985, -16.999999999999982, -15.51999999999998, -14.11999999999998, -12.79999999999998, -11.55999999999998, -10.399999999999979, -9.319999999999979, -8.319999999999979, -7.399999999999979, -6.559999999999979, -5.799999999999979, -5.119999999999981, -4.519999999999982, -3.9999999999999822, -3.559999999999983, -3.199999999999984, -2.9199999999999857, -2.719999999999987, -2.5999999999999885, -2.5599999999999903, -2.599999999999992, -2.719999999999994, -2.9199999999999964, -3.1999999999999984, -3.560000000000001, -4.0000000000000036, -4.520000000000007, -5.120000000000009, -5.800000000000011, -6.560000000000015, -7.400000000000018, -8.320000000000022, -9.320000000000025, -10.400000000000029, -11.560000000000032, -12.800000000000031, -14.12000000000004, -15.520000000000051, -17.00000000000005, -18.56000000000005, -20.20000000000006, -21.920000000000073, -23.72000000000007, -25.600000000000065 ], [ -26.95999999999999, -24.99999999999999, -23.119999999999987, -21.319999999999986, -19.599999999999984, -17.959999999999983, -16.399999999999984, -14.91999999999998, -13.519999999999982, -12.199999999999982, -10.95999999999998, -9.79999999999998, -8.719999999999981, -7.71999999999998, -6.79999999999998, -5.95999999999998, -5.199999999999981, -4.519999999999982, -3.919999999999982, -3.399999999999983, -2.959999999999984, -2.599999999999985, -2.3199999999999865, -2.1199999999999877, -1.9999999999999893, -1.959999999999991, -1.999999999999993, -2.1199999999999948, -2.319999999999997, -2.599999999999999, -2.9600000000000017, -3.4000000000000044, -3.920000000000007, -4.52000000000001, -5.200000000000013, -5.960000000000016, -6.800000000000019, -7.720000000000023, -8.720000000000027, -9.80000000000003, -10.960000000000033, -12.200000000000031, -13.520000000000042, -14.920000000000051, -16.400000000000052, -17.960000000000047, -19.60000000000006, -21.32000000000007, -23.12000000000007, -25.000000000000064 ], [ -26.43999999999999, -24.47999999999999, -22.599999999999987, -20.799999999999986, -19.079999999999984, -17.439999999999984, -15.879999999999985, -14.399999999999983, -12.999999999999982, -11.679999999999982, -10.439999999999982, -9.279999999999982, -8.199999999999982, -7.1999999999999815, -6.279999999999982, -5.439999999999982, -4.679999999999982, -3.9999999999999822, -3.399999999999983, -2.879999999999984, -2.439999999999985, -2.079999999999986, -1.7999999999999874, -1.5999999999999888, -1.4799999999999902, -1.439999999999992, -1.4799999999999938, -1.5999999999999959, -1.799999999999998, -2.08, -2.4400000000000026, -2.8800000000000052, -3.400000000000008, -4.000000000000011, -4.680000000000014, -5.440000000000017, -6.280000000000021, -7.200000000000024, -8.200000000000028, -9.280000000000031, -10.440000000000035, -11.680000000000033, -13.000000000000043, -14.400000000000054, -15.880000000000052, -17.440000000000047, -19.08000000000006, -20.80000000000007, -22.60000000000007, -24.480000000000064 ], [ -25.999999999999993, -24.039999999999992, -22.15999999999999, -20.35999999999999, -18.639999999999986, -16.999999999999986, -15.439999999999985, -13.959999999999983, -12.559999999999983, -11.239999999999982, -9.999999999999982, -8.839999999999982, -7.759999999999982, -6.759999999999982, -5.839999999999982, -4.999999999999982, -4.2399999999999824, -3.559999999999983, -2.959999999999984, -2.439999999999985, -1.9999999999999858, -1.6399999999999868, -1.3599999999999883, -1.1599999999999897, -1.0399999999999912, -0.9999999999999929, -1.0399999999999947, -1.1599999999999968, -1.359999999999999, -1.640000000000001, -2.0000000000000036, -2.440000000000006, -2.960000000000009, -3.5600000000000116, -4.240000000000014, -5.000000000000018, -5.840000000000021, -6.760000000000025, -7.760000000000028, -8.840000000000032, -10.000000000000036, -11.240000000000034, -12.560000000000043, -13.960000000000054, -15.440000000000053, -17.00000000000005, -18.64000000000006, -20.360000000000074, -22.16000000000007, -24.040000000000067 ], [ -25.639999999999993, -23.679999999999993, -21.79999999999999, -19.99999999999999, -18.279999999999987, -16.639999999999986, -15.079999999999986, -13.599999999999984, -12.199999999999983, -10.879999999999983, -9.639999999999983, -8.479999999999983, -7.3999999999999835, -6.3999999999999835, -5.4799999999999836, -4.639999999999984, -3.8799999999999835, -3.199999999999984, -2.599999999999985, -2.079999999999986, -1.6399999999999868, -1.279999999999988, -0.9999999999999893, -0.7999999999999907, -0.6799999999999923, -0.639999999999994, -0.6799999999999958, -0.7999999999999978, -1, -1.2800000000000022, -1.6400000000000046, -2.080000000000007, -2.60000000000001, -3.2000000000000126, -3.8800000000000154, -4.640000000000019, -5.480000000000023, -6.400000000000026, -7.40000000000003, -8.480000000000032, -9.640000000000036, -10.880000000000035, -12.200000000000044, -13.600000000000055, -15.080000000000053, -16.64000000000005, -18.28000000000006, -20.000000000000075, -21.80000000000007, -23.680000000000067 ], [ -25.359999999999996, -23.399999999999995, -21.519999999999992, -19.71999999999999, -17.99999999999999, -16.35999999999999, -14.799999999999988, -13.319999999999986, -11.919999999999986, -10.599999999999985, -9.359999999999985, -8.199999999999985, -7.119999999999984, -6.119999999999984, -5.199999999999984, -4.359999999999984, -3.599999999999985, -2.9199999999999857, -2.3199999999999865, -1.7999999999999874, -1.3599999999999883, -0.9999999999999893, -0.7199999999999906, -0.519999999999992, -0.39999999999999364, -0.3599999999999953, -0.3999999999999972, -0.5199999999999991, -0.7200000000000013, -1.0000000000000036, -1.360000000000006, -1.8000000000000087, -2.3200000000000114, -2.920000000000014, -3.600000000000017, -4.36000000000002, -5.200000000000023, -6.120000000000027, -7.12000000000003, -8.200000000000035, -9.360000000000039, -10.600000000000037, -11.920000000000046, -13.320000000000057, -14.800000000000056, -16.360000000000053, -18.000000000000064, -19.720000000000077, -21.520000000000074, -23.40000000000007 ], [ -25.159999999999997, -23.199999999999996, -21.319999999999993, -19.519999999999992, -17.79999999999999, -16.15999999999999, -14.599999999999989, -13.119999999999987, -11.719999999999986, -10.399999999999986, -9.159999999999986, -7.999999999999986, -6.919999999999986, -5.919999999999986, -4.999999999999986, -4.159999999999986, -3.399999999999986, -2.719999999999987, -2.1199999999999877, -1.5999999999999888, -1.1599999999999897, -0.7999999999999907, -0.519999999999992, -0.31999999999999346, -0.19999999999999502, -0.15999999999999673, -0.19999999999999857, -0.32000000000000056, -0.5200000000000027, -0.8000000000000049, -1.1600000000000075, -1.60000000000001, -2.1200000000000125, -2.7200000000000153, -3.400000000000018, -4.1600000000000215, -5.000000000000025, -5.920000000000028, -6.920000000000032, -8.000000000000036, -9.16000000000004, -10.400000000000038, -11.720000000000047, -13.120000000000058, -14.600000000000056, -16.160000000000053, -17.800000000000065, -19.520000000000078, -21.320000000000075, -23.20000000000007 ], [ -25.04, -23.08, -21.199999999999996, -19.399999999999995, -17.679999999999993, -16.039999999999992, -14.479999999999992, -12.99999999999999, -11.599999999999989, -10.279999999999989, -9.039999999999988, -7.8799999999999875, -6.799999999999987, -5.799999999999987, -4.8799999999999875, -4.039999999999988, -3.279999999999988, -2.5999999999999885, -1.9999999999999893, -1.4799999999999902, -1.0399999999999912, -0.6799999999999923, -0.39999999999999364, -0.19999999999999502, -0.07999999999999659, -0.039999999999998294, -0.08000000000000014, -0.20000000000000212, -0.4000000000000043, -0.6800000000000065, -1.040000000000009, -1.4800000000000115, -2.000000000000014, -2.600000000000017, -3.28000000000002, -4.040000000000023, -4.8800000000000265, -5.80000000000003, -6.800000000000034, -7.880000000000037, -9.040000000000042, -10.28000000000004, -11.60000000000005, -13.00000000000006, -14.480000000000059, -16.040000000000056, -17.680000000000067, -19.40000000000008, -21.200000000000077, -23.080000000000073 ], [ -25, -23.04, -21.159999999999997, -19.359999999999996, -17.639999999999993, -15.999999999999993, -14.439999999999992, -12.95999999999999, -11.55999999999999, -10.23999999999999, -8.99999999999999, -7.839999999999989, -6.759999999999989, -5.759999999999989, -4.839999999999989, -3.9999999999999893, -3.2399999999999896, -2.5599999999999903, -1.959999999999991, -1.439999999999992, -0.9999999999999929, -0.639999999999994, -0.3599999999999953, -0.15999999999999673, -0.039999999999998294, -3.944304526105059e-29, -0.04000000000000185, -0.16000000000000383, -0.360000000000006, -0.6400000000000082, -1.0000000000000107, -1.4400000000000133, -1.960000000000016, -2.5600000000000187, -3.2400000000000215, -4.000000000000025, -4.840000000000028, -5.760000000000032, -6.760000000000035, -7.840000000000039, -9.000000000000043, -10.240000000000041, -11.56000000000005, -12.960000000000061, -14.44000000000006, -16.000000000000057, -17.640000000000068, -19.36000000000008, -21.16000000000008, -23.040000000000074 ], [ -25.040000000000003, -23.080000000000002, -21.2, -19.4, -17.679999999999996, -16.039999999999996, -14.479999999999995, -12.999999999999993, -11.599999999999993, -10.279999999999992, -9.039999999999992, -7.879999999999991, -6.799999999999991, -5.799999999999991, -4.879999999999991, -4.039999999999991, -3.2799999999999914, -2.599999999999992, -1.999999999999993, -1.4799999999999938, -1.0399999999999947, -0.6799999999999958, -0.3999999999999972, -0.19999999999999857, -0.08000000000000014, -0.04000000000000185, -0.0800000000000037, -0.20000000000000567, -0.40000000000000785, -0.68000000000001, -1.0400000000000125, -1.480000000000015, -2.0000000000000178, -2.6000000000000205, -3.2800000000000233, -4.040000000000027, -4.88000000000003, -5.800000000000034, -6.800000000000037, -7.880000000000041, -9.040000000000045, -10.280000000000044, -11.600000000000053, -13.000000000000064, -14.480000000000063, -16.04000000000006, -17.68000000000007, -19.400000000000084, -21.20000000000008, -23.080000000000076 ], [ -25.160000000000004, -23.200000000000003, -21.32, -19.52, -17.799999999999997, -16.159999999999997, -14.599999999999996, -13.119999999999994, -11.719999999999994, -10.399999999999993, -9.159999999999993, -7.999999999999993, -6.919999999999993, -5.919999999999993, -4.999999999999993, -4.159999999999993, -3.3999999999999932, -2.719999999999994, -2.1199999999999948, -1.5999999999999959, -1.1599999999999968, -0.7999999999999978, -0.5199999999999991, -0.32000000000000056, -0.20000000000000212, -0.16000000000000383, -0.20000000000000567, -0.32000000000000767, -0.5200000000000098, -0.800000000000012, -1.1600000000000146, -1.6000000000000172, -2.1200000000000196, -2.7200000000000224, -3.4000000000000252, -4.160000000000029, -5.000000000000032, -5.9200000000000355, -6.920000000000039, -8.000000000000043, -9.160000000000046, -10.400000000000045, -11.720000000000054, -13.120000000000065, -14.600000000000064, -16.16000000000006, -17.80000000000007, -19.520000000000085, -21.320000000000082, -23.200000000000077 ], [ -25.360000000000007, -23.400000000000006, -21.520000000000003, -19.720000000000002, -18, -16.36, -14.799999999999999, -13.319999999999997, -11.919999999999996, -10.599999999999996, -9.359999999999996, -8.199999999999996, -7.119999999999995, -6.119999999999995, -5.199999999999995, -4.359999999999995, -3.5999999999999956, -2.9199999999999964, -2.319999999999997, -1.799999999999998, -1.359999999999999, -1, -0.7200000000000013, -0.5200000000000027, -0.4000000000000043, -0.360000000000006, -0.40000000000000785, -0.5200000000000098, -0.720000000000012, -1.0000000000000142, -1.3600000000000168, -1.8000000000000194, -2.320000000000022, -2.920000000000025, -3.6000000000000276, -4.3600000000000305, -5.200000000000034, -6.120000000000037, -7.120000000000041, -8.200000000000045, -9.36000000000005, -10.600000000000048, -11.920000000000057, -13.320000000000068, -14.800000000000066, -16.360000000000063, -18.000000000000075, -19.720000000000088, -21.520000000000085, -23.40000000000008 ], [ -25.640000000000008, -23.680000000000007, -21.800000000000004, -20.000000000000004, -18.28, -16.64, -15.08, -13.599999999999998, -12.199999999999998, -10.879999999999997, -9.639999999999997, -8.479999999999997, -7.399999999999998, -6.399999999999998, -5.479999999999998, -4.639999999999998, -3.8799999999999977, -3.1999999999999984, -2.599999999999999, -2.08, -1.640000000000001, -1.2800000000000022, -1.0000000000000036, -0.8000000000000049, -0.6800000000000065, -0.6400000000000082, -0.68000000000001, -0.800000000000012, -1.0000000000000142, -1.2800000000000165, -1.6400000000000188, -2.0800000000000214, -2.600000000000024, -3.200000000000027, -3.8800000000000296, -4.640000000000033, -5.480000000000037, -6.40000000000004, -7.400000000000044, -8.480000000000047, -9.64000000000005, -10.880000000000049, -12.200000000000058, -13.600000000000069, -15.080000000000068, -16.640000000000065, -18.280000000000076, -20.00000000000009, -21.800000000000086, -23.68000000000008 ], [ -26.00000000000001, -24.04000000000001, -22.160000000000007, -20.360000000000007, -18.640000000000004, -17.000000000000004, -15.440000000000003, -13.96, -12.56, -11.24, -10, -8.84, -7.76, -6.76, -5.84, -5, -4.24, -3.560000000000001, -2.9600000000000017, -2.4400000000000026, -2.0000000000000036, -1.6400000000000046, -1.360000000000006, -1.1600000000000075, -1.040000000000009, -1.0000000000000107, -1.0400000000000125, -1.1600000000000146, -1.3600000000000168, -1.6400000000000188, -2.0000000000000213, -2.440000000000024, -2.9600000000000266, -3.5600000000000294, -4.240000000000032, -5.0000000000000355, -5.840000000000039, -6.760000000000042, -7.760000000000046, -8.84000000000005, -10.000000000000053, -11.240000000000052, -12.560000000000061, -13.960000000000072, -15.44000000000007, -17.000000000000068, -18.64000000000008, -20.360000000000092, -22.16000000000009, -24.040000000000084 ], [ -26.440000000000012, -24.48000000000001, -22.60000000000001, -20.800000000000008, -19.080000000000005, -17.440000000000005, -15.880000000000006, -14.400000000000004, -13.000000000000004, -11.680000000000003, -10.440000000000003, -9.280000000000003, -8.200000000000003, -7.200000000000003, -6.280000000000003, -5.440000000000003, -4.680000000000003, -4.0000000000000036, -3.4000000000000044, -2.8800000000000052, -2.440000000000006, -2.080000000000007, -1.8000000000000087, -1.60000000000001, -1.4800000000000115, -1.4400000000000133, -1.480000000000015, -1.6000000000000172, -1.8000000000000194, -2.0800000000000214, -2.440000000000024, -2.8800000000000265, -3.400000000000029, -4.000000000000032, -4.680000000000035, -5.440000000000039, -6.280000000000042, -7.2000000000000455, -8.200000000000049, -9.280000000000053, -10.440000000000056, -11.680000000000055, -13.000000000000064, -14.400000000000075, -15.880000000000074, -17.44000000000007, -19.08000000000008, -20.800000000000093, -22.60000000000009, -24.480000000000086 ], [ -26.960000000000015, -25.000000000000014, -23.12000000000001, -21.32000000000001, -19.60000000000001, -17.960000000000008, -16.40000000000001, -14.920000000000005, -13.520000000000007, -12.200000000000006, -10.960000000000004, -9.800000000000004, -8.720000000000006, -7.720000000000005, -6.800000000000005, -5.960000000000005, -5.2000000000000055, -4.520000000000007, -3.920000000000007, -3.400000000000008, -2.960000000000009, -2.60000000000001, -2.3200000000000114, -2.1200000000000125, -2.000000000000014, -1.960000000000016, -2.0000000000000178, -2.1200000000000196, -2.320000000000022, -2.600000000000024, -2.9600000000000266, -3.400000000000029, -3.920000000000032, -4.520000000000035, -5.2000000000000375, -5.960000000000041, -6.800000000000044, -7.720000000000048, -8.720000000000052, -9.800000000000054, -10.960000000000058, -12.200000000000056, -13.520000000000067, -14.920000000000076, -16.400000000000077, -17.960000000000072, -19.600000000000083, -21.320000000000096, -23.120000000000093, -25.00000000000009 ], [ -27.56000000000002, -25.60000000000002, -23.720000000000017, -21.920000000000016, -20.200000000000014, -18.560000000000013, -17.00000000000001, -15.520000000000008, -14.120000000000008, -12.800000000000008, -11.560000000000008, -10.400000000000007, -9.320000000000007, -8.320000000000007, -7.4000000000000075, -6.560000000000008, -5.800000000000008, -5.120000000000009, -4.52000000000001, -4.000000000000011, -3.5600000000000116, -3.2000000000000126, -2.920000000000014, -2.7200000000000153, -2.600000000000017, -2.5600000000000187, -2.6000000000000205, -2.7200000000000224, -2.920000000000025, -3.200000000000027, -3.5600000000000294, -4.000000000000032, -4.520000000000035, -5.120000000000037, -5.80000000000004, -6.560000000000043, -7.4000000000000465, -8.32000000000005, -9.320000000000054, -10.400000000000057, -11.560000000000061, -12.80000000000006, -14.120000000000068, -15.52000000000008, -17.000000000000078, -18.560000000000077, -20.200000000000088, -21.9200000000001, -23.7200000000001, -25.600000000000094 ], [ -28.240000000000023, -26.280000000000022, -24.40000000000002, -22.600000000000016, -20.880000000000017, -19.240000000000016, -17.680000000000014, -16.20000000000001, -14.800000000000011, -13.480000000000011, -12.24000000000001, -11.08000000000001, -10.00000000000001, -9.00000000000001, -8.08000000000001, -7.240000000000011, -6.480000000000011, -5.800000000000011, -5.200000000000013, -4.680000000000014, -4.240000000000014, -3.8800000000000154, -3.600000000000017, -3.400000000000018, -3.28000000000002, -3.2400000000000215, -3.2800000000000233, -3.4000000000000252, -3.6000000000000276, -3.8800000000000296, -4.240000000000032, -4.680000000000035, -5.2000000000000375, -5.80000000000004, -6.480000000000043, -7.240000000000046, -8.08000000000005, -9.000000000000053, -10.000000000000057, -11.08000000000006, -12.240000000000064, -13.480000000000063, -14.800000000000072, -16.20000000000008, -17.68000000000008, -19.24000000000008, -20.880000000000088, -22.6000000000001, -24.400000000000098, -26.280000000000094 ], [ -29.000000000000025, -27.040000000000024, -25.16000000000002, -23.36000000000002, -21.64000000000002, -20.000000000000018, -18.44000000000002, -16.960000000000015, -15.560000000000015, -14.240000000000014, -13.000000000000014, -11.840000000000014, -10.760000000000014, -9.760000000000014, -8.840000000000014, -8.000000000000014, -7.240000000000014, -6.560000000000015, -5.960000000000016, -5.440000000000017, -5.000000000000018, -4.640000000000019, -4.36000000000002, -4.1600000000000215, -4.040000000000023, -4.000000000000025, -4.040000000000027, -4.160000000000029, -4.3600000000000305, -4.640000000000033, -5.0000000000000355, -5.440000000000039, -5.960000000000041, -6.560000000000043, -7.240000000000046, -8.00000000000005, -8.840000000000053, -9.760000000000057, -10.76000000000006, -11.840000000000064, -13.000000000000068, -14.240000000000066, -15.560000000000075, -16.960000000000086, -18.440000000000083, -20.00000000000008, -21.640000000000093, -23.360000000000106, -25.160000000000103, -27.0400000000001 ], [ -29.84000000000003, -27.880000000000027, -26.000000000000025, -24.200000000000024, -22.48000000000002, -20.84000000000002, -19.280000000000022, -17.80000000000002, -16.40000000000002, -15.080000000000018, -13.840000000000018, -12.680000000000017, -11.600000000000017, -10.600000000000017, -9.680000000000017, -8.840000000000018, -8.080000000000018, -7.400000000000018, -6.800000000000019, -6.280000000000021, -5.840000000000021, -5.480000000000023, -5.200000000000023, -5.000000000000025, -4.8800000000000265, -4.840000000000028, -4.88000000000003, -5.000000000000032, -5.200000000000034, -5.480000000000037, -5.840000000000039, -6.280000000000042, -6.800000000000044, -7.4000000000000465, -8.08000000000005, -8.840000000000053, -9.680000000000057, -10.60000000000006, -11.600000000000064, -12.680000000000067, -13.840000000000071, -15.08000000000007, -16.400000000000077, -17.80000000000009, -19.280000000000086, -20.840000000000085, -22.480000000000096, -24.20000000000011, -26.000000000000107, -27.880000000000102 ], [ -30.760000000000034, -28.800000000000033, -26.92000000000003, -25.120000000000026, -23.400000000000027, -21.760000000000026, -20.200000000000024, -18.72000000000002, -17.32000000000002, -16.00000000000002, -14.760000000000021, -13.600000000000021, -12.520000000000021, -11.520000000000021, -10.600000000000021, -9.760000000000021, -9.000000000000021, -8.320000000000022, -7.720000000000023, -7.200000000000024, -6.760000000000025, -6.400000000000026, -6.120000000000027, -5.920000000000028, -5.80000000000003, -5.760000000000032, -5.800000000000034, -5.9200000000000355, -6.120000000000037, -6.40000000000004, -6.760000000000042, -7.2000000000000455, -7.720000000000048, -8.32000000000005, -9.000000000000053, -9.760000000000057, -10.60000000000006, -11.520000000000064, -12.520000000000067, -13.60000000000007, -14.760000000000074, -16.00000000000007, -17.320000000000082, -18.72000000000009, -20.20000000000009, -21.76000000000009, -23.400000000000098, -25.12000000000011, -26.92000000000011, -28.800000000000104 ], [ -31.760000000000034, -29.800000000000033, -27.92000000000003, -26.120000000000033, -24.400000000000027, -22.760000000000026, -21.200000000000028, -19.720000000000027, -18.320000000000025, -17.000000000000025, -15.760000000000025, -14.600000000000025, -13.520000000000024, -12.520000000000024, -11.600000000000025, -10.760000000000025, -10.000000000000025, -9.320000000000025, -8.720000000000027, -8.200000000000028, -7.760000000000028, -7.40000000000003, -7.12000000000003, -6.920000000000032, -6.800000000000034, -6.760000000000035, -6.800000000000037, -6.920000000000039, -7.120000000000041, -7.400000000000044, -7.760000000000046, -8.200000000000049, -8.720000000000052, -9.320000000000054, -10.000000000000057, -10.76000000000006, -11.600000000000064, -12.520000000000067, -13.52000000000007, -14.600000000000074, -15.760000000000078, -17.000000000000078, -18.320000000000086, -19.7200000000001, -21.200000000000095, -22.76000000000009, -24.400000000000105, -26.12000000000012, -27.920000000000115, -29.80000000000011 ], [ -32.84000000000004, -30.880000000000038, -29.000000000000036, -27.200000000000035, -25.480000000000032, -23.840000000000032, -22.28000000000003, -20.80000000000003, -19.400000000000027, -18.080000000000027, -16.84000000000003, -15.680000000000028, -14.600000000000028, -13.600000000000028, -12.680000000000028, -11.840000000000028, -11.080000000000028, -10.400000000000029, -9.80000000000003, -9.280000000000031, -8.840000000000032, -8.480000000000032, -8.200000000000035, -8.000000000000036, -7.880000000000037, -7.840000000000039, -7.880000000000041, -8.000000000000043, -8.200000000000045, -8.480000000000047, -8.84000000000005, -9.280000000000053, -9.800000000000054, -10.400000000000057, -11.08000000000006, -11.840000000000064, -12.680000000000067, -13.60000000000007, -14.600000000000074, -15.680000000000078, -16.84000000000008, -18.08000000000008, -19.40000000000009, -20.8000000000001, -22.2800000000001, -23.840000000000096, -25.480000000000107, -27.20000000000012, -29.000000000000117, -30.880000000000113 ], [ -34.00000000000004, -32.04000000000004, -30.16000000000004, -28.36000000000004, -26.640000000000036, -25.000000000000036, -23.440000000000033, -21.960000000000033, -20.56000000000003, -19.24000000000003, -18.000000000000032, -16.840000000000032, -15.760000000000032, -14.760000000000032, -13.840000000000032, -13.000000000000032, -12.240000000000032, -11.560000000000032, -10.960000000000033, -10.440000000000035, -10.000000000000036, -9.640000000000036, -9.360000000000039, -9.16000000000004, -9.040000000000042, -9.000000000000043, -9.040000000000045, -9.160000000000046, -9.36000000000005, -9.64000000000005, -10.000000000000053, -10.440000000000056, -10.960000000000058, -11.560000000000061, -12.240000000000064, -13.000000000000068, -13.840000000000071, -14.760000000000074, -15.760000000000078, -16.84000000000008, -18.000000000000085, -19.240000000000084, -20.560000000000095, -21.960000000000104, -23.440000000000104, -25.0000000000001, -26.64000000000011, -28.360000000000124, -30.16000000000012, -32.04000000000012 ], [ -35.24000000000004, -33.280000000000044, -31.400000000000038, -29.600000000000037, -27.880000000000035, -26.240000000000034, -24.680000000000035, -23.20000000000003, -21.800000000000033, -20.480000000000032, -19.24000000000003, -18.08000000000003, -17.00000000000003, -16.00000000000003, -15.08000000000003, -14.24000000000003, -13.48000000000003, -12.800000000000031, -12.200000000000031, -11.680000000000033, -11.240000000000034, -10.880000000000035, -10.600000000000037, -10.400000000000038, -10.28000000000004, -10.240000000000041, -10.280000000000044, -10.400000000000045, -10.600000000000048, -10.880000000000049, -11.240000000000052, -11.680000000000055, -12.200000000000056, -12.80000000000006, -13.480000000000063, -14.240000000000066, -15.08000000000007, -16.00000000000007, -17.000000000000078, -18.08000000000008, -19.240000000000084, -20.480000000000082, -21.80000000000009, -23.200000000000102, -24.6800000000001, -26.240000000000098, -27.88000000000011, -29.600000000000122, -31.40000000000012, -33.280000000000115 ], [ -36.56000000000005, -34.60000000000005, -32.72000000000005, -30.920000000000044, -29.200000000000045, -27.560000000000045, -26.000000000000043, -24.52000000000004, -23.12000000000004, -21.80000000000004, -20.560000000000038, -19.40000000000004, -18.32000000000004, -17.32000000000004, -16.40000000000004, -15.56000000000004, -14.80000000000004, -14.12000000000004, -13.520000000000042, -13.000000000000043, -12.560000000000043, -12.200000000000044, -11.920000000000046, -11.720000000000047, -11.60000000000005, -11.56000000000005, -11.600000000000053, -11.720000000000054, -11.920000000000057, -12.200000000000058, -12.560000000000061, -13.000000000000064, -13.520000000000067, -14.120000000000068, -14.800000000000072, -15.560000000000075, -16.400000000000077, -17.320000000000082, -18.320000000000086, -19.40000000000009, -20.560000000000095, -21.80000000000009, -23.1200000000001, -24.52000000000011, -26.00000000000011, -27.56000000000011, -29.200000000000117, -30.92000000000013, -32.72000000000013, -34.60000000000012 ], [ -37.960000000000065, -36.00000000000006, -34.12000000000006, -32.32000000000006, -30.600000000000055, -28.960000000000054, -27.400000000000055, -25.92000000000005, -24.520000000000053, -23.200000000000053, -21.96000000000005, -20.80000000000005, -19.72000000000005, -18.72000000000005, -17.80000000000005, -16.96000000000005, -16.200000000000053, -15.520000000000051, -14.920000000000051, -14.400000000000054, -13.960000000000054, -13.600000000000055, -13.320000000000057, -13.120000000000058, -13.00000000000006, -12.960000000000061, -13.000000000000064, -13.120000000000065, -13.320000000000068, -13.600000000000069, -13.960000000000072, -14.400000000000075, -14.920000000000076, -15.52000000000008, -16.20000000000008, -16.960000000000086, -17.80000000000009, -18.72000000000009, -19.7200000000001, -20.8000000000001, -21.960000000000104, -23.200000000000102, -24.52000000000011, -25.920000000000122, -27.40000000000012, -28.960000000000118, -30.60000000000013, -32.32000000000014, -34.12000000000014, -36.000000000000135 ], [ -39.44000000000006, -37.48000000000006, -35.60000000000006, -33.800000000000054, -32.080000000000055, -30.440000000000055, -28.880000000000052, -27.40000000000005, -26.00000000000005, -24.68000000000005, -23.440000000000047, -22.28000000000005, -21.20000000000005, -20.20000000000005, -19.28000000000005, -18.440000000000047, -17.68000000000005, -17.00000000000005, -16.400000000000052, -15.880000000000052, -15.440000000000053, -15.080000000000053, -14.800000000000056, -14.600000000000056, -14.480000000000059, -14.44000000000006, -14.480000000000063, -14.600000000000064, -14.800000000000066, -15.080000000000068, -15.44000000000007, -15.880000000000074, -16.400000000000077, -17.000000000000078, -17.68000000000008, -18.440000000000083, -19.280000000000086, -20.20000000000009, -21.200000000000095, -22.2800000000001, -23.440000000000104, -24.6800000000001, -26.00000000000011, -27.40000000000012, -28.88000000000012, -30.44000000000012, -32.080000000000126, -33.80000000000014, -35.600000000000136, -37.48000000000013 ], [ -41.00000000000006, -39.040000000000056, -37.16000000000005, -35.360000000000056, -33.64000000000005, -32.00000000000005, -30.440000000000047, -28.960000000000047, -27.560000000000045, -26.240000000000045, -25.000000000000046, -23.840000000000046, -22.760000000000048, -21.760000000000048, -20.840000000000046, -20.000000000000046, -19.240000000000045, -18.56000000000005, -17.960000000000047, -17.440000000000047, -17.00000000000005, -16.64000000000005, -16.360000000000053, -16.160000000000053, -16.040000000000056, -16.000000000000057, -16.04000000000006, -16.16000000000006, -16.360000000000063, -16.640000000000065, -17.000000000000068, -17.44000000000007, -17.960000000000072, -18.560000000000077, -19.24000000000008, -20.00000000000008, -20.840000000000085, -21.76000000000009, -22.76000000000009, -23.840000000000096, -25.0000000000001, -26.240000000000098, -27.56000000000011, -28.960000000000118, -30.44000000000012, -32.000000000000114, -33.64000000000013, -35.36000000000014, -37.16000000000014, -39.040000000000134 ], [ -42.64000000000007, -40.680000000000064, -38.80000000000007, -37.000000000000064, -35.28000000000006, -33.64000000000006, -32.08000000000006, -30.60000000000006, -29.20000000000006, -27.88000000000006, -26.640000000000057, -25.480000000000057, -24.400000000000055, -23.400000000000055, -22.480000000000057, -21.640000000000057, -20.88000000000006, -20.20000000000006, -19.60000000000006, -19.08000000000006, -18.64000000000006, -18.28000000000006, -18.000000000000064, -17.800000000000065, -17.680000000000067, -17.640000000000068, -17.68000000000007, -17.80000000000007, -18.000000000000075, -18.280000000000076, -18.64000000000008, -19.08000000000008, -19.600000000000083, -20.200000000000088, -20.880000000000088, -21.640000000000093, -22.480000000000096, -23.400000000000098, -24.400000000000105, -25.480000000000107, -26.64000000000011, -27.88000000000011, -29.200000000000117, -30.60000000000013, -32.080000000000126, -33.64000000000013, -35.280000000000136, -37.00000000000015, -38.800000000000146, -40.68000000000014 ], [ -44.360000000000085, -42.40000000000008, -40.52000000000008, -38.72000000000008, -37.00000000000007, -35.36000000000007, -33.800000000000075, -32.32000000000007, -30.920000000000073, -29.600000000000072, -28.36000000000007, -27.20000000000007, -26.12000000000007, -25.12000000000007, -24.20000000000007, -23.36000000000007, -22.600000000000072, -21.920000000000073, -21.32000000000007, -20.80000000000007, -20.360000000000074, -20.000000000000075, -19.720000000000077, -19.520000000000078, -19.40000000000008, -19.36000000000008, -19.400000000000084, -19.520000000000085, -19.720000000000088, -20.00000000000009, -20.360000000000092, -20.800000000000093, -21.320000000000096, -21.9200000000001, -22.6000000000001, -23.360000000000106, -24.20000000000011, -25.12000000000011, -26.12000000000012, -27.20000000000012, -28.360000000000124, -29.600000000000122, -30.92000000000013, -32.32000000000014, -33.80000000000014, -35.36000000000014, -37.00000000000015, -38.72000000000016, -40.52000000000016, -42.400000000000155 ], [ -46.16000000000008, -44.200000000000074, -42.32000000000008, -40.520000000000074, -38.80000000000007, -37.16000000000007, -35.60000000000007, -34.12000000000007, -32.72000000000007, -31.40000000000007, -30.160000000000068, -29.000000000000068, -27.920000000000066, -26.920000000000066, -26.000000000000068, -25.160000000000068, -24.40000000000007, -23.72000000000007, -23.12000000000007, -22.60000000000007, -22.16000000000007, -21.80000000000007, -21.520000000000074, -21.320000000000075, -21.200000000000077, -21.16000000000008, -21.20000000000008, -21.320000000000082, -21.520000000000085, -21.800000000000086, -22.16000000000009, -22.60000000000009, -23.120000000000093, -23.7200000000001, -24.400000000000098, -25.160000000000103, -26.000000000000107, -26.92000000000011, -27.920000000000115, -29.000000000000117, -30.16000000000012, -31.40000000000012, -32.72000000000013, -34.12000000000014, -35.600000000000136, -37.16000000000014, -38.800000000000146, -40.52000000000016, -42.32000000000016, -44.20000000000015 ], [ -48.04000000000008, -46.08000000000007, -44.200000000000074, -42.40000000000007, -40.680000000000064, -39.04000000000006, -37.48000000000007, -36.000000000000064, -34.600000000000065, -33.280000000000065, -32.04000000000006, -30.880000000000063, -29.80000000000006, -28.80000000000006, -27.880000000000063, -27.040000000000063, -26.280000000000065, -25.600000000000065, -25.000000000000064, -24.480000000000064, -24.040000000000067, -23.680000000000067, -23.40000000000007, -23.20000000000007, -23.080000000000073, -23.040000000000074, -23.080000000000076, -23.200000000000077, -23.40000000000008, -23.68000000000008, -24.040000000000084, -24.480000000000086, -25.00000000000009, -25.600000000000094, -26.280000000000094, -27.0400000000001, -27.880000000000102, -28.800000000000104, -29.80000000000011, -30.880000000000113, -32.04000000000012, -33.280000000000115, -34.60000000000012, -36.000000000000135, -37.48000000000013, -39.040000000000134, -40.68000000000014, -42.400000000000155, -44.20000000000015, -46.08000000000015 ] ] }, { "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ], "scene": "scene2", "type": "surface", "x": [ [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ], [ -5, -4.8, -4.6, -4.3999999999999995, -4.199999999999999, -3.999999999999999, -3.799999999999999, -3.5999999999999988, -3.3999999999999986, -3.1999999999999984, -2.9999999999999982, -2.799999999999998, -2.599999999999998, -2.3999999999999977, -2.1999999999999975, -1.9999999999999973, -1.7999999999999972, -1.599999999999997, -1.3999999999999968, -1.1999999999999966, -0.9999999999999964, -0.7999999999999963, -0.5999999999999961, -0.3999999999999959, -0.19999999999999574, 4.440892098500626e-15, 0.20000000000000462, 0.4000000000000048, 0.600000000000005, 0.8000000000000052, 1.0000000000000053, 1.2000000000000055, 1.4000000000000057, 1.6000000000000059, 1.800000000000006, 2.000000000000006, 2.2000000000000064, 2.4000000000000066, 2.6000000000000068, 2.800000000000007, 3.000000000000007, 3.2000000000000064, 3.4000000000000075, 3.6000000000000085, 3.800000000000008, 4.000000000000007, 4.200000000000008, 4.400000000000009, 4.6000000000000085, 4.800000000000008 ] ], "y": [ [ -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5 ], [ -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8, -4.8 ], [ -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6, -4.6 ], [ -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995, -4.3999999999999995 ], [ -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999, -4.199999999999999 ], [ -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999, -3.999999999999999 ], [ -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999, -3.799999999999999 ], [ -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988, -3.5999999999999988 ], [ -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986, -3.3999999999999986 ], [ -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984, -3.1999999999999984 ], [ -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982, -2.9999999999999982 ], [ -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998, -2.799999999999998 ], [ -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998, -2.599999999999998 ], [ -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977, -2.3999999999999977 ], [ -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975, -2.1999999999999975 ], [ -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973, -1.9999999999999973 ], [ -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972, -1.7999999999999972 ], [ -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997, -1.599999999999997 ], [ -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968, -1.3999999999999968 ], [ -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966, -1.1999999999999966 ], [ -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964, -0.9999999999999964 ], [ -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963, -0.7999999999999963 ], [ -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961, -0.5999999999999961 ], [ -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959, -0.3999999999999959 ], [ -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574, -0.19999999999999574 ], [ 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15, 4.440892098500626e-15 ], [ 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462, 0.20000000000000462 ], [ 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048, 0.4000000000000048 ], [ 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005, 0.600000000000005 ], [ 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052, 0.8000000000000052 ], [ 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053, 1.0000000000000053 ], [ 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055, 1.2000000000000055 ], [ 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057, 1.4000000000000057 ], [ 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059, 1.6000000000000059 ], [ 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006, 1.800000000000006 ], [ 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006, 2.000000000000006 ], [ 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064, 2.2000000000000064 ], [ 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066, 2.4000000000000066 ], [ 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068, 2.6000000000000068 ], [ 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007, 2.800000000000007 ], [ 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007, 3.000000000000007 ], [ 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064, 3.2000000000000064 ], [ 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075, 3.4000000000000075 ], [ 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085, 3.6000000000000085 ], [ 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008, 3.800000000000008 ], [ 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007, 4.000000000000007 ], [ 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008, 4.200000000000008 ], [ 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009, 4.400000000000009 ], [ 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085, 4.6000000000000085 ], [ 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008, 4.800000000000008 ] ], "z": [ [ -12.642411176571153, -13.289394880814854, -13.96660076036068, -13.820456370551572, -12.851327347181744, -11.913518152857637, -12.565713630021122, -13.25011139365136, -13.113288899469204, -12.15576114846649, -11.231978796979323, -11.900766564610445, -12.60445196224333, -12.489728191387954, -11.557207225514428, -10.661412934927586, -11.361214289168226, -12.098948854400357, -12.02128068566406, -11.128748920637578, -10.275757266411311, -11.021004755847818, -11.806608317475758, -11.77896153502854, -10.938286093236453, -10.138626172095204, -10.938286093236506, -11.778961535028559, -11.806608317475742, -11.02100475584777, -10.275757266411313, -11.12874892063763, -12.021280685664081, -12.09894885440034, -11.36121428916818, -10.661412934927592, -11.557207225514487, -12.489728191387975, -12.604451962243314, -11.900766564610402, -11.23197879697933, -12.155761148466539, -13.113288899469229, -13.250111393651348, -12.565713630021076, -11.913518152857643, -12.851327347181801, -13.820456370551597, -13.96660076036067, -13.289394880814815 ], [ -13.289394880814854, -13.698338590858068, -14.128586219463543, -13.976024707652572, -13.241031656272627, -12.528476461737933, -12.941852545084211, -13.378591221582944, -13.234748053148877, -12.510870598140736, -11.811996773136189, -12.241783687120103, -12.697817462101638, -12.57629487753946, -11.87788578793983, -11.207725614414795, -11.669537919355367, -12.160937797885914, -12.078107217482838, -11.42165161930007, -10.796587502188709, -11.306461449123583, -11.848651773450102, -11.819044306780894, -11.217891928102437, -10.649807687858152, -11.217891928102475, -11.819044306780906, -11.848651773450092, -11.306461449123551, -10.796587502188713, -11.421651619300109, -12.078107217482854, -12.160937797885907, -11.669537919355335, -11.2077256144148, -11.87788578793987, -12.576294877539478, -12.69781746210163, -12.241783687120074, -11.811996773136196, -12.510870598140775, -13.234748053148897, -13.378591221582937, -12.94185254508418, -12.52847646173794, -13.241031656272668, -13.976024707652593, -14.128586219463536, -13.698338590858043 ], [ -13.96660076036068, -14.128586219463543, -14.302605427560742, -14.143399606965417, -13.65135700446906, -13.172514487383364, -13.33802626757008, -13.517703889875085, -13.36647743486373, -12.884927699662478, -12.419284937803, -12.600893980315231, -12.799749809979318, -12.670953503192344, -12.215238311798945, -11.778961535028548, -11.993562150723225, -12.22908851724309, -12.140646401939229, -11.728917552001088, -11.34014499003266, -11.605585594761111, -11.895034881128222, -11.863274787190699, -11.51059511121164, -11.182785012462768, -11.51059511121166, -11.863274787190706, -11.895034881128216, -11.605585594761095, -11.340144990032664, -11.728917552001112, -12.14064640193924, -12.229088517243085, -11.993562150723209, -11.778961535028552, -12.21523831179897, -12.670953503192358, -12.799749809979316, -12.600893980315217, -12.419284937803008, -12.8849276996625, -13.366477434863747, -13.517703889875083, -13.338026267570065, -13.17251448738337, -13.651357004469087, -14.143399606965431, -14.30260542756074, -14.12858621946353 ], [ -13.820456370551572, -13.976024707652572, -14.143399606965417, -13.977328015619394, -13.478207099939697, -12.992086377686359, -13.15013701673645, -13.32219242442963, -13.163210058898994, -12.67380419990967, -12.200245224950903, -12.37392516807629, -12.564893579095557, -12.428313534966899, -11.964987507758167, -11.52134867097084, -11.7289175520011, -11.957828297119285, -11.863274787190699, -11.44602684065556, -11.052412754318876, -11.313768862728292, -11.599961200780774, -11.565830286684003, -11.211709898234949, -10.883416479112666, -11.211709898234972, -11.565830286684012, -11.599961200780768, -11.313768862728272, -11.052412754318878, -11.446026840655584, -11.863274787190708, -11.957828297119281, -11.728917552001086, -11.521348670970845, -11.964987507758194, -12.428313534966913, -12.564893579095553, -12.373925168076278, -12.20024522495091, -12.673804199909693, -13.163210058899008, -13.322192424429632, -13.150137016736437, -12.992086377686366, -13.478207099939723, -13.977328015619413, -14.143399606965415, -13.976024707652561 ], [ -12.851327347181744, -13.241031656272627, -13.65135700446906, -13.478207099939697, -12.721985841778707, -11.987600067193826, -12.378593511508782, -12.792462492916261, -12.625344109840382, -11.877885787939828, -11.155245286860806, -11.561220893286663, -11.993562150723218, -11.848651773450086, -11.127367486031217, -10.43507287791631, -10.875737089703444, -11.347233919742399, -11.246011419050745, -10.572941341209988, -9.933298367539347, -10.430869929204498, -10.963248323507115, -10.926497292148259, -10.32100337325724, -9.751462577014419, -10.321003373257275, -10.926497292148273, -10.963248323507102, -10.430869929204464, -9.93329836753935, -10.57294134121003, -11.246011419050765, -11.347233919742393, -10.875737089703415, -10.435072877916316, -11.12736748603126, -11.848651773450108, -11.99356215072321, -11.561220893286636, -11.155245286860813, -11.877885787939867, -12.625344109840404, -12.792462492916258, -12.378593511508752, -11.987600067193835, -12.72198584177875, -13.47820709993972, -13.651357004469055, -13.2410316562726 ], [ -11.913518152857637, -12.528476461737933, -13.172514487383364, -12.992086377686359, -11.987600067193826, -11.013420717655567, -11.628315177979504, -12.274587442075273, -12.098948854400344, -11.102078899201112, -10.138626172095202, -10.76764920473097, -11.431746890395583, -11.277922038586922, -10.307133709508006, -9.37428781734064, -10.038665687111678, -10.743040506787828, -10.63452563248978, -9.71411110985258, -8.836638915350667, -9.561217898350048, -10.330329862517045, -10.2906722967268, -9.442695520570023, -8.640585759756156, -9.442695520570075, -10.29067229672682, -10.33032986251703, -9.561217898349998, -8.836638915350669, -9.714111109852634, -10.634525632489805, -10.743040506787812, -10.038665687111635, -9.374287817340647, -10.307133709508067, -11.277922038586947, -11.431746890395573, -10.767649204730928, -10.138626172095208, -11.102078899201167, -12.098948854400371, -12.27458744207526, -11.62831517797946, -11.013420717655574, -11.987600067193885, -12.992086377686388, -13.172514487383353, -12.528476461737895 ], [ -12.565713630021122, -12.941852545084211, -13.33802626757008, -13.15013701673645, -12.378593511508782, -11.628315177979504, -12.00286777016213, -12.399780909959588, -12.215238311798949, -11.449949483664271, -10.709151939031267, -11.096743532563536, -11.510595111211652, -11.347233919742395, -10.607706244661813, -9.897568088439078, -10.321003373257273, -10.776119960738315, -10.65961431833021, -9.972614088410744, -9.320648561668248, -9.807748204120527, -10.331725534888728, -10.288830373575717, -9.679590792721719, -9.108791388352943, -9.679590792721756, -10.28883037357573, -10.331725534888717, -9.807748204120493, -9.320648561668252, -9.972614088410785, -10.659614318330227, -10.776119960738306, -10.321003373257245, -9.897568088439083, -10.607706244661857, -11.347233919742417, -11.510595111211646, -11.09674353256351, -10.709151939031274, -11.44994948366431, -12.21523831179897, -12.399780909959581, -12.002867770162105, -11.62831517797951, -12.378593511508823, -13.150137016736474, -13.338026267570072, -12.941852545084187 ], [ -13.25011139365136, -13.378591221582944, -13.517703889875085, -13.32219242442963, -12.792462492916261, -12.274587442075273, -12.399780909959588, -12.53794113005159, -12.344116819686434, -11.819044306780894, -11.309151297216179, -11.446026840655575, -11.599961200780774, -11.426405066996015, -10.92649729214826, -10.447056127506297, -10.620033199462364, -10.816033846626945, -10.690753525148484, -10.245480236156222, -9.827059334123716, -10.067322022292792, -10.336583831327633, -10.290065865728328, -9.928391855906328, -9.597559930562056, -9.928391855906352, -10.290065865728337, -10.336583831327628, -10.067322022292778, -9.827059334123721, -10.245480236156247, -10.690753525148494, -10.816033846626942, -10.620033199462352, -10.447056127506302, -10.926497292148289, -11.426405066996029, -11.599961200780774, -11.446026840655565, -11.309151297216186, -11.81904430678092, -12.344116819686448, -12.537941130051593, -12.399780909959574, -12.27458744207528, -12.79246249291629, -13.32219242442965, -13.517703889875087, -13.378591221582933 ], [ -13.113288899469204, -13.234748053148877, -13.36647743486373, -13.163210058898994, -12.625344109840382, -12.098948854400344, -12.215238311798949, -12.344116819686434, -12.140646401939225, -11.605585594761093, -11.085395117068138, -11.211709898234957, -11.354881084629907, -11.17043732500679, -10.659614318330203, -10.169347323384647, -10.331725534888719, -10.51751155703265, -10.38257528984713, -9.928391855906323, -9.501999760699055, -9.735420692914017, -9.999147713177717, -9.94855545207261, -9.584386517393888, -9.252714350203338, -9.584386517393911, -9.948555452072616, -9.999147713177713, -9.735420692913998, -9.50199976069906, -9.92839185590635, -10.382575289847143, -10.51751155703265, -10.331725534888708, -10.169347323384654, -10.659614318330231, -11.170437325006803, -11.354881084629909, -11.211709898234949, -11.085395117068146, -11.605585594761124, -12.140646401939241, -12.344116819686436, -12.215238311798938, -12.098948854400353, -12.625344109840412, -13.16321005889901, -13.366477434863732, -13.234748053148868 ], [ -12.15576114846649, -12.510870598140736, -12.884927699662478, -12.67380419990967, -11.877885787939828, -11.102078899201112, -11.449949483664271, -11.819044306780894, -11.605585594761093, -10.810347829499317, -10.038665687111651, -10.39457260968299, -10.776119960738306, -10.580066805143787, -9.807748204120486, -9.065070003379432, -9.456628581830133, -9.881004650203113, -9.735420692914001, -9.021570244794937, -8.345568409077346, -8.812024978743558, -9.319293774403846, -9.264093767051097, -8.647316947379204, -8.073978752420247, -8.647316947379244, -9.264093767051111, -9.319293774403835, -8.812024978743526, -8.34556840907735, -9.02157024479498, -9.735420692914017, -9.881004650203106, -9.456628581830103, -9.065070003379441, -9.807748204120532, -10.580066805143804, -10.7761199607383, -10.394572609682967, -10.038665687111662, -10.81034782949936, -11.605585594761118, -11.81904430678089, -11.449949483664245, -11.102078899201123, -11.87788578793987, -12.673804199909693, -12.884927699662473, -12.510870598140713 ], [ -11.231978796979323, -11.811996773136189, -12.419284937803, -12.200245224950903, -11.155245286860806, -10.138626172095202, -10.709151939031267, -11.309151297216179, -11.085395117068138, -10.038665687111651, -9.023767278119468, -9.599977515837244, -10.210188951178694, -10.001783736586418, -8.976196107093736, -7.988910810518695, -8.59989541736731, -9.252714350203345, -9.095366329113347, -8.129799529934036, -7.211853616762054, -7.901629659606412, -8.64254139727292, -8.58209771931233, -7.721380653497336, -6.9149781629492875, -7.721380653497388, -8.582097719312351, -8.642541397272904, -7.901629659606364, -7.211853616762058, -8.12979952993409, -9.095366329113372, -9.25271435020333, -8.599895417367268, -7.988910810518703, -8.976196107093799, -10.001783736586447, -10.210188951178683, -9.599977515837205, -9.023767278119477, -10.038665687111706, -11.085395117068167, -11.30915129721617, -10.709151939031225, -10.138626172095211, -11.155245286860865, -12.200245224950933, -12.419284937802994, -11.811996773136153 ], [ -11.900766564610445, -12.241783687120103, -12.600893980315231, -12.37392516807629, -11.561220893286663, -10.76764920473097, -11.096743532563536, -11.446026840655575, -11.211709898234957, -10.39457260968299, -9.599977515837244, -9.93201503338402, -10.288830373575726, -10.067322022292784, -9.269019280491584, -8.50008611960788, -8.86544866682387, -9.264093767051111, -9.09372611525072, -8.35658979462316, -7.659401452659614, -8.107395175979912, -8.59953309209409, -8.533080124674846, -7.909363376374364, -7.33367944122107, -7.909363376374401, -8.533080124674862, -8.599533092094081, -8.10739517597988, -7.659401452659619, -8.356589794623202, -9.093726115250739, -9.264093767051108, -8.865448666823843, -8.50008611960789, -9.269019280491632, -10.067322022292808, -10.28883037357572, -9.932015033383994, -9.599977515837255, -10.394572609683035, -11.211709898234984, -11.446026840655572, -11.09674353256351, -10.767649204730981, -11.561220893286707, -12.373925168076315, -12.600893980315227, -12.24178368712008 ], [ -12.60445196224333, -12.697817462101638, -12.799749809979318, -12.564893579095557, -11.993562150723218, -11.431746890395583, -11.510595111211652, -11.599961200780774, -11.354881084629907, -10.776119960738306, -10.210188951178694, -10.288830373575726, -10.382575289847136, -10.147222391384862, -9.584386517393895, -9.041510163258275, -9.151334662149702, -9.28543099731894, -9.100635228115493, -8.599533092094076, -8.130385019387074, -8.326489569978603, -8.559592128484251, -8.486197883457285, -8.107954826411394, -7.771523448872303, -8.107954826411415, -8.486197883457294, -8.559592128484246, -8.326489569978587, -8.13038501938708, -8.599533092094102, -9.10063522811551, -9.285430997318942, -9.15133466214969, -9.041510163258286, -9.584386517393927, -10.14722239138488, -10.382575289847141, -10.288830373575717, -10.210188951178706, -10.776119960738333, -11.354881084629925, -11.599961200780777, -11.510595111211643, -11.431746890395596, -11.993562150723248, -12.564893579095575, -12.79974980997932, -12.697817462101629 ], [ -12.489728191387954, -12.57629487753946, -12.670953503192344, -12.428313534966899, -11.848651773450086, -11.277922038586922, -11.347233919742395, -11.426405066996015, -11.17043732500679, -10.580066805143787, -10.001783736586418, -10.067322022292784, -10.147222391384862, -9.897318413128199, -9.319293774403832, -8.760704061019055, -8.854459208336927, -8.972365427485169, -8.771569565628354, -8.255047430620678, -7.771523448872291, -7.954815695439699, -8.177210505390576, -8.095726386190918, -7.712438669259868, -7.374292407379144, -7.712438669259891, -8.095726386190927, -8.17721050539057, -7.954815695439683, -7.771523448872298, -8.255047430620705, -8.77156956562837, -8.97236542748517, -8.854459208336916, -8.760704061019066, -9.319293774403864, -9.897318413128216, -10.147222391384865, -10.067322022292776, -10.001783736586429, -10.580066805143813, -11.170437325006809, -11.42640506699602, -11.347233919742385, -11.277922038586933, -11.848651773450118, -12.42831353496692, -12.670953503192349, -12.57629487753945 ], [ -11.557207225514428, -11.87788578793983, -12.215238311798945, -11.964987507758167, -11.127367486031217, -10.307133709508006, -10.607706244661813, -10.92649729214826, -10.659614318330203, -9.807748204120486, -8.976196107093736, -9.269019280491584, -9.584386517393895, -9.319293774403832, -8.475467888697453, -7.659401452659585, -7.9765148307275435, -8.326489569978587, -8.107954826411387, -7.324323509993164, -6.583713245684322, -6.992939052947042, -7.452622031279661, -7.36161276318713, -6.722566574326212, -6.14166851072218, -6.722566574326249, -7.361612763187146, -7.4526220312796525, -6.992939052947012, -6.583713245684329, -7.324323509993208, -8.107954826411408, -8.326489569978586, -7.976514830727519, -7.659401452659598, -8.475467888697501, -9.319293774403858, -9.584386517393892, -9.26901928049156, -8.976196107093749, -9.80774820412053, -10.659614318330227, -10.92649729214826, -10.607706244661788, -10.307133709508017, -11.127367486031265, -11.964987507758194, -12.215238311798942, -11.877885787939807 ], [ -10.661412934927586, -11.207725614414795, -11.778961535028548, -11.52134867097084, -10.43507287791631, -9.37428781734064, -9.897568088439078, -10.447056127506297, -10.169347323384647, -9.065070003379432, -7.988910810518695, -8.50008611960788, -9.041510163258275, -8.760704061019055, -7.659401452659585, -6.5935990792872055, -7.124048778803203, -7.6954343365687485, -7.457265933108928, -6.413431400658972, -5.422131717799502, -6.046175712033971, -6.731864446075958, -6.629508444783948, -5.742567056939823, -4.9272336711246965, -5.742567056939876, -6.629508444783969, -6.731864446075946, -6.046175712033925, -5.422131717799509, -6.413431400659029, -7.457265933108957, -7.69543433656874, -7.124048778803164, -6.593599079287216, -7.659401452659653, -8.760704061019084, -9.04151016325827, -8.500086119607841, -7.988910810518705, -9.065070003379489, -10.169347323384676, -10.447056127506292, -9.897568088439037, -9.374287817340651, -10.43507287791637, -11.521348670970873, -11.778961535028541, -11.20772561441476 ], [ -11.361214289168226, -11.669537919355367, -11.993562150723225, -11.7289175520011, -10.875737089703444, -10.038665687111678, -10.321003373257273, -10.620033199462364, -10.331725534888719, -9.456628581830133, -8.59989541736731, -8.86544866682387, -9.151334662149702, -8.854459208336927, -7.9765148307275435, -7.124048778803203, -7.402669788939702, -7.712438669259891, -7.452622031279663, -6.627597475267393, -5.8468271967952425, -6.218856780162767, -6.646344801156111, -6.530289617480161, -5.8752916962735, -5.288910397660636, -5.8752916962735355, -6.530289617480182, -6.646344801156104, -6.218856780162742, -5.846827196795253, -6.627597475267439, -7.452622031279686, -7.712438669259887, -7.402669788939679, -7.124048778803214, -7.976514830727593, -8.854459208336952, -9.151334662149702, -8.865448666823847, -8.599895417367323, -9.456628581830175, -10.331725534888747, -10.620033199462364, -10.32100337325725, -10.03866568711169, -10.875737089703492, -11.728917552001125, -11.993562150723223, -11.669537919355344 ], [ -12.098948854400357, -12.160937797885914, -12.22908851724309, -11.957828297119285, -11.347233919742399, -10.743040506787828, -10.776119960738315, -10.816033846626945, -10.51751155703265, -9.881004650203113, -9.252714350203345, -9.264093767051111, -9.28543099731894, -8.972365427485169, -8.326489569978587, -7.6954343365687485, -7.712438669259891, -7.750005507777203, -7.466485176629362, -6.866714716512332, -6.3020676722773, -6.409847182875286, -6.566537850223732, -6.433686181623813, -6.017554479587538, -5.668093929283909, -6.017554479587563, -6.433686181623823, -6.566537850223732, -6.409847182875275, -6.30206767227731, -6.866714716512364, -7.46648517662938, -7.7500055077772085, -7.712438669259882, -7.6954343365687645, -8.326489569978621, -8.972365427485189, -9.285430997318946, -9.264093767051108, -9.252714350203355, -9.881004650203142, -10.517511557032671, -10.816033846626949, -10.776119960738306, -10.743040506787839, -11.34723391974243, -11.957828297119306, -12.229088517243092, -12.160937797885907 ], [ -12.02128068566406, -12.078107217482838, -12.140646401939229, -11.863274787190699, -11.246011419050745, -10.63452563248978, -10.65961431833021, -10.690753525148484, -10.38257528984713, -9.735420692914001, -9.095366329113347, -9.09372611525072, -9.100635228115493, -8.771569565628354, -8.107954826411387, -7.457265933108928, -7.452622031279663, -7.466485176629362, -7.157311420136502, -6.530289617480147, -5.937521668699237, -6.017554479587545, -6.148714783081058, -5.994902127106652, -5.564865983827982, -5.21051955596122, -5.564865983828, -5.994902127106663, -6.148714783081058, -6.017554479587535, -5.937521668699247, -6.530289617480179, -7.157311420136521, -7.466485176629368, -7.452622031279658, -7.457265933108943, -8.107954826411424, -8.771569565628377, -9.100635228115502, -9.093726115250714, -9.09536632911336, -9.735420692914031, -10.382575289847152, -10.690753525148493, -10.659614318330203, -10.63452563248979, -11.246011419050777, -11.863274787190717, -12.140646401939234, -12.078107217482831 ], [ -11.128748920637578, -11.42165161930007, -11.728917552001088, -11.44602684065556, -10.572941341209988, -9.71411110985258, -9.972614088410744, -10.245480236156222, -9.928391855906323, -9.021570244794937, -8.129799529934036, -8.35658979462316, -8.599533092094076, -8.255047430620678, -7.324323509993164, -6.413431400658972, -6.627597475267393, -6.866714716512332, -6.530289617480147, -5.623639089029201, -4.758263956770833, -5.046309234254682, -5.395942358173851, -5.215447231771993, -4.51739595934945, -3.9158211937892986, -4.517395959349493, -5.215447231772007, -5.395942358173848, -5.046309234254654, -4.758263956770847, -5.6236390890292505, -6.530289617480175, -6.866714716512334, -6.627597475267372, -6.413431400658986, -7.324323509993215, -8.255047430620706, -8.599533092094076, -8.356589794623138, -8.12979952993405, -9.021570244794983, -9.928391855906353, -10.245480236156224, -9.97261408841072, -9.71411110985259, -10.572941341210036, -11.446026840655588, -11.728917552001086, -11.421651619300048 ], [ -10.275757266411311, -10.796587502188709, -11.34014499003266, -11.052412754318876, -9.933298367539347, -8.836638915350667, -9.320648561668248, -9.827059334123716, -9.501999760699055, -8.345568409077346, -7.211853616762054, -7.659401452659614, -8.130385019387074, -7.771523448872291, -6.583713245684322, -5.422131717799502, -5.8468271967952425, -6.3020676722773, -5.937521668699237, -4.758263956770833, -3.6253849384403516, -4.10717544071289, -4.658923242132126, -4.443735450734838, -3.480175328584888, -2.6375310921082935, -3.480175328584945, -4.443735450734863, -4.6589232421321185, -4.1071754407128545, -3.625384938440366, -4.758263956770897, -5.937521668699269, -6.302067672277296, -5.846827196795207, -5.422131717799516, -6.58371324568439, -7.771523448872326, -8.13038501938707, -7.65940145265958, -7.21185361676207, -8.345568409077405, -9.50199976069909, -9.827059334123712, -9.32064856166821, -8.83663891535068, -9.933298367539411, -11.052412754318908, -11.340144990032657, -10.796587502188677 ], [ -11.021004755847818, -11.306461449123583, -11.605585594761111, -11.313768862728292, -10.430869929204498, -9.561217898350048, -9.807748204120527, -10.067322022292792, -9.735420692914017, -8.812024978743558, -7.901629659606412, -8.107395175979912, -8.326489569978603, -7.954815695439699, -6.992939052947042, -6.046175712033971, -6.218856780162767, -6.409847182875286, -6.017554479587545, -5.046309234254682, -4.10717544071289, -4.313320531036098, -4.577012137495945, -4.315841317730712, -3.5577131909328408, -2.933520672376769, -3.5577131909328834, -4.315841317730737, -4.577012137495945, -4.31332053103608, -4.107175440712904, -5.0463092342547355, -6.017554479587574, -6.409847182875293, -6.218856780162749, -6.046175712033989, -6.992939052947097, -7.954815695439729, -8.326489569978607, -8.107395175979892, -7.901629659606424, -8.812024978743603, -9.735420692914044, -10.067322022292796, -9.807748204120502, -9.56121789835006, -10.430869929204546, -11.313768862728317, -11.605585594761113, -11.306461449123564 ], [ -11.806608317475758, -11.848651773450102, -11.895034881128222, -11.599961200780774, -10.963248323507115, -10.330329862517045, -10.331725534888728, -10.336583831327633, -9.999147713177717, -9.319293774403846, -8.64254139727292, -8.59953309209409, -8.559592128484251, -8.177210505390576, -7.452622031279661, -6.731864446075958, -6.646344801156111, -6.566537850223732, -6.148714783081058, -5.395942358173851, -4.658923242132126, -4.577012137495945, -4.534577514907866, -4.212041049120636, -3.650668173243279, -3.24513262032821, -3.6506681732433073, -4.212041049120657, -4.534577514907877, -4.577012137495942, -4.658923242132143, -5.39594235817389, -6.148714783081083, -6.566537850223742, -6.646344801156108, -6.731864446075976, -7.4526220312797005, -8.177210505390601, -8.559592128484258, -8.599533092094088, -8.642541397272934, -9.319293774403876, -9.99914771317774, -10.336583831327642, -10.331725534888719, -10.330329862517056, -10.963248323507147, -11.599961200780799, -11.895034881128227, -11.848651773450094 ], [ -11.77896153502854, -11.819044306780894, -11.863274787190699, -11.565830286684003, -10.926497292148259, -10.2906722967268, -10.288830373575717, -10.290065865728328, -9.94855545207261, -9.264093767051097, -8.58209771931233, -8.533080124674846, -8.486197883457285, -8.095726386190918, -7.36161276318713, -6.629508444783948, -6.530289617480161, -6.433686181623813, -5.994902127106652, -5.215447231771993, -4.443735450734838, -4.315841317730712, -4.212041049120636, -3.810659321518287, -3.1652222170573054, -2.7180481650584767, -3.1652222170573374, -3.8106593215183118, -4.21204104912065, -4.315841317730712, -4.443735450734859, -5.215447231772032, -5.9949021271066805, -6.433686181623827, -6.530289617480161, -6.629508444783966, -7.361612763187168, -8.095726386190943, -8.486197883457294, -8.533080124674845, -8.582097719312348, -9.264093767051127, -9.94855545207263, -10.290065865728335, -10.28883037357571, -10.290672296726811, -10.926497292148294, -11.565830286684022, -11.863274787190704, -11.819044306780885 ], [ -10.938286093236453, -11.217891928102437, -11.51059511121164, -11.211709898234949, -10.32100337325724, -9.442695520570023, -9.679590792721719, -9.928391855906328, -9.584386517393888, -8.647316947379204, -7.721380653497336, -7.909363376374364, -8.107954826411394, -7.712438669259868, -6.722566574326212, -5.742567056939823, -5.8752916962735, -6.017554479587538, -5.564865983827982, -4.51739595934945, -3.480175328584888, -3.5577131909328408, -3.650668173243279, -3.1652222170573054, -2.1404075273137906, -1.3518456108914343, -2.1404075273138474, -3.165222217057341, -3.650668173243293, -3.5577131909328266, -3.4801753285849095, -4.5173959593495105, -5.564865983828014, -6.017554479587542, -5.875291696273482, -5.742567056939844, -6.722566574326265, -7.712438669259901, -8.107954826411397, -7.909363376374346, -7.72138065349735, -8.647316947379252, -9.584386517393916, -9.928391855906332, -9.679590792721699, -9.442695520570036, -10.321003373257286, -11.211709898234977, -11.510595111211641, -11.217891928102416 ], [ -10.138626172095204, -10.649807687858152, -11.182785012462768, -10.883416479112666, -9.751462577014419, -8.640585759756156, -9.108791388352943, -9.597559930562056, -9.252714350203338, -8.073978752420247, -6.9149781629492875, -7.33367944122107, -7.771523448872303, -7.374292407379144, -6.14166851072218, -4.9272336711246965, -5.288910397660636, -5.668093929283909, -5.21051955596122, -3.9158211937892986, -2.6375310921082935, -2.933520672376769, -3.24513262032821, -2.7180481650584767, -1.3518456108914343, -1.7763568394002505e-14, -1.351845610891509, -2.7180481650585193, -3.2451326203282136, -2.9335206723767406, -2.637531092108315, -3.915821193789366, -5.210519555961259, -5.668093929283913, -5.2889103976606044, -4.927233671124718, -6.141668510722251, -7.37429240737918, -7.771523448872301, -7.333679441221038, -6.9149781629493035, -8.073978752420306, -9.252714350203368, -9.597559930562053, -9.108791388352905, -8.640585759756167, -9.751462577014482, -10.883416479112698, -11.182785012462764, -10.649807687858118 ], [ -10.938286093236506, -11.217891928102475, -11.51059511121166, -11.211709898234972, -10.321003373257275, -9.442695520570075, -9.679590792721756, -9.928391855906352, -9.584386517393911, -8.647316947379244, -7.721380653497388, -7.909363376374401, -8.107954826411415, -7.712438669259891, -6.722566574326249, -5.742567056939876, -5.8752916962735355, -6.017554479587563, -5.564865983828, -4.517395959349493, -3.480175328584945, -3.5577131909328834, -3.6506681732433073, -3.1652222170573374, -2.1404075273138474, -1.351845610891509, -2.1404075273139007, -3.165222217057373, -3.6506681732433215, -3.557713190932869, -3.4801753285849664, -4.51739595934955, -5.564865983828035, -6.017554479587567, -5.875291696273521, -5.742567056939894, -6.722566574326304, -7.712438669259923, -8.10795482641142, -7.909363376374385, -7.721380653497405, -8.64731694737929, -9.58438651739394, -9.928391855906357, -9.679590792721735, -9.442695520570087, -10.321003373257325, -11.211709898234998, -11.510595111211662, -11.217891928102452 ], [ -11.778961535028559, -11.819044306780906, -11.863274787190706, -11.565830286684012, -10.926497292148273, -10.29067229672682, -10.28883037357573, -10.290065865728337, -9.948555452072616, -9.264093767051111, -8.582097719312351, -8.533080124674862, -8.486197883457294, -8.095726386190927, -7.361612763187146, -6.629508444783969, -6.530289617480182, -6.433686181623823, -5.994902127106663, -5.215447231772007, -4.443735450734863, -4.315841317730737, -4.212041049120657, -3.8106593215183118, -3.165222217057341, -2.7180481650585193, -3.165222217057373, -3.8106593215183366, -4.212041049120668, -4.315841317730733, -4.443735450734884, -5.215447231772046, -5.994902127106691, -6.4336861816238375, -6.530289617480179, -6.6295084447839905, -7.3616127631871855, -8.095726386190952, -8.486197883457304, -8.533080124674859, -8.582097719312367, -9.264093767051142, -9.94855545207264, -10.290065865728346, -10.288830373575724, -10.29067229672683, -10.926497292148307, -11.565830286684031, -11.863274787190711, -11.819044306780901 ], [ -11.806608317475742, -11.848651773450092, -11.895034881128216, -11.599961200780768, -10.963248323507102, -10.33032986251703, -10.331725534888717, -10.336583831327628, -9.999147713177713, -9.319293774403835, -8.642541397272904, -8.599533092094081, -8.559592128484246, -8.17721050539057, -7.4526220312796525, -6.731864446075946, -6.646344801156104, -6.566537850223732, -6.148714783081058, -5.395942358173848, -4.6589232421321185, -4.577012137495945, -4.534577514907877, -4.21204104912065, -3.650668173243293, -3.2451326203282136, -3.6506681732433215, -4.212041049120668, -4.534577514907884, -4.577012137495942, -4.658923242132136, -5.395942358173887, -6.148714783081083, -6.566537850223742, -6.646344801156101, -6.731864446075964, -7.452622031279692, -8.177210505390597, -8.559592128484255, -8.59953309209408, -8.642541397272918, -9.319293774403869, -9.999147713177734, -10.336583831327637, -10.33172553488871, -10.33032986251704, -10.963248323507138, -11.599961200780793, -11.895034881128224, -11.848651773450081 ], [ -11.02100475584777, -11.306461449123551, -11.605585594761095, -11.313768862728272, -10.430869929204464, -9.561217898349998, -9.807748204120493, -10.067322022292778, -9.735420692913998, -8.812024978743526, -7.901629659606364, -8.10739517597988, -8.326489569978587, -7.954815695439683, -6.992939052947012, -6.046175712033925, -6.218856780162742, -6.409847182875275, -6.017554479587535, -5.046309234254654, -4.1071754407128545, -4.31332053103608, -4.577012137495942, -4.315841317730712, -3.5577131909328266, -2.9335206723767406, -3.557713190932869, -4.315841317730733, -4.577012137495942, -4.313320531036059, -4.107175440712869, -5.046309234254714, -6.017554479587563, -6.4098471828752785, -6.218856780162724, -6.046175712033943, -6.992939052947065, -7.954815695439713, -8.32648956997859, -8.10739517597986, -7.901629659606382, -8.812024978743572, -9.735420692914028, -10.067322022292778, -9.807748204120472, -9.561217898350012, -10.430869929204514, -11.313768862728299, -11.605585594761095, -11.30646144912353 ], [ -10.275757266411313, -10.796587502188713, -11.340144990032664, -11.052412754318878, -9.93329836753935, -8.836638915350669, -9.320648561668252, -9.827059334123721, -9.50199976069906, -8.34556840907735, -7.211853616762058, -7.659401452659619, -8.13038501938708, -7.771523448872298, -6.583713245684329, -5.422131717799509, -5.846827196795253, -6.30206767227731, -5.937521668699247, -4.758263956770847, -3.625384938440366, -4.107175440712904, -4.658923242132143, -4.443735450734859, -3.4801753285849095, -2.637531092108315, -3.4801753285849664, -4.443735450734884, -4.658923242132136, -4.107175440712869, -3.62538493844038, -4.758263956770911, -5.937521668699279, -6.302067672277307, -5.846827196795218, -5.422131717799523, -6.583713245684397, -7.771523448872333, -8.130385019387077, -7.659401452659585, -7.211853616762074, -8.345568409077408, -9.501999760699093, -9.827059334123716, -9.320648561668214, -8.836638915350683, -9.933298367539413, -11.052412754318912, -11.340144990032659, -10.796587502188679 ], [ -11.12874892063763, -11.421651619300109, -11.728917552001112, -11.446026840655584, -10.57294134121003, -9.714111109852634, -9.972614088410785, -10.245480236156247, -9.92839185590635, -9.02157024479498, -8.12979952993409, -8.356589794623202, -8.599533092094102, -8.255047430620705, -7.324323509993208, -6.413431400659029, -6.627597475267439, -6.866714716512364, -6.530289617480179, -5.6236390890292505, -4.758263956770897, -5.0463092342547355, -5.39594235817389, -5.215447231772032, -4.5173959593495105, -3.915821193789366, -4.51739595934955, -5.215447231772046, -5.395942358173887, -5.046309234254714, -4.758263956770911, -5.6236390890293, -6.530289617480211, -6.866714716512366, -6.627597475267418, -6.4134314006590465, -7.3243235099932615, -8.255047430620733, -8.599533092094104, -8.356589794623183, -8.129799529934107, -9.021570244795024, -9.92839185590638, -10.24548023615625, -9.97261408841076, -9.714111109852649, -10.572941341210077, -11.446026840655612, -11.72891755200111, -11.421651619300087 ], [ -12.021280685664081, -12.078107217482854, -12.14064640193924, -11.863274787190708, -11.246011419050765, -10.634525632489805, -10.659614318330227, -10.690753525148494, -10.382575289847143, -9.735420692914017, -9.095366329113372, -9.093726115250739, -9.10063522811551, -8.77156956562837, -8.107954826411408, -7.457265933108957, -7.452622031279686, -7.46648517662938, -7.157311420136521, -6.530289617480175, -5.937521668699269, -6.017554479587574, -6.148714783081083, -5.9949021271066805, -5.564865983828014, -5.210519555961259, -5.564865983828035, -5.994902127106691, -6.148714783081083, -6.017554479587563, -5.937521668699279, -6.530289617480211, -7.157311420136542, -7.466485176629385, -7.452622031279681, -7.457265933108971, -8.107954826411444, -8.771569565628393, -9.100635228115516, -9.093726115250735, -9.095366329113386, -9.735420692914051, -10.382575289847162, -10.690753525148503, -10.65961431833022, -10.634525632489815, -11.246011419050795, -11.86327478719073, -12.140646401939245, -12.078107217482847 ], [ -12.09894885440034, -12.160937797885907, -12.229088517243085, -11.957828297119281, -11.347233919742393, -10.743040506787812, -10.776119960738306, -10.816033846626942, -10.51751155703265, -9.881004650203106, -9.25271435020333, -9.264093767051108, -9.285430997318942, -8.97236542748517, -8.326489569978586, -7.69543433656874, -7.712438669259887, -7.7500055077772085, -7.466485176629368, -6.866714716512334, -6.302067672277296, -6.409847182875293, -6.566537850223742, -6.433686181623827, -6.017554479587542, -5.668093929283913, -6.017554479587567, -6.4336861816238375, -6.566537850223742, -6.4098471828752785, -6.302067672277307, -6.866714716512366, -7.466485176629385, -7.750005507777212, -7.712438669259882, -7.695434336568754, -8.32648956997862, -8.97236542748519, -9.285430997318949, -9.264093767051103, -9.252714350203346, -9.881004650203135, -10.51751155703267, -10.816033846626949, -10.776119960738297, -10.743040506787827, -11.347233919742424, -11.957828297119303, -12.22908851724309, -12.160937797885897 ], [ -11.36121428916818, -11.669537919355335, -11.993562150723209, -11.728917552001086, -10.875737089703415, -10.038665687111635, -10.321003373257245, -10.620033199462352, -10.331725534888708, -9.456628581830103, -8.599895417367268, -8.865448666823843, -9.15133466214969, -8.854459208336916, -7.976514830727519, -7.124048778803164, -7.402669788939679, -7.712438669259882, -7.452622031279658, -6.627597475267372, -5.846827196795207, -6.218856780162749, -6.646344801156108, -6.530289617480161, -5.875291696273482, -5.2889103976606044, -5.875291696273521, -6.530289617480179, -6.646344801156101, -6.218856780162724, -5.846827196795218, -6.627597475267418, -7.452622031279681, -7.712438669259882, -7.402669788939654, -7.124048778803177, -7.97651483072757, -8.854459208336943, -9.15133466214969, -8.86544866682382, -8.599895417367282, -9.456628581830147, -10.331725534888733, -10.62003319946235, -10.321003373257218, -10.038665687111646, -10.875737089703462, -11.72891755200111, -11.993562150723209, -11.669537919355312 ], [ -10.661412934927592, -11.2077256144148, -11.778961535028552, -11.521348670970845, -10.435072877916316, -9.374287817340647, -9.897568088439083, -10.447056127506302, -10.169347323384654, -9.065070003379441, -7.988910810518703, -8.50008611960789, -9.041510163258286, -8.760704061019066, -7.659401452659598, -6.593599079287216, -7.124048778803214, -7.6954343365687645, -7.457265933108943, -6.413431400658986, -5.422131717799516, -6.046175712033989, -6.731864446075976, -6.629508444783966, -5.742567056939844, -4.927233671124718, -5.742567056939894, -6.6295084447839905, -6.731864446075964, -6.046175712033943, -5.422131717799523, -6.4134314006590465, -7.457265933108971, -7.695434336568754, -7.124048778803177, -6.59359907928723, -7.659401452659663, -8.760704061019094, -9.041510163258277, -8.50008611960785, -7.988910810518718, -9.065070003379496, -10.169347323384683, -10.447056127506299, -9.897568088439046, -9.374287817340658, -10.435072877916376, -11.521348670970877, -11.778961535028547, -11.207725614414764 ], [ -11.557207225514487, -11.87788578793987, -12.21523831179897, -11.964987507758194, -11.12736748603126, -10.307133709508067, -10.607706244661857, -10.926497292148289, -10.659614318330231, -9.807748204120532, -8.976196107093799, -9.269019280491632, -9.584386517393927, -9.319293774403864, -8.475467888697501, -7.659401452659653, -7.976514830727593, -8.326489569978621, -8.107954826411424, -7.324323509993215, -6.58371324568439, -6.992939052947097, -7.4526220312797005, -7.361612763187168, -6.722566574326265, -6.141668510722251, -6.722566574326304, -7.3616127631871855, -7.452622031279692, -6.992939052947065, -6.583713245684397, -7.3243235099932615, -8.107954826411444, -8.32648956997862, -7.97651483072757, -7.659401452659663, -8.47546788869755, -9.319293774403889, -9.584386517393925, -9.269019280491609, -8.976196107093811, -9.807748204120575, -10.659614318330256, -10.926497292148289, -10.60770624466183, -10.307133709508078, -11.127367486031305, -11.96498750775822, -12.215238311798968, -11.877885787939848 ], [ -12.489728191387975, -12.576294877539478, -12.670953503192358, -12.428313534966913, -11.848651773450108, -11.277922038586947, -11.347233919742417, -11.426405066996029, -11.170437325006803, -10.580066805143804, -10.001783736586447, -10.067322022292808, -10.14722239138488, -9.897318413128216, -9.319293774403858, -8.760704061019084, -8.854459208336952, -8.972365427485189, -8.771569565628377, -8.255047430620706, -7.771523448872326, -7.954815695439729, -8.177210505390601, -8.095726386190943, -7.712438669259901, -7.37429240737918, -7.712438669259923, -8.095726386190952, -8.177210505390597, -7.954815695439713, -7.771523448872333, -8.255047430620733, -8.771569565628393, -8.97236542748519, -8.854459208336943, -8.760704061019094, -9.319293774403889, -9.897318413128234, -10.147222391384885, -10.067322022292796, -10.00178373658646, -10.580066805143836, -11.170437325006825, -11.426405066996034, -11.347233919742404, -11.277922038586958, -11.848651773450138, -12.428313534966932, -12.67095350319236, -12.576294877539468 ], [ -12.604451962243314, -12.69781746210163, -12.799749809979316, -12.564893579095553, -11.99356215072321, -11.431746890395573, -11.510595111211646, -11.599961200780774, -11.354881084629909, -10.7761199607383, -10.210188951178683, -10.28883037357572, -10.382575289847141, -10.147222391384865, -9.584386517393892, -9.04151016325827, -9.151334662149702, -9.285430997318946, -9.100635228115502, -8.599533092094076, -8.13038501938707, -8.326489569978607, -8.559592128484258, -8.486197883457294, -8.107954826411397, -7.771523448872301, -8.10795482641142, -8.486197883457304, -8.559592128484255, -8.32648956997859, -8.130385019387077, -8.599533092094104, -9.100635228115516, -9.285430997318949, -9.15133466214969, -9.041510163258277, -9.584386517393925, -10.147222391384885, -10.382575289847145, -10.288830373575712, -10.210188951178697, -10.77611996073833, -11.354881084629929, -11.599961200780777, -11.510595111211638, -11.43174689039558, -11.993562150723243, -12.564893579095575, -12.799749809979318, -12.697817462101622 ], [ -11.900766564610402, -12.241783687120074, -12.600893980315217, -12.373925168076278, -11.561220893286636, -10.767649204730928, -11.09674353256351, -11.446026840655565, -11.211709898234949, -10.394572609682967, -9.599977515837205, -9.932015033383994, -10.288830373575717, -10.067322022292776, -9.26901928049156, -8.500086119607841, -8.865448666823847, -9.264093767051108, -9.093726115250714, -8.356589794623138, -7.65940145265958, -8.107395175979892, -8.599533092094088, -8.533080124674845, -7.909363376374346, -7.333679441221038, -7.909363376374385, -8.533080124674859, -8.59953309209408, -8.10739517597986, -7.659401452659585, -8.356589794623183, -9.093726115250735, -9.264093767051103, -8.86544866682382, -8.50008611960785, -9.269019280491609, -10.067322022292796, -10.288830373575712, -9.932015033383973, -9.599977515837216, -10.394572609683008, -11.211709898234972, -11.446026840655561, -11.096743532563481, -10.767649204730938, -11.56122089328668, -12.373925168076303, -12.600893980315215, -12.24178368712005 ], [ -11.23197879697933, -11.811996773136196, -12.419284937803008, -12.20024522495091, -11.155245286860813, -10.138626172095208, -10.709151939031274, -11.309151297216186, -11.085395117068146, -10.038665687111662, -9.023767278119477, -9.599977515837255, -10.210188951178706, -10.001783736586429, -8.976196107093749, -7.988910810518705, -8.599895417367323, -9.252714350203355, -9.09536632911336, -8.12979952993405, -7.21185361676207, -7.901629659606424, -8.642541397272934, -8.582097719312348, -7.72138065349735, -6.9149781629493035, -7.721380653497405, -8.582097719312367, -8.642541397272918, -7.901629659606382, -7.211853616762074, -8.129799529934107, -9.095366329113386, -9.252714350203346, -8.599895417367282, -7.988910810518718, -8.976196107093811, -10.00178373658646, -10.210188951178697, -9.599977515837216, -9.023767278119488, -10.038665687111713, -11.085395117068174, -11.309151297216177, -10.709151939031235, -10.138626172095218, -11.155245286860872, -12.20024522495094, -12.419284937802999, -11.811996773136158 ], [ -12.155761148466539, -12.510870598140775, -12.8849276996625, -12.673804199909693, -11.877885787939867, -11.102078899201167, -11.44994948366431, -11.81904430678092, -11.605585594761124, -10.81034782949936, -10.038665687111706, -10.394572609683035, -10.776119960738333, -10.580066805143813, -9.80774820412053, -9.065070003379489, -9.456628581830175, -9.881004650203142, -9.735420692914031, -9.021570244794983, -8.345568409077405, -8.812024978743603, -9.319293774403876, -9.264093767051127, -8.647316947379252, -8.073978752420306, -8.64731694737929, -9.264093767051142, -9.319293774403869, -8.812024978743572, -8.345568409077408, -9.021570244795024, -9.735420692914051, -9.881004650203135, -9.456628581830147, -9.065070003379496, -9.807748204120575, -10.580066805143836, -10.77611996073833, -10.394572609683008, -10.038665687111713, -10.810347829499399, -11.605585594761143, -11.819044306780917, -11.449949483664284, -11.102078899201175, -11.87788578793991, -12.673804199909718, -12.884927699662498, -12.510870598140752 ], [ -13.113288899469229, -13.234748053148897, -13.366477434863747, -13.163210058899008, -12.625344109840404, -12.098948854400371, -12.21523831179897, -12.344116819686448, -12.140646401939241, -11.605585594761118, -11.085395117068167, -11.211709898234984, -11.354881084629925, -11.170437325006809, -10.659614318330227, -10.169347323384676, -10.331725534888747, -10.517511557032671, -10.382575289847152, -9.928391855906353, -9.50199976069909, -9.735420692914044, -9.99914771317774, -9.94855545207263, -9.584386517393916, -9.252714350203368, -9.58438651739394, -9.94855545207264, -9.999147713177734, -9.735420692914028, -9.501999760699093, -9.92839185590638, -10.382575289847162, -10.51751155703267, -10.331725534888733, -10.169347323384683, -10.659614318330256, -11.170437325006825, -11.354881084629929, -11.211709898234972, -11.085395117068174, -11.605585594761143, -12.140646401939259, -12.344116819686452, -12.215238311798958, -12.09894885440038, -12.625344109840432, -13.163210058899026, -13.366477434863747, -13.234748053148886 ], [ -13.250111393651348, -13.378591221582937, -13.517703889875083, -13.322192424429632, -12.792462492916258, -12.27458744207526, -12.399780909959581, -12.537941130051593, -12.344116819686436, -11.81904430678089, -11.30915129721617, -11.446026840655572, -11.599961200780777, -11.42640506699602, -10.92649729214826, -10.447056127506292, -10.620033199462364, -10.816033846626949, -10.690753525148493, -10.245480236156224, -9.827059334123712, -10.067322022292796, -10.336583831327642, -10.290065865728335, -9.928391855906332, -9.597559930562053, -9.928391855906357, -10.290065865728346, -10.336583831327637, -10.067322022292778, -9.827059334123716, -10.24548023615625, -10.690753525148503, -10.816033846626949, -10.62003319946235, -10.447056127506299, -10.926497292148289, -11.426405066996034, -11.599961200780777, -11.446026840655561, -11.309151297216177, -11.819044306780917, -12.344116819686452, -12.537941130051594, -12.399780909959569, -12.27458744207527, -12.792462492916286, -13.322192424429648, -13.517703889875085, -13.378591221582925 ], [ -12.565713630021076, -12.94185254508418, -13.338026267570065, -13.150137016736437, -12.378593511508752, -11.62831517797946, -12.002867770162105, -12.399780909959574, -12.215238311798938, -11.449949483664245, -10.709151939031225, -11.09674353256351, -11.510595111211643, -11.347233919742385, -10.607706244661788, -9.897568088439037, -10.32100337325725, -10.776119960738306, -10.659614318330203, -9.97261408841072, -9.32064856166821, -9.807748204120502, -10.331725534888719, -10.28883037357571, -9.679590792721699, -9.108791388352905, -9.679590792721735, -10.288830373575724, -10.33172553488871, -9.807748204120472, -9.320648561668214, -9.97261408841076, -10.65961431833022, -10.776119960738297, -10.321003373257218, -9.897568088439046, -10.60770624466183, -11.347233919742404, -11.510595111211638, -11.096743532563481, -10.709151939031235, -11.449949483664284, -12.215238311798958, -12.399780909959569, -12.002867770162075, -11.628315177979468, -12.378593511508795, -13.15013701673646, -13.338026267570058, -12.941852545084156 ], [ -11.913518152857643, -12.52847646173794, -13.17251448738337, -12.992086377686366, -11.987600067193835, -11.013420717655574, -11.62831517797951, -12.27458744207528, -12.098948854400353, -11.102078899201123, -10.138626172095211, -10.767649204730981, -11.431746890395596, -11.277922038586933, -10.307133709508017, -9.374287817340651, -10.03866568711169, -10.743040506787839, -10.63452563248979, -9.71411110985259, -8.83663891535068, -9.56121789835006, -10.330329862517056, -10.290672296726811, -9.442695520570036, -8.640585759756167, -9.442695520570087, -10.29067229672683, -10.33032986251704, -9.561217898350012, -8.836638915350683, -9.714111109852649, -10.634525632489815, -10.743040506787827, -10.038665687111646, -9.374287817340658, -10.307133709508078, -11.277922038586958, -11.43174689039558, -10.767649204730938, -10.138626172095218, -11.102078899201175, -12.09894885440038, -12.27458744207527, -11.628315177979468, -11.013420717655581, -11.987600067193892, -12.992086377686395, -13.17251448738336, -12.528476461737903 ], [ -12.851327347181801, -13.241031656272668, -13.651357004469087, -13.478207099939723, -12.72198584177875, -11.987600067193885, -12.378593511508823, -12.79246249291629, -12.625344109840412, -11.87788578793987, -11.155245286860865, -11.561220893286707, -11.993562150723248, -11.848651773450118, -11.127367486031265, -10.43507287791637, -10.875737089703492, -11.34723391974243, -11.246011419050777, -10.572941341210036, -9.933298367539411, -10.430869929204546, -10.963248323507147, -10.926497292148294, -10.321003373257286, -9.751462577014482, -10.321003373257325, -10.926497292148307, -10.963248323507138, -10.430869929204514, -9.933298367539413, -10.572941341210077, -11.246011419050795, -11.347233919742424, -10.875737089703462, -10.435072877916376, -11.127367486031305, -11.848651773450138, -11.993562150723243, -11.56122089328668, -11.155245286860872, -11.87788578793991, -12.625344109840432, -12.792462492916286, -12.378593511508795, -11.987600067193892, -12.72198584177879, -13.478207099939747, -13.651357004469082, -13.241031656272643 ], [ -13.820456370551597, -13.976024707652593, -14.143399606965431, -13.977328015619413, -13.47820709993972, -12.992086377686388, -13.150137016736474, -13.32219242442965, -13.16321005889901, -12.673804199909693, -12.200245224950933, -12.373925168076315, -12.564893579095575, -12.42831353496692, -11.964987507758194, -11.521348670970873, -11.728917552001125, -11.957828297119306, -11.863274787190717, -11.446026840655588, -11.052412754318908, -11.313768862728317, -11.599961200780799, -11.565830286684022, -11.211709898234977, -10.883416479112698, -11.211709898234998, -11.565830286684031, -11.599961200780793, -11.313768862728299, -11.052412754318912, -11.446026840655612, -11.86327478719073, -11.957828297119303, -11.72891755200111, -11.521348670970877, -11.96498750775822, -12.428313534966932, -12.564893579095575, -12.373925168076303, -12.20024522495094, -12.673804199909718, -13.163210058899026, -13.322192424429648, -13.15013701673646, -12.992086377686395, -13.478207099939747, -13.977328015619428, -14.143399606965431, -13.97602470765258 ], [ -13.96660076036067, -14.128586219463536, -14.30260542756074, -14.143399606965415, -13.651357004469055, -13.172514487383353, -13.338026267570072, -13.517703889875087, -13.366477434863732, -12.884927699662473, -12.419284937802994, -12.600893980315227, -12.79974980997932, -12.670953503192349, -12.215238311798942, -11.778961535028541, -11.993562150723223, -12.229088517243092, -12.140646401939234, -11.728917552001086, -11.340144990032657, -11.605585594761113, -11.895034881128227, -11.863274787190704, -11.510595111211641, -11.182785012462764, -11.510595111211662, -11.863274787190711, -11.895034881128224, -11.605585594761095, -11.340144990032659, -11.72891755200111, -12.140646401939245, -12.22908851724309, -11.993562150723209, -11.778961535028547, -12.215238311798968, -12.67095350319236, -12.799749809979318, -12.600893980315215, -12.419284937802999, -12.884927699662498, -13.366477434863747, -13.517703889875085, -13.338026267570058, -13.17251448738336, -13.651357004469082, -14.143399606965431, -14.302605427560742, -14.128586219463525 ], [ -13.289394880814815, -13.698338590858043, -14.12858621946353, -13.976024707652561, -13.2410316562726, -12.528476461737895, -12.941852545084187, -13.378591221582933, -13.234748053148868, -12.510870598140713, -11.811996773136153, -12.24178368712008, -12.697817462101629, -12.57629487753945, -11.877885787939807, -11.20772561441476, -11.669537919355344, -12.160937797885907, -12.078107217482831, -11.421651619300048, -10.796587502188677, -11.306461449123564, -11.848651773450094, -11.819044306780885, -11.217891928102416, -10.649807687858118, -11.217891928102452, -11.819044306780901, -11.848651773450081, -11.30646144912353, -10.796587502188679, -11.421651619300087, -12.078107217482847, -12.160937797885897, -11.669537919355312, -11.207725614414764, -11.877885787939848, -12.576294877539468, -12.697817462101622, -12.24178368712005, -11.811996773136158, -12.510870598140752, -13.234748053148886, -13.378591221582925, -12.941852545084156, -12.528476461737903, -13.241031656272643, -13.97602470765258, -14.128586219463525, -13.698338590858016 ] ] } ], "layout": { "annotations": [ { "font": { "size": 16 }, "showarrow": false, "text": "Convex Function", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Non-convex Function", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "scene": { "domain": { "x": [ 0, 0.45 ], "y": [ 0, 1 ] } }, "scene2": { "domain": { "x": [ 0.55, 1 ], "y": [ 0, 1 ] } }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "Objective Function Surface" } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "compare_objective_functions(convex_function, ackley_function)" ] }, { "cell_type": "code", "execution_count": 16, "id": "found-ballot", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " x y score\n", "0 4.14 -3.61 -12.585249\n", "1 4.46 1.01 -11.228130\n", "2 -1.67 -1.67 -7.779507\n", "3 -1.67 1.66 -7.781677\n", "4 1.66 -1.67 -7.781677\n", "... ... ... ...\n", "1995 -4.39 -0.53 -11.600498\n", "1996 2.15 -2.98 -8.620673\n", "1997 -1.54 -3.53 -10.740426\n", "1998 2.02 2.12 -7.135418\n", "1999 2.79 1.63 -9.250090\n", "\n", "[2000 rows x 3 columns] \n", "\n" ] } ], "source": [ "hyper_ackley_0 = Hyperactive(verbosity=False)\n", "hyper_ackley_0.add_search(ackley_function, search_space, n_iter=2000)\n", "hyper_ackley_0.run()\n", "\n", "search_data_ackley_0 = hyper_ackley_0.search_data(ackley_function)" ] }, { "cell_type": "code", "execution_count": 17, "id": "fresh-boost", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "x=%{x}
y=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ -12.585248765407806, -11.228129854105209, -7.779506985578617, -7.781676992689018, -7.781676992689018, -7.783350109530206, -12.63773422539314, -12.642411176571153, -12.63773422539314, -12.633039893443026, -6.470564959652535, -1.0020166230313912, -7.910823701640991, -10.813702174038154, -9.30101710347614, -10.659746892437969, -8.1983992544405, -10.463578444309864, -12.317135641336773, -6.095771954721073, -11.29295933368188, -12.113091271133829, -8.491817305429947, -5.914668003272126, -2.7854959671206068, -7.259842999436339, -9.13944745729723, -10.040480111082054, -7.78873012897105, -9.585598785322338, -13.317685797544685, -12.13734641717496, -10.530188608072914, -12.099225858927849, -10.124855564797494, -10.535591141204153, -9.166204189240146, -8.049054927740881, -5.6980737680260525, -8.816094460832803, -11.941123807393467, -11.580035270616332, -9.763246646324058, -13.225135683015916, -10.025166001627959, -9.580031748200323, -7.649165484807673, -13.111447287175721, -8.74612749974491, -9.558054529253393, -3.867680567600093, -7.618369641866213, -4.325639137762952, -8.639876715969237, -10.376827054356893, -12.126040957997926, -4.49821270587239, -4.047229873018804, -8.040231604727143, -11.021215595233848, -9.21710621905406, -7.871186562184972, -5.187515357935471, -10.488049361580208, -8.136299454627695, -3.5659560127671277, -12.612734705513459, -12.134000040105848, -9.88878151140033, -10.338788470346913, -11.707324401244254, -8.379465990152688, -11.153340980561529, -8.389010997218097, -4.213637147300446, -11.379466617784779, -8.996642135387448, -11.538642280222692, -12.188110821398995, -11.436265514739722, -12.807720783967595, -7.040987776043918, -11.00747877387102, -12.646219518929207, -10.849228186459724, -9.879471224477012, -10.12042076347174, -4.492855526031828, -8.120168452962742, -8.344281836177522, -9.897299086378228, -7.635785405303011, -5.018187783249168, -10.839113200253632, -9.894524246378385, -6.6675550595673485, -11.066068167567666, -9.33280282202104, -11.626609190558803, -12.399726512558884, -8.851620828312766, -12.433181727580148, -11.490921291079564, -7.975098730753549, -11.317852154144017, -11.991220647516679, -8.018031654797483, -9.61491071361553, -11.466177184006842, -9.923826015248423, -13.449448991689874, -9.794580650634074, -10.644343168408305, -10.485543871854759, -6.666797693394521, -10.714652719908717, -11.1037638933398, -11.950555248927278, -6.593917302554502, -11.095325734079672, -12.243005346817739, -10.556341672568012, -10.51336247754184, -11.843699510970612, -10.217070971800979, -6.7080172823302995, -5.370536967615394, -10.609410986348339, -11.181115505369409, -10.631784990121979, -5.089079384859129, -10.035513909796618, -9.34423749716831, -12.884490072283581, -11.720996625832875, -11.515530282290836, -10.3601910248363, -12.126762937548214, -8.680068944011971, -9.939715783950918, -12.316967220720981, -10.215520096195302, -7.643343494810338, -11.582153546576928, -9.457390857611594, -5.894400341835741, -12.923896368982257, -13.387456750279274, -10.688139377317832, -9.836856238380713, -9.94744002285113, -8.26006435149308, -9.449148215916884, -9.920875388243003, -10.353015799456708, -3.846636785522275, -11.234136922769693, -6.1350342458177, -6.679177715082989, -2.9417235254252105, -13.553948542384402, -7.352429286865583, -13.113767751719282, -9.147279898349383, -12.054883315990857, -13.321726539519881, -10.44488691303026, -8.901605679720387, -12.390997271844935, -6.857433160615642, -12.386960427825683, -5.945422768132882, -8.772763401867145, -8.402852744754044, -12.67252573203924, -10.689706624953267, -4.1864557287045585, -8.440137445655358, -12.227584838900576, -8.153947330390649, -12.721240510441312, -6.535969123728119, -11.802897431638499, -11.939155994728402, -10.196619915086679, -12.692444260823068, -8.605503879837123, -11.29084254415596, -11.422627540596244, -12.173876057711354, -7.327400733307623, -6.473498746044488, -9.751679427919893, -12.098825740084415, -11.55305591696332, -10.61719476895782, -7.400789276392329, -9.024210793748392, -12.279526528151717, -6.245669680248351, -12.136028045927878, -12.55054271585588, -10.032008351266526, -8.89233622920537, -12.941852545084107, -10.102836891269195, -12.365683438468807, -9.997094535679787, -9.497306420243435, -8.816094460832803, -10.696474329199374, -9.443714937559232, -11.618315673231528, -3.3066980926538356, -9.675962858649461, -11.784625115946334, -6.210434777718149, -12.17566649182251, -7.5191775992499785, -13.828124456409235, -12.073659131711208, -11.611954441245226, -12.479739466224535, -10.608397135772542, -13.263594076160661, -5.441604012250277, -11.630328409588177, -11.316049803102516, -12.146485640130631, -11.62655458841775, -9.366661625375938, -6.650429639856391, -2.860429046967873, -6.203879523457431, -9.230088169997252, -13.383725004941635, -5.9912557317216795, -3.8288768085980713, -12.797676260084874, -7.44486148666892, -6.536502350386069, -11.444029467398696, -4.010095282007491, -5.875043841916792, -12.127748065436686, -9.608927148057205, -13.677322887971394, -5.409994939665168, -8.699515319522733, -8.132618262401135, -4.203457902147914, -10.943780178301896, -11.547769245430624, -9.4212024131793, -12.415271561149918, -8.996587691171495, -10.765100749327395, -13.237197286808716, -11.251503248468854, -6.579466117760205, -10.752991760254428, -3.508992182720611, -4.985125965585059, -6.801197187341057, -10.765038622982129, -9.41982568633363, -9.93674636094225, -7.839158316075638, -11.425912010686325, -11.49396628181864, -12.582676260810286, -10.631123346091579, -12.416117225055427, -7.319751543332048, -4.064807970057586, -6.0705090669452275, -8.918500826745229, -5.252582185241529, -8.956910478189855, -12.584181993713429, -10.935624969822202, -10.147222391384576, -7.126956964468224, -13.357587871727322, -10.380163547822303, -8.62139741376241, -12.622745089090284, -12.214494721855193, -13.303356073053637, -10.096378310620178, -12.302276203307954, -12.279294341985047, -13.2032054909527, -11.317852154144177, -11.500518156625754, -11.092176005747962, -13.06285792333022, -7.698441430646291, -13.302258676260122, -2.8951208266497126, -5.817293750160225, -11.799781597138734, -5.730532399780227, -13.306138411257052, -12.034509160691737, -9.312253411456693, -10.598138044171376, -10.377114871047796, -10.827996432598635, -10.620446202546496, -10.599863234288362, -10.819215947082125, -10.306444579463438, -10.79504900900764, -9.958906683342281, -10.066624158305032, -11.384273090335842, -11.840640851095607, -12.262043423155333, -5.634244028051736, -13.0998663699547, -13.410851889679188, -3.6222343912812924, -12.645366738436467, -13.378619270198344, -11.708462884461161, -12.234052219957428, -12.539569051652343, -10.341078357200715, -11.443120319352017, -6.080486370255343, -13.277145381536734, -10.141279751563317, -11.71770544721506, -6.281221449686299, -3.1746075687051416, -7.93085368102304, -4.965895156392094, -2.6675479767938377, -6.921652688029138, -13.968609082683336, -11.185225786308468, -11.989800100388155, -13.901523944377766, -12.773504482638199, -12.67115080019492, -9.679653287696159, -14.167676909572068, -10.754216788188131, -9.137048993819306, -10.090062110516909, -7.863931157012825, -10.788476014227628, -10.376055471405722, -6.548008501667802, -12.077738167353017, -9.360441511611983, -11.414334038848635, -10.109780409965314, -9.125346195310897, -6.266384103307843, -11.449918644969758, -12.62847368892303, -8.64145473833644, -11.738332652857311, -9.798826165433494, -9.727153090269955, -9.30578192350913, -9.087690788454712, -10.693664617271882, -9.436518901022366, -9.895568748449215, -12.893661780661706, -5.602541113314025, -9.45760352577376, -9.098674851056987, -11.598115120882099, -13.375217239843218, -12.11102023136493, -11.679721162075273, -9.426645266663108, -11.775115278609258, -12.617222357462692, -5.841076194478163, -11.747735828262556, -9.960839244646289, -10.792870021641946, -10.063294773564914, -11.132360109437428, -11.664457408195279, -7.6192966653956375, -8.63069029541274, -9.112020719337165, -11.488233331397122, -11.511696292060012, -11.860567967898897, -6.168410856797227, -10.358925796733885, -3.234497197584375, -7.904779017174258, -11.793254147528302, -12.032007013025241, -5.430788476049088, -13.393358846714138, -4.511937698585861, -2.9279514740302766, -8.649591212573618, -11.421171052295291, -8.071531127099112, -4.690304436492244, -9.36844617262874, -6.183960344548929, -12.18407389389452, -8.920494022415973, -3.8288768085980713, -11.547142288115557, -11.34438050700904, -10.215099425201922, -13.12369559376523, -4.985148997500911, -12.407430836066464, -6.662706640715868, -6.1670733939821645, -8.216004159082967, -11.782974846588587, -3.5065872174937844, -14.09052513036319, -10.28510189443224, -11.67689023591716, -6.159686670327538, -9.769393899746273, -12.776220686332104, -9.850019988420584, -11.12861502760324, -12.97317461222045, -10.28155559923262, -12.717234128803025, -9.856271873729238, -6.1268554501544585, -6.774909618534219, -12.148634929506546, -10.802910486194389, -5.663107572825147, -7.714847324631354, -10.484633481673251, -3.9610129064691293, -13.180997308562665, -11.678903524219882, -13.247129976578462, -2.664748284502547, -6.1545289131048975, -10.73510771097954, -11.91052633974548, -3.6362334082836902, -7.727406929589131, -1.3089685670671862, -6.867509299042727, -13.210665943054586, -12.608992152745863, -10.08419847736308, -10.571943233553935, -11.519758623100238, -7.608061198023309, -11.335682099505123, -7.9323877547752755, -13.239351186447447, -11.856367514550213, -10.355728631445587, -6.083553405066702, -10.40357339302606, -13.89079140558263, -10.705827336583916, -7.542986815849805, -4.702506316783989, -12.22569220592482, -13.381724318044082, -11.078376991003868, -4.470894356027337, -10.6912377408303, -10.34022735260093, -6.952753664000596, -13.76852921759267, -10.369385009021123, -10.662073044557614, -9.092991744322202, -12.012754206455659, -10.196351117250106, -10.601122118584012, -6.263886098324285, -12.596502469252728, -8.423837551005946, -10.022500408092228, -13.507503658854262, -10.995209704665134, -12.050620047588488, -9.014030697370213, -8.79254901189053, -6.077625998142739, -11.47515524004817, -7.4203113837681265, -10.592265053991056, -4.284177625585961, -10.997510476292565, -8.079788401373174, -13.158896962042434, -10.305475858684323, -9.597522761969493, -12.45229944858973, -9.088364487302032, -5.772907082407052, -11.561305451003243, -11.230196340547275, -12.8448976020056, -10.896301591543583, -13.975169034107559, -6.936274234747369, -10.520022005934281, -6.797628261091235, -9.505583913098803, -4.387428615231844, -5.81364777476724, -12.707071095400853, -10.673243600846561, -6.331420856792452, -6.062859838249338, -10.783637583049082, -5.295423264085375, -12.066661787131844, -7.551030222006881, -12.399220098679843, -6.374110959163129, -5.526513444660758, -12.856280419761616, -11.59645039137483, -12.466237960927224, -12.516470629276675, -7.976974013889926, -11.962542389923062, -10.29619338765554, -10.736259616029287, -13.434273128972439, -4.333936263741808, -10.75996778906399, -8.516243242919648, -10.04673375409421, -2.272369117879954, -13.160151461079108, -11.041104322690824, -8.111719033782743, -12.146539546220174, -7.510850398652591, -10.323658124658374, -7.345574618201528, -8.924570900151313, -14.196698032577945, -8.089435756025415, -11.290723010618247, -12.10044149674167, -11.604039522631028, -4.087827865383449, -9.694789788869517, -10.893589410482479, -12.636759912802926, -12.766906877246731, -5.401050980837194, -5.7977786891767025, -7.464597432030336, -10.857399969570226, -9.882352724302773, -12.203181820357024, -12.587215457716981, -12.026251775860198, -11.895868648755258, -10.499150320063844, -6.058961279162322, -6.816846497813561, -2.992083527782473, -9.009267040840795, -11.279757805734441, -10.711128580081308, -10.25614401160335, -5.45211213637522, -7.5544449521668575, -11.613891849211623, -7.820005482234446, -12.916025588553183, -11.818449716918723, -11.544886958374288, -9.798642688750562, -6.59391730255598, -10.85160073140545, -11.362465093516532, -9.07339810423027, -11.623927494762112, -11.60163785664164, -12.79929019826898, -13.783017684451165, -10.044163883360767, -10.609590932351622, -3.565953357329626, -9.009202661023789, -8.348468874462963, -2.9128874158817517, -12.43008342794929, -9.732668575732113, -11.109927359942072, -6.568460667773827, -8.749687810117756, -11.475630257313771, -9.871513164964771, -7.885801361698027, -13.325060433743047, -10.384246903836873, -11.810217063331322, -8.476364837859158, -9.729501482570281, -11.147503924911407, -9.460438854597182, -6.3805408904115986, -12.762948258425988, -8.697079151505994, -11.390990992751755, -9.978289253899492, -11.15126766728817, -11.207034819595657, -9.89063077145504, -9.426201722935529, -11.72942708701168, -8.252032641120412, -5.828775850691731, -7.254439064363076, -10.859080887873665, -11.202405623958127, -10.405248746660204, -9.893812651712137, -11.024384289358318, -5.213152917304523, -9.171620301410687, -10.15259414963429, -9.121379032324167, -14.279247619181934, -3.0373604346392042, -4.415266481497898, -10.353513017001816, -11.354428045542491, -10.926246748091469, -9.151782089544866, -11.41763795697913, -5.541803805727653, -8.418907176519598, -11.700300205919381, -12.308599430281044, -11.488632170317677, -8.993206238189812, -14.113201402975221, -7.89090707677884, -11.530497281794068, -10.557351899685122, -6.5521854184275234, -9.49429481254182, -7.078724585273962, -12.588503734061762, -11.440474148878737, -6.4278172794620225, -13.236622128214258, -3.1167036680018114, -7.978235228067543, -10.582169330975123, -9.09589813756068, -13.057432732918663, -11.557067873709777, -13.329875090160513, -4.5475354035193405, -9.010150880927112, -10.159736853981945, -11.68724893642035, -11.352511451554511, -9.268087100121171, -9.40499323342629, -9.84907525060877, -9.618368129477322, -5.487746380608584, -10.38399796713622, -6.872522957753173, -4.495725981364892, -12.232915006854634, -10.319722861519327, -5.028761150480701, -8.313769576289532, -12.292218921148438, -6.6537830833279585, -12.972947658148087, -10.108410430351736, -9.678302475790284, -10.777457159513705, -10.959601307857131, -2.710578388595046, -10.000345751685598, -12.297690129275294, -8.613403546632753, -9.959470346054971, -12.469468694307553, -11.052826562046675, -12.88725373172243, -11.005432651663973, -3.546434404200692, -7.45959879057094, -10.117146655406085, -12.540931433328897, -8.176933600475488, -9.867330657453955, -12.577939077246832, -11.832341119886458, -11.413706255989112, -8.290542192150555, -11.410526524314346, -7.338633178355641, -11.213738397066844, -8.620191370639478, -11.085996666871615, -11.35826717180936, -4.443735450734231, -9.654115533717862, -8.157318787011997, -9.87835803944467, -12.338338469854214, -8.264340658362547, -7.503605010088091, -10.183344380295342, -6.477235755756368, -10.365467150966571, -3.6014627060350257, -11.203659823168303, -13.492111397919032, -10.162222531166856, -7.570452823678471, -7.214298277134011, -7.773042446976396, -10.703349637434501, -10.110518352570262, -13.641435760802162, -5.101033820894635, -11.493385543802109, -8.878572596269258, -12.10532380738541, -12.17565315544249, -7.8588294007669965, -11.33929947972005, -8.741979678149695, -11.476168366451862, -8.050003357390148, -12.862534391623207, -10.44447684105832, -6.020220027095959, -9.383756506017095, -12.258454396015875, -8.10429552275902, -12.734511478334381, -6.956346923382156, -11.16114216247378, -6.6539626134590755, -9.312027334386796, -12.418425836063683, -12.591325479057263, -8.159751597134887, -6.6512304194516325, -10.705007442824057, -13.380320440943697, -11.721704799347272, -12.396078329710088, -11.500853415903856, -6.925467165661004, -12.15159988399387, -10.792394883406704, -10.331969206688832, -9.262447338941971, -2.9356801318334327, -10.706999605158874, -3.228696438595332, -10.766562307871258, -11.06484956439288, -13.486166188626028, -8.758095538782586, -9.597099363690045, -6.716130696506429, -11.90821798421839, -5.921805199449054, -7.213224697205668, -6.63494279684026, -11.309020572687343, -10.656091090120267, -5.492420398330765, -11.06417361077017, -10.818702127143393, -7.253854023749154, -11.82457378663818, -14.234817738332621, -11.642243897942594, -9.278740027949334, -4.12071066795626, -12.178294810565713, -14.194553381906445, -3.219742878106537, -12.802121150075772, -5.708751360748817, -5.842156328320673, -11.868923845250226, -10.60162551494977, -11.196962201806267, -9.952616675715852, -9.936308282286067, -11.368451769938826, -11.71645370020081, -12.117416060549017, -9.681154852449163, -12.270839929766396, -8.098740330950234, -9.262489131517993, -2.4154478123189875, -9.534222717544642, -9.826101835284984, -8.65830718184915, -2.648471319873938, -4.483123486665324, -9.708358590320797, -10.76825596548395, -9.15340964300915, -9.033731498055717, -5.056161020078509, -12.171964784775588, -6.31596050448854, -10.180504605581222, -10.617785461640214, -1.1242300492578714, -12.217535199193366, -3.671796855489273, -10.391625948541439, -4.7644035731532774, -10.693560404929514, -12.479083059398919, -12.767209437833497, -12.279146110470567, -6.524344024423357, -4.693016969083395, -3.152871280893617, -11.561191584746886, -10.335664470645865, -8.672422795148927, -12.242338951063182, -8.552137730670859, -12.237355479761046, -7.617070063547217, -12.138708049700591, -11.94532310859409, -13.412051769728423, -11.083942750279801, -9.29129827922716, -9.869635293987082, -7.56569711536684, -10.203769396205937, -11.801354562215032, -11.531252381616575, -11.39645611127152, -11.671687214059597, -11.366126758715364, -7.598889575310228, -6.266941663667517, -12.016957492058399, -12.553058021443295, -3.641819801453231, -12.63335601629791, -11.743905010384724, -11.83612930339681, -12.893436060598102, -11.85939712428729, -9.555402907419298, -13.486285140452683, -8.334465511402543, -10.742118709045325, -3.744239692234398, -8.243594966676843, -10.979845386376564, -10.355554177608779, -4.288906289978897, -10.369220462635365, -8.165626488360886, -9.724970863608243, -11.020638878776747, -14.023544857563644, -8.13019376517359, -5.7406157444377826, -9.239434445187186, -11.409267677424689, -10.005041828860357, -12.780216568797556, -10.790805899842471, -12.044561889948923, -12.678764460490171, -6.9801117509147765, -9.313391896204147, -10.382172535121596, -10.903368555067543, -8.82355886117475, -9.802814715893733, -9.091845015338189, -12.209948833194705, -8.131828674987041, -11.564099392795244, -10.364746589785117, -13.978181701087685, -14.296334379621982, -12.881721261936512, -11.371513432528962, -13.180243488449818, -9.576815910742216, -13.178269410136451, -11.498641295211007, -9.857308367793923, -12.600611971813686, -10.056313355887749, -13.49262777597764, -9.906062750980983, -7.2248848641828065, -11.102838870889034, -12.078940285243545, -10.806907345671911, -12.057976734718691, -9.8259719761036, -6.120031275757608, -12.230095066780407, -4.5531857281416315, -10.474477799272957, -3.6791473166704947, -8.718881975095544, -11.36453747458038, -7.909302379956202, -8.59551051784061, -5.681548377696192, -8.34322194134832, -12.237928286172735, -12.118839549641761, -13.393484029451097, -7.350848939795922, -12.631877351481446, -8.193647229671695, -9.71985023185443, -10.289371327343696, -11.910129201732532, -13.456127281585958, -11.713485144315895, -11.803973330489177, -11.696782993411167, -9.056470905086343, -11.68647455108222, -8.578007775005688, -11.430203686872291, -11.156951231181191, -6.489983056845471, -8.864144144081836, -11.45747340699547, -10.842081535742121, -9.428270475866697, -12.307582493644622, -10.884628515692633, -12.198737082318925, -9.579792921278244, -7.861511814915703, -6.6232705369099705, -11.008034867489062, -7.501664519599986, -12.130888989514567, -11.316660343168373, -11.808547969536459, -9.73120543618959, -5.664767220832548, -11.333021827926, -7.720685465737649, -12.585849438125553, -10.393908308872835, -11.65070671157115, -12.135060025673015, -11.354335421398444, -11.819216111626574, -11.48909941291615, -7.672879548693929, -10.437150877686229, -7.615488971411722, -12.433959638876738, -7.202352216893525, -12.745513581738217, -8.410442153706128, -12.559616480232062, -9.519659719069132, -10.789120910015715, -12.367561532273065, -11.446755491583327, -10.337810238701806, -11.030052420739253, -9.804725990533205, -8.97035931831014, -4.597494223364631, -11.366659440882856, -11.753532289856697, -12.329177051417243, -10.284321914695402, -13.749869807622705, -12.508907682865367, -6.643825585581972, -10.575734108194707, -8.640655482654926, -9.680092084982764, -8.745925604711323, -10.223530110571188, -7.248920402006284, -12.150795639286088, -10.310496369035597, -6.3914960784758925, -11.15813395485168, -11.18376637465605, -6.458026895686238, -10.746404159427955, -9.853398337248121, -12.314738310969789, -11.27558819381008, -9.005280266248594, -7.730353595004432, -9.153469450151913, -5.023708131249833, -10.284362519921883, -8.902149641090446, -9.001184985261553, -10.30340422280241, -6.206628622504418, -7.76092840037952, -10.431986725818394, -8.574365063263825, -10.845624110193363, -11.471157767361394, -8.087372453712204, -12.746678821204597, -10.497694130538711, -9.8640428906857, -8.427692186255994, -10.709837609137708, -13.950447410016066, -11.454703307138132, -4.564398510038437, -4.776221704613533, -9.036491679172713, -11.494202272645465, -12.621451483681492, -13.027005967545636, -12.417626760372384, -11.814249001555782, -14.230950259108965, -12.601293670275146, -6.729132854767734, -7.6662583876778925, -11.478429769641211, -11.50888803771879, -3.115422059487493, -11.738503162920686, -13.388392736078904, -11.010149698148618, -10.486852959793163, -10.94148578081673, -6.074038430435795, -11.311680068702863, -3.5690623115184508, -9.902996246059114, -9.882778347302702, -10.78693284169621, -7.762941581944277, -12.490376333707015, -6.1263577128463425, -8.414843046653436, -11.685519020965305, -11.75195007891531, -10.888208073719886, -10.749884690095136, -7.783693219317936, -12.376068991934694, -9.925883908404394, -11.564755147196184, -10.91337482559992, -10.925132115130516, -8.564896453610828, -11.02829883739669, -8.416425015969986, -5.27624252276232, -11.993640121290086, -6.188772939923098, -8.982991248355102, -4.449181141571334, -8.411854377018795, -10.380403289218883, -9.442389582111684, -5.97629050092851, -2.5945007559520725, -5.197349050142016, -9.640241073511605, -13.136985685014764, -12.339757089304124, -11.749303677276329, -13.06435338950072, -9.417595481735331, -6.228174874942592, -8.984431899024074, -12.066456134465918, -13.501404194005568, -10.047156574775691, -9.40195803278135, -10.671285072306013, -10.206120106587207, -4.607876289174843, -10.002515964711842, -10.83658644933424, -11.914952505861944, -8.909083134326725, -8.227978569528007, -10.178274204003275, -4.069097811516951, -2.7980686371812418, -10.935981514462844, -8.082858036094859, -5.689924466117805, -11.595407917085687, -3.3685732248024784, -6.431761800690428, -8.692621937552637, -12.525276220797657, -11.192587899676862, -11.161862500837374, -11.935254718465755, -13.48217548049214, -10.829093699647238, -11.348915107277744, -12.5066548912531, -2.9798435146034734, -9.495301888849276, -11.455490895897976, -11.529666550058355, -8.989874944274353, -7.392312660067246, -8.959824696501448, -5.396381296698628, -10.613265188465737, -11.922072451579373, -11.217214645920027, -4.569498946421252, -10.649838011825683, -11.482540042474321, -11.911144562520276, -13.21847823390407, -12.244321546725812, -8.982629871700968, -6.897095581109758, -9.84378987161646, -4.408747445043211, -10.95816150188036, -11.446031738694506, -11.793859835015876, -9.158266967084332, -3.908329897636868, -5.94673393891377, -10.449552207169512, -9.336548137520158, -11.229051647570552, -10.349922456048267, -11.287749450068938, -9.450287795738644, -13.159389049240097, -9.067135271682135, -11.581401883764187, -12.374004716721167, -7.716491792025188, -7.033907327538719, -6.938361157247572, -9.570294880614465, -12.526426747555282, -8.901606050398398, -4.44605569375468, -7.519005737409291, -8.358318417023805, -11.070512839880092, -6.238567279516822, -6.599024730216556, -4.518892382243042, -8.07024258414376, -12.465109005802908, -5.770595139050009, -9.301394405237968, -13.931867240028321, -3.677338137465771, -10.18244970075819, -10.610655103018525, -6.155192480865534, -8.995642136821973, -6.058713155333123, -8.462228119963461, -12.198757318226445, -9.71533349261854, -8.581202776149908, -9.205413696528645, -10.674374449821517, -9.007806220180509, -8.470281404287535, -9.592122382293883, -9.77465693504838, -13.221953164219597, -11.386269616795836, -8.25891716568141, -10.447988911031628, -4.39303357148372, -11.937150481524128, -12.801707289832471, -7.883886024063154, -12.242483388974021, -10.851388099664742, -11.345936591777225, -7.725231943848266, -14.137417339306998, -10.765949523913013, -5.824712220605615, -9.806212161839802, -10.728752649612666, -4.81133174943259, -11.271606411045676, -6.819543479762306, -7.668836360830095, -12.440575197900223, -6.278354463660261, -13.06109168609316, -10.607118943115855, -11.868713486501804, -10.647559312833273, -7.731041450501927, -7.726502953856059, -10.476762776997072, -10.11051835256988, -6.869881569315725, -7.662809659459814, -13.085757233389069, -12.217238548635713, -8.19826526745495, -5.417662379681477, -2.626377325381444, -11.675157153896969, -8.574992577233173, -7.894348320742754, -6.633629495361443, -9.024786081604143, -13.39863613827701, -13.763603141698884, -10.137710438526566, -4.202229406075617, -12.65246602737971, -10.041120259505725, -9.686092660298598, -9.638788268861118, -6.4497390365520175, -3.243952451745482, -12.330452125135823, -11.165827273856205, -11.205290370279705, -11.388115565484119, -5.388095149911397, -7.627019854504525, -9.980259983828093, -10.215020370046801, -13.189857153479599, -8.427302387109137, -10.212050822811554, -12.524273620706579, -12.064205371150276, -9.10304052990069, -4.922375138710887, -9.482601864391945, -7.575576429224974, -8.440022882324667, -9.98816428918548, -7.331592618250452, -11.718674898074347, -8.538686810129386, -7.111873869693008, -11.583492738483523, -12.740267267960178, -6.273132838849062, -5.4284590070639105, -12.721734429785133, -7.760981596397455, -4.205863040525141, -11.49038326606958, -9.207062946546644, -9.572852131586696, -11.109927359942471, -11.24680492109032, -8.373991354789965, -10.748879247173095, -7.813867898188565, -11.814557715860593, -13.375651858295488, -12.273243396170871, -11.217670282790072, -3.4497389625393495, -8.577626580252579, -11.363215841390428, -8.599293808961137, -11.812185351096048, -11.3811220998503, -5.310601934806513, -6.31596050448799, -10.823766730239909, -6.565692865860605, -6.998040991988288, -4.934784344333028, -11.815751472579171, -2.8142303631005134, -6.709560706701545, -11.241525127274873, -7.495525216141525, -12.454856948776005, -11.243037555674228, -10.815429782874793, -6.559837396560525, -8.278460771516869, -11.877653255728374, -12.484303475605513, -8.60161091884953, -10.92819477977812, -5.3877394813838535, -6.887311323602338, -11.846876583931637, -5.980398409090206, -12.841692548052212, -9.00077820466699, -10.37114812995613, -3.736767793708367, -11.695907256570583, -11.552088749687172, -5.573471826273593, -11.113359417301632, -10.266744976873442, -11.733021144527825, -12.121083185335435, -11.076962686646262, -12.151105659080194, -9.712875432652554, -6.026492437532749, -12.34026740996404, -13.4118513444615, -9.947315666553408, -10.818309549442834, -11.694820182233812, -7.895344456846887, -5.809952496508846, -7.910250820130013, -6.643494608118488, -12.104107154468677, -11.407966878536286, -9.949318948417304, -11.293510074749577, -12.255160307081484, -8.117968362990146, -9.034667375942117, -10.762105021039872, -2.4359890875464814, -4.571333230612037, -11.409249439709306, -13.197615878058865, -4.145503637690485, -6.722868571893287, -8.03797288832981, -10.441938245170483, -9.426908309089011, -12.538804450025701, -7.206438300257412, -10.055971723022264, -13.485203007653617, -9.598264910806524, -5.6649742628912705, -7.76092840037952, -8.334327517671843, -9.833514632612287, -11.802680556017059, -14.140127719060908, -8.183461669981071, -10.399246644551166, -12.52753127925618, -9.851140461731482, -6.709207350204142, -9.648215807866904, -8.414421305316587, -11.520288265488619, -11.886779720432065, -9.903236761323805, -11.44044312625019, -12.212012899496493, -11.602792987896157, -9.173220170980507, -10.384034743105815, -11.182945339626226, -7.543840664686648, -11.644378172736886, -11.382500636482147, -10.4208226133361, -11.07850844325077, -7.590862471248357, -8.524351915084583, -10.903454150329399, -3.4234460724456177, -11.357323685125031, -10.430682897941029, -7.21319065634637, -9.04739224847599, -11.904851032988505, -10.662477040347378, -7.4352831501033005, -9.461825418372005, -6.950424203575798, -4.5098530375313, -6.934581768463847, -8.895595505913962, -11.376393720475747, -9.72684027642923, -4.3524492619182595, -10.377766982170948, -10.728853700539705, -10.581017227295295, -12.628397832268767, -9.690954722704914, -11.728307110320877, -10.767617329446088, -8.989452502203974, -9.721031610336654, -14.201007710193615, -8.745292321991622, -8.038523022726338, -4.947038809535616, -11.078149080699736, -8.440137445656399, -11.630264867977637, -6.338402987265038, -9.008250756819299, -11.345697910079657, -8.227605281523404, -8.642488698833361, -9.685369738861304, -3.96234961704992, -10.400841910547014, -4.701078533580304, -5.9441607265063645, -10.032281336166301, -11.304744494463508, -9.86135422554773, -13.172514487383612, -5.7657569492454215, -1.9934994694962676, -8.95574363108853, -14.17299997118844, -2.332056205093604, -10.761625708953382, -9.200617309267503, -10.007595773086205, -10.81291444554331, -10.281755899788328, -9.042460699062772, -12.775566241501807, -10.236888692178042, -8.84406577296955, -3.825408959573352, -4.694927365087004, -6.379949560735, -8.474613237832086, -13.189155802686889, -11.347010875460954, -9.55474253409517, -10.918401022413804, -12.157241049056307, -10.248731557982957, -12.339266720996864, -6.371130981656911, -11.597731662131157, -11.965245610353396, -5.684324884181528, -11.336394445176312, -11.98023550687337, -10.231470517429841, -12.023942369219014, -13.228634489728476, -8.56822803269007, -6.081597282489664, -4.310711538289041, -9.200919117940245, -7.998770945174748, -11.460365269712929, -8.414556714455331, -10.390331124937008, -7.619890402498502, -13.058479384560236, -9.515279449810482, -2.4359890875452024, -11.548103564602584, -8.020556974853974, -10.184368508766129, -10.187605444970368, -10.468541153387546, -11.975661416662962, -12.774241027842818, -4.2604168962894455, -10.560130824605034, -11.149615372276811, -11.316421243481422, -11.453468282120127, -11.44460044128693, -12.058383069410663, -9.591849543857704, -6.465644878728607, -5.613738724452293, -11.329342083820508, -9.689857431525704, -9.386377091155458, -10.898113068171755, -12.162799451923235, -6.404889299876032, -11.590736081448442, -5.527095241345236, -13.16580829106481, -12.36965260593766, -10.316551975173354, -6.346413041209029, -8.480778611707862, -6.648041546784551, -9.336165264326995, -12.25822773873656, -3.546442189918679, -11.560985042402429, -9.4351173160217, -11.158385565144034, -10.800182178440972, -11.71784631305854, -11.538384105253604, -7.7799969894316625, -4.201268804323469, -8.680068944011186, -11.6923135335386, -11.966540146929365, -9.920268580297535, -9.722677666052892, -9.035455618154865, -9.550918223555037, -9.842627614012503, -11.66708614493016, -7.7310483388160485, -12.861239698231676, -10.182518906174945, -14.179605829137856, -8.521333825228604, -5.15402316778551, -8.77006667765368, -11.115105205887792, -5.19617614942873, -5.667390678253842, -8.68281294568714, -9.27364495238288, -12.097924549814788, -8.212435950838449, -10.141775211817968, -11.099859633149187, -9.349990404864723, -9.689977273688408, -8.068586674482441, -11.826805247779777, -10.966279545975846, -11.937420252849575, -11.288900392629786, -6.575216801026183, -11.779769677011743, -13.201812393749112, -4.384528217525091, -5.802726067107642, -12.80116127314165, -8.85529100175033, -10.483543709016816, -12.397388254483305, -8.504467838138007, -9.476093678290864, -7.577969454325455, -9.337828011983431, -9.314882567154614, -5.213796217657929, -13.389843239288666, -11.164574017769219, -10.723717435525186, -8.689826538955163, -10.929642964347938, -10.02280141140092, -9.189890439568478, -8.622534192154566, -11.465902657266845, -13.397855785687472, -7.946157696114733, -9.37746463011498, -10.759092792406591, -9.648073760897757, -9.374020519934946, -10.453263295530212, -10.948413163288459, -6.629977145729583, -4.020508103727817, -11.517788615213329, -9.61189463568838, -12.533661612482694, -9.674673944991817, -10.586427284430506, -6.920066409551497, -12.634917411195788, -6.564009772461954, -5.8501542140327984, -10.08814209045725, -8.910262683674352, -7.869625982793247, -5.094176348012155, -11.388701977082226, -6.342874381675653, -10.298659860681367, -4.663784679774508, -9.611100326952716, -12.939594035393895, -10.312649143416902, -9.173107013740953, -2.686335974636325, -12.530306366743568, -12.395064157852275, -10.333038323565795, -12.625837890962345, -11.992186281426774, -9.807238653235133, -11.874278803304797, -9.720924635921094, -9.017206040515525, -7.655532382401377, -12.381680231398285, -10.284672460201108, -3.925109125232126, -10.10265486496903, -9.215737470418327, -8.709575422078977, -12.238885313188575, -10.796559610856484, -9.976574399376268, -12.405790723340889, -8.094304136353966, -10.505156833262939, -9.970258288124072, -10.838807174982884, -12.090391592075953, -3.9504068992049284, -11.286533021208209, -12.055190418364813, -10.088146511882952, -8.5302870933807, -11.388115565483215, -3.239810997347437, -9.090076651849778, -8.638796977769768, -10.58512901921422, -10.398639033916188, -2.6259695580194915, -11.509597282999032, -4.703295527070146, -12.438749262059472, -10.17827420400357, -5.725524195461464, -12.644739618609908, -11.414098855257008, -9.225214545373456, -9.550324317258214, -10.175781718371677, -10.677827224012644, -9.51999129514694, -9.933629627933993, -12.632971137671802, -5.672416302131822, -2.2255478373907245, -10.96927992141189, -11.075067481378191, -6.544234674623933, -12.5385563694515, -8.300062783170738, -4.061190578170983, -12.819631182270593, -12.09731048017671, -11.903153064213923, -11.658970559661373, -6.974079352208284, -11.205182159383533, -9.239365743402994, -12.241533168010486, -14.062777156850407, -8.332782472363112, -6.454230603336811, -13.198595372604654, -10.104994776555644, -10.57523471746101, -7.3875141073173864, -13.724880728180501, -10.44189106674742, -8.068586674482441, -6.636877684601831, -8.504090891740365, -13.452461312559969, -10.227111711795866, -10.941666843066054, -9.912007730629218, -4.110591640698736, -10.441544200760966, -13.22877722917644, -13.144302940036596, -9.828850192569853, -9.667803405991714, -4.341906269583365, -7.951384609560307, -12.699697047315816, -12.890352105393548, -11.459102405708862, -8.801048283510891, -9.81837379367723, -13.400608927690268, -11.518742025061805, -8.398156194311952, -7.230394139719429, -10.647789679530513, -9.653600116684522, -9.65393666841953, -10.143116227923924, -12.197302260406296, -12.505221067305843, -10.248210192016634, -12.33486136546007, -11.548961862007687, -12.449345152653258, -12.2216022519385, -12.557585804356293, -7.1599559389075615, -10.159257617204142, -8.455854597408885, -10.76913694600149, -11.916456866131426, -9.531544193038442, -12.652086923786644, -11.538267771451608, -11.985482194770244, -8.53773642402202, -8.360425288384544, -6.183703961276144, -3.9336823005216743, -12.0276171542078, -5.75998743480854, -12.540140823095363, -11.062005812465085, -12.640307250146867, -9.008899455733879, -9.192066173751646, -10.969661282763553, -9.951109595404994, -6.172746363505269, -7.548483963854533, -10.910730329952665, -11.545206367832684, -7.793811176046852, -11.210914978906665, -13.447726503802555, -9.916679540379798, -5.359611255956455, -7.70349394405306, -12.39879722624112, -4.34803770449594, -13.898027465574593, -10.709151939031983, -7.085954271062699, -12.242025584172495, -9.399561825890968, -11.185017861672774, -10.567268725522883, -11.090240525020777, -13.1646182021803, -9.882924265590272, -8.91236825229491, -7.597685839820151, -6.865082088655216, -8.669311712768106, -6.6642000356686175, -9.931982164754952, -2.7959885949276284, -11.548103564601996, -12.262035709017107, -6.5894809403286985, -7.966580772715588, -6.973387016641283, -9.125589800272467, -11.698820749953127, -7.2648551064090885, -12.077596807294178, -10.406871693673995, -11.00869177980999, -10.198119764411892, -10.32453328623899, -12.777129853579904, -6.912071600507213, -9.699255053337248, -10.711813830270012, -11.412326434148932, -5.928627789919872, -8.746469330215989, -10.844817127473297, -12.126889898706684, -11.146760386973401, -10.983651794609353, -10.954331630862331, -11.656457669488683, -8.759079613017379, -9.462302369923988, -5.981860050059126, -10.800758122059921, -7.9678246560844705, -10.289156362909237, -7.869059464044726, -6.9718256527746405, -12.531138302281061, -11.499718124300779, -10.007439751509926, -4.433668785287285, -11.879637434638457, -10.15555949759749, -12.147340148556816, -11.527966836898623, -10.299580177522481, -10.016168802932746, -13.129060508889037, -3.940497374876731, -8.629063579958784, -12.805294965289162, -9.2258912587828, -11.684236439729125, -11.260780790602983, -11.57507684480546, -10.805265824381147, -11.141907536019504, -11.143231000303125, -11.673683808952427, -12.846998158706581, -13.29762550664168, -10.141197108294778, -12.173240586047001, -12.786268804257665, -12.13845388850228, -6.103383455291752, -9.808487388105787, -6.449750807082147, -10.566870096466563, -5.519511470497623, -11.405826710674358, -13.423766131778946, -8.787578428599748, -5.3228904907137, -7.77600020310204, -12.023956960168269, -12.283306503471334, -11.31844497224784, -9.930138845867978, -5.926275544830837, -12.408361640990185, -10.52115893710816, -6.0088281457641415, -3.820546924586811, -11.642352849418215, -8.405286486596282, -7.924351101336416, -4.397735168308966, -9.537793554849685, -11.634694409695912, -10.259292351658813, -2.992083527782473, -11.174341814289596, -10.392159252708039, -9.57993570381058, -3.9245636607236634, -11.024965133078783, -9.400269280088661, -12.22310695959624, -12.426957615701005, -12.333963510004766, -12.092990826300191, -12.018401099606544, -11.043785714371666, -8.349100472890502, -9.39416694910222, -4.4649304660270275, -7.760910210096412, -10.44488691303026, -11.129874290157744, -10.097497768951659, -8.963825614930595, -4.595051589407028, -12.043892083145392, -12.483040785367976, -11.940820242886817, -11.297437004010439, -9.761716860091855, -9.630816762467834, -8.570383213821598, -11.84289973401452, -12.250378706501015, -11.444907492738448, -11.618369997906868, -8.086624885428135, -12.455041217530326, -2.6240531348112484, -10.510873349864557, -8.010086589694021, -12.472701398572529, -11.475127237898086, -12.495838590118145, -12.55896699315576, -10.951170816357479, -4.073414252453571, -12.949777981924177, -7.627003029642529, -9.261079101973271, -8.955332914929565, -9.10425013752209, -9.89770474462504, -12.95879603440826, -7.435507211885691, -6.330307676657885, -7.62967069266853, -10.747679719772952, -9.454868121058109, -8.531746740165957, -8.622767670855206, -5.403371059629297, -11.418825118801001, -5.629727267624194, -13.727665878575507, -8.629855740064476, -9.253455833719208, -9.704618514287855, -9.42126077718611, -9.096987927058878, -8.764953336435687, -12.75037614645457, -9.826674208550413, -11.663097439035841, -14.10030487892338, -9.860007364922925, -10.210188951178646, -10.461357404095676, -12.377207553360227, -12.203472085175017, -7.905986881645518, -4.25365402656837, -10.919146191680621, -7.197999316055084, -9.607429291332991, -10.714283790652832, -9.993927556110304, -11.660444088629353, -10.438289814957672, -11.505269561084354, -12.502478593992647, -11.153390105180861, -11.040051661444458, -12.503101647707847, -9.473350729981876, -9.864447226577774, -7.428798929945174, -9.002861082557228, -9.456001198575125, -9.653649549335409, -13.34330343004948, -9.214050912252642, -12.427336859291762, -9.928391855906593, -3.5294405275042635, -2.9249991455369972, -8.379078722488797, -10.412049081105451, -11.433948336751598, -9.27094427635572, -8.401061534611392, -5.89497914158207, -6.446028688108321, -11.366942984307672, -3.4979672212307342, -11.600498274787405, -8.620672664517345, -10.74042624479389, -7.135418014207657, -9.250090356811695 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "showlegend": false, "type": "scattergl", "x": [ 4.139999999999805, 4.459999999999798, -1.670000000000071, -1.670000000000071, 1.659999999999858, 1.659999999999858, -5, -5, 4.989999999999787, 4.989999999999787, -1.8700000000000667, -0.050000000000105516, 2.9499999999998305, 4.289999999999802, -1.7400000000000695, 3.4699999999998195, 1.8899999999998531, 2.1199999999998482, -3.0100000000000424, -0.7600000000000904, 4.3599999999998005, -4.000000000000021, 0.6999999999998785, 1.3699999999998642, -0.11000000000010424, -0.11000000000010424, 3.059999999999828, 3.2399999999998244, 2.6299999999998374, 2.889999999999832, 3.779999999999813, -1.4900000000000748, -4.230000000000016, 4.709999999999793, -3.460000000000033, 2.219999999999846, 2.5699999999998386, 0.18999999999988937, 1.0799999999998704, 1.9299999999998523, -4.55000000000001, 1.2699999999998663, 1.189999999999868, 4.049999999999807, 3.2799999999998235, -2.410000000000055, 1.6199999999998589, 4.6399999999997945, 0.20999999999988894, -3.5200000000000315, 1.0599999999998708, -1.5100000000000744, -0.1400000000001036, 2.699999999999836, -3.8000000000000256, -4.7400000000000055, -0.7000000000000917, -0.3100000000001, -0.19000000000010253, 3.109999999999827, 1.4499999999998625, -0.16000000000010317, -0.09000000000010466, -1.0200000000000848, -2.1500000000000608, -0.18000000000010274, -3.8900000000000237, -4.890000000000002, 3.2399999999998244, -3.8500000000000245, -4.670000000000007, 2.3799999999998427, 4.3899999999998, 2.2999999999998444, -0.3100000000001, 3.969999999999809, -1.5500000000000735, -2.4800000000000537, 2.219999999999846, -1.5000000000000746, 4.86999999999979, 1.3099999999998655, -3.7100000000000275, 4.7799999999997915, 2.449999999999841, -3.8000000000000256, 3.309999999999823, -0.6800000000000921, -2.410000000000055, 1.849999999999854, -2.460000000000054, -2.240000000000059, -0.050000000000105516, -1.0700000000000838, -2.3000000000000576, -0.5400000000000951, -4.1800000000000175, 3.0999999999998273, 3.819999999999812, -3.460000000000033, 2.139999999999848, 2.3899999999998425, 4.729999999999793, -2.2100000000000595, 2.9899999999998297, 4.909999999999789, 2.219999999999846, 0.9799999999998725, -0.8600000000000882, 1.2199999999998674, -4.560000000000009, 1.0799999999998704, 4.149999999999805, -3.080000000000041, 1.4499999999998625, 4.019999999999808, -2.7200000000000486, 4.749999999999792, -0.8800000000000878, 3.729999999999814, 3.4599999999998197, -2.3500000000000565, 3.669999999999815, 4.579999999999796, 2.4799999999998406, 2.03999999999985, -0.0600000000001053, -3.60000000000003, -4.1800000000000175, -3.2300000000000377, 0.0899999999998915, -1.1600000000000819, 0.12999999999989065, 4.809999999999791, 0.62999999999988, -2.1200000000000614, 0.6099999999998804, -1.4600000000000755, 1.8599999999998538, -2.3400000000000567, 4.009999999999808, 3.7399999999998137, -2.0300000000000633, -4.090000000000019, -1.72000000000007, -1.900000000000066, 4.339999999999801, 3.5999999999998167, -1.4000000000000767, -1.010000000000085, 3.669999999999815, 2.6599999999998367, -3.880000000000024, 1.8199999999998546, -3.6200000000000294, -0.25000000000010125, -0.09000000000010466, -0.9800000000000857, -1.9900000000000642, -0.8000000000000895, -4.910000000000002, 1.609999999999859, -4.900000000000002, -1.6600000000000712, -4.270000000000016, -4.120000000000019, 3.2399999999998244, 3.119999999999827, 4.659999999999794, -1.2300000000000804, -4.850000000000003, 0.8299999999998757, -0.5000000000000959, 0.6199999999998802, 4.259999999999803, -3.180000000000039, -0.3600000000000989, 3.109999999999827, -4.590000000000009, -0.23000000000010168, 3.309999999999823, -0.6700000000000923, -0.4400000000000972, -2.570000000000052, -2.420000000000055, -4.52000000000001, 0.5199999999998823, 3.389999999999821, -0.8900000000000876, 1.6699999999998578, 2.309999999999844, 0.7199999999998781, -4.100000000000019, 4.879999999999789, 2.549999999999839, 3.44999999999982, -1.480000000000075, -1.7000000000000703, 3.8299999999998118, -1.9400000000000652, 4.729999999999793, -4.430000000000012, -1.900000000000066, 3.1499999999998263, -3.8000000000000256, -4.110000000000019, 3.539999999999818, 2.839999999999833, 0.8899999999998744, 1.9299999999998523, 1.4199999999998631, 3.999999999999808, -4.940000000000001, -0.4300000000000974, 0.2399999999998883, 4.569999999999796, 0.7899999999998766, 3.349999999999822, -1.8500000000000671, -4.270000000000016, 2.1499999999998476, -4.470000000000011, -4.810000000000004, 0.009999999999893205, 4.949999999999788, -0.5500000000000949, 0.3099999999998868, -4.880000000000003, -4.710000000000006, -3.870000000000024, 2.8599999999998325, -0.5000000000000959, 0.8699999999998749, -1.6800000000000708, 3.059999999999828, -4.800000000000004, 0.21999999999988873, 0.34999999999988596, 4.199999999999804, 2.8999999999998316, -1.6900000000000706, 4.429999999999799, -0.8300000000000889, -0.33000000000009955, 2.9399999999998307, 0.8299999999998757, 4.229999999999803, 2.089999999999849, 2.319999999999844, 2.549999999999839, 1.0399999999998712, 4.759999999999792, 3.639999999999816, -4.020000000000021, -3.2500000000000373, -1.860000000000067, 1.5299999999998608, 4.719999999999793, -4.1800000000000175, -2.0600000000000627, 1.5299999999998608, -0.16000000000010317, 1.939999999999852, -0.39000000000009827, -4.140000000000018, 3.3799999999998214, -2.320000000000057, 1.0599999999998708, 4.649999999999794, 1.1699999999998685, -3.260000000000037, -3.9000000000000234, -3.450000000000033, -3.040000000000042, -0.0600000000001053, -1.810000000000068, 1.0199999999998717, 0.17999999999988958, -1.0400000000000844, -4.430000000000012, -2.2100000000000595, -2.600000000000051, 1.469999999999862, 3.44999999999982, -3.6700000000000284, 0.579999999999881, 4.909999999999789, 1.5099999999998612, -4.430000000000012, 0.26999999999988766, 4.719999999999793, -3.5100000000000318, -4.490000000000011, -2.990000000000043, 2.0999999999998487, 4.909999999999789, -4.770000000000005, -2.330000000000057, 3.579999999999817, -0.23000000000010168, 1.6499999999998582, -0.47000000000009656, -1.3600000000000776, -3.640000000000029, 2.589999999999838, 1.6199999999998589, -1.010000000000085, 0.4699999999998834, -4.150000000000018, 0.34999999999988596, 3.6899999999998148, -0.730000000000091, 3.769999999999813, 3.999999999999808, -1.1300000000000825, 2.359999999999843, 4.099999999999806, 3.3799999999998214, -3.6100000000000296, 1.8599999999998538, 4.0299999999998075, 4.599999999999795, 1.009999999999872, -3.3800000000000345, 4.709999999999793, 3.169999999999826, -2.2100000000000595, -4.800000000000004, 0.49999999999988276, -4.220000000000017, -1.9100000000000659, -4.340000000000014, 2.02999999999985, -1.3000000000000789, -1.5000000000000746, 0.009999999999893205, -2.790000000000047, 0.42999999999988425, -1.010000000000085, 0.42999999999988425, 4.889999999999789, -4.05000000000002, 3.3699999999998216, 4.529999999999797, 4.739999999999792, -4.280000000000015, -3.460000000000033, 4.669999999999794, -3.8600000000000243, -3.1900000000000386, -4.210000000000017, 2.639999999999837, -3.3900000000000343, -1.0900000000000833, 1.2199999999998674, 1.4499999999998625, -0.1300000000001038, 2.739999999999835, -3.450000000000033, -0.12000000000010402, -0.9500000000000863, -4.350000000000014, 2.5399999999998393, -2.330000000000057, -0.32000000000009976, 2.889999999999832, -2.940000000000044, -0.6400000000000929, -2.5900000000000514, -2.3600000000000563, 0.43999999999988404, -0.8200000000000891, 4.239999999999803, 0.11999999999989086, -1.8800000000000665, 2.699999999999836, 4.259999999999803, 3.4199999999998205, 3.779999999999813, -1.7300000000000697, -1.1300000000000825, 4.339999999999801, -4.260000000000016, -1.8200000000000678, 4.519999999999797, -2.0300000000000633, -3.6100000000000296, 0.5199999999998823, 0.17999999999988958, 4.279999999999802, 1.6799999999998576, 2.929999999999831, 1.7899999999998553, -2.5000000000000533, 2.4699999999998408, 3.2699999999998237, 1.4099999999998634, -1.8400000000000674, -0.23000000000010168, -2.0700000000000625, -4.240000000000016, -4.310000000000015, -0.020000000000106155, 4.609999999999795, -0.49000000000009614, -0.010000000000106368, -2.4500000000000544, -4.300000000000015, -2.520000000000053, -1.1400000000000823, 3.0899999999998276, 0.8099999999998762, -1.430000000000076, -1.5900000000000727, -0.4600000000000968, 2.1999999999998465, 2.4399999999998414, 3.6999999999998145, -3.410000000000034, 1.0599999999998708, -2.4000000000000554, -1.8000000000000682, -2.0800000000000622, 2.5199999999998397, -3.2500000000000373, -0.4300000000000974, 4.509999999999797, 0.5199999999998823, 1.7699999999998557, 0.2399999999998883, -1.3800000000000772, -2.66000000000005, 1.9099999999998527, -2.790000000000047, -4.370000000000013, -3.790000000000026, -4.9300000000000015, -3.410000000000034, 0.8799999999998747, 2.0699999999998493, -1.6300000000000718, 1.469999999999862, -1.340000000000078, -2.050000000000063, -3.830000000000025, 0.5199999999998823, 4.679999999999794, 0.7499999999998774, -3.790000000000026, 0.11999999999989086, 1.799999999999855, 4.069999999999807, -3.160000000000039, 0.9299999999998736, 1.5999999999998593, 0.18999999999988937, -0.6500000000000927, 3.539999999999818, -4.610000000000008, -2.840000000000046, -3.4400000000000333, 3.499999999999819, 2.1899999999998467, 0.7599999999998772, 0.8199999999998759, -4.960000000000001, -3.9000000000000234, 2.889999999999832, 0.8499999999998753, 4.0799999999998064, -4.920000000000002, 3.679999999999815, -1.4600000000000755, -0.11000000000010424, 2.7999999999998337, 4.2199999999998035, -1.430000000000076, 0.4799999999998832, 2.2599999999998452, -2.65000000000005, 0.04999999999989235, 4.819999999999791, 4.979999999999787, -3.830000000000025, -1.4200000000000763, -3.8000000000000256, -3.550000000000031, 2.8499999999998327, -0.8200000000000891, 4.419999999999799, 2.309999999999844, -1.3300000000000782, -3.5600000000000307, 4.279999999999802, -1.6500000000000714, -3.1100000000000403, -3.160000000000039, 0.6599999999998793, -3.5100000000000318, 2.8999999999998316, -1.670000000000071, -0.39000000000009827, 4.069999999999807, 1.9799999999998512, -4.500000000000011, -3.8500000000000245, 3.1399999999998265, 3.639999999999816, -2.890000000000045, 2.0499999999998497, 2.6299999999998374, 4.319999999999801, 4.2199999999998035, -4.100000000000019, 4.899999999999789, 2.229999999999846, -1.3200000000000784, 0.49999999999988276, 2.0599999999998495, 0.42999999999988425, 0.20999999999988894, 4.1699999999998045, -1.760000000000069, -1.6000000000000725, 0.8799999999998747, 1.5899999999998595, 1.2299999999998672, -3.2800000000000367, -2.8100000000000467, 2.8699999999998322, -0.2900000000001004, 1.329999999999865, 4.739999999999792, 4.649999999999794, -4.370000000000013, -2.5000000000000533, -2.9500000000000437, 2.8499999999998327, -4.210000000000017, 3.90999999999981, 3.4599999999998197, -0.730000000000091, -3.8500000000000245, 0.42999999999988425, -0.6900000000000919, -0.32000000000009976, -3.3800000000000345, -2.380000000000056, -1.200000000000081, 3.719999999999814, 2.3399999999998435, -2.510000000000053, -0.9900000000000855, -3.9400000000000226, 4.439999999999799, -0.26000000000010104, -4.98, 2.7699999999998344, -2.5800000000000516, -0.33000000000009955, -2.7800000000000473, 3.8299999999998118, 3.119999999999827, -3.160000000000039, 1.1999999999998678, -2.0200000000000635, -0.41000000000009784, -3.3400000000000354, 3.209999999999825, -4.590000000000009, 2.589999999999838, 2.369999999999843, 1.5599999999998602, 0.9799999999998725, -2.140000000000061, -1.5700000000000731, -0.08000000000010488, -2.320000000000057, 2.9899999999998297, -4.910000000000002, 2.319999999999844, -0.9300000000000868, -2.940000000000044, 4.479999999999798, -0.07000000000010509, -4.370000000000013, -4.560000000000009, 0.6099999999998804, -3.7200000000000273, -0.8800000000000878, 3.90999999999981, 4.899999999999789, 2.9899999999998297, -1.4200000000000763, 3.6199999999998163, -3.780000000000026, -4.710000000000006, -1.2600000000000797, -2.2200000000000593, -0.6100000000000936, 2.359999999999843, -1.4900000000000748, -0.42000000000009763, 2.9399999999998307, 3.7999999999998124, -4.660000000000007, -1.8500000000000671, -4.0400000000000205, 4.769999999999792, 0.8499999999998753, -2.3100000000000573, -4.7400000000000055, 2.6799999999998363, -4.610000000000008, -0.39000000000009827, -0.6700000000000923, -1.0000000000000853, 3.389999999999821, 0.41999999999988447, -4.54000000000001, -3.0500000000000416, -2.4000000000000554, 2.359999999999843, 4.709999999999793, -4.960000000000001, 3.389999999999821, 0.9499999999998732, -4.9300000000000015, 3.199999999999825, -1.3100000000000787, -2.18000000000006, -2.2200000000000593, -3.6700000000000284, 1.7699999999998557, 3.209999999999825, 2.309999999999844, 0.5199999999998823, -1.4200000000000763, -0.7100000000000914, 3.259999999999824, -4.670000000000007, 0.43999999999988404, -0.5500000000000949, -0.4400000000000972, 4.2199999999998035, -0.20000000000010232, -2.560000000000052, 2.559999999999839, 0.06999999999989193, -3.2000000000000384, -2.9200000000000443, 3.1499999999998263, 4.299999999999802, -2.5000000000000533, -4.810000000000004, -2.6300000000000505, -3.820000000000025, -3.9100000000000232, 1.759999999999856, 0.3299999999998864, -1.2600000000000797, 2.4699999999998408, 3.8799999999998107, 0.769999999999877, 4.3699999999998, 0.22999999999988852, 0.42999999999988425, 2.079999999999849, 1.8399999999998542, -4.660000000000007, 3.1299999999998267, -4.430000000000012, -0.5500000000000949, -2.4400000000000546, 3.3299999999998224, 0.34999999999988596, -2.1700000000000603, 1.5099999999998612, -2.4500000000000544, -1.1600000000000819, 2.919999999999831, -0.5700000000000944, -3.7000000000000277, 0.2499999999998881, 0.8599999999998751, -2.2100000000000595, -3.3900000000000343, 1.1999999999998678, 2.359999999999843, -3.490000000000032, -1.860000000000067, 4.819999999999791, -3.5300000000000313, -0.10000000000010445, -4.230000000000016, -3.3400000000000354, 0.07999999999989171, -0.33000000000009955, 3.499999999999819, -2.8100000000000467, -3.9500000000000224, 4.8299999999997905, 4.119999999999806, -4.370000000000013, -3.780000000000026, 0.1699999999998898, -1.4000000000000767, 3.5199999999998184, -3.2000000000000384, 3.109999999999827, -0.9100000000000872, 4.159999999999805, -4.54000000000001, 3.779999999999813, -2.890000000000045, 1.5299999999998608, -2.940000000000044, 2.309999999999844, -1.7900000000000684, -4.410000000000013, 3.6499999999998156, 0.9999999999998721, -3.1500000000000394, 3.0799999999998278, 3.4699999999998195, -4.4600000000000115, 2.319999999999844, -0.24000000000010147, -3.550000000000031, -1.9400000000000652, 0.5099999999998825, -0.18000000000010274, -0.050000000000105516, -4.6000000000000085, -3.350000000000035, -0.920000000000087, 2.359999999999843, 2.5799999999998384, -3.8000000000000256, -2.9100000000000446, 4.419999999999799, -0.07000000000010509, 2.6299999999998374, -3.080000000000041, 4.909999999999789, -3.4200000000000337, -1.530000000000074, -3.970000000000022, 3.0799999999998278, 4.889999999999789, 0.6199999999998802, 4.899999999999789, -4.020000000000021, -1.150000000000082, 2.4399999999998414, 4.409999999999799, -2.6300000000000505, -2.5800000000000516, -1.110000000000083, -2.280000000000058, -1.8000000000000682, 2.699999999999836, -4.920000000000002, 4.429999999999799, -2.420000000000055, 1.7799999999998555, 1.3399999999998649, 4.439999999999799, 4.429999999999799, 4.909999999999789, 3.219999999999825, 1.899999999999853, -3.410000000000034, -3.7600000000000264, -0.7100000000000914, -1.3000000000000789, -0.4400000000000972, -1.5000000000000746, 0.3199999999998866, 4.229999999999803, -0.11000000000010424, -4.53000000000001, 2.3399999999998435, -2.5400000000000524, -1.200000000000081, 1.609999999999859, 1.2999999999998657, -1.8500000000000671, 1.709999999999857, -3.2300000000000377, -2.1200000000000614, -0.010000000000106368, -1.5900000000000727, -2.9100000000000446, -0.9900000000000855, 0.6699999999998791, -4.680000000000007, -4.840000000000003, -0.23000000000010168, -0.8500000000000885, -4.840000000000003, -4.630000000000008, -0.010000000000106368, -3.630000000000029, 0.06999999999989193, 0.7299999999998779, -3.220000000000038, -3.320000000000036, -1.3600000000000776, 3.299999999999823, 2.359999999999843, 3.529999999999818, 4.479999999999798, 2.169999999999847, -3.3000000000000362, -4.210000000000017, -1.430000000000076, -1.5400000000000738, 0.12999999999989065, 2.9999999999998295, -4.210000000000017, -3.070000000000041, 0.3199999999998866, -0.49000000000009614, -1.900000000000066, 3.6099999999998165, 2.139999999999848, -3.3000000000000362, 1.1199999999998695, -4.340000000000014, 0.7999999999998764, 2.179999999999847, 1.3999999999998636, 0.10999999999989107, -3.8900000000000237, -0.2100000000001021, -2.9600000000000435, -0.050000000000105516, 1.4499999999998625, -3.6200000000000294, -4.7400000000000055, -3.5700000000000305, -1.670000000000071, 0.42999999999988425, -0.49000000000009614, 4.799999999999791, 0.8099999999998762, 3.059999999999828, 3.4699999999998195, -2.740000000000048, 2.7699999999998344, 1.4799999999998619, -4.870000000000003, 3.5599999999998175, -4.480000000000011, 0.9799999999998725, -1.580000000000073, 0.8499999999998753, -1.4600000000000755, -0.730000000000091, -4.160000000000018, 2.4799999999998406, 3.029999999999829, -0.2700000000001008, -4.210000000000017, -0.39000000000009827, -1.620000000000072, 3.7099999999998143, 2.639999999999837, -0.6300000000000932, 4.7799999999997915, -0.5100000000000957, -4.4600000000000115, 4.879999999999789, 4.509999999999797, 3.1799999999998256, -4.390000000000013, -2.4500000000000544, 4.129999999999805, -1.0000000000000853, -2.840000000000046, -0.41000000000009784, -2.410000000000055, -0.4300000000000974, 2.559999999999839, 2.5799999999998384, -0.870000000000088, 3.359999999999822, 4.599999999999795, -2.840000000000046, 1.289999999999866, -1.5100000000000744, -2.4900000000000535, -0.7500000000000906, -4.820000000000004, 3.5699999999998173, 1.609999999999859, -4.780000000000005, -1.3700000000000774, 1.609999999999859, -3.360000000000035, 2.2899999999998446, -2.0300000000000633, -3.3700000000000347, 2.50999999999984, 2.9899999999998297, -2.5900000000000514, -3.630000000000029, -4.0400000000000205, 4.989999999999787, 4.629999999999795, 4.889999999999789, 1.1199999999998695, -3.630000000000029, -3.260000000000037, -4.1800000000000175, 2.749999999999835, -3.6100000000000296, -2.790000000000047, 1.3599999999998644, -4.700000000000006, 4.189999999999804, 1.4199999999998631, 4.159999999999805, -1.6500000000000714, -1.6400000000000716, -3.7100000000000275, -2.940000000000044, 0.2399999999998883, -4.300000000000015, -0.6100000000000936, 0.7599999999998772, 0.019999999999892992, 0.4799999999998832, -3.030000000000042, 0.2399999999998883, 2.829999999999833, 0.06999999999989193, 0.49999999999988276, 4.6399999999997945, 4.229999999999803, 4.519999999999797, -1.4000000000000767, -4.98, -1.2100000000000808, -3.59000000000003, 3.109999999999827, -0.5000000000000959, -4.610000000000008, 4.459999999999798, 3.90999999999981, -3.930000000000023, 3.2799999999998235, 4.109999999999806, 2.96999999999983, 3.9399999999998094, -2.6300000000000505, 0.3999999999998849, -1.0658141036401503e-13, -0.5800000000000942, 4.239999999999803, 0.8399999999998755, -4.05000000000002, 0.22999999999988852, 4.2199999999998035, -2.9200000000000443, -0.2100000000001021, 1.6899999999998574, 3.5599999999998175, -1.4100000000000765, 4.809999999999791, 2.6799999999998363, 0.5899999999998808, -3.4300000000000335, -1.9600000000000648, 2.5399999999998393, 2.3299999999998438, 4.419999999999799, 1.289999999999866, -4.5100000000000104, 1.429999999999863, 4.049999999999807, -2.040000000000063, 2.6599999999998367, -1.5700000000000731, -4.280000000000015, -1.4900000000000748, 4.559999999999796, 1.3999999999998636, 2.589999999999838, 0.04999999999989235, 4.289999999999802, -3.460000000000033, -3.2900000000000365, -4.840000000000003, -2.790000000000047, -3.680000000000028, 3.9399999999998094, 3.159999999999826, -2.6700000000000497, -0.7200000000000912, 4.039999999999807, -1.3300000000000782, 2.4699999999998408, -3.8900000000000237, -4.690000000000007, -3.7000000000000277, -0.5400000000000951, 0.3199999999998866, -0.5300000000000953, -0.010000000000106368, -3.9800000000000217, 3.529999999999818, 2.9499999999998305, -4.350000000000014, -3.220000000000038, -1.9900000000000642, -0.3800000000000985, -3.3700000000000347, 2.0599999999998495, -1.5200000000000742, 0.1699999999998898, -2.240000000000059, 3.3799999999998214, -4.05000000000002, -0.25000000000010125, -1.6900000000000706, 1.9799999999998512, 3.059999999999828, -1.3500000000000778, -0.12000000000010402, -3.7500000000000266, -1.4500000000000757, -1.6000000000000725, -3.780000000000026, -0.7500000000000906, 3.669999999999815, 4.299999999999802, 2.3799999999998427, -4.920000000000002, -3.4700000000000326, -3.9500000000000224, -1.3100000000000787, -1.4700000000000752, -4.98, 2.5199999999998397, 0.5599999999998815, 0.8499999999998753, 1.989999999999851, -3.070000000000041, 3.309999999999823, 3.2699999999998237, -4.52000000000001, 4.909999999999789, -4.670000000000007, 4.1699999999998045, -1.9800000000000644, 0.02999999999989278, -2.7600000000000477, -4.800000000000004, 0.7799999999998768, -1.900000000000066, -4.7900000000000045, -3.320000000000036, 3.5699999999998173, 3.249999999999824, 1.9999999999998508, -3.540000000000031, -0.20000000000010232, -3.12000000000004, 2.7299999999998352, -3.7600000000000264, -1.200000000000081, -4.810000000000004, 0.7599999999998772, -2.230000000000059, -4.330000000000014, 4.4999999999997975, -3.160000000000039, 3.6499999999998156, 1.659999999999858, -4.850000000000003, 4.1699999999998045, 1.1399999999998691, 3.8799999999998107, 4.339999999999801, -2.990000000000043, -2.740000000000048, -2.9200000000000443, 0.8899999999998744, -2.1200000000000614, 1.2499999999998668, 3.0999999999998273, -0.870000000000088, -0.34000000000009933, 1.2999999999998657, 2.179999999999847, 1.3499999999998646, -0.2200000000001019, 1.239999999999867, 1.9099999999998527, 3.91999999999981, 2.279999999999845, -1.3200000000000784, 3.779999999999813, -3.400000000000034, -0.8600000000000882, 0.909999999999874, -4.250000000000016, 4.599999999999795, 4.109999999999806, 4.059999999999807, -1.4900000000000748, 3.0399999999998286, 0.6699999999998791, -0.7000000000000917, 2.829999999999833, 2.139999999999848, -1.8700000000000667, 1.0699999999998706, 3.309999999999823, 0.7999999999998764, -0.2900000000001004, -3.2500000000000373, -3.1500000000000394, 1.9099999999998527, 2.6199999999998376, 0.5299999999998821, -0.7900000000000897, 2.369999999999843, 3.6499999999998156, 4.85999999999979, -2.2900000000000578, 2.789999999999834, 3.5699999999998173, 3.7499999999998135, 0.9999999999998721, 4.739999999999792, -0.3000000000001002, -1.6100000000000723, 4.549999999999796, -4.910000000000002, 2.639999999999837, -0.17000000000010296, -1.390000000000077, -2.100000000000062, -4.020000000000021, 2.0999999999998487, 3.769999999999813, 0.579999999999881, 2.1499999999998476, -0.7200000000000912, -0.450000000000097, 4.909999999999789, -4.6000000000000085, -3.9100000000000232, -2.040000000000063, 0.8399999999998755, -0.9100000000000872, 4.149999999999805, -2.890000000000045, 0.6699999999998791, 1.659999999999858, 0.9899999999998723, -1.0800000000000836, -4.9300000000000015, 2.96999999999983, 0.06999999999989193, 3.159999999999826, -4.220000000000017, -0.7200000000000912, 4.2199999999998035, -4.06000000000002, -4.620000000000008, 3.4299999999998203, 1.4599999999998623, -2.1600000000000605, -1.2200000000000806, 0.009999999999893205, 4.679999999999794, -2.6300000000000505, -0.4600000000000968, 0.5599999999998815, 0.3099999999998868, 3.8799999999998107, 1.1799999999998683, 2.0099999999998506, -0.7200000000000912, -1.0200000000000848, -2.8200000000000465, 0.5599999999998815, -0.18000000000010274, 4.729999999999793, 0.9799999999998725, 3.579999999999817, 1.6899999999998574, -0.9000000000000874, 2.50999999999984, -1.4500000000000757, -0.3800000000000985, 2.7199999999998354, 3.8399999999998116, -2.7800000000000473, -3.3000000000000362, -4.200000000000017, 2.6199999999998376, 1.2099999999998676, -3.3700000000000347, 3.2399999999998244, 3.7399999999998137, 3.539999999999818, -2.9700000000000433, 1.7199999999998568, -1.1200000000000827, -4.98, -4.680000000000007, -1.2300000000000804, -4.830000000000004, -1.6100000000000723, -0.8500000000000885, -2.1700000000000603, 4.409999999999799, 4.3599999999998005, 0.7099999999998783, -3.8100000000000254, -3.540000000000031, -0.730000000000091, -1.950000000000065, -2.2200000000000593, 2.7599999999998346, -3.1000000000000405, 0.3199999999998866, -4.6000000000000085, -1.4000000000000767, -2.6900000000000492, -2.000000000000064, 2.02999999999985, -2.520000000000053, 1.4199999999998631, -2.9100000000000446, 2.0999999999998487, 1.8399999999998542, 4.889999999999789, -3.3900000000000343, 0.2399999999998883, -1.3000000000000789, -0.9400000000000865, 4.459999999999798, 0.42999999999988425, -2.6200000000000507, -1.2400000000000801, -2.6200000000000507, -4.53000000000001, -4.710000000000006, 2.7699999999998344, -0.7000000000000917, 4.789999999999791, 3.779999999999813, 2.609999999999838, -3.0000000000000426, 1.7799999999998555, -0.050000000000105516, 4.9699999999997875, 4.3999999999998, 2.3499999999998433, -4.920000000000002, -0.9800000000000857, 2.0199999999998504, 2.0499999999998497, 2.5199999999998397, 4.339999999999801, 2.559999999999839, 2.4699999999998408, -2.850000000000046, -4.210000000000017, -3.1400000000000396, -0.16000000000010317, -2.1700000000000603, -1.860000000000067, 1.2599999999998666, 0.7299999999998779, -0.9500000000000863, -1.2100000000000808, -1.6400000000000716, 0.5999999999998806, -3.7000000000000277, 2.699999999999836, 1.609999999999859, -1.9200000000000657, -4.330000000000014, -1.0000000000000853, -1.0900000000000833, 3.4799999999998192, -1.7100000000000701, 2.2999999999998444, -1.0658141036401503e-13, 1.09999999999987, 2.4299999999998416, -3.880000000000024, 2.779999999999834, 2.369999999999843, 4.099999999999806, 3.969999999999809, 3.6099999999998165, -0.16000000000010317, -1.8800000000000665, 0.27999999999988745, -2.6700000000000497, -4.940000000000001, 4.649999999999794, 0.07999999999989171, -0.8000000000000895, -0.48000000000009635, -0.49000000000009614, 1.239999999999867, -0.3800000000000985, 4.929999999999788, -0.3800000000000985, -1.9600000000000648, 4.409999999999799, -0.030000000000105942, -4.660000000000007, -4.470000000000011, -1.6300000000000718, -1.1900000000000812, 1.1699999999998685, -4.700000000000006, 2.7999999999998337, -2.740000000000048, -2.3100000000000573, -1.7800000000000686, -1.2400000000000801, 4.949999999999788, -1.6500000000000714, -4.98, 2.4399999999998414, 1.899999999999853, 0.26999999999988766, 4.039999999999807, -1.2400000000000801, 1.1699999999998685, -4.310000000000015, 2.3899999999998425, 4.749999999999792, -3.59000000000003, -0.050000000000105516, 4.749999999999792, 4.159999999999805, 0.26999999999988766, -4.3200000000000145, -3.490000000000032, 1.0899999999998702, 2.2599999999998452, -3.5800000000000303, -2.600000000000051, 1.519999999999861, 2.9099999999998314, 1.8099999999998548, -4.98, 2.0499999999998497, -0.9600000000000861, -4.430000000000012, -3.4400000000000333, 2.169999999999847, -1.8800000000000665, -1.6400000000000716, -0.2900000000001004, -0.7100000000000914, 2.3899999999998425, -4.53000000000001, -0.78000000000009, -1.6100000000000723, 2.7099999999998357, -1.3200000000000784, 2.879999999999832, 4.559999999999796, 2.1599999999998474, 0.6999999999998785, 4.549999999999796, -3.2800000000000367, 1.6299999999998587, -1.620000000000072, -2.940000000000044, 3.5699999999998173, -0.3600000000000989, 4.769999999999792, 0.3199999999998866, -2.1700000000000603, -4.430000000000012, -1.4000000000000767, -1.9700000000000646, 0.17999999999988958, 2.5199999999998397, 4.679999999999794, 1.799999999999855, 3.6999999999998145, 2.1499999999998476, -3.3700000000000347, 4.759999999999792, -2.990000000000043, -1.0500000000000842, 2.7299999999998352, -1.9200000000000657, 1.2199999999998674, -0.24000000000010147, 4.119999999999806, 4.2199999999998035, -1.9700000000000646, -1.2400000000000801, 2.8499999999998327, -1.1200000000000827, 4.3799999999998, -4.200000000000017, 1.8899999999998531, 2.3499999999998433, 4.229999999999803, -1.3700000000000774, -0.3500000000000991, -3.960000000000022, 0.26999999999988766, -0.8000000000000895, -1.9400000000000652, 3.209999999999825, -4.52000000000001, -2.320000000000057, -0.050000000000105516, 2.129999999999848, -4.110000000000019, 2.879999999999832, 3.109999999999827, -3.2000000000000384, 2.6799999999998363, 1.4899999999998617, 2.4799999999998406, 3.5499999999998177, -4.55000000000001, 1.3499999999998646, 0.2399999999998883, 0.19999999999988916, -0.7900000000000897, -3.1100000000000403, 1.6399999999998585, -0.3100000000001, -3.970000000000022, 3.1299999999998267, 2.49999999999984, -0.48000000000009635, 4.049999999999807, -0.2900000000001004, -1.8400000000000674, 0.3899999999998851, -1.1300000000000825, 2.7199999999998354, -4.390000000000013, -4.190000000000017, 4.599999999999795, 0.8599999999998751, -1.0658141036401503e-13, 2.639999999999837, -4.6000000000000085, -0.1400000000001036, -2.0100000000000637, 1.759999999999856, 3.499999999999819, 4.989999999999787, -2.8100000000000467, -2.560000000000052, 4.229999999999803, -2.6400000000000503, -3.0600000000000414, 0.33999999999988617, -1.1700000000000816, 1.0899999999998702, 0.3899999999998851, -4.160000000000018, -4.870000000000003, 2.8699999999998322, 2.229999999999846, 2.97999999999983, -1.110000000000083, 3.4099999999998207, -1.580000000000073, 1.4899999999998617, 4.489999999999798, 1.6299999999998587, 4.909999999999789, -2.1500000000000608, 3.299999999999823, 4.429999999999799, 4.949999999999788, 2.6299999999998374, -0.8900000000000876, 0.6499999999998796, 2.8999999999998316, -0.16000000000010317, 3.489999999999819, 2.5399999999998393, 2.639999999999837, -2.890000000000045, -4.730000000000006, -3.2400000000000375, 0.14999999999989022, -0.23000000000010168, 0.7599999999998772, -2.800000000000047, 3.5699999999998173, 4.939999999999788, 2.559999999999839, -2.6400000000000503, -0.6700000000000923, 3.6199999999998163, 4.179999999999804, -3.640000000000029, 3.2399999999998244, 4.599999999999795, 4.4999999999997975, -2.6900000000000492, 1.1099999999998698, -2.0200000000000635, -1.0400000000000844, -3.2000000000000384, -0.9300000000000868, 4.989999999999787, 4.8299999999997905, 0.3799999999998853, 1.149999999999869, 0.5999999999998806, -4.330000000000014, 4.979999999999787, 0.4099999999998847, 1.5799999999998597, 2.3299999999998438, 1.239999999999867, -3.1100000000000403, 4.229999999999803, -0.3800000000000985, -2.6900000000000492, -0.17000000000010296, 2.309999999999844, -4.160000000000018, -4.820000000000004, 4.119999999999806, 2.7699999999998344, 0.7799999999998768, 1.8599999999998538, 1.9599999999998516, 3.2699999999998237, -0.3000000000001002, 0.2499999999998881, 1.849999999999854, -3.8000000000000256, 1.9499999999998519, 4.419999999999799, 1.3099999999998655, -3.2800000000000367, -0.32000000000009976, -4.660000000000007, 3.199999999999825, -1.8400000000000674, 2.919999999999831, 1.4099999999998634, 1.239999999999867, 1.2699999999998663, 3.019999999999829, 2.779999999999834, -4.5100000000000104, -2.0800000000000622, -3.4800000000000324, -3.790000000000026, 1.6399999999998585, -2.240000000000059, -1.060000000000084, -1.340000000000078, -4.08000000000002, 4.789999999999791, -4.450000000000012, -1.1900000000000812, -3.180000000000039, 4.109999999999806, 0.5599999999998815, 2.089999999999849, 4.2199999999998035, 3.9299999999998096, 1.289999999999866, 3.499999999999819, -2.280000000000058, -3.2300000000000377, 1.4399999999998627, 3.2299999999998246, 2.6699999999998365, 1.2599999999998666, 3.579999999999817, 0.9399999999998734, -1.7100000000000701, -2.9500000000000437, -4.280000000000015, -0.7100000000000914, -1.7700000000000689, 2.6799999999998363, 3.999999999999808, 4.4999999999997975, 2.4299999999998416, -1.9600000000000648, -3.8600000000000243, -3.260000000000037, 2.169999999999847, -3.5200000000000315, 1.8799999999998533, -1.6800000000000708, 0.299999999999887, 2.739999999999835, -0.4300000000000974, -3.820000000000025, -3.1400000000000396, 1.7399999999998563, 2.169999999999847, -4.190000000000017, -0.6700000000000923, 1.3099999999998655, -2.940000000000044, 0.909999999999874, -3.1100000000000403, 1.2299999999998672, -1.0500000000000842, -0.920000000000087, 3.5499999999998177, 1.0699999999998706, -3.12000000000004, -4.350000000000014, 3.8299999999998118, 2.7099999999998357, -0.11000000000010424, -4.310000000000015, -4.06000000000002, -3.9000000000000234, -4.620000000000008, -1.3500000000000778, 3.3299999999998224, 2.7599999999998346, 3.44999999999982, -1.3500000000000778, -1.7000000000000703, 4.83999999999979, 3.6499999999998156, -0.24000000000010147, 3.4599999999998197, -2.4400000000000546, -1.7000000000000703, -3.360000000000035, -2.9000000000000448, 1.2299999999998672, -2.270000000000058, -2.6400000000000503, -2.8600000000000456, 3.219999999999825, -1.060000000000084, -1.6000000000000725, 0.9799999999998725, -4.08000000000002, -4.3200000000000145, -1.3300000000000782, 2.0199999999998504, 0.28999999999988724, -1.0800000000000836, 3.219999999999825, -0.5700000000000944, 0.6699999999998791, -0.5000000000000959, 0.3299999999998864, -3.260000000000037, 0.34999999999988596, -3.770000000000026, -1.430000000000076, 1.3399999999998649, -4.820000000000004, -0.16000000000010317, -2.890000000000045, -0.920000000000087, 2.449999999999841, -4.280000000000015, -2.0600000000000627, 3.209999999999825, -4.480000000000011, -0.6500000000000927, 0.12999999999989065, -4.08000000000002, 4.1699999999998045, -1.6900000000000706, 3.6199999999998163, 2.2899999999998446, 1.189999999999868, 4.109999999999806, -4.760000000000005, 0.5699999999998813, -4.670000000000007, -2.090000000000062, 1.8699999999998536, -2.560000000000052, 1.5899999999998595, 4.619999999999795, -2.5000000000000533, -0.7100000000000914, -4.390000000000013, 3.5699999999998173, -1.670000000000071, 1.3899999999998638, 4.84999999999979, 3.389999999999821, -1.060000000000084, -2.1600000000000605, 1.429999999999863, -4.500000000000011, -2.980000000000043, -0.8100000000000893, 0.7599999999998772, 0.5099999999998825, 3.4799999999998192, 3.7899999999998126, 4.86999999999979, -2.0800000000000622, -0.10000000000010445, 0.43999999999988404, -2.140000000000061, 3.2799999999998235, 3.809999999999812, -4.880000000000003, 1.7499999999998561, 3.539999999999818, 4.679999999999794, -4.1300000000000185, 2.49999999999984, 2.169999999999847, -3.830000000000025, 3.43999999999982, -1.9200000000000657, 2.639999999999837, 4.659999999999794, 2.3899999999998425, 1.6799999999998576, 2.96999999999983, -2.6900000000000492, -3.4800000000000324, -4.220000000000017, -4.07000000000002, -1.810000000000068, 4.159999999999805, -2.6300000000000505, -3.7200000000000273, 4.599999999999795, -0.8500000000000885, 4.099999999999806, -0.7600000000000904, 3.259999999999824, 1.2299999999998672, -2.3000000000000576, -1.9700000000000646, 0.02999999999989278, 3.7499999999998135, 0.7599999999998772, -4.120000000000019, 0.6399999999998798, 4.9199999999997885, -1.7800000000000686, -3.830000000000025, -2.6300000000000505, 2.4799999999998406, 1.4899999999998617, -2.140000000000061, -4.750000000000005, -1.1900000000000812, 2.0599999999998495, -4.030000000000021, -4.170000000000018, -3.0500000000000416, -0.10000000000010445, 0.959999999999873, 3.529999999999818, -0.8300000000000889, -4.950000000000001, 2.9999999999998295, -1.8800000000000665, -2.280000000000058, -3.080000000000041, -4.640000000000008, -4.0400000000000205, -4.210000000000017, -3.4300000000000335, 4.0299999999998075, -2.1500000000000608, -2.8200000000000465, -0.07000000000010509, 2.889999999999832, 0.5399999999998819, 0.11999999999989086, 0.0899999999998915, 0.22999999999988852, -4.920000000000002, -1.9400000000000652, -2.1200000000000614, -2.9100000000000446, -1.6900000000000706, -3.870000000000024, 0.9699999999998727, 3.3799999999998214, -1.9700000000000646, -4.230000000000016, 2.96999999999983, -3.830000000000025, -4.560000000000009, 1.2599999999998666, 2.889999999999832, 4.279999999999802, -4.9300000000000015, -0.1400000000001036, -3.020000000000042, 3.8999999999998103, 2.9899999999998297, -4.240000000000016, -2.5800000000000516, -2.3000000000000576, -0.6400000000000929, 0.4499999999998838, -2.6700000000000497, 1.4999999999998614, -3.9800000000000217, 2.9599999999998303, 3.5499999999998177, 2.319999999999844, 2.0699999999998493, 3.6099999999998165, -4.360000000000014, -0.41000000000009784, -1.1400000000000823, -4.420000000000012, 1.899999999999853, 4.85999999999979, -4.100000000000019, 3.959999999999809, 0.5299999999998821, -4.330000000000014, -0.2900000000001004, 3.0799999999998278, 4.649999999999794, -1.580000000000073, 3.299999999999823, -2.5000000000000533, -3.960000000000022, 4.0799999999998064, -1.2700000000000795, 4.279999999999802, -3.770000000000026, 4.529999999999797, 4.799999999999791, -2.6300000000000505, 2.779999999999834, 2.5299999999998395, 4.799999999999791, -0.9700000000000859, -1.1200000000000827, 0.4699999999998834, -3.830000000000025, 0.7899999999998766, -0.2200000000001019, -4.6500000000000075, 2.8599999999998325, -0.12000000000010402, -0.18000000000010274, 2.4399999999998414, -3.3900000000000343, 3.959999999999809, 2.309999999999844, -1.9700000000000646, 3.7499999999998135, 3.069999999999828, -1.6400000000000716, -0.8600000000000882, -2.1600000000000605, -0.7500000000000906, 2.919999999999831, -0.6400000000000929, 2.8099999999998335, -0.26000000000010104, -3.9500000000000224, -0.08000000000010488, 1.0399999999998712, 3.719999999999814, 0.9399999999998734, 0.8799999999998747, 1.4399999999998627, 0.009999999999893205, -2.2600000000000584, 4.489999999999798, 2.4299999999998416, -3.5800000000000303, 1.659999999999858, 0.3199999999998866, -2.6200000000000507, 0.8599999999998751, -0.8300000000000889, 2.3999999999998423, 3.2399999999998244, 0.13999999999989043, -1.9700000000000646, -2.050000000000063, 0.7499999999998774, 1.6799999999998576, 4.269999999999802, -4.670000000000007, 1.8099999999998548, -3.960000000000022, 1.1199999999998695, 1.5399999999998606, -1.3300000000000782, -4.710000000000006, -2.600000000000051, -1.1700000000000816, -2.5500000000000522, 4.449999999999799, 0.02999999999989278, -4.250000000000016, 2.0099999999998506, -2.890000000000045, 2.649999999999837, 4.319999999999801, -4.630000000000008, -4.360000000000014, 0.5399999999998819, 4.719999999999793, -1.5200000000000742, 2.97999999999983, 2.9499999999998305, 1.4199999999998631, 1.1299999999998693, -4.260000000000016, 2.879999999999832, -0.8600000000000882, 1.8299999999998544, -3.9800000000000217, 1.939999999999852, -2.6400000000000503, 2.8199999999998333, 1.2499999999998668, 4.449999999999799, 1.4799999999998619, 4.3799999999998, 3.2299999999998246, -3.3900000000000343, 3.169999999999826, -0.8200000000000891, 1.8399999999998542, 0.1699999999998898, 4.739999999999792, -0.9600000000000861, 3.1499999999998263, 4.749999999999792, -3.3800000000000345, 2.599999999999838, -0.09000000000010466, -4.690000000000007, -3.450000000000033, 2.309999999999844, 0.49999999999988276, -1.3600000000000776, 1.5899999999998595, 0.8999999999998742, -2.9000000000000448, -2.420000000000055, 4.6399999999997945, 1.9599999999998516, 4.1699999999998045, 4.809999999999791, -3.680000000000028, 1.5799999999998597, -4.55000000000001, 0.2399999999998883, -3.4700000000000326, 1.9799999999998512, 3.249999999999824, -1.0900000000000833, -2.850000000000046, -3.4400000000000333, -3.820000000000025, 4.819999999999791, -0.20000000000010232, -0.16000000000010317, 0.9899999999998723, -2.2200000000000593, 3.359999999999822, -3.5100000000000318, -3.770000000000026, 1.1399999999998691, 1.2199999999998674, 1.7199999999998568, -0.1400000000001036, 0.5999999999998806, -4.390000000000013, 2.1499999999998476, -1.5400000000000738, 2.0199999999998504, 2.789999999999834 ], "xaxis": "x", "y": [ -3.6100000000000296, 1.009999999999872, -1.670000000000071, 1.659999999999858, -1.670000000000071, 1.659999999999858, 4.989999999999787, -5, -5, 4.989999999999787, 0.3799999999998853, 0.14999999999989022, -0.7900000000000897, -0.8300000000000889, -3.13000000000004, -1.5900000000000727, 3.0099999999998293, -3.730000000000027, -4.440000000000012, -1.4100000000000765, 0.7499999999998774, 3.679999999999815, 3.0499999999998284, 0.4099999999998847, 0.3799999999998853, -3.020000000000042, -1.3700000000000774, 1.5399999999998606, 0.04999999999989235, -3.1100000000000403, -4.660000000000007, 4.85999999999979, 1.149999999999869, 2.1999999999998465, -0.450000000000097, -3.3400000000000354, 1.7799999999998555, 2.699999999999836, -1.3100000000000787, -3.160000000000039, -2.100000000000062, 4.86999999999979, -3.360000000000035, 4.479999999999798, -1.7300000000000697, -2.270000000000058, 1.9799999999998512, 3.9299999999998096, 3.209999999999825, -0.010000000000106368, -0.23000000000010168, 1.9099999999998527, 1.2099999999998676, 0.5199999999998823, 0.5399999999998819, -3.040000000000042, 0.8099999999998762, -0.6700000000000923, -2.5500000000000522, 4.129999999999805, -2.7300000000000484, 2.4899999999998403, -2.0200000000000635, -4.280000000000015, 1.5499999999998604, 0.579999999999881, 4.809999999999791, -4.020000000000021, -1.4000000000000767, -0.5600000000000946, 1.9199999999998525, -1.2600000000000797, 1.09999999999987, 1.3599999999998644, 1.0399999999998712, 2.459999999999841, -2.420000000000055, -3.6100000000000296, -4.700000000000006, -4.220000000000017, 3.299999999999823, -1.900000000000066, -2.2500000000000586, -4.020000000000021, -3.2500000000000373, 0.2499999999998881, 2.139999999999848, -0.5100000000000957, 0.7099999999998783, -2.2900000000000578, 2.3499999999998433, 0.49999999999988276, -2.0100000000000637, 4.329999999999801, 2.6699999999998365, -1.7700000000000689, 1.709999999999857, -1.6900000000000706, 4.049999999999807, -3.730000000000027, 2.3299999999998438, 4.409999999999799, -1.9800000000000644, 1.3399999999998649, 3.539999999999818, -1.6400000000000716, -1.340000000000078, 3.719999999999814, 4.629999999999795, -3.3800000000000345, 3.4799999999998192, 3.6999999999998145, 1.7899999999998553, -4.05000000000002, 1.1999999999998678, -1.4200000000000763, -3.970000000000022, -1.3200000000000784, 2.179999999999847, 2.879999999999832, -3.880000000000024, -3.2300000000000377, -2.090000000000062, 2.0499999999998497, 2.5199999999998397, -0.6600000000000925, 1.429999999999863, 1.3499999999998646, -2.880000000000045, 3.1299999999998267, -1.950000000000065, -3.490000000000032, 3.3799999999998214, -3.2900000000000365, -4.470000000000011, 4.799999999999791, -3.730000000000027, -4.52000000000001, -2.370000000000056, -2.7600000000000477, 3.539999999999818, 1.899999999999853, 1.6699999999998578, -3.2400000000000375, -3.1500000000000394, 0.8299999999998757, -3.830000000000025, -4.440000000000012, -4.030000000000021, -3.5800000000000303, 0.8399999999998755, 1.09999999999987, -1.1400000000000823, 3.299999999999823, 0.4599999999998836, -0.730000000000091, -4.640000000000008, 1.469999999999862, -2.040000000000063, -0.020000000000106155, -4.3200000000000145, -1.3500000000000778, 3.399999999999821, 2.8499999999998327, 3.159999999999826, -4.440000000000012, -2.8600000000000456, -0.6500000000000927, -2.270000000000058, 1.759999999999856, 2.7199999999998354, -1.390000000000077, 3.029999999999829, 2.49999999999984, -4.120000000000019, 2.50999999999984, -0.6700000000000923, 0.769999999999877, 1.5299999999998608, -2.7200000000000486, 4.299999999999802, 1.8299999999998544, 4.899999999999789, 4.179999999999804, 2.6699999999998365, -2.470000000000054, 2.639999999999837, 2.50999999999984, 4.549999999999796, -4.580000000000009, -0.9600000000000861, 1.56999999999986, -1.9900000000000642, -2.7800000000000473, 3.5499999999998177, 1.5499999999998604, -1.4400000000000759, 2.9999999999998295, -3.450000000000033, -0.3100000000001, -1.4000000000000767, -2.7200000000000486, -4.120000000000019, 1.2299999999998672, -4.800000000000004, 2.0999999999998487, 3.4299999999998203, 2.3899999999998425, -3.3800000000000345, -3.160000000000039, 3.7999999999998124, -2.040000000000063, -3.1100000000000403, 0.20999999999988894, -3.8900000000000237, -1.9700000000000646, -1.8200000000000678, 3.729999999999814, 1.4399999999998627, -4.6500000000000075, -4.570000000000009, -0.32000000000009976, 3.999999999999808, 4.809999999999791, -3.5600000000000307, 1.2099999999998676, -4.490000000000011, 1.8299999999998544, 1.3999999999998636, 3.8399999999998116, 2.919999999999831, -1.7500000000000693, -0.08000000000010488, 0.9299999999998736, 2.929999999999831, 3.5099999999998186, 1.5399999999998606, -0.4600000000000968, -3.59000000000003, -1.0700000000000838, 0.7199999999998781, 0.26999999999988766, 0.8899999999998744, 1.3999999999998636, 4.3799999999998, -3.830000000000025, -4.670000000000007, -0.0600000000001053, -1.7100000000000701, 0.909999999999874, -1.110000000000083, -0.09000000000010466, 2.4799999999998406, 2.0099999999998506, 4.269999999999802, 2.4899999999998403, -3.8500000000000245, 3.769999999999813, 1.4499999999998625, 0.7199999999998781, -3.960000000000022, -0.7200000000000912, 0.05999999999989214, 2.0699999999998493, -0.450000000000097, -0.9700000000000859, -2.700000000000049, -2.7800000000000473, 1.09999999999987, 4.449999999999799, -4.890000000000002, 1.4199999999998631, -3.680000000000028, -0.9900000000000855, 1.2099999999998676, 0.2499999999998881, -3.2800000000000367, 1.3399999999998649, -3.9100000000000232, -2.4900000000000535, 3.6499999999998156, 2.3999999999998423, 1.329999999999865, 4.769999999999792, -0.4600000000000968, -2.790000000000047, 2.50999999999984, 4.579999999999796, -3.730000000000027, -3.6200000000000294, -2.8600000000000456, -3.9100000000000232, 3.8799999999998107, 3.539999999999818, -4.350000000000014, -0.7900000000000897, -3.790000000000026, 0.299999999999887, 4.86999999999979, 0.3099999999998868, -0.12000000000010402, -4.500000000000011, -0.9400000000000865, -4.820000000000004, 4.209999999999804, 2.689999999999836, -4.300000000000015, -3.6500000000000288, 0.489999999999883, 4.149999999999805, 1.7799999999998555, 4.229999999999803, -1.8300000000000676, -1.530000000000074, 3.6899999999998148, 2.6199999999998376, -3.2000000000000384, -3.2900000000000365, 3.999999999999808, 0.9399999999998734, -4.660000000000007, -3.7500000000000266, 0.959999999999873, -4.220000000000017, -4.170000000000018, -3.5200000000000315, -4.55000000000001, -2.700000000000049, 3.91999999999981, 1.5599999999998602, 1.149999999999869, 4.189999999999804, 3.7499999999998135, -4.410000000000013, 0.41999999999988447, 0.7099999999998783, 0.19999999999988916, 1.1299999999998693, -0.010000000000106368, 2.079999999999849, 4.459999999999798, -3.930000000000023, -3.920000000000023, 4.289999999999802, 2.559999999999839, 4.089999999999806, -0.9700000000000859, -4.730000000000006, -1.5600000000000733, 0.6799999999998789, -0.920000000000087, -0.10000000000010445, -2.2500000000000586, -4.240000000000016, 1.9999999999998508, -4.900000000000002, -3.8100000000000254, 3.729999999999814, 0.6099999999998804, -4.05000000000002, -1.5700000000000731, -2.0800000000000622, 4.9699999999997875, 2.079999999999849, 4.719999999999793, 2.7699999999998344, 2.7699999999998344, 3.209999999999825, 1.849999999999854, -3.260000000000037, 3.2299999999998246, 4.119999999999806, -3.640000000000029, 1.469999999999862, -3.2400000000000375, -1.7900000000000684, 2.9099999999998314, -4.55000000000001, -4.1300000000000185, -4.950000000000001, 3.869999999999811, -2.2000000000000597, -3.320000000000036, 1.0399999999998712, -0.6800000000000921, 3.3399999999998222, -1.4900000000000748, 3.4199999999998205, 4.3899999999998, -2.9500000000000437, 2.02999999999985, -0.5100000000000957, 2.689999999999836, 3.5199999999998184, -3.5800000000000303, 4.119999999999806, 0.5599999999998815, -3.740000000000027, 0.3899999999998851, -2.2100000000000595, 2.789999999999834, -3.1000000000000405, -1.760000000000069, 3.7599999999998133, -0.9100000000000872, 0.4499999999998838, 1.289999999999866, 1.7799999999998555, -0.9700000000000859, -0.32000000000009976, 2.929999999999831, -1.4600000000000755, -4.620000000000008, -2.380000000000056, 0.34999999999988596, 4.989999999999787, 3.969999999999809, -1.2100000000000808, -4.98, -1.2100000000000808, -4.390000000000013, 0.5699999999998813, -1.1200000000000827, -0.8200000000000891, -3.7600000000000264, -0.2700000000001008, 4.8299999999997905, -3.540000000000031, -4.840000000000003, -2.0600000000000627, 3.219999999999825, 4.629999999999795, 3.3199999999998226, -3.8000000000000256, 3.859999999999811, -0.6400000000000929, -5, 0.7499999999998774, 2.1099999999998484, 0.3799999999998853, -4.52000000000001, 3.669999999999815, -0.33000000000009955, 1.659999999999858, -1.7300000000000697, 0.34999999999988596, -3.320000000000036, 4.679999999999794, -4.440000000000012, 0.34999999999988596, -0.8200000000000891, 1.379999999999864, -4.940000000000001, -0.24000000000010147, 1.56999999999986, 0.039999999999892566, 2.079999999999849, 4.349999999999801, 3.109999999999827, -2.710000000000049, 1.6799999999998576, -2.6200000000000507, 1.2699999999998663, 4.3799999999998, 3.0499999999998284, 4.809999999999791, 3.3299999999998224, -4.010000000000021, 2.079999999999849, -2.18000000000006, 4.429999999999799, -1.7300000000000697, 1.9099999999998527, -1.2800000000000793, 4.349999999999801, -4.340000000000014, 4.139999999999805, -0.730000000000091, -4.010000000000021, 2.5199999999998397, 2.9499999999998305, -4.760000000000005, 0.09999999999989129, -1.6600000000000712, 2.559999999999839, -3.3300000000000356, 0.6999999999998785, 4.039999999999807, 1.7499999999998561, 2.6299999999998374, -1.3600000000000776, 3.3199999999998226, 4.669999999999794, -2.0300000000000633, -4.4600000000000115, -0.5400000000000951, -1.200000000000081, -1.3800000000000772, -2.740000000000048, 0.909999999999874, -3.870000000000024, 0.8399999999998755, -2.2900000000000578, -3.040000000000042, 3.3199999999998226, -1.7700000000000689, 2.219999999999846, 3.4699999999998195, 1.6699999999998578, 0.8899999999998744, -3.540000000000031, 1.2199999999998674, 3.6199999999998163, -1.6500000000000714, 4.609999999999795, 0.8799999999998747, -3.5600000000000307, -2.0100000000000637, 3.949999999999809, -0.870000000000088, -1.4700000000000752, 3.539999999999818, 3.6499999999998156, -0.8600000000000882, -1.7700000000000689, -3.8000000000000256, 0.769999999999877, -3.59000000000003, -0.12000000000010402, -4.670000000000007, -2.050000000000063, -0.2900000000001004, 4.039999999999807, -0.24000000000010147, 2.6699999999998365, 4.3899999999998, -1.9900000000000642, -4.830000000000004, 0.17999999999988958, 1.5099999999998612, -4.690000000000007, 0.8999999999998742, 1.5099999999998612, 2.599999999999838, 3.969999999999809, 0.009999999999893205, -4.820000000000004, -3.360000000000035, -2.9200000000000443, -4.08000000000002, 1.009999999999872, -2.700000000000049, 2.879999999999832, 0.10999999999989107, -4.640000000000008, -2.840000000000046, -1.810000000000068, 4.899999999999789, 3.6199999999998163, 0.9299999999998736, 2.929999999999831, -3.1100000000000403, -4.610000000000008, -4.590000000000009, 0.579999999999881, 0.8599999999998751, -2.2200000000000593, -3.0000000000000426, -1.6400000000000716, 1.6399999999998585, 4.949999999999788, -4.270000000000016, 4.3699999999998, 4.86999999999979, 0.13999999999989043, -1.1900000000000812, -1.0500000000000842, 2.1899999999998467, 4.929999999999788, -1.1200000000000827, -3.180000000000039, 1.9699999999998514, 1.1099999999998698, -0.3100000000001, -2.5800000000000516, 3.309999999999823, -1.2400000000000801, -4.380000000000013, 1.09999999999987, -2.18000000000006, -2.790000000000047, 1.8099999999998548, 1.6799999999998576, -4.300000000000015, 2.5699999999998386, 4.279999999999802, -4.280000000000015, 3.91999999999981, 4.069999999999807, -0.17000000000010296, 2.1499999999998476, -2.2200000000000593, 0.0899999999998915, 4.539999999999797, 0.8199999999998759, -1.0658141036401503e-13, -1.2100000000000808, 0.009999999999893205, -0.25000000000010125, -4.140000000000018, -0.48000000000009635, 3.4099999999998207, -2.6300000000000505, -0.3500000000000991, 2.8999999999998316, 3.3299999999998224, 4.429999999999799, -0.9900000000000855, -1.5500000000000735, -2.570000000000052, -2.140000000000061, -4.06000000000002, 2.779999999999834, -1.0200000000000848, -0.7600000000000904, -0.6900000000000919, 3.779999999999813, 1.7199999999998568, 0.09999999999989129, -0.6600000000000925, -0.39000000000009827, -3.460000000000033, -3.0100000000000424, 4.0799999999998064, -1.6300000000000718, 3.4199999999998205, -1.1600000000000819, -2.7200000000000486, 3.769999999999813, 0.2399999999998883, -4.6000000000000085, -0.11000000000010424, 0.5499999999998817, 3.639999999999816, 1.4199999999998631, -4.99, 2.0599999999998495, -3.4400000000000333, -1.480000000000075, -1.0500000000000842, -4.840000000000003, 4.789999999999791, 1.7499999999998561, 1.4099999999998634, 4.599999999999795, 0.10999999999989107, 3.999999999999808, -1.670000000000071, 0.4099999999998847, 3.2899999999998233, 1.6699999999998578, -4.920000000000002, 3.8799999999998107, 1.7199999999998568, 3.669999999999815, 0.35999999999988574, 2.3499999999998433, 3.579999999999817, -2.570000000000052, 3.949999999999809, 3.489999999999819, -4.1300000000000185, 0.6899999999998787, -2.0700000000000625, 1.379999999999864, -4.490000000000011, -4.960000000000001, 2.639999999999837, -2.19000000000006, -3.780000000000026, 3.1299999999998267, 1.2199999999998674, -1.8700000000000667, -2.19000000000006, -0.49000000000009614, -4.630000000000008, 2.0999999999998487, -0.33000000000009955, 1.2599999999998666, -3.870000000000024, 0.5699999999998813, -4.160000000000018, -0.32000000000009976, -3.640000000000029, -1.2100000000000808, 2.839999999999833, -0.3800000000000985, -3.460000000000033, 4.009999999999808, -0.5800000000000942, 0.7099999999998783, 2.369999999999843, 1.5999999999998593, 3.9899999999998084, -2.280000000000058, -0.7200000000000912, 1.7899999999998553, 0.7299999999998779, -4.7900000000000045, -1.1400000000000823, -3.6100000000000296, 4.209999999999804, -0.4300000000000974, -3.180000000000039, -0.7100000000000914, 4.209999999999804, 0.13999999999989043, -3.550000000000031, 2.96999999999983, -0.9700000000000859, 3.069999999999828, 0.3999999999998849, -1.6300000000000718, -0.8000000000000895, -0.25000000000010125, -2.9500000000000437, 1.289999999999866, 2.319999999999844, -0.7100000000000914, 0.6799999999998789, -3.830000000000025, -0.7000000000000917, -4.610000000000008, -3.6700000000000284, -1.3500000000000778, 3.069999999999828, -0.04000000000010573, 0.019999999999892992, 2.219999999999846, -2.460000000000054, -4.240000000000016, 1.8899999999998531, 3.4799999999998192, -1.7900000000000684, -1.480000000000075, 3.399999999999821, -2.0700000000000625, -2.6400000000000503, -1.8200000000000678, -2.930000000000044, 2.359999999999843, 3.7499999999998135, -0.5200000000000955, 1.3199999999998653, -2.19000000000006, -2.3000000000000576, -0.9800000000000857, 4.509999999999797, -2.2100000000000595, -3.640000000000029, 0.49999999999988276, 1.5599999999998602, 3.8399999999998116, 2.49999999999984, 0.6999999999998785, 0.5999999999998806, 4.099999999999806, -3.5600000000000307, 1.849999999999854, -3.8500000000000245, 3.3399999999998222, 1.8599999999998538, -3.870000000000024, 1.6199999999998589, -4.110000000000019, -3.160000000000039, -0.0600000000001053, -3.5200000000000315, -0.3000000000001002, -1.8900000000000663, 4.739999999999792, -3.6100000000000296, 1.7199999999998568, -2.2100000000000595, -1.760000000000069, -4.380000000000013, -1.150000000000082, 1.3399999999998649, -0.6300000000000932, 4.009999999999808, 3.5199999999998184, 1.4799999999998619, -4.120000000000019, -3.3300000000000356, -3.020000000000042, -4.590000000000009, 4.489999999999798, 0.6799999999998789, 3.299999999999823, -0.34000000000009933, 3.159999999999826, -4.7400000000000055, -0.5700000000000944, -4.210000000000017, 2.129999999999848, -1.3300000000000782, -3.630000000000029, -2.2600000000000584, 4.209999999999804, 1.329999999999865, 2.4699999999998408, -2.850000000000046, 1.989999999999851, 4.539999999999797, 0.42999999999988425, -4.090000000000019, -2.18000000000006, 2.599999999999838, -0.3000000000001002, 2.2899999999998446, 0.009999999999893205, -0.6700000000000923, 0.15999999999989, 0.7199999999998781, -3.2900000000000365, 1.4599999999998623, -2.740000000000048, -0.9500000000000863, 0.5399999999998819, -2.8100000000000467, 1.7499999999998561, 3.9299999999998096, -3.8500000000000245, -0.1300000000001038, -3.450000000000033, -0.7500000000000906, 3.869999999999811, -1.3100000000000787, 3.959999999999809, 4.109999999999806, 2.5199999999998397, -3.970000000000022, 0.7299999999998779, 1.0599999999998708, 0.0899999999998915, -1.8300000000000676, -4.190000000000017, -2.130000000000061, 3.9899999999998084, -0.40000000000009805, -4.840000000000003, -1.6000000000000725, 1.5099999999998612, 3.2399999999998244, -4.1300000000000185, 4.409999999999799, -2.7700000000000475, 3.489999999999819, 1.8399999999998542, 3.6499999999998156, 2.449999999999841, 3.729999999999814, 4.229999999999803, 4.619999999999795, 2.879999999999832, -2.2600000000000584, -0.9600000000000861, 3.9299999999998096, 4.3999999999998, 0.18999999999988937, -4.010000000000021, 4.459999999999798, -1.8200000000000678, 4.989999999999787, -1.2800000000000793, 1.3599999999998644, -4.210000000000017, -1.1800000000000814, 1.7399999999998563, 0.8899999999998744, -0.3000000000001002, 4.209999999999804, 3.1299999999998267, 0.6099999999998804, 2.6799999999998363, 0.22999999999988852, -4.1300000000000185, -2.370000000000056, 4.86999999999979, 0.26999999999988766, -0.6700000000000923, -2.8300000000000463, 3.90999999999981, -3.790000000000026, 3.2699999999998237, -1.6100000000000723, -4.950000000000001, -2.4500000000000544, -1.3600000000000776, -2.700000000000049, -1.670000000000071, -3.3900000000000343, 3.159999999999826, -0.3500000000000991, -1.4700000000000752, 4.699999999999793, 0.9999999999998721, 3.1299999999998267, 1.7499999999998561, 4.489999999999798, -4.570000000000009, 3.7499999999998135, 4.449999999999799, -4.960000000000001, 1.8799999999998533, 4.789999999999791, -3.550000000000031, 1.0399999999998712, -4.53000000000001, -3.310000000000036, -3.5800000000000303, 1.0399999999998712, 1.4099999999998634, -1.6600000000000712, -4.810000000000004, 3.7099999999998143, 4.0299999999998075, 2.359999999999843, -1.6000000000000725, -2.700000000000049, 0.8599999999998751, 4.179999999999804, 1.1699999999998685, -3.0100000000000424, 3.5699999999998173, -2.9700000000000433, -0.590000000000094, 1.5499999999998604, 2.4699999999998408, -1.6100000000000723, -3.220000000000038, -4.110000000000019, 1.4999999999998614, -2.5500000000000522, -3.0000000000000426, 0.10999999999989107, -2.410000000000055, -4.7900000000000045, -3.7100000000000275, -0.4300000000000974, -4.160000000000018, 4.149999999999805, -0.19000000000010253, -3.8600000000000243, 0.41999999999988447, -2.5500000000000522, -3.320000000000036, -1.6500000000000714, 3.859999999999811, -4.340000000000014, 0.28999999999988724, -3.880000000000024, 3.629999999999816, -4.290000000000015, -3.9000000000000234, 2.8099999999998335, -2.4400000000000546, 1.149999999999869, -2.230000000000059, 1.6199999999998589, -1.4400000000000759, 3.859999999999811, -4.5100000000000104, 0.8499999999998753, 0.8699999999998749, -3.400000000000034, -0.3100000000001, -2.520000000000053, 3.769999999999813, 0.7399999999998776, 4.789999999999791, -4.08000000000002, -4.53000000000001, 3.769999999999813, -1.9700000000000646, -0.10000000000010445, -1.8400000000000674, -2.990000000000043, -1.4200000000000763, -4.52000000000001, -3.2400000000000375, -4.020000000000021, 0.0899999999998915, 2.8199999999998333, -2.3400000000000567, 3.629999999999816, 0.6399999999998798, 2.739999999999835, -2.240000000000059, 1.8899999999998531, -0.6100000000000936, 3.869999999999811, -4.400000000000013, 4.319999999999801, 0.42999999999988425, 4.9699999999997875, 3.579999999999817, 1.6899999999998574, -4.160000000000018, -2.790000000000047, -4.190000000000017, 0.06999999999989193, -0.6500000000000927, 0.9399999999998734, 2.839999999999833, 2.2899999999998446, 0.7199999999998781, 4.269999999999802, 2.7599999999998346, 0.7499999999998774, -3.9500000000000224, -3.59000000000003, 4.549999999999796, 2.699999999999836, 0.959999999999873, -2.370000000000056, 2.7999999999998337, -0.07000000000010509, 2.6299999999998374, 2.97999999999983, 3.91999999999981, -0.39000000000009827, 0.4599999999998836, -1.620000000000072, -2.1500000000000608, -3.12000000000004, 1.56999999999986, -2.1600000000000605, 0.4499999999998838, 4.9699999999997875, -1.8000000000000682, -1.2300000000000804, 2.3499999999998433, -3.9400000000000226, 4.469999999999798, 3.4799999999998192, -0.7100000000000914, 1.1799999999998683, 2.549999999999839, 4.229999999999803, 4.269999999999802, 4.6899999999997934, -2.980000000000043, 1.3399999999998649, -4.480000000000011, -3.6700000000000284, -0.5700000000000944, 2.699999999999836, -3.6500000000000288, -1.2200000000000806, 0.07999999999989171, -4.470000000000011, -3.500000000000032, 2.419999999999842, -1.3000000000000789, -2.5400000000000524, 1.149999999999869, 2.919999999999831, -1.0300000000000846, 2.8199999999998333, -2.980000000000043, 1.469999999999862, 2.279999999999845, 3.949999999999809, 1.4199999999998631, 1.6499999999998582, -1.4000000000000767, -0.6500000000000927, -4.030000000000021, 1.709999999999857, -1.6400000000000716, 4.049999999999807, 0.14999999999989022, -4.610000000000008, -2.790000000000047, 0.8799999999998747, 0.3999999999998849, 3.9799999999998086, -1.8400000000000674, -1.2700000000000795, 4.569999999999796, -1.2600000000000797, 0.5099999999998825, 0.4599999999998836, -2.790000000000047, -4.05000000000002, -2.4900000000000535, 0.5399999999998819, 0.2599999999998879, -0.32000000000009976, 3.8899999999998105, 4.629999999999795, 4.479999999999798, -4.900000000000002, -4.780000000000005, 0.12999999999989065, -1.72000000000007, -3.960000000000022, -2.6900000000000492, 3.529999999999818, 1.8799999999998533, -1.1200000000000827, -3.500000000000032, 3.9899999999998084, -0.6900000000000919, 3.959999999999809, 4.0799999999998064, -4.450000000000012, 2.449999999999841, 2.609999999999838, 1.429999999999863, 0.9699999999998727, -0.23000000000010168, -2.5300000000000527, 1.0599999999998708, 1.0799999999998704, 3.589999999999817, -0.1400000000001036, -1.620000000000072, -1.8500000000000671, -3.5600000000000307, 0.7899999999998766, -3.690000000000028, -4.970000000000001, 4.529999999999797, 1.5599999999998602, 4.569999999999796, 2.3399999999998435, 0.2599999999998879, -3.1000000000000405, 0.8699999999998749, -0.6900000000000919, 1.989999999999851, 2.3399999999998435, -2.5000000000000533, 0.019999999999892992, -1.3800000000000772, -4.52000000000001, -3.13000000000004, -0.920000000000087, -3.460000000000033, 4.85999999999979, -4.750000000000005, -3.6200000000000294, 2.2099999999998463, 1.049999999999871, -1.2600000000000797, 3.729999999999814, 0.6999999999998785, -1.3600000000000776, 4.9699999999997875, -4.7400000000000055, -2.840000000000046, 1.0799999999998704, -1.3600000000000776, 0.10999999999989107, -2.2600000000000584, 4.609999999999795, 2.369999999999843, 2.9599999999998303, 3.2799999999998235, -4.860000000000003, 0.9499999999998732, 0.22999999999988852, -4.100000000000019, 2.0499999999998497, -0.7200000000000912, 1.5999999999998593, 3.529999999999818, 2.7999999999998337, 1.2999999999998657, 0.8299999999998757, 2.2099999999998463, -3.030000000000042, -2.750000000000048, -1.950000000000065, -1.9900000000000642, 0.769999999999877, -2.5000000000000533, 4.6899999999997934, -1.290000000000079, 3.9799999999998086, 4.339999999999801, -1.0300000000000846, -0.33000000000009955, -3.820000000000025, -1.72000000000007, -1.9800000000000644, -0.3500000000000991, -2.600000000000051, 4.299999999999802, -1.9800000000000644, 1.239999999999867, -1.1000000000000831, -2.100000000000062, 1.9799999999998512, -2.470000000000054, 1.1199999999998695, -1.3500000000000778, -4.760000000000005, -2.840000000000046, 2.089999999999849, -3.870000000000024, 0.8499999999998753, -2.790000000000047, -2.5400000000000524, 2.279999999999845, -2.3000000000000576, 4.059999999999807, -4.450000000000012, 1.3199999999998653, 4.529999999999797, -0.0600000000001053, -1.3200000000000784, -1.1700000000000816, -2.140000000000061, 1.1199999999998695, 4.789999999999791, 1.049999999999871, 0.10999999999989107, 4.6899999999997934, 1.9199999999998525, -3.260000000000037, -3.5200000000000315, -4.200000000000017, -4.98, 1.4999999999998614, 0.039999999999892566, 3.4299999999998203, 2.459999999999841, 0.6799999999998789, -1.6300000000000718, 3.6899999999998148, 3.779999999999813, -3.080000000000041, -0.9300000000000868, -0.04000000000010573, -1.9700000000000646, -2.9500000000000437, 0.10999999999989107, 1.9499999999998519, 1.9999999999998508, -3.450000000000033, -4.890000000000002, -2.750000000000048, 0.35999999999988574, 2.649999999999837, 2.03999999999985, -2.230000000000059, 2.309999999999844, 0.35999999999988574, 0.6899999999998787, 2.699999999999836, -1.0900000000000833, -3.4700000000000326, -0.2900000000001004, 1.9799999999998512, 1.4499999999998625, 3.7999999999998124, -2.8600000000000456, -3.630000000000029, 1.149999999999869, -3.030000000000042, -4.55000000000001, 4.009999999999808, -0.4600000000000968, 1.289999999999866, 3.169999999999826, -1.6800000000000708, -2.4000000000000554, -3.450000000000033, 2.309999999999844, -4.670000000000007, 2.2599999999998452, 2.1199999999998482, -2.5300000000000527, -4.610000000000008, -0.9300000000000868, -0.9800000000000857, -3.9800000000000217, 3.119999999999827, -1.0700000000000838, -2.65000000000005, -2.7300000000000484, -2.370000000000056, 4.659999999999794, 4.729999999999793, -1.2100000000000808, 1.5299999999998608, -1.0500000000000842, -4.210000000000017, -4.55000000000001, -3.5100000000000318, -2.3000000000000576, -0.7600000000000904, -2.3500000000000565, -4.900000000000002, -0.6200000000000934, 0.4499999999998838, -1.0800000000000836, -1.8300000000000676, 1.7499999999998561, -4.150000000000018, -1.6300000000000718, -1.6800000000000708, -1.150000000000082, -0.5600000000000946, 0.11999999999989086, -0.5000000000000959, -1.110000000000083, 2.7599999999998346, -2.850000000000046, 0.9899999999998723, -3.7100000000000275, 2.0699999999998493, 2.4399999999998414, 1.8099999999998548, 4.449999999999799, -0.4400000000000972, 3.9899999999998084, 0.039999999999892566, 1.4899999999998617, 0.5099999999998825, 0.17999999999988958, -4.200000000000017, -1.760000000000069, 3.4699999999998195, 0.5199999999998823, 3.7999999999998124, -4.400000000000013, -1.2200000000000806, 0.7499999999998774, -3.12000000000004, -1.25000000000008, 3.299999999999823, 4.6899999999997934, 1.6499999999998582, 1.049999999999871, -1.5000000000000746, -2.66000000000005, -4.760000000000005, -3.6100000000000296, 3.389999999999821, 3.159999999999826, 0.10999999999989107, 0.15999999999989, 0.2399999999998883, 0.5999999999998806, -2.3000000000000576, 4.769999999999792, 4.199999999999804, 0.19999999999988916, -4.0400000000000205, 1.4599999999998623, -2.6300000000000505, -3.770000000000026, 0.14999999999989022, -0.730000000000091, 3.5999999999998167, -3.920000000000023, 0.959999999999873, 1.1599999999998687, -1.0500000000000842, -4.0400000000000205, -3.070000000000041, 3.069999999999828, -0.6500000000000927, -3.8600000000000243, -3.6600000000000286, 1.239999999999867, -1.0658141036401503e-13, -1.6000000000000725, -2.100000000000062, -0.9500000000000863, 4.739999999999792, -4.640000000000008, 2.4799999999998406, -3.820000000000025, 2.739999999999835, -3.2300000000000377, -0.48000000000009635, -3.780000000000026, -0.450000000000097, -0.2200000000001019, 4.469999999999798, 0.8399999999998755, -4.860000000000003, 4.099999999999806, 1.2199999999998674, 1.6399999999998585, 4.929999999999788, 4.049999999999807, 1.4599999999998623, 4.739999999999792, 4.429999999999799, -2.1500000000000608, 1.7699999999998557, 1.469999999999862, 2.8099999999998335, -3.3300000000000356, 0.07999999999989171, -1.1900000000000812, 1.989999999999851, -1.7900000000000684, -2.1700000000000603, 2.7299999999998352, -3.7000000000000277, -2.240000000000059, 2.0499999999998497, -2.19000000000006, 0.6999999999998785, -1.8400000000000674, -1.1600000000000819, 1.0599999999998708, 2.409999999999842, -1.25000000000008, -3.770000000000026, -1.72000000000007, 3.869999999999811, 4.539999999999797, -1.72000000000007, -4.160000000000018, -3.9900000000000215, 1.429999999999863, -0.12000000000010402, -4.450000000000012, 2.4299999999998416, 2.8199999999998333, 1.2699999999998663, -4.920000000000002, 0.769999999999877, 4.289999999999802, -1.7300000000000697, 1.0799999999998704, -3.3900000000000343, 0.7899999999998766, 2.9599999999998303, 0.7999999999998764, -0.7900000000000897, 3.44999999999982, -1.0900000000000833, 1.9999999999998508, 3.0499999999998284, 0.7899999999998766, -0.9800000000000857, -4.000000000000021, -1.900000000000066, 0.27999999999988745, 1.3199999999998653, -4.770000000000005, -0.2800000000001006, -4.910000000000002, 2.599999999999838, 0.7799999999998768, -2.0700000000000625, 2.5799999999998384, -1.8900000000000663, -4.1800000000000175, 2.8199999999998333, 0.5499999999998817, -0.47000000000009656, -1.060000000000084, -2.140000000000061, 2.599999999999838, -4.760000000000005, 0.7499999999998774, 3.0899999999998276, -3.7200000000000273, 4.3899999999998, -4.210000000000017, 3.6999999999998145, -0.8200000000000891, 4.269999999999802, -1.7700000000000689, -0.04000000000010573, 2.97999999999983, -4.470000000000011, 1.6199999999998589, -1.580000000000073, 3.5999999999998167, 1.189999999999868, 1.4499999999998625, 0.41999999999988447, 2.229999999999846, -2.5800000000000516, -2.510000000000053, -0.41000000000009784, 2.599999999999838, -0.18000000000010274, 4.099999999999806, -0.5300000000000953, 0.28999999999988724, 4.679999999999794, -2.4000000000000554, -3.160000000000039, -0.34000000000009933, 1.0799999999998704, 4.189999999999804, 4.569999999999796, -0.39000000000009827, 2.079999999999849, -1.390000000000077, -2.3400000000000567, -4.05000000000002, -0.8800000000000878, -1.7100000000000701, -2.2200000000000593, 1.4999999999998614, -0.17000000000010296, -4.640000000000008, 1.379999999999864, -4.110000000000019, 0.8199999999998759, -3.1500000000000394, 1.5999999999998593, -4.560000000000009, 1.2299999999998672, -3.540000000000031, 2.689999999999836, -3.780000000000026, 0.8399999999998755, -2.0100000000000637, 1.3899999999998638, -1.3800000000000772, -3.9000000000000234, -0.33000000000009955, -3.6600000000000286, 4.059999999999807, -3.7500000000000266, 1.2999999999998657, 2.9999999999998295, -3.920000000000023, 0.14999999999989022, -0.8800000000000878, 2.369999999999843, 4.659999999999794, 3.4799999999998192, -3.920000000000023, -4.000000000000021, -2.510000000000053, 0.8699999999998749, -4.110000000000019, -0.5200000000000955, 2.179999999999847, 4.3799999999998, -3.60000000000003, 4.729999999999793, 0.1699999999998898, 0.019999999999892992, 1.759999999999856, 4.159999999999805, 1.0599999999998708, 0.5099999999998825, -0.6000000000000938, 1.609999999999859, 1.4399999999998627, -2.9600000000000435, 0.6499999999998796, 2.309999999999844, 3.069999999999828, -2.6700000000000497, 2.4699999999998408, -4.99, 1.5499999999998604, 3.0499999999998284, 1.0799999999998704, 1.799999999999855, -4.200000000000017, 4.3999999999998, -0.5200000000000955, -1.010000000000085, 3.6599999999998154, -1.0000000000000853, -3.60000000000003, 3.4799999999998192, -1.7400000000000695, -0.5400000000000951, 1.7399999999998563, 0.6699999999998791, 1.609999999999859, 0.8899999999998744, 4.799999999999791, 4.429999999999799, -3.7000000000000277, 2.1599999999998474, 0.2599999999998879, -3.8600000000000243, -2.6200000000000507, 0.489999999999883, -4.120000000000019, -3.730000000000027, -0.26000000000010104, 3.959999999999809, -1.530000000000074, -1.2800000000000793, -2.700000000000049, -1.3100000000000787, -5, -0.5200000000000955, 0.7499999999998774, 3.629999999999816, -3.2800000000000367, 4.909999999999789, -1.5500000000000735, 3.7599999999998133, 0.28999999999988724, -3.4200000000000337, -1.620000000000072, -0.6000000000000938, -2.4500000000000544, -3.270000000000037, 0.14999999999989022, -1.0500000000000842, -4.560000000000009, -2.1600000000000605, -0.5200000000000955, -1.1600000000000819, -3.0500000000000416, 3.3399999999998222, 0.42999999999988425, -1.4200000000000763, -0.3600000000000989, 3.2399999999998244, -3.5700000000000305, -1.7400000000000695, -2.7800000000000473, 4.7799999999997915, 1.239999999999867, 4.239999999999803, -0.8900000000000876, -2.7300000000000484, -1.480000000000075, 3.9299999999998096, -0.3700000000000987, -1.060000000000084, -0.6500000000000927, 2.139999999999848, 2.319999999999844, 3.6899999999998148, 3.7999999999998124, 3.859999999999811, -4.630000000000008, 0.9799999999998725, 4.019999999999808, 1.5499999999998604, 4.809999999999791, -4.470000000000011, -0.8300000000000889, -2.3500000000000565, 2.829999999999833, 3.3399999999998222, 3.119999999999827, -4.920000000000002, 0.09999999999989129, -1.1900000000000812, -2.7200000000000486, 4.139999999999805, -3.7500000000000266, -0.1400000000001036, 3.299999999999823, -1.1200000000000827, 3.5999999999998167, 3.309999999999823, -0.3600000000000989, 3.2399999999998244, 4.579999999999796, -1.5600000000000733, -3.7600000000000264, 3.029999999999829, -1.110000000000083, 3.249999999999824, -1.5500000000000735, 2.7199999999998354, 1.2699999999998663, -0.2700000000001008, -1.5400000000000738, 1.6899999999998574, 1.1299999999998693, 3.629999999999816, 1.3399999999998649, -0.11000000000010424, 4.799999999999791, 1.3899999999998638, -4.590000000000009, 0.26999999999988766, 0.43999999999988404, -4.860000000000003, -1.5700000000000731, 4.699999999999793, 4.8299999999997905, -0.40000000000009805, 1.5499999999998604, 3.7499999999998135, 0.28999999999988724, -3.9900000000000215, 1.849999999999854, -4.340000000000014, 1.469999999999862, 2.4699999999998408, 0.8299999999998757, 2.2899999999998446, -3.6500000000000288, 3.2299999999998246, 4.959999999999788, 3.819999999999812, 0.41999999999988447, -2.050000000000063, -4.430000000000012, 3.679999999999815, -3.8600000000000243, -3.6500000000000288, -0.6400000000000929, -1.7000000000000703, -4.880000000000003, 4.319999999999801, 2.9599999999998303, 2.96999999999983, 0.1699999999998898, -3.4300000000000335, 3.9799999999998086, -0.6300000000000932, -0.40000000000009805, -2.9100000000000446, 0.17999999999988958, 4.059999999999807, -2.990000000000043, 1.659999999999858, 4.449999999999799, 3.3199999999998226, 4.649999999999794, 3.679999999999815, -3.5700000000000305, -3.8900000000000237, -4.270000000000016, 1.8799999999998533, -1.9600000000000648, 0.7299999999998779, 1.4399999999998627, -0.5500000000000949, -3.3700000000000347, 4.269999999999802, 4.759999999999792, 3.6499999999998156, -2.7800000000000473, -1.8700000000000667, 0.28999999999988724, -1.200000000000081, 3.309999999999823, 1.3199999999998653, -3.59000000000003, -4.240000000000016, 3.2799999999998235, 3.0999999999998273, 0.9899999999998723, 3.259999999999824, -2.3600000000000563, -0.9000000000000874, -1.3100000000000787, 0.05999999999989214, 4.449999999999799, 1.6299999999998587, -4.06000000000002, 4.679999999999794, 2.3499999999998433, 1.4099999999998634, -2.790000000000047, 4.059999999999807, 0.7599999999998772, 4.439999999999799, 3.7999999999998124, 1.8299999999998544, 4.789999999999791, -1.6300000000000718, 0.05999999999989214, -1.7000000000000703, 1.7499999999998561, -4.970000000000001, -0.7500000000000906, 2.829999999999833, 0.9499999999998732, 2.2899999999998446, 2.139999999999848, -1.8300000000000676, -4.190000000000017, 0.9299999999998736, 4.679999999999794, 4.089999999999806, -1.9700000000000646, 1.4799999999998619, 0.02999999999989278, 2.8199999999998333, 4.119999999999806, 2.9099999999998314, -4.030000000000021, 3.5099999999998186, 1.2699999999998663, 2.5799999999998384, -0.4400000000000972, 2.559999999999839, -1.810000000000068, -3.13000000000004, 0.18999999999988937, -3.0600000000000414, -2.1200000000000614, -0.5400000000000951, -2.2900000000000578, 4.3799999999998, 0.4699999999998834, -3.260000000000037, 3.399999999999821, -4.440000000000012, -3.040000000000042, 2.179999999999847, 0.2499999999998881, -2.790000000000047, -1.9900000000000642, -0.48000000000009635, -0.6700000000000923, -0.5200000000000955, 3.5699999999998173, -0.4400000000000972, -3.4200000000000337, -0.26000000000010104, -1.3800000000000772, 4.139999999999805, 2.779999999999834, 2.409999999999842, 0.42999999999988425, 3.399999999999821, 3.679999999999815, -0.8200000000000891, -0.34000000000009933, -2.5400000000000524, -2.8700000000000454, -3.9800000000000217, -3.3800000000000345, -3.2800000000000367, 1.659999999999858, -4.260000000000016, 0.34999999999988596, 3.2299999999998246, 3.1899999999998254, -4.950000000000001, -2.9600000000000435, -4.850000000000003, -4.590000000000009, -2.8300000000000463, -1.72000000000007, -3.740000000000027, 1.5599999999998602, 2.9599999999998303, 1.2799999999998661, -4.4600000000000115, 3.43999999999982, 1.7799999999998555, 1.8799999999998533, 2.8199999999998333, 4.229999999999803, 3.719999999999814, 2.4299999999998416, -2.5900000000000514, -0.24000000000010147, 3.4799999999998192, -4.07000000000002, 0.18999999999988937, -0.9500000000000863, 4.339999999999801, -2.700000000000049, 0.7899999999998766, 0.4699999999998834, -2.980000000000043, 4.669999999999794, -0.6100000000000936, -1.0500000000000842, -4.710000000000006, -0.47000000000009656, -3.4200000000000337, 1.049999999999871, 4.119999999999806, -3.7200000000000273, 4.439999999999799, 2.889999999999832, 4.339999999999801, 3.2899999999998233, 4.83999999999979, 4.269999999999802, -0.3000000000001002, 3.859999999999811, 1.1199999999998695, -0.9800000000000857, -2.8600000000000456, 4.419999999999799, 3.7499999999998135, 2.4399999999998414, -0.6200000000000934, -4.470000000000011, -3.9000000000000234, 0.5599999999998815, -4.99, -1.8500000000000671, 3.3799999999998214, -2.270000000000058, -4.850000000000003, -1.5400000000000738, 4.009999999999808, -4.6500000000000075, 0.20999999999988894, 2.8199999999998333, -0.9800000000000857, -0.8600000000000882, -2.940000000000044, -4.55000000000001, -3.790000000000026, -3.220000000000038, 3.0999999999998273, -0.15000000000010338, -0.3800000000000985, 3.259999999999824, -1.5500000000000735, 1.5399999999998606, -2.2000000000000597, 2.5699999999998386, 3.4599999999998197, 3.639999999999816, 0.13999999999989043, 1.6299999999998587, -1.670000000000071, 1.4799999999998619, 3.9299999999998096, -1.1800000000000814, -0.5600000000000946, -0.7700000000000902, 1.1399999999998691, -0.12000000000010402, -4.300000000000015, -0.15000000000010338, 0.05999999999989214, -1.6500000000000714, -3.960000000000022, 2.609999999999838, 3.2399999999998244, -2.4900000000000535, 3.5999999999998167, -3.5800000000000303, 4.429999999999799, -0.3700000000000987, 2.9999999999998295, -4.890000000000002, -2.8600000000000456, 3.90999999999981, -0.5400000000000951, -0.5000000000000959, -4.140000000000018, 1.2999999999998657, -4.1300000000000185, 3.819999999999812, -2.430000000000055, -0.7700000000000902, 3.5499999999998177, 2.319999999999844, 4.019999999999808, -2.980000000000043, 4.109999999999806, 2.319999999999844, -3.3400000000000354, 0.8299999999998757, -1.4000000000000767, -0.2200000000001019, 4.099999999999806, -2.3000000000000576, -4.770000000000005, -0.10000000000010445, 2.3499999999998433, -3.60000000000003, -0.7000000000000917, -0.11000000000010424, -1.5200000000000742, -1.5400000000000738, -2.7800000000000473, 0.04999999999989235, 2.639999999999837, 1.2299999999998672, -1.1200000000000827, -4.570000000000009, 0.14999999999989022, -0.5300000000000953, -2.980000000000043, -3.5300000000000313, 2.1199999999998482, 1.6299999999998587 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "x" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "y" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_ackley_0, x=\"x\", y=\"y\", color=\"score\", color_continuous_scale=color_scale)\n", "fig.update_layout(width=900, height=800)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "unsigned-fleet", "metadata": {}, "source": [ "\n", "\n", "The plot above shows the random search exploring the ackley function. Random search is not affected by the many local optima in the search space. Lets try out the hill climbing algorithm on the ackley function and see the results." ] }, { "cell_type": "markdown", "id": "subsequent-twenty", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": 18, "id": "nervous-denial", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " x y score\n", "0 -5.00 4.99 -12.637734\n", "1 -4.55 4.99 -13.998385\n", "2 -4.75 4.99 -13.522024\n", "3 -4.85 4.75 -13.719183\n", "4 -4.81 4.99 -13.233342\n", ".. ... ... ...\n", "95 -4.82 4.68 -13.984259\n", "96 -5.00 4.99 -12.637734\n", "97 -5.00 4.99 -12.637734\n", "98 -4.66 4.55 -14.280916\n", "99 -5.00 4.99 -12.637734\n", "\n", "[100 rows x 3 columns] \n", "\n" ] } ], "source": [ "optimizer = HillClimbingOptimizer(rand_rest_p=0)\n", "\n", "hyper_ackley_1 = Hyperactive(verbosity=False)\n", "hyper_ackley_1.add_search(ackley_function, \n", " search_space, \n", " n_iter=100, \n", " optimizer=optimizer, \n", " initialize={\"vertices\": 1})\n", "hyper_ackley_1.run()\n", "\n", "search_data_ackley_1 = hyper_ackley_1.search_data(ackley_function)" ] }, { "cell_type": "code", "execution_count": 19, "id": "oriented-stone", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "x=%{x}
y=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ -12.63773422539314, -13.99838467810746, -13.522023938559908, -13.719183095616176, -13.23334245248374, -14.005416090758251, -12.63773422539314, -12.650569689555564, -13.339219254818714, -12.666522088415933, -13.457207132787142, -13.754780201672801, -13.087732784735302, -13.03836008533632, -12.816406058864704, -13.968835425829134, -13.42770042669916, -13.63571007077306, -12.63773422539314, -13.685174689069648, -13.436197953066547, -13.130902557251082, -12.63773422539314, -12.63839218074661, -14.001772885546801, -13.94857762144471, -12.63773422539314, -12.824889111904746, -12.63773422539314, -12.851030667444382, -13.2893948808159, -13.647254647598617, -13.907590732779939, -13.98757858461055, -12.63773422539314, -14.105236170843675, -13.001674762403498, -12.741620865899048, -12.63773422539314, -12.712328979746102, -13.382333121649978, -13.436197953066547, -13.03836008533632, -12.63773422539314, -13.8495565891491, -12.63773422539314, -12.938157275663741, -12.63773422539314, -13.910434723399582, -13.893785345829334, -13.952314054388506, -12.63773422539314, -12.63773422539314, -12.63773422539314, -12.727786171026672, -12.639561542646591, -13.032960493728229, -13.653552615690007, -12.687196277323526, -12.727786171026974, -12.63773422539314, -13.873140118090825, -13.382333121649978, -12.63773422539314, -13.79288731505283, -13.457207132787142, -13.252800680854202, -12.893436060597171, -13.76852921759267, -12.63773422539314, -13.791786456822388, -13.945699511323863, -13.382333121649978, -13.873569480704017, -13.890637950457995, -12.692054471295332, -12.687196277323526, -13.936328864638384, -12.63773422539314, -12.774741826488345, -13.941395537459183, -12.63773422539314, -12.633675184948402, -13.647254647598617, -13.03512008682262, -13.8495565891491, -12.63773422539314, -12.63773422539314, -12.63773422539314, -13.430226929505066, -12.633039893443069, -12.63773422539314, -12.63773422539314, -13.92423849099691, -13.23334245248374, -13.984259182699233, -12.63773422539314, -12.63773422539314, -14.28091573847518, -12.63773422539314 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ -5, -4.55000000000001, -4.750000000000005, -4.850000000000003, -4.810000000000004, -5, -5, -4.960000000000001, -5, -4.950000000000001, -4.7900000000000045, -4.690000000000007, -5, -5, -5, -4.490000000000011, -4.770000000000005, -4.200000000000017, -5, -4.710000000000006, -5, -4.98, -5, -5, -5, -5, -5, -4.960000000000001, -5, -4.890000000000002, -5, -4.720000000000006, -4.900000000000002, -4.960000000000001, -5, -4.810000000000004, -4.900000000000002, -4.920000000000002, -5, -4.9300000000000015, -4.780000000000005, -5, -5, -5, -5, -5, -4.870000000000003, -5, -4.920000000000002, -5, -4.860000000000003, -5, -5, -5, -4.9300000000000015, -4.970000000000001, -4.850000000000003, -5, -4.940000000000001, -4.960000000000001, -5, -4.660000000000007, -4.780000000000005, -5, -5, -4.7900000000000045, -4.830000000000004, -4.880000000000003, -4.760000000000005, -5, -5, -4.610000000000008, -4.780000000000005, -4.330000000000014, -4.640000000000008, -5, -4.940000000000001, -5, -5, -4.910000000000002, -4.4600000000000115, -5, -4.98, -4.720000000000006, -4.850000000000003, -5, -5, -5, -5, -4.770000000000005, -4.99, -5, -5, -4.410000000000013, -4.810000000000004, -4.820000000000004, -5, -5, -4.660000000000007, -5 ], "xaxis": "x", "y": [ 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.749999999999792, 4.989999999999787, 4.549999999999796, 4.989999999999787, 4.989999999999787, 4.789999999999791, 4.989999999999787, 4.889999999999789, 4.989999999999787, 4.83999999999979, 4.84999999999979, 4.899999999999789, 4.949999999999788, 4.979999999999787, 4.4999999999997975, 4.989999999999787, 4.989999999999787, 4.769999999999792, 4.8299999999997905, 4.989999999999787, 4.979999999999787, 4.559999999999796, 4.459999999999798, 4.989999999999787, 4.899999999999789, 4.989999999999787, 4.989999999999787, 4.799999999999791, 4.989999999999787, 4.649999999999794, 4.519999999999797, 4.989999999999787, 4.609999999999795, 4.889999999999789, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.769999999999792, 4.84999999999979, 4.989999999999787, 4.659999999999794, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.439999999999799, 4.429999999999799, 4.429999999999799, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.959999999999788, 4.989999999999787, 4.989999999999787, 4.719999999999793, 4.989999999999787, 4.929999999999788, 4.989999999999787, 4.909999999999789, 4.989999999999787, 4.989999999999787, 4.679999999999794, 4.889999999999789, 4.899999999999789, 4.989999999999787, 4.819999999999791, 4.989999999999787, 4.3899999999998, 4.989999999999787, 4.989999999999787, 4.749999999999792, 4.989999999999787, 4.939999999999788, 4.989999999999787, 4.619999999999795, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.9699999999997875, 4.659999999999794, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.989999999999787, 4.84999999999979, 4.989999999999787, 4.679999999999794, 4.989999999999787, 4.989999999999787, 4.549999999999796, 4.989999999999787 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "x" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "y" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_ackley_1, x=\"x\", y=\"y\", color=\"score\", color_continuous_scale=color_scale)\n", "fig.update_layout(width=900, height=800, xaxis_range=[-5, 5], yaxis_range=[-5, 5])\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "auburn-projector", "metadata": {}, "source": [ "\n", "\n", "Maybe you already expected, that the hill climbing algorithm delivers bad results optimizing the ackley function. That does not mean, that hill climbing is a bad algorithm in general. It means that it is bad for this kind of objective functions. This is a very important idea in mathematical optimization. It is very useful to know about the properties of the objective function, because you can choose an optimization algorithm that works really well for this problem." ] }, { "cell_type": "markdown", "id": "original-province", "metadata": {}, "source": [ "\n", " \n", "### Repulsing Hill Climbing Optimizer\n", " \n", "The repulsing hill climbing optimizer tries to improve how hill climbing solves non-convex objective functions. It does so by increasing the radius in which hill climbing selects a neighbour position if the last position wasn't an improvement over the current one. This means the hill climber will jump away from its current position of it does not find better position in its close environment." ] }, { "cell_type": "markdown", "id": "destroyed-pilot", "metadata": {}, "source": [ "\n", "\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 20, "id": "antique-rating", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " x y score\n", "0 4.99 4.99 -12.633040\n", "1 4.92 4.99 -12.741621\n", "2 -1.34 0.02 -4.914987\n", "3 4.99 4.93 -12.712329\n", "4 -0.81 1.72 -6.338818\n", ".. ... ... ...\n", "95 -3.54 0.89 -9.877024\n", "96 0.50 0.26 -3.662621\n", "97 -1.50 0.79 -6.295015\n", "98 -1.65 0.34 -6.387955\n", "99 -0.08 0.71 -3.271799\n", "\n", "[100 rows x 3 columns] \n", "\n" ] } ], "source": [ "from hyperactive.optimizers import RepulsingHillClimbingOptimizer\n", "\n", "optimizer = RepulsingHillClimbingOptimizer()\n", "\n", "hyper_ackley_2 = Hyperactive(verbosity=False)\n", "hyper_ackley_2.add_search(ackley_function, \n", " search_space, \n", " n_iter=100, \n", " optimizer=optimizer, \n", " initialize={\"vertices\": 1})\n", "hyper_ackley_2.run()\n", "\n", "search_data_ackley_2 = hyper_ackley_2.search_data(ackley_function)" ] }, { "cell_type": "code", "execution_count": 21, "id": "architectural-crown", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "x=%{x}
y=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ -12.633039893443026, -12.741620865899712, -4.914986993321801, -12.712328979746683, -6.338818228398505, -7.035122609311411, -8.700732623357963, -5.038936263647248, -4.927840439662866, -6.341804763619894, -6.095954671431354, -11.99624661584065, -10.094732293167922, -6.117698257392796, -6.124969314772262, -5.351660596096632, -7.783789021881949, -9.453835027434911, -8.090604434195164, -6.50338541514969, -6.409428905146008, -5.785151304509068, -7.417073593402394, -10.996031048338725, -8.329116391354878, -5.423220348246922, -8.56891872891406, -6.579221910370244, -5.444731262459193, -8.389772363137904, -8.933866300709003, -10.509445600721467, -3.900364433607205, -4.2205087135881385, -9.571620295147238, -6.190057311187804, -7.937743555193856, -6.779911414508575, -8.51794603013101, -6.903679721878106, -8.061038571148016, -12.10645740346628, -6.364728682122859, -6.164164556830606, -2.8031051972319077, -4.671982290767751, -6.495398918051908, -10.524974796501269, -6.363046942033982, -6.188772939923098, -7.814036658733551, -9.217431736739297, -3.896715818219949, -4.647755723333368, -9.198725999070207, -4.547118924915729, -9.047278839669778, -4.371317922280841, -7.182091953423692, -5.81154454471282, -6.7267580096001005, -8.965945501760691, -4.532074507060088, -5.4740220365442696, -6.560039619998847, -4.549970891192327, -5.232952269246695, -3.613429444186913, -6.416632806042827, -11.821004389483463, -7.6022678446672, -6.532830396719426, -7.802749821709956, -8.320152581427415, -3.939536575361977, -11.544392823482069, -4.888908767986319, -4.15246666774323, -6.723775863342224, -5.774292298873199, -9.124740541318573, -6.455216318424323, -11.59978159947337, -5.074876601337422, -11.327686363679637, -8.113081439047361, -7.3084827770660645, -6.151483578720164, -9.212326478074429, -5.5630535065616975, -2.2117487208028166, -10.589587722302271, -6.009727543684118, -4.5238647878694564, -4.699346743094178, -9.87702394936716, -3.6626206589287236, -6.29501534843671, -6.387955007558702, -3.271798982576854 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 4.989999999999787, 4.9199999999997885, -1.340000000000078, 4.989999999999787, -0.8100000000000893, 1.2699999999998663, -2.1600000000000605, -1.3600000000000776, -0.33000000000009955, -2.1200000000000614, 1.2199999999998674, -5, -3.8400000000000247, -1.480000000000075, 0.9699999999998727, -1.1900000000000812, 1.009999999999872, -3.870000000000024, -2.6700000000000497, 0.34999999999988596, -1.060000000000084, -2.1500000000000608, -1.4400000000000759, -5, -2.2000000000000597, -1.2100000000000808, -2.8700000000000454, -1.6400000000000716, -1.3300000000000782, -3.210000000000038, -3.960000000000022, -3.4200000000000337, -1.0800000000000836, -1.2200000000000806, -3.3900000000000343, 1.2099999999998676, -0.8100000000000893, -0.6700000000000923, -1.25000000000008, -0.4400000000000972, -2.3900000000000556, -4.290000000000015, -1.2800000000000793, -1.810000000000068, 0.04999999999989235, -1.1800000000000814, 0.7399999999998776, -3.180000000000039, 1.049999999999871, -1.2600000000000797, 1.149999999999869, -1.0300000000000846, 1.0899999999998702, 0.7799999999998768, 2.5399999999998393, 0.6499999999998796, 1.6799999999998576, 0.5999999999998806, -2.2600000000000584, -0.920000000000087, 2.0199999999998504, 2.229999999999846, -0.7600000000000904, -0.2100000000001021, -1.6500000000000714, -0.8100000000000893, -0.7500000000000906, 0.17999999999988958, 0.7799999999998768, -0.39000000000009827, 0.8499999999998753, 1.7299999999998565, 0.8999999999998742, -1.9300000000000654, 0.6099999999998804, 0.28999999999988724, -1.2600000000000797, 0.35999999999988574, 1.429999999999863, -1.290000000000079, 3.0099999999998293, 1.3999999999998636, 3.6499999999998156, 0.07999999999989171, -3.8000000000000256, -1.0000000000000853, -0.450000000000097, 1.519999999999861, -1.4700000000000752, -1.0400000000000844, -0.3100000000001, -3.6600000000000286, 0.21999999999988873, -0.5800000000000942, 0.12999999999989065, -3.540000000000031, 0.49999999999988276, -1.5000000000000746, -1.6500000000000714, -0.08000000000010488 ], "xaxis": "x", "y": [ 4.989999999999787, 4.989999999999787, 0.019999999999892992, 4.929999999999788, 1.7199999999998568, -1.760000000000069, 2.279999999999845, 0.039999999999892566, -1.1800000000000814, 0.21999999999988873, -1.2700000000000795, 1.3899999999998638, -0.33000000000009955, 0.33999999999988617, 2.139999999999848, -0.6000000000000938, -2.7700000000000475, 0.8399999999998755, -1.0300000000000846, -2.0100000000000637, 1.5799999999998597, -0.04000000000010573, -1.4900000000000748, -0.2100000000001021, -1.5700000000000731, -0.6300000000000932, 0.43999999999988404, 0.6699999999998791, 0.2599999999998879, 0.11999999999989086, -1.060000000000084, 1.469999999999862, 0.9799999999998725, 0.0899999999998915, 1.0799999999998704, 1.2999999999998657, 2.3999999999998423, 2.0699999999998493, 2.4399999999998414, 2.0699999999998493, 1.149999999999869, 2.749999999999835, 1.2699999999998663, -0.8100000000000893, 1.0299999999998715, 0.8899999999998744, 1.709999999999857, 2.7199999999998354, 1.6399999999998585, 1.2499999999998668, 2.3299999999998438, 3.3299999999998224, -0.2100000000001021, 1.1199999999998695, 1.5399999999998606, -0.8200000000000891, 2.9599999999998303, 0.4799999999998832, 1.0799999999998704, 1.8199999999998546, 2.03999999999985, 2.269999999999845, -0.5200000000000955, -1.3700000000000774, 1.1199999999998695, 0.5399999999998819, -1.2100000000000808, 0.6399999999998798, 1.709999999999857, 4.7799999999997915, 2.9599999999998303, 0.3999999999998849, 3.0999999999998273, 3.0799999999998278, -0.2900000000001004, 4.459999999999798, 0.19999999999988916, 0.62999999999988, 1.2299999999998672, 0.579999999999881, 2.219999999999846, 1.1799999999998683, 2.549999999999839, -1.9900000000000642, 2.7199999999998354, 2.6299999999998374, -2.1700000000000603, -0.3100000000001, 2.97999999999983, 1.899999999999853, -0.020000000000106155, -2.1100000000000616, -2.050000000000063, 0.6099999999998804, 1.2699999999998663, 0.8899999999998744, 0.2599999999998879, 0.7899999999998766, 0.33999999999988617, 0.7099999999998783 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "x" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "range": [ -5, 5 ], "title": { "text": "y" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_ackley_2, x=\"x\", y=\"y\", color=\"score\", color_continuous_scale=color_scale)\n", "fig.update_layout(width=900, height=800, xaxis_range=[-5, 5], yaxis_range=[-5, 5])\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "mounted-speech", "metadata": {}, "source": [ "\n", " \n", "The plot above shows how the repulsing hill climbing optimizer explored the search space of the ackley function. It does a much better job finding new optima in the space, while also exploring local regions." ] }, { "cell_type": "markdown", "id": "minor-butler", "metadata": {}, "source": [ "\n", " \n", "## Hyperactive memory \n", " \n", "Some objective functions (especially with machein- or deep- learning models) can take a long time to evaluate, which slows down the optimization run. This means, that we want to avoid any unnecessary evaluation of the objective function. Unfortunately most optimization algorithms won't avoid positions in the search space that were already evaluated. For example:\n", " \n", " - Random Search, which could select a position it already selected before\n", " - Hill climbing stuck in an optimum\n", " - Particle swarms that converge on one position\n", " \n", "The bottom line is, that optimization algorithms don't \"remember\" already explored positions and won't avoid them. But Hyperactive has a feature that solves this problem, by saving the position and score in a memory-dictionary. If a position is selected Hyperactive will look up if this position is already known. If it knows the position and score it won't reevaluate the objective function but extract the score, which saves time. This is very useful for computationally expensive objective functions.\n", " \n", "You can even pass the search data from a previous optimization run into the ```memory_warm_start```-argument, so that the optimization run \"remembers\" the memory from the previous one." ] }, { "cell_type": "code", "execution_count": 22, "id": "enormous-diagnosis", "metadata": {}, "outputs": [], "source": [ "def dtr_model(opt):\n", " dtr = DecisionTreeRegressor(\n", " max_depth=opt[\"max_depth\"],\n", " min_samples_split=opt[\"min_samples_split\"],\n", " )\n", " scores = cross_val_score(dtr, X_boston, y_boston, cv=5)\n", "\n", " return scores.mean()\n", "\n", "\n", "search_space_dtr = {\n", " \"max_depth\": list(range(10, 35)),\n", " \"min_samples_split\": list(range(2, 35)),\n", "}" ] }, { "cell_type": "code", "execution_count": 23, "id": "latter-morning", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " max_depth min_samples_split score\n", "0 10 21 0.229979\n", "1 33 5 0.294730\n", "2 18 12 0.229979\n", "3 18 22 0.317003\n", "4 26 12 0.205503\n", ".. ... ... ...\n", "295 15 2 0.131560\n", "296 11 12 0.317003\n", "297 20 32 0.317003\n", "298 34 32 0.292527\n", "299 21 9 0.207706\n", "\n", "[300 rows x 3 columns] \n", "\n", "Optimization time 1: 1.1\n" ] } ], "source": [ "c_time1 = time.time()\n", "\n", "hyper_dtr_0 = Hyperactive(verbosity=False)\n", "hyper_dtr_0.add_search(dtr_model, search_space_dtr, n_iter=300)\n", "hyper_dtr_0.run()\n", "\n", "d_time1 = time.time() - c_time1\n", "print(\"Optimization time 1:\", round(d_time1, 2))\n", "\n", "# Hyperactive collects the search data\n", "search_data_dtr_0 = hyper_dtr_0.search_data(dtr_model, times=True)" ] }, { "cell_type": "markdown", "id": "charming-entity", "metadata": {}, "source": [ "\n", "\n", "After the first optimization run we start an additional run and pass the search data into ```memory_warm_start```. We expect that the next run will be faster, because we already have explored 300 positions in the search space from the previous optimization run." ] }, { "cell_type": "markdown", "id": "baking-monthly", "metadata": {}, "source": [ "\n", " \n", " " ] }, { "cell_type": "markdown", "id": "informed-berlin", "metadata": {}, "source": [ "\n", " \n" ] }, { "cell_type": "code", "execution_count": 24, "id": "perfect-philadelphia", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " max_depth min_samples_split score\n", "0 17 29 0.229979\n", "1 31 34 0.317003\n", "2 18 12 0.229979\n", "3 18 22 0.317003\n", "4 26 12 0.205503\n", ".. ... ... ...\n", "295 12 6 0.183230\n", "296 34 6 0.245779\n", "297 13 21 0.317003\n", "298 32 33 0.292527\n", "299 18 18 0.292527\n", "\n", "[300 rows x 3 columns] \n", "\n", "Optimization time 2: 0.76\n" ] } ], "source": [ "c_time2 = time.time()\n", "\n", "hyper_dtr_1 = Hyperactive(verbosity=False)\n", "hyper_dtr_1.add_search(dtr_model, search_space_dtr, n_iter=300, memory_warm_start=search_data_dtr_0)\n", "hyper_dtr_1.run()\n", "\n", "d_time2 = time.time() - c_time2\n", "print(\"Optimization time 2:\", round(d_time2, 2))" ] }, { "cell_type": "code", "execution_count": 25, "id": "prerequisite-graham", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " The second optimization run is 31.12% faster than the first one.\n" ] } ], "source": [ "print(\"\\n The second optimization run is \"+'{}'.format(round((1-d_time2/d_time1)*100,2))+\"% faster than the first one.\")" ] }, { "cell_type": "code", "execution_count": 26, "id": "hollow-swift", "metadata": {}, "outputs": [], "source": [ "search_data_dtr_1 = hyper_dtr_1.search_data(dtr_model, times=True)\n", "search_data_dtr = search_data_dtr_1.append(search_data_dtr_0, ignore_index=True)" ] }, { "cell_type": "code", "execution_count": 27, "id": "exterior-forward", "metadata": {}, "outputs": [], "source": [ "# times in seconds\n", "eval_times = search_data_dtr_0[\"eval_times\"]\n", "eval_times_mem = search_data_dtr_1[\"eval_times\"]\n", "\n", "opt_times = search_data_dtr[\"iter_times\"]-search_data_dtr[\"eval_times\"]" ] }, { "cell_type": "code", "execution_count": 28, "id": "adult-terminal", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "name": "evaluation time", "nbinsx": 15, "type": "histogram", "x": [ 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906, 1.0681495666503906 ] }, { "name": "evaluation time second run", "nbinsx": 15, "type": "histogram", "x": [ 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291, 0.7258913516998291 ] }, { "name": "optimization time", "nbinsx": 15, "type": "histogram", "x": [ 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0054361820220947266, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125, 0.0062580108642578125 ] } ], "layout": { "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = go.Figure()\n", "fig.add_trace(go.Histogram(x=eval_times, name=\"evaluation time\", nbinsx=15))\n", "fig.add_trace(go.Histogram(x=eval_times_mem, name=\"evaluation time second run\", nbinsx=15))\n", "fig.add_trace(go.Histogram(x=opt_times, name=\"optimization time\", nbinsx=15))\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "abroad-traveler", "metadata": {}, "source": [ "\n", " \n", "The evaluation- and optimization- times from the two optimization runs is shown in the histogram above. There are several interesting things to see:\n", " \n", " - Even for simple machine learning models the optimization algorithm is much faster.\n", " - The evaluations, which are faster than the optimization are from the memory-dictionary lookup.\n", " - The second optimization run has much more memory lookups than the first.\n", "\n" ] }, { "cell_type": "markdown", "id": "chicken-truck", "metadata": {}, "source": [ "\n", " \n", "## Machine Learning Hyperparameter Optimization \n", "\n", "Until now we mostly optimized test functions to show how an objective function and the search space can look like. These problems were easy to solve, because the objective funtion evaluates very fast and the search space is very small. Real optimization problems often have one of those two problems:\n", " - The objective function is computationally expensive, so it takes a long time to evaluate. This increases the iteration time and slowes down the optimization progress.\n", " - The search space is very large. This can makes it very difficult to find positions with a high score.\n", " \n", "In the first case you would want to use optimization algorithms that are very inteligent in finding new positions with high scores. You don't want to waste too much time exploring the search space, because each evaluation takes such a long time. You want to get to a good position with a high score in as few steps as possible.\n", " \n", "In the second case you would want a fast algorithm, that looks for a good score but also explores the search space very well." ] }, { "cell_type": "markdown", "id": "moved-shield", "metadata": {}, "source": [ "\n", " \n", "Lets take a look at a (kind of) real optimization problem. We want to optimize the hyperparameters of a gradient boosting regressor that is trained on the boston housing regression dataset." ] }, { "cell_type": "code", "execution_count": 29, "id": "fiscal-stick", "metadata": {}, "outputs": [], "source": [ "def gbr_model_0(opt):\n", " gbr = GradientBoostingRegressor(\n", " n_estimators=opt[\"n_estimators\"],\n", " max_depth=opt[\"max_depth\"],\n", " )\n", " scores = cross_val_score(gbr, X_boston, y_boston, cv=5)\n", " score = scores.mean()\n", " return score\n", "\n", "\n", "search_space_gbr_0 = {\n", " \"n_estimators\": list(range(10, 100)),\n", " \"max_depth\": list(range(2, 12)),\n", "}" ] }, { "cell_type": "code", "execution_count": 30, "id": "whole-fever", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 41 11 0.286493\n", "1 10 11 0.158985\n", "2 39 5 0.269010\n", "3 39 8 0.289922\n", "4 68 5 0.285994\n", "5 68 8 0.287513\n", "6 10 11 0.158985\n", "7 99 11 0.275180\n", "8 10 2 0.130693\n", "9 99 2 0.326558\n", "10 83 3 0.309352\n", "11 49 2 0.324265\n", "12 31 6 0.284971\n", "13 84 4 0.272758\n", "14 53 9 0.280875\n", "15 53 2 0.324899\n", "16 16 10 0.260080\n", "17 15 3 0.265057\n", "18 66 5 0.261062\n", "19 30 6 0.293972\n", "20 50 3 0.315253\n", "21 63 4 0.271137\n", "22 33 6 0.273074\n", "23 68 10 0.285651\n", "24 22 5 0.269218\n", "25 12 7 0.199622\n", "26 42 7 0.271228\n", "27 44 3 0.305420\n", "28 96 5 0.276716\n", "29 43 2 0.323368\n", "30 12 8 0.199739\n", "31 45 11 0.289907\n", "32 41 11 0.286493\n", "33 43 6 0.260725\n", "34 77 10 0.282316\n", "35 81 8 0.267947\n", "36 65 10 0.260475\n", "37 31 11 0.266904\n", "38 47 3 0.306781\n", "39 87 7 0.276266\n", "40 39 4 0.279326\n", "41 49 4 0.273381\n", "42 59 5 0.275943\n", "43 47 11 0.288363\n", "44 23 4 0.263894\n", "45 26 6 0.267758\n", "46 11 5 0.195222\n", "47 69 10 0.275137\n", "48 84 2 0.325823\n", "49 69 6 0.268853 \n", "\n" ] } ], "source": [ "hyper_gbr_0 = Hyperactive(verbosity=False)\n", "hyper_gbr_0.add_search(gbr_model_0, search_space_gbr_0, n_iter=50)\n", "hyper_gbr_0.run()\n", "\n", "search_data_gbr_0 = hyper_gbr_0.search_data(gbr_model_0)" ] }, { "cell_type": "code", "execution_count": 31, "id": "special-postcard", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "n_estimators=%{x}
max_depth=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.2864932483046703, 0.1589850511243395, 0.26901037829997904, 0.28992169693052544, 0.28599404679335455, 0.28751333137906476, 0.1589850511243395, 0.2751800172862603, 0.1306925369713726, 0.32655778323546214, 0.30935162265031246, 0.32426502490016196, 0.2849709177998162, 0.2727575953446778, 0.28087530253171955, 0.32489915895782406, 0.26008003019382936, 0.2650569881201294, 0.26106247861848986, 0.29397176343174775, 0.31525331864453837, 0.2711370261553595, 0.2730744498289349, 0.2856513571192761, 0.2692179483899363, 0.19962194274266837, 0.2712280243350414, 0.30541997088676426, 0.2767164177354497, 0.3233675374273507, 0.19973894957557853, 0.28990679666152813, 0.2864932483046703, 0.26072535848597533, 0.28231589476905794, 0.26794708070659545, 0.2604750111709577, 0.2669043365444564, 0.3067813824106441, 0.27626625375088537, 0.2793262279809792, 0.2733814131528477, 0.2759425741187724, 0.2883627862008063, 0.26389443143689467, 0.2677576687564798, 0.19522194680826815, 0.27513653004035454, 0.3258234859577797, 0.26885271431979885 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 41, 10, 39, 39, 68, 68, 10, 99, 10, 99, 83, 49, 31, 84, 53, 53, 16, 15, 66, 30, 50, 63, 33, 68, 22, 12, 42, 44, 96, 43, 12, 45, 41, 43, 77, 81, 65, 31, 47, 87, 39, 49, 59, 47, 23, 26, 11, 69, 84, 69 ], "xaxis": "x", "y": [ 11, 11, 5, 8, 5, 8, 11, 11, 2, 2, 3, 2, 6, 4, 9, 2, 10, 3, 5, 6, 3, 4, 6, 10, 5, 7, 7, 3, 5, 2, 8, 11, 11, 6, 10, 8, 10, 11, 3, 7, 4, 4, 5, 11, 4, 6, 5, 10, 2, 6 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "n_estimators" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "max_depth" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_gbr_0, \n", " x=\"n_estimators\", \n", " y=\"max_depth\", \n", " color=\"score\", \n", " color_continuous_scale=color_scale)\n", "\n", "fig.update_layout(width=900, height=800)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "reported-start", "metadata": {}, "source": [ "\n", " \n", "The scatter plot above contains the samples from the search data from the gbr-model. It seams that high ```max_depth``` delivers bad scores but we should explore higher values for ```n_estimators```." ] }, { "cell_type": "markdown", "id": "colonial-ordering", "metadata": {}, "source": [ "\n", "\n", "## Continuing the Search \n", "\n", "
\n", "\n", "Hyperactive makes it very easy to continue a search. The search data you already used for data exploration can just be passed to Hyperactive. This is done in multiple ways:\n", " \n", " - You can extract the best parameters via the \"best_para\"-method. This can than be passed to \"initialize\" to start at this position in the search space\n", " - The search data from the \"results\"-method can be passed to \"memory_warm_start\". The search data is automaticaly added into the memory-dictionary.\n", " - You can also pass the search data to \"warm_start_smbo\". This has the effect that the Bayesian optimizer can do more precise approximations in the beginning of the optimization run.\n", " " ] }, { "cell_type": "code", "execution_count": 32, "id": "rapid-clearing", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 45 6 0.260725\n", "1 25 6 0.278369\n", "2 30 2 0.311317\n", "3 130 6 0.288743\n", "4 100 2 0.326558\n", "5 80 3 0.313228\n", "6 240 7 0.282410\n", "7 20 2 0.289883\n", "8 210 7 0.285183\n", "9 25 3 0.305628\n", "10 55 6 0.276450\n", "11 105 4 0.290260\n", "12 180 3 0.306254\n", "13 175 4 0.258077\n", "14 20 5 0.243912\n", "15 175 7 0.264526\n", "16 235 2 0.329066\n", "17 200 5 0.279308\n", "18 80 7 0.267073\n", "19 195 2 0.329525\n", "20 70 2 0.326380\n", "21 165 4 0.268225\n", "22 245 5 0.260622\n", "23 105 3 0.308383\n", "24 145 4 0.276206\n", "25 75 3 0.312939\n", "26 60 3 0.312158\n", "27 155 3 0.310263\n", "28 150 5 0.262560\n", "29 155 5 0.280357\n", "30 130 7 0.272747\n", "31 155 3 0.310263\n", "32 220 3 0.313533\n", "33 40 2 0.320701\n", "34 75 4 0.228214\n", "35 230 7 0.272366\n", "36 180 2 0.328918\n", "37 75 6 0.266215\n", "38 10 7 0.176882\n", "39 175 6 0.271295\n", "40 95 4 0.292781\n", "41 25 4 0.263894\n", "42 50 4 0.273381\n", "43 240 3 0.312109\n", "44 90 4 0.271041\n", "45 145 2 0.328595\n", "46 65 3 0.315951\n", "47 220 4 0.275379\n", "48 30 2 0.311317\n", "49 175 2 0.329184 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 40 4 0.279326\n", "1 95 2 0.326284\n", "2 40 3 0.305565\n", "3 215 7 0.278450\n", "4 100 2 0.326558\n", "5 115 7 0.263180\n", "6 130 3 0.312360\n", "7 115 4 0.272206\n", "8 50 4 0.273381\n", "9 175 6 0.270705\n", "10 195 5 0.275754\n", "11 165 5 0.269701\n", "12 75 3 0.313818\n", "13 190 7 0.273318\n", "14 30 7 0.288718\n", "15 245 3 0.308790\n", "16 105 3 0.316376\n", "17 100 4 0.278197\n", "18 185 5 0.267187\n", "19 125 5 0.269066\n", "20 35 6 0.273074\n", "21 30 6 0.267758\n", "22 170 4 0.280237\n", "23 180 4 0.264906\n", "24 120 4 0.270004\n", "25 140 3 0.310266\n", "26 155 7 0.291970\n", "27 155 3 0.313230\n", "28 130 5 0.282460\n", "29 40 3 0.305565\n", "30 190 5 0.250403\n", "31 220 7 0.270510\n", "32 190 4 0.271958\n", "33 235 2 0.329018\n", "34 185 3 0.312214\n", "35 205 7 0.256337\n", "36 65 2 0.325732\n", "37 70 3 0.313170\n", "38 70 3 0.313170\n", "39 205 6 0.280859\n", "40 125 3 0.313643\n", "41 55 6 0.288828\n", "42 80 7 0.272950\n", "43 150 3 0.311705\n", "44 40 7 0.271878\n", "45 45 2 0.323368\n", "46 35 5 0.278078\n", "47 240 7 0.270968\n", "48 130 4 0.264316\n", "49 120 6 0.281479 \n", "\n" ] } ], "source": [ "best_para_gbr_0 = hyper_gbr_0.best_para(gbr_model_0)\n", "initialize = {\"random\": 4, \"warm_start\": [best_para_gbr_0]}\n", "\n", "\n", "search_space_gbr_01 = {\n", " \"n_estimators\": list(range(10, 250, 5)),\n", " \"max_depth\": list(range(2, 8)),\n", "}\n", "\n", "\n", "hyper_gbr_01 = Hyperactive(verbosity=False)\n", "hyper_gbr_01.add_search(gbr_model_0, \n", " search_space_gbr_01, \n", " n_iter=50,\n", " n_jobs=2,\n", " memory_warm_start=search_data_gbr_0, \n", " initialize=initialize)\n", "hyper_gbr_01.run()\n", "\n", "search_data_gbr_01 = hyper_gbr_01.search_data(gbr_model_0)" ] }, { "cell_type": "code", "execution_count": 33, "id": "handy-tiffany", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
n_estimatorsmax_depthscore
04040.279326
19520.326284
24030.305565
321570.278450
410020.326558
............
1452660.267758
1461150.195222
14769100.275137
1488420.325823
1496960.268853
\n", "

150 rows × 3 columns

\n", "
" ], "text/plain": [ " n_estimators max_depth score\n", "0 40 4 0.279326\n", "1 95 2 0.326284\n", "2 40 3 0.305565\n", "3 215 7 0.278450\n", "4 100 2 0.326558\n", ".. ... ... ...\n", "145 26 6 0.267758\n", "146 11 5 0.195222\n", "147 69 10 0.275137\n", "148 84 2 0.325823\n", "149 69 6 0.268853\n", "\n", "[150 rows x 3 columns]" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# merge the search data from the previous run and the current run\n", "search_data_gbr_01_ = search_data_gbr_01.append(search_data_gbr_0, ignore_index=True)\n", "search_data_gbr_01_" ] }, { "cell_type": "code", "execution_count": 34, "id": "significant-athletics", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "n_estimators=%{x}
max_depth=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.2793262279809792, 0.32628436333833566, 0.30556542918028207, 0.2784496317993343, 0.32655778323546214, 0.2631801900931725, 0.3123600417741349, 0.2722058632348008, 0.2733814131528477, 0.27070538752913464, 0.2757538949436727, 0.2697011023377223, 0.3138177746433362, 0.2733176162495467, 0.28871782209119223, 0.3087895165038163, 0.3163763349936094, 0.278197256704461, 0.2671865634065352, 0.26906550995438183, 0.2730744498289349, 0.2677576687564798, 0.28023700133485563, 0.26490622380969203, 0.27000360383532784, 0.31026610438997093, 0.29197002011805095, 0.31322965300501293, 0.2824595854074706, 0.30556542918028207, 0.2504027827265195, 0.2705101915427627, 0.27195825677798025, 0.3290183773070441, 0.3122140822660534, 0.2563366502483721, 0.32573172216096413, 0.3131702255861782, 0.3131702255861782, 0.2808591204416624, 0.3136429899642813, 0.28882774034954284, 0.272949615989452, 0.31170450086522034, 0.27187755967318583, 0.3233675374273507, 0.2780780530151393, 0.27096754672095724, 0.2643160697016852, 0.28147941985501945, 0.26072535848597533, 0.27836868761364547, 0.3113172242643901, 0.2887431592927675, 0.32655778323546214, 0.31322774774696127, 0.28240969395278936, 0.2898829830848285, 0.28518339704536133, 0.30562754701639305, 0.2764499820479058, 0.2902598363052285, 0.3062537519476814, 0.2580767448520175, 0.2439118923739763, 0.26452636605430124, 0.3290661535293872, 0.27930759700224694, 0.26707262244274144, 0.32952498670547914, 0.32637969644693027, 0.2682253153751989, 0.2606219851948429, 0.30838332340293617, 0.27620608233397725, 0.31293928637957963, 0.31215804605278435, 0.3102632753195741, 0.2625599396449668, 0.2803571224719586, 0.27274690150813063, 0.3102632753195741, 0.3135331557130284, 0.32070109205697134, 0.22821376437457935, 0.2723663274906047, 0.32891804736359875, 0.2662153161820551, 0.17688158230293244, 0.27129487361761084, 0.29278050228082797, 0.26389443143689467, 0.2733814131528477, 0.3121091570051557, 0.2710408694709569, 0.32859540416320143, 0.31595129234752184, 0.2753792304173115, 0.3113172242643901, 0.3291836898231087, 0.2864932483046703, 0.1589850511243395, 0.26901037829997904, 0.28992169693052544, 0.28599404679335455, 0.28751333137906476, 0.1589850511243395, 0.2751800172862603, 0.1306925369713726, 0.32655778323546214, 0.30935162265031246, 0.32426502490016196, 0.2849709177998162, 0.2727575953446778, 0.28087530253171955, 0.32489915895782406, 0.26008003019382936, 0.2650569881201294, 0.26106247861848986, 0.29397176343174775, 0.31525331864453837, 0.2711370261553595, 0.2730744498289349, 0.2856513571192761, 0.2692179483899363, 0.19962194274266837, 0.2712280243350414, 0.30541997088676426, 0.2767164177354497, 0.3233675374273507, 0.19973894957557853, 0.28990679666152813, 0.2864932483046703, 0.26072535848597533, 0.28231589476905794, 0.26794708070659545, 0.2604750111709577, 0.2669043365444564, 0.3067813824106441, 0.27626625375088537, 0.2793262279809792, 0.2733814131528477, 0.2759425741187724, 0.2883627862008063, 0.26389443143689467, 0.2677576687564798, 0.19522194680826815, 0.27513653004035454, 0.3258234859577797, 0.26885271431979885 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 40, 95, 40, 215, 100, 115, 130, 115, 50, 175, 195, 165, 75, 190, 30, 245, 105, 100, 185, 125, 35, 30, 170, 180, 120, 140, 155, 155, 130, 40, 190, 220, 190, 235, 185, 205, 65, 70, 70, 205, 125, 55, 80, 150, 40, 45, 35, 240, 130, 120, 45, 25, 30, 130, 100, 80, 240, 20, 210, 25, 55, 105, 180, 175, 20, 175, 235, 200, 80, 195, 70, 165, 245, 105, 145, 75, 60, 155, 150, 155, 130, 155, 220, 40, 75, 230, 180, 75, 10, 175, 95, 25, 50, 240, 90, 145, 65, 220, 30, 175, 41, 10, 39, 39, 68, 68, 10, 99, 10, 99, 83, 49, 31, 84, 53, 53, 16, 15, 66, 30, 50, 63, 33, 68, 22, 12, 42, 44, 96, 43, 12, 45, 41, 43, 77, 81, 65, 31, 47, 87, 39, 49, 59, 47, 23, 26, 11, 69, 84, 69 ], "xaxis": "x", "y": [ 4, 2, 3, 7, 2, 7, 3, 4, 4, 6, 5, 5, 3, 7, 7, 3, 3, 4, 5, 5, 6, 6, 4, 4, 4, 3, 7, 3, 5, 3, 5, 7, 4, 2, 3, 7, 2, 3, 3, 6, 3, 6, 7, 3, 7, 2, 5, 7, 4, 6, 6, 6, 2, 6, 2, 3, 7, 2, 7, 3, 6, 4, 3, 4, 5, 7, 2, 5, 7, 2, 2, 4, 5, 3, 4, 3, 3, 3, 5, 5, 7, 3, 3, 2, 4, 7, 2, 6, 7, 6, 4, 4, 4, 3, 4, 2, 3, 4, 2, 2, 11, 11, 5, 8, 5, 8, 11, 11, 2, 2, 3, 2, 6, 4, 9, 2, 10, 3, 5, 6, 3, 4, 6, 10, 5, 7, 7, 3, 5, 2, 8, 11, 11, 6, 10, 8, 10, 11, 3, 7, 4, 4, 5, 11, 4, 6, 5, 10, 2, 6 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "n_estimators" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "max_depth" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_gbr_01_, \n", " x=\"n_estimators\", \n", " y=\"max_depth\", \n", " color=\"score\", \n", " color_continuous_scale=color_scale)\n", "\n", "fig.update_layout(width=900, height=800)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "familiar-washington", "metadata": {}, "source": [ "\n", "\n", " \n", "The scatter plot shows the search data from the previous run on the left side and the new search data on the bottom, because of the different search spaces. " ] }, { "cell_type": "markdown", "id": "useful-measurement", "metadata": {}, "source": [ "\n", "\n", "## Parallel Computing \n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "better-production", "metadata": {}, "source": [ "\n", " \n", "Lets throw more computational resources at this problem:\n", " \n", " - 1 job does a hill climbing search starting at the best position from the last run\n", " - 1 job does a hill climbing search with four random initial positions\n", " - 2 jobs doing a random search\n", " \n", "All of those jobs run in parallel and merge their results into one search data file." ] }, { "cell_type": "code", "execution_count": 35, "id": "partial-recipe", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 178 4 0.263013\n", "1 162 4 0.280177\n", "2 168 2 0.328659\n", "3 248 4 0.275434\n", "4 178 4 0.263013\n", "5 172 4 0.289256\n", "6 178 4 0.263013\n", "7 168 4 0.283338\n", "8 178 4 0.263013\n", "9 176 4 0.258077\n", "10 174 4 0.269407\n", "11 168 4 0.283338\n", "12 174 4 0.269407\n", "13 164 4 0.281814\n", "14 172 4 0.289256\n", "15 178 4 0.263013\n", "16 170 4 0.280237\n", "17 168 4 0.283338\n", "18 166 4 0.268225\n", "19 178 4 0.263013\n", "20 174 4 0.269407\n", "21 168 4 0.283338\n", "22 176 4 0.258077\n", "23 170 4 0.280237\n", "24 176 4 0.258077\n", "25 166 4 0.268225\n", "26 174 4 0.269407\n", "27 182 4 0.282154\n", "28 178 4 0.263013\n", "29 164 4 0.281814\n", "30 170 4 0.280237\n", "31 176 4 0.258077\n", "32 164 4 0.281814\n", "33 166 4 0.268225\n", "34 180 4 0.264906\n", "35 176 4 0.258077\n", "36 170 4 0.280237\n", "37 172 4 0.289256\n", "38 168 4 0.283338\n", "39 168 4 0.283338\n", "40 172 4 0.289256\n", "41 174 4 0.269407\n", "42 174 4 0.269407\n", "43 174 4 0.269407\n", "44 160 4 0.261971\n", "45 174 4 0.269407\n", "46 162 4 0.280177\n", "47 178 4 0.263013\n", "48 178 4 0.263013\n", "49 174 4 0.269407 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 194 2 0.329392\n", "1 194 2 0.329392\n", "2 198 2 0.328777\n", "3 190 2 0.328667\n", "4 194 2 0.329392\n", "5 202 2 0.328845\n", "6 192 2 0.329001\n", "7 190 2 0.328667\n", "8 194 2 0.329392\n", "9 196 2 0.329525\n", "10 200 2 0.329898\n", "11 202 2 0.328845\n", "12 200 2 0.329898\n", "13 200 2 0.329898\n", "14 196 2 0.329525\n", "15 202 2 0.328845\n", "16 206 2 0.329736\n", "17 206 2 0.329736\n", "18 198 2 0.328777\n", "19 192 2 0.329001\n", "20 202 2 0.328845\n", "21 204 2 0.328983\n", "22 200 2 0.329898\n", "23 194 2 0.329392\n", "24 200 2 0.329898\n", "25 202 2 0.328845\n", "26 200 2 0.329898\n", "27 202 2 0.328845\n", "28 198 2 0.328777\n", "29 200 2 0.329898\n", "30 198 2 0.328777\n", "31 200 2 0.329898\n", "32 186 2 0.329217\n", "33 198 2 0.328777\n", "34 200 2 0.329898\n", "35 198 2 0.328777\n", "36 206 2 0.329736\n", "37 194 2 0.329392\n", "38 196 2 0.329525\n", "39 198 2 0.328777\n", "40 202 2 0.328845\n", "41 214 2 0.330057\n", "42 196 2 0.329525\n", "43 222 2 0.328906\n", "44 208 2 0.329644\n", "45 212 2 0.329418\n", "46 218 2 0.329348\n", "47 212 2 0.329418\n", "48 206 2 0.329736\n", "49 214 2 0.330057 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 288 4 0.237881\n", "1 294 2 0.328950\n", "2 198 2 0.328207\n", "3 198 2 0.328207\n", "4 246 2 0.328708\n", "5 246 2 0.328708\n", "6 150 4 0.263894\n", "7 298 2 0.329070\n", "8 298 4 0.264539\n", "9 150 2 0.325823\n", "10 192 4 0.286485\n", "11 248 4 0.275830\n", "12 262 2 0.329647\n", "13 286 2 0.328659\n", "14 192 4 0.286485\n", "15 280 4 0.238565\n", "16 288 4 0.237881\n", "17 282 4 0.285908\n", "18 230 4 0.271318\n", "19 230 4 0.271318\n", "20 262 2 0.329647\n", "21 200 3 0.316830\n", "22 208 3 0.316408\n", "23 246 3 0.308790\n", "24 218 3 0.315333\n", "25 150 4 0.263894\n", "26 232 3 0.316344\n", "27 168 3 0.310756\n", "28 286 4 0.278173\n", "29 162 2 0.328547\n", "30 230 4 0.271318\n", "31 250 2 0.329677\n", "32 182 4 0.283171\n", "33 194 3 0.309264\n", "34 238 2 0.329388\n", "35 260 2 0.329516\n", "36 250 4 0.275661\n", "37 266 3 0.313437\n", "38 274 4 0.267048\n", "39 264 3 0.311236\n", "40 216 3 0.308580\n", "41 196 2 0.329525\n", "42 276 3 0.314968\n", "43 256 3 0.314426\n", "44 198 4 0.278145\n", "45 252 2 0.329357\n", "46 250 4 0.275661\n", "47 260 3 0.311066\n", "48 280 4 0.238565\n", "49 176 4 0.258077 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth score\n", "0 284 4 0.269907\n", "1 224 3 0.312947\n", "2 198 2 0.329464\n", "3 198 2 0.329464\n", "4 246 2 0.328885\n", "5 246 2 0.328885\n", "6 150 4 0.263894\n", "7 298 2 0.329071\n", "8 150 2 0.325823\n", "9 298 4 0.271417\n", "10 254 2 0.329152\n", "11 284 4 0.269907\n", "12 200 4 0.276576\n", "13 262 4 0.275647\n", "14 298 3 0.308374\n", "15 192 2 0.329062\n", "16 236 3 0.311532\n", "17 240 4 0.289443\n", "18 236 2 0.329066\n", "19 224 2 0.329289\n", "20 222 3 0.309378\n", "21 164 2 0.328603\n", "22 252 2 0.329503\n", "23 284 2 0.329192\n", "24 294 3 0.316338\n", "25 228 4 0.279851\n", "26 246 3 0.308790\n", "27 280 4 0.272449\n", "28 268 4 0.276267\n", "29 150 4 0.263894\n", "30 206 2 0.329596\n", "31 290 3 0.313317\n", "32 194 4 0.276474\n", "33 244 4 0.272497\n", "34 226 2 0.329349\n", "35 226 4 0.253493\n", "36 174 4 0.283864\n", "37 232 4 0.281243\n", "38 150 4 0.263894\n", "39 200 3 0.316185\n", "40 216 4 0.282455\n", "41 280 2 0.328181\n", "42 276 3 0.309901\n", "43 284 3 0.310843\n", "44 252 4 0.278502\n", "45 278 3 0.309817\n", "46 188 2 0.329047\n", "47 258 4 0.283216\n", "48 258 3 0.308893\n", "49 282 4 0.294080 \n", "\n" ] } ], "source": [ "best_para_gbr_01 = hyper_gbr_01.best_para(gbr_model_0)\n", "initialize = {\"warm_start\": [best_para_gbr_01]}\n", "\n", "\n", "search_space_gbr_02 = {\n", " \"n_estimators\": list(range(150, 300, 2)),\n", " \"max_depth\": list(range(2, 5)),\n", "}\n", "\n", "optimizer = HillClimbingOptimizer(rand_rest_p=0)\n", "\n", "hyper_gbr_02 = Hyperactive(verbosity=False)\n", "hyper_gbr_02.add_search(gbr_model_0, \n", " search_space_gbr_02, \n", " n_iter=50,\n", " n_jobs=1,\n", " optimizer=optimizer,\n", " memory_warm_start=search_data_gbr_01_, \n", " initialize=initialize)\n", "hyper_gbr_02.add_search(gbr_model_0, \n", " search_space_gbr_02, \n", " n_iter=50,\n", " n_jobs=1,\n", " optimizer=optimizer,\n", " memory_warm_start=search_data_gbr_01_, \n", " initialize={\"random\": 4})\n", "hyper_gbr_02.add_search(gbr_model_0, \n", " search_space_gbr_02, \n", " n_iter=50,\n", " n_jobs=2,\n", " memory_warm_start=search_data_gbr_01_)\n", "hyper_gbr_02.run()\n", "\n", "search_data_gbr_02 = hyper_gbr_02.search_data(gbr_model_0)" ] }, { "cell_type": "code", "execution_count": 36, "id": "dressed-catalog", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "n_estimators=%{x}
max_depth=%{y}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.3293917486617307, 0.3293917486617307, 0.32877688010316336, 0.3286665411657868, 0.3293917486617307, 0.3288453834969599, 0.32900061222687554, 0.3286665411657868, 0.3293917486617307, 0.32952498670547914, 0.3298977268223462, 0.3288453834969599, 0.3298977268223462, 0.3298977268223462, 0.32952498670547914, 0.3288453834969599, 0.32973625336832313, 0.32973625336832313, 0.32877688010316336, 0.32900061222687554, 0.3288453834969599, 0.32898347012548423, 0.3298977268223462, 0.3293917486617307, 0.3298977268223462, 0.3288453834969599, 0.3298977268223462, 0.3288453834969599, 0.32877688010316336, 0.3298977268223462, 0.32877688010316336, 0.3298977268223462, 0.329216944305858, 0.32877688010316336, 0.3298977268223462, 0.32877688010316336, 0.32973625336832313, 0.3293917486617307, 0.32952498670547914, 0.32877688010316336, 0.3288453834969599, 0.33005692837951284, 0.32952498670547914, 0.32890638579889825, 0.32964379790861315, 0.32941824216232485, 0.32934798732556425, 0.32941824216232485, 0.32973625336832313, 0.33005692837951284, 0.2630133816266802, 0.2801771461127559, 0.32865910776012647, 0.2754344347235066, 0.2630133816266802, 0.2892557180029174, 0.2630133816266802, 0.2833379865132187, 0.2630133816266802, 0.2580767448520175, 0.26940743492103125, 0.2833379865132187, 0.26940743492103125, 0.2818136299569188, 0.2892557180029174, 0.2630133816266802, 0.28023700133485563, 0.2833379865132187, 0.2682253153751989, 0.2630133816266802, 0.26940743492103125, 0.2833379865132187, 0.2580767448520175, 0.28023700133485563, 0.2580767448520175, 0.2682253153751989, 0.26940743492103125, 0.2821536393224085, 0.2630133816266802, 0.2818136299569188, 0.28023700133485563, 0.2580767448520175, 0.2818136299569188, 0.2682253153751989, 0.26490622380969203, 0.2580767448520175, 0.28023700133485563, 0.2892557180029174, 0.2833379865132187, 0.2833379865132187, 0.2892557180029174, 0.26940743492103125, 0.26940743492103125, 0.26940743492103125, 0.26197093468204546, 0.26940743492103125, 0.2801771461127559, 0.2630133816266802, 0.2630133816266802, 0.26940743492103125, 0.23788071420612358, 0.3289501547987626, 0.328207044280269, 0.328207044280269, 0.32870798458431205, 0.32870798458431205, 0.26389443143689467, 0.3290703207910613, 0.26453924302336584, 0.3258234859577797, 0.28648527276301333, 0.2758300630908391, 0.3296472044125486, 0.32865942022239886, 0.28648527276301333, 0.2385647029619995, 0.23788071420612358, 0.28590782493821465, 0.2713182542271232, 0.2713182542271232, 0.3296472044125486, 0.3168296550824806, 0.316407830293174, 0.3087895165038163, 0.3153330938056592, 0.26389443143689467, 0.3163438632862293, 0.310755678043422, 0.2781734158207554, 0.32854679131863423, 0.2713182542271232, 0.32967705101771755, 0.28317136377261376, 0.3092636599561764, 0.3293877370356907, 0.32951642651664576, 0.27566065767591363, 0.3134365720510646, 0.26704775439635997, 0.3112363596192861, 0.30857961493007047, 0.32952498670547914, 0.31496827461115207, 0.3144259805575336, 0.2781452558482044, 0.3293565266352624, 0.27566065767591363, 0.3110661442493511, 0.2385647029619995, 0.2580767448520175, 0.269906833958413, 0.31294686730396987, 0.3294637303822108, 0.3294637303822108, 0.3288849227096001, 0.3288849227096001, 0.26389443143689467, 0.3290713010305122, 0.3258234859577797, 0.27141695200946037, 0.32915153051856494, 0.269906833958413, 0.2765759254242133, 0.27564734328579815, 0.308374390840915, 0.32906231102337724, 0.31153212024436927, 0.2894425765794341, 0.3290661535293872, 0.3292888473461112, 0.3093778162635238, 0.32860303649812306, 0.3295025662974611, 0.3291920832437507, 0.31633827378765367, 0.27985130514724854, 0.3087895165038163, 0.2724491298841304, 0.2762665290703617, 0.26389443143689467, 0.32959612914078734, 0.31331657496186927, 0.27647386405277746, 0.272496836004126, 0.32934923144390665, 0.2534934637403325, 0.28386375540633024, 0.28124276522857866, 0.26389443143689467, 0.3161851024818067, 0.2824546783968906, 0.32818056991806566, 0.30990067886167233, 0.3108431884346843, 0.2785023661006526, 0.309816969962426, 0.32904681175667794, 0.28321577792523844, 0.3088926985977952, 0.29407994440972773, 0.2793262279809792, 0.32628436333833566, 0.30556542918028207, 0.2784496317993343, 0.32655778323546214, 0.2631801900931725, 0.3123600417741349, 0.2722058632348008, 0.2733814131528477, 0.27070538752913464, 0.2757538949436727, 0.2697011023377223, 0.3138177746433362, 0.2733176162495467, 0.28871782209119223, 0.3087895165038163, 0.3163763349936094, 0.278197256704461, 0.2671865634065352, 0.26906550995438183, 0.2730744498289349, 0.2677576687564798, 0.28023700133485563, 0.26490622380969203, 0.27000360383532784, 0.31026610438997093, 0.29197002011805095, 0.31322965300501293, 0.2824595854074706, 0.30556542918028207, 0.2504027827265195, 0.2705101915427627, 0.27195825677798025, 0.3290183773070441, 0.3122140822660534, 0.2563366502483721, 0.32573172216096413, 0.3131702255861782, 0.3131702255861782, 0.2808591204416624, 0.3136429899642813, 0.28882774034954284, 0.272949615989452, 0.31170450086522034, 0.27187755967318583, 0.3233675374273507, 0.2780780530151393, 0.27096754672095724, 0.2643160697016852, 0.28147941985501945, 0.26072535848597533, 0.27836868761364547, 0.3113172242643901, 0.2887431592927675, 0.32655778323546214, 0.31322774774696127, 0.28240969395278936, 0.2898829830848285, 0.28518339704536133, 0.30562754701639305, 0.2764499820479058, 0.2902598363052285, 0.3062537519476814, 0.2580767448520175, 0.2439118923739763, 0.26452636605430124, 0.3290661535293872, 0.27930759700224694, 0.26707262244274144, 0.32952498670547914, 0.32637969644693027, 0.2682253153751989, 0.2606219851948429, 0.30838332340293617, 0.27620608233397725, 0.31293928637957963, 0.31215804605278435, 0.3102632753195741, 0.2625599396449668, 0.2803571224719586, 0.27274690150813063, 0.3102632753195741, 0.3135331557130284, 0.32070109205697134, 0.22821376437457935, 0.2723663274906047, 0.32891804736359875, 0.2662153161820551, 0.17688158230293244, 0.27129487361761084, 0.29278050228082797, 0.26389443143689467, 0.2733814131528477, 0.3121091570051557, 0.2710408694709569, 0.32859540416320143, 0.31595129234752184, 0.2753792304173115, 0.3113172242643901, 0.3291836898231087, 0.2864932483046703, 0.1589850511243395, 0.26901037829997904, 0.28992169693052544, 0.28599404679335455, 0.28751333137906476, 0.1589850511243395, 0.2751800172862603, 0.1306925369713726, 0.32655778323546214, 0.30935162265031246, 0.32426502490016196, 0.2849709177998162, 0.2727575953446778, 0.28087530253171955, 0.32489915895782406, 0.26008003019382936, 0.2650569881201294, 0.26106247861848986, 0.29397176343174775, 0.31525331864453837, 0.2711370261553595, 0.2730744498289349, 0.2856513571192761, 0.2692179483899363, 0.19962194274266837, 0.2712280243350414, 0.30541997088676426, 0.2767164177354497, 0.3233675374273507, 0.19973894957557853, 0.28990679666152813, 0.2864932483046703, 0.26072535848597533, 0.28231589476905794, 0.26794708070659545, 0.2604750111709577, 0.2669043365444564, 0.3067813824106441, 0.27626625375088537, 0.2793262279809792, 0.2733814131528477, 0.2759425741187724, 0.2883627862008063, 0.26389443143689467, 0.2677576687564798, 0.19522194680826815, 0.27513653004035454, 0.3258234859577797, 0.26885271431979885 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 194, 194, 198, 190, 194, 202, 192, 190, 194, 196, 200, 202, 200, 200, 196, 202, 206, 206, 198, 192, 202, 204, 200, 194, 200, 202, 200, 202, 198, 200, 198, 200, 186, 198, 200, 198, 206, 194, 196, 198, 202, 214, 196, 222, 208, 212, 218, 212, 206, 214, 178, 162, 168, 248, 178, 172, 178, 168, 178, 176, 174, 168, 174, 164, 172, 178, 170, 168, 166, 178, 174, 168, 176, 170, 176, 166, 174, 182, 178, 164, 170, 176, 164, 166, 180, 176, 170, 172, 168, 168, 172, 174, 174, 174, 160, 174, 162, 178, 178, 174, 288, 294, 198, 198, 246, 246, 150, 298, 298, 150, 192, 248, 262, 286, 192, 280, 288, 282, 230, 230, 262, 200, 208, 246, 218, 150, 232, 168, 286, 162, 230, 250, 182, 194, 238, 260, 250, 266, 274, 264, 216, 196, 276, 256, 198, 252, 250, 260, 280, 176, 284, 224, 198, 198, 246, 246, 150, 298, 150, 298, 254, 284, 200, 262, 298, 192, 236, 240, 236, 224, 222, 164, 252, 284, 294, 228, 246, 280, 268, 150, 206, 290, 194, 244, 226, 226, 174, 232, 150, 200, 216, 280, 276, 284, 252, 278, 188, 258, 258, 282, 40, 95, 40, 215, 100, 115, 130, 115, 50, 175, 195, 165, 75, 190, 30, 245, 105, 100, 185, 125, 35, 30, 170, 180, 120, 140, 155, 155, 130, 40, 190, 220, 190, 235, 185, 205, 65, 70, 70, 205, 125, 55, 80, 150, 40, 45, 35, 240, 130, 120, 45, 25, 30, 130, 100, 80, 240, 20, 210, 25, 55, 105, 180, 175, 20, 175, 235, 200, 80, 195, 70, 165, 245, 105, 145, 75, 60, 155, 150, 155, 130, 155, 220, 40, 75, 230, 180, 75, 10, 175, 95, 25, 50, 240, 90, 145, 65, 220, 30, 175, 41, 10, 39, 39, 68, 68, 10, 99, 10, 99, 83, 49, 31, 84, 53, 53, 16, 15, 66, 30, 50, 63, 33, 68, 22, 12, 42, 44, 96, 43, 12, 45, 41, 43, 77, 81, 65, 31, 47, 87, 39, 49, 59, 47, 23, 26, 11, 69, 84, 69 ], "xaxis": "x", "y": [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 4, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 4, 2, 4, 2, 4, 4, 2, 2, 4, 4, 4, 4, 4, 4, 2, 3, 3, 3, 3, 4, 3, 3, 4, 2, 4, 2, 4, 3, 2, 2, 4, 3, 4, 3, 3, 2, 3, 3, 4, 2, 4, 3, 4, 4, 4, 3, 2, 2, 2, 2, 4, 2, 2, 4, 2, 4, 4, 4, 3, 2, 3, 4, 2, 2, 3, 2, 2, 2, 3, 4, 3, 4, 4, 4, 2, 3, 4, 4, 2, 4, 4, 4, 4, 3, 4, 2, 3, 3, 4, 3, 2, 4, 3, 4, 4, 2, 3, 7, 2, 7, 3, 4, 4, 6, 5, 5, 3, 7, 7, 3, 3, 4, 5, 5, 6, 6, 4, 4, 4, 3, 7, 3, 5, 3, 5, 7, 4, 2, 3, 7, 2, 3, 3, 6, 3, 6, 7, 3, 7, 2, 5, 7, 4, 6, 6, 6, 2, 6, 2, 3, 7, 2, 7, 3, 6, 4, 3, 4, 5, 7, 2, 5, 7, 2, 2, 4, 5, 3, 4, 3, 3, 3, 5, 5, 7, 3, 3, 2, 4, 7, 2, 6, 7, 6, 4, 4, 4, 3, 4, 2, 3, 4, 2, 2, 11, 11, 5, 8, 5, 8, 11, 11, 2, 2, 3, 2, 6, 4, 9, 2, 10, 3, 5, 6, 3, 4, 6, 10, 5, 7, 7, 3, 5, 2, 8, 11, 11, 6, 10, 8, 10, 11, 3, 7, 4, 4, 5, 11, 4, 6, 5, 10, 2, 6 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "n_estimators" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "max_depth" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "search_data_gbr_02_ = search_data_gbr_02.append(search_data_gbr_01_, ignore_index=True)\n", "\n", "fig = px.scatter(search_data_gbr_02_, \n", " x=\"n_estimators\", \n", " y=\"max_depth\", \n", " color=\"score\", \n", " color_continuous_scale=color_scale)\n", "\n", "fig.update_layout(width=900, height=800)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "dried-slope", "metadata": {}, "source": [ "\n", "\n", "All the search data we collected shows a clear pattern. We should keep the ```max_depth``` at 2. In the following plot the search data is filtered to show only ```max_depth``` == 2, while showing the ```n_estimators``` dependend on the score." ] }, { "cell_type": "code", "execution_count": 37, "id": "planned-liechtenstein", "metadata": {}, "outputs": [], "source": [ "search_data_gbr_02_f = search_data_gbr_02_[search_data_gbr_02_[\"score\"] > 0.68]\n", "search_data_gbr_02_f_max_depth2 = search_data_gbr_02_f[search_data_gbr_02_[\"max_depth\"] == 2]" ] }, { "cell_type": "code", "execution_count": 38, "id": "copyrighted-verse", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "n_estimators=%{x}
score=%{y}", "legendgroup": "", "marker": { "color": "#636efa", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [], "xaxis": "x", "y": [], "yaxis": "y" } ], "layout": { "height": 800, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "n_estimators" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "score" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_gbr_02_f_max_depth2, \n", " x=\"n_estimators\", \n", " y=\"score\")\n", "\n", "fig.update_layout(width=900, height=800)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "subtle-extent", "metadata": {}, "source": [ "\n", "\n", "The filtering and visualization of the search data in the last few plots was an example how you can explore the model and search space yourself. Hyperactive makes it very easy to collect and reuse search data. Let's take a look how to collect more data:" ] }, { "cell_type": "markdown", "id": "transsexual-refund", "metadata": {}, "source": [ "\n", " \n", "## Collecting more Data \n", " \n", "Until now you have seen, that the objective function always returns only one variable: The score, which is always a real number. But Hyperactive has the capability to accept more variables. Those additional variables won't affect the score or the decision making of the optimization algorithm, but they will be collected in each iteration and accessed in the search data.\n", " \n", "This feature can be very useful, because you can add any variable you want to the search data, which might help you understand the model better. To collect additional data in the objective function you just put it into a dictionary and return it alongside with the score. The key will be the column name in the search data and the value will be collected." ] }, { "cell_type": "code", "execution_count": 39, "id": "correct-observer", "metadata": {}, "outputs": [], "source": [ "def gbr_model_1(opt):\n", " gbr = GradientBoostingRegressor(\n", " n_estimators=opt[\"n_estimators\"],\n", " max_depth=opt[\"max_depth\"],\n", " )\n", " c_time = time.time()\n", " scores = cross_val_score(gbr, X_boston, y_boston, cv=5)\n", " cv_time = time.time() - c_time\n", " \n", " # add the dictionary to collect more data\n", " return scores.mean(), {\"cv_time\": cv_time}\n", "\n", "\n", "search_space_gbr_1 = {\n", " \"n_estimators\": list(range(10, 250, 5)),\n", " \"max_depth\": list(range(2, 8)),\n", "}" ] }, { "cell_type": "code", "execution_count": 40, "id": "exterior-heath", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 45 6 0.118185 0.266934\n", "1 25 6 0.066240 0.274931\n", "2 30 2 0.064459 0.311317\n", "3 130 6 0.313416 0.279695\n", "4 80 3 0.194099 0.311537\n", "5 240 7 0.481668 0.286414\n", "6 20 2 0.043417 0.289883\n", "7 210 7 0.446462 0.275240\n", "8 25 3 0.061512 0.300796\n", "9 55 6 0.135543 0.291698\n", "10 105 4 0.263462 0.270263\n", "11 180 3 0.429693 0.313951\n", "12 175 4 0.449600 0.274124\n", "13 20 5 0.053172 0.268555\n", "14 175 7 0.404586 0.288742 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 170 4 0.429751 0.273267\n", "1 115 7 0.278648 0.296102\n", "2 85 4 0.211211 0.279554\n", "3 135 4 0.342682 0.267721\n", "4 220 6 0.454925 0.268529\n", "5 135 5 0.323710 0.270547\n", "6 35 3 0.084315 0.306963\n", "7 60 5 0.149962 0.260005\n", "8 25 2 0.053171 0.303257\n", "9 140 2 0.278299 0.328258\n", "10 210 2 0.428252 0.330240\n", "11 80 2 0.163116 0.326211\n", "12 25 4 0.064566 0.293585\n", "13 115 6 0.277513 0.270716\n", "14 45 3 0.110079 0.313095 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 40 4 0.104604 0.270946\n", "1 95 2 0.191430 0.326538\n", "2 40 3 0.096477 0.306560\n", "3 215 7 0.447988 0.268221\n", "4 115 7 0.277140 0.278463\n", "5 130 3 0.307548 0.310494\n", "6 115 4 0.285706 0.271133\n", "7 50 4 0.124728 0.275723\n", "8 175 6 0.403750 0.272474\n", "9 195 5 0.425691 0.282180\n", "10 165 5 0.390918 0.282236\n", "11 75 3 0.183286 0.316066\n", "12 190 7 0.423599 0.274997\n", "13 30 7 0.077578 0.269481\n", "14 245 3 0.560118 0.312848 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 50 3 0.149075 0.310441\n", "1 195 3 0.459050 0.316121\n", "2 230 2 0.474948 0.328721\n", "3 70 3 0.166681 0.308237\n", "4 40 7 0.101941 0.262045\n", "5 100 2 0.203159 0.326424\n", "6 125 6 0.300996 0.285009\n", "7 145 5 0.351124 0.281005\n", "8 195 7 0.435357 0.282233\n", "9 30 7 0.080386 0.262075\n", "10 175 7 0.413364 0.268366\n", "11 175 3 0.413322 0.310711\n", "12 110 7 0.265733 0.275947\n", "13 140 6 0.335133 0.279896\n", "14 95 2 0.193122 0.326247 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 175 6 0.412572 0.280366\n", "1 100 4 0.250836 0.290253\n", "2 240 2 0.479053 0.328462\n", "3 195 5 0.432045 0.258361\n", "4 140 6 0.336825 0.284946\n", "5 10 3 0.027291 0.175655\n", "6 80 3 0.189595 0.314869\n", "7 115 2 0.236703 0.326597\n", "8 135 7 0.328682 0.277129\n", "9 65 7 0.164024 0.281355\n", "10 220 2 0.447159 0.329678\n", "11 70 3 0.166764 0.307975\n", "12 80 2 0.161863 0.325785\n", "13 225 3 0.515723 0.313758\n", "14 200 6 0.429268 0.280143 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 220 3 0.528340 0.311830\n", "1 125 7 0.306709 0.280501\n", "2 195 2 0.394514 0.329501\n", "3 150 4 0.377696 0.264062\n", "4 235 5 0.466406 0.286418\n", "5 40 5 0.101020 0.265324\n", "6 150 5 0.367667 0.268419\n", "7 235 3 0.550423 0.308684\n", "8 10 2 0.024037 0.130693\n", "9 100 3 0.237929 0.311202\n", "10 65 2 0.132017 0.325816\n", "11 195 4 0.463978 0.271931\n", "12 170 7 0.396408 0.288230\n", "13 180 3 0.419057 0.311722\n", "14 35 7 0.088092 0.282675 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 180 7 0.414299 0.262485\n", "1 190 2 0.376027 0.329066\n", "2 80 4 0.199736 0.262852\n", "3 205 4 0.481331 0.288552\n", "4 180 3 0.418108 0.310108\n", "5 240 5 0.469577 0.278504\n", "6 85 6 0.205165 0.270781\n", "7 190 7 0.421955 0.264020\n", "8 190 4 0.460415 0.281676\n", "9 210 7 0.442884 0.276648\n", "10 105 4 0.261704 0.285338\n", "11 10 7 0.029574 0.179460\n", "12 135 7 0.324964 0.269716\n", "13 90 5 0.217990 0.278937\n", "14 130 4 0.323176 0.286018 \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score\n", "0 190 5 0.448730 0.280003\n", "1 165 6 0.392935 0.287462\n", "2 125 5 0.304988 0.283130\n", "3 210 3 0.486952 0.315794\n", "4 115 4 0.286834 0.263519\n", "5 110 7 0.266535 0.282376\n", "6 55 4 0.139961 0.283277\n", "7 240 4 0.567436 0.278804\n", "8 85 6 0.214452 0.285471\n", "9 225 3 0.516657 0.308658\n", "10 160 4 0.397832 0.260815\n", "11 100 4 0.248304 0.281050\n", "12 225 2 0.444328 0.329629\n", "13 90 3 0.213261 0.310404\n", "14 85 3 0.200542 0.311639 \n", "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
n_estimatorsmax_depthcv_timescore
04040.1046040.270946
19520.1914300.326538
24030.0964770.306560
321570.4479880.268221
411570.2771400.278463
\n", "
" ], "text/plain": [ " n_estimators max_depth cv_time score\n", "0 40 4 0.104604 0.270946\n", "1 95 2 0.191430 0.326538\n", "2 40 3 0.096477 0.306560\n", "3 215 7 0.447988 0.268221\n", "4 115 7 0.277140 0.278463" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hyper_gbr_1 = Hyperactive(verbosity=False)\n", "hyper_gbr_1.add_search(gbr_model_1, search_space_gbr_1, n_iter=15, n_jobs=8, initialize={\"random\": 10})\n", "hyper_gbr_1.run()\n", "\n", "search_data_gbr_1 = hyper_gbr_1.search_data(gbr_model_1)\n", "search_data_gbr_1.head()" ] }, { "cell_type": "code", "execution_count": 41, "id": "celtic-mainstream", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "n_estimators=%{x}
max_depth=%{y}
cv_time=%{marker.size}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.2709461969064999, 0.3265375606679304, 0.30655981239034585, 0.2682214989219318, 0.27846260324265487, 0.31049360080462163, 0.2711326281177766, 0.27572323829056744, 0.27247420602272027, 0.28217999254723375, 0.2822357688579039, 0.3160658279981439, 0.2749969055514188, 0.26948075145554257, 0.31284776596212405, 0.26693419165110144, 0.27493145429054544, 0.3113172242643901, 0.2796948048861001, 0.3115367869020273, 0.28641426002744025, 0.2898829830848285, 0.2752400974595218, 0.3007959793714179, 0.2916984274696913, 0.2702632040040044, 0.31395074690466285, 0.2741244314690546, 0.26855471289869143, 0.2887416559848542, 0.26248546420384156, 0.3290662172960134, 0.2628521246533159, 0.2885522753575302, 0.3101077587762274, 0.2785041278980528, 0.2707810837389413, 0.26402011519742496, 0.2816761486072048, 0.27664775358563115, 0.2853378910529163, 0.1794603025236365, 0.26971647236583146, 0.2789370526570093, 0.28601817976472493, 0.28036588196851314, 0.2902527658213353, 0.3284623406334463, 0.25836105907245743, 0.28494560076595093, 0.1756549051838017, 0.3148693131954209, 0.3265974673594415, 0.27712932161000214, 0.2813554243332576, 0.3296782845461366, 0.30797500060391264, 0.3257847120442463, 0.3137575314728778, 0.28014313202265695, 0.3104409654057541, 0.3161206054765948, 0.32872055748965656, 0.3082368192381738, 0.2620448120031044, 0.32642440352891616, 0.2850088371840315, 0.2810048434023715, 0.28223255072950415, 0.26207477387389005, 0.2683664028112568, 0.3107109650073628, 0.27594747672476105, 0.27989556679129335, 0.32624690344056984, 0.31183026475484293, 0.2805011440686025, 0.3295014209349489, 0.2640618758815732, 0.2864183585199447, 0.2653237440439869, 0.26841866960462574, 0.30868371915039544, 0.1306925369713726, 0.3112018035387637, 0.32581601477019667, 0.27193140723631704, 0.2882298969615945, 0.31172219937720974, 0.2826754081701782, 0.27326653870179457, 0.2961018244711651, 0.27955420340164516, 0.26772138371350784, 0.2685292840855814, 0.27054718035943537, 0.3069625106006734, 0.26000534154713695, 0.30325694709963996, 0.3282581796830334, 0.3302399370714225, 0.32621135464712847, 0.2935853774990184, 0.2707158762066942, 0.3130948557354734, 0.2800025372757483, 0.2874619733888228, 0.28312955845116816, 0.3157943234652518, 0.2635190591951778, 0.28237596984928004, 0.2832774352120103, 0.278803762672842, 0.28547067992840774, 0.30865804264881896, 0.26081463632167, 0.2810500627115669, 0.32962889470964524, 0.3104044382590432, 0.31163937285104 ], "coloraxis": "coloraxis", "size": [ 0.10460376739501953, 0.19142961502075195, 0.09647727012634277, 0.44798779487609863, 0.27713990211486816, 0.30754828453063965, 0.2857058048248291, 0.12472844123840332, 0.403749942779541, 0.4256908893585205, 0.390918493270874, 0.18328595161437988, 0.4235990047454834, 0.07757782936096191, 0.5601177215576172, 0.11818528175354004, 0.06623959541320801, 0.06445932388305664, 0.31341552734375, 0.19409894943237305, 0.48166751861572266, 0.04341745376586914, 0.44646191596984863, 0.06151247024536133, 0.1355433464050293, 0.2634620666503906, 0.42969298362731934, 0.4495997428894043, 0.05317211151123047, 0.4045860767364502, 0.41429853439331055, 0.37602686882019043, 0.1997356414794922, 0.48133063316345215, 0.4181082248687744, 0.4695765972137451, 0.20516514778137207, 0.4219553470611572, 0.4604146480560303, 0.4428839683532715, 0.2617042064666748, 0.02957439422607422, 0.3249642848968506, 0.21798968315124512, 0.32317590713500977, 0.41257190704345703, 0.2508363723754883, 0.4790525436401367, 0.43204545974731445, 0.3368253707885742, 0.027291297912597656, 0.18959498405456543, 0.23670339584350586, 0.32868242263793945, 0.16402435302734375, 0.4471588134765625, 0.1667635440826416, 0.16186261177062988, 0.5157229900360107, 0.42926836013793945, 0.14907479286193848, 0.45905017852783203, 0.4749481678009033, 0.16668057441711426, 0.10194087028503418, 0.20315909385681152, 0.3009963035583496, 0.351123571395874, 0.43535661697387695, 0.08038568496704102, 0.4133639335632324, 0.41332197189331055, 0.2657332420349121, 0.3351328372955322, 0.19312191009521484, 0.5283398628234863, 0.30670857429504395, 0.3945140838623047, 0.37769556045532227, 0.46640586853027344, 0.10101985931396484, 0.36766743659973145, 0.5504229068756104, 0.024036884307861328, 0.2379288673400879, 0.13201665878295898, 0.463977575302124, 0.3964083194732666, 0.41905736923217773, 0.08809208869934082, 0.4297513961791992, 0.27864789962768555, 0.2112107276916504, 0.342681884765625, 0.4549250602722168, 0.32370972633361816, 0.08431458473205566, 0.1499621868133545, 0.05317115783691406, 0.27829861640930176, 0.4282519817352295, 0.1631155014038086, 0.06456613540649414, 0.2775130271911621, 0.11007881164550781, 0.4487297534942627, 0.39293527603149414, 0.30498766899108887, 0.4869523048400879, 0.2868344783782959, 0.26653456687927246, 0.13996076583862305, 0.5674357414245605, 0.21445202827453613, 0.5166566371917725, 0.39783215522766113, 0.24830389022827148, 0.44432830810546875, 0.21326112747192383, 0.20054244995117188 ], "sizemode": "area", "sizeref": 0.0014185893535614013, "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 40, 95, 40, 215, 115, 130, 115, 50, 175, 195, 165, 75, 190, 30, 245, 45, 25, 30, 130, 80, 240, 20, 210, 25, 55, 105, 180, 175, 20, 175, 180, 190, 80, 205, 180, 240, 85, 190, 190, 210, 105, 10, 135, 90, 130, 175, 100, 240, 195, 140, 10, 80, 115, 135, 65, 220, 70, 80, 225, 200, 50, 195, 230, 70, 40, 100, 125, 145, 195, 30, 175, 175, 110, 140, 95, 220, 125, 195, 150, 235, 40, 150, 235, 10, 100, 65, 195, 170, 180, 35, 170, 115, 85, 135, 220, 135, 35, 60, 25, 140, 210, 80, 25, 115, 45, 190, 165, 125, 210, 115, 110, 55, 240, 85, 225, 160, 100, 225, 90, 85 ], "xaxis": "x", "y": [ 4, 2, 3, 7, 7, 3, 4, 4, 6, 5, 5, 3, 7, 7, 3, 6, 6, 2, 6, 3, 7, 2, 7, 3, 6, 4, 3, 4, 5, 7, 7, 2, 4, 4, 3, 5, 6, 7, 4, 7, 4, 7, 7, 5, 4, 6, 4, 2, 5, 6, 3, 3, 2, 7, 7, 2, 3, 2, 3, 6, 3, 3, 2, 3, 7, 2, 6, 5, 7, 7, 7, 3, 7, 6, 2, 3, 7, 2, 4, 5, 5, 5, 3, 2, 3, 2, 4, 7, 3, 7, 4, 7, 4, 4, 6, 5, 3, 5, 2, 2, 2, 2, 4, 6, 3, 5, 6, 5, 3, 4, 7, 4, 4, 6, 3, 4, 4, 2, 3, 3 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 800, "legend": { "itemsizing": "constant", "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 900, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "n_estimators" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "max_depth" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_gbr_1, \n", " x=\"n_estimators\", \n", " y=\"max_depth\", \n", " color=\"score\", \n", " size='cv_time', \n", " color_continuous_scale=color_scale)\n", "\n", "fig.update_layout(width=900, height=800)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "decent-transcription", "metadata": {}, "source": [ "\n", " \n", "The scatter plot above shows the samples od the search data, but adds a visualization of the cross-validation-time with the size of the scatter-points." ] }, { "cell_type": "markdown", "id": "differential-tattoo", "metadata": {}, "source": [ "\n", " \n", "## Managing multiple objectives \n", " \n", "In the last chapter you were able to collect additional data during the optimization run. This data did not affect the score. But you can still try to create one score that represents information from multiple scores. In the following example we want to optimize a model to get a high score and at the same time a low training time." ] }, { "cell_type": "code", "execution_count": 42, "id": "approximate-county", "metadata": {}, "outputs": [], "source": [ "def gbr_model_2(opt):\n", " gbr = GradientBoostingRegressor(\n", " n_estimators=opt[\"n_estimators\"],\n", " max_depth=opt[\"max_depth\"],\n", " )\n", " c_time = time.time()\n", " scores = cross_val_score(gbr, X_boston, y_boston, cv=5)\n", " cv_time = time.time() - c_time\n", " \n", " score_cv_avg = scores.mean()\n", " score_cv_std = scores.std()\n", " \n", " # the score is calculated from the cv-score and the cv-training time\n", " score = score_cv_avg / (cv_time**0.1)\n", " \n", " # independed from the score we want some additional data\n", " return score, {\"cv_time\": cv_time, \n", " \"score_cv_avg\": score_cv_avg,\n", " \"score_cv_std\": score_cv_std,\n", " \"scores\": scores,\n", " }\n", "\n", "\n", "search_space_gbr_2 = {\n", " \"n_estimators\": list(range(10, 250, 5)),\n", " \"max_depth\": list(range(2, 12)),\n", "}" ] }, { "cell_type": "markdown", "id": "conceptual-boxing", "metadata": {}, "source": [ "\n", " \n", "The objective function above enables us to return a score that is composed of multiple variables. At the same time, we also want to collect data about the variables the score is composed from. This helps us understand the score later during the data visualization." ] }, { "cell_type": "code", "execution_count": 43, "id": "aging-postage", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 45 10 0.136457 0.323857 0.265371 0.330783 \n", "1 25 10 0.064831 0.357932 0.272257 0.336918 \n", "2 30 3 0.072968 0.403566 0.310619 0.380542 \n", "3 130 10 0.312861 0.318940 0.283952 0.349164 \n", "4 80 4 0.200744 0.313594 0.267074 0.332184 \n", "5 240 2 0.484914 0.354671 0.329908 0.406825 \n", "6 10 2 0.023994 0.189776 0.130693 0.160954 \n", "7 70 4 0.178738 0.328887 0.276865 0.341739 \n", "8 175 6 0.407856 0.292649 0.267545 0.332618 \n", "9 95 10 0.234092 0.309486 0.267658 0.332724 \n", "10 55 10 0.135570 0.332052 0.271908 0.336840 \n", "11 95 2 0.192210 0.385327 0.326744 0.402988 \n", "12 130 10 0.312861 0.318940 0.283952 0.349164 \n", "13 220 3 0.505374 0.336861 0.314639 0.385742 \n", "14 200 9 0.434827 0.301456 0.277367 0.342242 \n", "\n", " scores \n", "0 [0.0, 0.7606979573331243, 0.0, 0.5661560583542... \n", "1 [0.0, 0.756928785905888, 0.0, 0.60435498531228... \n", "2 [0.0, 0.7618800535985606, 0.0, 0.7912142241900... \n", "3 [0.0, 0.7591975497210028, 0.0, 0.6605601524950... \n", "4 [0.0, 0.7592408879023933, 0.0, 0.5761300406532... \n", "5 [0.0, 0.7498086663521787, 0.0, 0.8997297401252... \n", "6 [0.0, 0.3000208717930798, 0.0, 0.3534418130637... \n", "7 [0.0, 0.7593214379685758, 0.0, 0.6250028639511... \n", "8 [0.0, 0.7591973311141902, 0.0, 0.5785281255638... \n", "9 [0.0, 0.7592063189382285, 0.0, 0.5790821541598... \n", "10 [0.0, 0.7597763989791204, 0.0, 0.5997639441451... \n", "11 [0.0, 0.7417446308421538, 0.0, 0.8919733614070... \n", "12 [0.0, 0.7591975497210028, 0.0, 0.6605601524950... \n", "13 [0.0, 0.7591973311141902, 0.0, 0.8139963950064... \n", "14 [0.0, 0.7591973311141902, 0.0, 0.6276383127284... \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 170 7 0.413711 0.311161 0.284875 0.350164 \n", "1 115 5 0.277382 0.332062 0.292097 0.358213 \n", "2 105 8 0.256741 0.320797 0.280014 0.344981 \n", "3 125 11 0.300331 0.323629 0.286951 0.352436 \n", "4 135 9 0.323920 0.319626 0.285552 0.350901 \n", "5 35 4 0.088649 0.354309 0.278067 0.343188 \n", "6 60 9 0.147740 0.348017 0.287442 0.352994 \n", "7 25 2 0.054084 0.405979 0.303257 0.372563 \n", "8 140 3 0.329806 0.345044 0.308817 0.378309 \n", "9 210 2 0.414936 0.359749 0.329456 0.406356 \n", "10 80 3 0.188538 0.369897 0.313055 0.383698 \n", "11 25 6 0.064801 0.370514 0.281814 0.346738 \n", "12 115 11 0.275769 0.296294 0.260482 0.326254 \n", "13 45 4 0.113226 0.354023 0.284725 0.350083 \n", "14 210 8 0.440172 0.283969 0.261597 0.327226 \n", "\n", " scores \n", "0 [0.0, 0.7591973314288802, 0.0, 0.6651778198072... \n", "1 [0.0, 0.7591984186362237, 0.0, 0.7012863352115... \n", "2 [0.0, 0.7592004620942316, 0.0, 0.6408702197439... \n", "3 [0.0, 0.7591977059680856, 0.0, 0.6755583577510... \n", "4 [0.0, 0.7591974574568147, 0.0, 0.6685633062254... \n", "5 [0.0, 0.7621839309593008, 0.0, 0.6281502604805... \n", "6 [0.0, 0.7595464902301229, 0.0, 0.6776650535732... \n", "7 [0.0, 0.7118804031055922, 0.0, 0.8044043323926... \n", "8 [0.0, 0.7591974029750898, 0.0, 0.7848865763114... \n", "9 [0.0, 0.7476038105049081, 0.0, 0.8996785062807... \n", "10 [0.0, 0.7592408879023933, 0.0, 0.8060341746144... \n", "11 [0.0, 0.756928785905888, 0.0, 0.65214339070086... \n", "12 [0.0, 0.7591984186362237, 0.0, 0.5432121318684... \n", "13 [0.0, 0.7606979573331243, 0.0, 0.6629268123550... \n", "14 [0.0, 0.7591973311141902, 0.0, 0.5487874146446... \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 180 11 0.415904 0.292008 0.267482 0.332559 \n", "1 15 5 0.041179 0.314475 0.228589 0.285959 \n", "2 95 11 0.229800 0.316754 0.273436 0.338287 \n", "3 95 10 0.232698 0.339406 0.293358 0.359660 \n", "4 60 9 0.148902 0.323756 0.267614 0.332720 \n", "5 85 11 0.207217 0.334708 0.285962 0.351350 \n", "6 190 11 0.419905 0.315123 0.288932 0.354635 \n", "7 90 6 0.221628 0.308532 0.265376 0.330614 \n", "8 90 2 0.199622 0.383714 0.326609 0.402842 \n", "9 230 8 0.476602 0.282412 0.262240 0.327792 \n", "10 245 6 0.475372 0.292884 0.271894 0.336772 \n", "11 155 8 0.368208 0.314119 0.284252 0.349489 \n", "12 90 3 0.211561 0.368240 0.315264 0.386551 \n", "13 20 3 0.049844 0.386807 0.286586 0.351208 \n", "14 60 11 0.147057 0.354909 0.292998 0.359256 \n", "\n", " scores \n", "0 [0.0, 0.7591973311141902, 0.0, 0.5782101756953... \n", "1 [0.0, 0.6635783675193356, 0.0, 0.4793645774808... \n", "2 [0.0, 0.7592063189382285, 0.0, 0.6079760893920... \n", "3 [0.0, 0.7592063189382285, 0.0, 0.7075857737728... \n", "4 [0.0, 0.7595464902301229, 0.0, 0.5785220373474... \n", "5 [0.0, 0.7592230854704327, 0.0, 0.6705870066605... \n", "6 [0.0, 0.7591973311141902, 0.0, 0.6854609574533... \n", "7 [0.0, 0.7592125490494327, 0.0, 0.5676686658178... \n", "8 [0.0, 0.7411685862901474, 0.0, 0.8918770859385... \n", "9 [0.0, 0.7591973311141902, 0.0, 0.5520006952587... \n", "10 [0.0, 0.7591973311141902, 0.0, 0.6002726153686... \n", "11 [0.0, 0.7591973405896815, 0.0, 0.6620637527677... \n", "12 [0.0, 0.7592125490494327, 0.0, 0.8171062246438... \n", "13 [0.0, 0.7358195863474217, 0.0, 0.6971118540768... \n", "14 [0.0, 0.7595464902301229, 0.0, 0.7054459913405... \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 220 5 0.455640 0.304106 0.281117 0.346140 \n", "1 125 11 0.302580 0.316119 0.280501 0.345492 \n", "2 10 9 0.030269 0.196829 0.138736 0.179144 \n", "3 110 9 0.265424 0.302174 0.264637 0.329940 \n", "4 40 9 0.100850 0.344517 0.273891 0.338938 \n", "5 150 8 0.358505 0.323525 0.291983 0.358083 \n", "6 235 5 0.472267 0.292650 0.271498 0.336387 \n", "7 10 3 0.028313 0.244344 0.171081 0.209606 \n", "8 100 5 0.252887 0.325643 0.283814 0.349017 \n", "9 65 2 0.134255 0.398686 0.326155 0.402016 \n", "10 195 7 0.432069 0.310959 0.285929 0.351313 \n", "11 170 10 0.398265 0.306941 0.279944 0.344908 \n", "12 60 3 0.143180 0.376253 0.309791 0.379532 \n", "13 230 3 0.526460 0.331901 0.311275 0.381422 \n", "14 225 2 0.451513 0.357549 0.330219 0.407264 \n", "\n", " scores \n", "0 [0.0, 0.7591973311141902, 0.0, 0.6463874468818... \n", "1 [0.0, 0.7591977059680856, 0.0, 0.6433080143749... \n", "2 [0.0, 0.43657746289686405, 0.0, 0.257100661469... \n", "3 [0.0, 0.7591991773750808, 0.0, 0.5639876512819... \n", "4 [0.0, 0.761449900875621, 0.0, 0.60800697241692... \n", "5 [0.0, 0.7591973518071372, 0.0, 0.7007177747400... \n", "6 [0.0, 0.7591973311141902, 0.0, 0.5982950718976... \n", "7 [0.0, 0.43657746289686405, 0.0, 0.418826802922... \n", "8 [0.0, 0.759202637170055, 0.0, 0.65986925166211... \n", "9 [0.0, 0.7437668317273509, 0.0, 0.8870068552202... \n", "10 [0.0, 0.7591973311141902, 0.0, 0.6704501104566... \n", "11 [0.0, 0.7591973314288802, 0.0, 0.6405251282294... \n", "12 [0.0, 0.7595464902301229, 0.0, 0.7894066997405... \n", "13 [0.0, 0.7591973311141902, 0.0, 0.7971785803182... \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "14 [0.0, 0.7497741442819785, 0.0, 0.9013226961820... \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 40 6 0.117105 0.339244 0.273760 0.338809 \n", "1 95 2 0.192001 0.385126 0.326538 0.402701 \n", "2 40 5 0.100304 0.346020 0.274937 0.339974 \n", "3 215 7 0.456671 0.290090 0.268221 0.333253 \n", "4 225 8 0.460565 0.300552 0.278131 0.343026 \n", "5 65 7 0.159255 0.353656 0.294300 0.360753 \n", "6 100 4 0.248682 0.298836 0.260015 0.325850 \n", "7 100 10 0.254327 0.326997 0.285156 0.350470 \n", "8 180 11 0.416745 0.300223 0.275062 0.339906 \n", "9 150 9 0.362403 0.289249 0.261331 0.326992 \n", "10 165 5 0.390241 0.257560 0.234429 0.307286 \n", "11 55 11 0.136349 0.343151 0.281158 0.346221 \n", "12 245 3 0.558418 0.334188 0.315273 0.386564 \n", "13 210 4 0.489017 0.307314 0.286098 0.351498 \n", "14 105 5 0.255243 0.330143 0.288003 0.353600 \n", "\n", " scores \n", "0 [0.0, 0.761449900875621, 0.0, 0.60735008583143... \n", "1 [0.0, 0.74171543851855, 0.0, 0.890972364821101... \n", "2 [0.0, 0.761449900875621, 0.0, 0.61323489059761... \n", "3 [0.0, 0.7591973311141902, 0.0, 0.5819101634954... \n", "4 [0.0, 0.7591973311141902, 0.0, 0.6314568449037... \n", "5 [0.0, 0.7594060236112754, 0.0, 0.7120962035144... \n", "6 [0.0, 0.759202637170055, 0.0, 0.54087068362118... \n", "7 [0.0, 0.759202637170055, 0.0, 0.66657920962673... \n", "8 [0.0, 0.7591973311141902, 0.0, 0.6161127493205... \n", "9 [0.0, 0.7591973518071372, 0.0, 0.5474566324285... \n", "10 [0.0, 0.7591973319805099, 0.0, 0.4129485460500... \n", "11 [0.0, 0.7597763989791204, 0.0, 0.6460118139438... \n", "12 [0.0, 0.7591973311141902, 0.0, 0.8171690398746... \n", "13 [0.0, 0.7591973311141902, 0.0, 0.6712930359609... \n", "14 [0.0, 0.7592004620942316, 0.0, 0.6808160058318... \n", "\n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 175 10 0.435712 0.300426 0.276476 0.341333 \n", "1 100 7 0.242967 0.335800 0.291498 0.357530 \n", "2 240 3 0.553861 0.332909 0.313809 0.384670 \n", "3 195 9 0.428541 0.292784 0.268997 0.333985 \n", "4 140 10 0.339210 0.320703 0.287839 0.353418 \n", "5 10 4 0.029351 0.241386 0.169619 0.207891 \n", "6 80 5 0.195049 0.313984 0.266638 0.331779 \n", "7 115 2 0.231999 0.377976 0.326597 0.402624 \n", "8 135 4 0.336672 0.302481 0.271280 0.336175 \n", "9 215 2 0.426309 0.358897 0.329566 0.406412 \n", "10 70 5 0.170563 0.334608 0.280366 0.345358 \n", "11 80 2 0.164777 0.390169 0.325794 0.401664 \n", "12 225 5 0.458695 0.289719 0.267997 0.333041 \n", "13 200 10 0.437606 0.297075 0.273511 0.338360 \n", "14 85 8 0.204948 0.331356 0.282787 0.347915 \n", "\n", " scores \n", "0 [0.0, 0.7591973311141902, 0.0, 0.6231820929314... \n", "1 [0.0, 0.759202637170055, 0.0, 0.69828739724825... \n", "2 [0.0, 0.7591973311141902, 0.0, 0.8098490909401... \n", "3 [0.0, 0.7591973311141902, 0.0, 0.5857860644759... \n", "4 [0.0, 0.7591974029750898, 0.0, 0.6799969200704... \n", "5 [0.0, 0.43657746289686405, 0.0, 0.411517709726... \n", "6 [0.0, 0.7592408879023933, 0.0, 0.5739502865654... \n", "7 [0.0, 0.743909637104236, 0.0, 0.88907769969297... \n", "8 [0.0, 0.7591974574568147, 0.0, 0.5972046427982... \n", "9 [0.0, 0.7489188791452149, 0.0, 0.8989108289244... \n", "10 [0.0, 0.7593214379685758, 0.0, 0.6425066168565... \n", "11 [0.0, 0.741652812212936, 0.0, 0.88731529803996... \n", "12 [0.0, 0.7591973311141902, 0.0, 0.5807874747754... \n", "13 [0.0, 0.7591973311141902, 0.0, 0.6083562144719... \n", "14 [0.0, 0.7592230854704327, 0.0, 0.6547128579433... \n", "\n", "\n", " hyper_optimizer search_data \n", "\n", " hyper_optimizer search_data \n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 190 8 0.425644 0.304549 0.279615 0.344565 \n", "1 165 10 0.395935 0.313447 0.285710 0.351074 \n", "2 125 8 0.301297 0.314241 0.278716 0.343631 \n", "3 210 5 0.442195 0.302296 0.278608 0.343518 \n", "4 115 6 0.277537 0.301964 0.265636 0.330850 \n", "5 110 4 0.274258 0.325823 0.286284 0.351703 \n", "6 105 6 0.252859 0.289422 0.252243 0.319459 \n", "7 85 11 0.205357 0.341245 0.291284 0.357288 \n", "8 225 4 0.513295 0.301982 0.282499 0.347606 \n", "9 160 7 0.378928 0.291803 0.264817 0.330102 \n", "10 100 7 0.240047 0.317631 0.275392 0.340238 \n", "11 225 2 0.443083 0.356974 0.329067 0.405642 \n", "12 90 4 0.223649 0.326463 0.281055 0.346075 \n", "13 85 5 0.204921 0.314538 0.268431 0.333452 \n", "14 35 4 0.088334 0.331140 0.259791 0.326065 \n", "\n", " scores \n", "0 [0.0, 0.7591973311141902, 0.0, 0.6388801546981... \n", "1 [0.0, 0.7591973319805099, 0.0, 0.6693539423234... \n", "2 [0.0, 0.7591977059680856, 0.0, 0.6343847529870... \n", "3 [0.0, 0.7591973311141902, 0.0, 0.6338418345754... \n", "4 [0.0, 0.7591984186362237, 0.0, 0.5689810757456... \n", "5 [0.0, 0.7591991773750808, 0.0, 0.6722232597348... \n", "6 [0.0, 0.7592004620942316, 0.0, 0.5020131961419... \n", "7 [0.0, 0.7592230854704327, 0.0, 0.6971975231665... \n", "8 [0.0, 0.7591973311141902, 0.0, 0.6532994289108... \n", "9 [0.0, 0.7591973339658764, 0.0, 0.5648870415883... \n", "10 [0.0, 0.759202637170055, 0.0, 0.61775959245503... \n", "11 [0.0, 0.749907029558438, 0.0, 0.89542940208227... \n", "12 [0.0, 0.7592125490494327, 0.0, 0.6460601110964... \n", "13 [0.0, 0.7592230854704327, 0.0, 0.5829309386913... \n", "14 [0.0, 0.7621839309593008, 0.0, 0.5367704553759... \n", "\n", " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 50 4 0.134709 0.325120 0.266062 0.331350 \n", "1 195 4 0.465079 0.288511 0.267248 0.332341 \n", "2 230 2 0.460997 0.355187 0.328721 0.405174 \n", "3 70 5 0.171615 0.311101 0.260830 0.326572 \n", "4 40 6 0.103232 0.328847 0.262045 0.327909 \n", "5 25 7 0.068596 0.341303 0.261078 0.326479 \n", "6 195 8 0.429352 0.293719 0.269907 0.334852 \n", "7 140 11 0.341563 0.308038 0.276663 0.341524 \n", "8 230 3 0.534215 0.333307 0.313052 0.383695 \n", "9 235 10 0.466711 0.302454 0.280262 0.345240 \n", "10 215 10 0.461447 0.298496 0.276281 0.341135 \n", "11 50 7 0.124703 0.330690 0.268540 0.333653 \n", "12 215 8 0.447357 0.311976 0.287864 0.353445 \n", "13 170 6 0.404659 0.303095 0.276878 0.341742 \n", "14 20 4 0.054090 0.362930 0.271103 0.334056 \n", "\n", " scores \n", "0 [0.0, 0.7601428859741948, 0.0, 0.5701674479267... \n", "1 [0.0, 0.7591973311141902, 0.0, 0.5770435925038... \n", "2 [0.0, 0.7496839489591742, 0.0, 0.8939188384891... \n", "3 [0.0, 0.7593214379685758, 0.0, 0.5448284353802... \n", "4 [0.0, 0.761449900875621, 0.0, 0.54877415913990... \n", "5 [0.0, 0.756928785905888, 0.0, 0.54846060837347... \n", "6 [0.0, 0.7591973311141902, 0.0, 0.5903374255550... \n", "7 [0.0, 0.7591974029750898, 0.0, 0.6241196406447... \n", "8 [0.0, 0.7591973311141902, 0.0, 0.8060615508503... \n", "9 [0.0, 0.7591973311141902, 0.0, 0.6421115372831... \n", "10 [0.0, 0.7591973311141902, 0.0, 0.6222059494211... \n", "11 [0.0, 0.7601428859741948, 0.0, 0.5825584181845... \n", "12 [0.0, 0.7591973311141902, 0.0, 0.6801219906510... \n", "13 [0.0, 0.7591973314288802, 0.0, 0.6251904453111... \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "14 [0.0, 0.7358195863474217, 0.0, 0.6196937978507... \n", "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
n_estimatorsmax_depthcv_timescorescore_cv_avgscore_cv_stdscores
04060.1171050.3392440.2737600.338809[0.0, 0.761449900875621, 0.0, 0.60735008583143...
19520.1920010.3851260.3265380.402701[0.0, 0.74171543851855, 0.0, 0.890972364821101...
24050.1003040.3460200.2749370.339974[0.0, 0.761449900875621, 0.0, 0.61323489059761...
321570.4566710.2900900.2682210.333253[0.0, 0.7591973311141902, 0.0, 0.5819101634954...
422580.4605650.3005520.2781310.343026[0.0, 0.7591973311141902, 0.0, 0.6314568449037...
\n", "
" ], "text/plain": [ " n_estimators max_depth cv_time score score_cv_avg score_cv_std \\\n", "0 40 6 0.117105 0.339244 0.273760 0.338809 \n", "1 95 2 0.192001 0.385126 0.326538 0.402701 \n", "2 40 5 0.100304 0.346020 0.274937 0.339974 \n", "3 215 7 0.456671 0.290090 0.268221 0.333253 \n", "4 225 8 0.460565 0.300552 0.278131 0.343026 \n", "\n", " scores \n", "0 [0.0, 0.761449900875621, 0.0, 0.60735008583143... \n", "1 [0.0, 0.74171543851855, 0.0, 0.890972364821101... \n", "2 [0.0, 0.761449900875621, 0.0, 0.61323489059761... \n", "3 [0.0, 0.7591973311141902, 0.0, 0.5819101634954... \n", "4 [0.0, 0.7591973311141902, 0.0, 0.6314568449037... " ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hyper_gbr_2 = Hyperactive(verbosity=False)\n", "hyper_gbr_2.add_search(gbr_model_2, search_space_gbr_2, n_iter=15, n_jobs=8, initialize={\"random\": 10})\n", "hyper_gbr_2.run()\n", "\n", "search_data_gbr_2 = hyper_gbr_2.search_data(gbr_model_2)\n", "search_data_gbr_2.head()" ] }, { "cell_type": "code", "execution_count": 44, "id": "selective-gender", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "n_estimators=%{x}
max_depth=%{y}
cv_time=%{marker.size}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.33924420792142596, 0.3851258922780077, 0.34601995098991506, 0.29009029289844657, 0.3005522879079682, 0.3536564671188295, 0.2988363931730489, 0.3269970387178019, 0.30022274227294743, 0.28924866768987223, 0.25755996326179464, 0.3431505579577497, 0.3341882929630856, 0.3073141353501457, 0.3301430051422143, 0.3238572646865594, 0.3579320089665409, 0.40356573355183306, 0.3189400255904115, 0.313593911871331, 0.3546712378739036, 0.18977569002089342, 0.3288866435592627, 0.29264851757011345, 0.3094860802506066, 0.3320518256975791, 0.3853270990704408, 0.3189400255904115, 0.3368611254915471, 0.30145561614745797, 0.29200782370560063, 0.3144752896917005, 0.3167535913796781, 0.3394057948248868, 0.3237564174136239, 0.33470786699429, 0.3151229655122107, 0.30853158836745775, 0.38371378450285987, 0.2824117169425673, 0.2928844145537176, 0.31411924810896447, 0.36823955820026566, 0.3868067957777741, 0.3549088876233846, 0.30042586369218316, 0.3358001785354896, 0.3329091533897728, 0.292784217225847, 0.3207026839002266, 0.24138632080764544, 0.3139843791755568, 0.3779759926151066, 0.30248074960131527, 0.3588970687899826, 0.3346078638186708, 0.39016913264848024, 0.28971934702808283, 0.2970749594162761, 0.331356368823197, 0.3251196812879458, 0.28851079151870934, 0.3551870044553276, 0.3111014706214051, 0.3288472170891202, 0.3413030092918394, 0.2937194892979895, 0.3080382255554449, 0.3333071014436668, 0.30245379363559505, 0.2984958489446542, 0.33069039007407264, 0.31197639355633244, 0.3030950389041828, 0.3629296684724518, 0.3041059340443084, 0.31611902394881375, 0.19682880781942907, 0.3021739805974345, 0.3445171317037266, 0.3235251967414948, 0.29265005338465494, 0.24434429722205842, 0.325643025692899, 0.39868577747060474, 0.31095934009536513, 0.30694085553627704, 0.37625309889634434, 0.3319005286050237, 0.35754895336747883, 0.3111606679130351, 0.33206199641700784, 0.32079715413853216, 0.3236294730376744, 0.3196257003493371, 0.3543087800636328, 0.34801725406094836, 0.4059794205671839, 0.3450444495168897, 0.3597492635752339, 0.36989704767222, 0.37051411696737185, 0.29629436715588414, 0.35402263524163274, 0.2839686613711984, 0.3045485907280507, 0.31344654347297474, 0.3142412249739643, 0.3022956508557088, 0.30196364620187766, 0.32582309438173357, 0.28942150628839547, 0.3412446049731384, 0.3019817780592667, 0.29180312480028087, 0.3176306468274738, 0.3569736607742887, 0.32646297811921476, 0.3145383866372641, 0.33113955870824013 ], "coloraxis": "coloraxis", "size": [ 0.11710500717163086, 0.1920013427734375, 0.10030436515808105, 0.45667147636413574, 0.46056509017944336, 0.1592550277709961, 0.24868154525756836, 0.25432705879211426, 0.4167449474334717, 0.36240339279174805, 0.3902411460876465, 0.13634896278381348, 0.558417797088623, 0.4890170097351074, 0.25524282455444336, 0.13645672798156738, 0.06483078002929688, 0.0729684829711914, 0.31286144256591797, 0.2007441520690918, 0.48491382598876953, 0.023993968963623047, 0.17873764038085938, 0.407855749130249, 0.23409175872802734, 0.13556957244873047, 0.19220972061157227, 0.31286144256591797, 0.5053741931915283, 0.4348266124725342, 0.41590356826782227, 0.04117941856384277, 0.22979950904846191, 0.2326982021331787, 0.14890170097351074, 0.20721650123596191, 0.4199047088623047, 0.22162771224975586, 0.19962239265441895, 0.47660183906555176, 0.4753718376159668, 0.3682079315185547, 0.2115612030029297, 0.04984402656555176, 0.14705681800842285, 0.4357118606567383, 0.2429666519165039, 0.5538606643676758, 0.4285411834716797, 0.33920979499816895, 0.029350757598876953, 0.19504904747009277, 0.23199892044067383, 0.3366720676422119, 0.42630863189697266, 0.1705634593963623, 0.16477704048156738, 0.4586946964263916, 0.437605619430542, 0.20494818687438965, 0.13470911979675293, 0.46507930755615234, 0.4609968662261963, 0.1716146469116211, 0.10323190689086914, 0.06859612464904785, 0.4293515682220459, 0.34156322479248047, 0.534214973449707, 0.46671128273010254, 0.46144700050354004, 0.12470340728759766, 0.4473569393157959, 0.4046590328216553, 0.05408954620361328, 0.4556396007537842, 0.3025798797607422, 0.03026866912841797, 0.2654244899749756, 0.10084986686706543, 0.3585047721862793, 0.47226738929748535, 0.028313398361206055, 0.2528867721557617, 0.13425469398498535, 0.4320693016052246, 0.39826512336730957, 0.14317989349365234, 0.5264601707458496, 0.45151305198669434, 0.4137108325958252, 0.27738213539123535, 0.2567405700683594, 0.30033087730407715, 0.32391977310180664, 0.08864879608154297, 0.14774036407470703, 0.054083824157714844, 0.3298060894012451, 0.4149360656738281, 0.18853759765625, 0.06480145454406738, 0.2757687568664551, 0.11322569847106934, 0.4401721954345703, 0.4256439208984375, 0.3959345817565918, 0.3012971878051758, 0.44219541549682617, 0.27753710746765137, 0.2742581367492676, 0.25285911560058594, 0.20535731315612793, 0.5132951736450195, 0.37892842292785645, 0.24004697799682617, 0.4430832862854004, 0.2236487865447998, 0.20492124557495117, 0.0883340835571289 ], "sizemode": "area", "sizeref": 0.0013960444927215577, "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 40, 95, 40, 215, 225, 65, 100, 100, 180, 150, 165, 55, 245, 210, 105, 45, 25, 30, 130, 80, 240, 10, 70, 175, 95, 55, 95, 130, 220, 200, 180, 15, 95, 95, 60, 85, 190, 90, 90, 230, 245, 155, 90, 20, 60, 175, 100, 240, 195, 140, 10, 80, 115, 135, 215, 70, 80, 225, 200, 85, 50, 195, 230, 70, 40, 25, 195, 140, 230, 235, 215, 50, 215, 170, 20, 220, 125, 10, 110, 40, 150, 235, 10, 100, 65, 195, 170, 60, 230, 225, 170, 115, 105, 125, 135, 35, 60, 25, 140, 210, 80, 25, 115, 45, 210, 190, 165, 125, 210, 115, 110, 105, 85, 225, 160, 100, 225, 90, 85, 35 ], "xaxis": "x", "y": [ 6, 2, 5, 7, 8, 7, 4, 10, 11, 9, 5, 11, 3, 4, 5, 10, 10, 3, 10, 4, 2, 2, 4, 6, 10, 10, 2, 10, 3, 9, 11, 5, 11, 10, 9, 11, 11, 6, 2, 8, 6, 8, 3, 3, 11, 10, 7, 3, 9, 10, 4, 5, 2, 4, 2, 5, 2, 5, 10, 8, 4, 4, 2, 5, 6, 7, 8, 11, 3, 10, 10, 7, 8, 6, 4, 5, 11, 9, 9, 9, 8, 5, 3, 5, 2, 7, 10, 3, 3, 2, 7, 5, 8, 11, 9, 4, 9, 2, 3, 2, 3, 6, 11, 4, 8, 8, 10, 8, 5, 6, 4, 6, 11, 4, 7, 7, 2, 4, 5, 4 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 700, "legend": { "itemsizing": "constant", "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 800, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "n_estimators" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "max_depth" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_gbr_2, \n", " x=\"n_estimators\", \n", " y=\"max_depth\", \n", " color=\"score\", \n", " size='cv_time', \n", " color_continuous_scale=color_scale)\n", "\n", "fig.update_layout(width=800, height=700)\n", "fig.show()" ] }, { "cell_type": "code", "execution_count": 45, "id": "potential-dayton", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "cv_time=%{x}
score_cv_avg=%{y}
score_cv_std=%{marker.size}
score=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.33924420792142596, 0.3851258922780077, 0.34601995098991506, 0.29009029289844657, 0.3005522879079682, 0.3536564671188295, 0.2988363931730489, 0.3269970387178019, 0.30022274227294743, 0.28924866768987223, 0.25755996326179464, 0.3431505579577497, 0.3341882929630856, 0.3073141353501457, 0.3301430051422143, 0.3238572646865594, 0.3579320089665409, 0.40356573355183306, 0.3189400255904115, 0.313593911871331, 0.3546712378739036, 0.18977569002089342, 0.3288866435592627, 0.29264851757011345, 0.3094860802506066, 0.3320518256975791, 0.3853270990704408, 0.3189400255904115, 0.3368611254915471, 0.30145561614745797, 0.29200782370560063, 0.3144752896917005, 0.3167535913796781, 0.3394057948248868, 0.3237564174136239, 0.33470786699429, 0.3151229655122107, 0.30853158836745775, 0.38371378450285987, 0.2824117169425673, 0.2928844145537176, 0.31411924810896447, 0.36823955820026566, 0.3868067957777741, 0.3549088876233846, 0.30042586369218316, 0.3358001785354896, 0.3329091533897728, 0.292784217225847, 0.3207026839002266, 0.24138632080764544, 0.3139843791755568, 0.3779759926151066, 0.30248074960131527, 0.3588970687899826, 0.3346078638186708, 0.39016913264848024, 0.28971934702808283, 0.2970749594162761, 0.331356368823197, 0.3251196812879458, 0.28851079151870934, 0.3551870044553276, 0.3111014706214051, 0.3288472170891202, 0.3413030092918394, 0.2937194892979895, 0.3080382255554449, 0.3333071014436668, 0.30245379363559505, 0.2984958489446542, 0.33069039007407264, 0.31197639355633244, 0.3030950389041828, 0.3629296684724518, 0.3041059340443084, 0.31611902394881375, 0.19682880781942907, 0.3021739805974345, 0.3445171317037266, 0.3235251967414948, 0.29265005338465494, 0.24434429722205842, 0.325643025692899, 0.39868577747060474, 0.31095934009536513, 0.30694085553627704, 0.37625309889634434, 0.3319005286050237, 0.35754895336747883, 0.3111606679130351, 0.33206199641700784, 0.32079715413853216, 0.3236294730376744, 0.3196257003493371, 0.3543087800636328, 0.34801725406094836, 0.4059794205671839, 0.3450444495168897, 0.3597492635752339, 0.36989704767222, 0.37051411696737185, 0.29629436715588414, 0.35402263524163274, 0.2839686613711984, 0.3045485907280507, 0.31344654347297474, 0.3142412249739643, 0.3022956508557088, 0.30196364620187766, 0.32582309438173357, 0.28942150628839547, 0.3412446049731384, 0.3019817780592667, 0.29180312480028087, 0.3176306468274738, 0.3569736607742887, 0.32646297811921476, 0.3145383866372641, 0.33113955870824013 ], "coloraxis": "coloraxis", "size": [ 0.33880891298225785, 0.4027007956229687, 0.3399739188256062, 0.33325250592624, 0.34302610019982477, 0.3607533093041874, 0.32585015353692354, 0.3504698880250669, 0.3399058528407925, 0.3269924942493912, 0.307285501999161, 0.3462205482873712, 0.3865642611995317, 0.35149804825678777, 0.3536004213406131, 0.33078263222631354, 0.3369176261487008, 0.3805419286996104, 0.34916421579717694, 0.3321835174236155, 0.4068245937781696, 0.16095399506270475, 0.34173875154238076, 0.3326179490497899, 0.33272426077360095, 0.33684031119381297, 0.40298751325648247, 0.34916421579717694, 0.3857416282107277, 0.3422419666169009, 0.33255851424808736, 0.2859590645102471, 0.3382873371731827, 0.35965985411595075, 0.33272004496837737, 0.35135032292607443, 0.35463496658349947, 0.33061417305130547, 0.4028418985678026, 0.3277917943647058, 0.33677176047121693, 0.34948888268129097, 0.38655144592313495, 0.3512084560928626, 0.359255987648806, 0.3413332489273117, 0.3575299968830507, 0.3846699371552749, 0.3339849799085177, 0.3534177268027596, 0.20789113601246947, 0.3317788166159992, 0.4026241860196415, 0.33617520231969467, 0.40641151965425576, 0.34535761749355726, 0.4016640165996775, 0.3330413855435821, 0.3383599654948783, 0.34791540622746053, 0.33134966693080287, 0.33234076910012883, 0.4051742559815926, 0.3265717966164192, 0.32790901536691147, 0.3264787685994155, 0.334852221641193, 0.34152388317328586, 0.38369486257086943, 0.3452403524946762, 0.34113511293069754, 0.33365316100912745, 0.3534454661575796, 0.34174194289833865, 0.33405615090852325, 0.34613974213619014, 0.3454915046675505, 0.17914394390686458, 0.32993962630131757, 0.3389383438000036, 0.3580825557000356, 0.336386817750334, 0.20960557261652865, 0.34901663812498895, 0.4020163423487334, 0.3513134147009638, 0.34490822986056063, 0.3795319779602451, 0.3814218349679718, 0.40726397207918247, 0.3501637347643085, 0.3582126797224804, 0.3449812740918548, 0.3524358832465137, 0.3509009872251476, 0.3431883627330608, 0.3529944450914186, 0.3725630573411186, 0.37830901848625004, 0.4063557716843763, 0.38369795731584877, 0.3467377421597836, 0.3262535856218669, 0.3500833740285446, 0.32722573096127827, 0.3445647423550403, 0.35107365048581113, 0.3436308360983268, 0.34351839409260426, 0.3308502832270494, 0.35170256273253664, 0.31945910924574705, 0.3572875262217605, 0.34760647634068204, 0.33010243105375026, 0.34023831706418867, 0.4056422000822265, 0.3460748797336686, 0.33345245174689525, 0.32606452331437896 ], "sizemode": "area", "sizeref": 0.0010181599301979562, "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.11710500717163086, 0.1920013427734375, 0.10030436515808105, 0.45667147636413574, 0.46056509017944336, 0.1592550277709961, 0.24868154525756836, 0.25432705879211426, 0.4167449474334717, 0.36240339279174805, 0.3902411460876465, 0.13634896278381348, 0.558417797088623, 0.4890170097351074, 0.25524282455444336, 0.13645672798156738, 0.06483078002929688, 0.0729684829711914, 0.31286144256591797, 0.2007441520690918, 0.48491382598876953, 0.023993968963623047, 0.17873764038085938, 0.407855749130249, 0.23409175872802734, 0.13556957244873047, 0.19220972061157227, 0.31286144256591797, 0.5053741931915283, 0.4348266124725342, 0.41590356826782227, 0.04117941856384277, 0.22979950904846191, 0.2326982021331787, 0.14890170097351074, 0.20721650123596191, 0.4199047088623047, 0.22162771224975586, 0.19962239265441895, 0.47660183906555176, 0.4753718376159668, 0.3682079315185547, 0.2115612030029297, 0.04984402656555176, 0.14705681800842285, 0.4357118606567383, 0.2429666519165039, 0.5538606643676758, 0.4285411834716797, 0.33920979499816895, 0.029350757598876953, 0.19504904747009277, 0.23199892044067383, 0.3366720676422119, 0.42630863189697266, 0.1705634593963623, 0.16477704048156738, 0.4586946964263916, 0.437605619430542, 0.20494818687438965, 0.13470911979675293, 0.46507930755615234, 0.4609968662261963, 0.1716146469116211, 0.10323190689086914, 0.06859612464904785, 0.4293515682220459, 0.34156322479248047, 0.534214973449707, 0.46671128273010254, 0.46144700050354004, 0.12470340728759766, 0.4473569393157959, 0.4046590328216553, 0.05408954620361328, 0.4556396007537842, 0.3025798797607422, 0.03026866912841797, 0.2654244899749756, 0.10084986686706543, 0.3585047721862793, 0.47226738929748535, 0.028313398361206055, 0.2528867721557617, 0.13425469398498535, 0.4320693016052246, 0.39826512336730957, 0.14317989349365234, 0.5264601707458496, 0.45151305198669434, 0.4137108325958252, 0.27738213539123535, 0.2567405700683594, 0.30033087730407715, 0.32391977310180664, 0.08864879608154297, 0.14774036407470703, 0.054083824157714844, 0.3298060894012451, 0.4149360656738281, 0.18853759765625, 0.06480145454406738, 0.2757687568664551, 0.11322569847106934, 0.4401721954345703, 0.4256439208984375, 0.3959345817565918, 0.3012971878051758, 0.44219541549682617, 0.27753710746765137, 0.2742581367492676, 0.25285911560058594, 0.20535731315612793, 0.5132951736450195, 0.37892842292785645, 0.24004697799682617, 0.4430832862854004, 0.2236487865447998, 0.20492124557495117, 0.0883340835571289 ], "xaxis": "x", "y": [ 0.27375999734141077, 0.3265375606679304, 0.27493695829464726, 0.2682214989219318, 0.27813083520359383, 0.2943004454251512, 0.26001466415824853, 0.2851563693593577, 0.27506201608694175, 0.2613307968471414, 0.23442917560611942, 0.2811576425845854, 0.3152732741977754, 0.286098073415036, 0.2880032935852236, 0.265370803137467, 0.27225675424363427, 0.31061885555772917, 0.28395154044320814, 0.26707418571113173, 0.32990768129548564, 0.1306925369713726, 0.2768648603839529, 0.26754509133561005, 0.26765769461961114, 0.27190806862485084, 0.3267435984498438, 0.28395154044320814, 0.3146387452241239, 0.27736712876852254, 0.26748150136191506, 0.22858858900003615, 0.27343648166605095, 0.29335841854222194, 0.2676137055155068, 0.2859620184261979, 0.2889316577135093, 0.2653762429734476, 0.32660913444573225, 0.262239605274579, 0.27189398929656694, 0.2842522186714961, 0.31526375473866597, 0.2865862880848501, 0.2929984963141261, 0.2764758848091252, 0.291498006883662, 0.31380928441087297, 0.2689966791180187, 0.28783886460911645, 0.16961903452474703, 0.2666382348935611, 0.3265974673594415, 0.2712804200510225, 0.3295659416139386, 0.2803656109650182, 0.32579362205058077, 0.2679969611779314, 0.27351070911721914, 0.28278718868275277, 0.26606206678019484, 0.26724818472360345, 0.32872055748965656, 0.26082997466975605, 0.2620448120031044, 0.26107787885587186, 0.26990695133384823, 0.2766634087239607, 0.31305177639290555, 0.28026177367947047, 0.27628065610706076, 0.2685402608317394, 0.28786386435305766, 0.27687755534800906, 0.2711026768396313, 0.28111695559921646, 0.2805011440686025, 0.13873562487318788, 0.2646373657314148, 0.27389137465850855, 0.29198302530943404, 0.2714984806023774, 0.1710808531639382, 0.2838143777664346, 0.32615473738951045, 0.28592948831417403, 0.2799444919316753, 0.3097906379941303, 0.31127518228648293, 0.33021936809280084, 0.28487503024721955, 0.29209695076954756, 0.28001413636763905, 0.2869512127438316, 0.2855521527364613, 0.2780668382879645, 0.287442308760673, 0.30325694709963996, 0.30881679585730587, 0.3294564633571228, 0.3130550125033654, 0.28181443532135025, 0.26048211010093214, 0.2847249539376283, 0.2615969491517714, 0.2796154971624697, 0.2857102548607898, 0.2787164917910327, 0.2786078331379239, 0.2656358988763655, 0.2862844874219782, 0.25224273164722766, 0.2912841217273999, 0.2824993520050137, 0.26481687511084934, 0.2753924459250179, 0.3290672863281433, 0.28105453202916786, 0.2684308048323529, 0.2597908772670526 ], "yaxis": "y" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 700, "legend": { "itemsizing": "constant", "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 800, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "cv_time" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "score_cv_avg" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(search_data_gbr_2, \n", " x=\"cv_time\", \n", " y=\"score_cv_avg\", \n", " color=\"score\", \n", " size='score_cv_std', \n", " color_continuous_scale=color_scale)\n", "\n", "fig.update_layout(width=800, height=700)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "everyday-toner", "metadata": {}, "source": [ "\n", " \n", "## Non-numerical Search Spaces \n", " \n", "This chapter describes a very unique and helpful feature of Hyperactive: non-numerical values in the search space. You are not constrained to use numeric values in your search space, but also strings or even functions. Because of this you can do some really interesting stuff like:\n", " \n", " - hyperparameter optimization of any parameter\n", " - preprocessing-optimization\n", " - neural architecture search\n", " \n", "Lets take a look at the following example:" ] }, { "cell_type": "code", "execution_count": 46, "id": "asian-original", "metadata": {}, "outputs": [], "source": [ "def mlp_model(opt): \n", " scaler = MinMaxScaler()\n", " X_norm = scaler.fit_transform(X_iris)\n", " \n", " mlp = MLPClassifier(\n", " hidden_layer_sizes=opt[\"hidden_layer_sizes\"],\n", " activation=opt[\"activation\"],\n", " alpha=opt[\"alpha\"],\n", " learning_rate_init=opt[\"learning_rate_init\"],\n", "\n", " )\n", " scores = cross_val_score(mlp, X_norm, y_iris, cv=5)\n", "\n", " return scores.mean()\n", "\n", "\n", "search_space_mlp = {\n", " \"hidden_layer_sizes\": list(range(10, 100, 10)),\n", " \"activation\": [\"identity\", \"logistic\", \"tanh\", \"relu\"],\n", " \"solver\": [\"lbfgs\", \"sgd\", \"adam\"],\n", " \"alpha\": [1/(10**x) for x in range(1, 9)],\n", " \"learning_rate_init\": [1/(10**x) for x in range(1, 9)],\n", "\n", "}" ] }, { "cell_type": "code", "execution_count": 47, "id": "statewide-uganda", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " hidden_layer_sizes activation solver alpha learning_rate_init \\\n", "0 20 tanh sgd 1.000000e-01 1.000000e-02 \n", "1 40 tanh adam 1.000000e-07 1.000000e-03 \n", "2 50 logistic sgd 1.000000e-04 1.000000e-04 \n", "3 90 relu lbfgs 1.000000e-08 1.000000e-08 \n", "4 90 relu lbfgs 1.000000e-01 1.000000e-01 \n", "5 10 relu lbfgs 1.000000e-08 1.000000e-08 \n", "6 90 relu adam 1.000000e-01 1.000000e-01 \n", "7 90 tanh adam 1.000000e-05 1.000000e-06 \n", "8 60 relu lbfgs 1.000000e-08 1.000000e-08 \n", "9 40 relu sgd 1.000000e-02 1.000000e-04 \n", "10 70 tanh adam 1.000000e-02 1.000000e-03 \n", "11 30 identity lbfgs 1.000000e-04 1.000000e-04 \n", "12 40 tanh lbfgs 1.000000e-03 1.000000e-04 \n", "13 80 logistic lbfgs 1.000000e-02 1.000000e-01 \n", "14 20 relu adam 1.000000e-07 1.000000e-05 \n", "15 60 relu sgd 1.000000e-07 1.000000e-01 \n", "16 60 tanh adam 1.000000e-03 1.000000e-01 \n", "17 10 relu adam 1.000000e-01 1.000000e-03 \n", "18 20 identity sgd 1.000000e-01 1.000000e-01 \n", "19 30 logistic lbfgs 1.000000e-02 1.000000e-06 \n", "20 40 tanh lbfgs 1.000000e-01 1.000000e-03 \n", "21 70 logistic lbfgs 1.000000e-02 1.000000e-06 \n", "22 40 identity adam 1.000000e-06 1.000000e-08 \n", "23 70 logistic sgd 1.000000e-07 1.000000e-05 \n", "24 30 identity adam 1.000000e-07 1.000000e-01 \n", "25 50 logistic sgd 1.000000e-07 1.000000e-08 \n", "26 20 relu lbfgs 1.000000e-03 1.000000e-06 \n", "27 80 logistic lbfgs 1.000000e-05 1.000000e-08 \n", "28 10 tanh sgd 1.000000e-05 1.000000e-08 \n", "29 30 identity lbfgs 1.000000e-04 1.000000e-03 \n", "30 40 tanh sgd 1.000000e-08 1.000000e-07 \n", "31 90 tanh sgd 1.000000e-08 1.000000e-03 \n", "32 50 tanh adam 1.000000e-03 1.000000e-01 \n", "33 80 relu lbfgs 1.000000e-05 1.000000e-07 \n", "34 50 logistic adam 1.000000e-08 1.000000e-04 \n", "35 70 tanh sgd 1.000000e-03 1.000000e-03 \n", "36 60 tanh adam 1.000000e-03 1.000000e-03 \n", "37 70 relu sgd 1.000000e-06 1.000000e-07 \n", "38 20 tanh adam 1.000000e-06 1.000000e-07 \n", "39 20 identity lbfgs 1.000000e-04 1.000000e-01 \n", "\n", " score \n", "0 0.973333 \n", "1 0.900000 \n", "2 0.420000 \n", "3 0.306667 \n", "4 0.973333 \n", "5 0.393333 \n", "6 0.973333 \n", "7 0.386667 \n", "8 0.313333 \n", "9 0.466667 \n", "10 0.920000 \n", "11 0.433333 \n", "12 0.480000 \n", "13 0.973333 \n", "14 0.380000 \n", "15 0.973333 \n", "16 0.973333 \n", "17 0.740000 \n", "18 0.973333 \n", "19 0.333333 \n", "20 0.900000 \n", "21 0.266667 \n", "22 0.340000 \n", "23 0.333333 \n", "24 0.973333 \n", "25 0.333333 \n", "26 0.360000 \n", "27 0.320000 \n", "28 0.400000 \n", "29 0.866667 \n", "30 0.253333 \n", "31 0.940000 \n", "32 0.973333 \n", "33 0.333333 \n", "34 0.406667 \n", "35 0.926667 \n", "36 0.926667 \n", "37 0.266667 \n", "38 0.306667 \n", "39 0.973333 \n", "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
hidden_layer_sizesactivationsolveralphalearning_rate_initscore
020tanhsgd1.000000e-011.000000e-020.973333
140tanhadam1.000000e-071.000000e-030.900000
250logisticsgd1.000000e-041.000000e-040.420000
390relulbfgs1.000000e-081.000000e-080.306667
490relulbfgs1.000000e-011.000000e-010.973333
\n", "
" ], "text/plain": [ " hidden_layer_sizes activation solver alpha learning_rate_init \\\n", "0 20 tanh sgd 1.000000e-01 1.000000e-02 \n", "1 40 tanh adam 1.000000e-07 1.000000e-03 \n", "2 50 logistic sgd 1.000000e-04 1.000000e-04 \n", "3 90 relu lbfgs 1.000000e-08 1.000000e-08 \n", "4 90 relu lbfgs 1.000000e-01 1.000000e-01 \n", "\n", " score \n", "0 0.973333 \n", "1 0.900000 \n", "2 0.420000 \n", "3 0.306667 \n", "4 0.973333 " ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hyper_mlp_0 = Hyperactive(verbosity=False)\n", "hyper_mlp_0.add_search(mlp_model, search_space_mlp, n_iter=40)\n", "hyper_mlp_0.run()\n", "\n", "mlp_search_data = hyper_mlp_0.search_data(mlp_model)\n", "mlp_search_data.head()" ] }, { "cell_type": "code", "execution_count": 48, "id": "corporate-certification", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "dimensions": [ { "label": "hidden_layer_sizes", "values": [ 90, 90, 90, 90, 90, 80, 80, 80, 70, 70, 70, 70, 70, 60, 60, 60, 60, 50, 50, 50, 50, 40, 40, 40, 40, 40, 40, 30, 30, 30, 30, 20, 20, 20, 20, 20, 20, 10, 10, 10 ] }, { "label": "activation", "values": [ "tanh", "relu", "relu", "relu", "tanh", "relu", "logistic", "logistic", "logistic", "logistic", "tanh", "tanh", "relu", "relu", "tanh", "tanh", "relu", "tanh", "logistic", "logistic", "logistic", "tanh", "tanh", "relu", "identity", "tanh", "tanh", "identity", "logistic", "identity", "identity", "tanh", "tanh", "relu", "identity", "relu", "identity", "tanh", "relu", "relu" ] }, { "label": "solver", "values": [ "sgd", "lbfgs", "lbfgs", "adam", "adam", "lbfgs", "lbfgs", "lbfgs", "sgd", "lbfgs", "adam", "sgd", "sgd", "sgd", "adam", "adam", "lbfgs", "adam", "sgd", "adam", "sgd", "lbfgs", "adam", "sgd", "adam", "sgd", "lbfgs", "lbfgs", "lbfgs", "adam", "lbfgs", "adam", "sgd", "lbfgs", "sgd", "adam", "lbfgs", "sgd", "adam", "lbfgs" ] }, { "label": "alpha", "values": [ 1e-08, 1e-08, 0.1, 0.1, 1e-05, 1e-05, 1e-05, 0.01, 1e-07, 0.01, 0.01, 0.001, 1e-06, 1e-07, 0.001, 0.001, 1e-08, 0.001, 1e-07, 1e-08, 0.0001, 0.001, 1e-07, 0.01, 1e-06, 1e-08, 0.1, 0.0001, 0.01, 1e-07, 0.0001, 1e-06, 0.1, 0.001, 0.1, 1e-07, 0.0001, 1e-05, 0.1, 1e-08 ] }, { "label": "learning_rate_init", "values": [ 0.001, 1e-08, 0.1, 0.1, 1e-06, 1e-07, 1e-08, 0.1, 1e-05, 1e-06, 0.001, 0.001, 1e-07, 0.1, 0.001, 0.1, 1e-08, 0.1, 1e-08, 0.0001, 0.0001, 0.0001, 0.001, 0.0001, 1e-08, 1e-07, 0.001, 0.0001, 1e-06, 0.1, 0.001, 1e-07, 0.01, 1e-06, 0.1, 1e-05, 0.1, 1e-08, 0.001, 1e-08 ] } ], "domain": { "x": [ 0, 1 ], "y": [ 0, 1 ] }, "line": { "color": [ 0.9400000000000001, 0.3066666666666667, 0.9733333333333334, 0.9733333333333334, 0.3866666666666666, 0.3333333333333333, 0.31999999999999995, 0.9733333333333334, 0.3333333333333333, 0.26666666666666666, 0.9199999999999999, 0.9266666666666667, 0.26666666666666666, 0.9733333333333334, 0.9266666666666667, 0.9733333333333334, 0.3133333333333333, 0.9733333333333334, 0.3333333333333333, 0.4066666666666666, 0.42000000000000004, 0.48, 0.9, 0.4666666666666667, 0.33999999999999997, 0.2533333333333333, 0.9, 0.4333333333333334, 0.3333333333333333, 0.9733333333333334, 0.8666666666666666, 0.30666666666666664, 0.9733333333333334, 0.36, 0.9733333333333334, 0.37999999999999995, 0.9733333333333334, 0.4, 0.74, 0.3933333333333333 ], "coloraxis": "coloraxis" }, "name": "", "type": "parcats" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 700, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 950 } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "parameter_names = list(search_space_mlp.keys())\n", "\n", "mlp_search_data = mlp_search_data.sort_values('hidden_layer_sizes', ascending=False)\n", "\n", "fig = px.parallel_categories(mlp_search_data, \n", " color=\"score\", \n", " color_continuous_scale=color_scale, \n", " dimensions=parameter_names, \n", " )\n", "fig.update_layout(width=950, height=700)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "educational-yugoslavia", "metadata": {}, "source": [ "\n", " \n", "## Deep Learning Optimization \n", " \n", "The optimization of deep learning models can be very difficult because the evaluation times of the objective functions are very high, due to the long training times. There is also the challenge of finding a way to find the optimal structure/architecture of the neural network. Hyperactive can help with both of those problems.\n", " \n", "The optimization of the structure/architecture of a neural network is called **n**eural **a**rchitecture **s**earch. Because Hyperactive can handle functions in its search spaces performing **nas** is very easy.\n", " " ] }, { "cell_type": "code", "execution_count": 49, "id": "alert-series", "metadata": {}, "outputs": [], "source": [ "def deep_learning_model(params):\n", " filters_0 = params[\"filters.0\"]\n", " kernel_size_0 = params[\"kernel_size.0\"]\n", " \n", " model = Sequential()\n", " model.add(Conv2D(filters_0, (kernel_size_0, kernel_size_0), input_shape=(img_width, img_height, 1), activation=\"relu\"))\n", " \n", " model.add(MaxPooling2D(pool_size=(2, 2)))\n", " \n", " # the next two lines are layers that are put in during the optimization run\n", " model = params[\"layer.0\"](params, model)\n", " model = params[\"layer.1\"](params, model)\n", "\n", " model.add(Flatten())\n", " model.add(Dense(params[\"dense.0\"], activation=\"relu\"))\n", " model.add(Dense(num_classes, activation=\"softmax\"))\n", "\n", " model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n", " model.fit(\n", " x_train,\n", " y_train,\n", " validation_data=(x_test, y_test),\n", " epochs=5,\n", " verbose=False,\n", " )\n", " _, score = model.evaluate(x=x_test, y=y_test, verbose=False)\n", "\n", " return score" ] }, { "cell_type": "markdown", "id": "returning-cornell", "metadata": {}, "source": [ "\n", " \n", "The following functions are the layers and layer-compositions that we will use in the search space. The ```params```-argument enables the optimization of parameters inside the layer-function. There is also a ```no_layer```-function because we want to test of it might be better for the score of the neural network if its number of layers is reduced." ] }, { "cell_type": "code", "execution_count": 50, "id": "voluntary-volleyball", "metadata": {}, "outputs": [], "source": [ "def Conv2D_MaxPooling2D_layer(params, model):\n", " filters_1 = params[\"layer.0.filters\"]\n", " kernel_size_1 = params[\"layer.0.kernel_size\"]\n", " model.add(Conv2D(filters_1, (kernel_size_1, kernel_size_1), activation='relu'))\n", " model.add(MaxPooling2D(pool_size=(2, 2)))\n", " return model\n", "\n", "def Conv2D_layer(params, model):\n", " filters_1 = params[\"layer.0.filters\"]\n", " kernel_size_1 = params[\"layer.0.kernel_size\"]\n", " model.add(Conv2D(filters_1, (kernel_size_1, kernel_size_1), activation='relu'))\n", " return model\n", "\n", "def Dropout_layer(params, model):\n", " model.add(Dropout(params[\"layer.1.rate\"]))\n", " return model\n", "\n", "def no_layer(params, model):\n", " return model" ] }, { "cell_type": "markdown", "id": "opposed-playback", "metadata": {}, "source": [ "\n", " \n", "In the search space you can see that the layer-functions are put inside lists. During the optimization run Hyperactive will select those layer-functions similar to any other variable inside the search space." ] }, { "cell_type": "code", "execution_count": 51, "id": "hindu-cradle", "metadata": {}, "outputs": [], "source": [ "# you can put the layers into lists like any other variable\n", "search_space_dl = {\n", " \"filters.0\": list(range(7, 15)),\n", " \"kernel_size.0\": list(range(3, 6)),\n", " \n", " \"layer.0\": [Conv2D_MaxPooling2D_layer, Conv2D_layer, no_layer],\n", " \"layer.0.filters\": list(range(5, 12)),\n", " \"layer.0.kernel_size\": list(range(3, 6)),\n", " \n", " \"layer.1\": [Dropout_layer, no_layer],\n", " \"layer.1.rate\": list(np.arange(0.2, 0.8, 0.1)),\n", "\n", " \"dense.0\": list(range(10, 200, 20)),\n", "}" ] }, { "cell_type": "markdown", "id": "gorgeous-duration", "metadata": {}, "source": [ "\n", " \n", "### Bayesian Optimization\n", "\n", "
\n", "\n", "Bayesian optimization is a global optimization technique that uses a machine learning model (surrogate model) to approximate the objective function. It relies on a gaussian process regressor fitting to known positions and scores in the search space and predicting where to search next. It follows the following steps:\n", " \n", " - fit the gaussian process regressor to the training data (positions in search space) and the target (score of each position).\n", " - the regressor makes a prediction of every position in the search space\n", " - from the predictions an acquisition function is calculated that determines which position to evaluate next\n", " - after the evaluation the algorithm adds the position and score to the training data\n", " \n", "\n", "Since the regressor is trained in every iteration the optimization step takes a long time compared to other algorithms. This is why it is often used for objective functions with a long evaluation time. The long optimization time does not matter if the evaluation time is even longer. In those cases it is much more important that each new position is carfully selected to avoid wasted time of an evaluation that has a low score. \n", "\n", "The following plots show an example of the path a bayesian optimization algorithm would take in different objective functions:" ] }, { "cell_type": "markdown", "id": "given-brush", "metadata": {}, "source": [ "\n", "\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 52, "id": "supreme-invention", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-01-02 08:47:40.065310: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n", "2024-01-02 08:47:40.065497: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n", "Skipping registering GPU devices...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", " hyper_optimizer search_data \n", " filters.0 kernel_size.0 \\\n", "0 11 3 \n", "1 13 4 \n", "2 10 4 \n", "3 7 5 \n", "4 7 5 \n", "5 14 5 \n", "6 7 3 \n", "7 7 3 \n", "8 8 3 \n", "9 14 5 \n", "10 13 3 \n", "11 11 3 \n", "12 11 4 \n", "13 12 5 \n", "14 10 3 \n", "15 11 3 \n", "16 10 4 \n", "17 11 3 \n", "18 12 3 \n", "19 11 3 \n", "20 12 3 \n", "21 11 3 \n", "22 11 3 \n", "23 11 3 \n", "24 12 3 \n", "25 10 3 \n", "26 10 3 \n", "27 10 3 \n", "28 11 3 \n", "29 10 3 \n", "\n", " layer.0 layer.0.filters \\\n", "0 8 \n", "3 11 \n", "4 11 \n", "5 8 \n", "8 9 \n", "12 8 \n", "14 10 \n", "20 10 \n", "22 9 \n", "24 \n", "1 5 \n", "2 4 \n", "3 5 \n", "4 5 \n", "5 3 \n", "6 3 \n", "7 3 \n", "8 5 \n", "9 5 \n", "10 5 \n", "11 5 \n", "12 3 \n", "13 3 \n", "14 4 \n", "15 5 \n", "16 4 \n", "17 4 \n", "18 4 \n", "19 3 \n", "20 3 \n", "21 4 \n", "22 4 \n", "23 3 \n", "24 4 \n", "25 5 \n", "26 4 \n", "27 3 \n", "28 5 \n", "29 5 \n", "\n", " layer.1.rate dense.0 score \n", "0 0.4 150 0.9879 \n", "1 0.2 110 0.9898 \n", "2 0.5 90 0.9905 \n", "3 0.2 190 0.9871 \n", "4 0.2 10 0.9746 \n", "5 0.8 10 0.9750 \n", "6 0.8 190 0.9836 \n", "7 0.2 130 0.9864 \n", "8 0.5 70 0.9853 \n", "9 0.3 70 0.9824 \n", "10 0.2 190 0.9887 \n", "11 0.4 150 0.9895 \n", "12 0.3 150 0.9900 \n", "13 0.2 190 0.9860 \n", "14 0.4 130 0.9905 \n", "15 0.2 130 0.9893 \n", "16 0.4 130 0.9860 \n", "17 0.3 150 0.9900 \n", "18 0.3 130 0.9908 \n", "19 0.4 130 0.9909 \n", "20 0.3 150 0.9891 \n", "21 0.3 130 0.9902 \n", "22 0.4 130 0.9885 \n", "23 0.4 130 0.9870 \n", "24 0.2 130 0.9882 \n", "25 0.3 190 0.9904 \n", "26 0.3 170 0.9905 \n", "27 0.3 190 0.9842 \n", "28 0.3 150 0.9892 \n", "29 0.3 170 0.9856 \n", "\n" ] } ], "source": [ "optimizer = BayesianOptimizer()\n", "\n", "hyper_dl = Hyperactive(verbosity=False)\n", "hyper_dl.add_search(deep_learning_model, search_space_dl, n_iter=30, optimizer=optimizer)\n", "hyper_dl.run()\n", "\n", "dl_search_data = hyper_dl.search_data(deep_learning_model, times=True)" ] }, { "cell_type": "code", "execution_count": 53, "id": "indonesian-excuse", "metadata": {}, "outputs": [], "source": [ "# we need to replace the functions with their names for the plot\n", "def func2str(row):\n", " return row.__name__\n", "\n", "dl_search_data[\"layer.0\"] = dl_search_data[\"layer.0\"].apply(func2str)\n", "dl_search_data[\"layer.1\"] = dl_search_data[\"layer.1\"].apply(func2str)\n", "\n", "dl_search_data = dl_search_data.drop([\"eval_times\", \"iter_times\"], axis=1)" ] }, { "cell_type": "code", "execution_count": null, "id": "intelligent-hunter", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 54, "id": "honest-effect", "metadata": {}, "outputs": [], "source": [ "score_max = np.amax(search_data_0[\"score\"])\n", "score_std = search_data_0[\"score\"].std()\n", "dl_search_data_f = dl_search_data[abs(search_data_0[\"score\"]-score_max) < score_std*2]" ] }, { "cell_type": "code", "execution_count": 55, "id": "7a405506", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
filters.0kernel_size.0layer.0layer.0.filterslayer.0.kernel_sizelayer.1layer.1.ratedense.0score
0113Conv2D_MaxPooling2D_layer54no_layer0.41500.9879
1134Conv2D_MaxPooling2D_layer115Dropout_layer0.21100.9898
2104Conv2D_layer84Dropout_layer0.5900.9905
375no_layer115no_layer0.21900.9871
475no_layer115no_layer0.2100.9746
5145Conv2D_MaxPooling2D_layer113Dropout_layer0.8100.9750
673Conv2D_MaxPooling2D_layer53no_layer0.81900.9836
773no_layer83no_layer0.21300.9864
883Conv2D_MaxPooling2D_layer65no_layer0.5700.9853
9145Conv2D_MaxPooling2D_layer55Dropout_layer0.3700.9824
10133Conv2D_MaxPooling2D_layer105Dropout_layer0.21900.9887
11113Conv2D_layer95Dropout_layer0.41500.9895
12114Conv2D_MaxPooling2D_layer93Dropout_layer0.31500.9900
13125no_layer83Dropout_layer0.21900.9860
14103Conv2D_MaxPooling2D_layer94Dropout_layer0.41300.9905
15113Conv2D_MaxPooling2D_layer95Dropout_layer0.21300.9893
16104Conv2D_MaxPooling2D_layer84Dropout_layer0.41300.9860
17113Conv2D_MaxPooling2D_layer114Dropout_layer0.31500.9900
18123Conv2D_MaxPooling2D_layer104Dropout_layer0.31300.9908
19113Conv2D_layer103Dropout_layer0.41300.9909
20123Conv2D_MaxPooling2D_layer103no_layer0.31500.9891
21113Conv2D_layer104Dropout_layer0.31300.9902
22113Conv2D_MaxPooling2D_layer104Dropout_layer0.41300.9885
23113no_layer93Dropout_layer0.41300.9870
24123Conv2D_MaxPooling2D_layer114Dropout_layer0.21300.9882
25103Conv2D_MaxPooling2D_layer115Dropout_layer0.31900.9904
26103Conv2D_MaxPooling2D_layer104Dropout_layer0.31700.9905
27103Conv2D_MaxPooling2D_layer113Dropout_layer0.31900.9842
28113Conv2D_MaxPooling2D_layer105Dropout_layer0.31500.9892
29103Conv2D_MaxPooling2D_layer105no_layer0.31700.9856
\n", "
" ], "text/plain": [ " filters.0 kernel_size.0 layer.0 layer.0.filters \\\n", "0 11 3 Conv2D_MaxPooling2D_layer 5 \n", "1 13 4 Conv2D_MaxPooling2D_layer 11 \n", "2 10 4 Conv2D_layer 8 \n", "3 7 5 no_layer 11 \n", "4 7 5 no_layer 11 \n", "5 14 5 Conv2D_MaxPooling2D_layer 11 \n", "6 7 3 Conv2D_MaxPooling2D_layer 5 \n", "7 7 3 no_layer 8 \n", "8 8 3 Conv2D_MaxPooling2D_layer 6 \n", "9 14 5 Conv2D_MaxPooling2D_layer 5 \n", "10 13 3 Conv2D_MaxPooling2D_layer 10 \n", "11 11 3 Conv2D_layer 9 \n", "12 11 4 Conv2D_MaxPooling2D_layer 9 \n", "13 12 5 no_layer 8 \n", "14 10 3 Conv2D_MaxPooling2D_layer 9 \n", "15 11 3 Conv2D_MaxPooling2D_layer 9 \n", "16 10 4 Conv2D_MaxPooling2D_layer 8 \n", "17 11 3 Conv2D_MaxPooling2D_layer 11 \n", "18 12 3 Conv2D_MaxPooling2D_layer 10 \n", "19 11 3 Conv2D_layer 10 \n", "20 12 3 Conv2D_MaxPooling2D_layer 10 \n", "21 11 3 Conv2D_layer 10 \n", "22 11 3 Conv2D_MaxPooling2D_layer 10 \n", "23 11 3 no_layer 9 \n", "24 12 3 Conv2D_MaxPooling2D_layer 11 \n", "25 10 3 Conv2D_MaxPooling2D_layer 11 \n", "26 10 3 Conv2D_MaxPooling2D_layer 10 \n", "27 10 3 Conv2D_MaxPooling2D_layer 11 \n", "28 11 3 Conv2D_MaxPooling2D_layer 10 \n", "29 10 3 Conv2D_MaxPooling2D_layer 10 \n", "\n", " layer.0.kernel_size layer.1 layer.1.rate dense.0 score \n", "0 4 no_layer 0.4 150 0.9879 \n", "1 5 Dropout_layer 0.2 110 0.9898 \n", "2 4 Dropout_layer 0.5 90 0.9905 \n", "3 5 no_layer 0.2 190 0.9871 \n", "4 5 no_layer 0.2 10 0.9746 \n", "5 3 Dropout_layer 0.8 10 0.9750 \n", "6 3 no_layer 0.8 190 0.9836 \n", "7 3 no_layer 0.2 130 0.9864 \n", "8 5 no_layer 0.5 70 0.9853 \n", "9 5 Dropout_layer 0.3 70 0.9824 \n", "10 5 Dropout_layer 0.2 190 0.9887 \n", "11 5 Dropout_layer 0.4 150 0.9895 \n", "12 3 Dropout_layer 0.3 150 0.9900 \n", "13 3 Dropout_layer 0.2 190 0.9860 \n", "14 4 Dropout_layer 0.4 130 0.9905 \n", "15 5 Dropout_layer 0.2 130 0.9893 \n", "16 4 Dropout_layer 0.4 130 0.9860 \n", "17 4 Dropout_layer 0.3 150 0.9900 \n", "18 4 Dropout_layer 0.3 130 0.9908 \n", "19 3 Dropout_layer 0.4 130 0.9909 \n", "20 3 no_layer 0.3 150 0.9891 \n", "21 4 Dropout_layer 0.3 130 0.9902 \n", "22 4 Dropout_layer 0.4 130 0.9885 \n", "23 3 Dropout_layer 0.4 130 0.9870 \n", "24 4 Dropout_layer 0.2 130 0.9882 \n", "25 5 Dropout_layer 0.3 190 0.9904 \n", "26 4 Dropout_layer 0.3 170 0.9905 \n", "27 3 Dropout_layer 0.3 190 0.9842 \n", "28 5 Dropout_layer 0.3 150 0.9892 \n", "29 5 no_layer 0.3 170 0.9856 " ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dl_search_data" ] }, { "cell_type": "code", "execution_count": 56, "id": "significant-result", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "dimensions": [ { "label": "filters.0", "values": [ 11, 13, 10, 7, 7, 14, 7, 7, 8, 14, 13, 11, 11, 12, 10, 11, 10, 11, 12, 11, 12, 11, 11, 11, 12, 10, 10, 10, 11, 10 ] }, { "label": "kernel_size.0", "values": [ 3, 4, 4, 5, 5, 5, 3, 3, 3, 5, 3, 3, 4, 5, 3, 3, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 ] }, { "label": "layer.0", "values": [ "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_layer", "no_layer", "no_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "no_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_layer", "Conv2D_MaxPooling2D_layer", "no_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_layer", "Conv2D_MaxPooling2D_layer", "no_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer", "Conv2D_MaxPooling2D_layer" ] }, { "label": "layer.0.filters", "values": [ 5, 11, 8, 11, 11, 11, 5, 8, 6, 5, 10, 9, 9, 8, 9, 9, 8, 11, 10, 10, 10, 10, 10, 9, 11, 11, 10, 11, 10, 10 ] }, { "label": "layer.0.kernel_size", "values": [ 4, 5, 4, 5, 5, 3, 3, 3, 5, 5, 5, 5, 3, 3, 4, 5, 4, 4, 4, 3, 3, 4, 4, 3, 4, 5, 4, 3, 5, 5 ] }, { "label": "layer.1", "values": [ "no_layer", "Dropout_layer", "Dropout_layer", "no_layer", "no_layer", "Dropout_layer", "no_layer", "no_layer", "no_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "no_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "Dropout_layer", "no_layer" ] }, { "label": "layer.1.rate", "values": [ 0.4000000000000001, 0.2, 0.5000000000000001, 0.2, 0.2, 0.8000000000000003, 0.8000000000000003, 0.2, 0.5000000000000001, 0.30000000000000004, 0.2, 0.4000000000000001, 0.30000000000000004, 0.2, 0.4000000000000001, 0.2, 0.4000000000000001, 0.30000000000000004, 0.30000000000000004, 0.4000000000000001, 0.30000000000000004, 0.30000000000000004, 0.4000000000000001, 0.4000000000000001, 0.2, 0.30000000000000004, 0.30000000000000004, 0.30000000000000004, 0.30000000000000004, 0.30000000000000004 ] }, { "label": "dense.0", "values": [ 150, 110, 90, 190, 10, 10, 190, 130, 70, 70, 190, 150, 150, 190, 130, 130, 130, 150, 130, 130, 150, 130, 130, 130, 130, 190, 170, 190, 150, 170 ] }, { "label": "score", "values": [ 0.9879000186920166, 0.989799976348877, 0.9904999732971191, 0.9871000051498413, 0.9746000170707703, 0.9750000238418579, 0.9836000204086304, 0.9864000082015991, 0.9853000044822693, 0.9824000000953674, 0.9886999726295471, 0.9894999861717224, 0.9900000095367432, 0.9860000014305115, 0.9904999732971191, 0.989300012588501, 0.9860000014305115, 0.9900000095367432, 0.9908000230789185, 0.9908999800682068, 0.9890999794006348, 0.9901999831199646, 0.9884999990463257, 0.9869999885559082, 0.9882000088691711, 0.9904000163078308, 0.9904999732971191, 0.9842000007629395, 0.9891999959945679, 0.9855999946594238 ] } ], "domain": { "x": [ 0, 1 ], "y": [ 0, 1 ] }, "line": { "color": [ 0.9879000186920166, 0.989799976348877, 0.9904999732971191, 0.9871000051498413, 0.9746000170707703, 0.9750000238418579, 0.9836000204086304, 0.9864000082015991, 0.9853000044822693, 0.9824000000953674, 0.9886999726295471, 0.9894999861717224, 0.9900000095367432, 0.9860000014305115, 0.9904999732971191, 0.989300012588501, 0.9860000014305115, 0.9900000095367432, 0.9908000230789185, 0.9908999800682068, 0.9890999794006348, 0.9901999831199646, 0.9884999990463257, 0.9869999885559082, 0.9882000088691711, 0.9904000163078308, 0.9904999732971191, 0.9842000007629395, 0.9891999959945679, 0.9855999946594238 ], "coloraxis": "coloraxis" }, "name": "", "type": "parcats" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "score" } }, "colorscale": [ [ 0, "rgb(0,0,131)" ], [ 0.2, "rgb(0,60,170)" ], [ 0.4, "rgb(5,255,255)" ], [ 0.6, "rgb(255,255,0)" ], [ 0.8, "rgb(250,0,0)" ], [ 1, "rgb(128,0,0)" ] ] }, "height": 700, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 950 } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "parameter_names = list(dl_search_data_f.keys())\n", "\n", "fig = px.parallel_categories(dl_search_data, \n", " color=\"score\", \n", " color_continuous_scale=color_scale, \n", " dimensions=parameter_names, \n", " )\n", "fig.update_layout(width=950, height=700)\n", "fig.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 5 }