{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Hi, Are you in Google Colab?\n", "In Google colab you can easily run Optimus. If you not you may want to go here\n", "https://colab.research.google.com/github/ironmussa/Optimus/blob/master/examples/10_min_from_spark_to_pandas_with_optimus.ipynb" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Install Optimus all the dependencies." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import sys\n", "if 'google.colab' in sys.modules:\n", " !apt-get install openjdk-8-jdk-headless -qq > /dev/null\n", " !wget -q https://archive.apache.org/dist/spark/spark-2.4.1/spark-2.4.1-bin-hadoop2.7.tgz\n", " !tar xf spark-2.4.1-bin-hadoop2.7.tgz\n", " !pip install optimuspyspark" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Restart Runtime\n", "Before you continue, please go to the 'Runtime' Menu above, and select 'Restart Runtime (Ctrl + M + .)'." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "if 'google.colab' in sys.modules:\n", " import os\n", " os.environ[\"JAVA_HOME\"] = \"/usr/lib/jvm/java-8-openjdk-amd64\"\n", " os.environ[\"SPARK_HOME\"] = \"/content/spark-2.4.1-bin-hadoop2.7\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## You are done. Enjoy Optimus!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Hacking Optimus!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To hacking Optimus we recommend to clone the repo and change ```repo_path``` relative to this notebook." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "repo_path=\"..\"\n", "\n", "# This will reload the change you make to Optimus in real time\n", "%load_ext autoreload\n", "%autoreload 2\n", "import sys\n", "sys.path.append(repo_path)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install Optimus \n", "\n", "from command line:\n", "\n", "`pip install optimuspyspark`\n", "\n", "from a notebook you can use:\n", "\n", "`!pip install optimuspyspark`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import Optimus and start it" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\argenisleon\\Anaconda3\\lib\\site-packages\\socks.py:58: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n", " from collections import Callable\n", "\n", " You are using PySparkling of version 2.4.10, but your PySpark is of\n", " version 2.3.1. Please make sure Spark and PySparkling versions are compatible. \n", "`formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly\n" ] } ], "source": [ "from optimus import Optimus" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "op = Optimus(master=\"local\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Dataframe creation\n", "\n", "Create a dataframe to passing a list of values for columns and rows. Unlike pandas you need to specify the column names.\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 6 of 6 rows / 8 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height(ft)
\n", "
2 (float)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (int)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
weight(t)
\n", "
5 (float)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
japanese name
\n", "
6 (array<string>)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
last position
\n", "
7 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
attributes
\n", "
8 (array<float>)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
Optim'us\n", "
\n", " 		\n", "
28.0\n", "
\n", " 		\n", "
Leader\n", "
\n", " 		\n", "
10\n", "
\n", " 		\n", "
4.300000190734863\n", "
\n", " 		\n", "
['Inochi',⋅'Convoy']\n", "
\n", " 		\n", "
19.442735,-99.201111\n", "
\n", " 		\n", "
[8.53439998626709,⋅4300.0]\n", "
\n", "
\n", "
bumbl#ebéé⋅⋅\n", "
\n", " 		\n", "
17.5\n", "
\n", " 		\n", "
Espionage\n", "
\n", " 		\n", "
7\n", "
\n", " 		\n", "
2.0\n", "
\n", " 		\n", "
['Bumble',⋅'Goldback']\n", "
\n", " 		\n", "
10.642707,-71.612534\n", "
\n", " 		\n", "
[5.334000110626221,⋅2000.0]\n", "
\n", "
\n", "
ironhide&\n", "
\n", " 		\n", "
26.0\n", "
\n", " 		\n", "
Security\n", "
\n", " 		\n", "
7\n", "
\n", " 		\n", "
4.0\n", "
\n", " 		\n", "
['Roadbuster']\n", "
\n", " 		\n", "
37.789563,-122.400356\n", "
\n", " 		\n", "
[7.924799919128418,⋅4000.0]\n", "
\n", "
\n", "
Jazz\n", "
\n", " 		\n", "
13.0\n", "
\n", " 		\n", "
First⋅Lieutenant\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
1.7999999523162842\n", "
\n", " 		\n", "
['Meister']\n", "
\n", " 		\n", "
33.670666,-117.841553\n", "
\n", " 		\n", "
[3.962399959564209,⋅1800.0]\n", "
\n", "
\n", "
Megatron\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
5.699999809265137\n", "
\n", " 		\n", "
['Megatron']\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
[None,⋅5700.0]\n", "
\n", "
\n", "
Metroplex_)^$\n", "
\n", " 		\n", "
300.0\n", "
\n", " 		\n", "
Battle⋅Station\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
['Metroflex']\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
[91.44000244140625,⋅None]\n", "
\n", "
\n", "\n", "\n", "
Viewing 6 of 6 rows / 8 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df = op.create.df(\n", " [\n", " \"names\",\n", " \"height(ft)\",\n", " \"function\",\n", " \"rank\",\n", " \"weight(t)\",\n", " \"japanese name\",\n", " \"last position\",\n", " \"attributes\"\n", " ],\n", " [\n", "\n", " (\"Optim'us\", 28.0, \"Leader\", 10, 4.3, [\"Inochi\", \"Convoy\"], \"19.442735,-99.201111\", [8.5344, 4300.0]),\n", " (\"bumbl#ebéé \", 17.5, \"Espionage\", 7, 2.0, [\"Bumble\", \"Goldback\"], \"10.642707,-71.612534\", [5.334, 2000.0]),\n", " (\"ironhide&\", 26.0, \"Security\", 7, 4.0, [\"Roadbuster\"], \"37.789563,-122.400356\", [7.9248, 4000.0]),\n", " (\"Jazz\", 13.0, \"First Lieutenant\", 8, 1.8, [\"Meister\"], \"33.670666,-117.841553\", [3.9624, 1800.0]),\n", " (\"Megatron\", None, \"None\", None, 5.7, [\"Megatron\"], None, [None, 5700.0]),\n", " (\"Metroplex_)^$\", 300.0, \"Battle Station\", 8, None, [\"Metroflex\"], None, [91.44, None]),\n", "\n", " ]).h_repartition(1)\n", "df.table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Creating a dataframe by passing a list of tuples specifyng the column data type. You can specify as data type an string or a Spark Datatypes. https://spark.apache.org/docs/2.3.1/api/java/org/apache/spark/sql/types/package-summary.html\n", "\n", "Also you can use some Optimus predefined types:\n", "* \"str\" = StringType() \n", "* \"int\" = IntegerType() \n", "* \"float\" = FloatType()\n", "* \"bool\" = BoleanType()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 5 of 5 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (float)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (int)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
bumbl#ebéé⋅⋅\n", "
\n", " 		\n", "
17.5\n", "
\n", " 		\n", "
Espionage\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
Optim'us\n", "
\n", " 		\n", "
28.0\n", "
\n", " 		\n", "
Leader\n", "
\n", " 		\n", "
10\n", "
\n", "
\n", "
ironhide&\n", "
\n", " 		\n", "
26.0\n", "
\n", " 		\n", "
Security\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
Jazz\n", "
\n", " 		\n", "
13.0\n", "
\n", " 		\n", "
First⋅Lieutenant\n", "
\n", " 		\n", "
8\n", "
\n", "
\n", "
Megatron\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
None\n", "
\n", "
\n", "\n", "\n", "
Viewing 5 of 5 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df = op.create.df(\n", " [\n", " (\"names\", \"str\"),\n", " (\"height\", \"float\"),\n", " (\"function\", \"str\"),\n", " (\"rank\", \"int\"),\n", " ],\n", " [\n", " (\"bumbl#ebéé \", 17.5, \"Espionage\", 7),\n", " (\"Optim'us\", 28.0, \"Leader\", 10),\n", " (\"ironhide&\", 26.0, \"Security\", 7),\n", " (\"Jazz\", 13.0, \"First Lieutenant\", 8),\n", " (\"Megatron\", None, \"None\", None),\n", "\n", " ])\n", "df.table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Creating a dataframe and specify if the column accepts null values" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 5 of 5 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (float)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (int)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
bumbl#ebéé⋅⋅\n", "
\n", " 		\n", "
17.5\n", "
\n", " 		\n", "
Espionage\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
Optim'us\n", "
\n", " 		\n", "
28.0\n", "
\n", " 		\n", "
Leader\n", "
\n", " 		\n", "
10\n", "
\n", "
\n", "
ironhide&\n", "
\n", " 		\n", "
26.0\n", "
\n", " 		\n", "
Security\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
Jazz\n", "
\n", " 		\n", "
13.0\n", "
\n", " 		\n", "
First⋅Lieutenant\n", "
\n", " 		\n", "
8\n", "
\n", "
\n", "
Megatron\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
None\n", "
\n", "
\n", "\n", "\n", "
Viewing 5 of 5 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df = op.create.df(\n", " [\n", " (\"names\", \"str\", True),\n", " (\"height\", \"float\", True),\n", " (\"function\", \"str\", True),\n", " (\"rank\", \"int\", True),\n", " ],\n", " [\n", " (\"bumbl#ebéé \", 17.5, \"Espionage\", 7),\n", " (\"Optim'us\", 28.0, \"Leader\", 10),\n", " (\"ironhide&\", 26.0, \"Security\", 7),\n", " (\"Jazz\", 13.0, \"First Lieutenant\", 8),\n", " (\"Megatron\", None, \"None\", None),\n", "\n", " ])\n", "df.table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Creating a Daframe using a pandas dataframe" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
bumbl#ebéé⋅⋅\n", "
\n", " 		\n", "
17.5\n", "
\n", " 		\n", "
Espionage\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
Optim'us\n", "
\n", " 		\n", "
28.0\n", "
\n", " 		\n", "
Leader\n", "
\n", " 		\n", "
10\n", "
\n", "
\n", "
ironhide&\n", "
\n", " 		\n", "
26.0\n", "
\n", " 		\n", "
Security\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import pandas as pd\n", "\n", "data = [(\"bumbl#ebéé \", 17.5, \"Espionage\", 7),\n", " (\"Optim'us\", 28.0, \"Leader\", 10),\n", " (\"ironhide&\", 26.0, \"Security\", 7)]\n", "labels = [\"names\", \"height\", \"function\", \"rank\"]\n", "\n", "# Create pandas dataframe\n", "pdf = pd.DataFrame.from_records(data, columns=labels)\n", "\n", "df = op.create.df(pdf=pdf)\n", "df.table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Viewing data\n", "Here is how to View the first 10 elements in a dataframe." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
bumbl#ebéé⋅⋅\n", "
\n", " 		\n", "
17.5\n", "
\n", " 		\n", "
Espionage\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
Optim'us\n", "
\n", " 		\n", "
28.0\n", "
\n", " 		\n", "
Leader\n", "
\n", " 		\n", "
10\n", "
\n", "
\n", "
ironhide&\n", "
\n", " 		\n", "
26.0\n", "
\n", " 		\n", "
Security\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.table(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## About Spark\n", "Spark and Optimus work differently than pandas or R. If you are not familiar with Spark, we recommend taking the time to take a look at the links below.\n", "\n", "### Partitions\n", "Partition are the way Spark divide the data in your local computer or cluster to better optimize how it will be processed.It can greatly impact the Spark performance.\n", "\n", "Take 5 minutes to read this article:\n", "https://www.dezyre.com/article/how-data-partitioning-in-spark-helps-achieve-more-parallelism/297\n", "\n", "### Lazy operations\n", "Lazy evaluation in Spark means that the execution will not start until an action is triggered.\n", "\n", "https://stackoverflow.com/questions/38027877/spark-transformation-why-its-lazy-and-what-is-the-advantage\n", "\n", "### Inmutability\n", "Immutability rules out a big set of potential problems due to updates from multiple threads at once. Immutable data is definitely safe to share across processes.\n", "\n", "https://www.quora.com/Why-is-RDD-immutable-in-Spark\n", "\n", "### Spark Architecture\n", "https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-architecture.html" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Columns and Rows\n", "\n", "Optimus organized operations in columns and rows. This is a little different of how pandas works in which all operations are aroud the pandas class. We think this approach can better help you to access and transform data. For a deep dive about the designing decision please read:\n", "\n", "https://towardsdatascience.com/announcing-optimus-v2-agile-data-science-workflows-made-easy-c127a12d9e13" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sort by cols names" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
function
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
names
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " Espionage\n", " \n", " 17.5\n", " \n", " bumbl#ebéé⸱⸱\n", " \n", " 7\n", "
\n", " Leader\n", " \n", " 28.0\n", " \n", " Optim'us\n", " \n", " 10\n", "
\n", " Security\n", " \n", " 26.0\n", " \n", " ironhide&\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.cols.sort().table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sort by rows rank value" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
3 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
3 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.rows.sort(\"rank\").table()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 5 of 5 rows / 5 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
summary
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
names
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
5 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
count\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
3\n", "
\n", "
\n", "
mean\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
23.833333333333332\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
8.0\n", "
\n", "
\n", "
stddev\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
5.575242894559244\n", "
\n", " 		\n", "
None\n", "
\n", " 		\n", "
1.7320508075688772\n", "
\n", "
\n", "
min\n", "
\n", " 		\n", "
Optim'us\n", "
\n", " 		\n", "
17.5\n", "
\n", " 		\n", "
Espionage\n", "
\n", " 		\n", "
7\n", "
\n", "
\n", "
max\n", "
\n", " 		\n", "
ironhide&\n", "
\n", " 		\n", "
28.0\n", "
\n", " 		\n", "
Security\n", "
\n", " 		\n", "
10\n", "
\n", "
\n", "\n", "\n", "
Viewing 5 of 5 rows / 5 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.describe().table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Selection\n", "\n", "Unlike Pandas, Spark DataFrames don't support random row access. So methods like `loc` in pandas are not available.\n", "\n", "Also Pandas don't handle indexes. So methods like `iloc` are not available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Select an show an specific column" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 1 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", "
\n", " Optim'us\n", "
\n", " ironhide&\n", "
\n", "\n", "
Viewing 3 of 3 rows / 1 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.cols.select(\"names\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Select rows from a Dataframe where a the condition is meet" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 1 of 1 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", "\n", "
Viewing 1 of 1 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.rows.select(df[\"rank\"] > 7).table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Select rows by specific values on it" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.rows.is_in(\"rank\", [7, 10]).table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create and unique id for every row." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df.rows.create_id().table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create wew columns" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 5 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
Affiliation
\n", "
5 (string)
\n", "
\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", " \n", " Autobot\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", " \n", " Autobot\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", " \n", " Autobot\n", "
\n", "\n", "
Viewing 3 of 3 rows / 5 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.cols.append(\"Affiliation\", \"Autobot\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Missing Data" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.rows.drop_na(\"*\", how='any').table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Filling missing data." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.cols.fill_na(\"*\", \"N//A\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get the boolean mask where values are nan." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (boolean)
\n", "
\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (boolean)
\n", "
\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " False\n", " \n", " Espionage\n", " \n", " False\n", "
\n", " Optim'us\n", " \n", " False\n", " \n", " Leader\n", " \n", " False\n", "
\n", " ironhide&\n", " \n", " False\n", " \n", " Security\n", " \n", " False\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.cols.is_na(\"*\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Operations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Stats" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/plain": [ "23.833333333333332" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.cols.mean(\"height\")" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/plain": [ "{'rank': {'mean': 8.0}, 'height': {'mean': 23.833333333333332}}" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.cols.mean(\"*\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Apply" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (float)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 18.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 29.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 27.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def func(value, args):\n", " return value + 1\n", "\n", "\n", "df.cols.apply(\"height\", func, \"float\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Histogramming" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'names': {'approx_count_distinct': 3},\n", " 'height': {'approx_count_distinct': 3},\n", " 'function': {'approx_count_distinct': 3},\n", " 'rank': {'approx_count_distinct': 2}}" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.cols.count_uniques(\"*\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### String Methods" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " ESPIONAGE\n", " \n", " 7\n", "
\n", " optim'us\n", " \n", " 28.0\n", " \n", " LEADER\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " SECURITY\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df \\\n", " .cols.lower(\"names\") \\\n", " .cols.upper(\"function\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Merge" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Concat\n", "\n", "Optimus provides and intuitive way to concat Dataframes by columns or rows." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'op' is not defined", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m df_new = op.create.df(\n\u001b[0m\u001b[0;32m 2\u001b[0m [\n\u001b[0;32m 3\u001b[0m \u001b[1;34m\"class\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m ],\n\u001b[0;32m 5\u001b[0m [\n", "\u001b[1;31mNameError\u001b[0m: name 'op' is not defined" ] } ], "source": [ "df_new = op.create.df(\n", " [\n", " \"class\"\n", " ],\n", " [\n", " (\"Autobot\"),\n", " (\"Autobot\"),\n", " (\"Autobot\"),\n", " (\"Autobot\"),\n", " (\"Decepticons\"),\n", "\n", " ]).h_repartition(1)\n", "\n", "op.append([df, df_new], \"columns\").table()" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 4 of 4 rows / 4 columns
\n", "
2 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", " Grimlock\n", " \n", " 22.9\n", " \n", " Dinobot⸱Commander\n", " \n", " 9\n", "
\n", "\n", "
Viewing 4 of 4 rows / 4 columns
\n", "
2 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df_new = op.create.df(\n", " [\n", " \"names\",\n", " \"height\",\n", " \"function\",\n", " \"rank\",\n", " ],\n", " [\n", " (\"Grimlock\", 22.9, \"Dinobot Commander\", 9),\n", " ]).h_repartition(1)\n", "\n", "op.append([df, df_new], \"rows\").table()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [], "source": [ "# Operations like `join` and `group` are handle using Spark directly" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df_melt = df.melt(id_vars=[\"names\"], value_vars=[\"height\", \"function\", \"rank\"])\n", "df.table()" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
200 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " Espionage\n", " \n", " 17.5\n", " \n", " 7\n", "
\n", " ironhide&\n", " \n", " Security\n", " \n", " 26.0\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " Leader\n", " \n", " 28.0\n", " \n", " 10\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
200 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df_melt.pivot(\"names\", \"variable\", \"value\").table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Ploting" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "bucketizer() executed in 0.1 sec\n", "hist() executed in 1.27 sec\n", "hist() executed in 3.39 sec\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAEHCAYAAACp5ActAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAFU9JREFUeJzt3X+w5WddH/D3h6zR2gABd0FIdtkUQzVlquiaglqJQmc2KFn+wDYpVNIypoWCFUWMxAbE4vDDgcIYf6TK8EMkBKq4tUtDrTAUh2A2YIAkZNiGQNYEEgIhZDDEhU//OGfD4e79cfJwN/fc7Os1c+ee7/N9zvl+zplnzv2+7/Oc76nuDgAAAPfeAza6AAAAgM1KoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQD3qKqrq+qMja7jvlJVO6uqq2rLwH13VNWdVXXc0TwOAItNoAI4RlTVDVX15CVt51bVBw5vd/c/6e73rfE4mzocTJ/zG7/Vx+nuz3T3Cd39tXWo6aVV9UdL2t53LIVbgM1KoAJgoWzWoAbAsUmgAuAes7NYVXV6Ve2vqjuq6nNV9Zppt/dPf98+XfL2hKp6QFX9WlV9uqpuqao3V9WDZx73Z6f7bquq/7zkOC+tqndW1R9V1R1Jzp0e+4NVdXtV3VxVv11Vx888XlfVc6vqk1X15ar6jap69PQ+d1TVpbP95/CMqvpMVX2+qi6YOc4Dqur8qvp/09ovraqHTvd900xdVZ1SVe+f1vMXVXXR0lmn5Y5TVbuTvDjJv5q+nlfdi7oB2GACFQAreV2S13X3g5I8Osml0/Yfn/4+cbrk7YNJzp3+/ESSf5TkhCS/nSRVdVqS30nyjCSPSPLgJCctOdaeJO9McmKStyb5WpIXJNma5AlJnpTkuUvuszvJDyV5fJIXJbl4eoztSR6b5JzlnlR3v7G7z13S/GNJ/vH0OBdW1fdN238+ydOSPDHJI5N8MclFyz1ukj9O8tdJvivJS5P8m2X6HHGc7v5fSX4zydunr+f3T+s8Y63llwBsPIEK4Njyrumsz+1VdXsmQWclf5/ke6pqa3ff2d2Xr9L3GUle093Xd/edSX41ydnT2ZunJ/kf3f2B7r47yYVJesn9P9jd7+rur3f333X3ld19eXcf6u4bkvx+JqFm1iu7+47uvjrJx5O8Z3r8LyV5d5LHzfeSJEl+fXrcq5JcleT7p+3/PskF3X2wu7+aSVB6+tJliVW1I8kPJ7mwu+/u7g8k2XsvjgPAJiVQARxbntbdJx7+yZGzPrOeneQxST5RVVdU1U+v0veRST49s/3pJFuSPHy678bDO7r7K0luW3L/G2c3quoxVfXnVfXZ6TLA38xktmrW52Zu/90y2yesUu9Sn525/ZWZ+z4qyZ/OBNBrM5k9e/iS+z8yyRemz23Z57TGcQDYpAQqAJbV3Z/s7nOSPCzJK5O8s6r+YY6cXUqSmzIJH4ftSHIok5Bzc5KTD++oqn+QybK4bzrcku3fTfKJJKdOlxy+OEmNP5thNyY5czaEdvd3dPffLul3c5KHVtV3zrRtvxfHWe41BWATEKgAWFZVPbOqtnX315PcPm3+WpJbk3w9k89KHfa2JC+YXpjhhHzjM0GHMvls1FOr6kemF4r49awdjh6Y5I4kd1bV9yZ5zro9sXvn95K8vKoelSRVta2q9izt1N2fTrI/yUur6viqekKSp96L43wuyc6q8ncZYJPxxg3ASnYnubqq7szkAhVnd/dd02VtL0/yV9OlcI9P8oYkb8nkCoCfSnJXkucnyfQzTs9PckkmMzlfTnJLkq+ucuwXJvnX077/Lcnb1//pzeV1mXwW6j1V9eUklyf5Zyv0fUYmF9C4Lcl/yaTm1Z7jrHdMf99WVR8eLxeA+1p1W2UAwH1nOoN1eybL+T610fUcLVX19iSf6O6XbHQtABw9ZqgAOOqq6qlV9Z3Tz2D9VpKPJblhY6taX1X1w9PvwnrA9Lul9iR510bXBcDRJVABcF/Yk8mFK25Kcmomywfvb0skvjvJ+5LcmeT1SZ7T3R/Z0IoAOOos+QMAABhkhgoAAGCQQAUAADBoy0YdeOvWrb1z586NOjwAAMCKrrzyys9397a1+m1YoNq5c2f279+/UYcHAABYUVV9ep5+lvwBAAAMEqgAAAAGCVQAAACDBCoAAIBBawaqqnpDVd1SVR9fYX9V1eur6kBVfbSqfnD9ywQAAFg888xQvTHJ7lX2n5nk1OnPeUl+91svCwAAYPGtGai6+/1JvrBKlz1J3twTlyc5saoesV4FAgAALKr1+AzVSUlunNk+OG0DAAC4X1uPL/atZdp62Y5V52WyLDA7duxYh0Ovr53n/8+NLmFh3PCKn9roEgCATco51Tesdk7ldfqGzXzuuR4zVAeTbJ/ZPjnJTct17O6Lu3tXd+/atm3bOhwaAABg46xHoNqb5GenV/t7fJIvdffN6/C4AAAAC23NJX9V9bYkZyTZWlUHk7wkybclSXf/XpJ9SZ6S5ECSryT5t0erWAAAgEWyZqDq7nPW2N9J/uO6VQQAALBJrMeSPwAAgGOSQAUAADBIoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAIIEKAABgkEAFAAAwSKACAAAYJFABAAAMEqgAAAAGCVQAAACDBCoAAIBBAhUAAMAggQoAAGCQQAUAADBIoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYNBcgaqqdlfVdVV1oKrOX2b/jqp6b1V9pKo+WlVPWf9SAQAAFsuagaqqjktyUZIzk5yW5JyqOm1Jt19Lcml3Py7J2Ul+Z70LBQAAWDTzzFCdnuRAd1/f3XcnuSTJniV9OsmDprcfnOSm9SsRAABgMc0TqE5KcuPM9sFp26yXJnlmVR1Msi/J85d7oKo6r6r2V9X+W2+9daBcAACAxTFPoKpl2nrJ9jlJ3tjdJyd5SpK3VNURj93dF3f3ru7etW3btntfLQAAwAKZJ1AdTLJ9ZvvkHLmk79lJLk2S7v5gku9IsnU9CgQAAFhU8wSqK5KcWlWnVNXxmVx0Yu+SPp9J8qQkqarvyyRQWdMHAADcr60ZqLr7UJLnJbksybWZXM3v6qp6WVWdNe32S0l+rqquSvK2JOd299JlgQAAAPcrW+bp1N37MrnYxGzbhTO3r0nyo+tbGgAAwGKb64t9AQAAOJJABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAIIEKAABgkEAFAAAwSKACAAAYJFABAAAMEqgAAAAGCVQAAACDBCoAAIBBAhUAAMAggQoAAGCQQAUAADBIoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAoLkCVVXtrqrrqupAVZ2/Qp9/WVXXVNXVVfXH61smAADA4tmyVoeqOi7JRUn+RZKDSa6oqr3dfc1Mn1OT/GqSH+3uL1bVw45WwQAAAItinhmq05Mc6O7ru/vuJJck2bOkz88luai7v5gk3X3L+pYJAACweOYJVCcluXFm++C0bdZjkjymqv6qqi6vqt3LPVBVnVdV+6tq/6233jpWMQAAwIKYJ1DVMm29ZHtLklOTnJHknCR/UFUnHnGn7ou7e1d379q2bdu9rRUAAGChzBOoDibZPrN9cpKblunzZ9399939qSTXZRKwAAAA7rfmCVRXJDm1qk6pquOTnJ1k75I+70ryE0lSVVszWQJ4/XoWCgAAsGjWDFTdfSjJ85JcluTaJJd299VV9bKqOmva7bIkt1XVNUnem+SXu/u2o1U0AADAIljzsulJ0t37kuxb0nbhzO1O8ovTHwAAgGPCXF/sCwAAwJEEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAIIEKAABgkEAFAAAwSKACAAAYJFABAAAMEqgAAAAGCVQAAACDBCoAAIBBAhUAAMAggQoAAGCQQAUAADBIoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADBKoAAAABs0VqKpqd1VdV1UHqur8Vfo9vaq6qnatX4kAAACLac1AVVXHJbkoyZlJTktyTlWdtky/Byb5+SQfWu8iAQAAFtE8M1SnJznQ3dd3991JLkmyZ5l+v5HkVUnuWsf6AAAAFtY8geqkJDfObB+ctt2jqh6XZHt3//k61gYAALDQ5glUtUxb37Oz6gFJXpvkl9Z8oKrzqmp/Ve2/9dZb568SAABgAc0TqA4m2T6zfXKSm2a2H5jksUneV1U3JHl8kr3LXZiiuy/u7l3dvWvbtm3jVQMAACyAeQLVFUlOrapTqur4JGcn2Xt4Z3d/qbu3dvfO7t6Z5PIkZ3X3/qNSMQAAwIJYM1B196Ekz0tyWZJrk1za3VdX1cuq6qyjXSAAAMCi2jJPp+7el2TfkrYLV+h7xrdeFgAAwOKb64t9AQAAOJJABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAIIEKAABgkEAFAAAwSKACAAAYJFABAAAMEqgAAAAGCVQAAACDBCoAAIBBAhUAAMAggQoAAGCQQAUAADBIoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAoLkCVVXtrqrrqupAVZ2/zP5frKprquqjVfV/qupR618qAADAYlkzUFXVcUkuSnJmktOSnFNVpy3p9pEku7r7nyZ5Z5JXrXehAAAAi2aeGarTkxzo7uu7++4klyTZM9uhu9/b3V+Zbl6e5OT1LRMAAGDxzBOoTkpy48z2wWnbSp6d5N3fSlEAAACbwZY5+tQybb1sx6pnJtmV5Ikr7D8vyXlJsmPHjjlLBAAAWEzzzFAdTLJ9ZvvkJDct7VRVT05yQZKzuvuryz1Qd1/c3bu6e9e2bdtG6gUAAFgY8wSqK5KcWlWnVNXxSc5Osne2Q1U9LsnvZxKmbln/MgEAABbPmoGquw8leV6Sy5Jcm+TS7r66ql5WVWdNu706yQlJ3lFVf1NVe1d4OAAAgPuNeT5Dle7el2TfkrYLZ24/eZ3rAgAAWHhzfbEvAAAARxKoAAAABglUAAAAgwQqAACAQQIVAADAIIEKAABgkEAFAAAwSKACAAAYJFABAAAMEqgAAAAGCVQAAACDBCoAAIBBAhUAAMAggQoAAGCQQAUAADBIoAIAABgkUAEAAAwSqAAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADBKoAAAABglUAAAAgwQqAACAQQIVAADAIIEKAABgkEAFAAAwSKACAAAYNFegqqrdVXVdVR2oqvOX2f/tVfX26f4PVdXO9S4UAABg0awZqKrquCQXJTkzyWlJzqmq05Z0e3aSL3b39yR5bZJXrnehAAAAi2aeGarTkxzo7uu7++4klyTZs6TPniRvmt5+Z5InVVWtX5kAAACLZ55AdVKSG2e2D07blu3T3YeSfCnJd61HgQAAAItqyxx9lptp6oE+qarzkpw33byzqq6b4/jHkq1JPr/RRSRJWbS5mSzMuGFTMW4YYdww4pgeN86p5rPM67QI4+ZR83SaJ1AdTLJ9ZvvkJDet0OdgVW1J8uAkX1j6QN19cZKL5ynsWFRV+7t710bXweZi3DDCuGGEccMI44YRm2nczLPk74okp1bVKVV1fJKzk+xd0mdvkmdNbz89yV929xEzVAAAAPcna85QdfehqnpeksuSHJfkDd19dVW9LMn+7t6b5A+TvKWqDmQyM3X20SwaAABgEcyz5C/dvS/JviVtF87cvivJz6xvacckyyEZYdwwwrhhhHHDCOOGEZtm3JSVeQAAAGPm+QwVAAAAyxCoNkhVvaGqbqmqj8+0/UBVXV5Vf1NV+6vq9I2skcVSVdur6r1VdW1VXV1V/2na/tCq+t9V9cnp74dsdK0sjlXGzaur6hNV9dGq+tOqOnGja2VxrDRuZva/sKq6qrZuVI0sntXGTVU9v6qum7a/aiPrZLGs8ndq05wXW/K3Qarqx5PcmeTN3f3Yadt7kry2u99dVU9J8qLuPmMDy2SBVNUjkjyiuz9cVQ9McmWSpyU5N8kXuvsVVXV+kod0969sYKkskFXGzcmZXJH1UNXk2z+MGw5badx09zVVtT3JHyT53iQ/1N0b/T0xLIhV3m8enuSCJD/V3V+tqod19y0bWSuLY5Vx81+zSc6LzVBtkO5+f478rq5O8qDp7QfnyO/74hjW3Td394ent7+c5NokJyXZk+RN025vyuRNCJKsPG66+z3dfWja7fJMAhYkWfX9Jklem+RFmfzNgnusMm6ek+QV3f3V6T5hinusMm42zXnxXFf54z7zC0kuq6rfyiTs/sgG18OCqqqdSR6X5ENJHt7dNyeTN6WqetgGlsYCWzJuZv27JG+/r+thc5gdN1V1VpK/7e6rqmpD62KxLXm/eXWSf15VL09yV5IXdvcVG1cdi2rJuNk058VmqBbLc5K8oLu3J3lBJt/vBd+kqk5I8t+T/EJ337HR9bA5rDRuquqCJIeSvHWjamNxzY6bTMbJBUkuXPVOHPOWeb/ZkuQhSR6f5JeTXFoSOUssM242zXmxQLVYnpXkT6a335FkYT98x8aoqm/L5M3mrd19eKx8brr++PA6ZEsp+CYrjJtU1bOS/HSSZ7QP1LLEMuPm0UlOSXJVVd2QyTLRD1fVd29clSyaFd5vDib5k5746yRfT+KCJtxjhXGzac6LBarFclOSJ05v/2SST25gLSyY6X/z/jDJtd39mpldezN508n095/d17WxuFYaN1W1O8mvJDmru7+yUfWxmJYbN939se5+WHfv7O6dmZwk/2B3f3YDS2WBrPJ36l2ZnNekqh6T5PgkLmZCklXHzaY5L3aVvw1SVW9LckYm/6H5XJKXJLkuyesymRq/K8lzu/vKjaqRxVJVP5bk/yb5WCb/3UuSF2eyzvjSJDuSfCbJz3T30guecIxaZdy8Psm3J7lt2nZ5d/+H+75CFtFK46a79830uSHJLlf547BV3m/+IskbkvxAkrsz+QzVX25IkSycVcbNHdkk58UCFQAAwCBL/gAAAAYJVAAAAIMEKgAAgEECFQAAwCCBCgAAYJBABQAAMEigAgAAGCRQAQAADPr/lmi1DcBwbDcAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.plot.hist(\"height\", 10)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAE2CAYAAABmwtMlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XuYXWV9t/H7S0IERVBIUDkGa7RQi9JGpKJCX0BOQkA5+qJU0FQFrFKroBYVWtvaqkDF+uIBEayAIBglitaqFJWaUIQKiAREEqgQkPMpHH7vH2sN7g4TMiwms3dm7s91zZW91npmz28iruzvek6pKiRJkiRJT95q/S5AkiRJklZVBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiaJJNcnuT/JPT1fG/S7rn5IsspswphkZpLr+12HJGlkBipJmlz2qKq1er5uGt4gydR+FCZJ0qrIQCVJk1zbA1JJDk1yA/Dv7fltkvw4yR1JLkuyfc/3bJbkh0nuTvLdJJ9Kcnp7bfskS4b9jOuT7Ni+Xi3JUUmuTXJbkrOSrDusloOT3JDk1iQf6HmfKUne337v3UkuSbJxkpOSfHzYz/xGkneN4vf/QZLjkvyofc/vJJnec/2rSX6T5M4kFyb5g55rX0zy6STfanv8fpTkuUmOT3J7kl8k2aqn/QZJzkmyNMmvkryz59rWSRYmuSvJzUk+seL/9SRJ/WagkiQN2Q7YHNg5yYbA+cDfAOsC7wHOSTKjbfuvwCXAdOA44OAn8XPeCezV/rwNgNuBk4a1eSXwImAH4Jgkm7fnjwQOBHYD1gYOAe4DTgUOTLIaQBuIdgC+MlIBVZVhp94AvBlYH5jW/r5DvgXMaq/9F/DlYd+7H/BBmr+LB4GftO2mA2cDn2hrWg34BnAZsGFb37uS7Ny+zwnACVW1NvB7wFltrddX1cyRfg9JUv8ZqCRpcjmv7XG6I8l5w659uKrurar7gYOA+VU1v6oerarvAguB3ZJsArwM+OuqerCqLqQJCqP158AHqmpJVT0IfBjYZ9hQw49U1f1VdRlNAHlJe/4twAer6upqXFZVt1XVT4E7aUIKwAHAD6rq5lHWdEpV/bL93c8CXjp0oaq+UFV399T6kiTr9HzvuVV1SVU9AJwLPFBVX6qqR4AzgaEeqpcBM6rq2KpaVlXXAZ9tawV4CHhBkulVdU9VXTzK2iVJfWSgkqTJZa+qelb7tdewa4t7Xm8K7NsTvu6g6TV6Hm2vUlXd29P+10+ihk2Bc3ve9yrgEeA5PW1+0/P6PmCt9vXGwLXLed9TaYIg7Z+nPYmaRvx57RDDv2+HGN4FXN+2md7Tvje03T/C8VDtmwIbDPs7fT+/+70PBV4I/CLJgiSvfRL1S5L6xInHkqQhvSvfLQZOq6q3Dm+UZFPg2Ume0ROqNun5/nuBp/e0nwLM6HmLxcAhVfWjEd575gpqXEwzHO7nI1w7Hfh5kpfQDF0c3gPXxRuAOcCONGFqHZohisOHDI7GYuBXVTVrpItVdQ2/G7b4OuDsJOsNC66SpAFjD5UkaSSnA3sk2bntpVmjXWxio6r6Nc3wv48kmZbklcAePd/7S2CNJLsnWZ1mftHTeq5/BvjbNpiRZEaSOaOs63PAcUlmpbFlkvUAqmoJsICmZ+qcdvjeU/VMmnlRt9GExI8+hff6KXBXkvclWbP9e31xkpcBJDkoyYyqehS4o/2eR55K8ZKklc9AJUl6nKpaTNMz835gKU3vyl/xu3833gC8HPgt8CHgSz3feyfwDprwcyNNj1Xvqn8nAPOA7yS5G7i4fa/R+ATNHKfvAHcBnwfW7Ll+KvCHPLnhfk/kSzTDGW8Ermxr7aSdU7UHzfysXwG30vwdDc3H2gW4Isk9NH9HB7TzsiRJAyxVq8zehpKkAZXkw8ALquqgFbVdyXW8mqZ3bWbb0yNJ0kplD5UkaUJohxf+BfA5w5QkabwYqCRJq7x2n6o7aFYhPL7P5UiSJhGH/EmSJElSR/ZQSZIkSVJHBipJkiRJ6qhvG/tOnz69Zs6c2a8fL0mSJEnLdckll9xaVTNW1K5vgWrmzJksXLiwXz9ekiRJkpYrya9H084hf5IkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSepohYEqyReS3JLk58u5niQnJlmU5PIkfzT2ZUqSJEnS4BlND9UXgV2e4PquwKz2ay7wL0+9LEmSJEkafCsMVFV1IfDbJ2gyB/hSNS4GnpXkeWNVoCRJkiQNqrGYQ7UhsLjneEl7TpIkSZImtLHY2DcjnKsRGyZzaYYFsskmm4zBjx5bM486v98lSBPC9X+/e79LGHPeH6SxMdHuD94bpLGxKt8bxqKHagmwcc/xRsBNIzWsqpOranZVzZ4xY8YY/GhJkiRJ6p+xCFTzgDe1q/1tA9xZVf8zBu8rSZIkSQNthUP+knwF2B6YnmQJ8CFgdYCq+gwwH9gNWATcB7x5ZRUrSZIkSYNkhYGqqg5cwfUCDhuziiRJkiRpFTEWQ/4kSZIkaVIyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6GlWgSrJLkquTLEpy1AjXN0ny/SSXJrk8yW5jX6okSZIkDZYVBqokU4CTgF2BLYADk2wxrNkHgbOqaivgAODTY12oJEmSJA2a0fRQbQ0sqqrrqmoZcAYwZ1ibAtZuX68D3DR2JUqSJEnSYBpNoNoQWNxzvKQ91+vDwEFJlgDzgSNGeqMkc5MsTLJw6dKlHcqVJEmSpMExmkCVEc7VsOMDgS9W1UbAbsBpSR733lV1clXNrqrZM2bMePLVSpIkSdIAGU2gWgJs3HO8EY8f0ncocBZAVf0EWAOYPhYFSpIkSdKgGk2gWgDMSrJZkmk0i07MG9bmBmAHgCSb0wQqx/RJkiRJmtBWGKiq6mHgcOAC4Cqa1fyuSHJskj3bZn8JvDXJZcBXgD+rquHDAiVJkiRpQpk6mkZVNZ9msYnec8f0vL4S2HZsS5MkSZKkwTaqjX0lSZIkSY9noJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqaNRBaokuyS5OsmiJEctp81+Sa5MckWSfx3bMiVJkiRp8ExdUYMkU4CTgJ2AJcCCJPOq6sqeNrOAo4Ftq+r2JOuvrIIlSZIkaVCMpodqa2BRVV1XVcuAM4A5w9q8FTipqm4HqKpbxrZMSZIkSRo8owlUGwKLe46XtOd6vRB4YZIfJbk4yS4jvVGSuUkWJlm4dOnSbhVLkiRJ0oAYTaDKCOdq2PFUYBawPXAg8Lkkz3rcN1WdXFWzq2r2jBkznmytkiRJkjRQRhOolgAb9xxvBNw0QpuvV9VDVfUr4GqagCVJkiRJE9ZoAtUCYFaSzZJMAw4A5g1rcx7wpwBJptMMAbxuLAuVJEmSpEGzwkBVVQ8DhwMXAFcBZ1XVFUmOTbJn2+wC4LYkVwLfB/6qqm5bWUVLkiRJ0iBY4bLpAFU1H5g/7NwxPa8LOLL9kiRJkqRJYVQb+0qSJEmSHs9AJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSR6MKVEl2SXJ1kkVJjnqCdvskqSSzx65ESZIkSRpMKwxUSaYAJwG7AlsABybZYoR2zwTeCfznWBcpSZIkSYNoND1UWwOLquq6qloGnAHMGaHdccDHgAfGsD5JkiRJGlijCVQbAot7jpe05x6TZCtg46r65hjWJkmSJEkDbTSBKiOcq8cuJqsBnwT+coVvlMxNsjDJwqVLl46+SkmSJEkaQKMJVEuAjXuONwJu6jl+JvBi4AdJrge2AeaNtDBFVZ1cVbOravaMGTO6Vy1JkiRJA2A0gWoBMCvJZkmmAQcA84YuVtWdVTW9qmZW1UzgYmDPqlq4UiqWJEmSpAGxwkBVVQ8DhwMXAFcBZ1XVFUmOTbLnyi5QkiRJkgbV1NE0qqr5wPxh545ZTtvtn3pZkiRJkjT4RrWxryRJkiTp8QxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1NKpAlWSXJFcnWZTkqBGuH5nkyiSXJ/lekk3HvlRJkiRJGiwrDFRJpgAnAbsCWwAHJtliWLNLgdlVtSVwNvCxsS5UkiRJkgbNaHqotgYWVdV1VbUMOAOY09ugqr5fVfe1hxcDG41tmZIkSZI0eEYTqDYEFvccL2nPLc+hwLeeSlGSJEmStCqYOoo2GeFcjdgwOQiYDWy3nOtzgbkAm2yyyShLlCRJkqTBNJoeqiXAxj3HGwE3DW+UZEfgA8CeVfXgSG9UVSdX1eyqmj1jxowu9UqSJEnSwBhNoFoAzEqyWZJpwAHAvN4GSbYC/h9NmLpl7MuUJEmSpMGzwkBVVQ8DhwMXAFcBZ1XVFUmOTbJn2+wfgbWAryb5WZJ5y3k7SZIkSZowRjOHiqqaD8wfdu6Yntc7jnFdkiRJkjTwRrWxryRJkiTp8QxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1NKpAlWSXJFcnWZTkqBGuPy3Jme31/0wyc6wLlSRJkqRBs8JAlWQKcBKwK7AFcGCSLYY1OxS4vapeAHwS+IexLlSSJEmSBs1oeqi2BhZV1XVVtQw4A5gzrM0c4NT29dnADkkydmVKkiRJ0uAZTaDaEFjcc7ykPTdim6p6GLgTWG8sCpQkSZKkQTV1FG1G6mmqDm1IMheY2x7ek+TqUfx8qdd04NZ+F6EnFgf9qj+8P6wCvD+oT7w/DLgBvTdsOppGowlUS4CNe443Am5aTpslSaYC6wC/Hf5GVXUycPJoCpNGkmRhVc3udx2SBo/3B0nL4/1BK9NohvwtAGYl2SzJNOAAYN6wNvOAg9vX+wD/XlWP66GSJEmSpIlkhT1UVfVwksOBC4ApwBeq6ookxwILq2oe8HngtCSLaHqmDliZRUuSJEnSIIgdSVqVJJnbDh2VpP/F+4Ok5fH+oJXJQCVJkiRJHY1mDpUkSZIkaQQGKkmSJEnqyEAlSZIkSR0ZqCRJkrTKSpJ+16DJzUClgZBk/X7XIEmSVi1JUlWVZNckc/pdjyYnA5X6LskewLwks5Ks3nPe/z4lSdJytWHqtcA/Asv6XY8mJz+wqq+SbAkcBxxZVddU1UPt+UOAV7Sv/e9UmkSSbNjvGiQNvjSeCbwLOLSqvpXkFUkOTbJJv+vT5OEHVfXbA8Ai4IZh56cBRwNU1aPjXZSkvtoZnBchaYWeVVV3A/8D7J3kHOCtwDuBA/tamSYVA5X6pv2wdCfwC+BrSb6b5ItJjgeeDWyU5J/aIYGSJoEkGwE7tocvTfLcJOv0syZJg6e9V3w9yUtohvstBU6oqjcDRwF/kuTpPpjReEhV9bsGTTJDE0h7jp9L8wFqJnAJsDbwEPC29vjUqvpFH0qV1CdJ9qN5yvwD4I+B/avqrr4WJamvej8/JFkLeDMwBzi6qha053cATgD+qqq+1bdiNanYQ6VxNexm+Pz2ydHtwBLgT4EfVtWZbfNlwEcMU9LElmTasOM/Bd4O7EbzgOU+4G6fNEuTU5Ip8NgCFH/Yvr4HOAU4E/hYkm3aha3eBLzXMKXxZA+V+iLJEcCuwFU0Y59PAv4JeCVwHrA3cGBVXdG3IiWtdEnWA15cVT9MslpVPZrkT4CX0Dz02wvYo6oeTLITzUMXV/KSJol2W5X9gPlVdV2S7wO3VNX+7fW1gY8CL6fpsbq2qu7vW8GalOyh0rhrlzfdp/36Q2AWzeIU7wFOpJlTtY9hSpoUZgOvbl8P9VT9Gng/cFhVvaYNU4cChwJr9KFGSf2zOk1Y2i/JGsBOwLpJTgFohwJfSvPZYS3DlPrBHiqtdEleDaxbVee1xwfQfHCaCuwP7Nl+YHpRVV3dx1IljbP26fP9wO7Aa4FrgAXt5Q8AZ9B8oHoD8Oaq+u9+1Clp/PX0Wm8CvAP4dFXdkGQq8G3gJuBi4BCaZdMv62O5msQMVFrp2p3LPw+8qarmJ3k5cBpwa1UN7TV1BPAimv2oHM4jTSJJ3kwzVOfdNKt1XUKzStfWNPMhbgW+UlVX9q1ISeNqaM51kqdX1X0955/WPoSdSnOfWBu4qKrm9a1YTXoGKq00Q0+W2tdH06za9y7g68Df0fRSXQisAxwBHFxVP+9TuZL6oN24+y+As4HXAAfQzK8EWKOdeC5pEkqyO3AkcD5wWVV9rz2/elU9NKzt/1pBWBpPzqHSyjS0mt9hwO8BFwFfptm08wTgv2iePr8Cw5Q04aXVe65n4+6fAvtW1U5V9TDNfKk3tk+hJU0ySTam2aT3DGBNYN92DjZV9VD7MOYxhin1k/9QacwleUFVLWq76l8AzAVeV1XXtjfDLwFvqarTknwFoP0AJWliW7+qbgZIshcwA1gIfAd4Ic3y6CQ5GDgc2M97gzT5tEujbwUsrKrPtnOodgV2b3unzu15GCP1nT1UGlNJngUc0v4JcC1wObBaexP8Js0S6ecm2amqHvYDkzTxtRt4n5JklyR7Ax+kWd3vCOAtwHzgGUkuAP6MZtuEq/pVr6TxNdTjlGR74Bs0i9S8O8mGVXUDzbC/q4A92sVspIHhHCqNqXbzvak0y6HvU1VHJfkccDfNTuYPJNmTZsn0j1TVtX0sV9I4afeK2RvYA3g2zf3h9iTb0Kzwd1VV/WuSdYCHeiehS5q4kqxTVXe2rzcH/gH4UFVdmuREmsVpXl9VN7bDANMGLGlg2EOlMTE0L6KqHgGWAU8HtkjyDuDtwPOBzyU5Ffgw8EHDlDTx9dwb7qJZeOIc4I9plkGnqi4G7gBe1R7faZiSJockTwc+k+R57altgM1p5lpTVe8EfgJ8p+2pWmyY0iByDpWest6VdZJsANxbVRcmeRB4X9tsL5rFJ54P/I03RGniG3Zv2Bi4l2aC+TTggCS3VtWZNMuib5tkTeABJ5dLk0NV3dcuXLVukv9bVackeQTYLskbq+q0qnp3kk8BM4Eb+1qwtBwGKj0lSbZqX16a5EjgIODGJN+uqpOS/APwl8Bzq+oY4Ef9qlXS+OoJU++l6YFaE5jXfj0CHJ9k/7b5X1fV/X0pVNK4SrJmz//fpwHPAf4myUNV9aV2PtWrkkyrqs9X1eH9q1ZaMQOVnqrXA7OTfJamq35/YDrN8L7Vq+r4dgz0oUlmVNXSfhYraeUavhdMkt2AHapq5yTnANtU1YlJfgM8g2ZhiqPttZYmh3YY8A5J/ohmL8pD+N3iNJ9o7yFfTLI68Mr2Aa09UxpoLkqhToYN5TkG2Am4tB3vTJItgdOBM6vqb5OsUVUP9K9iSeMhyfpVdcvQxt5J5tA8fV4X2B6YU1UPJvk9YDHwtKq6u48lSxpnbahaAMwCdqqqn7bnd6RZlOKTVXV6O2/KMKWB56IUetKGP4GuqmOBrwEvTfKSJFOq6nKapY/3TLKuYUqa+NoJ5ucmmQW8rD19F3AAsB2wZxum3g18AljNMCVNWpe2X4cMnaiqfwOOAt6bZAPDlFYV9lDpSRnWM/VGmifP32uXN30fsC3wIeDyqnokydOq6sE+lixpHLTzKacAAT4GbAi8tL38j8DDNHvSPQ14G80+U1f0oVRJfTD0+SHJi2jmUN4MPAicSrNVwpuSvABYH7hiaCl1aVVgoFInSV5HszHnz9pTF1XVF5K8h2afmSPaXipJk0CSV9Mshz4dOBiYW1Xz26E9a9L0WG9KM2/q01V1Zb9qldQfSXYGPkmzN+VVNBt6/xj4F2AtmqHBh1fVf/StSKkDF6XQk5bk9cA7gB2r6rdJDqRZ8riq6p+SLANu72+VksbZQuAwmq0RjgVek+Re4P/QbJXwaYB2sZqH+lempH5I8mLgXTRDfxclmUuzae9VwBya+8elVXVRH8uUOnEOlVZoaGPOHo/SzIfYrz3+KnARzb4RB1XViVW1eDxrlDT+2jlTQ3anCU/3V9XJwOrAN4GFVfVQkiltu4fHuUxJfZDk+UkOS3JwO0VgO5pe7KFNfL8CrAccUFWPVtU/G6a0qrKHSk9o2JypjWg+LJ2bZG/go0l+W1VntcshP0wTrCRNcO08iH2TnF1Vv6CZD7ExsHOSC4G1gTPbtqmqR+B3e1NJmriSbEGz39z5NPOk9gV+CHwGOCjJ3VX1syTn09wz1gAe9P6gVZWBSk+oJ0y9B3gNsF6S04HvAUcCf99uvHc6cHb/KpU0zp4DPBfYO8mZVXUhQJIlwEzg8zQfoF5M82+Nw/ykSSDJOjTB6W+r6pT23PHAF2hGRv0U+EKSs4A9gY+6ErBWdS5KoREN65naFjgeeBXNKn7b0Kzm9XGaYT5/AewC3OPTJWliG3ZveBWwN/Bbmgcq1wBfB34O/AY4jWZpdDf0liaJtrfps8DbqureoX0ok2wI/AfwZZohwc8HvlpVXx3at66PZUtPiXOo9DjDPjBNA5YBt1XVA1X1PeDbwK7AH1TVWcDOVXW3YUqa2EbYg+4/aJ46Twf2AZ5dVa+tqqNoNuyMYUqadNakmSu1PUAbpqa1e0r9C3Ab8EWaVYJ3TfL7himt6gxUekwaq/WEqbfRPHW+Ebg9yT4AVXUJcBnNfAmq6p4+lSxpnCSZ1XNvOCLJ55N8CriPpgd7fWBuki3bbzm8qm7tU7mS+qSqbgdOAF6fZGgvukfaP6cCa7XzLr8D/AK4Y/yrlMaWgUq9pgw9JUqyG7Aj8Naquommm37bJCcneSuwE3BJ/0qVNB7aBy3TgK8l+WiSbYC9gPOAW4CLaTbrPZFm7tRObXtJk9fXgP8B/jzJDlX1SJJXAHOBnwBU1ULg+Kr6TR/rlMaEc6gEQJIZNJPI59DMj/oA8D6avaZ+nGRt4PdpPkgFOM2NOaXJo13l82s0K3adXFWnteffAbwJGNrY99qquqVvhUoaCEmeA+xPs7/UAmBz4Niq+rpzpjTRGKj0mHZPmW2BBVV1R5K/A/4A+GBVXd7TbsrQEsiSJq7hc6aSPA84B7ixqvbtOf9l4B1VdWcfypQ0wJI8t325RlVdP/y+Ik0EDvnTY6rqPuAZwM+TrFNVR9MM5zmmZxw0hilpcuiZM7V7klfT7DW1B/D8JB9PsmWS/YDZwBp9LFXSgKqq37Rf17fHhilNOPZQ6XGS7AL8M82HpLuA42jmRhxSVcv6WJqkcTBspc+DgaOB64H/Bj4HLAXOpdmH6hTgvHaSuSRJk46BSiNqF6X4OPAn7fC/9arqtn7XJWnlGham1gHeQ7Ni12rAnwPPotlj5mbgDOAtVbW4T+VKktR3BiotV5K9gL8GZttFL018w8LUe2gWmXg5sH9VLUjyQppJ5hsBnwCucWK5JGmycw6VlquqzgO2M0xJk0NPmHoNzaacxwHfolkyfYOq+iXN3nTXAXcYpiRJajZYk5bLTXuliW9Yz9RWNMscX99ujXBYknuBi5K8uqquSrKoqh7qZ82SJA0Ke6gkaZLrCVOzgMuAHwEbJnlte/29wLeBbyeZAjzcr1olSRo0zqGSJJFkY5o9pv65qk5L8l7gecC/VdX5bZv13bRXkqT/zR4qSZqEkmTYqZuBY4A3JjmQZpXPG4G9kuzctlk6jiVKkrRKcA6VJE1CPcP8XgdcVFW3JPl34CHgA8B9NMulvx34We/3SJKk37GHSpImkSSr9f4J7AXMTzKj3bj7x8DFNMui71JVJ1bVzf2pVpKkwWegkqRJpGep85nt8ZsqPC2XAAABN0lEQVSAC4B57Ryp+2mWRT8T+O++FClJ0irEIX+SNAkkeQWwSVWdkeQw4JAkvwSuoZk79ShwdpLLafag2q2qbuhbwZIkrSIMVJI0OTwb+Lskm9P0Tu3b/rkj8KmqenuSPYB1gZMMU5IkjY6BSpImgao6P8ky4JPAZVV1XZIlwGLgw0m2rKpv9LdKSZJWPc6hkqRJoqq+S7OC325J9q+qZVV1DbAG8Pv9rU6SpFWTPVSSNIlU1deTvBE4MckWwE+BzYBL+luZJEmrJgOVJE0yVfXNJFOBc4CvAq+rquv7W5UkSaumuE+jJE1OSbYDrq+qX/e7FkmSVlUGKkmSJEnqyEUpJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnq6P8D4PX/UdvK5aEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAEUCAYAAAAspncYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAF41JREFUeJzt3X20ZWV9H/DvD0ZEQcU4Y6LMwGAEKzXxZU2Ib1Wy1Mqggm2tC5SqiXFMItVUa0OiomLSptpItSEafAmoVSSxMWMyLrT1bYWKMoRCBUsc8WUmYBx59RWC/vrHOZjj5c7MYXPnngP381nrrnv23s/Z+3fPgofz5Xn2s6u7AwAAwO23z6wLAAAAuLMSqAAAAAYSqAAAAAYSqAAAAAYSqAAAAAYSqAAAAAYSqABY8arqU1X1qwPf+9Gqev7evg4A80mgAlhBquqrVfX9qvrOxM8DZ13XLFTVkjyIsbs3dvfZS1DP+qrqqlo1se8FVXXWHT03AHuPQAWw8jyjuw+c+LlqYYPJL/UAwK4JVABMjo68sKq+nuQT4/2Prqr/XVXXV9UlVXX0xHsOq6pPV9W3q+rjVfWHVfW+8bGjq2rHgmt8taqePH69T1WdUlVfrqprqurcqvqpBbU8v6q+XlXfqqpXTZxn36r6nfF7v11VF1XVuqo6o6r+YME1P1JVvznlx3BoVZ0/PufHqmr1xHl29zn8eBrfuLY/GNf8lao6eeGo026u85nx7+vHI4ePmbJuAGZIoAJg0hOTPDTJU6vq4CR/leR3k/xUkn+f5ENVtWbc9v1JLkqyOskbkkx1H9HYS5M8c3y9Bya5LskZC9o8PslDkjwpyalV9dDx/pcnOTHJsUnuneRXknwvydlJTqyqfZJkHFSelOQDixXQ3bVg13OS/HKS+yfZb/z3ZorPYdKLkmxM8ogkjxr/jQstep0kTxj/Pmg8cvjZ7j6ru1+wWP0AzAeBCmDl+fB4pOX6qvrwgmOv6+7vdvf3k5yUZEt3b+nuH3X3x5NsTXJsVR2S5BeSvKa7b+ruzyT5yO2o4cVJXtXdO7r7piSvS/KsBSM5r+/u73f3JUkuSfLw8f5fTfLq7r6iRy7p7mu6+/NJbsgoRCXJCUk+1d1/P2VNf9Ldfzv+28/NKBRld5/DIud4dpK3jP+u65L8/u24DgB3QgIVwMrzzO4+aPyzcARl+8TrQ5P864nwdX1Go0YPyHhUqbu/O9H+a7ejhkOT/PnEeb+Y5IdJfnqizTcmXn8vyYHj1+uSfHkX5z07owCU8e/33o6adnW93X0OCz0wP/kZbl+kza6uA8CdkJuOAZg0ufLd9iTv7e4XLWxUVYcmuW9VHTARqg6ZeP93k9xzov2+SSanyG1P8ivdff4i516/hxq3J/nZJF9Y5Nj7knyhqh6e0dTFhSNwQ+zyc1jE1UnWTmyvux3XWZJVBwFYXkaoANiV9yV5RlU9dbzYwv7jxSbWdvfXMpr29vqq2q+qHp/kGRPv/dsk+1fV06rqbkleneTuE8ffnuT3xsEsVbWmqo6fsq53JnlDVR1eIz9fVfdLku7ekeTCjEamPjSeVndH7fJzWKTtuUleVlUHV9VBSX7rdlxnZ5IfJXnQEtQMwDIRqABYVHdvT3J8kt/J6Mv+9iSvzD/+t+M5SX4xybVJXpvkPRPvvSHJb2QUfv4uoxGryVX/3pJkc5KPVdW3k1wwPtc03pxRcPlYkhuTvCvJPSaOn53k53L7pvvt0hSfw6R3jOu6NMnFSbYkuSWj6Yx7us73kvxekvPHUwsfvRT1A7B3VbcZBgDccVX1uiQP7u6T9tR2L9fxhIxGldZ3949mXMvGJG/v7kNnWQcAe48RKgDuMsbTC1+W5J2zCFNVdY+qOraqVo2XW39tkj9f7joAWD4CFQB3CePnVF2f0ep7/3VWZSR5fUbP1bo4o9ULT51RLQAsA1P+AAAABjJCBQAAMJBABQAAMNDMHuy7evXqXr9+/awuDwAAsEsXXXTRt7p7zZ7azSxQrV+/Plu3bp3V5QEAAHapqr42TTtT/gAAAAYSqAAAAAYSqAAAAAYSqAAAAAbaY6CqqndX1Ter6gu7OF5V9daq2lZVl1bVo5a+TAAAgPkzzQjVWUmO2c3xjUkOH/9sSvK2O14WAADA/NtjoOruzyS5djdNjk/ynh65IMlBVfWApSoQAABgXi3FPVQHJ9k+sb1jvA8AAOAubSke7FuL7OtFG1ZtymhaYA455JAluPTSWn/KX826BLhL+OrvP23WJSw5/QMsjbta/6BvgKVxZ+4blmKEakeSdRPba5NctVjD7j6zuzd094Y1a9YswaUBAABmZykC1eYkzxuv9vfoJDd099VLcF4AAIC5tscpf1X1gSRHJ1ldVTuSvDbJ3ZKku9+eZEuSY5NsS/K9JL+8t4oFAACYJ3sMVN194h6Od5KXLFlFAAAAdxJLMeUPAABgRRKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABhKoAAAABpoqUFXVMVV1RVVtq6pTFjl+SFV9sqourqpLq+rYpS8VAABgvuwxUFXVvknOSLIxyZFJTqyqIxc0e3WSc7v7kUlOSPJHS10oAADAvJlmhOqoJNu6+8ruvjnJOUmOX9Cmk9x7/Po+Sa5auhIBAADm0zSB6uAk2ye2d4z3TXpdkpOqakeSLUn+7WInqqpNVbW1qrbu3LlzQLkAAADzY5pAVYvs6wXbJyY5q7vXJjk2yXur6jbn7u4zu3tDd29Ys2bN7a8WAABgjkwTqHYkWTexvTa3ndL3wiTnJkl3fzbJ/klWL0WBAAAA82qaQHVhksOr6rCq2i+jRSc2L2jz9SRPSpKqemhGgcqcPgAA4C5tj4Gqu29JcnKS85J8MaPV/C6rqtOq6rhxs1ckeVFVXZLkA0le0N0LpwUCAADcpayaplF3b8losYnJfadOvL48yeOWtjQAAID5NtWDfQEAALgtgQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGAggQoAAGCgqQJVVR1TVVdU1baqOmUXbZ5dVZdX1WVV9f6lLRMAAGD+rNpTg6raN8kZSZ6SZEeSC6tqc3dfPtHm8CS/neRx3X1dVd1/bxUMAAAwL6YZoToqybbuvrK7b05yTpLjF7R5UZIzuvu6JOnuby5tmQAAAPNnmkB1cJLtE9s7xvsmHZHkiKo6v6ouqKpjFjtRVW2qqq1VtXXnzp3DKgYAAJgT0wSqWmRfL9heleTwJEcnOTHJO6vqoNu8qfvM7t7Q3RvWrFlze2sFAACYK9MEqh1J1k1sr01y1SJt/qK7/6G7v5LkiowCFgAAwF3WNIHqwiSHV9VhVbVfkhOSbF7Q5sNJfilJqmp1RlMAr1zKQgEAAObNHgNVd9+S5OQk5yX5YpJzu/uyqjqtqo4bNzsvyTVVdXmSTyZ5ZXdfs7eKBgAAmAd7XDY9Sbp7S5ItC/adOvG6k7x8/AMAALAiTPVgXwAAAG5LoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhIoAIAABhoqkBVVcdU1RVVta2qTtlNu2dVVVfVhqUrEQAAYD7tMVBV1b5JzkiyMcmRSU6sqiMXaXevJC9N8rmlLhIAAGAeTTNCdVSSbd19ZXffnOScJMcv0u4NSd6Y5AdLWB8AAMDcmiZQHZxk+8T2jvG+H6uqRyZZ191/uYS1AQAAzLVpAlUtsq9/fLBqnySnJ3nFHk9UtamqtlbV1p07d05fJQAAwByaJlDtSLJuYnttkqsmtu+V5GFJPlVVX03y6CSbF1uYorvP7O4N3b1hzZo1w6sGAACYA9MEqguTHF5Vh1XVfklOSLL51oPdfUN3r+7u9d29PskFSY7r7q17pWIAAIA5scdA1d23JDk5yXlJvpjk3O6+rKpOq6rj9naBAAAA82rVNI26e0uSLQv2nbqLtkff8bIAAADm31QP9gUAAOC2BCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBBCoAAICBpgpUVXVMVV1RVduq6pRFjr+8qi6vqkur6n9V1aFLXyoAAMB82WOgqqp9k5yRZGOSI5OcWFVHLmh2cZIN3f3zSf4syRuXulAAAIB5M80I1VFJtnX3ld19c5Jzkhw/2aC7P9nd3xtvXpBk7dKWCQAAMH+mCVQHJ9k+sb1jvG9XXpjko3ekKAAAgDuDVVO0qUX29aINq05KsiHJE3dxfFOSTUlyyCGHTFkiAADAfJpmhGpHknUT22uTXLWwUVU9OcmrkhzX3TctdqLuPrO7N3T3hjVr1gypFwAAYG5ME6guTHJ4VR1WVfslOSHJ5skGVfXIJH+cUZj65tKXCQAAMH/2GKi6+5YkJyc5L8kXk5zb3ZdV1WlVddy42ZuSHJjkT6vq/1TV5l2cDgAA4C5jmnuo0t1bkmxZsO/UiddPXuK6AAAA5t5UD/YFAADgtgQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgQQqAACAgaYKVFV1TFVdUVXbquqURY7fvao+OD7+uapav9SFAgAAzJs9Bqqq2jfJGUk2JjkyyYlVdeSCZi9Mcl13PzjJ6Un+81IXCgAAMG+mGaE6Ksm27r6yu29Ock6S4xe0OT7J2ePXf5bkSVVVS1cmAADA/JkmUB2cZPvE9o7xvkXbdPctSW5Icr+lKBAAAGBerZqizWIjTT2gTapqU5JN483vVNUVU1wfJq1O8q1ZF8HulUm/zIb+4U5A/8CM6B/m3Jz2DYdO02iaQLUjybqJ7bVJrtpFmx1VtSrJfZJcu/BE3X1mkjOnKQwWU1Vbu3vDrOsA5o/+AdgV/QN70zRT/i5McnhVHVZV+yU5IcnmBW02J3n++PWzknyiu28zQgUAAHBXsscRqu6+papOTnJekn2TvLu7L6uq05Js7e7NSd6V5L1VtS2jkakT9mbRAAAA86AMJHFnUlWbxlNHAX6C/gHYFf0De5NABQAAMNA091ABAACwCIEKAABgIIGKuVNViz3XDED/AMDcEaiYG1V1tySx5D6wUFUdmOgfAJg/FqVgLlTVcUk2Jrlnkjcmubq7b/NwaGDlGfcPz03SSd6T5Avd/fXZVgXMg6o6Nsm+3f2RWdfCymWEipmrqocleVuSP03y90lekuR5VXXwTAsDZq6qjsiof3hrks8meVySV1TVQ2ZaGDBzVfWUJG9K8t1Z18LKJlAxD+6f5DPd/Ynu/g9JtiRZm+RfVdVBsy0NmLG7J/nr7j6/u9+S5ENJdiZ5cVWtnW1pwKxU1dFJ3pHk+d39iao6sKruV1W+27Ls/EPHPLgoybqq+hdJ0t1/meQzSR6WZPUsCwNm7v8leVBV/XqSdPffJPlokluSHJFYqAJWqHsmuVeS66rqvknOSfLfk/y3qjp+ppWx4ghUzFRV7dPdNyQ5O8ljq+qJSdLdm5P8IMlvzrI+YHaqat/u/ockr07yi1X17CTp7ouSXJvkpPG2m4FhhenuLUl+I8mnM5oO/BdJXpTkK0mOqaoDZlgeK8yqWRfAyjUOUz8ab/7PjKb+Pb2qfrq7z01ySZKHLmgH3IVV1aruviVJuvuH492fz6h/eFpV/Ux3vzXJ3yX5J1V19+6+aUblAsusqvZN8qMe+WBV/SDJEd39jvHxP07y4YxmuLi3imVhlT+WVVU9NEkluba7vzEZlqrqsCRPTPLrSXYkOSrJ07r70pkVDCybqvqlJI9O8uaFIWk8peeRGd2A/pUkv5DkuO6+ZNkLBZZdVR3e3V+a2K5bR6cXvP6XSV6RUf9wzWyqZaURqFg2VfXUjFbq+nSSn8uos9t56w2kE8HqwIzun/p6d181q3qB5VNVG5P8UZIXd/fHJvYv7B9WJVmf5Mbu/uYMSgWW2Xg1v/cn+a3ufvdu2v1aRtMAn9PdX1iu+sA9VCyL8RLHb07ya929KcnnknRVHTD+orTPuN3a7v5Od18gTMHKUFX7JTk2yUu6+2NVdVBV3b+qVo/7h1v/z/ODuvuW7t4mTMHKUFXHZDQy/dEkPzPet6uFaL6f5ERhiuUmULFcbkry6e7+ZFWtz+hm8jcmOb+qHtbdt4xD12uq6gCrdsHK0d03J7k5yerxUugfz+gL1MVV9bju7nG/cXpV3Vv/ACvDeGn0/5TkhRlN43tpVT1l4UI0VfWUqtq/u8/u7stmUCornCl/7FVV9eAk907yzST/I6Ml0p+Z5PQk/yXJy5K8NMmGjFb1u1t3Xz+baoHlVFU/m+R+SS5NcmKSQzLqB27s7rdV1aYkr0/yqO6+uqru1d3fnl3FwHKqqmOTXNPdnxtvn5zkUUn+3XiF4FtHq16T5D3d/dVZ1crKZpU/9pqqenqS/5jk+iR/k+SUJF/KaLTqreOpPKdX1SOS7O/mUVg5JvqHa5N8OckHMwpVqzNaJj3dfWZVPSbJfZJcLUzBylBVD+nuK8ZLo0+uCvz5JP88yUFJbpjYf9oMywVT/tg7quqxGY1APb+7n5DkHkme293bM5oD/bJxu+cmeURGD+kEVoAF/cPRGf37/+Qkz05yXZInVNXjq+rfJHlMkhtmVSuwvMb/s+XiqvrAxO5bF6f5fJKdSf5wvO2RKswFU/7YK8ZfmI7o7rPG22uSvKu7j6uqI5J8Ksknkzw8ybO7+/JZ1Qosr130D3/S3U+vqoOTPC/JARlN7XmleyJgZRg/jPdDGd0i8Ngkq7r7pPGxu3f3TVW1OsmZGT1e4a9nVy38I4GKvWL84L0DuvvG8esHJPlIRs+Vuqqq/mlGN6HfYLUuWFl20z9sHD+f7oHjfuKA7vZgTlhBquqBSW5Msn+Styf5wa2hanz8nklem+T07v7GbKqEn2TKH3tFd/+wu28cb1ZG91FdO/6S9LyMnhOxQ5iClWc3/cM3quqkJK+tqnsIU7DydPdV48enfCvJi5Pco6relyRVtSGjha5+W5hinhihYtlU1VlJrs7ohtIXdPf/nW1FwLzQPwCLGU/xe1NG91OuSnJ0d++YbVXwk6zyx143XtL0bkn+2fj3k7r7S7OtCpgH+gdgd7r7W1V1aZKNSZ4iTDGPjFCxbKrqBUkudIM5sJD+AVhMVd03yblJXtHdl866HliMQMWyqapa+HRzgET/AOxaVe3f3T+YdR2wKwIVAADAQFb5AwAAGEigAgAAGEigAgAAGEigAgAAGEigAgAAGEigAgAAGOj/A0SqalfpVPQZAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAEoCAYAAABfHrDOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHlpJREFUeJzt3XmYXHWd7/H3h4RFREBMFGULCKJcFZeIOjKIwsjihaiDQryMykWQuSCKooOAiCgoqIN4iQvjgqKCqCMEDSCOouM4KlFBBQYnskiISJBdGNbv/HFOO2XToSuHTleRfr+ep59UnfPrqm938pzU5/y2VBWSJEmSpOW3yqALkCRJkqRHKgOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJ0pSS5AlJfpDk9iQfmeT3viPJZpP5npKkFctAJUkrqSRXJ7mr/RA/8vWkQdc1CEl6N13cH7gRWLuq3r4C3/PCJG/sPVZVa1XVlRP02ts/3NeRJD18BipJWrnt1n6IH/laMrpBkumDKGyANgEuK3e2lyRNAAOVJE0xSWYlqST7Jvkd8N32+AuS/CjJLUku6e0BSbJpku+3w+QuSHJyki+257ZPsnjUe1ydZMf28SpJDkvy2yR/THJmkvVG1fL6JL9LcmOSI3peZ1qSw9vvvT3Jz5JslGTe6OF6Sc5J8tZxfvZTgdcD72x77HZMcmqS9/e0+Yufp/1ZDk3yyyS3JvlKkjV6zs9JcnGS29o6d05yLPDXwMnt+5zctq0km7eP10nyhSRLk1yT5Mgkq7Tn3pDkh0k+nOTmJFcl2WXcv1xJ0qQzUEnS1PVi4GnATkk2AL4FvB9YDzgU+HqSmW3bLwM/A2YA76MJJf06GHhF+35PAm4G5o1qsy2wJbADcFSSp7XH3wbMBXYF1gb+L3An8Hlgbk8AmdF+7+ljFVBVaf98A/Al4IS2x+47ff4MrwF2BjYFngm8oX3fbYAvAO8A1gW2A66uqiOAfwUOat/noDFe8/8D6wCb0fxuXgfs03P++cAVNL/zE4DPJBn5Obavqgv7rF2StAIZqCRp5XZW2+N0S5KzRp07uqr+VFV3AXsDC6pqQVU9UFUXAAuBXZNsDDwPeHdV3V1VPwDOWY4a3gQcUVWLq+pu4Ghgj1FDDd9bVXdV1SXAJcDW7fE3AkdW1RXVuKSq/lhVPwVupQlRAHsBF1bVH5ajruXxsapaUlU30fzsz2qP7wt8tqouaH9v11XVf4z3YkmmAXsC76qq26vqauAjwN/1NLumqv6pqu6nCZBPBJ4wgT+TJGkCGKgkaeX2iqpat/16xahz1/Y83gR4dU/4uoWm1+iJtL1KVfWnnvbXLEcNmwDf6Hndy4H7+ctwcH3P4zuBtdrHGwG/Xcbrfp4mCNL+edpy1LS8utT3UGYAq/GXv8drgA3Ges+qurN9uBaSpKFioJKkqat3UYZrgdN6wte6VfXoqvog8HvgsUke3dN+457HfwLWHHnS9r7M7Dl/LbDLqNdeo6qu66PGa4EnL+PcF4E5SbamGbo4ugeuX39RP7D+cnzvQ9X3UIte3AjcSxM2R2wM9PM7kSQNEQOVJAmacLJbkp3ahSDWaBdn2LCqrqEZ/vfeJKsl2RbYred7fwOskeTlSVYFjgRW7zn/SeDYJJsAJJmZZE6fdX0aeF+SLdJ4ZpLHAVTVYuAimp6pr7dDF7u4mGZo43pJ1gcecmGLUT4D7JNkh3bxjQ2SPLU99wea+VEP0g7jO5Pm9/KY9nfzNpq/B0nSI4iBSpJEVV0LzAEOB5bS9Ly8g//5f+K1NIsk3AS8h2YhhpHvvRX4fzTh5zqaHp/eVf9OAuYD305yO/Dj9rX68Y80wePbwG00AeZRPec/DzyDhzfc7zSaeVtXt+/zlX6/sZ3LtQ9wIs2cru/zP71OJ9HMFbs5ycfG+PY30/yurgR+SLPwx2e7/QiSpEGJ23BIkpZXkqOBzatq7/HaruA6tqPp1ZlVVQ8MshZJ0tRkD5Uk6RGpHV74FuDThilJ0qAYqCRJjzjtPlW30KxC+NEBlyNJmsIc8idJkiRJHdlDJUmSJEkdTR+/yYoxY8aMmjVr1qDeXpIkSZKW6Wc/+9mNVTVzvHYDC1SzZs1i4cKFg3p7SZIkSVqmJNf0084hf5IkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHY0bqJJ8NskNSX69jPNJ8rEki5L8MslzJr5MSZIkSRo+/fRQnQrs/BDndwG2aL/2Bz7x8MuSJEmSpOE3bqCqqh8ANz1EkznAF6rxY2DdJE+cqAIlSZIkaVhNxByqDYBre54vbo9JkiRJ0kpt+gS8RsY4VmM2TPanGRbIxhtvPAFvPbFmHfatQZcgrRSu/uDLB13ChPP6IE2Mle364LVBmhiP5GvDRPRQLQY26nm+IbBkrIZVdUpVza6q2TNnzpyAt5YkSZKkwZmIQDUfeF272t8LgFur6vcT8LqSJEmSNNTGHfKX5HRge2BGksXAe4BVAarqk8ACYFdgEXAnsM+KKlaSJEmShsm4gaqq5o5zvoADJ6wiSZIkSXqEmIghf5IkSZI0JRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR31FaiS7JzkiiSLkhw2xvmNk3wvyS+S/DLJrhNfqiRJkiQNl3EDVZJpwDxgF2ArYG6SrUY1OxI4s6qeDewFfHyiC5UkSZKkYdNPD9U2wKKqurKq7gHOAOaMalPA2u3jdYAlE1eiJEmSJA2nfgLVBsC1Pc8Xt8d6HQ3snWQxsAB481gvlGT/JAuTLFy6dGmHciVJkiRpePQTqDLGsRr1fC5walVtCOwKnJbkQa9dVadU1eyqmj1z5szlr1aSJEmShkg/gWoxsFHP8w158JC+fYEzAarq34E1gBkTUaAkSZIkDat+AtVFwBZJNk2yGs2iE/NHtfkdsANAkqfRBCrH9EmSJElaqY0bqKrqPuAg4HzgcprV/C5NckyS3dtmbwf2S3IJcDrwhqoaPSxQkiRJklYq0/tpVFULaBab6D12VM/jy4AXTWxpkiRJkjTc+trYV5IkSZL0YAYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI66itQJdk5yRVJFiU5bBltXpPksiSXJvnyxJYpSZIkScNn+ngNkkwD5gF/AywGLkoyv6ou62mzBfAu4EVVdXOSx6+ogiVJkiRpWPTTQ7UNsKiqrqyqe4AzgDmj2uwHzKuqmwGq6oaJLVOSJEmShk8/gWoD4Nqe54vbY72eAjwlyb8l+XGSncd6oST7J1mYZOHSpUu7VSxJkiRJQ6KfQJUxjtWo59OBLYDtgbnAp5Os+6BvqjqlqmZX1eyZM2cub62SJEmSNFT6CVSLgY16nm8ILBmjzdlVdW9VXQVcQROwJEmSJGml1U+gugjYIsmmSVYD9gLmj2pzFvASgCQzaIYAXjmRhUqSJEnSsBk3UFXVfcBBwPnA5cCZVXVpkmOS7N42Ox/4Y5LLgO8B76iqP66ooiVJkiRpGIy7bDpAVS0AFow6dlTP4wLe1n5JkiRJ0pTQ18a+kiRJkqQHM1BJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktRRX4Eqyc5JrkiyKMlhD9FujySVZPbElShJkiRJw2ncQJVkGjAP2AXYCpibZKsx2j0GOBj4yUQXKUmSJEnDqJ8eqm2ARVV1ZVXdA5wBzBmj3fuAE4D/msD6JEmSJGlo9ROoNgCu7Xm+uD32Z0meDWxUVd98qBdKsn+ShUkWLl26dLmLlSRJkqRh0k+gyhjH6s8nk1WAE4G3j/dCVXVKVc2uqtkzZ87sv0pJkiRJGkL9BKrFwEY9zzcElvQ8fwzwdODCJFcDLwDmuzCFJEmSpJVdP4HqImCLJJsmWQ3YC5g/crKqbq2qGVU1q6pmAT8Gdq+qhSukYkmSJEkaEuMGqqq6DzgIOB+4HDizqi5NckyS3Vd0gZIkSZI0rKb306iqFgALRh07ahltt3/4ZUmSJEnS8OtrY19JkiRJ0oMZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6qivQJVk5yRXJFmU5LAxzr8tyWVJfpnkX5JsMvGlSpIkSdJwGTdQJZkGzAN2AbYC5ibZalSzXwCzq+qZwNeAEya6UEmSJEkaNv30UG0DLKqqK6vqHuAMYE5vg6r6XlXd2T79MbDhxJYpSZIkScOnn0C1AXBtz/PF7bFl2Rc4d6wTSfZPsjDJwqVLl/ZfpSRJkiQNoX4CVcY4VmM2TPYGZgMfGut8VZ1SVbOravbMmTP7r1KSJEmShtD0PtosBjbqeb4hsGR0oyQ7AkcAL66quyemPEmSJEkaXv30UF0EbJFk0ySrAXsB83sbJHk28Clg96q6YeLLlCRJkqThM26gqqr7gIOA84HLgTOr6tIkxyTZvW32IWAt4KtJLk4yfxkvJ0mSJEkrjX6G/FFVC4AFo44d1fN4xwmuS5IkSZKGXl8b+0qSJEmSHsxAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSRwYqSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSODFSSJEmS1JGBSpIkSZI6MlBJkiRJUkcGKkmSJEnqyEAlSZIkSR0ZqCRJkiSpIwOVJEmSJHVkoJIkSZKkjgxUkiRJktSRgUqSJEmSOjJQSZIkSVJHBipJkiRJ6shAJUmSJEkdGagkSZIkqSMDlSRJkiR1ZKCSJEmSpI4MVJIkSZLUkYFKkiRJkjoyUEmSJElSR30FqiQ7J7kiyaIkh41xfvUkX2nP/yTJrIkuVJIkSZKGzbiBKsk0YB6wC7AVMDfJVqOa7QvcXFWbAycCx090oZIkSZI0bPrpodoGWFRVV1bVPcAZwJxRbeYAn28ffw3YIUkmrkxJkiRJGj7T+2izAXBtz/PFwPOX1aaq7ktyK/A44MbeRkn2B/Zvn96R5IouRWtKm8Gof1caPrGPWoPh9eERwOuDBsTrw5Ab0mvDJv006idQjdXTVB3aUFWnAKf08Z7SmJIsrKrZg65D0vDx+iBpWbw+aEXqZ8jfYmCjnucbAkuW1SbJdGAd4KaJKFCSJEmShlU/geoiYIskmyZZDdgLmD+qzXzg9e3jPYDvVtWDeqgkSZIkaWUy7pC/dk7UQcD5wDTgs1V1aZJjgIVVNR/4DHBakkU0PVN7rciiNaU5ZFTSsnh9kLQsXh+0wsSOJEmSJEnqpq+NfSVJkiRJD2agkiRJkqSODFSSJEmS1JGBSkMryVj7m0magrweSJKGlYFKQyfJVkke79L7kqAJUyPXgyQvSbLZoGuSJGmEgUpDJclawD7A09vn/huVprieMLUd8C7ghsFWJGlYjPReJ1m13S/VHm1NOj+saqhU1R3AA7QbRVfVA4OtSNIwSPJa4HDgU1V1hzdbJEFzwyXJ7sAXgNOTvNIRLpps/oekgeq5s7RZkh3bw0cCj0ryvwdXmaRBGuMO8y+BWcCO4M0WSY0k29P0XB8G3AYcNtJTJU0WA5UGIsmaSVZv7yw9F9gB+EiS9wDbAguBdQdapKSBGDVnaqck2wB/pLlOvCTJIQMtUNIw2Qg4AngOsCWwZ1Xdk2T9wZalqcRApUF5AXBSklcB84DzgF2AorkDfShwXJJnD65ESYPQE6YOoRnmtzPwFSDAHsC+SY4YXIWSBi3JRiMPgfcDBwKvrqqrk+wGHJ1kzYEVqCnFQKVJleQJSbaqqu8Cm9J8SDq6qq6tqiXAcVV1BPBO4Fzgue33+W9VWsklWaPn8ZbA31TVi4G1gRuBG6rq18CewJwk6w2mUkmD0DNNYEvgg0n2Ar4I3ArcBNyc5GXA8cDZVXXnwIrVlOKHVE2a9kL4mvbh6jRh6hvAm5M8FqCq7mv/PBVYgPMlpCkhydOBTyTZvD10H3BDOwz4qcBr22E8r6qqS4Ftq+qmQdUrafK10wR2Az4EbEyzgNWrgN1oPtN+kmaEy6FVde7ACtWUExdC0WRLsiFwAHBeVf0wyWeAWVW1Q5K/Ap5RVZ9qL5onAC+sqlsGWbOkFS/J54FbgI9W1VVJzgBm01wT7kqyL7AvsHtV3TjIWiVNviRPAL4FvBb4A02Qeinwz1X1zbbNDK8Pmmz2UGlSJXkR8CLgscDcJM+qqn2B3yU5F/gccH3b/D+BVxmmpJVXGiP/F50FbA98NskmwMdp5lee1c6ZOhjY3w9L0pS1Gs1n17uq6laaqQH3Am9NMhfA64MGwR4qTZokmwH/CBxYVdcl+SDwGOCUqrokyfOAP1XVZUmmVdX9Ay1Y0qRJ8iaaoTsfBt5Nc0PleJobLHsDNwM/q6rfDKxISZNqZMXPJGuOzIdKciywJvCRqlqcZA+aFUDvBo6qqtsGWLKmKHuoNCmSPBn4DrCoqq5rDx9DM7zn4CQvqKqLquoyAMOUNHUkmQZsB5xaVRdU1XbAo4FPAI+tqo9X1emGKWnq6AlTuwPzknwuyVOAs2k+O3w1yZtpVvj7OrAF8PjBVaypzEClFa4dz/xb4EvAy5NsDNDebTqWZn8Z7yhJU8ToTXvbGyiXAhsnWbs9fADwTOCAJKtOcomSBqwNUzsBR7dfmwP/RHOz5WTgU8BawF40nyEeD9wxiFolA5VWqCSzgA8k2a2q3k0zR+JrSZ4Efw5Vh430TElauY3atPclSbZO8kTg2zS9VC9pF655Ps314pNVde/gKpY02XpuumxHc3PlGTSfWX8KfAR4blWdWlUfoJmT/VFgv6q6fqzXk1Y051BphUtyOLAhcE5VnZvkOGAO8LKe4X+SppAkBwGvo9k6YTfgb4EtaZZBngmsD+xdVf8xsCIlTaqeYX7Tqur+dsGaxwNfAF5XVdcnWQhcTbNAzU1JngHcVlXXDLB0TXHTB12AVk5JnkOzFPo/V9VxSQ4BXpnk3qo6vJ0zMQswUElTTJIdafak2x44nGbYzoXALlW1T9uDfX9V/WFgRUqadG2Y2gF4YZLfVtXpSW4G/gvYOckvgKuAD4/sQ1dVvxpgyRLgkD9NoJ4dzKcBW9Fs2DunPf3R9s/jk7y8qv6hqv5tEHVKmlyj50zRrOA3F9gTeFFVPRP4EfDv7VYKSwxT0tTTbq0yj2bFvhOTvINmk+/Tgd2BrwKfq6qfDK5K6cHsodKE6Omm36E9dB7NRfCNSVapqm8k+TLwJOC3AytU0qQaNWdqc+C+qrq6fb4pzV5TAL8EHgBuH0SdkgarXcFvP+C9bc/UecCpwJ00i1HMB55UVf85uCqlsRmoNCHaMPUympV33lRVNyZZAAQ4Jsn2NMN7DnZOhDR19ISptwCvAG5KMh14JTANeFmSrYGXAq+0Z0qasrammW+9Y5LvVtWvkryeZkn0tdsFKAxTGkouSqEJkeRRwNeAj1fVt3onlgLPAXYCflhVFw6yTkmTL8muwFuBXYD3ANtV1fbtEumvo5lP+XnnQkhTR8/nhA2B26vq1iTb0iyDfinwtapamuTpwLpV9cOBFiw9BAOVJkR7x/krwNHtXaU1q+rOJBtX1e8GXZ+kydE7xK/n2POADWjmVm4PvLyq7k3yfOdCSFNPOxXggSS7ACcAV9Cs5vdKYBuamy9XAV+sqqWDq1Tqj4tSqJOeBSg2be8yPwBcBHwmyaptmHohcHqS9QdZq6RJNQ3+fJNlxLrAiTQ9Uy9rw9Q+wBFJ1hlEkZImX5K1ANow9WTgfTTTBPYALqbZRuFC4Ds0G/muOaBSpeVioFInbTf9LjQr7hxKM1l0Hk2o+nmSo2l2MT/BjfakqSHJDGBRkvWq6r6RUFVVFwAnAU9L8ur2+vBW4PCqunVwFUuaLO3N12OTPK49dCtNz9RvAKrqYJqtVA6rqvnA+91bSo8UBir1LcmT2h6pJNkSOJZmL5nbgBk0OetA4Eia3cwPqKqzx1gyWdJKqKpuBN4M/CjJY9tQtXp77qPAEcCTae4671lVvx5ctZIm2f3AccBjkrwCuAlYD9ixp80C4B6Aqvr9pFcodeQqf+pLkqfSrLRzNLCUZojfacAWNKFqblXd0e4h8e2qumvke0fPp5C08qqqc5LcByxMMruqbk6yRlX9F7AI+HlVXTbgMiVNsqr6U5I7gb+jWR7998A/AF9sP2PcBuwDvHNwVUrduCiFxpVkFvBN4MSq+kx7bF1gIbAq8L/aMLUdzfC/v6+q6wZUrqQh0A4JPhkYCVUHAocAOziMR5o6elbzW51mH7r7k+wNvBY4iqanajeahWu+XVXfGWtxG2mYGag0rnby+LOq6i1JVgGeRbPM8dY0K/IcD9wLvItmlb+zB1WrpOHRhqrjaTbn3I+mJ/vigRYladIlmUNzDViLZlW/nwM70CyRfnJVnT/A8qSHzSF/6seVwBuT7ATsCTyKJkydD9wNzAUup5lgfq53liQBtNeDVYBzgGdX1SWDrknS5EqyFXAw8G6alfsOAT4NnE7zeeKQJBcDS6vqgYEVKj0M9lBpXEnWBPYH3kAzB+Ik4Nc0vVT/Bziqqu5s2xqmJP2FkX3pBl2HpMmVZHPgA8DdVbV3e2xnmlWAR/aaemxVLRlcldLD5yp/GldV3dmu0PXSqtqjqv61qm4G1gFeCKw7spKfYUrSaIYpaero2adyJnAN8FtgRpIXJ1mtqs6j6bXeqKruMkxpZWCgUt+q6iaAJKsm2ZWmp+q4qlpikJIkaWpLMq1nn8pP0Wzq/X7gV8AewN8n+Wtgd5p9qKSVgkP+tFySrApsA7wXOKmqzhlwSZIkaYB6h/Um2Rb4LLB3Vf20PTadZjn0vwV+BMyvqguSrOK8Ka0MXJRCy6Wq7k3yU5oL5fXOmZIkaepKsiVwYJLjgSXAXwFfAi5NcgBNiLq6qvZLshawOnBT25t1/8AKlyaQQ/603Krq3qq6vn1smJIkaQpK8hTgDODiqrqu/UxwNs0GvQto5lofDWyZZDPgOGBN4FXAagMpWloB7KGSJEnScmmXQ/8S8N6qOivJNOAA4BSaBauqHcnyFJpgNb2q7kjyDmDNqrprYMVLE8weKkmSJC2v9YCtq+qs9vl5wKx2FMvvgT+0i1MsoNle5TftML87quqGQRUtrQguSiFJkqTl1gamecCVwA+q6piec48DZgDrV9X3nXOtlZmBSpIkSZ0k2QE4H1htZMW+JC8EDgQOqaqlg6xPmgwO+ZMkSVInVfUvNPtK/QYgyRY086i+bJjSVOGiFJIkSeqsqhYkeSDJncBVwKFVde6g65Imi0P+JEmS9LC1w//WrqpvDLoWaTIZqCRJkjRhXIBCU42BSpIkSZI6clEKSZIkSerIQCVJkiRJHRmoJEmSJKkjA5UkSZIkdWSgkiRJkqSO/hsr99FmIQkVygAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1sAAAENCAYAAAD5dH/GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAGKNJREFUeJzt3X+wpXddH/D3h01CFBACe+VHks3GmjpA5YeuwRYqcYRkE34EpnSaVTRQmHWsqVqt06CWYKIzoK1axmiIuhMETUQjupaVkMqPtNBoNhp+JBhdIrjXxSZk+Q2CGz794z7bHm7u7j27e57cszev18yZ+zzf7/d5zudkJjl5z/f7fE91dwAAAJith6x1AQAAAOuRsAUAADACYQsAAGAEwhYAAMAIhC0AAIARCFsAAAAjELYAYI1V1TVV9TNrXQcAsyVsAXBIVfXRqvpiVX1u4vWEta5rLVTVmvww5RDEXrYW7w3AsRG2AFjNC7r74ROvfcsHVNUJa1HYPKglvk8BuB9fDgAcsaraXFVdVa+oqr9N8s6h/dur6n1V9amqen9VnTNxzZlV9Z6q+mxV3VhVv1xVbx76zqmqxWXv8dGqes5w/JCqurSqPlJV91bVW6rq0ctqubiq/raqPlFVPzlxnw1V9RPDtZ+tqlur6vSqurKq/uuy9/yjqvqRKT7/u6vqZ6vqvUm+kOQbqurlVfXh4T3uqqrvnxh/TlUtVtWPVdXdVfXxqnr5Ie79iKp6V1W9vqpqtVoAmF/CFgDH4tlJnpjkvKo6NcnbkvxMkkcn+Y9Jrq+qhWHsbye5NcnGJFckufgI3ueHkrxoeL8nJPlkkiuXjXlWkm9K8l1JXl1VTxzafzTJtiQXJPm6JP82SwHpjUm2HZyVqqqNw7XXrlRAdy8PPt+bZHuSRyT5WJK7kzx/eI+XJ/nFqvqWifGPS/LIJKcmeUWSK6vqlMkbVtVjkvxJkvd29w/1kpd19zWH+4cDwHwStgBYzR8MM1Wfqqo/WNb3mu7+fHd/MclLk+zq7l3d/ZXuvjHJ7iQXVNWmJN+W5D9395e6+6Ykf3QENXx/kp/s7sXu/lKS1yR5ybLliz/d3V/s7vcneX+Spw7tr0zyU9195xBe3t/d93b3nyX5dJYCVpJclOTd3f1/pqzpmu6+vbsPdPc/dvfbuvsjw3u8J8k7kvzLifH/mOTyYeyuJJ/LUjg86AlJ3pPkd7v7p6asAYA5JmwBsJoXdfejhteLlvXtnTg+I8m/nghmn8rSbNPjM8xGdffnJ8Z/7AhqOCPJWyfu++Ek9yV57MSYv584/kKShw/Hpyf5yCHu+8YshcQMf990BDVNfvZU1flVdXNV7R9qvCBLs3gH3dvdBw5RY5I8L8nXJLnqCGoAYI4JWwAci8kd+vYmedNEMHtUdz+su1+b5ONJTqmqh02M3zRx/PkkX3vwpKo2JFmY6N+b5Pxl9z65u/9uihr3Jvknh+h7c5ILq+qpWVoOuXzm7nD+32evqocmuT7Jf0ny2O5+VJJdSY7kmatfS/L2JLuW/XMC4DglbAEwK29O8oKqOm/YlOLkYWOI07r7Y1laUvjTVXVSVT0ryQsmrv2rJCdX1fOq6sQkP5XkoRP9VyX52ao6I0mqaqGqLpyyrl9PckVVnTXsHPiU4dmodPdikluyNKN1/bAc8micNNR7T5IDVXV+knOP4j6XJLkzyX+vqq85yloAmBPCFgAz0d17k1yY5CeyFDr2Jvnx/P/vmu9O8owk+5NcluQ3J679dJJ/l6Vg9HdZmuma3J3wvyXZmeQdVfXZJDcP95rGLyR5S5aeofpMkt/I0nK9g96Y5JtzZEsIv0p3fzZLm3i8JUubd3z3UO+R3qeztOnG3iR/WFUnH21NAKy9WvrvOgA8sKrqNUm+sbtfutrYkev4jizNym3u7q+sZS0ArC9mtgB40BqWLP5wkl8XtACYNWELgAel4Xe4PpWl3RJ/aY3LAWAdsowQAABgBGa2AAAARiBsAQAAjOCEtS5gJRs3buzNmzevdRkAAAD3c+utt36iuxdWGzeXYWvz5s3ZvXv3WpcBAABwP1X1sWnGWUYIAAAwAmELAABgBMIWAADACIQtAACAEawatqrq9Kp6V1V9uKpur6ofXmFMVdXrq2pPVX2gqr5lou/iqvrr4XXxrD8AAADAPJpmN8IDSX6su/+8qh6R5NaqurG775gYc36Ss4bXM5L8apJnVNWjk1yWZEuSHq7d2d2fnOmnAAAAmDOrzmx198e7+8+H488m+XCSU5cNuzDJb/aSm5M8qqoen+S8JDd29/4hYN2YZOtMPwEAAMAcOqJntqpqc5KnJ/nTZV2nJtk7cb44tB2qHQAAYF2b+keNq+rhSa5P8iPd/Znl3Stc0odpX+n+25NsT5JNmzZNW9YDYvOlb1vrEgDm1kdf+7y1LgEA5tJUM1tVdWKWgtZvdffvrzBkMcnpE+enJdl3mPb76e6ru3tLd29ZWFiYpiwAAIC5Nc1uhJXkN5J8uLt/4RDDdib5vmFXwm9P8unu/niSG5KcW1WnVNUpSc4d2gAAANa1aZYRPjPJ9yb5YFXdNrT9RJJNSdLdVyXZleSCJHuSfCHJy4e+/VV1RZJbhusu7+79sysfAABgPq0atrr7f2XlZ68mx3SSHzxE344kO46qOgAAgOPUEe1GCAAAwHSELQAAgBEIWwAAACMQtgAAAEYgbAEAAIxA2AIAABiBsAUAADACYQsAAGAEwhYAAMAIhC0AAIARCFsAAAAjELYAAABGIGwBAACMQNgCAAAYgbAFAAAwAmELAABgBMIWAADACE5YbUBV7Ujy/CR3d/c/W6H/x5N8z8T9nphkobv3V9VHk3w2yX1JDnT3llkVDgAAMM+mmdm6JsnWQ3V2989399O6+2lJXpXkPd29f2LIdw79ghYAAPCgsWrY6u6bkuxfbdxgW5Jrj6kiAACAdWBmz2xV1ddmaQbs+onmTvKOqrq1qrbP6r0AAADm3arPbB2BFyR577IlhM/s7n1V9fVJbqyqvxxmyu5nCGPbk2TTpk0zLAsAAOCBN8vdCC/KsiWE3b1v+Ht3krcmOftQF3f31d29pbu3LCwszLAsAACAB95MwlZVPTLJs5P84UTbw6rqEQePk5yb5EOzeD8AAIB5N83W79cmOSfJxqpaTHJZkhOTpLuvGoa9OMk7uvvzE5c+Nslbq+rg+/x2d799dqUDAADMr1XDVndvm2LMNVnaIn6y7a4kTz3awgAAAI5ns3xmCwAAgIGwBQAAMAJhCwAAYATCFgAAwAiELQAAgBEIWwAAACMQtgAAAEYgbAEAAIxA2AIAABiBsAUAADACYQsAAGAEwhYAAMAIhC0AAIARCFsAAAAjELYAAABGIGwBAACMQNgCAAAYwaphq6p2VNXdVfWhQ/SfU1WfrqrbhterJ/q2VtWdVbWnqi6dZeEAAADzbJqZrWuSbF1lzP/s7qcNr8uTpKo2JLkyyflJnpRkW1U96ViKBQAAOF6sGra6+6Yk+4/i3mcn2dPdd3X3l5Ncl+TCo7gPAADAcWdWz2z986p6f1X9cVU9eWg7NcneiTGLQxsAAMC6d8IM7vHnSc7o7s9V1QVJ/iDJWUlqhbF9qJtU1fYk25Nk06ZNMygLAABg7RzzzFZ3f6a7Pzcc70pyYlVtzNJM1ukTQ09Lsu8w97m6u7d095aFhYVjLQsAAGBNHXPYqqrHVVUNx2cP97w3yS1JzqqqM6vqpCQXJdl5rO8HAABwPFh1GWFVXZvknCQbq2oxyWVJTkyS7r4qyUuS/EBVHUjyxSQXdXcnOVBVlyS5IcmGJDu6+/ZRPgUAAMCcWTVsdfe2Vfp/OckvH6JvV5JdR1caAADA8WtWuxECAAAwQdgCAAAYgbAFAAAwAmELAABgBMIWAADACIQtAACAEQhbAAAAIxC2AAAARiBsAQAAjEDYAgAAGIGwBQAAMAJhCwAAYATCFgAAwAiELQAAgBEIWwAAACMQtgAAAEYgbAEAAIxg1bBVVTuq6u6q+tAh+r+nqj4wvN5XVU+d6PtoVX2wqm6rqt2zLBwAAGCeTTOzdU2SrYfp/5skz+7upyS5IsnVy/q/s7uf1t1bjq5EAACA488Jqw3o7puqavNh+t83cXpzktOOvSwAAIDj26yf2XpFkj+eOO8k76iqW6tq++EurKrtVbW7qnbfc889My4LAADggbXqzNa0quo7sxS2njXR/Mzu3ldVX5/kxqr6y+6+aaXru/vqDEsQt2zZ0rOqCwAAYC3MZGarqp6S5NeTXNjd9x5s7+59w9+7k7w1ydmzeD8AAIB5d8xhq6o2Jfn9JN/b3X810f6wqnrEweMk5yZZcUdDAACA9WbVZYRVdW2Sc5JsrKrFJJclOTFJuvuqJK9O8pgkv1JVSXJg2HnwsUneOrSdkOS3u/vtI3wGAACAuTPNboTbVul/ZZJXrtB+V5Kn3v8KAACA9W/WuxECAAAQYQsAAGAUwhYAAMAIhC0AAIARCFsAAAAjELYAAABGIGwBAACMQNgCAAAYgbAFAAAwAmELAABgBMIWAADACIQtAACAEQhbAAAAIxC2AAAARiBsAQAAjEDYAgAAGMFUYauqdlTV3VX1oUP0V1W9vqr2VNUHqupbJvourqq/Hl4Xz6pwAACAeTbtzNY1SbYepv/8JGcNr+1JfjVJqurRSS5L8owkZye5rKpOOdpiAQAAjhdTha3uvinJ/sMMuTDJb/aSm5M8qqoen+S8JDd29/7u/mSSG3P40AYAALAuzOqZrVOT7J04XxzaDtUOAACwrp0wo/vUCm19mPb736Bqe5aWIGbTpk0zKgsAjg+bL33bWpcAMJc++trnrXUJR21WM1uLSU6fOD8tyb7DtN9Pd1/d3Vu6e8vCwsKMygIAAFgbswpbO5N837Ar4bcn+XR3fzzJDUnOrapTho0xzh3aAAAA1rWplhFW1bVJzkmysaoWs7TD4IlJ0t1XJdmV5IIke5J8IcnLh779VXVFkluGW13e3YfbaAMAAGBdmCpsdfe2Vfo7yQ8eom9Hkh1HXhoAAMDxa1bLCAEAAJggbAEAAIxA2AIAABiBsAUAADACYQsAAGAEwhYAAMAIhC0AAIARCFsAAAAjELYAAABGIGwBAACMQNgCAAAYgbAFAAAwAmELAABgBMIWAADACIQtAACAEQhbAAAAIxC2AAAARjBV2KqqrVV1Z1XtqapLV+j/xaq6bXj9VVV9aqLvvom+nbMsHgAAYF6dsNqAqtqQ5Mokz02ymOSWqtrZ3XccHNPd/2Fi/L9P8vSJW3yxu582u5IBAADm3zQzW2cn2dPdd3X3l5Ncl+TCw4zfluTaWRQHAABwvJombJ2aZO/E+eLQdj9VdUaSM5O8c6L55KraXVU3V9WLjrpSAACA48iqywiT1AptfYixFyX5ve6+b6JtU3fvq6pvSPLOqvpgd3/kfm9StT3J9iTZtGnTFGUBAADMr2lmthaTnD5xflqSfYcYe1GWLSHs7n3D37uSvDtf/TzX5Liru3tLd29ZWFiYoiwAAID5NU3YuiXJWVV1ZlWdlKVAdb9dBavqm5KckuR/T7SdUlUPHY43JnlmkjuWXwsAALDerLqMsLsPVNUlSW5IsiHJju6+vaouT7K7uw8Gr21JruvuySWGT0zyhqr6SpaC3WsndzEEAABYr6Z5ZivdvSvJrmVtr152/poVrntfkm8+hvoAAACOS1P9qDEAAABHRtgCAAAYgbAFAAAwAmELAABgBMIWAADACIQtAACAEQhbAAAAIxC2AAAARiBsAQAAjEDYAgAAGIGwBQAAMAJhCwAAYATCFgAAwAiELQAAgBEIWwAAACMQtgAAAEYgbAEAAIxgqrBVVVur6s6q2lNVl67Q/7Kquqeqbhter5zou7iq/np4XTzL4gEAAObVCasNqKoNSa5M8twki0luqaqd3X3HsqG/092XLLv20UkuS7IlSSe5dbj2kzOpHgAAYE5NM7N1dpI93X1Xd385yXVJLpzy/uclubG79w8B68YkW4+uVAAAgOPHNGHr1CR7J84Xh7bl/lVVfaCqfq+qTj/Ca1NV26tqd1Xtvueee6YoCwAAYH5NE7ZqhbZedv5HSTZ391OS/I8kbzyCa5cau6/u7i3dvWVhYWGKsgAAAObXNGFrMcnpE+enJdk3OaC77+3uLw2nv5bkW6e9FgAAYD2aJmzdkuSsqjqzqk5KclGSnZMDqurxE6cvTPLh4fiGJOdW1SlVdUqSc4c2AACAdW3V3Qi7+0BVXZKlkLQhyY7uvr2qLk+yu7t3JvmhqnphkgNJ9id52XDt/qq6IkuBLUku7+79I3wOAACAubJq2EqS7t6VZNeytldPHL8qyasOce2OJDuOoUYAAIDjzlQ/agwAAMCREbYAAABGIGwBAACMQNgCAAAYgbAFAAAwAmELAABgBMIWAADACIQtAACAEQhbAAAAIxC2AAAARiBsAQAAjEDYAgAAGIGwBQAAMAJhCwAAYATCFgAAwAiELQAAgBFMFbaqamtV3VlVe6rq0hX6f7Sq7qiqD1TVn1TVGRN991XVbcNr5yyLBwAAmFcnrDagqjYkuTLJc5MsJrmlqnZ29x0Tw/4iyZbu/kJV/UCSn0vyb4a+L3b302ZcNwAAwFybZmbr7CR7uvuu7v5ykuuSXDg5oLvf1d1fGE5vTnLabMsEAAA4vkwTtk5NsnfifHFoO5RXJPnjifOTq2p3Vd1cVS86ihoBAACOO6suI0xSK7T1igOrXppkS5JnTzRv6u59VfUNSd5ZVR/s7o+scO32JNuTZNOmTVOUBQAAML+mmdlaTHL6xPlpSfYtH1RVz0nyk0le2N1fOtje3fuGv3cleXeSp6/0Jt19dXdv6e4tCwsLU38AAACAeTRN2LolyVlVdWZVnZTkoiRftatgVT09yRuyFLTunmg/paoeOhxvTPLMJJMbawAAAKxLqy4j7O4DVXVJkhuSbEiyo7tvr6rLk+zu7p1Jfj7Jw5P8blUlyd929wuTPDHJG6rqK1kKdq9dtoshAADAujTNM1vp7l1Jdi1re/XE8XMOcd37knzzsRQIAABwPJrqR40BAAA4MsIWAADACIQtAACAEQhbAAAAIxC2AAAARiBsAQAAjEDYAgAAGIGwBQAAMAJhCwAAYATCFgAAwAiELQAAgBEIWwAAACMQtgAAAEYgbAEAAIxA2AIAABiBsAUAADACYQsAAGAEU4WtqtpaVXdW1Z6qunSF/odW1e8M/X9aVZsn+l41tN9ZVefNrnQAAID5tWrYqqoNSa5Mcn6SJyXZVlVPWjbsFUk+2d3fmOQXk7xuuPZJSS5K8uQkW5P8ynA/AACAdW2ama2zk+zp7ru6+8tJrkty4bIxFyZ543D8e0m+q6pqaL+uu7/U3X+TZM9wPwAAgHVtmrB1apK9E+eLQ9uKY7r7QJJPJ3nMlNcCAACsOydMMaZWaOspx0xz7dINqrYn2T6cfq6q7pyiNniw2pjkE2tdBCRJvW6tKwBG4HuGuTGn3zNnTDNomrC1mOT0ifPTkuw7xJjFqjohySOT7J/y2iRJd1+d5OppioYHu6ra3d1b1roOANYn3zMwG9MsI7wlyVlVdWZVnZSlDS92LhuzM8nFw/FLkryzu3tov2jYrfDMJGcl+bPZlA4AADC/Vp3Z6u4DVXVJkhuSbEiyo7tvr6rLk+zu7p1JfiPJm6pqT5ZmtC4arr29qt6S5I4kB5L8YHffN9JnAQAAmBu1NAEFHE+qavuw9BYAZs73DMyGsAUAADCCaZ7ZAgAA4AgJWzBnqupxw4+CAwBwHBO2YI5U1XlJ3pqv/skEAACOQ8IWzImqOjfJ65I8PsmPrXE5ADwIWEkB4xK2YA5U1XOS/EqS78nS79E9saq+Y22rAmC9qapnVNWzq+rbkqS7W+CC8az6O1vAA2JDku8bfpvuUUnuTPLkJDdVVbVtQwE4RlV1fpLXJ3lXkq+vqnu7+xUHA5fvGpg9W7/DHKmqh3T3V6pqa5Jrkjy3uz+4xmUBcJyrqg1JfivJ27r7TVX1dUl2Jfn77n7JMEbgghmzjBDmSHd/Zfj79iRXJ3l+LfHvKgBHrbvvS/IXE+ef6e5nJXlsVb1haBO0YMb8DxzMr/cneV6ShxwMYQBwJKrqn06c/l2S/1RVmybaXpzkMVX1pAe2MnhwELZgTnX39Un2xTbwAByFqnp+ktuq6rok6e43Z+nnRd57MHB19yeSHEjyiDUrFNYxz2zBHLJuHoBjUVUPS3J9kt9P8i+SPLS7tw19VyR5YZZ2wd2Y5KVJLujuv1mjcmHdErYAANahqnpCks8kOTnJVUn+cSJwvTjJ45J8a5Jf6u4PrVmhsI4JWwAA61xVPSZLGy99ubu3VdWTk3yuuz+2xqXBuuaZLQCAda67703y/Un+oaruTPKHSe5b26pg/RO2AAAeBIbNMD6Q5JFJXtzdi2tcEqx7whYAwINAVZ2S5IIk53b3B9e6Hngw8MwWAMCDRFWd3N3/sNZ1wIOFsAUAADACywgBAABGIGwBAACMQNgCAAAYgbAFAAAwAmELAABgBMIWAADACP4vTn+aEffHME4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.plot.frequency(\"*\", 10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting Data In/Out" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/plain": [ "['names', 'height', 'function', 'rank']" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.cols.names()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [], "source": [ "df.to_json()" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "inputHidden": false, "outputHidden": false }, "outputs": [ { "data": { "text/plain": [ "StructType(List(StructField(names,StringType,true),StructField(height,DoubleType,true),StructField(function,StringType,true),StructField(rank,LongType,true)))" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.schema" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.table()" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "inputHidden": false, "lines_to_next_cell": 0, "outputHidden": false, "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Processing column 'height'...\n", "_count_data_types() executed in 1.11 sec\n", "count_data_types() executed in 1.11 sec\n", "cast_columns() executed in 0.0 sec\n", "_exprs() executed in 1.18 sec\n", "general_stats() executed in 1.19 sec\n", "------------------------------\n", "Processing column 'height'...\n", "frequency() executed in 1.19 sec\n", "stats_by_column() executed in 0.0 sec\n", "percentile() executed in 0.04 sec\n", "extra_numeric_stats() executed in 0.17 sec\n", "bucketizer() executed in 0.19 sec\n", "hist() executed in 1.38 sec\n", "dataset_info() executed in 1.21 sec\n" ] }, { "data": { "text/html": [ "\n", "\n", "
\n", "

Overview

\n", "
\n", "
\n", "
\n", "

Dataset info

\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "
Number of columns4
Number of rows3
Total Missing (%)0.0%
Total size in memory81.7 MB
\n", "
\n", "
\n", "

Column types

\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "
String0
Numeric1
Date0
Bool0
Array0
Not available0
\n", "
\n", "
\n", "\n", "
\n", "
\n", "\n", " \n", "\n", "
\n", "
\n", "

height

\n", "
numeric
\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Unique 3
Unique (%) 100.0
Missing0.0
Missing (%)0
\n", "
\n", "

\n", " Datatypes\n", "

\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "
\n", " String\n", " \n", " 0\n", "
\n", " Integer\n", " \n", " 0\n", "
\n", " Float\n", " \n", " 0\n", "
\n", " Bool\n", " \n", " 0\n", "
\n", " Date\n", " \n", " 0\n", "
\n", " Missing\n", " \n", " 0\n", "
\n", " Null\n", " \n", " 0\n", "
\n", " \n", "
\n", "

\n", " Basic Stats\n", "

\n", "\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "
Mean23.833333333333332
Minimum17.5
Maximum28.0
Zeros(%)0
\n", " \n", "\n", "
\n", "
\n", "

Frequency

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ValueCountFrequency (%)
28.0133.333%
26.0133.333%
17.5133.333%
\"Missing\"00.0%
\n", "
\n", " \n", "\n", " \n", "
\n", "\n", "\n", "

Quantile statistics

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Minimum17.5
5-th percentile17.5
Q117.5
Median17.5
Q317.5
95-th percentile17.5
Maximum28.0
Range10.5
Interquartile range0.0
\n", "
\n", "
\n", "

Descriptive statistics

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Standard deviation5.575242894559244
Coef of variation0.23393
Kurtosis-1.5000000000000004
Mean23.833333333333332
MAD0.0
Skewness0
Sum71.5
Variance31.083333333333336
\n", "
\n", " \n", "
\n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\n", "\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", "
\n", "\n", "
\n", "
\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "Pika version 0.12.0 connecting to ::1:5672\n", "Created channel=1\n", "Closing channel (0): 'Normal shutdown' on ('::1', 5672, 0, 0) params=>>\n", "Received on ('::1', 5672, 0, 0) params=>>\n", "run() executed in 8.76 sec\n" ] } ], "source": [ "op.profiler.run(df, \"height\", infer=True)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Downloading foo.csv from https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.csv\n", "Downloaded 967 bytes\n", "Creating DataFrame for foo.csv. Please wait...\n", "Successfully created DataFrame for 'foo.csv'\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 5 of 5 rows / 8 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
id
\n", "
1 (int)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
firstName
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
lastName
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
billingId
\n", "
4 (int)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
product
\n", "
5 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
price
\n", "
6 (int)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
birth
\n", "
7 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
dummyCol
\n", "
8 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " 1\n", " \n", " Luis\n", " \n", " Alvarez$$%!\n", " \n", " 123\n", " \n", " Cake\n", " \n", " 10\n", " \n", " 1980/07/07\n", " \n", " never\n", "
\n", " 2\n", " \n", " André\n", " \n", " Ampère\n", " \n", " 423\n", " \n", " piza\n", " \n", " 8\n", " \n", " 1950/07/08\n", " \n", " gonna\n", "
\n", " 3\n", " \n", " NiELS\n", " \n", " Böhr//((%%\n", " \n", " 551\n", " \n", " pizza\n", " \n", " 8\n", " \n", " 1990/07/09\n", " \n", " give\n", "
\n", " 4\n", " \n", " PAUL\n", " \n", " dirac$\n", " \n", " 521\n", " \n", " pizza\n", " \n", " 8\n", " \n", " 1954/07/10\n", " \n", " you\n", "
\n", " 5\n", " \n", " Albert\n", " \n", " Einstein\n", " \n", " 634\n", " \n", " pizza\n", " \n", " 8\n", " \n", " 1990/07/11\n", " \n", " up\n", "
\n", "\n", "
Viewing 5 of 5 rows / 8 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df_csv = op.load.csv(\"https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.csv\").limit(5)\n", "df_csv.table()" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Downloading foo.json from https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.json\n", "Downloaded 2596 bytes\n", "Creating DataFrame for foo.json. Please wait...\n", "Successfully created DataFrame for 'foo.json'\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 5 of 5 rows / 8 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
billingId
\n", "
1 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
birth
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
dummyCol
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
firstName
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
id
\n", "
5 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
lastName
\n", "
6 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
price
\n", "
7 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
product
\n", "
8 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " 123\n", " \n", " 1980/07/07\n", " \n", " never\n", " \n", " Luis\n", " \n", " 1\n", " \n", " Alvarez$$%!\n", " \n", " 10\n", " \n", " Cake\n", "
\n", " 423\n", " \n", " 1950/07/08\n", " \n", " gonna\n", " \n", " André\n", " \n", " 2\n", " \n", " Ampère\n", " \n", " 8\n", " \n", " piza\n", "
\n", " 551\n", " \n", " 1990/07/09\n", " \n", " give\n", " \n", " NiELS\n", " \n", " 3\n", " \n", " Böhr//((%%\n", " \n", " 8\n", " \n", " pizza\n", "
\n", " 521\n", " \n", " 1954/07/10\n", " \n", " you\n", " \n", " PAUL\n", " \n", " 4\n", " \n", " dirac$\n", " \n", " 8\n", " \n", " pizza\n", "
\n", " 634\n", " \n", " 1990/07/11\n", " \n", " up\n", " \n", " Albert\n", " \n", " 5\n", " \n", " Einstein\n", " \n", " 8\n", " \n", " pizza\n", "
\n", "\n", "
Viewing 5 of 5 rows / 8 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df_json = op.load.json(\"https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.json\").limit(5)\n", "df_json.table()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df_csv.save.csv(\"test.csv\")" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
names
\n", "
1 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
height
\n", "
2 (double)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
function
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
rank
\n", "
4 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", " bumbl#ebéé⸱⸱\n", " \n", " 17.5\n", " \n", " Espionage\n", " \n", " 7\n", "
\n", " Optim'us\n", " \n", " 28.0\n", " \n", " Leader\n", " \n", " 10\n", "
\n", " ironhide&\n", " \n", " 26.0\n", " \n", " Security\n", " \n", " 7\n", "
\n", "\n", "
Viewing 3 of 3 rows / 4 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Enrichment" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "df = op.load.json(\"https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.json\")" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 10 of 19 rows / 8 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
billingId
\n", "
1 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
birth
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
dummyCol
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
firstName
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
id
\n", "
5 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
lastName
\n", "
6 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
price
\n", "
7 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
product
\n", "
8 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
123\n", "
\n", " 		\n", "
1980/07/07\n", "
\n", " 		\n", "
never\n", "
\n", " 		\n", "
Luis\n", "
\n", " 		\n", "
1\n", "
\n", " 		\n", "
Alvarez$$%!\n", "
\n", " 		\n", "
10\n", "
\n", " 		\n", "
Cake\n", "
\n", "
\n", "
423\n", "
\n", " 		\n", "
1950/07/08\n", "
\n", " 		\n", "
gonna\n", "
\n", " 		\n", "
André\n", "
\n", " 		\n", "
2\n", "
\n", " 		\n", "
Ampère\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
piza\n", "
\n", "
\n", "
551\n", "
\n", " 		\n", "
1990/07/09\n", "
\n", " 		\n", "
give\n", "
\n", " 		\n", "
NiELS\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
Böhr//((%%\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
pizza\n", "
\n", "
\n", "
521\n", "
\n", " 		\n", "
1954/07/10\n", "
\n", " 		\n", "
you\n", "
\n", " 		\n", "
PAUL\n", "
\n", " 		\n", "
4\n", "
\n", " 		\n", "
dirac$\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
pizza\n", "
\n", "
\n", "
634\n", "
\n", " 		\n", "
1990/07/11\n", "
\n", " 		\n", "
up\n", "
\n", " 		\n", "
Albert\n", "
\n", " 		\n", "
5\n", "
\n", " 		\n", "
Einstein\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
pizza\n", "
\n", "
\n", "
672\n", "
\n", " 		\n", "
1930/08/12\n", "
\n", " 		\n", "
never\n", "
\n", " 		\n", "
Galileo\n", "
\n", " 		\n", "
6\n", "
\n", " 		\n", "
⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅GALiLEI\n", "
\n", " 		\n", "
5\n", "
\n", " 		\n", "
arepa\n", "
\n", "
\n", "
323\n", "
\n", " 		\n", "
1970/07/13\n", "
\n", " 		\n", "
gonna\n", "
\n", " 		\n", "
CaRL\n", "
\n", " 		\n", "
7\n", "
\n", " 		\n", "
Ga%%%uss\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taco\n", "
\n", "
\n", "
624\n", "
\n", " 		\n", "
1950/07/14\n", "
\n", " 		\n", "
let\n", "
\n", " 		\n", "
David\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
H$$$ilbert\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taaaccoo\n", "
\n", "
\n", "
735\n", "
\n", " 		\n", "
1920/04/22\n", "
\n", " 		\n", "
you\n", "
\n", " 		\n", "
Johannes\n", "
\n", " 		\n", "
9\n", "
\n", " 		\n", "
KEPLER\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taco\n", "
\n", "
\n", "
875\n", "
\n", " 		\n", "
1923/03/12\n", "
\n", " 		\n", "
down\n", "
\n", " 		\n", "
JaMES\n", "
\n", " 		\n", "
10\n", "
\n", " 		\n", "
M$$ax%%well\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taco\n", "
\n", "
\n", "\n", "\n", "
Viewing 10 of 19 rows / 8 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.table()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "count is deprecated. Use Collection.count_documents instead.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0720079ce4774eba846f02c6b8a49949", "version_major": 2, "version_minor": 0 }, "text/plain": [ "HBox(children=(IntProgress(value=0, description='Processing...', max=19, style=ProgressStyle(description_width…" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "find_and_modify is deprecated, use find_one_and_delete, find_one_and_replace, or find_one_and_update instead\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "import requests\n", "\n", "\n", "def func_request(params):\n", " # You can use here whatever header or auth info you need to send. \n", " # For more information see the requests library\n", " \n", " url= \"https://jsonplaceholder.typicode.com/todos/\" + str(params[\"id\"])\n", " return requests.get(url)\n", "\n", "def func_response(response):\n", " # Here you can parse de response\n", " return response[\"title\"]\n", "\n", "\n", "e = op.enrich(host=\"localhost\", port=27017, db_name=\"jazz\")\n", "e.flush()\n", "df_result = e.run(df, func_request, func_response, calls= 60, period = 60, max_tries = 8)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
Viewing 10 of 19 rows / 9 columns
\n", "
1 partition(s)
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
billingId
\n", "
1 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
birth
\n", "
2 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
dummyCol
\n", "
3 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
firstName
\n", "
4 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
id
\n", "
5 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
lastName
\n", "
6 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
price
\n", "
7 (bigint)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
product
\n", "
8 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", " 		\n", "
jazz_results
\n", "
9 (string)
\n", "
\n", " \n", " nullable\n", " \n", "
\n", "
\n", "
123\n", "
\n", " 		\n", "
1980/07/07\n", "
\n", " 		\n", "
never\n", "
\n", " 		\n", "
Luis\n", "
\n", " 		\n", "
1\n", "
\n", " 		\n", "
Alvarez$$%!\n", "
\n", " 		\n", "
10\n", "
\n", " 		\n", "
Cake\n", "
\n", " 		\n", "
delectus⋅aut⋅autem\n", "
\n", "
\n", "
423\n", "
\n", " 		\n", "
1950/07/08\n", "
\n", " 		\n", "
gonna\n", "
\n", " 		\n", "
André\n", "
\n", " 		\n", "
2\n", "
\n", " 		\n", "
Ampère\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
piza\n", "
\n", " 		\n", "
quis⋅ut⋅nam⋅facilis⋅et⋅officia⋅qui\n", "
\n", "
\n", "
551\n", "
\n", " 		\n", "
1990/07/09\n", "
\n", " 		\n", "
give\n", "
\n", " 		\n", "
NiELS\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
Böhr//((%%\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
pizza\n", "
\n", " 		\n", "
fugiat⋅veniam⋅minus\n", "
\n", "
\n", "
521\n", "
\n", " 		\n", "
1954/07/10\n", "
\n", " 		\n", "
you\n", "
\n", " 		\n", "
PAUL\n", "
\n", " 		\n", "
4\n", "
\n", " 		\n", "
dirac$\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
pizza\n", "
\n", " 		\n", "
et⋅porro⋅tempora\n", "
\n", "
\n", "
634\n", "
\n", " 		\n", "
1990/07/11\n", "
\n", " 		\n", "
up\n", "
\n", " 		\n", "
Albert\n", "
\n", " 		\n", "
5\n", "
\n", " 		\n", "
Einstein\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
pizza\n", "
\n", " 		\n", "
laboriosam⋅mollitia⋅et⋅enim⋅quasi⋅adipisci⋅quia⋅provident⋅illum\n", "
\n", "
\n", "
672\n", "
\n", " 		\n", "
1930/08/12\n", "
\n", " 		\n", "
never\n", "
\n", " 		\n", "
Galileo\n", "
\n", " 		\n", "
6\n", "
\n", " 		\n", "
⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅GALiLEI\n", "
\n", " 		\n", "
5\n", "
\n", " 		\n", "
arepa\n", "
\n", " 		\n", "
qui⋅ullam⋅ratione⋅quibusdam⋅voluptatem⋅quia⋅omnis\n", "
\n", "
\n", "
323\n", "
\n", " 		\n", "
1970/07/13\n", "
\n", " 		\n", "
gonna\n", "
\n", " 		\n", "
CaRL\n", "
\n", " 		\n", "
7\n", "
\n", " 		\n", "
Ga%%%uss\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taco\n", "
\n", " 		\n", "
illo⋅expedita⋅consequatur⋅quia⋅in\n", "
\n", "
\n", "
624\n", "
\n", " 		\n", "
1950/07/14\n", "
\n", " 		\n", "
let\n", "
\n", " 		\n", "
David\n", "
\n", " 		\n", "
8\n", "
\n", " 		\n", "
H$$$ilbert\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taaaccoo\n", "
\n", " 		\n", "
quo⋅adipisci⋅enim⋅quam⋅ut⋅ab\n", "
\n", "
\n", "
735\n", "
\n", " 		\n", "
1920/04/22\n", "
\n", " 		\n", "
you\n", "
\n", " 		\n", "
Johannes\n", "
\n", " 		\n", "
9\n", "
\n", " 		\n", "
KEPLER\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taco\n", "
\n", " 		\n", "
molestiae⋅perspiciatis⋅ipsa\n", "
\n", "
\n", "
875\n", "
\n", " 		\n", "
1923/03/12\n", "
\n", " 		\n", "
down\n", "
\n", " 		\n", "
JaMES\n", "
\n", " 		\n", "
10\n", "
\n", " 		\n", "
M$$ax%%well\n", "
\n", " 		\n", "
3\n", "
\n", " 		\n", "
taco\n", "
\n", " 		\n", "
illo⋅est⋅ratione⋅doloremque⋅quia⋅maiores⋅aut\n", "
\n", "
\n", "\n", "\n", "
Viewing 10 of 19 rows / 9 columns
\n", "
1 partition(s)
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df_result.table()" ] } ], "metadata": { "jupytext": { "formats": "ipynb,py" }, "kernel_info": { "name": "python3" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.1" }, "nteract": { "version": "0.11.7" } }, "nbformat": 4, "nbformat_minor": 2 }