{ "cells": [ { "cell_type": "markdown", "id": "dea00c9e-ab6a-4a56-8e91-782b06002f6c", "metadata": { "id": "dea00c9e-ab6a-4a56-8e91-782b06002f6c" }, "source": [ "# Fine-tuning a BERT model with skorch and Hugging Face" ] }, { "cell_type": "markdown", "id": "11b4d0cc-40c5-48a9-bd52-fdd522498acf", "metadata": { "id": "11b4d0cc-40c5-48a9-bd52-fdd522498acf" }, "source": [ "In this notebook, we follow the fine-tuning guideline from [Hugging Face documentation](https://huggingface.co/docs/transformers/training). Please check it out if we you want to know more about BERT and fine-tuning. Here, we assume that you're familiar with the general ideas.\n", "\n", "You will learn how to:\n", "- integrate the [Hugging Face transformers](https://huggingface.co/docs/transformers/index) library with skorch\n", "- use skorch to fine-tune a BERT model on a text classification task\n", "- use skorch with the [Hugging Face accelerate](https://huggingface.co/docs/accelerate/index) library for automatic mixed precision (AMP) training" ] }, { "cell_type": "markdown", "id": "922bfcd7", "metadata": { "id": "922bfcd7" }, "source": [ "
\n", "\n", " Run in Google Colab \n", "\n", "View source on GitHub
" ] }, { "cell_type": "markdown", "id": "9737e20e", "metadata": { "id": "9737e20e" }, "source": [ "The first part of the notebook requires hugginface `transformers` as an additional dependency. If you have not already installed it, you can do so like this:\n", "\n", "`python -m pip install transformers`" ] }, { "cell_type": "code", "execution_count": 1, "id": "465f48cc", "metadata": { "id": "465f48cc" }, "outputs": [], "source": [ "import subprocess\n", "\n", "# Installation on Google Colab\n", "try:\n", " import google.colab\n", " subprocess.run(['python', '-m', 'pip', 'install', 'skorch', 'transformers'])\n", "except ImportError:\n", " pass" ] }, { "cell_type": "markdown", "id": "f9e0f846-ea36-4a07-835d-c2b8c69c27fd", "metadata": { "id": "f9e0f846-ea36-4a07-835d-c2b8c69c27fd" }, "source": [ "## Imports" ] }, { "cell_type": "code", "execution_count": 2, "id": "5164c02c-9d4f-4b1c-bf72-0e8f21a23b89", "metadata": { "id": "5164c02c-9d4f-4b1c-bf72-0e8f21a23b89" }, "outputs": [], "source": [ "import numpy as np\n", "import torch\n", "from sklearn.datasets import fetch_20newsgroups\n", "from sklearn.metrics import accuracy_score\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.pipeline import Pipeline\n", "from skorch import NeuralNetClassifier\n", "from skorch.callbacks import LRScheduler, ProgressBar\n", "from skorch.hf import HuggingfacePretrainedTokenizer\n", "from torch import nn\n", "from torch.optim.lr_scheduler import LambdaLR\n", "from transformers import AutoModelForSequenceClassification\n", "from transformers import AutoTokenizer" ] }, { "cell_type": "markdown", "id": "380c4fee-8c1d-42e2-933f-27570d1c7ea3", "metadata": { "id": "380c4fee-8c1d-42e2-933f-27570d1c7ea3" }, "source": [ "## Parameters" ] }, { "cell_type": "markdown", "id": "83c77bdf-9929-47b7-8e19-39b623fd4a52", "metadata": { "id": "83c77bdf-9929-47b7-8e19-39b623fd4a52" }, "source": [ "Change the values below if you want to try out different model architectures and hyper-parameters." ] }, { "cell_type": "code", "execution_count": 3, "id": "e6f17a28-52e3-4f0c-8c3e-308ecc9288f3", "metadata": { "id": "e6f17a28-52e3-4f0c-8c3e-308ecc9288f3" }, "outputs": [], "source": [ "# Choose a tokenizer and BERT model that work together\n", "TOKENIZER = \"distilbert-base-uncased\"\n", "PRETRAINED_MODEL = \"distilbert-base-uncased\"\n", "\n", "# model hyper-parameters\n", "OPTMIZER = torch.optim.AdamW\n", "LR = 5e-5\n", "MAX_EPOCHS = 3\n", "CRITERION = nn.CrossEntropyLoss\n", "BATCH_SIZE = 8\n", "\n", "# device\n", "DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'" ] }, { "cell_type": "markdown", "id": "556ed4ac-27e8-4b80-bcc6-2431f1d9ec12", "metadata": { "id": "556ed4ac-27e8-4b80-bcc6-2431f1d9ec12" }, "source": [ "## Data" ] }, { "cell_type": "code", "execution_count": 4, "id": "8ecfd881-f589-493a-a028-68becdf240ea", "metadata": { "id": "8ecfd881-f589-493a-a028-68becdf240ea" }, "outputs": [], "source": [ "dataset = fetch_20newsgroups()" ] }, { "cell_type": "markdown", "id": "27e33d0d-5bfb-49ce-82e7-d0e05d10c0e6", "metadata": { "id": "27e33d0d-5bfb-49ce-82e7-d0e05d10c0e6" }, "source": [ "For this notebook, we're making use the 20 newsgroups dataset. It is a text classification dataset with 20 classes. A decent score would be to reach 89% accuracy out of sample. For more details, read the description below:" ] }, { "cell_type": "code", "execution_count": 5, "id": "a9d84f34-b506-40c1-a9da-2d40c0a2feca", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "a9d84f34-b506-40c1-a9da-2d40c0a2feca", "outputId": "de9fd9e4-4b32-49e2-9284-08000c15c71e" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ ".. _20newsgroups_dataset:\n", "\n", "The 20 newsgroups text dataset\n", "------------------------------\n", "\n", "The 20 newsgroups dataset comprises around 18000 newsgroups posts on\n", "20 topics split in two subsets: one for training (or development)\n", "and the other one for testing (or for performance evaluation). The split\n", "between the train and test set is based upon a messages posted before\n", "and after a specific date.\n", "\n", "This module contains two loaders. The first one,\n", ":func:`sklearn.datasets.fetch_20newsgroups`,\n", "returns a list of the raw texts that can be fed to text feature\n", "extractors such as :class:`~sklearn.feature_extraction.text.CountVectorizer`\n", "with custom parameters so as to extract feature vectors.\n", "The second one, :func:`sklearn.datasets.fetch_20newsgroups_vectorized`,\n", "returns ready-to-use features, i.e., it is not necessary to use a feature\n", "extractor.\n", "\n", "**Data Set Characteristics:**\n", "\n", " ================= ==========\n", " Classes 20\n", " Samples total 18846\n", " Dimensionality 1\n", " Features text\n", " ================= ==========\n", "\n", "\n" ] } ], "source": [ "print(dataset.DESCR.split('Usage')[0])" ] }, { "cell_type": "code", "execution_count": 6, "id": "cec3f23b-7906-4804-a517-db17001c22e6", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cec3f23b-7906-4804-a517-db17001c22e6", "outputId": "8301ef90-6246-462a-9ceb-62eb481daf9f" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['alt.atheism',\n", " 'comp.graphics',\n", " 'comp.os.ms-windows.misc',\n", " 'comp.sys.ibm.pc.hardware',\n", " 'comp.sys.mac.hardware',\n", " 'comp.windows.x',\n", " 'misc.forsale',\n", " 'rec.autos',\n", " 'rec.motorcycles',\n", " 'rec.sport.baseball',\n", " 'rec.sport.hockey',\n", " 'sci.crypt',\n", " 'sci.electronics',\n", " 'sci.med',\n", " 'sci.space',\n", " 'soc.religion.christian',\n", " 'talk.politics.guns',\n", " 'talk.politics.mideast',\n", " 'talk.politics.misc',\n", " 'talk.religion.misc']" ] }, "metadata": {}, "execution_count": 6 } ], "source": [ "dataset.target_names" ] }, { "cell_type": "code", "execution_count": 7, "id": "061eb1d4-4805-4a7f-91fe-b64b43939905", "metadata": { "id": "061eb1d4-4805-4a7f-91fe-b64b43939905" }, "outputs": [], "source": [ "X = dataset.data\n", "y = dataset.target" ] }, { "cell_type": "code", "execution_count": 8, "id": "b491705e-95b0-4699-86b0-71128e7d54e3", "metadata": { "id": "b491705e-95b0-4699-86b0-71128e7d54e3" }, "outputs": [], "source": [ "X_train, X_test, y_train, y_test, = train_test_split(X, y, stratify=y, random_state=0)" ] }, { "cell_type": "code", "execution_count": 9, "id": "d4ec04ba-57fd-47bf-9528-1dc56fcad7ea", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "d4ec04ba-57fd-47bf-9528-1dc56fcad7ea", "outputId": "77d2d8c4-89c5-47c1-edc3-5d7309f53438" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['From: hilmi-er@dsv.su.se (Hilmi Eren)\\nSubject: Re: ARMENIA SAYS IT COULD SHOOT DOWN TURKISH PLANES (Henrik)\\nLines: 53\\nNntp-Posting-Host: alban.dsv.su.se\\nReply-To: hilmi-er@dsv.su.se (Hilmi Eren)\\nOrganization: Dept. of Computer and Systems Sciences, Stockholm University\\n\\n\\n \\n|> henrik@quayle.kpc.com writes:\\n\\n\\n|>\\tThe Armenians in Nagarno-Karabagh are simply DEFENDING their RIGHTS\\n|> to keep their homeland and it is the AZERIS that are INVADING their \\n|> territorium...\\n\\t\\n\\n\\tHomeland? First Nagarno-Karabagh was Armenians homeland today\\n\\tFizuli, Lacin and several villages (in Azerbadjan)\\n\\tare their homeland. Can\\'t you see the\\n\\tthe \"Great Armenia\" dream in this? With facist methods like\\n\\tkilling, raping and bombing villages. The last move was the \\n\\tblast of a truck with 60 kurdish refugees, trying to\\n\\tescape the from Lacin, a city that was \"given\" to the Kurds\\n\\tby the Armenians. \\n\\n\\n|> However, I hope that the Armenians WILL force a TURKISH airplane \\n|> to LAND for purposes of SEARCHING for ARMS similar to the one\\n|> that happened last SUMMER. Turkey searched an AMERICAN plane\\n|> (carrying humanitarian aid) bound to ARMENIA.\\n|>\\n\\n\\tDon\\'t speak about things you don\\'t know: 8 American Cargo planes\\n\\twere heading to Armenia. When the Turkish authorities\\n\\tannounced that they were going to search these cargo \\n\\tplanes 3 of these planes returned to it\\'s base in Germany.\\n\\t5 of these planes were searched in Turkey. The content of\\n\\tof the other 3 planes? Not hard to guess, is it? It was sure not\\n\\thumanitarian aid.....\\n\\n\\tSearch Turkish planes? You don\\'t know what you are talking about.\\n\\tTurkey\\'s government has announced that it\\'s giving weapons\\n\\tto Azerbadjan since Armenia started to attack Azerbadjan\\n\\tit self, not the Karabag province. So why search a plane for weapons\\n\\tsince it\\'s content is announced to be weapons? \\n\\n\\n\\n\\n\\n\\nHilmi Eren\\nDept. of Computer and Systems Sciences, Stockholm University\\nSweden\\nHilmi-er@dsv.su.se\\n\\n\\n\\n\\n',\n", " 'Subject: VHS movie for sale\\nFrom: koutd@hirama.hiram.edu (DOUGLAS KOU)\\nOrganization: Hiram College\\nNntp-Posting-Host: hirama.hiram.edu\\nLines: 13\\n\\nVHS movie for sale.\\n\\nDance with Wovies\\t($12.00)\\n\\nThe tape is new and just open, buyer pay shipping cost.\\nIf you are interested, please send your offer to\\nkoutd@hirama.hiram.edu\\n\\nthanks,\\n\\nDouglas Kou\\nHiram College\\n\\n']" ] }, "metadata": {}, "execution_count": 9 } ], "source": [ "X_train[:2]" ] }, { "cell_type": "markdown", "id": "ad5a8f51-98bc-4f09-b14d-5708a1788b80", "metadata": { "id": "ad5a8f51-98bc-4f09-b14d-5708a1788b80" }, "source": [ "## Prepare the training" ] }, { "cell_type": "markdown", "id": "9f7c0f71-4c62-4c3c-b411-02075d81169f", "metadata": { "id": "9f7c0f71-4c62-4c3c-b411-02075d81169f" }, "source": [ "We want to use a linear learning rate schedule that linearly decreases the learning rate during training." ] }, { "cell_type": "code", "execution_count": 10, "id": "acde3950-a9dc-434f-b76c-587e98fa8d35", "metadata": { "id": "acde3950-a9dc-434f-b76c-587e98fa8d35" }, "outputs": [], "source": [ "num_training_steps = MAX_EPOCHS * (len(X_train) // BATCH_SIZE + 1)\n", "\n", "def lr_schedule(current_step):\n", " factor = float(num_training_steps - current_step) / float(max(1, num_training_steps))\n", " assert factor > 0\n", " return factor" ] }, { "cell_type": "markdown", "id": "3207018e-f281-4fff-b394-b35d41694bad", "metadata": { "id": "3207018e-f281-4fff-b394-b35d41694bad" }, "source": [ "Next we wrap the BERT module itself inside a simple `nn.Module`. The only real work for us here is to load the pretrained model and to return the _logits_ from the model output. The rest of the outputs is not needed." ] }, { "cell_type": "code", "execution_count": 11, "id": "57e0924d-65d4-4fd0-af48-338acf40ec53", "metadata": { "id": "57e0924d-65d4-4fd0-af48-338acf40ec53" }, "outputs": [], "source": [ "class BertModule(nn.Module):\n", " def __init__(self, name, num_labels):\n", " super().__init__()\n", " self.name = name\n", " self.num_labels = num_labels\n", " \n", " self.reset_weights()\n", " \n", " def reset_weights(self):\n", " self.bert = AutoModelForSequenceClassification.from_pretrained(\n", " self.name, num_labels=self.num_labels\n", " )\n", " \n", " def forward(self, **kwargs):\n", " pred = self.bert(**kwargs)\n", " return pred.logits" ] }, { "cell_type": "markdown", "id": "ac6234ba", "metadata": { "id": "ac6234ba" }, "source": [ "### Tokenizer" ] }, { "cell_type": "markdown", "id": "37f08038", "metadata": { "id": "37f08038" }, "source": [ "We make use of `HuggingfacePretrainedTokenizer`, which is a wrapper that skorch provides to use the tokenizers from Hugging Face. In this instance, we use a tokenizer that was pretrained in conjunction with BERT. The tokenizer is automatically downloaded if not already present. More on Hugging Face tokenizers can be found [here](https://huggingface.co/docs/tokenizers/index)." ] }, { "cell_type": "markdown", "id": "87512ddc-8e1e-4db1-b203-f26e6e8f3730", "metadata": { "id": "87512ddc-8e1e-4db1-b203-f26e6e8f3730" }, "source": [ "## Training" ] }, { "cell_type": "markdown", "id": "82675bea-7f6e-470b-a468-e803a94cfcf3", "metadata": { "id": "82675bea-7f6e-470b-a468-e803a94cfcf3" }, "source": [ "### Putting it all togther" ] }, { "cell_type": "markdown", "id": "6a626328-b4a1-445c-9fad-5347263e889e", "metadata": { "id": "6a626328-b4a1-445c-9fad-5347263e889e" }, "source": [ "Now we can put together all the parts from above. There is nothing special going on here, we simply use an sklearn `Pipeline` to chain the `HuggingfacePretrainedTokenizer` and the neural net. Using skorch's `NeuralNetClassifier`, we make sure to pass the `BertModule` as the first argument and to set the number of labels based on `y_train`. The criterion is `CrossEntropyLoss` because we return the logits. Moreover, we make use of the learning rate schedule we defined above, and we add the `ProgressBar` callback to monitor our progress." ] }, { "cell_type": "code", "execution_count": 12, "id": "b798511a-c389-4a77-a77a-35ecc7c30e45", "metadata": { "id": "b798511a-c389-4a77-a77a-35ecc7c30e45" }, "outputs": [], "source": [ "pipeline = Pipeline([\n", " ('tokenizer', HuggingfacePretrainedTokenizer(TOKENIZER)),\n", " ('net', NeuralNetClassifier(\n", " BertModule,\n", " module__name=PRETRAINED_MODEL,\n", " module__num_labels=len(set(y_train)),\n", " optimizer=OPTMIZER,\n", " lr=LR,\n", " max_epochs=MAX_EPOCHS,\n", " criterion=CRITERION,\n", " batch_size=BATCH_SIZE,\n", " iterator_train__shuffle=True,\n", " device=DEVICE,\n", " callbacks=[\n", " LRScheduler(LambdaLR, lr_lambda=lr_schedule, step_every='batch'),\n", " ProgressBar(),\n", " ],\n", " )),\n", "])" ] }, { "cell_type": "markdown", "id": "be19feb6-aeaf-43f7-8409-1baebe9a4c77", "metadata": { "id": "be19feb6-aeaf-43f7-8409-1baebe9a4c77" }, "source": [ "Since we are using skorch, we could now take this pipeline to run a grid search or other kind of hyper-parameter sweep to figure out the best hyper-parameters for this model. E.g. we could try out a different BERT model or a different `max_length`." ] }, { "cell_type": "markdown", "id": "744c78c9-6941-4de9-a43c-d4f154bdd13e", "metadata": { "id": "744c78c9-6941-4de9-a43c-d4f154bdd13e" }, "source": [ "### Fitting" ] }, { "cell_type": "code", "execution_count": 13, "id": "3159ab90-1c5e-411f-937a-5ac9addbedbc", "metadata": { "id": "3159ab90-1c5e-411f-937a-5ac9addbedbc" }, "outputs": [], "source": [ "torch.manual_seed(0)\n", "torch.cuda.manual_seed(0)\n", "torch.cuda.manual_seed_all(0)\n", "np.random.seed(0)" ] }, { "cell_type": "code", "execution_count": 14, "id": "9e8d0e55-dd20-4f13-ac3f-4ff9b807cf33", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 860, "referenced_widgets": [ "7737eb5f4b0147fca1d57fcff3b94310", "731489d30da74c22a941c2c40eac2f9e", "24c355db75d6402fb3d592b979efd97d", "97ede09e513643dbbe06a7a495d0e338", "5e48eaee9fec42439607252b57abf573", "2032954a84d64dc88d011ff804fc159e", "f47d171dfb1a49b0be475751052b9dd6", "9c74749827474c259b3de86723fa458a", "01aebe710a0448f298b8ee7cf168f158", "e0957d3f9d4f41499eb8fd103a858941", "d963b1ec70254a83aa9b7a6158cee474", "a351265b57e748df9bd4e3385bd82df1", "6a9891abe74043488b320c0721180e85", "8431059e4ad847279ab27f799eb47e7e", "3919403b5b4a422ab9a2f8b17004a8f8", "f67d5799925545e8beb0a5e138dc338d", "0d4b749541ba470ebdaaa8276cf63519", "e4d9c0c46410454988e56e6bd374d0dd", "05527a2144d046ddaf80ba6ea7d05da4", "cb45242338134614a9383354bb5e3d15", "cc5eeb0ad24c46818273525abe40754c", "257643eab22642a5b6d3930836dacdc9", "4e281b399814454cb89e687b3cba3a76", "77ef0a692bcc48018722b6d428a9523f", "641592e304264892a87f030243c7bfaa", "9d7acf7bb42c4cd68db0f906429243c4", "d3ea83287e134c0383d5ea9ecc3bd66d", "ae73e7099644414eb26fe669ae30edc2", "5e1d2c6e1dd34078a2a65f6fdedb2120", "7c954494c5b743318fe151cb212ec143", "7fd494a95eb442b2a07f490c8475685f", "c7f268cf4e144fd9a78b7390f7906b12", "42286df0e74b475d8701e8538e9f13ab", "1d9803aa96634b1fb07c06805fc258f5", "69d65608f1e94268b74e1992ece58318", "3889afc887b3425b8a4fe11984a15825", "c6e07ba089694552997dc3b85992ff14", "3dd66f23558e49afa6c2d552c9bdc886", "fc6f07f74486405e8d012bf4a114c3d3", "d685d3ad9c2c46648b07e8c8c66268ce", "45f636f84742412bb9aaad4edf9a87aa", "789b6dde0c844dcdacf197bb1133115b", "ba3a2b60fbb8426c95a48eb4c1581fc2", "de7b7750d12a4f9cbc12f94c31abd3d1", "ee66c812582e4892b5cb77c12af5befe", "77e92bb077da47299f9d40665a3e3ee3", "3198d6e184d647889319b76f5b03e44c", "bf9383b18e1f491984984189c75f6c60", "9a761ec322c4412aaeb6609e84e0cadc", "a69f39b12d754641b34316abd6c20467", "5a0ab8b67c7a4e22a2b812103809fba9", "17d766cdf6034912bc2b367f0e5510b4", "3457c65dafb440ec87f701c6a8f62f92", "59feac5daa344bcd8ab6389bcd1157e0", "b1936e7d43164d92b07a5ddb9d2bfba0" ] }, "id": "9e8d0e55-dd20-4f13-ac3f-4ff9b807cf33", "outputId": "a9fa6552-a6ed-47b6-ef64-67e73c17c8cb" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "Downloading: 0%| | 0.00/28.0 [00:00[initialized](\n", " module_=BertModule(\n", " (bert): DistilBertForSequenceClassification(\n", " (distilbert): DistilBertModel(\n", " (embeddings): Embeddings(\n", " (word_embeddings): Embedding(30522, 768, padding_idx=0)\n", " (position_embeddin...\n", " (lin1): Linear(in_features=768, out_features=3072, bias=True)\n", " (lin2): Linear(in_features=3072, out_features=768, bias=True)\n", " (activation): GELUActivation()\n", " )\n", " (output_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", " )\n", " )\n", " )\n", " )\n", " (pre_classifier): Linear(in_features=768, out_features=768, bias=True)\n", " (classifier): Linear(in_features=768, out_features=20, bias=True)\n", " (dropout): Dropout(p=0.2, inplace=False)\n", " )\n", " ),\n", "))])" ] }, "metadata": {}, "execution_count": 14 } ], "source": [ "%time pipeline.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "id": "4385305a-af65-4c19-b5a3-809f933e7794", "metadata": { "id": "4385305a-af65-4c19-b5a3-809f933e7794" }, "source": [ "### Evaluation" ] }, { "cell_type": "code", "execution_count": 15, "id": "cc50f549-ce59-4440-a1e4-c5b30c4e8e4b", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cc50f549-ce59-4440-a1e4-c5b30c4e8e4b", "outputId": "404bc0ac-ed6e-4aa9-d063-7512bdf4266f" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "CPU times: user 26.8 s, sys: 68.5 ms, total: 26.9 s\n", "Wall time: 24.6 s\n" ] } ], "source": [ "%%time\n", "with torch.inference_mode():\n", " y_pred = pipeline.predict(X_test)" ] }, { "cell_type": "code", "execution_count": 16, "id": "eb04fc96-baa3-4592-a236-a022c96d1e2a", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "eb04fc96-baa3-4592-a236-a022c96d1e2a", "outputId": "b14fb031-e3de-458e-b3f7-43b4efeb0aa4" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "0.8985507246376812" ] }, "metadata": {}, "execution_count": 16 } ], "source": [ "accuracy_score(y_test, y_pred)" ] }, { "cell_type": "markdown", "id": "9799b81f-57d5-4239-ba7b-82cd2f76ed0d", "metadata": { "id": "9799b81f-57d5-4239-ba7b-82cd2f76ed0d" }, "source": [ "We can be happy with the results. We set ourselves the goal to reach or exceed 89% accuracy on the test set and we managed to do that." ] }, { "cell_type": "markdown", "id": "707bd2da-89ef-4684-a45c-7ccc44fb7596", "metadata": { "id": "707bd2da-89ef-4684-a45c-7ccc44fb7596" }, "source": [ "## Training with automatic mixed precision (AMP)" ] }, { "cell_type": "markdown", "id": "a8376427-6d9d-421b-9078-3357f041726d", "metadata": { "id": "a8376427-6d9d-421b-9078-3357f041726d" }, "source": [ "For this to work, you need:\n", "- A GPU that is capable of mixed precision training\n", "- The [accelerate library](https://huggingface.co/docs/accelerate/index), which you can install as: `python -m pip install 'accelerate>=0.11'`.\n", "- skorch version 0.12 or installed from the current master branch (`python -m pip install git+https://github.com/skorch-dev/skorch.git`)\n", "\n", "Again, we assume that you're familiar with the general concept of mixed precision training. For more information on how skorch integrates with accelerate, please consult the [skorch docs](https://skorch.readthedocs.io/en/latest/user/huggingface.html#accelerate)." ] }, { "cell_type": "code", "execution_count": 17, "id": "a6981f1d", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "a6981f1d", "outputId": "80f42c00-b23f-41f4-976d-8648239d8755" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "CompletedProcess(args=['python', '-m', 'pip', 'install', 'accelerate>=0.11'], returncode=0)" ] }, "metadata": {}, "execution_count": 17 } ], "source": [ "import subprocess\n", "\n", "subprocess.run(['python', '-m', 'pip', 'install', 'accelerate>=0.11'])" ] }, { "cell_type": "code", "execution_count": 18, "id": "c47aa1a6-f466-4a2c-84ab-034e4d6bdbcd", "metadata": { "id": "c47aa1a6-f466-4a2c-84ab-034e4d6bdbcd" }, "outputs": [], "source": [ "from accelerate import Accelerator\n", "from skorch.hf import AccelerateMixin" ] }, { "cell_type": "code", "execution_count": 19, "id": "63dfa5a1-c75b-4b61-93b6-cedc203ce9a6", "metadata": { "id": "63dfa5a1-c75b-4b61-93b6-cedc203ce9a6" }, "outputs": [], "source": [ "class AcceleratedNet(AccelerateMixin, NeuralNetClassifier):\n", " \"\"\"NeuralNetClassifier with accelerate support\"\"\"" ] }, { "cell_type": "code", "execution_count": 20, "id": "2eeac984-cc13-48e7-979f-9e9f208211df", "metadata": { "id": "2eeac984-cc13-48e7-979f-9e9f208211df" }, "outputs": [], "source": [ "accelerator = Accelerator(mixed_precision='fp16')" ] }, { "cell_type": "code", "execution_count": 21, "id": "783df165-369c-448b-8867-12a8eb85daa2", "metadata": { "id": "783df165-369c-448b-8867-12a8eb85daa2" }, "outputs": [], "source": [ "pipeline2 = Pipeline([\n", " ('tokenizer', HuggingfacePretrainedTokenizer(TOKENIZER)),\n", " ('net', AcceleratedNet( # <= changed\n", " BertModule,\n", " accelerator=accelerator, # <= changed\n", " module__name=PRETRAINED_MODEL,\n", " module__num_labels=len(set(y_train)),\n", " optimizer=OPTMIZER,\n", " lr=LR,\n", " max_epochs=MAX_EPOCHS,\n", " criterion=CRITERION,\n", " batch_size=BATCH_SIZE,\n", " iterator_train__shuffle=True,\n", " # device=DEVICE, # <= changed\n", " callbacks=[\n", " LRScheduler(LambdaLR, lr_lambda=lr_schedule, step_every='batch'),\n", " ProgressBar(),\n", " ],\n", " )),\n", "])" ] }, { "cell_type": "code", "execution_count": 22, "id": "123f7967-e2d5-4bff-877d-cef395af9f14", "metadata": { "id": "123f7967-e2d5-4bff-877d-cef395af9f14" }, "outputs": [], "source": [ "torch.manual_seed(0)\n", "torch.cuda.manual_seed(0)\n", "torch.cuda.manual_seed_all(0)\n", "np.random.seed(0)" ] }, { "cell_type": "code", "execution_count": 23, "id": "4f7e22db-eced-4360-ab2d-92ea9d1dbe79", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "4f7e22db-eced-4360-ab2d-92ea9d1dbe79", "outputId": "2e39249e-7072-4de2-e7b6-87a3375c4f89" }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_layer_norm.weight', 'vocab_projector.bias', 'vocab_projector.weight', 'vocab_transform.weight', 'vocab_layer_norm.bias', 'vocab_transform.bias']\n", "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['pre_classifier.weight', 'classifier.weight', 'pre_classifier.bias', 'classifier.bias']\n", "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ " epoch train_loss valid_acc valid_loss dur\n", "------- ------------ ----------- ------------ -------\n", " 1 \u001b[36m1.1681\u001b[0m \u001b[32m0.8397\u001b[0m \u001b[35m0.5697\u001b[0m 79.4182\n" ] }, { "output_type": "stream", "name": "stderr", "text": [] }, { "output_type": "stream", "name": "stdout", "text": [ " 2 \u001b[36m0.3479\u001b[0m \u001b[32m0.8863\u001b[0m \u001b[35m0.4011\u001b[0m 78.2318\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ " " ] }, { "output_type": "stream", "name": "stdout", "text": [ " 3 \u001b[36m0.1438\u001b[0m \u001b[32m0.8933\u001b[0m \u001b[35m0.3853\u001b[0m 77.5366\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\r" ] }, { "output_type": "execute_result", "data": { "text/plain": [ "Pipeline(steps=[('tokenizer',\n", " HuggingfacePretrainedTokenizer(tokenizer='distilbert-base-uncased')),\n", " ('net',\n", " [initialized](\n", " module_=BertModule(\n", " (bert): DistilBertForSequenceClassification(\n", " (distilbert): DistilBertModel(\n", " (embeddings): Embeddings(\n", " (word_embeddings): Embedding(30522, 768, padding_idx=0)\n", " (position_embeddings): Embedding(...\n", " (lin1): Linear(in_features=768, out_features=3072, bias=True)\n", " (lin2): Linear(in_features=3072, out_features=768, bias=True)\n", " (activation): GELUActivation()\n", " )\n", " (output_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", " )\n", " )\n", " )\n", " )\n", " (pre_classifier): Linear(in_features=768, out_features=768, bias=True)\n", " (classifier): Linear(in_features=768, out_features=20, bias=True)\n", " (dropout): Dropout(p=0.2, inplace=False)\n", " )\n", " ),\n", "))])" ] }, "metadata": {}, "execution_count": 23 } ], "source": [ "pipeline2.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 24, "id": "8fda6530-49b4-4423-b6fa-cf941dbf91db", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "8fda6530-49b4-4423-b6fa-cf941dbf91db", "outputId": "c840cf7b-dc4a-4e12-a985-1ab5bc17794a" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "CPU times: user 23.6 s, sys: 219 ms, total: 23.8 s\n", "Wall time: 24.7 s\n" ] } ], "source": [ "%%time\n", "with torch.inference_mode():\n", " y_pred = pipeline2.predict(X_test)" ] }, { "cell_type": "code", "execution_count": 25, "id": "ab4ce9f3-85ab-4b6a-af6c-b7926320d4b0", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ab4ce9f3-85ab-4b6a-af6c-b7926320d4b0", "outputId": "ac94dfbd-6e3c-4195-f3be-921c5279bd57" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "0.9070342877341817" ] }, "metadata": {}, "execution_count": 25 } ], "source": [ "accuracy_score(y_test, y_pred)" ] }, { "cell_type": "markdown", "id": "c4163616-3b62-46b5-976d-355378209fa7", "metadata": { "id": "c4163616-3b62-46b5-976d-355378209fa7" }, "source": [ "Using AMP, we could reduce our training and prediction time by half, while attaining the same scores." ] } ], "metadata": { "accelerator": "GPU", "colab": { "provenance": [] }, "gpuClass": "standard", "kernelspec": { "display_name": "base", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.13 (default, Mar 28 2022, 08:03:21) [MSC v.1916 64 bit (AMD64)]" }, "vscode": { "interpreter": { "hash": "bd97b8bffa4d3737e84826bc3d37be3046061822757ce35137ab82ad4c5a2016" } }, "widgets": { "application/vnd.jupyter.widget-state+json": { "7737eb5f4b0147fca1d57fcff3b94310": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_731489d30da74c22a941c2c40eac2f9e", "IPY_MODEL_24c355db75d6402fb3d592b979efd97d", "IPY_MODEL_97ede09e513643dbbe06a7a495d0e338" ], "layout": "IPY_MODEL_5e48eaee9fec42439607252b57abf573" } }, "731489d30da74c22a941c2c40eac2f9e": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2032954a84d64dc88d011ff804fc159e", "placeholder": "​", "style": "IPY_MODEL_f47d171dfb1a49b0be475751052b9dd6", "value": "Downloading: 100%" } }, "24c355db75d6402fb3d592b979efd97d": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9c74749827474c259b3de86723fa458a", "max": 28, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_01aebe710a0448f298b8ee7cf168f158", "value": 28 } }, "97ede09e513643dbbe06a7a495d0e338": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e0957d3f9d4f41499eb8fd103a858941", "placeholder": "​", "style": "IPY_MODEL_d963b1ec70254a83aa9b7a6158cee474", "value": " 28.0/28.0 [00:00<00:00, 1.38kB/s]" } }, "5e48eaee9fec42439607252b57abf573": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2032954a84d64dc88d011ff804fc159e": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f47d171dfb1a49b0be475751052b9dd6": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9c74749827474c259b3de86723fa458a": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "01aebe710a0448f298b8ee7cf168f158": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e0957d3f9d4f41499eb8fd103a858941": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d963b1ec70254a83aa9b7a6158cee474": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a351265b57e748df9bd4e3385bd82df1": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_6a9891abe74043488b320c0721180e85", "IPY_MODEL_8431059e4ad847279ab27f799eb47e7e", "IPY_MODEL_3919403b5b4a422ab9a2f8b17004a8f8" ], "layout": "IPY_MODEL_f67d5799925545e8beb0a5e138dc338d" } }, "6a9891abe74043488b320c0721180e85": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0d4b749541ba470ebdaaa8276cf63519", "placeholder": "​", "style": "IPY_MODEL_e4d9c0c46410454988e56e6bd374d0dd", "value": "Downloading: 100%" } }, "8431059e4ad847279ab27f799eb47e7e": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_05527a2144d046ddaf80ba6ea7d05da4", "max": 483, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_cb45242338134614a9383354bb5e3d15", "value": 483 } }, "3919403b5b4a422ab9a2f8b17004a8f8": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cc5eeb0ad24c46818273525abe40754c", "placeholder": "​", "style": "IPY_MODEL_257643eab22642a5b6d3930836dacdc9", "value": " 483/483 [00:00<00:00, 27.6kB/s]" } }, "f67d5799925545e8beb0a5e138dc338d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0d4b749541ba470ebdaaa8276cf63519": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e4d9c0c46410454988e56e6bd374d0dd": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "05527a2144d046ddaf80ba6ea7d05da4": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cb45242338134614a9383354bb5e3d15": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "cc5eeb0ad24c46818273525abe40754c": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "257643eab22642a5b6d3930836dacdc9": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4e281b399814454cb89e687b3cba3a76": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_77ef0a692bcc48018722b6d428a9523f", "IPY_MODEL_641592e304264892a87f030243c7bfaa", "IPY_MODEL_9d7acf7bb42c4cd68db0f906429243c4" ], "layout": "IPY_MODEL_d3ea83287e134c0383d5ea9ecc3bd66d" } }, "77ef0a692bcc48018722b6d428a9523f": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ae73e7099644414eb26fe669ae30edc2", "placeholder": "​", "style": "IPY_MODEL_5e1d2c6e1dd34078a2a65f6fdedb2120", "value": "Downloading: 100%" } }, "641592e304264892a87f030243c7bfaa": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7c954494c5b743318fe151cb212ec143", "max": 231508, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_7fd494a95eb442b2a07f490c8475685f", "value": 231508 } }, "9d7acf7bb42c4cd68db0f906429243c4": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c7f268cf4e144fd9a78b7390f7906b12", "placeholder": "​", "style": "IPY_MODEL_42286df0e74b475d8701e8538e9f13ab", "value": " 232k/232k [00:00<00:00, 204kB/s]" } }, "d3ea83287e134c0383d5ea9ecc3bd66d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ae73e7099644414eb26fe669ae30edc2": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5e1d2c6e1dd34078a2a65f6fdedb2120": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7c954494c5b743318fe151cb212ec143": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7fd494a95eb442b2a07f490c8475685f": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c7f268cf4e144fd9a78b7390f7906b12": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "42286df0e74b475d8701e8538e9f13ab": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "1d9803aa96634b1fb07c06805fc258f5": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_69d65608f1e94268b74e1992ece58318", "IPY_MODEL_3889afc887b3425b8a4fe11984a15825", "IPY_MODEL_c6e07ba089694552997dc3b85992ff14" ], "layout": "IPY_MODEL_3dd66f23558e49afa6c2d552c9bdc886" } }, "69d65608f1e94268b74e1992ece58318": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_fc6f07f74486405e8d012bf4a114c3d3", "placeholder": "​", "style": "IPY_MODEL_d685d3ad9c2c46648b07e8c8c66268ce", "value": "Downloading: 100%" } }, "3889afc887b3425b8a4fe11984a15825": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_45f636f84742412bb9aaad4edf9a87aa", "max": 466062, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_789b6dde0c844dcdacf197bb1133115b", "value": 466062 } }, "c6e07ba089694552997dc3b85992ff14": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ba3a2b60fbb8426c95a48eb4c1581fc2", "placeholder": "​", "style": "IPY_MODEL_de7b7750d12a4f9cbc12f94c31abd3d1", "value": " 466k/466k [00:01<00:00, 500kB/s]" } }, "3dd66f23558e49afa6c2d552c9bdc886": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "fc6f07f74486405e8d012bf4a114c3d3": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d685d3ad9c2c46648b07e8c8c66268ce": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "45f636f84742412bb9aaad4edf9a87aa": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "789b6dde0c844dcdacf197bb1133115b": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "ba3a2b60fbb8426c95a48eb4c1581fc2": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "de7b7750d12a4f9cbc12f94c31abd3d1": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ee66c812582e4892b5cb77c12af5befe": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_77e92bb077da47299f9d40665a3e3ee3", "IPY_MODEL_3198d6e184d647889319b76f5b03e44c", "IPY_MODEL_bf9383b18e1f491984984189c75f6c60" ], "layout": "IPY_MODEL_9a761ec322c4412aaeb6609e84e0cadc" } }, "77e92bb077da47299f9d40665a3e3ee3": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a69f39b12d754641b34316abd6c20467", "placeholder": "​", "style": "IPY_MODEL_5a0ab8b67c7a4e22a2b812103809fba9", "value": "Downloading: 100%" } }, "3198d6e184d647889319b76f5b03e44c": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_17d766cdf6034912bc2b367f0e5510b4", "max": 267967963, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_3457c65dafb440ec87f701c6a8f62f92", "value": 267967963 } }, "bf9383b18e1f491984984189c75f6c60": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_59feac5daa344bcd8ab6389bcd1157e0", "placeholder": "​", "style": "IPY_MODEL_b1936e7d43164d92b07a5ddb9d2bfba0", "value": " 268M/268M [00:03<00:00, 75.0MB/s]" } }, "9a761ec322c4412aaeb6609e84e0cadc": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a69f39b12d754641b34316abd6c20467": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5a0ab8b67c7a4e22a2b812103809fba9": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "17d766cdf6034912bc2b367f0e5510b4": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3457c65dafb440ec87f701c6a8f62f92": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "59feac5daa344bcd8ab6389bcd1157e0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b1936e7d43164d92b07a5ddb9d2bfba0": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 5 }