{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Image labeling vs classification models \n", "> Comparing the loss functions of label and classification models\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- categories: [models, labels, loss functions]\n", "- search_exclude: false" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The 'hello world' example for introducing deep learning based computer vision often involves classifying images as 🐶 or 🐱. An alternative approach to classifying images is to instead apply labels. This is usually introduced in the context of multi-label classification i.e. where an image can have more than one label. In this blog post I discuss some of the differences between these two approaches, specifically the difference in loss functions, and how these two approaches might work better depending on the application. The post starts with a conceptual overview of the differences between these two approaches, before showing the different loss functions and then moving to a practical example of training these two different types of model. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Image Classification vs Image Labeling " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In a classification model, an input can have only one label. This could be one of a few or one of a hundred, regardless of the number of potential classes, it is assumed that the input only belongs to one of these. With a model that applies labels this is not true an input can have one, multiple or no labels. \n", "\n", "### Sorting through family photos \n", "\n", "We can use an analogy to illustrate the difference between these two approaches. Let's say you were sorting through some old family photographs. You might \"classify\" the photos into one (and only one) of two photo albums, depending on whether they are black-and-white or colour. This would be comparable to using a classification model since each photo will go into exactly one of these two albums - a photo cannot be both simultaneously colour *and* black-and-white, and it cannot be neither colour *nor* black-and-white.\n", "\n", "You may at the same time also want to make it easier to find photos of particular people in your family. You could do this by assigning labels to each photo, indicating or \"tagging\" the family members who appear in the photo. In this case, a photo may have one label (a photo of your sister), more than one label (a photo of your sister *and* aunt), or it may have no labels (a photograph of a landscape taken on a holiday). This would be analogous to a multi-label classification model. \n", "\n", "The choice between using a model which performs classification or a model which assigns labels should be considered in relation to the role your model has. It is also useful to look a little bit more closely as how these different types of models work under the hood. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## CrossEntropyLoss vs BCEWithLogitsLoss\n", "\n", "When we create a model which does classifications or applies labels, the distinction, if using the same data is that they use different loss functions. \n", "\n", "A classification model will use a variant of Cross Entropy Loss whilst the label model will use a BCE with Logits Loss. We'll see how this is inferred by fastai below but fore now take my word for it...\n", "\n", "Let's take a look at a snippet of the Pytorch docs for each of these loss functions \n", "\n", "### CrossEntropyLoss \n", ">This criterion combines nn.LogSoftmax() and nn.NLLLoss() in one single class.\n", "It is useful when training a classification problem with C classes. If provided, the optional argument weight should be a 1D Tensor assigning weight to each of the classes. This is particularly useful when you have an unbalanced training set. [Read more](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#crossentropyloss) \n", "\n", "\n", "### BCEWithLogitsLoss\n", "> This loss combines a Sigmoid layer and the BCELoss in one single class. This version is more numerically stable than using a plain Sigmoid followed by a BCELoss as, by combining the operations into one layer, we take advantage of the log-sum-exp trick for numerical stability. [Read more](https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html?highlight=bcewithlogitsloss) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see what these do to some activations. First we'll import required packages" ] }, { "cell_type": "code", "execution_count": 174, "metadata": {}, "outputs": [], "source": [ "import torch.nn as nn\n", "import numpy as np\n", "import torch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exploring CrossEntropyLoss " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can create some fake activations. To start we'll just consider one output with three classes. We'll start with one to keep things simple for now." ] }, { "cell_type": "code", "execution_count": 175, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.9924, 0.8698, -0.0100]])" ] }, "execution_count": 175, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act = torch.randn((1, 3)) * 1\n", "one_act" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can think of these activations as probabilities for one of three classes. Let's see what these sum to. " ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(1.0875)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act.sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that these activations don't sum to 1. If we want our image input to belong to only one class, then the labels are not mutually exclusive of each other i.e. if one label probability is higher, another needs to be lower i.e. the probabilities need to add up to 1. Going back to the Pytorch explanation of `CrossEntropyLoss` we see that one component is `nn.LogSoftmax()`. What is particularly relevant here is that 'softmax' part. Let's see what this does to our activation" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[0.4525, 0.0381, 0.5093]])" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "softmax_acts = torch.softmax(one_act, dim=1)\n", "softmax_acts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can probably already see how this has changed the nature of these activations. Let's call sum on these outputs again. " ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(1.)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "softmax_acts.sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now have a sum of 1! We can now treat this as the probability of an input image belonging to a particular class. We could then call argmax to find out which class the model is most confident about and use that as our prediction. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([2])" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "softmax_acts.argmax(dim=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the potential issues that was mentioned about using a classification model was that it doesn't account for ambiguities in the labels very well. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### What is softmax doing?\n", "Digging into what `softmax` does in a little bit more detail will show what is going on here. \n", "\n", "First lets see what softmax actually does, I'll skip the LaTeX formula from Wikepedia because it makes is look much scarier than the [Python code example](https://en.wikipedia.org/wiki/Softmax_function#Example):" ] }, { "cell_type": "code", "execution_count": 180, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.02364054, 0.06426166, 0.1746813 , 0.474833 , 0.02364054,\n", " 0.06426166, 0.1746813 ])" ] }, "execution_count": 180, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = [1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0]\n", "np.exp(a) / np.sum(np.exp(a)) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is much easier for me to parse compared to the Greek. Let's look at the different parts. Working with one set of activations again:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1.1479, -1.3265, 1.2661]])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Starting from `np.exp(a)` we can do this in Pytorch like:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[3.1515, 0.2654, 3.5471]])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act.exp()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can convert the rest of the numpy code as follows " ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([6.9641])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act.exp().sum(dim=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Putting it all together we get" ] }, { "cell_type": "code", "execution_count": 181, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([1.])" ] }, "execution_count": 181, "metadata": {}, "output_type": "execute_result" } ], "source": [ "(one_act.exp() /one_act.exp().sum(dim=1)).sum(dim=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This seems to work as expected, i.e. we get the probabilities to sum to 1. To make it clearer what's going on though, it's useful to look a little more closely at the difference using `exp` makes. Let's import the standard python version of `exp` and check the docs. " ] }, { "cell_type": "code", "execution_count": 201, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
exp
[source]\n", "\n", "
exp
(x
)
Return e raised to the power of x.
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from math import exp\n", "doc(exp)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What difference does using the exponent make? We'll use a simple array of values to keep things simple " ] }, { "cell_type": "code", "execution_count": 208, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 4, 1])" ] }, "execution_count": 208, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = np.array([1,2,4,1])\n", "x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now if we want these to be converted to probabilities for different classes we need them to sum to 1. We could just do this by dividing each element by the sum. " ] }, { "cell_type": "code", "execution_count": 205, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.125, 0.25 , 0.5 , 0.125])" ] }, "execution_count": 205, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x/x.sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can confirm this add to 1" ] }, { "cell_type": "code", "execution_count": 206, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1.0" ] }, "execution_count": 206, "metadata": {}, "output_type": "execute_result" } ], "source": [ "(x/x.sum()).sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now this seems to work to get us probabilities for each class. Let's compare doing the same thing but using `exp` to create exponents of the inputs" ] }, { "cell_type": "code", "execution_count": 209, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.04031637, 0.10959126, 0.80977599, 0.04031637])" ] }, "execution_count": 209, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.exp(x)/np.sum(np.exp(x))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again we get an array of probabilities, let's confirm they add to one. " ] }, { "cell_type": "code", "execution_count": 189, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[0.4441, 0.3929, 0.1630]])" ] }, "execution_count": 189, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act.exp()/one_act.exp().sum(dim=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### So what is different here? \n", "Let's put the two arrays next to each other so we can compare the values for each index" ] }, { "cell_type": "code", "execution_count": 190, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array([0.04031637, 0.10959126, 0.80977599, 0.04031637]),\n", " array([0.125, 0.25 , 0.5 , 0.125]))" ] }, "execution_count": 190, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.exp(x)/ np.sum(np.exp(x)), (x/ x.sum())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Other than the difference in decimals, you will probably notice that when we use exponent, some labels for a class have been pushed much higher. Index `2` is `0.80` when we use `exp` and only `0.5` when we don't use the exponent. This is an important difference here. By using the magic properties of $e$ we 'push' one probability to be higher than the others. \n", "\n", "This property is useful when we have a clear distinction between classes. If we were predicting handwritten digits there (should) only be one correct answer. In this case having one class prediction being pushed much higher would be a good thing. \n", "\n", "If however, we have labels which are more ambiguous, this would be less of a desirable property. Even if we try and capture ambiguity by using the raw probabilities of the labels, rather than taking the `argmax` value, the numerical properties of the softmax function mean that it likely that one label value will be pushed higher than the others. \n", "\n", "We'll look at a practical example later on to illustrate this. Let's now quickly compare our other loss function " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exploring BCEWithLogitsLoss\n", "\n", "As a reminder\n", "> This loss combines a Sigmoid layer and the BCELoss in one single class.\n", "\n", "The part here that we are particularly interested in is the `Sigmoid`. Let's use `one_acts` again" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1.1479, -1.3265, 1.2661]])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "one_act" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As a reminder sigmoid function can be plotted as" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhNUlEQVR4nO3deXxV9Z3/8deHEEggkEAStgRI2ISAIBJAcMGNCozLWG3dF6SlONDaOu1PLTPavbVOR0dri4wibhR3xRW3uiuSsENYwhZCEhII2ffk+/sjwYkxwA3c5NybvJ+PRx5y7j259y3kvvnyPd9zjjnnEBGR4NfJ6wAiIuIfKnQRkXZChS4i0k6o0EVE2gkVuohIO9HZqzeOiYlxCQkJXr29iEhQSk1NPeici23uOc8KPSEhgZSUFK/eXkQkKJnZ3qM9pykXEZF24riFbmZLzCzXzDYd5XkzswfNLN3MNpjZ6f6PKSIix+PLCH0pMOMYz88Ehjd8zQX+fvKxRESkpY5b6M65j4H8Y+xyGfCkq/clEGVm/f0VUEREfOOPOfQ4YF+j7cyGx0REpA35o9CtmceaveKXmc01sxQzS8nLy/PDW4uIyBH+KPRMYGCj7Xggq7kdnXOLnXPJzrnk2Nhml1GKiMgJ8sc69BXAAjNbDkwGCp1z2X54XRGRoOacI7+0ipyiCnKLKsktruBAUSXjB0Vx9nD/D2qPW+hm9g/gXCDGzDKBe4DQhrCLgDeBWUA6UAbM9ntKEZEAVFVTx/6CcjIPl5F5uJz9h8vJKihnf0E52YUV5BRVUFVT963vu/Xcod4UunPumuM874D5fkskIhJAqmvryMgvY1deKbsPlrD7YBl7DpaSkV9GdmE5dY2OGIZ0Mvr1DGNAVBinDYyif1QY/XrWf/XpGUafHl2J7dGVsNCQVsnq2an/IiKBpLbOsftgCVtzitl+oIQdB4rZkVvC3kOlVNf+X2v36hZKQkx3Jib0YlB0PIN6d2Ngr3Die3ejb4+udA7x7gR8FbqIdDgV1bVsyylm4/5CNmcVsjmriG05xVQ2TI90Mhgc3Z1hfSKYntSXYbERDIntzpCYCCK7hXqc/uhU6CLSrjnnyMgvI3XvYdZmFLA+s4C07KKvR92R4aGMHtCTG84YzKj+PRnZvwdDYyNabVqkNanQRaRdqatzpOUUsWpXPl/tzidl72EOllQC0L1LCGPjo/jB2UMYGxfJmLhI4nuFY9bc6TTBR4UuIkFvz8FSPk0/yGfpB/l85yEKy6sBiO8VztnDY5gwuBfJCb0Y3qcHIZ3aR3k3R4UuIkGnorqWL3Ye4sNtuXy4PY+9h8oAGBAZxneS+jJlaDSTh0QTFxXucdK2pUIXkaBQUFbFu1sO8F7aAT7efpDy6lrCQjsxdWgMc85K5OzhsSREd2s30ycnQoUuIgHrcGkVb2/O4c2N2Xyx8xA1dY7+kWFcOSGeC0b14Ywh0UF58LK1qNBFJKCUV9XyzpYcVqzL4qPtedTUOQZHd+OH5wxh5ph+nBoX2aFH4ceiQhcRzznnWJNxmBdSM3l9fTbFlTX06xnGLWclcum4AYwe0FMl7gMVuoh4prCsmhfXZLLsqwzSc0sIDw1h1qn9uWJCHGckRtOpHa9IaQ0qdBFpc1uyilj6+W5eXZdFZU0d4wZG8ecrxjJrbH8iuqqWTpR+50SkTdTVOd5LO8Bjn+5m1e58wkI78d3T47n+jEGMHhDpdbx2QYUuIq2qsqaWV9bu55GPd7Err5S4qHB+OWskVyUPCujrogQjFbqItIqK6lqWf5XBoo92kVNUwegBPXnwmvHMGtPP0ysStmcqdBHxq4rqWp5ZlcGij3aSV1zJpMTe3Pe9sZw1LEYrVVqZCl1E/KKmto4XUjP5n/d3kF1YwdSh0Tx0zXjOGBLtdbQOQ4UuIifFOcd7abn88a00duWVctrAKP7yvXFMHRbjdbQOR4UuIids0/5Cfvv6FlbtzmdIbHcW3zCB6Ul9NbXiERW6iLRYfmkV963cxvLVGfTq1oXfXjaaqycNIlQHOz2lQhcRn9XVOZZ9lcF9K7dRUlnD7KmJ/HT6cHqGaflhIFChi4hPtuYUcddLG1mbUcCUIdH8+rLRjOjbw+tY0ogKXUSOqaK6lgff38Hij3fRMzyU+68ax7+eFqd58gCkQheRo1qbcZhfvLCB9NwSrpwQz8JZo+jVvYvXseQoVOgi8i2VNbXc/+4OFn+8k749w3jilklMGxHrdSw5DhW6iHzD9gPF3LZ8HWnZRVyVPJCFF4/SQc8goUIXEaD+BKEnPt/DH9/aSkTXzjx6YzIXJvX1Opa0gApdRCgoq+Lnz2/gvbQDnHdKLH++chyxPbp6HUtaSIUu0sGl7j3MT/6xltziCv7z4iRuOTNBK1iClApdpINyzvH4Z3v4w5tp9I8K44V5Uxk3MMrrWHISVOgiHVBZVQ13vbSRV9dlceGovvzl++OIDNeBz2CnQhfpYDIOlTH3qRS2HSjmFxedwq3ThupmzO2ET1fSMbMZZrbNzNLN7M5mno80s9fMbL2ZbTaz2f6PKiIn64udh7js4U/JLqxg6exJzD9vmMq8HTluoZtZCPAwMBNIAq4xs6Qmu80HtjjnxgHnAn8xM51OJhJAlq3K4IbHVhEd0ZVX55+pE4XaIV+mXCYB6c65XQBmthy4DNjSaB8H9LD6Q+MRQD5Q4+esInICauscf3gzjcc+3c25p8Ty4DXjdaJQO+VLoccB+xptZwKTm+zzV2AFkAX0AK5yztX5JaGInLDyqlp++uxaVm4+wM1TE/jPi5MI0RRLu+VLoTf3p++abF8ErAPOB4YC75rZJ865om+8kNlcYC7AoEGDWhxWRHx3qKSSOU+ksD6zgLsvTuKWsxK9jiStzJeDopnAwEbb8dSPxBubDbzk6qUDu4GRTV/IObfYOZfsnEuOjdX8nUhr2ZdfxpWLvmBrThGLrp+gMu8gfCn01cBwM0tsONB5NfXTK41lABcAmFlf4BRglz+Diohv0rKLuOLvn5NfWsUzP5jMRaP7eR1J2shxp1ycczVmtgBYCYQAS5xzm81sXsPzi4DfAkvNbCP1UzR3OOcOtmJuEWnG6j353LJ0Nd27dOb5eVN0R6EOxqcTi5xzbwJvNnlsUaNfZwHf8W80EWmJT3bk8cMnUxgQGc6TcyYR36ub15GkjelMUZF24J3NOSxYtpYhsd15as5kXSmxg1KhiwS519Zn8dNn1zEmLpInZk8kqpvO6euoVOgiQezVdfv52bPrSB7cm8duTqaHThjq0FToIkHqlbX7uf25dUxM6M2SmyfSvas+zh2dfgJEgtCRMp+UWF/m3brooywqdJGg88aGbG5/bh2TE6NZcvNEwruEeB1JAoRPl88VkcDwzuYcblu+ltMH9eKxm5NV5vINKnSRIPHR9jwWLFvL6LhIHp+taRb5NhW6SBBI2ZPPj55KYWifCJ6cPUmrWaRZKnSRALclq4jZS1czIDKcp+ZMIrKbylyap0IXCWC7D5Zy45KviOjamad+MJmYCJ0BKkenQhcJULlFFdzw2CrqnOOpOZOJiwr3OpIEOBW6SAAqqqjmpsdXk19axdLZExnWJ8LrSBIEVOgiAaayppZ5T6Wy40Axi66fwNj4KK8jSZDQuieRAFJX5/j58xv4fOch7r9qHOeM0J29xHcaoYsEkHtXbuW19VncOXMkl4+P9zqOBBkVukiAePrLvTzy0S6uP2MQPzpniNdxJAip0EUCwAdbD3D3q5s4f2QffnXJaMzM60gShFToIh7bklXEgmVrSRrQk4euGU/nEH0s5cToJ0fEQ7lFFfzgidVEhofy2E26prmcHP30iHikvKqWHz6ZQkF5Nc/Pm0LfnmFeR5Igp0IX8UD98sT1bNhfyCPXT2D0gEivI0k7oCkXEQ88+MEO3tiYzZ0zRvKd0f28jiPthApdpI29tTGbB97bwRWnxzNXyxPFj1ToIm1oc1Yhtz+3nvGDovj95WO0PFH8SoUu0kYOllQy98lUorqF8sgNEwgL1e3jxL90UFSkDVTX1jH/mTUcLKnkhXlT6dNDK1rE/1ToIm3g92+ksWp3PvdfNY5T47WiRVqHplxEWtnzKftY+vke5pyVqAtuSatSoYu0og2ZBSx8ZRNTh0Zz18yRXseRdk6FLtJKDpVUMu+pVGIjuvLXa0/XNVqk1WkOXaQV1NTW8ZPlazlYWsWL86bSu3sXryNJB+DTkMHMZpjZNjNLN7M7j7LPuWa2zsw2m9lH/o0pElz+653tfJZ+iN/96xgdBJU2c9wRupmFAA8D04FMYLWZrXDObWm0TxTwN2CGcy7DzPq0Ul6RgPf2phwWfbSTaycP4vvJA72OIx2ILyP0SUC6c26Xc64KWA5c1mSfa4GXnHMZAM65XP/GFAkOu/JK+Pnz6xk3MIp7LknyOo50ML4Uehywr9F2ZsNjjY0AepnZh2aWamY3+iugSLAoq6rh1qfXEBpi/O260+naWWeCStvy5aBocxebcM28zgTgAiAc+MLMvnTObf/GC5nNBeYCDBo0qOVpRQKUc46FL29ie24xS2dPIi4q3OtI0gH5MkLPBBpPBMYDWc3s87ZzrtQ5dxD4GBjX9IWcc4udc8nOueTY2NgTzSwScJ5ZlcHLa/fz0wtGMG2EfrbFG74U+mpguJklmlkX4GpgRZN9XgXONrPOZtYNmAyk+TeqSGDamFnIb17bwjkjYvnx+cO8jiMd2HGnXJxzNWa2AFgJhABLnHObzWxew/OLnHNpZvY2sAGoAx51zm1qzeAigaCwrJp/W5ZKdEQXHrjqNDp10uVwxTs+nVjknHsTeLPJY4uabN8H3Oe/aCKBzTnHz19YT3ZBBc/+aIpOHhLP6VxkkRP06Ce7eXfLAe6aNYoJg3t5HUdEhS5yIlL3Hubet7cyY3Q/bjkzwes4IoAKXaTFDpdW8eNla+gfFca9V47VbeQkYOjiXCItUFfn+Pfn13OwpIoXb51KZHio15FEvqYRukgL/O8nu/hgay4L/2WULrolAUeFLuKj1L35/HnlNmad2o8bpwz2Oo7It6jQRXxQP2++lriocP50hebNJTBpDl3kOJxz/LzRvHnPMM2bS2DSCF3kOB79ZDfvb83ll7NGat5cApoKXeQY1mbUrze/aHRfbpqa4HUckWNSoYscRWFZNQuWraVfZBh/vnKc5s0l4GkOXaQZzjnueHEDB4oqeH7eFK03l6CgEbpIM578Yi9vb87hjhkjGT9I12mR4KBCF2li0/5Cfv9GGueP7MMPzk70Oo6Iz1ToIo0UV1SzYNkaoiO68Jfvad5cgovm0EUaOOf45cub2He4nOVzz6CXrm8uQUYjdJEGz67ex2vrs7h9+ggmJvT2Oo5Ii6nQRYBtOcXcs2IzZw2L4dZpQ72OI3JCVOjS4ZVV1TB/2Rp6hIVyv+4LKkFMc+jS4d396mZ25pXw9JzJxPbo6nUckROmEbp0aC+mZvJCaiY/Pn84Zw6L8TqOyElRoUuHlZ5bzH+8solJib257YLhXscROWkqdOmQyqtqmf/MWsK7hPDg1eMJ0by5tAOaQ5cO6VcrNrPtQDFP3DKJfpFhXscR8QuN0KXDeWXtfp5N2ce/nTuUaSNivY4j4jcqdOlQ0nNL+OXLG5mY0Ivbp4/wOo6IX6nQpcOonzdfQ1hoCA9eM57OIfrxl/ZFc+jSYRyZN186eyL9I8O9jiPidxqiSIfw0ppMnk3Zx/zzhnLuKX28jiPSKlTo0u7tOFDMwpfr15v/7ELNm0v7pUKXdq20soZbn1lD964h/FXz5tLOaQ5d2i3nHAtf3siuhuu09Omp9ebSvvk0XDGzGWa2zczSzezOY+w30cxqzexK/0UUOTHLvsrglXVZ/OzCEUzVdVqkAzhuoZtZCPAwMBNIAq4xs6Sj7HcvsNLfIUVaakNmAb9esYVpI2KZf94wr+OItAlfRuiTgHTn3C7nXBWwHLismf1+DLwI5Poxn0iLFZRVcevTa4jt0VXXN5cOxZdCjwP2NdrObHjsa2YWB1wOLPJfNJGWq6tz/PTZdeQVV/K3606nt+4LKh2IL4Xe3PDGNdl+ALjDOVd7zBcym2tmKWaWkpeX52NEEd899EE6H27L4+5Lkhg3MMrrOCJtypdVLpnAwEbb8UBWk32SgeVmBhADzDKzGufcK413cs4tBhYDJCcnN/1LQeSkfLgtlwfe387l4+O4bvIgr+OItDlfCn01MNzMEoH9wNXAtY13cM4lHvm1mS0FXm9a5iKtKeNQGbctX8cpfXvwh8tPpWFwIdKhHLfQnXM1ZraA+tUrIcAS59xmM5vX8LzmzcVT5VW1zHs6Feccj9wwgfAuIV5HEvGETycWOefeBN5s8lizRe6cu/nkY4n4xjnHwlc2kpZTxJKbJjI4urvXkUQ8o/OgJag9+cVeXlqzn9suGM55I3XRLenYVOgStL7YeYjfvL6FC0f15Sfn6ybPIip0CUr7C8qZv2wNCdHduP+qcTp5SAQVugShiupafvRUCtU1dSy+MZkeYaFeRxIJCLraogQV5xy/eGEDm7OKePTGZIbGRngdSSRgaIQuQeVvH+7ktfVZ/OKiU7hgVF+v44gEFBW6BI13Nudw38ptXHbaAG6dNtTrOCIBR4UuQWFrThE/e3Yd4+IjufeKsToTVKQZKnQJeHnFlcxZmkJEWGceuSGZsFCdCSrSHB0UlYBWUV3L3KdSyC+t4vl5U+gXqdvIiRyNCl0C1pEVLWszClh0/QTGxEV6HUkkoGnKRQLW/e9u57X1WdwxYyQzxvTzOo5IwFOhS0B6bvU+HvwgnauSBzJv2hCv44gEBRW6BJxPduTxy5c3cs6IWH53+RitaBHxkQpdAkpadhG3Pr2GYX0iePja8YSG6EdUxFf6tEjAyDxcxs2Pf0VE1848PnuirtEi0kIqdAkIh0uruHHJV5RX1fLknEn0jwz3OpJI0NGyRfFceVUttzyxmszD5Tw9ZzIj+vbwOpJIUNIIXTxVVVPHvz2Tyvp9BTx49XgmJfb2OpJI0NIIXTxTW+f49+fX889tefzxu6dqrbnISdIIXTzhnOPuVzfx2vos7pw5kmsmDfI6kkjQU6FLm3PO8eeV23hmVQbzpg1lni6FK+IXKnRpcw++n87fP9zJtZMHcceMU7yOI9JuqNClTT3y0U7uf287V06I53eX6SxQEX9SoUubefyz3fzxra1cMm4A914xlk6dVOYi/qRVLtImlny6m9+8voUZo/vx398fR4jKXMTvVOjS6h79ZBe/eyONGaP78ZCuzyLSavTJklZ1pMxnjlGZi7Q2jdClVTjneOiDdP773e38y6n9eeDq01TmIq1MhS5+55zjT29v5ZGPdnHF6fHce8WpdFaZi7Q6Fbr4VW2d454Vm3j6ywxuOGMwv750tFaziLQRFbr4TWVNLbc/u543Nmbzo2lDuHPGSK0zF2lDPv072MxmmNk2M0s3szubef46M9vQ8PW5mY3zf1QJZCWVNdyydDVvbMxm4axR3DVzlMpcpI0dd4RuZiHAw8B0IBNYbWYrnHNbGu22G5jmnDtsZjOBxcDk1ggsgSe3qIJbnlhNWnYxf/neOK6YEO91JJEOyZcpl0lAunNuF4CZLQcuA74udOfc5432/xLQJ7qD2H6gmNmPr+ZwWRWP3pjMeSP7eB1JpMPypdDjgH2NtjM59uh7DvDWyYSS4PBZ+kHmPZVKWJcQnvvRFMbERXodSaRD86XQm5sIdc3uaHYe9YV+1lGenwvMBRg0SNe/DmbPrNrLPa9uZkhsdx6fPYm4KN0DVMRrvhR6JjCw0XY8kNV0JzMbCzwKzHTOHWruhZxzi6mfXyc5ObnZvxQksNXU1vHb17fwxBd7mTYiloeuHU/PsFCvY4kIvhX6amC4mSUC+4GrgWsb72Bmg4CXgBucc9v9nlICQn5pFT/5x1o+TT/ID89O5M6Zo3SRLZEActxCd87VmNkCYCUQAixxzm02s3kNzy8C7gaigb81LFWrcc4lt15saWsbMwuZ93QqeSWV/PnKsXw/eeDxv0lE2pQ5583MR3JysktJSfHkvaVlnkvZx3+8sonYiK78/frTGRsf5XUkkQ7LzFKPNmDWmaJyVGVVNdz96mZeSM3krGExPHjNeHp37+J1LBE5ChW6NGtbTjHzl61hZ14JP7lgOLddMFzz5SIBToUu3+Cc4+lVGfz+jS1EdA3l6TmTOXNYjNexRMQHKnT5Wl5xJXe8uIEPtuZyzohY/ut7Y+nTI8zrWCLiIxW6APD2phwWvryR4soafnVJEjdOSdBlb0WCjAq9g8svreKeFZt5bX0Wowf05B9XncaIvj28jiUiJ0CF3kE553hjYza/WrGZwvJqbp8+glvPHarbxIkEMRV6B7Qvv4y7X93EP7flcWpcJE/Nmcyo/j29jiUiJ0mF3oFU1tTy2Ke7eej9dMzgPy9O4qYpg3W/T5F2QoXeQXy4LZdfv7aF3QdLmZ7Ul19dOlpXSBRpZ1To7dyOA8X88a2tfLA1lyEx3XnilklMGxHrdSwRaQUq9HYqr7iSB97bzvLV++jWJYS7Zo5k9pmJdOms6RWR9kqF3s4UllWz+JOdLPl0D9W1ddxwxmB+csFwXYNFpANQobcTRRXVPPHZHv73k10UVdRw6bgB/Gz6CBJjunsdTUTaiAo9yBWUVfH4Z3tY8tluiitquHBUH26ffgpJA7QMUaSjUaEHqf0F5Tz2yW6Wr86grKqWi0b35cfnD9eNmkU6MBV6kFm3r4DHP9vN6xuyMeCScQOYe84QnRgkIir0YFBRXcvbm3JY+vke1u0rIKJrZ26aksCcsxO1llxEvqZCD2C78kr4x1cZvJCayeGyahJjuvOrS5K4MnkgEV31Ryci36RWCDBFFdW8sSGbF1IzSd17mM6djOlJfblu8mCmDo3WJW1F5KhU6AGgorqWD7flsWL9ft5Py6Wypo5hfSK4c+ZIvjs+jj49dZMJETk+FbpHKqpr+Xh7Hm9tyuG9tAMUV9QQE9GFqycO5F/Hx3HawCjMNBoXEd+p0NtQfmkV/9yay3tpB/h4ex6lVbVEhody0eh+XDpuAFOHRuvKhyJywlTorai2zrFpfyEfbsvjo+25rNtXQJ2Dvj27culpccwc048pQ6N1UwkR8QsVuh8559iZV8qXuw7xWfpBPt95iMLyasxgbFwkC84fzoWj+jBmQKQOboqI36nQT0J1bR1p2UWk7DlMyt58vtqdz8GSKgAGRIbxnaS+nDU8hrOGxRAd0dXjtCLS3qnQfeScI/NwORv3F7JuXwHr9hWwMbOQ8upaoL7Azx4ey+TE3kweEk1CdDcd1BSRNqVCb0ZVTR0780rYmlNEWnYxW7KK2JRVSEFZNQBdQjqRNKAnV00cSHJCL04f1IsBOmNTRDzWoQu9orqW3QdL2ZlXQnpuCTtyS9ieU8zug6XU1DkAunTuxIi+Ecwc048xcZGMGRDJqP49daMIEQk47b7QC8uryTxcxr78MvYeKiMjv4w9h0rZc7CMrMJyXH1vYwYDe3VjRN8ILkzqy8h+PRjVvydDYrprKaGIBIWgLvTSyhpyiyvJLiznQFEFOYWVZBWUk11Yzv6CCjIPl1FcUfON74nqFkpCdHcmJfYmIbo7Q2K7M6xPBIkx3QkLDfHo/0RE5OQFXaH/c2suv3l9C7lFFZRW1X7r+cjwUAZEhTMgMoxJCb2I79WNuF7hDOrdjYG9uxEZHupBahGR1udToZvZDOB/gBDgUefcn5o8bw3PzwLKgJudc2v8nBWoH2En9e/JuafE0qdHGH16dKV/ZBj9Gr66dQm6v6NERPziuO1nZiHAw8B0IBNYbWYrnHNbGu02Exje8DUZ+HvDf/1u/KBePHxdr9Z4aRGRoObL0b5JQLpzbpdzrgpYDlzWZJ/LgCddvS+BKDPr7+esIiJyDL4Uehywr9F2ZsNjLd1HRERakS+F3tzpju4E9sHM5ppZipml5OXl+ZJPRER85EuhZwIDG23HA1knsA/OucXOuWTnXHJsbGxLs4qIyDH4UuirgeFmlmhmXYCrgRVN9lkB3Gj1zgAKnXPZfs4qIiLHcNxVLs65GjNbAKykftniEufcZjOb1/D8IuBN6pcsplO/bHF260UWEZHm+LRo2zn3JvWl3fixRY1+7YD5/o0mIiItoYuUiIi0E+bctxajtM0bm+UBe0/w22OAg36M4y+BmgsCN5tytYxytUx7zDXYOdfsqhLPCv1kmFmKcy7Z6xxNBWouCNxsytUyytUyHS2XplxERNoJFbqISDsRrIW+2OsARxGouSBwsylXyyhXy3SoXEE5hy4iIt8WrCN0ERFpQoUuItJOBH2hm9nPzcyZWYzXWQDM7LdmtsHM1pnZO2Y2wOtMAGZ2n5ltbcj2splFeZ0JwMy+Z2abzazOzDxfXmZmM8xsm5mlm9mdXuc5wsyWmFmumW3yOssRZjbQzP5pZmkNf4a3eZ0JwMzCzOwrM1vfkOvXXmdqzMxCzGytmb3u79cO6kI3s4HU30kpw+ssjdznnBvrnDsNeB242+M8R7wLjHHOjQW2A3d5nOeITcB3gY+9DtLo7lwzgSTgGjNL8jbV15YCM7wO0UQN8O/OuVHAGcD8APn9qgTOd86NA04DZjRcNDBQ3AaktcYLB3WhA/cD/49mrr3uFedcUaPN7gRINufcO865mobNL6m/xLHnnHNpzrltXudo4MvduTzhnPsYyPc6R2POuewj9w52zhVTX1Ke39im4c5pJQ2boQ1fAfE5NLN44F+AR1vj9YO20M3sUmC/c26911maMrPfm9k+4DoCZ4Te2C3AW16HCEC689YJMrMEYDywyuMowNfTGuuAXOBd51xA5AIeoH4QWtcaL+7T1Ra9YmbvAf2aeWoh8EvgO22bqN6xcjnnXnXOLQQWmtldwALgnkDI1bDPQur/qfxMW2TyNVeA8OnOW/JNZhYBvAj8tMm/UD3jnKsFTms4VvSymY1xznl6/MHMLgZynXOpZnZua7xHQBe6c+7C5h43s1OBRGC9mUH99MEaM5vknMvxKlczlgFv0EaFfrxcZnYTcDFwgWvDExBa8PvlNZ/uvCX/x8xCqS/zZ5xzL3mdpynnXIGZfUj98QevDyifCVxqZrOAMKCnmT3tnLveX28QlFMuzrmNzrk+zrkE51wC9R/E09uizI/HzIY32rwU2OpVlsbMbAZwB3Cpc67M6zwBype7c0kDqx9NPQakOef+2+s8R5hZ7JFVXGYWDlxIAHwOnXN3OefiGzrrauADf5Y5BGmhB7g/mdkmM9tA/ZRQQCzlAv4K9ADebVhSueh439AWzOxyM8sEpgBvmNlKr7I0HDQ+cneuNOA559xmr/I0Zmb/AL4ATjGzTDOb43Um6kecNwDnN/xMrWsYfXqtP/DPhs/gaurn0P2+RDAQ6dR/EZF2QiN0EZF2QoUuItJOqNBFRNoJFbqISDuhQhcRaSdU6CIi7YQKXUSknfj/EjrZdogmficAAAAASUVORK5CYII=\n", "text/plain": [ "epoch | \n", "train_loss | \n", "valid_loss | \n", "accuracy | \n", "f1_score | \n", "time | \n", "
---|---|---|---|---|---|
0 | \n", "1.023169 | \n", "0.786303 | \n", "0.785714 | \n", "0.769231 | \n", "00:03 | \n", "
1 | \n", "0.721281 | \n", "0.576258 | \n", "0.821429 | \n", "0.814815 | \n", "00:03 | \n", "
2 | \n", "0.477446 | \n", "0.339626 | \n", "0.821429 | \n", "0.782609 | \n", "00:04 | \n", "
3 | \n", "0.423173 | \n", "0.331097 | \n", "0.821429 | \n", "0.782609 | \n", "00:03 | \n", "
4 | \n", "0.351390 | \n", "0.239433 | \n", "0.857143 | \n", "0.818182 | \n", "00:03 | \n", "
epoch | \n", "train_loss | \n", "valid_loss | \n", "f1_score | \n", "time | \n", "
---|---|---|---|---|
0 | \n", "0.979886 | \n", "1.107939 | \n", "0.520422 | \n", "00:03 | \n", "
1 | \n", "0.742499 | \n", "0.530855 | \n", "0.749681 | \n", "00:03 | \n", "
2 | \n", "0.540384 | \n", "0.294314 | \n", "0.858553 | \n", "00:03 | \n", "
3 | \n", "0.428714 | \n", "0.197953 | \n", "0.874603 | \n", "00:03 | \n", "
4 | \n", "0.358649 | \n", "0.157847 | \n", "0.973684 | \n", "00:03 | \n", "
epoch | \n", "train_loss | \n", "valid_loss | \n", "f1_score | \n", "time | \n", "
---|---|---|---|---|
0 | \n", "0.746691 | \n", "0.450183 | \n", "0.655556 | \n", "00:03 | \n", "
1 | \n", "0.572374 | \n", "0.352565 | \n", "0.843939 | \n", "00:03 | \n", "
2 | \n", "0.477375 | \n", "0.328633 | \n", "0.856250 | \n", "00:03 | \n", "
3 | \n", "0.359104 | \n", "0.320807 | \n", "0.841642 | \n", "00:03 | \n", "
4 | \n", "0.306263 | \n", "0.327382 | \n", "0.860795 | \n", "00:03 | \n", "