{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# CNN Tutorial on Fashion MNIST Data Set\n",
"\n",
"\n",
"This turorial gives a breif intro on using CNN for train and prediction (i.e. inference)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Let's Load Some Packages"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Plots.GRBackend()"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using MLDatasets\n",
"using NumNN\n",
"using Plots\n",
"gr()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Temp for ProgressMeter.jl Package\n",
"\n",
"**Uncomment the following line if you run this code for the first time***"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# ] add https://github.com/timholy/ProgressMeter.jl.git ;"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"using ProgressMeter\n",
"ProgressMeter.ijulia_behavior(:clear);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load the Train/Test Data/Labels"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"X_train, Y_train = FashionMNIST.traindata(Float64);\n",
"X_test, Y_test = FashionMNIST.testdata(Float64);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Let's Prepare the data/labels\n",
"\n",
"Since the shape of the MNIST data is `(28,28,size)` and to use it in CNN 2D it must be as 4D Array"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"X_train = reshape(X_train, (size(X_train)[1:2]..., 1, size(X_train)[end]))\n",
"X_test = reshape(X_test, (size(X_test)[1:2]...,1,size(X_test)[end]))\n",
"\n",
"Y_train = oneHot(Y_train)\n",
"Y_test = oneHot(Y_test);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Define the Layers"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"X_Input = Input(X_train)\n",
"X = Conv2D(10, (3,3))(X_Input)\n",
"X = BatchNorm(dim=3)(X) #to normalize across the channels\n",
"X = Activation(:relu)(X)\n",
"X = MaxPool2D((2,2))(X)\n",
"X = Conv2D(20, (5,5))(X)\n",
"X = BatchNorm(dim=3)(X)\n",
"X = Activation(:relu)(X)\n",
"X = AveragePool2D((3,3))(X)\n",
"X = Flatten()(X)\n",
"X_Output = FCLayer(10, :softmax)(X);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Another way when there is no side branches is to use the `chain` function as follows:\n",
"\n",
"```julia\n",
"X_Input, X_Ouput = chain(X_train,[Conv2D(10, (3,3)),\n",
" BatchNorm(dim=3),\n",
" Activation(:relu),\n",
" MaxPool2D((2,2)),\n",
" Conv2D(20, (5,5)),\n",
" BatchNorm(dim=3),\n",
" Activation(:relu),\n",
" AveragePool2D((3,3)),\n",
" Flatten(),\n",
" FCLayer(10,:softmax)]);\n",
"```\n",
"\n",
"`chain` returns a `Tuple` of two pointers of the Input `Layer` and Output `Layer`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Let's Define Our Model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This will also initialize the `Layer`s' parameters"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"model = Model(X_train,Y_train,X_Input,X_Output, 0.005; optimizer=:adam);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Let's use `predict` to see the current Accuracy\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:00:15\u001b[39m\n",
"\u001b[34m Instances 10000: 10000\u001b[39m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"The accuracy of Test Data before the training process 0.1278\n",
"The cost of Test Data before the training process 2.4328\n"
]
}
],
"source": [
"TestP = predict(model, X_test, Y_test);\n",
"\n",
"println()\n",
"println(\"The accuracy of Test Data before the training process $(round(TestP[:accuracy], digits=4))\")\n",
"println(\"The cost of Test Data before the training process $(round(TestP[:cost], digits=4))\")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:00:22\u001b[39m\n",
"\u001b[34m Instances 60000: 60000\u001b[39m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"The accuracy of Train Data before the training process 0.1287\n",
"The cost of Train Data before the training process 2.4326\n"
]
}
],
"source": [
"TrainP = predict(model, X_train, Y_train);\n",
"\n",
"println()\n",
"println(\"The accuracy of Train Data before the training process $(round(TrainP[:accuracy], digits=4))\")\n",
"println(\"The cost of Train Data before the training process $(round(TrainP[:cost], digits=4))\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train the model"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:16:24\u001b[39m\n",
"\u001b[34m Epoch 10: 10\u001b[39m\n",
"\u001b[34m Instances 60000: 60000\u001b[39m\n",
"\u001b[34m Train Cost: 0.2496\u001b[39m\n",
"\u001b[34m Train Accuracy: 0.9099\u001b[39m\n"
]
}
],
"source": [
"TrainD = train(X_train, Y_train, model, 10);# testData = X_test, testLabels = Y_test);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`train` function provides an extra `kwargs` to use test Data/Labels to get the Costs and Accuracies during each training epoch. \n",
"\n",
"**Note** This will take extra time to do the training\n",
"\n",
"Instead it can be used as follows:\n",
"\n",
"```julia\n",
"TrainD = train(X_train, Y_train, model, 10)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
"<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 2400 1600\">\n",
"<defs>\n",
" <clipPath id=\"clip310\">\n",
" <rect x=\"0\" y=\"0\" width=\"2400\" height=\"1600\"/>\n",
" </clipPath>\n",
"</defs>\n",
"<path clip-path=\"url(#clip310)\" d=\"\n",
"M0 1600 L2400 1600 L2400 0 L0 0 Z\n",
" \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
"<defs>\n",
" <clipPath id=\"clip311\">\n",
" <rect x=\"480\" y=\"0\" width=\"1681\" height=\"1600\"/>\n",
" </clipPath>\n",
"</defs>\n",
"<path clip-path=\"url(#clip310)\" d=\"\n",
"M211.602 1486.45 L2352.76 1486.45 L2352.76 47.2441 L211.602 47.2441 Z\n",
" \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
"<defs>\n",
" <clipPath id=\"clip312\">\n",
" <rect x=\"211\" y=\"47\" width=\"2142\" height=\"1440\"/>\n",
" </clipPath>\n",
"</defs>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 496.641,1486.45 496.641,47.2441 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 945.52,1486.45 945.52,47.2441 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 1394.4,1486.45 1394.4,47.2441 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 1843.28,1486.45 1843.28,47.2441 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 2292.16,1486.45 2292.16,47.2441 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,1342.15 2352.76,1342.15 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,1136.52 2352.76,1136.52 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,930.881 2352.76,930.881 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,725.245 2352.76,725.245 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,519.608 2352.76,519.608 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,313.971 2352.76,313.971 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
" 211.602,108.334 2352.76,108.334 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,1486.45 2352.76,1486.45 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,1486.45 211.602,47.2441 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 496.641,1486.45 496.641,1469.18 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 945.52,1486.45 945.52,1469.18 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 1394.4,1486.45 1394.4,1469.18 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 1843.28,1486.45 1843.28,1469.18 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 2292.16,1486.45 2292.16,1469.18 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,1342.15 237.296,1342.15 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,1136.52 237.296,1136.52 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,930.881 237.296,930.881 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,725.245 237.296,725.245 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,519.608 237.296,519.608 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,313.971 237.296,313.971 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 211.602,108.334 237.296,108.334 \n",
" \"/>\n",
"<path clip-path=\"url(#clip310)\" d=\"M 0 0 M491.293 1535.98 L507.613 1535.98 L507.613 1539.92 L485.669 1539.92 L485.669 1535.98 Q488.331 1533.23 492.914 1528.6 Q497.52 1523.95 498.701 1522.61 Q500.946 1520.08 501.826 1518.35 Q502.729 1516.59 502.729 1514.9 Q502.729 1512.14 500.784 1510.41 Q498.863 1508.67 495.761 1508.67 Q493.562 1508.67 491.108 1509.43 Q488.678 1510.2 485.9 1511.75 L485.9 1507.03 Q488.724 1505.89 491.178 1505.31 Q493.631 1504.73 495.668 1504.73 Q501.039 1504.73 504.233 1507.42 Q507.428 1510.11 507.428 1514.6 Q507.428 1516.73 506.617 1518.65 Q505.83 1520.54 503.724 1523.14 Q503.145 1523.81 500.043 1527.03 Q496.942 1530.22 491.293 1535.98 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M948.529 1509.43 L936.724 1527.88 L948.529 1527.88 L948.529 1509.43 M947.302 1505.36 L953.182 1505.36 L953.182 1527.88 L958.112 1527.88 L958.112 1531.77 L953.182 1531.77 L953.182 1539.92 L948.529 1539.92 L948.529 1531.77 L932.927 1531.77 L932.927 1527.26 L947.302 1505.36 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1394.8 1520.78 Q1391.66 1520.78 1389.8 1522.93 Q1387.98 1525.08 1387.98 1528.83 Q1387.98 1532.56 1389.8 1534.73 Q1391.66 1536.89 1394.8 1536.89 Q1397.95 1536.89 1399.78 1534.73 Q1401.63 1532.56 1401.63 1528.83 Q1401.63 1525.08 1399.78 1522.93 Q1397.95 1520.78 1394.8 1520.78 M1404.09 1506.12 L1404.09 1510.38 Q1402.33 1509.55 1400.52 1509.11 Q1398.74 1508.67 1396.98 1508.67 Q1392.35 1508.67 1389.9 1511.8 Q1387.47 1514.92 1387.12 1521.24 Q1388.48 1519.23 1390.54 1518.16 Q1392.6 1517.07 1395.08 1517.07 Q1400.29 1517.07 1403.3 1520.24 Q1406.33 1523.39 1406.33 1528.83 Q1406.33 1534.16 1403.18 1537.37 Q1400.04 1540.59 1394.8 1540.59 Q1388.81 1540.59 1385.64 1536.01 Q1382.47 1531.4 1382.47 1522.67 Q1382.47 1514.48 1386.36 1509.62 Q1390.24 1504.73 1396.79 1504.73 Q1398.55 1504.73 1400.34 1505.08 Q1402.14 1505.43 1404.09 1506.12 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1843.28 1523.51 Q1839.94 1523.51 1838.02 1525.29 Q1836.13 1527.07 1836.13 1530.2 Q1836.13 1533.32 1838.02 1535.11 Q1839.94 1536.89 1843.28 1536.89 Q1846.61 1536.89 1848.53 1535.11 Q1850.45 1533.3 1850.45 1530.2 Q1850.45 1527.07 1848.53 1525.29 Q1846.63 1523.51 1843.28 1523.51 M1838.6 1521.52 Q1835.59 1520.78 1833.9 1518.72 Q1832.24 1516.66 1832.24 1513.69 Q1832.24 1509.55 1835.18 1507.14 Q1838.14 1504.73 1843.28 1504.73 Q1848.44 1504.73 1851.38 1507.14 Q1854.32 1509.55 1854.32 1513.69 Q1854.32 1516.66 1852.63 1518.72 Q1850.96 1520.78 1847.98 1521.52 Q1851.36 1522.3 1853.23 1524.6 Q1855.13 1526.89 1855.13 1530.2 Q1855.13 1535.22 1852.05 1537.91 Q1849 1540.59 1843.28 1540.59 Q1837.56 1540.59 1834.48 1537.91 Q1831.43 1535.22 1831.43 1530.2 Q1831.43 1526.89 1833.32 1524.6 Q1835.22 1522.3 1838.6 1521.52 M1836.89 1514.13 Q1836.89 1516.82 1838.56 1518.32 Q1840.25 1519.83 1843.28 1519.83 Q1846.29 1519.83 1847.98 1518.32 Q1849.69 1516.82 1849.69 1514.13 Q1849.69 1511.45 1847.98 1509.94 Q1846.29 1508.44 1843.28 1508.44 Q1840.25 1508.44 1838.56 1509.94 Q1836.89 1511.45 1836.89 1514.13 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2269.03 1535.98 L2276.67 1535.98 L2276.67 1509.62 L2268.36 1511.29 L2268.36 1507.03 L2276.62 1505.36 L2281.3 1505.36 L2281.3 1535.98 L2288.94 1535.98 L2288.94 1539.92 L2269.03 1539.92 L2269.03 1535.98 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2304.01 1508.44 Q2300.4 1508.44 2298.57 1512 Q2296.76 1515.55 2296.76 1522.67 Q2296.76 1529.78 2298.57 1533.35 Q2300.4 1536.89 2304.01 1536.89 Q2307.64 1536.89 2309.45 1533.35 Q2311.28 1529.78 2311.28 1522.67 Q2311.28 1515.55 2309.45 1512 Q2307.64 1508.44 2304.01 1508.44 M2304.01 1504.73 Q2309.82 1504.73 2312.87 1509.34 Q2315.95 1513.92 2315.95 1522.67 Q2315.95 1531.4 2312.87 1536.01 Q2309.82 1540.59 2304.01 1540.59 Q2298.2 1540.59 2295.12 1536.01 Q2292.06 1531.4 2292.06 1522.67 Q2292.06 1513.92 2295.12 1509.34 Q2298.2 1504.73 2304.01 1504.73 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M139.339 1327.95 Q135.728 1327.95 133.899 1331.52 Q132.093 1335.06 132.093 1342.19 Q132.093 1349.3 133.899 1352.86 Q135.728 1356.4 139.339 1356.4 Q142.973 1356.4 144.779 1352.86 Q146.607 1349.3 146.607 1342.19 Q146.607 1335.06 144.779 1331.52 Q142.973 1327.95 139.339 1327.95 M139.339 1324.25 Q145.149 1324.25 148.204 1328.86 Q151.283 1333.44 151.283 1342.19 Q151.283 1350.92 148.204 1355.52 Q145.149 1360.11 139.339 1360.11 Q133.529 1360.11 130.45 1355.52 Q127.394 1350.92 127.394 1342.19 Q127.394 1333.44 130.45 1328.86 Q133.529 1324.25 139.339 1324.25 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M156.353 1353.56 L161.237 1353.56 L161.237 1359.43 L156.353 1359.43 L156.353 1353.56 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M180.473 1340.8 Q183.829 1341.52 185.704 1343.79 Q187.602 1346.06 187.602 1349.39 Q187.602 1354.5 184.084 1357.31 Q180.565 1360.11 174.084 1360.11 Q171.908 1360.11 169.593 1359.67 Q167.302 1359.25 164.848 1358.39 L164.848 1353.88 Q166.792 1355.01 169.107 1355.59 Q171.422 1356.17 173.945 1356.17 Q178.343 1356.17 180.635 1354.43 Q182.95 1352.7 182.95 1349.39 Q182.95 1346.33 180.797 1344.62 Q178.667 1342.88 174.848 1342.88 L170.82 1342.88 L170.82 1339.04 L175.033 1339.04 Q178.482 1339.04 180.311 1337.68 Q182.139 1336.29 182.139 1333.69 Q182.139 1331.03 180.241 1329.62 Q178.366 1328.19 174.848 1328.19 Q172.927 1328.19 170.728 1328.6 Q168.528 1329.02 165.89 1329.9 L165.89 1325.73 Q168.552 1324.99 170.866 1324.62 Q173.204 1324.25 175.265 1324.25 Q180.589 1324.25 183.69 1326.68 Q186.792 1329.09 186.792 1333.21 Q186.792 1336.08 185.149 1338.07 Q183.505 1340.04 180.473 1340.8 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M138.205 1122.32 Q134.593 1122.32 132.765 1125.88 Q130.959 1129.42 130.959 1136.55 Q130.959 1143.66 132.765 1147.22 Q134.593 1150.77 138.205 1150.77 Q141.839 1150.77 143.644 1147.22 Q145.473 1143.66 145.473 1136.55 Q145.473 1129.42 143.644 1125.88 Q141.839 1122.32 138.205 1122.32 M138.205 1118.61 Q144.015 1118.61 147.07 1123.22 Q150.149 1127.8 150.149 1136.55 Q150.149 1145.28 147.07 1149.89 Q144.015 1154.47 138.205 1154.47 Q132.394 1154.47 129.316 1149.89 Q126.26 1145.28 126.26 1136.55 Q126.26 1127.8 129.316 1123.22 Q132.394 1118.61 138.205 1118.61 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M155.218 1147.92 L160.103 1147.92 L160.103 1153.8 L155.218 1153.8 L155.218 1147.92 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M178.019 1123.31 L166.214 1141.76 L178.019 1141.76 L178.019 1123.31 M176.792 1119.24 L182.672 1119.24 L182.672 1141.76 L187.602 1141.76 L187.602 1145.65 L182.672 1145.65 L182.672 1153.8 L178.019 1153.8 L178.019 1145.65 L162.417 1145.65 L162.417 1141.14 L176.792 1119.24 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M139.686 916.68 Q136.075 916.68 134.246 920.245 Q132.441 923.787 132.441 930.916 Q132.441 938.023 134.246 941.587 Q136.075 945.129 139.686 945.129 Q143.32 945.129 145.126 941.587 Q146.954 938.023 146.954 930.916 Q146.954 923.787 145.126 920.245 Q143.32 916.68 139.686 916.68 M139.686 912.976 Q145.496 912.976 148.552 917.583 Q151.63 922.166 151.63 930.916 Q151.63 939.643 148.552 944.249 Q145.496 948.833 139.686 948.833 Q133.876 948.833 130.797 944.249 Q127.742 939.643 127.742 930.916 Q127.742 922.166 130.797 917.583 Q133.876 912.976 139.686 912.976 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M156.7 942.282 L161.584 942.282 L161.584 948.161 L156.7 948.161 L156.7 942.282 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M166.7 913.601 L185.056 913.601 L185.056 917.537 L170.982 917.537 L170.982 926.009 Q172.001 925.662 173.019 925.499 Q174.038 925.314 175.056 925.314 Q180.843 925.314 184.223 928.486 Q187.602 931.657 187.602 937.073 Q187.602 942.652 184.13 945.754 Q180.658 948.833 174.339 948.833 Q172.163 948.833 169.894 948.462 Q167.649 948.092 165.241 947.351 L165.241 942.652 Q167.325 943.786 169.547 944.342 Q171.769 944.898 174.246 944.898 Q178.251 944.898 180.589 942.791 Q182.927 940.685 182.927 937.073 Q182.927 933.462 180.589 931.356 Q178.251 929.249 174.246 929.249 Q172.371 929.249 170.496 929.666 Q168.644 930.083 166.7 930.962 L166.7 913.601 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M138.529 711.043 Q134.918 711.043 133.089 714.608 Q131.283 718.15 131.283 725.279 Q131.283 732.386 133.089 735.951 Q134.918 739.492 138.529 739.492 Q142.163 739.492 143.968 735.951 Q145.797 732.386 145.797 725.279 Q145.797 718.15 143.968 714.608 Q142.163 711.043 138.529 711.043 M138.529 707.34 Q144.339 707.34 147.394 711.946 Q150.473 716.529 150.473 725.279 Q150.473 734.006 147.394 738.613 Q144.339 743.196 138.529 743.196 Q132.718 743.196 129.64 738.613 Q126.584 734.006 126.584 725.279 Q126.584 716.529 129.64 711.946 Q132.718 707.34 138.529 707.34 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M155.542 736.645 L160.427 736.645 L160.427 742.525 L155.542 742.525 L155.542 736.645 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M176.075 723.381 Q172.927 723.381 171.075 725.534 Q169.246 727.687 169.246 731.437 Q169.246 735.164 171.075 737.339 Q172.927 739.492 176.075 739.492 Q179.223 739.492 181.052 737.339 Q182.903 735.164 182.903 731.437 Q182.903 727.687 181.052 725.534 Q179.223 723.381 176.075 723.381 M185.357 708.729 L185.357 712.988 Q183.598 712.154 181.792 711.715 Q180.01 711.275 178.251 711.275 Q173.621 711.275 171.167 714.4 Q168.737 717.525 168.39 723.844 Q169.755 721.83 171.815 720.765 Q173.876 719.678 176.352 719.678 Q181.561 719.678 184.57 722.849 Q187.602 725.997 187.602 731.437 Q187.602 736.761 184.454 739.978 Q181.306 743.196 176.075 743.196 Q170.079 743.196 166.908 738.613 Q163.737 734.006 163.737 725.279 Q163.737 717.085 167.626 712.224 Q171.515 707.34 178.065 707.34 Q179.825 707.34 181.607 707.687 Q183.413 708.034 185.357 708.729 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M139.593 505.407 Q135.982 505.407 134.154 508.971 Q132.348 512.513 132.348 519.643 Q132.348 526.749 134.154 530.314 Q135.982 533.855 139.593 533.855 Q143.228 533.855 145.033 530.314 Q146.862 526.749 146.862 519.643 Q146.862 512.513 145.033 508.971 Q143.228 505.407 139.593 505.407 M139.593 501.703 Q145.404 501.703 148.459 506.309 Q151.538 510.893 151.538 519.643 Q151.538 528.369 148.459 532.976 Q145.404 537.559 139.593 537.559 Q133.783 537.559 130.705 532.976 Q127.649 528.369 127.649 519.643 Q127.649 510.893 130.705 506.309 Q133.783 501.703 139.593 501.703 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M156.607 531.008 L161.491 531.008 L161.491 536.888 L156.607 536.888 L156.607 531.008 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M165.38 502.328 L187.602 502.328 L187.602 504.319 L175.056 536.888 L170.172 536.888 L181.977 506.263 L165.38 506.263 L165.38 502.328 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M138.783 299.77 Q135.172 299.77 133.343 303.335 Q131.538 306.876 131.538 314.006 Q131.538 321.112 133.343 324.677 Q135.172 328.219 138.783 328.219 Q142.417 328.219 144.223 324.677 Q146.052 321.112 146.052 314.006 Q146.052 306.876 144.223 303.335 Q142.417 299.77 138.783 299.77 M138.783 296.066 Q144.593 296.066 147.649 300.673 Q150.728 305.256 150.728 314.006 Q150.728 322.733 147.649 327.339 Q144.593 331.922 138.783 331.922 Q132.973 331.922 129.894 327.339 Q126.839 322.733 126.839 314.006 Q126.839 305.256 129.894 300.673 Q132.973 296.066 138.783 296.066 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M155.797 325.371 L160.681 325.371 L160.681 331.251 L155.797 331.251 L155.797 325.371 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M175.751 314.839 Q172.417 314.839 170.496 316.622 Q168.598 318.404 168.598 321.529 Q168.598 324.654 170.496 326.436 Q172.417 328.219 175.751 328.219 Q179.084 328.219 181.005 326.436 Q182.927 324.631 182.927 321.529 Q182.927 318.404 181.005 316.622 Q179.107 314.839 175.751 314.839 M171.075 312.848 Q168.065 312.108 166.376 310.047 Q164.709 307.987 164.709 305.024 Q164.709 300.881 167.649 298.473 Q170.612 296.066 175.751 296.066 Q180.913 296.066 183.852 298.473 Q186.792 300.881 186.792 305.024 Q186.792 307.987 185.102 310.047 Q183.436 312.108 180.45 312.848 Q183.829 313.635 185.704 315.927 Q187.602 318.219 187.602 321.529 Q187.602 326.552 184.524 329.237 Q181.468 331.922 175.751 331.922 Q170.033 331.922 166.954 329.237 Q163.899 326.552 163.899 321.529 Q163.899 318.219 165.797 315.927 Q167.695 313.635 171.075 312.848 M169.362 305.464 Q169.362 308.149 171.028 309.654 Q172.718 311.159 175.751 311.159 Q178.76 311.159 180.45 309.654 Q182.163 308.149 182.163 305.464 Q182.163 302.779 180.45 301.274 Q178.76 299.77 175.751 299.77 Q172.718 299.77 171.028 301.274 Q169.362 302.779 169.362 305.464 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M138.876 94.133 Q135.265 94.133 133.436 97.6978 Q131.63 101.239 131.63 108.369 Q131.63 115.475 133.436 119.04 Q135.265 122.582 138.876 122.582 Q142.51 122.582 144.316 119.04 Q146.144 115.475 146.144 108.369 Q146.144 101.239 144.316 97.6978 Q142.51 94.133 138.876 94.133 M138.876 90.4293 Q144.686 90.4293 147.742 95.0358 Q150.82 99.6191 150.82 108.369 Q150.82 117.096 147.742 121.702 Q144.686 126.286 138.876 126.286 Q133.066 126.286 129.987 121.702 Q126.931 117.096 126.931 108.369 Q126.931 99.6191 129.987 95.0358 Q133.066 90.4293 138.876 90.4293 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M155.89 119.735 L160.774 119.735 L160.774 125.614 L155.89 125.614 L155.89 119.735 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M165.982 124.897 L165.982 120.637 Q167.741 121.471 169.547 121.911 Q171.353 122.35 173.089 122.35 Q177.718 122.35 180.149 119.249 Q182.602 116.124 182.95 109.781 Q181.607 111.772 179.547 112.837 Q177.487 113.901 174.987 113.901 Q169.802 113.901 166.769 110.776 Q163.76 107.628 163.76 102.189 Q163.76 96.8645 166.908 93.6469 Q170.056 90.4293 175.288 90.4293 Q181.283 90.4293 184.431 95.0358 Q187.602 99.6191 187.602 108.369 Q187.602 116.54 183.714 121.425 Q179.848 126.286 173.297 126.286 Q171.538 126.286 169.732 125.938 Q167.927 125.591 165.982 124.897 M175.288 110.244 Q178.436 110.244 180.264 108.091 Q182.116 105.938 182.116 102.189 Q182.116 98.4617 180.264 96.3089 Q178.436 94.133 175.288 94.133 Q172.14 94.133 170.288 96.3089 Q168.459 98.4617 168.459 102.189 Q168.459 105.938 170.288 108.091 Q172.14 110.244 175.288 110.244 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M40.4842 868.395 L40.4842 838.349 L45.895 838.349 L45.895 861.966 L59.9632 861.966 L59.9632 839.335 L65.3741 839.335 L65.3741 861.966 L82.5933 861.966 L82.5933 837.776 L88.0042 837.776 L88.0042 868.395 L40.4842 868.395 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M82.657 825.967 L101.563 825.967 L101.563 831.856 L52.3562 831.856 L52.3562 825.967 L57.7671 825.967 Q54.5842 824.121 53.0564 821.321 Q51.4968 818.488 51.4968 814.573 Q51.4968 808.08 56.6531 804.038 Q61.8093 799.964 70.212 799.964 Q78.6147 799.964 83.771 804.038 Q88.9272 808.08 88.9272 814.573 Q88.9272 818.488 87.3994 821.321 Q85.8398 824.121 82.657 825.967 M70.212 806.043 Q63.7508 806.043 60.0905 808.716 Q56.3984 811.358 56.3984 816.005 Q56.3984 820.652 60.0905 823.326 Q63.7508 825.967 70.212 825.967 Q76.6732 825.967 80.3653 823.326 Q84.0256 820.652 84.0256 816.005 Q84.0256 811.358 80.3653 808.716 Q76.6732 806.043 70.212 806.043 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M56.4621 780.007 Q56.4621 784.718 60.1542 787.455 Q63.8145 790.192 70.212 790.192 Q76.6095 790.192 80.3017 787.487 Q83.9619 784.75 83.9619 780.007 Q83.9619 775.328 80.2698 772.591 Q76.5777 769.854 70.212 769.854 Q63.8781 769.854 60.186 772.591 Q56.4621 775.328 56.4621 780.007 M51.4968 780.007 Q51.4968 772.368 56.4621 768.008 Q61.4273 763.647 70.212 763.647 Q78.9649 763.647 83.9619 768.008 Q88.9272 772.368 88.9272 780.007 Q88.9272 787.678 83.9619 792.038 Q78.9649 796.367 70.212 796.367 Q61.4273 796.367 56.4621 792.038 Q51.4968 787.678 51.4968 780.007 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M53.7248 731.851 L59.1993 731.851 Q57.8307 734.333 57.1623 736.848 Q56.4621 739.33 56.4621 741.876 Q56.4621 747.574 60.0905 750.725 Q63.6872 753.876 70.212 753.876 Q76.7369 753.876 80.3653 750.725 Q83.9619 747.574 83.9619 741.876 Q83.9619 739.33 83.2935 736.848 Q82.5933 734.333 81.2247 731.851 L86.6355 731.851 Q87.7814 734.301 88.3543 736.943 Q88.9272 739.553 88.9272 742.513 Q88.9272 750.566 83.8664 755.308 Q78.8057 760.051 70.212 760.051 Q61.491 760.051 56.4939 755.276 Q51.4968 750.47 51.4968 742.131 Q51.4968 739.426 52.0697 736.848 Q52.6108 734.269 53.7248 731.851 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M66.4881 696.075 L88.0042 696.075 L88.0042 701.932 L66.679 701.932 Q61.6183 701.932 59.1038 703.905 Q56.5894 705.878 56.5894 709.825 Q56.5894 714.568 59.6131 717.305 Q62.6368 720.042 67.8567 720.042 L88.0042 720.042 L88.0042 725.93 L38.479 725.93 L38.479 720.042 L57.8944 720.042 Q54.6797 717.941 53.0883 715.109 Q51.4968 712.244 51.4968 708.52 Q51.4968 702.377 55.3163 699.226 Q59.1038 696.075 66.4881 696.075 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M53.4065 667.207 L58.9447 667.207 Q57.6716 669.689 57.035 672.363 Q56.3984 675.037 56.3984 677.901 Q56.3984 682.262 57.7352 684.458 Q59.072 686.622 61.7456 686.622 Q63.7826 686.622 64.9603 685.063 Q66.1061 683.503 67.1565 678.792 L67.6021 676.787 Q68.9389 670.549 71.3897 667.939 Q73.8086 665.297 78.1691 665.297 Q83.1344 665.297 86.0308 669.244 Q88.9272 673.159 88.9272 680.034 Q88.9272 682.898 88.3543 686.017 Q87.8132 689.105 86.6992 692.542 L80.6518 692.542 Q82.3387 689.296 83.198 686.145 Q84.0256 682.994 84.0256 679.906 Q84.0256 675.769 82.6251 673.541 Q81.1929 671.313 78.6147 671.313 Q76.2276 671.313 74.9545 672.936 Q73.6813 674.527 72.5037 679.97 L72.0262 682.007 Q70.8804 687.45 68.5251 689.869 Q66.138 692.288 62.0002 692.288 Q56.9713 692.288 54.2341 688.723 Q51.4968 685.158 51.4968 678.601 Q51.4968 675.355 51.9743 672.49 Q52.4517 669.626 53.4065 667.207 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><polyline clip-path=\"url(#clip312)\" style=\"stroke:#009af9; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 272.201,274.146 496.641,175.715 721.08,143.087 945.52,132.085 1169.96,118.273 1394.4,110.459 1618.84,106.072 1843.28,96.8872 2067.72,94.7966 2292.16,87.9763 \n",
" \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip312)\" style=\"stroke:#e26f46; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 272.201,923.799 496.641,1207.98 721.08,1293.15 945.52,1331.05 1169.96,1365.41 1394.4,1382.71 1618.84,1402.34 1843.28,1422 2067.72,1434.24 2292.16,1445.72 \n",
" \n",
" \"/>\n",
"<path clip-path=\"url(#clip310)\" d=\"\n",
"M1635.6 312.204 L2280.76 312.204 L2280.76 130.764 L1635.6 130.764 Z\n",
" \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 1635.6,312.204 2280.76,312.204 2280.76,130.764 1635.6,130.764 1635.6,312.204 \n",
" \"/>\n",
"<polyline clip-path=\"url(#clip310)\" style=\"stroke:#009af9; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 1659.6,191.244 1803.6,191.244 \n",
" \"/>\n",
"<path clip-path=\"url(#clip310)\" d=\"M 0 0 M1827.6 173.964 L1856.83 173.964 L1856.83 177.899 L1844.56 177.899 L1844.56 208.524 L1839.86 208.524 L1839.86 177.899 L1827.6 177.899 L1827.6 173.964 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1869.36 186.58 Q1868.64 186.163 1867.78 185.978 Q1866.95 185.77 1865.93 185.77 Q1862.32 185.77 1860.37 188.131 Q1858.45 190.469 1858.45 194.867 L1858.45 208.524 L1854.17 208.524 L1854.17 182.598 L1858.45 182.598 L1858.45 186.626 Q1859.79 184.265 1861.95 183.131 Q1864.1 181.973 1867.18 181.973 Q1867.62 181.973 1868.15 182.043 Q1868.68 182.089 1869.33 182.205 L1869.36 186.58 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1885.6 195.492 Q1880.44 195.492 1878.45 196.672 Q1876.46 197.853 1876.46 200.7 Q1876.46 202.969 1877.94 204.311 Q1879.45 205.631 1882.02 205.631 Q1885.56 205.631 1887.69 203.131 Q1889.84 200.607 1889.84 196.441 L1889.84 195.492 L1885.6 195.492 M1894.1 193.733 L1894.1 208.524 L1889.84 208.524 L1889.84 204.589 Q1888.38 206.95 1886.21 208.084 Q1884.03 209.195 1880.88 209.195 Q1876.9 209.195 1874.54 206.973 Q1872.2 204.728 1872.2 200.978 Q1872.2 196.603 1875.12 194.381 Q1878.06 192.158 1883.87 192.158 L1889.84 192.158 L1889.84 191.742 Q1889.84 188.802 1887.9 187.205 Q1885.98 185.584 1882.48 185.584 Q1880.26 185.584 1878.15 186.117 Q1876.04 186.649 1874.1 187.714 L1874.1 183.779 Q1876.44 182.876 1878.64 182.436 Q1880.84 181.973 1882.92 181.973 Q1888.54 181.973 1891.32 184.89 Q1894.1 187.807 1894.1 193.733 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1898.57 182.598 L1902.83 182.598 L1902.83 208.524 L1898.57 208.524 L1898.57 182.598 M1898.57 172.506 L1902.83 172.506 L1902.83 177.899 L1898.57 177.899 L1898.57 172.506 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1928.85 192.876 L1928.85 208.524 L1924.59 208.524 L1924.59 193.015 Q1924.59 189.334 1923.15 187.506 Q1921.72 185.677 1918.85 185.677 Q1915.4 185.677 1913.41 187.876 Q1911.42 190.075 1911.42 193.871 L1911.42 208.524 L1907.13 208.524 L1907.13 182.598 L1911.42 182.598 L1911.42 186.626 Q1912.94 184.288 1915 183.131 Q1917.09 181.973 1919.79 181.973 Q1924.26 181.973 1926.55 184.751 Q1928.85 187.506 1928.85 192.876 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1933.31 182.598 L1937.57 182.598 L1937.57 208.524 L1933.31 208.524 L1933.31 182.598 M1933.31 172.506 L1937.57 172.506 L1937.57 177.899 L1933.31 177.899 L1933.31 172.506 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1963.59 192.876 L1963.59 208.524 L1959.33 208.524 L1959.33 193.015 Q1959.33 189.334 1957.9 187.506 Q1956.46 185.677 1953.59 185.677 Q1950.14 185.677 1948.15 187.876 Q1946.16 190.075 1946.16 193.871 L1946.16 208.524 L1941.88 208.524 L1941.88 182.598 L1946.16 182.598 L1946.16 186.626 Q1947.69 184.288 1949.75 183.131 Q1951.83 181.973 1954.54 181.973 Q1959.01 181.973 1961.3 184.751 Q1963.59 187.506 1963.59 192.876 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1985.12 195.26 Q1985.12 190.631 1983.2 188.084 Q1981.3 185.538 1977.85 185.538 Q1974.42 185.538 1972.5 188.084 Q1970.6 190.631 1970.6 195.26 Q1970.6 199.867 1972.5 202.413 Q1974.42 204.959 1977.85 204.959 Q1981.3 204.959 1983.2 202.413 Q1985.12 199.867 1985.12 195.26 M1989.38 205.307 Q1989.38 211.927 1986.44 215.144 Q1983.5 218.385 1977.43 218.385 Q1975.19 218.385 1973.2 218.038 Q1971.21 217.714 1969.33 217.019 L1969.33 212.876 Q1971.21 213.894 1973.04 214.381 Q1974.86 214.867 1976.76 214.867 Q1980.95 214.867 1983.04 212.668 Q1985.12 210.492 1985.12 206.07 L1985.12 203.964 Q1983.8 206.256 1981.74 207.39 Q1979.68 208.524 1976.81 208.524 Q1972.04 208.524 1969.12 204.89 Q1966.21 201.256 1966.21 195.26 Q1966.21 189.242 1969.12 185.608 Q1972.04 181.973 1976.81 181.973 Q1979.68 181.973 1981.74 183.108 Q1983.8 184.242 1985.12 186.533 L1985.12 182.598 L1989.38 182.598 L1989.38 205.307 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2020.65 178.571 L2014.31 195.77 L2027.02 195.77 L2020.65 178.571 M2018.01 173.964 L2023.31 173.964 L2036.48 208.524 L2031.62 208.524 L2028.47 199.658 L2012.9 199.658 L2009.75 208.524 L2004.82 208.524 L2018.01 173.964 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2058.78 183.594 L2058.78 187.575 Q2056.97 186.58 2055.14 186.094 Q2053.34 185.584 2051.48 185.584 Q2047.34 185.584 2045.05 188.223 Q2042.76 190.839 2042.76 195.584 Q2042.76 200.33 2045.05 202.969 Q2047.34 205.584 2051.48 205.584 Q2053.34 205.584 2055.14 205.098 Q2056.97 204.589 2058.78 203.594 L2058.78 207.529 Q2056.99 208.362 2055.07 208.779 Q2053.17 209.195 2051.02 209.195 Q2045.16 209.195 2041.72 205.515 Q2038.27 201.834 2038.27 195.584 Q2038.27 189.242 2041.74 185.608 Q2045.23 181.973 2051.3 181.973 Q2053.27 181.973 2055.14 182.39 Q2057.02 182.783 2058.78 183.594 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2081.9 183.594 L2081.9 187.575 Q2080.09 186.58 2078.27 186.094 Q2076.46 185.584 2074.61 185.584 Q2070.47 185.584 2068.17 188.223 Q2065.88 190.839 2065.88 195.584 Q2065.88 200.33 2068.17 202.969 Q2070.47 205.584 2074.61 205.584 Q2076.46 205.584 2078.27 205.098 Q2080.09 204.589 2081.9 203.594 L2081.9 207.529 Q2080.12 208.362 2078.2 208.779 Q2076.3 209.195 2074.15 209.195 Q2068.29 209.195 2064.84 205.515 Q2061.39 201.834 2061.39 195.584 Q2061.39 189.242 2064.86 185.608 Q2068.36 181.973 2074.42 181.973 Q2076.39 181.973 2078.27 182.39 Q2080.14 182.783 2081.9 183.594 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2085.93 198.293 L2085.93 182.598 L2090.19 182.598 L2090.19 198.131 Q2090.19 201.811 2091.62 203.663 Q2093.06 205.492 2095.93 205.492 Q2099.38 205.492 2101.37 203.293 Q2103.38 201.094 2103.38 197.297 L2103.38 182.598 L2107.64 182.598 L2107.64 208.524 L2103.38 208.524 L2103.38 204.543 Q2101.83 206.904 2099.77 208.061 Q2097.73 209.195 2095.03 209.195 Q2090.56 209.195 2088.24 206.418 Q2085.93 203.64 2085.93 198.293 M2096.65 181.973 L2096.65 181.973 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2127.13 186.58 Q2126.41 186.163 2125.56 185.978 Q2124.72 185.77 2123.71 185.77 Q2120.09 185.77 2118.15 188.131 Q2116.23 190.469 2116.23 194.867 L2116.23 208.524 L2111.95 208.524 L2111.95 182.598 L2116.23 182.598 L2116.23 186.626 Q2117.57 184.265 2119.72 183.131 Q2121.88 181.973 2124.96 181.973 Q2125.4 181.973 2125.93 182.043 Q2126.46 182.089 2127.11 182.205 L2127.13 186.58 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2143.38 195.492 Q2138.22 195.492 2136.23 196.672 Q2134.24 197.853 2134.24 200.7 Q2134.24 202.969 2135.72 204.311 Q2137.22 205.631 2139.79 205.631 Q2143.34 205.631 2145.46 203.131 Q2147.62 200.607 2147.62 196.441 L2147.62 195.492 L2143.38 195.492 M2151.88 193.733 L2151.88 208.524 L2147.62 208.524 L2147.62 204.589 Q2146.16 206.95 2143.98 208.084 Q2141.81 209.195 2138.66 209.195 Q2134.68 209.195 2132.32 206.973 Q2129.98 204.728 2129.98 200.978 Q2129.98 196.603 2132.9 194.381 Q2135.84 192.158 2141.65 192.158 L2147.62 192.158 L2147.62 191.742 Q2147.62 188.802 2145.67 187.205 Q2143.75 185.584 2140.26 185.584 Q2138.03 185.584 2135.93 186.117 Q2133.82 186.649 2131.88 187.714 L2131.88 183.779 Q2134.21 182.876 2136.41 182.436 Q2138.61 181.973 2140.7 181.973 Q2146.32 181.973 2149.1 184.89 Q2151.88 187.807 2151.88 193.733 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2175 183.594 L2175 187.575 Q2173.2 186.58 2171.37 186.094 Q2169.56 185.584 2167.71 185.584 Q2163.57 185.584 2161.27 188.223 Q2158.98 190.839 2158.98 195.584 Q2158.98 200.33 2161.27 202.969 Q2163.57 205.584 2167.71 205.584 Q2169.56 205.584 2171.37 205.098 Q2173.2 204.589 2175 203.594 L2175 207.529 Q2173.22 208.362 2171.3 208.779 Q2169.4 209.195 2167.25 209.195 Q2161.39 209.195 2157.94 205.515 Q2154.49 201.834 2154.49 195.584 Q2154.49 189.242 2157.96 185.608 Q2161.46 181.973 2167.52 181.973 Q2169.49 181.973 2171.37 182.39 Q2173.24 182.783 2175 183.594 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2179.47 182.598 L2183.73 182.598 L2183.73 208.524 L2179.47 208.524 L2179.47 182.598 M2179.47 172.506 L2183.73 172.506 L2183.73 177.899 L2179.47 177.899 L2179.47 172.506 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2210.37 194.496 L2210.37 196.58 L2190.79 196.58 Q2191.07 200.978 2193.43 203.293 Q2195.81 205.584 2200.05 205.584 Q2202.5 205.584 2204.79 204.982 Q2207.11 204.381 2209.38 203.177 L2209.38 207.205 Q2207.08 208.177 2204.68 208.686 Q2202.27 209.195 2199.79 209.195 Q2193.59 209.195 2189.96 205.584 Q2186.34 201.973 2186.34 195.816 Q2186.34 189.45 2189.77 185.723 Q2193.22 181.973 2199.05 181.973 Q2204.28 181.973 2207.32 185.353 Q2210.37 188.709 2210.37 194.496 M2206.11 193.246 Q2206.07 189.751 2204.14 187.668 Q2202.25 185.584 2199.1 185.584 Q2195.53 185.584 2193.38 187.598 Q2191.25 189.612 2190.93 193.27 L2206.11 193.246 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2231.37 183.362 L2231.37 187.39 Q2229.56 186.464 2227.62 186.001 Q2225.67 185.538 2223.59 185.538 Q2220.42 185.538 2218.82 186.51 Q2217.25 187.483 2217.25 189.427 Q2217.25 190.908 2218.38 191.765 Q2219.52 192.598 2222.94 193.362 L2224.4 193.686 Q2228.94 194.658 2230.83 196.441 Q2232.76 198.2 2232.76 201.371 Q2232.76 204.982 2229.89 207.089 Q2227.04 209.195 2222.04 209.195 Q2219.96 209.195 2217.69 208.779 Q2215.44 208.385 2212.94 207.575 L2212.94 203.177 Q2215.3 204.404 2217.59 205.029 Q2219.89 205.631 2222.13 205.631 Q2225.14 205.631 2226.76 204.612 Q2228.38 203.57 2228.38 201.695 Q2228.38 199.959 2227.2 199.033 Q2226.04 198.107 2222.08 197.251 L2220.6 196.904 Q2216.64 196.07 2214.89 194.358 Q2213.13 192.621 2213.13 189.612 Q2213.13 185.955 2215.72 183.964 Q2218.31 181.973 2223.08 181.973 Q2225.44 181.973 2227.52 182.321 Q2229.61 182.668 2231.37 183.362 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><polyline clip-path=\"url(#clip310)\" style=\"stroke:#e26f46; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
" 1659.6,251.724 1803.6,251.724 \n",
" \"/>\n",
"<path clip-path=\"url(#clip310)\" d=\"M 0 0 M1827.6 234.444 L1856.83 234.444 L1856.83 238.379 L1844.56 238.379 L1844.56 269.004 L1839.86 269.004 L1839.86 238.379 L1827.6 238.379 L1827.6 234.444 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1869.36 247.06 Q1868.64 246.643 1867.78 246.458 Q1866.95 246.25 1865.93 246.25 Q1862.32 246.25 1860.37 248.611 Q1858.45 250.949 1858.45 255.347 L1858.45 269.004 L1854.17 269.004 L1854.17 243.078 L1858.45 243.078 L1858.45 247.106 Q1859.79 244.745 1861.95 243.611 Q1864.1 242.453 1867.18 242.453 Q1867.62 242.453 1868.15 242.523 Q1868.68 242.569 1869.33 242.685 L1869.36 247.06 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1885.6 255.972 Q1880.44 255.972 1878.45 257.152 Q1876.46 258.333 1876.46 261.18 Q1876.46 263.449 1877.94 264.791 Q1879.45 266.111 1882.02 266.111 Q1885.56 266.111 1887.69 263.611 Q1889.84 261.087 1889.84 256.921 L1889.84 255.972 L1885.6 255.972 M1894.1 254.213 L1894.1 269.004 L1889.84 269.004 L1889.84 265.069 Q1888.38 267.43 1886.21 268.564 Q1884.03 269.675 1880.88 269.675 Q1876.9 269.675 1874.54 267.453 Q1872.2 265.208 1872.2 261.458 Q1872.2 257.083 1875.12 254.861 Q1878.06 252.638 1883.87 252.638 L1889.84 252.638 L1889.84 252.222 Q1889.84 249.282 1887.9 247.685 Q1885.98 246.064 1882.48 246.064 Q1880.26 246.064 1878.15 246.597 Q1876.04 247.129 1874.1 248.194 L1874.1 244.259 Q1876.44 243.356 1878.64 242.916 Q1880.84 242.453 1882.92 242.453 Q1888.54 242.453 1891.32 245.37 Q1894.1 248.287 1894.1 254.213 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1898.57 243.078 L1902.83 243.078 L1902.83 269.004 L1898.57 269.004 L1898.57 243.078 M1898.57 232.986 L1902.83 232.986 L1902.83 238.379 L1898.57 238.379 L1898.57 232.986 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1928.85 253.356 L1928.85 269.004 L1924.59 269.004 L1924.59 253.495 Q1924.59 249.814 1923.15 247.986 Q1921.72 246.157 1918.85 246.157 Q1915.4 246.157 1913.41 248.356 Q1911.42 250.555 1911.42 254.351 L1911.42 269.004 L1907.13 269.004 L1907.13 243.078 L1911.42 243.078 L1911.42 247.106 Q1912.94 244.768 1915 243.611 Q1917.09 242.453 1919.79 242.453 Q1924.26 242.453 1926.55 245.231 Q1928.85 247.986 1928.85 253.356 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1933.31 243.078 L1937.57 243.078 L1937.57 269.004 L1933.31 269.004 L1933.31 243.078 M1933.31 232.986 L1937.57 232.986 L1937.57 238.379 L1933.31 238.379 L1933.31 232.986 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1963.59 253.356 L1963.59 269.004 L1959.33 269.004 L1959.33 253.495 Q1959.33 249.814 1957.9 247.986 Q1956.46 246.157 1953.59 246.157 Q1950.14 246.157 1948.15 248.356 Q1946.16 250.555 1946.16 254.351 L1946.16 269.004 L1941.88 269.004 L1941.88 243.078 L1946.16 243.078 L1946.16 247.106 Q1947.69 244.768 1949.75 243.611 Q1951.83 242.453 1954.54 242.453 Q1959.01 242.453 1961.3 245.231 Q1963.59 247.986 1963.59 253.356 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M1985.12 255.74 Q1985.12 251.111 1983.2 248.564 Q1981.3 246.018 1977.85 246.018 Q1974.42 246.018 1972.5 248.564 Q1970.6 251.111 1970.6 255.74 Q1970.6 260.347 1972.5 262.893 Q1974.42 265.439 1977.85 265.439 Q1981.3 265.439 1983.2 262.893 Q1985.12 260.347 1985.12 255.74 M1989.38 265.787 Q1989.38 272.407 1986.44 275.624 Q1983.5 278.865 1977.43 278.865 Q1975.19 278.865 1973.2 278.518 Q1971.21 278.194 1969.33 277.499 L1969.33 273.356 Q1971.21 274.374 1973.04 274.861 Q1974.86 275.347 1976.76 275.347 Q1980.95 275.347 1983.04 273.148 Q1985.12 270.972 1985.12 266.55 L1985.12 264.444 Q1983.8 266.736 1981.74 267.87 Q1979.68 269.004 1976.81 269.004 Q1972.04 269.004 1969.12 265.37 Q1966.21 261.736 1966.21 255.74 Q1966.21 249.722 1969.12 246.088 Q1972.04 242.453 1976.81 242.453 Q1979.68 242.453 1981.74 243.588 Q1983.8 244.722 1985.12 247.013 L1985.12 243.078 L1989.38 243.078 L1989.38 265.787 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2034.98 237.106 L2034.98 242.037 Q2032.62 239.838 2029.93 238.75 Q2027.27 237.662 2024.26 237.662 Q2018.34 237.662 2015.19 241.296 Q2012.04 244.907 2012.04 251.759 Q2012.04 258.587 2015.19 262.222 Q2018.34 265.833 2024.26 265.833 Q2027.27 265.833 2029.93 264.745 Q2032.62 263.657 2034.98 261.458 L2034.98 266.342 Q2032.53 268.009 2029.77 268.842 Q2027.04 269.675 2023.98 269.675 Q2016.14 269.675 2011.62 264.884 Q2007.11 260.069 2007.11 251.759 Q2007.11 243.426 2011.62 238.634 Q2016.14 233.819 2023.98 233.819 Q2027.09 233.819 2029.82 234.652 Q2032.57 235.463 2034.98 237.106 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2049.49 246.064 Q2046.07 246.064 2044.08 248.75 Q2042.09 251.412 2042.09 256.064 Q2042.09 260.717 2044.05 263.402 Q2046.04 266.064 2049.49 266.064 Q2052.9 266.064 2054.89 263.379 Q2056.88 260.694 2056.88 256.064 Q2056.88 251.458 2054.89 248.773 Q2052.9 246.064 2049.49 246.064 M2049.49 242.453 Q2055.05 242.453 2058.22 246.064 Q2061.39 249.675 2061.39 256.064 Q2061.39 262.43 2058.22 266.064 Q2055.05 269.675 2049.49 269.675 Q2043.91 269.675 2040.74 266.064 Q2037.59 262.43 2037.59 256.064 Q2037.59 249.675 2040.74 246.064 Q2043.91 242.453 2049.49 242.453 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2082.39 243.842 L2082.39 247.87 Q2080.58 246.944 2078.64 246.481 Q2076.69 246.018 2074.61 246.018 Q2071.44 246.018 2069.84 246.99 Q2068.27 247.963 2068.27 249.907 Q2068.27 251.388 2069.4 252.245 Q2070.53 253.078 2073.96 253.842 L2075.42 254.166 Q2079.96 255.138 2081.85 256.921 Q2083.78 258.68 2083.78 261.851 Q2083.78 265.462 2080.9 267.569 Q2078.06 269.675 2073.06 269.675 Q2070.97 269.675 2068.71 269.259 Q2066.46 268.865 2063.96 268.055 L2063.96 263.657 Q2066.32 264.884 2068.61 265.509 Q2070.9 266.111 2073.15 266.111 Q2076.16 266.111 2077.78 265.092 Q2079.4 264.05 2079.4 262.175 Q2079.4 260.439 2078.22 259.513 Q2077.06 258.587 2073.1 257.731 L2071.62 257.384 Q2067.66 256.55 2065.9 254.838 Q2064.15 253.101 2064.15 250.092 Q2064.15 246.435 2066.74 244.444 Q2069.33 242.453 2074.1 242.453 Q2076.46 242.453 2078.54 242.801 Q2080.63 243.148 2082.39 243.842 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2092.46 235.717 L2092.46 243.078 L2101.23 243.078 L2101.23 246.388 L2092.46 246.388 L2092.46 260.462 Q2092.46 263.634 2093.31 264.537 Q2094.19 265.439 2096.85 265.439 L2101.23 265.439 L2101.23 269.004 L2096.85 269.004 Q2091.92 269.004 2090.05 267.175 Q2088.17 265.324 2088.17 260.462 L2088.17 246.388 L2085.05 246.388 L2085.05 243.078 L2088.17 243.078 L2088.17 235.717 L2092.46 235.717 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /><path clip-path=\"url(#clip310)\" d=\"M 0 0 M2122.22 243.842 L2122.22 247.87 Q2120.42 246.944 2118.47 246.481 Q2116.53 246.018 2114.45 246.018 Q2111.28 246.018 2109.68 246.99 Q2108.1 247.963 2108.1 249.907 Q2108.1 251.388 2109.24 252.245 Q2110.37 253.078 2113.8 253.842 L2115.26 254.166 Q2119.79 255.138 2121.69 256.921 Q2123.61 258.68 2123.61 261.851 Q2123.61 265.462 2120.74 267.569 Q2117.9 269.675 2112.9 269.675 Q2110.81 269.675 2108.54 269.259 Q2106.3 268.865 2103.8 268.055 L2103.8 263.657 Q2106.16 264.884 2108.45 265.509 Q2110.74 266.111 2112.99 266.111 Q2116 266.111 2117.62 265.092 Q2119.24 264.05 2119.24 262.175 Q2119.24 260.439 2118.06 259.513 Q2116.9 258.587 2112.94 257.731 L2111.46 257.384 Q2107.5 256.55 2105.74 254.838 Q2103.98 253.101 2103.98 250.092 Q2103.98 246.435 2106.58 244.444 Q2109.17 242.453 2113.94 242.453 Q2116.3 242.453 2118.38 242.801 Q2120.46 243.148 2122.22 243.842 Z\" fill=\"#000000\" fill-rule=\"evenodd\" fill-opacity=\"1\" /></svg>\n"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"plot(1:10, TrainD[:trainAccuracies], label=\"Training Accuracies\")\n",
"plot!(1:10, TrainD[:trainCosts], label=\"Training Costs\")\n",
"# plot!(1:10, TrainD[:testAccuracies], label=\"Test Accuracies\")\n",
"# plot!(1:10, TrainD[:testCosts], label=\"Test Costs\")\n",
"ylabel!(\"Epochs\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Predict After Training"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:00:22\u001b[39m\n",
"\u001b[34m Instances 60000: 60000\u001b[39m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"The accuracy of Train Data before the training process 0.902\n",
"The cost of Train Data before the training process 0.275\n"
]
}
],
"source": [
"TrainP = predict(model, X_train, Y_train);\n",
"\n",
"println()\n",
"println(\"The accuracy of Train Data before the training process $(round(TrainP[:accuracy], digits=4))\")\n",
"println(\"The cost of Train Data before the training process $(round(TrainP[:cost], digits=4))\")"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:00:03\u001b[39m\n",
"\u001b[34m Instances 10000: 10000\u001b[39m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"The accuracy of Test Data before the training process 0.8857\n",
"The cost of Test Data before the training process 0.342\n"
]
}
],
"source": [
"TestP = predict(model, X_test, Y_test);\n",
"\n",
"println()\n",
"println(\"The accuracy of Test Data before the training process $(round(TestP[:accuracy], digits=4))\")\n",
"println(\"The cost of Test Data before the training process $(round(TestP[:cost], digits=4))\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.4.1",
"language": "julia",
"name": "julia-1.4"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.4.1"
}
},
"nbformat": 4,
"nbformat_minor": 4
}