{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Get Started with Keras 3.0 + MLflow\n", "\n", "This tutorial is an end-to-end tutorial on training a MINST classifier with **Keras 3.0** and logging results with **MLflow**. It will demonstrate the use of `mlflow.keras.MlflowCallback`, and how to subclass it to implement custom logging logic.\n", "\n", "**Keras** is a high-level api that is designed to be simple, flexible, and powerful - allowing everyone from beginners to advanced users to quickly build, train, and evaluate models. **Keras 3.0**, or Keras Core, is a full rewrite of the Keras codebase that rebases it on top of a modular backend architecture. It makes it possible to run Keras workflows on top of arbitrary frameworks — starting with TensorFlow, JAX, and PyTorch." ] }, { "cell_type": "raw", "metadata": {}, "source": [ " Download this Notebook
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install Packages\n", "\n", "`pip install -q keras mlflow jax jaxlib torch tensorflow`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import Packages / Configure Backend\n", "Keras 3.0 is inherently multi-backend, so you will need to set the backend environment variable **before** importing the package." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "# You can use 'tensorflow', 'torch' or 'jax' as backend. Make sure to set the environment variable before importing.\n", "os.environ[\"KERAS_BACKEND\"] = \"tensorflow\"" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using TensorFlow backend\n" ] } ], "source": [ "import keras\n", "import numpy as np\n", "\n", "import mlflow" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load Dataset\n", "We will use the MNIST dataset. This is a dataset of handwritten digits and will be used for an image classification task. There are 10 classes corresponding to the 10 digits." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(28, 28, 1)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n", "x_train = np.expand_dims(x_train, axis=3)\n", "x_test = np.expand_dims(x_test, axis=3)\n", "x_train[0].shape" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAJOCAYAAAC++60XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8WgzjOAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbsElEQVR4nO3de3xU1b3///ckJJMASSBcElISiCCFgoU23AIKqDmgtgoKlXpqRev5ghi0iB780a/Ct9SeeGkVL1i0x0K1R0WsgHosHoUIagNKAOsFAlbQaEgANRcDuZBZvz9ynHazBknCJLOz83o+HvNo12fW7PlMmI/5ZM/aa3zGGCMAAAAPiop0AgAAAK2FRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzaHQAAIBn0egAAADPotEBAACeRaMDAAA8i0aniVatWiWfz6cDBw4063GTJk3SsGHDwppL//79dfXVV4f1mEBTUQsAddCe0Oh0YK+99pp8Pl/I29atWyOdHtCmamtrdeuttyotLU3x8fEaM2aMXnnllUinBUTMr3/9a/l8vrA3Zm2tU6QTQOTdeOONGjVqlCM2cODACGUDRMbVV1+tZ599VvPnz9eZZ56pVatW6aKLLlJ+fr7OPvvsSKcHtKlPP/1U//Ef/6EuXbpEOpXTRqMDnXPOOZoxY0ak0wAi5q233tLTTz+te+65R7fccosk6aqrrtKwYcO0cOFC/fWvf41whkDbuuWWWzR27Fg1NDToyJEjkU7ntPDRVQutX79eP/jBD5SWlia/368BAwboV7/6lRoaGkLOLyws1Lhx4xQfH6/MzEytWLHCmlNbW6slS5Zo4MCB8vv9Sk9P18KFC1VbW9vaL0dVVVU6fvx4qz8PvMcLtfDss88qOjpas2fPDsbi4uJ07bXXqqCgQMXFxa3yvPAOL9TB17Zs2aJnn31Wy5Yta9XnaSuc0WmhVatWqWvXrlqwYIG6du2qTZs2afHixaqsrNQ999zjmPvll1/qoosu0uWXX64rrrhCzzzzjObOnavY2Fj97Gc/kyQFAgFdcskleuONNzR79mwNGTJE7777ru677z7t3btX69atO2kugUBAX3zxRZPyTkpKUkxMjCN2zTXX6KuvvlJ0dLTOOecc3XPPPRo5cmTzfiDosLxQCzt37tSgQYOUmJjomDN69GhJ0q5du5Sent7UHwk6IC/UgSQ1NDTohhtu0L/927/prLPOav4Pwo0MmmTlypVGktm/f78xxpijR49ac+bMmWM6d+5sampqgrGJEycaSea3v/1tMFZbW2tGjBhhevfuberq6owxxjzxxBMmKirKvP76645jrlixwkgyb775ZjDWr18/M2vWrOB4//79RlKTbvn5+cHHvfnmm2b69OnmscceM+vXrzd5eXmmR48eJi4uzuzYseN0flzwMC/WwtChQ815551nvY7333/fSDIrVqxo1s8I3ufFOjDGmIceesgkJSWZQ4cOBfMdOnRoi35GbsEZnRaKj48P/v+qqirV1tbqnHPO0SOPPKI9e/Zo+PDhwfs7deqkOXPmBMexsbGaM2eO5s6dq8LCQo0dO1Zr1qzRkCFDNHjwYMfnoeedd54kKT8/X+PGjQuZS2pqapOvDvnnvMaNG+c45iWXXKIZM2bou9/9rhYtWqQNGzY06Zjo2LxQC8eOHZPf77fmxMXFBe8HvokX6uDzzz/X4sWLdfvtt6tXr15Ne+HtAI1OC73//vu67bbbtGnTJlVWVjruq6iocIzT0tKsleuDBg2SJB04cEBjx47Vvn37tHv37pO+uQ4dOnTSXOLi4pSTk9OSl2EZOHCgpk6dqueee04NDQ2Kjo4Oy3HhXV6ohfj4+JDrHmpqaoL3A9/EC3Vw2223KTk5WTfccEOzH+tmNDotUF5erokTJyoxMVFLly7VgAEDFBcXpx07dujWW29VIBBo9jEDgYDOOuss3XvvvSHv/6b1AQ0NDTp8+HCTnic5OVmxsbHfOCc9PV11dXWqrq621iwA/8wrtdCnTx999tln1pyDBw9KavzFBJyMF+pg3759evTRR7Vs2TKVlJQE76+pqVF9fb0OHDigxMREJScnN++FuACNTgu89tpr+vzzz/Xcc89pwoQJwfj+/ftDzi8pKVF1dbWjg9+7d6+kxh0tJWnAgAF65513dP7558vn8zUrn+LiYmVmZjZpbn5+viZNmvSNcz766CPFxcWpa9euzcoDHY9XamHEiBHKz89XZWWlo7nftm1b8H7gZLxQB5999pkCgYBuvPFG3Xjjjda8zMxM/fznP2+XV2LR6LTA1x/nGGOCsbq6Oj388MMh5x8/flyPPPKIFixYEJz7yCOPqFevXsrKypIkXX755XrppZf0+9//3nGJq9S4PiAQCJx046aWfh57+PBh67ToO++8o+eff14XXnihoqLYfQDfzCu1MGPGDP3mN7/Ro48+GtxHp7a2VitXrtSYMWO44grfyAt1MGzYMK1du9a6/7bbblNVVZXuv/9+DRgwoEnHdBsanRYYN26cunfvrlmzZunGG2+Uz+fTE0884XiT/7O0tDTdddddOnDggAYNGqTVq1dr165devTRR4OX9f30pz/VM888o+uuu075+fkaP368GhoatGfPHj3zzDN6+eWXT3rJd0s/j505c6bi4+M1btw49e7dWx988IEeffRRde7cWXfeeWezj4eOxyu1MGbMGP3oRz/SokWLdOjQIQ0cOFB//OMfdeDAAT322GPNPh46Fi/UQc+ePTVt2jQr/vUZnFD3tRuRvOSrPTnxUsI333zTjB071sTHx5u0tDSzcOFC8/LLL1uX6319ad727dtNdna2iYuLM/369TMPPfSQ9Rx1dXXmrrvuMkOHDjV+v990797dZGVlmV/+8pemoqIiOO/ESwlb6v777zejR482ycnJplOnTqZPnz7myiuvNPv27TvtY8O7vFgLxhhz7Ngxc8stt5jU1FTj9/vNqFGjzIYNG8JybHiPV+vgRF64vNxnzElaTgAAgHaORRgAAMCzaHQAAIBn0egAAADPotEBAACeRaMDAAA8q9UaneXLl6t///6Ki4vTmDFj9NZbb7XWUwGuRi0A1AEip1UuL1+9erWuuuoqrVixQmPGjNGyZcu0Zs0aFRUVqXfv3t/42EAgoJKSEiUkJDR722t4lzFGVVVVSktLa1c7NlMLCLf2WAunUwcStYDQmlwLrbE5z+jRo01ubm5w3NDQYNLS0kxeXt4pH1tcXGwkceMW8lZcXNwab9lWQy1wa61be6qF06kDY6gFbt98O1UthP0rIOrq6lRYWKhFixYFY1FRUcrJyVFBQcEpH5+QkCBJOlsXqZNiwp0e2qnjqtcbein4/mgPqAW0hvZWC6dbBxK1gNCaWgthb3SOHDmihoYGpaSkOOIpKSnas2ePNb+2tla1tbXBcVVV1f8mFqNOPt7Q+F+m8X/a02lragGtop3VQnPrQKIW0ERNrIWIf8Cbl5enpKSk4I1vCUZHRS0AjagFhFPYG52ePXsqOjpaZWVljnhZWZlSU1Ot+YsWLVJFRUXwVlxcHO6UgIigFoDm14FELSC8wt7oxMbGKisrSxs3bgzGAoGANm7cqOzsbGu+3+9XYmKi4wZ4AbUANL8OJGoB4RX2NTqStGDBAs2aNUsjR47U6NGjtWzZMlVXV+uaa65pjacDXItaAKgDRFarNDozZ87U4cOHtXjxYpWWlmrEiBHasGGDtRgN8DpqAaAOEFmtsmHg6aisrFRSUpImaSqr6xF03NTrNa1XRUVFhzmNTS0gFGqBWkCjptZCxK+6AgAAaC00OgAAwLNodAAAgGfR6AAAAM+i0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzWuW7rgCgLRw/L8uKHby+1jF+J/uP1pzhBbOsWNryWCsWnb/jNLID4Aac0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsFiNHiK+T/aOP7tWzxccruqW/Y9zQOWDN6TfgkBXrfL3PipXe61yUuWPkamvOkYZqKzZmzc1WbOCCrVYMaInAxO9ZsQf+8JAVGxjjrC27EqSd2SutWNHIBiv27/3HNj1BwMOqZ4xxjO+6+3fWnF9dfpUVM9vfa7WcmoozOgAAwLNodAAAgGfR6AAAAM+i0QEAAJ7FYuRmiB5yphUz/hgrVjKxmxU7Nta5eDc5yV7M+/pwe9FvOP3laIIVu+uhC6zYtrOedIz31x+z5txZ9i9WLO11cxrZAf9QP3mkFVv48BNWbFCMvZtx4ITlxx/V11tzKgJ+K/Y9O6TaC0c5xvH579rPV1NjPxCudmzqaDvWI9qKJf+hoC3SaRcOjXSeF/nVgYsjlEnzcUYHAAB4Fo0OAADwLBodAADgWazROYmGSd+3YveuWm7FQq0RcIN6Y29+tvjBq61Yp2p7XU32mnmOccJnx605/iP2up3O27c1I0N0VNGJiY5x9YTB1pyb7nvSip0b/1WIo536b7VVX46zYhsfzrZib/6/B6zYK/+5wjH+zp/mWXPOuJV1HO1NyQT7fdN5QLk98Q+tn4srRdnrlUyG87/55/feY83Z6LNrzQ04owMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZLEY+CX9RiRUrrEm3YoNiylo1j5sP2t+e/NFX9recrxrwrGNcEbAXGac88New5cXWgGipTx//lmP89ih7kX84Le39thXb0NVeNHnNgclW7I/9X3WME7/zefgSQ8T88odrrNhdu+1//44qekA/K7ZnonNl9oi3rrTmpL1tb6jpBpzRAQAAnkWjAwAAPItGBwAAeBaNDgAA8CwWI5/E8YOlVuzBu35kxX59gf0t5NF/62rF3rn+wVM+5x1HvmvFPszpbMUayg9asX/Nvt4xPnCjffxMvXPKHIBwOn5elhV7asRDjnGUmra7+DUfn2/Ftr86xIq9e63z+PnH4qw5vbfbO3t/+KW9Q3PMf+Q7xlG+U6aJdiDGZ+/2jn/o9J9HTznn2N8TTznHLTijAwAAPItGBwAAeFazG50tW7bo4osvVlpamnw+n9atW+e43xijxYsXq0+fPoqPj1dOTo727dsXrnwBV6AOgEbUAtyu2Y1OdXW1hg8fruXLQ2/ydffdd+uBBx7QihUrtG3bNnXp0kVTpkxRTU3NaScLuAV1ADSiFuB2zV6MfOGFF+rCCy8MeZ8xRsuWLdNtt92mqVOnSpIef/xxpaSkaN26dfrxj398etlGWPLKAivW64UeVqzh8y+s2NBhP3OM35/wB2vO849OtGK9y5u2m7GvwLnQONNOFWHUkevgZAITv2fFHvjDQ1ZsYIzzPzsBBaw5l+y51IpFz7AX/nf7gb1H93eemOcYD1pebM2JKt5pxbq/boVU/+sGx/jP37Xr9mfn2iv/o/N32AfzKLfXQuDsEVbsnLg3Wv1527P+XU69A3j6qw2nnOMWYV2js3//fpWWlionJycYS0pK0pgxY1RQwG9edAzUAdCIWoAbhPXy8tLSxkuyU1JSHPGUlJTgfSeqra1VbW1tcFxZWRnOlIA215I6kKgFeA+1ADeI+FVXeXl5SkpKCt7S0+0vzgQ6AmoBaEQtIJzC2uikpqZKksrKnN/oXVZWFrzvRIsWLVJFRUXwVlxsf54OtCctqQOJWoD3UAtwg7B+dJWZmanU1FRt3LhRI0aMkNR4ynHbtm2aO3duyMf4/X75/f5wptGmGo6cetGWJNVXnnr316E/+cCKHf5dtD0x0H4WgXVELakDqX3Vgi9rqBU7ssDebXhQjP2+L6x1jjd99R1rzudP23/B9/jSXtOR9KetduyEcTj3wE2Jtv99Pp9v7yLbO98KdUhuqIWPfxhvxXpH2zvOd1Sd+mdYsRnJz5/ycfH7v7Ribv3N1OxG56uvvtKHH34YHO/fv1+7du1ScnKyMjIyNH/+fN1xxx0688wzlZmZqdtvv11paWmaNm1aOPMGIoo6ABpRC3C7Zjc627dv17nnnhscL1iwQJI0a9YsrVq1SgsXLlR1dbVmz56t8vJynX322dqwYYPi4uzvmwHaK+oAaEQtwO2a3ehMmjRJxth7V3zN5/Np6dKlWrp06WklBrgZdQA0ohbgdnx7eRsZcutex/ias+xvYl7Zb6MVm/ijXCuWsNpelwC0lqjO9nqG43fbl/tuHfycFdt/vM6KLfjFzY5x99c/seb07nLIirn18//RfT62YgfaPg2cRKeBVU2aV7OnW+sm4lLFy7pYsfF+exPPxyr7OgPl7eeS/4hfXg4AANBaaHQAAIBn0egAAADPotEBAACexWLkNtJQXuEYfz53iDXnk+ftDdf+vzset2KLLre/2dnsdG6Tlv7rEF+Y9w1XRgAnc2yivTngy4MfbtJj/+3nN1mxhHXOxfTh3NAPaKne2+0FuO1JdM8eVqxs+iDHOPnyT605mwc9FuJo9qX/v1s+zTHuXfbXZuUXSZzRAQAAnkWjAwAAPItGBwAAeBaNDgAA8CwWI0dI4J3dVuzHv/x3K/ZfS35jxXaNtRcoa6xzOLTLPGvKmb8/aMWOf3Tg5EkCkr77q11WLCrE30jXfGzv9h2/7q3WSKnNxPiiHeP6EOv5o30s8veCY8n2e9reM7hpAud8z4qZaJ8VK85xfkN7XVq9NScq1t4T/H/OedCKxdiHV2mD8/i3f2RfyPJFwF6E3TnKfs6Ubc4dptvTu54zOgAAwLNodAAAgGfR6AAAAM+i0QEAAJ7FYmQXSf6DvZvxvKJcK5Z4p7275VNnvOwYv3/VQ9acwen/ZsW+/Uu7123Y99E35glvK/9ptmN8W4q9ID6gWCtW+D/fsWIZaj+7p4ZSb5yLMgOyF25u2G2/7jO1o9VyQvPU1sRYsUCIpbQrf3GfFXt+3ogWPeetPf7TikXJXi18zNQ5xiUN9iLghw5PsmI5r863Yt122jXZ53/KHGPfx/bvjsO7461YSrS9KNq8/a4Vay84owMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZLEZ2Od+bu6zY0Rm9rdiomTc4xttuvd+as+dce4HcT/pPtmIVZzcjQXjO8RPWJiZF2YscC2r8VuyMx0vsY4Utq/CK6tzZiu35zbAQMwsdo598dKE1Y/DP91sxe0kpImXglTut2NA8e+f49FGfhe058w8NsmKH/9LXivV437noN3bD2yGOZi8MHqTtTcrjxPfhZ7eOs+aM8tsXwTz91beadPz2gjM6AADAs2h0AACAZ9HoAAAAz2KNTjvUUHbIiqU84IzVLLRXR3T22Wstft//RSv2w0vnOx+3dlszM4TXfd7Q1Yod/+hA2yfSBKHW4xTdeZYV2zPV3mTzL0eTHOOS5QOtOQlfbj2N7BAJmYvsdSmtrY8+afPnPFHnCYebNO+2/OlWbJDeCnc6bYYzOgAAwLNodAAAgGfR6AAAAM+i0QEAAJ7FYmSXC5w9wor9/UdxVmzYiAOOcaiFx6E8+MX3rFjn9U3bjAod1y1v/siKDTphc71ICUx0vqcPLThmzdk90l54fP67M61Ylws+cowTxMJjeF+/9fa3u7dnnNEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBYjR4hvpP1NyXtvDLFz8fg/WrEJcXUtes5aY38L7tYvMu2JgYMtOj48wuccRoX4e+j+s5+yYstlf2Nza/t4abYV+/NV9zrGg2Lsuvr+W7OsWNqlH4QvMQCuwRkdAADgWTQ6AADAs5rV6OTl5WnUqFFKSEhQ7969NW3aNBUVFTnm1NTUKDc3Vz169FDXrl01ffp0lZWVhTVpINKoBaARtQC3a1ajs3nzZuXm5mrr1q165ZVXVF9fr8mTJ6u6ujo456abbtILL7ygNWvWaPPmzSopKdFll10W9sSBSKIWgEbUAtyuWYuRN2zY4BivWrVKvXv3VmFhoSZMmKCKigo99thjevLJJ3XeeedJklauXKkhQ4Zo69atGjt2bPgyd7FOmf2s2N+vSXOM/9/Mp60507seCVsOvygbacU232///Lv/sSBsz9mReLoWTtgUNaCANWVi/OdWbP6qLCs2YKX92JjSKse4bGIva07yzE+t2A0ZG63YhZ3t3Zifr05xjK969wJrTs9HulgxtIyna6EDiPbZ5zu+HBRjxVL/0hbZtI7TWqNTUVEhSUpOTpYkFRYWqr6+Xjk5OcE5gwcPVkZGhgoKQv9Cra2tVWVlpeMGtDfUAtCIWoDbtLjRCQQCmj9/vsaPH69hwxovlS4tLVVsbKy6devmmJuSkqLS0tKQx8nLy1NSUlLwlp6e3tKUgIigFoBG1ALcqMWNTm5urt577z09/bT9EUxzLFq0SBUVFcFbcXHxaR0PaGvUAtCIWoAbtWjDwHnz5unFF1/Uli1b1Ldv32A8NTVVdXV1Ki8vd3TvZWVlSk1NDXksv98vv9/fkjTaXKf+GVasIquPFZu5dIMVu67bc2HL4+aD9mfaBQ871+Qkr3rLmtM9wHqccOuotRDns//TsftfVlixN86Js2L7ap2v/5qkAy3O4+cl51ixDX8d4Rif+XO+cbwtdNRaaO8ajL2OzmsbzzTr5RhjNG/ePK1du1abNm1SZqZzV92srCzFxMRo48Z/LBosKirSJ598ouxsewdToL2iFoBG1ALcrllndHJzc/Xkk09q/fr1SkhICH6+mpSUpPj4eCUlJenaa6/VggULlJycrMTERN1www3Kzs5mZT08hVoAGlELcLtmNTq/+93vJEmTJk1yxFeuXKmrr75aknTfffcpKipK06dPV21traZMmaKHH344LMkCbkEtAI2oBbhdsxodY8wp58TFxWn58uVavnx5i5MC3I5aABpRC3A7vr38f3Xq41wU98Uf7A3F5mZutmJXJIRvG/N5n51txXb8boQV6/nse1YsuYqFxgiPlNcOOca3zrHXUdyV2rT324S4Oit2dtyBUz5uZ629fPCKzbOt2KBr7A0DzxSLj4HTcXTU0UinEFYeW1sNAADwDzQ6AADAs2h0AACAZ9HoAAAAz/L8YuS6Kfa3eNfd9IUV+8XAlxzjyfHVYc2jrOGYYzzh+ZutOYNv22PFksvtRZ8h9rEEwqZh798d430/6m/N+c4NN1ixDy5/sEXPN/il663Ytx+2F0MO2mkvPAZwekJ9e7nXeP8VAgCADotGBwAAeBaNDgAA8CwaHQAA4FmeX4x8YJrdy+09a02LjrW8fIAVu3/zZCvma/BZscF37HeMzyzbZs1paFFWQOs6/tEBKzbwJjt2yU2jWnT8QXrbip36SwUANFftq72sWMMI71/ewhkdAADgWTQ6AADAs2h0AACAZ9HoAAAAz/L8YuRBc9+yYj+cmxW+48s+figsNAYARFLqfX+1Yhfd930rdoZ2tUE2bYczOgAAwLNodAAAgGfR6AAAAM+i0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzaHQAAIBn0egAAADPotEBAACe5bov9TTGSJKOq14yEU4GrnFc9ZL+8f7oCKgFhEItRDgZuEZTa8F1jU5VVZUk6Q29FOFM4EZVVVVKSkqKdBptglrAN6EWgEanqgWfcdmfBYFAQCUlJUpISFBVVZXS09NVXFysxMTESKfWbJWVle02f7flboxRVVWV0tLSFBXVMT5x9UotuO291Fxuy78j14IxRhkZGa75t2gut72Xmstt+Te1Flx3RicqKkp9+/aVJPl8PklSYmKiK36oLdWe83dT7h3lr9evea0W2nPukrvy76i1UFlZKcld/xYtQf7h05Ra6Bh/DgAAgA6JRgcAAHiWqxsdv9+vJUuWyO/3RzqVFmnP+bfn3L2oPf97tOfcpfafv5e0938L8o8M1y1GBgAACBdXn9EBAAA4HTQ6AADAs2h0AACAZ9HoAAAAz3Jto7N8+XL1799fcXFxGjNmjN56661IpxTSli1bdPHFFystLU0+n0/r1q1z3G+M0eLFi9WnTx/Fx8crJydH+/bti0yyIeTl5WnUqFFKSEhQ7969NW3aNBUVFTnm1NTUKDc3Vz169FDXrl01ffp0lZWVRSjjjodaaBvUgvtRC63Pi3XgykZn9erVWrBggZYsWaIdO3Zo+PDhmjJlig4dOhTp1CzV1dUaPny4li9fHvL+u+++Ww888IBWrFihbdu2qUuXLpoyZYpqamraONPQNm/erNzcXG3dulWvvPKK6uvrNXnyZFVXVwfn3HTTTXrhhRe0Zs0abd68WSUlJbrssssimHXHQS20HWrB3aiFtuHJOjAuNHr0aJObmxscNzQ0mLS0NJOXlxfBrE5Nklm7dm1wHAgETGpqqrnnnnuCsfLycuP3+81TTz0VgQxP7dChQ0aS2bx5szGmMd+YmBizZs2a4Jzdu3cbSaagoCBSaXYY1ELkUAvuQi1EhhfqwHVndOrq6lRYWKicnJxgLCoqSjk5OSooKIhgZs23f/9+lZaWOl5LUlKSxowZ49rXUlFRIUlKTk6WJBUWFqq+vt7xGgYPHqyMjAzXvgavoBYii1pwD2ohcrxQB65rdI4cOaKGhgalpKQ44ikpKSotLY1QVi3zdb7t5bUEAgHNnz9f48eP17BhwyQ1vobY2Fh169bNMdetr8FLqIXIoRbchVqIDK/Ugeu+vRyRk5ubq/fee09vvPFGpFMBIopaALxTB647o9OzZ09FR0dbK7jLysqUmpoaoaxa5ut828NrmTdvnl588UXl5+erb9++wXhqaqrq6upUXl7umO/G1+A11EJkUAvuQy20PS/VgesandjYWGVlZWnjxo3BWCAQ0MaNG5WdnR3BzJovMzNTqampjtdSWVmpbdu2uea1GGM0b948rV27Vps2bVJmZqbj/qysLMXExDheQ1FRkT755BPXvAavohbaFrXgXtRC2/FkHUR4MXRITz/9tPH7/WbVqlXmgw8+MLNnzzbdunUzpaWlkU7NUlVVZXbu3Gl27txpJJl7773X7Ny503z88cfGGGPuvPNO061bN7N+/Xrzt7/9zUydOtVkZmaaY8eORTjzRnPnzjVJSUnmtddeMwcPHgzejh49Gpxz3XXXmYyMDLNp0yazfft2k52dbbKzsyOYdcdBLbQdasHdqIW24cU6cGWjY4wxDz74oMnIyDCxsbFm9OjRZuvWrZFOKaT8/HwjybrNmjXLGNN4KeHtt99uUlJSjN/vN+eff74pKiqKbNL/JFTukszKlSuDc44dO2auv/560717d9O5c2dz6aWXmoMHD0Yu6Q6GWmgb1IL7UQutz4t14DPGmNY9ZwQAABAZrlujAwAAEC40OgAAwLNodAAAgGfR6AAAAM+i0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzaHQAAIBn0egAAADPotEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBodAADgWTQ6TbRq1Sr5fD4dOHCgWY+bNGmShg0bFtZc+vfvr6uvvjqsxwSailoAqIP2hEanA/vqq6+0ZMkSXXDBBUpOTpbP59OqVasinRYQEYWFhbrggguUmJiohIQETZ48Wbt27Yp0WkCbefvttzVv3jwNHTpUXbp0UUZGhi6//HLt3bs30qmdFhqdDuzIkSNaunSpdu/ereHDh0c6HSBiduzYobPPPlsfffSRlixZosWLF2vfvn2aOHGiioqKIp0e0Cbuuusu/fnPf9b555+v+++/X7Nnz9aWLVv0/e9/X++9916k02uxTpFOAJHTp08fHTx4UKmpqdq+fbtGjRoV6ZSAiLj99tsVHx+vgoIC9ejRQ5J05ZVXatCgQfrFL36hP//5zxHOEGh9CxYs0JNPPqnY2NhgbObMmTrrrLN055136k9/+lMEs2s5zui00Pr16/WDH/xAaWlp8vv9GjBggH71q1+poaEh5PzCwkKNGzdO8fHxyszM1IoVK6w5tbW1WrJkiQYOHCi/36/09HQtXLhQtbW1rfIa/H6/UlNTW+XY6Di8UAuvv/66cnJygk2O1PiHwMSJE/Xiiy/qq6++apXnhXd4oQ7GjRvnaHIk6cwzz9TQoUO1e/fuVnnOtsAZnRZatWqVunbtqgULFqhr167atGmTFi9erMrKSt1zzz2OuV9++aUuuugiXX755briiiv0zDPPaO7cuYqNjdXPfvYzSVIgENAll1yiN954Q7Nnz9aQIUP07rvv6r777tPevXu1bt26k+YSCAT0xRdfNCnvpKQkxcTEtPh1AyfyQi3U1tYqPj7emtO5c2fV1dXpvffe09ixY5v4E0FH5IU6CMUYo7KyMg0dOrRJx3MlgyZZuXKlkWT2799vjDHm6NGj1pw5c+aYzp07m5qammBs4sSJRpL57W9/G4zV1taaESNGmN69e5u6ujpjjDFPPPGEiYqKMq+//rrjmCtWrDCSzJtvvhmM9evXz8yaNSs43r9/v5HUpFt+fn7I1/f2228bSWblypXN/Mmgo/FiLZx11llm0KBB5vjx447cMjIyjCTz7LPPtuhnBe/yYh2E8sQTTxhJ5rHHHmvqj8Z1OKPTQv/8119VVZVqa2t1zjnn6JFHHtGePXsci3s7deqkOXPmBMexsbGaM2eO5s6dq8LCQo0dO1Zr1qzRkCFDNHjwYB05ciQ497zzzpMk5efna9y4cSFzSU1N1SuvvNKkvFl0jHDzQi1cf/31mjt3rq699lotXLhQgUBAd9xxhw4ePChJOnbsWJOOiY7LC3Vwoj179ig3N1fZ2dmaNWtWk47nRjQ6LfT+++/rtttu06ZNm1RZWem4r6KiwjFOS0tTly5dHLFBgwZJkg4cOKCxY8dq37592r17t3r16hXy+Q4dOnTSXOLi4pSTk9OSlwGcNi/UwnXXXafi4mLdc889+uMf/yhJGjlypBYuXKhf//rX6tq1a7OPiY7FC3Xwz0pLS/WDH/xASUlJevbZZxUdHX1ax4skGp0WKC8v18SJE5WYmKilS5dqwIABiouL044dO3TrrbcqEAg0+5iBQEBnnXWW7r333pD3p6enn/SxDQ0NOnz4cJOeJzk52VpsBrSUl2rh17/+tW655Ra9//77SkpK0llnnaVf/OIXkv7xSwgIxUt1IDU2ZhdeeKHKy8v1+uuvKy0tremJuxCNTgu89tpr+vzzz/Xcc89pwoQJwfj+/ftDzi8pKVF1dbWjg/96A6b+/ftLkgYMGKB33nlH559/vnw+X7PyKS4uVmZmZpPm5ufna9KkSc06PnAyXquF7t276+yzzw6OX331VfXt21eDBw9uVh7oWLxUBzU1Nbr44ou1d+9evfrqq/rOd77TrOd2IxqdFvj6FJ4xJhirq6vTww8/HHL+8ePH9cgjj2jBggXBuY888oh69eqlrKwsSdLll1+ul156Sb///e81e/Zsx+OPHTumQCBgner8Gmt0ECleroXVq1fr7bff1m9+8xtFRbETB07OK3XQ0NCgmTNnqqCgQOvXr1d2dnaTjuF2NDotMG7cOHXv3l2zZs3SjTfeKJ/PpyeeeMLxJv9naWlpuuuuu3TgwAENGjRIq1ev1q5du/Too48GL+v76U9/qmeeeUbXXXed8vPzNX78eDU0NGjPnj165pln9PLLL2vkyJEhj386n8c+9NBDKi8vV0lJiSTphRde0KeffipJuuGGG5SUlNSi46Jj8EotbNmyRUuXLtXkyZPVo0cPbd26VStXrtQFF1ygn//8580+HjoWr9TBzTffrOeff14XX3yxvvjiC2uDwCuvvLLZx3SFSF7y1Z6ceCnhm2++acaOHWvi4+NNWlqaWbhwoXn55Zety/UmTpxohg4darZv326ys7NNXFyc6devn3nooYes56irqzN33XWXGTp0qPH7/aZ79+4mKyvL/PKXvzQVFRXBeSdeSng6+vXrd9LLDr9+rcA/82ItfPjhh2by5MmmZ8+exu/3m8GDB5u8vDxTW1t72seGN3mxDr6+9P1kt/bKZ8xJWk4AAIB2jg+eAQCAZ9HoAAAAz6LRAQAAnkWjAwAAPItGBwAAeFarNTrLly9X//79FRcXpzFjxuitt95qracCXI1aAKgDRE6rXF6+evVqXXXVVVqxYoXGjBmjZcuWac2aNSoqKlLv3r2/8bGBQEAlJSVKSEho9rbX8C5jjKqqqpSWltaudqmlFhBu7bEWTqcOJGoBoTW5Flpjc57Ro0eb3Nzc4LihocGkpaWZvLy8Uz62uLj4Gzcs4taxb8XFxa3xlm011AK31rq1p1o4nTowhlrg9s23U9VC2L8Coq6uToWFhVq0aFEwFhUVpZycHBUUFJzy8QkJCZKks3WROikm3OmhnTquer2hl4Lvj/aAWkBraG+1cLp1IFELCK2ptRD2RufIkSNqaGhQSkqKI56SkqI9e/ZY82tra1VbWxscV1VV/W9iMerk4w2N/2Ua/6c9nbamFtAq2lktNLcOJGoBTdTEWoj4B7x5eXlKSkoK3tLT0yOdEhAR1ALQiFpAOIW90enZs6eio6NVVlbmiJeVlSk1NdWav2jRIlVUVARvxcXF4U4JiAhqAWh+HUjUAsIr7I1ObGyssrKytHHjxmAsEAho48aNys7Otub7/X4lJiY6boAXUAtA8+tAohYQXmFfoyNJCxYs0KxZszRy5EiNHj1ay5YtU3V1ta655prWeDrAtagFgDpAZLVKozNz5kwdPnxYixcvVmlpqUaMGKENGzZYi9EAr6MWAOoAkdUqGwaejsrKSiUlJWmSprK6HkHHTb1e03pVVFR0mNPY1AJCoRaoBTRqai1E/KorAACA1kKjAwAAPItGBwAAeBaNDgAA8CwaHQAA4Fk0OgAAwLNodAAAgGfR6AAAAM+i0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzaHQAAIBndYp0Amg7f78n24rt/teHrFiML9oxnnD9bGtO/Lq3wpcYAOC0RPdItmK+pETH+JPpadacmp7Gig385TtWLHD06GlkF1mc0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsFiN7VOlN46zYazPvtmL1JvbUB7PXqgEA2kDUsMFWbN+ieCv2s7P+asVu7vFyi55zSMp1VuzMqwtbdCw34IwOAADwLBodAADgWTQ6AADAs1ij41FfpQesWHJUE9bjAG2sbspIK/bxT+z379zvb7Zi87vvPeXxz/rPG6xY54P2wrPycbVWrN9/Of8WjH15+ymfD2gq36izHOMPb4q25rx2tr2pa69ovxWLCnHe4r+PdneMP6rtbc3J7V5kxZ6Y8Hsr9qtRsxxj8/a71hy34owOAADwLBodAADgWTQ6AADAs2h0AACAZ7EY2SO++tEYx/jPl94fYpbPiqwotzejevVy5+LQLh+/b82xl4oCTXP4umzH+MGFy605I/0NVizUYstZB3Ks2PeSPnGM3/m3ULVgC3X8cclXOMbJLdt/DR1MdK9eVmzv/d+yYi+Me9gxPiMmJsTR7IXHoaysTLdi66af7RgH/Pbxc1+0FyOHqr9jKc5NCuOalJU7cEYHAAB4Fo0OAADwLBodAADgWTQ6AADAs1iM3A7V/HC0FVuS9wfHeFCMvfA4lD/+/gIrlvqB/S24wKn4Yuydt2tyhluxPy+6xzFO62Qvtrz243+xYh//5ttWrMt/77Ji+Z0zHOPNawfZOZz5vBULpXJXD8c4uUmPQkf32ZVnWrH3J4ZaFB9q8fGp/SnUwuNp46xYQ5Fz53Df94a26PnaO87oAAAAz6LRAQAAntXsRmfLli26+OKLlZaWJp/Pp3Xr1jnuN8Zo8eLF6tOnj+Lj45WTk6N9+/aFK1/AFagDoBG1ALdrdqNTXV2t4cOHa/lye5MvSbr77rv1wAMPaMWKFdq2bZu6dOmiKVOmqKam5rSTBdyCOgAaUQtwu2YvRr7wwgt14YUXhrzPGKNly5bptttu09SpUyVJjz/+uFJSUrRu3Tr9+Mc/Pr1sIUk6eKX9H4hz40+MRVtzQu0im3o/C49bgjqwHZw30oq9dUuoBZjOxcc/+vBia8bx6fVWrPORbVbMhDh6yewsx3jbmU3bGfkvRxOs2MBHip15NelIHQu1YPvWJQda9Lhnv0q1YvfuPd+KpSy03/kNRac+S/blWYktyqu9C+sanf3796u0tFQ5Of/4hZqUlKQxY8aooKAgnE8FuBZ1ADSiFuAGYb28vLS0VJKUkpLiiKekpATvO1Ftba1qa2uD48rKynCmBLS5ltSBRC3Ae6gFuEHEr7rKy8tTUlJS8Jaebu8PAHQE1ALQiFpAOIW10UlNbfx8sayszBEvKysL3neiRYsWqaKiIngrLi4OOQ9oL1pSBxK1AO+hFuAGYf3oKjMzU6mpqdq4caNGjBghqfGU47Zt2zR37tyQj/H7/fL7m/Y19B1Rp77fsmLvn7PSitWbBsd4t72WU5/ca+8Q20X2Ak+cnpbUgdS+amHfg2OsWNFlD1qxQIjHDnnlOsd48C0HrDkNRz5vaWq6bu76Fj3ujl/PsmLdi1lHcjo6Qi2E9H/s3L+Te4MVS3/F+d/tLu/bH+f1/HivFWuwIk1zNKVpO+Z7TbMbna+++koffvhhcLx//37t2rVLycnJysjI0Pz583XHHXfozDPPVGZmpm6//XalpaVp2rRp4cwbiCjqAGhELcDtmt3obN++Xeeee25wvGDBAknSrFmztGrVKi1cuFDV1dWaPXu2ysvLdfbZZ2vDhg2Ki4sLX9ZAhFEHQCNqAW7X7EZn0qRJMibU7hWNfD6fli5dqqVLl55WYoCbUQdAI2oBbse3l7tI9FD725lHPvlei44187kbrdiAP29t0bHQsf39t2OtWNFl9i64FQF7I8sf7flXK/btG5xrDhqqqpqUR1SXLlbs8xnftWJTuzq/HT1K8dacwWtyrdjAVazHQXg0fLjfig28yY6dqLU3pKwf1bRa85qIX14OAADQWmh0AACAZ9HoAAAAz6LRAQAAnsViZBf5+JIeVuzZHjtDzLS/mfxf/+78BuhBd/7dmtPSTabQsUSn9HaM/3jpw9acQIitAEMtPI79l49DPPbUokZ8x4oN+8NuK3ZHygMhHu3crG38Lvsbsr/9/+xjUR9wo08Wj7NixzuHuMot1F6AJ0y77MymLbif9+kkKxa/Ycc3HdrVOKMDAAA8i0YHAAB4Fo0OAADwLBodAADgWSxGjpAvrsm2YmuvuyfEzBgrcl3xRCtWP8u5ALPh8Cctzg0dmy/O+V4a6W/aMt34G2PtY/VLt2L7ruvrGE/O2WHNuan3o1Yso5O9w3Gohc0NJ3wdgW91T3tO+b4QjwRaT3RiohWrGX2mYxyzqMya87fBDzbp+DE++yKVenPq2s0/1tmKfTo7w4qZ4/YC/vaCMzoAAMCzaHQAAIBn0egAAADPotEBAACexWLkNhI99NuO8V/veCjErLgmHavg0/5WLP3Aey3ICrCZmlrHeFutvSB+jL/eiq1/9WkrFmoH5aZ49Zi9gHhfvb0X67nxX1mx7XXORdHdHm/abrBAS/j8fitWN/EsK3bTw09YsXPjNzrGZQ211pz8Y92t2OK9U63YU0NXWbG0TnZuJ4qLsmv5o8u7WbEzipy/nwI1Nac8tltwRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzWIzcRvb+wrn7ZFN2rDyZjDvtmL1ME2iZhrJDjvGSuf9mzfnNioet2HftjZH1p0p7Z+Q7Nl/iGA9aZS9q7FRWYcV6P/WFFTs3fZMVm5XvzHeQttuJAS0QFWdfMPL5zO9Zsdf/44EmHW/oUzc4xn3z7d8L/v9+24r16GMvwn/q5SwrdnOPU1+kEurCgr9dbeefXXyjY5zy+DvWnMDRo6d8vkjgjA4AAPAsGh0AAOBZNDoAAMCzWKPTCgIT7c9s7xi5rkXH+pf3fmzFum5nc0C0ndiX7TUuv8gc3eLjDdJbp5xTNdU+/n9nrLdi9cb+Wy3+QIjFQkALnLgZ4J57v2vN2TO1aetxphZNs2KD7vnIMT5xfZwkdUrva8WGP/+JFfv3Hh9YsYpAnWM85s83W3P6DLafc+NZq61Ywe3O1znzih9ac448YG+UGPe5vQYolOjXdjRpXktwRgcAAHgWjQ4AAPAsGh0AAOBZNDoAAMCzWIzcCn696lErNizm1Fv63XJwghVLuuJLK9byrQaB9uF4vP03WKhNNkN9O3rmKudCzePhSwse5utk/zosWjbcMd5zyXJrzqfH7W8cv+SRhVas/x/+bsWOn7D4uD7H3vRv2F07rdiS3oVWbGVlPyv2xP+92DEe+NxWa050zx5WbNK/3GDFqmc6N/Fc+73fW3P6PnDqb0uXpBer7ed8dNAZTXpsS3BGBwAAeBaNDgAA8CwaHQAA4Fk0OgAAwLNYjNwKvhfbtIWUJypY+X0r1vvLv4YlJ6A9SXjaXjSp37Z9Hug4iv/d3o17zyX3O8YlIRYe/+jOf7di/dd9ZMW+OC/TipkrExzjZ4fdb83pFW0v8B36tL1YeNCjR6xY56JtVuxEDUc+t2KJT4WKOcczrrcXXKfM+PiUzydJurlbiOD7TXtsC3BGBwAAeBaNDgAA8KxmNTp5eXkaNWqUEhIS1Lt3b02bNk1FRUWOOTU1NcrNzVWPHj3UtWtXTZ8+XWVlZWFNGog0agFoRC3A7ZrV6GzevFm5ubnaunWrXnnlFdXX12vy5Mmqrq4Ozrnpppv0wgsvaM2aNdq8ebNKSkp02WWXhT1xIJKoBaARtQC38xljTr1l70kcPnxYvXv31ubNmzVhwgRVVFSoV69eevLJJzVjxgxJ0p49ezRkyBAVFBRo7NixpzxmZWWlkpKSNElT1ckX09LU2kzxs8Os2Afj/mTFmrIYeeq/XGHFGj7Y27LEPOa4qddrWq+KigolJiZGOh0LtRBeVT+2fz6bf2vvShtqZ+TLxl7qGB8v/jR8ibkAtdA6tfB/P9plxcb46x3jLxrsxcgrvhxjxb4Va+9oPyuxiQt1TzD0yRut2MBFb1sxc7zj7QHe1Fo4rTU6FRWNW0InJydLkgoLC1VfX6+cnJzgnMGDBysjI0MFBQUhj1FbW6vKykrHDWhvqAWgEbUAt2lxoxMIBDR//nyNHz9ew4Y1ntUoLS1VbGysunXr5pibkpKi0tLSkMfJy8tTUlJS8Jaent7SlICIoBaARtQC3KjFjU5ubq7ee+89Pf3006eVwKJFi1RRURG8FRcXn9bxgLZGLQCNqAW4UYs2DJw3b55efPFFbdmyRX379g3GU1NTVVdXp/Lyckf3XlZWptTU1JDH8vv98vub9o2nkRaY+D0rtmxE09bjVARqHONRf5lvzRn88QctTw4R0VFrobVVnMHOF+1Ne6+FLV8NtmJj/O86xskhNu/7Rc9dTTr+D/fYi68/KejrGJ/xbIU1Z+D79jeVd8T1OKejWf81McZo3rx5Wrt2rTZt2qTMTOdOj1lZWYqJidHGjRuDsaKiIn3yySfKzs4OT8aAC1ALQCNqAW7XrDM6ubm5evLJJ7V+/XolJCQEP19NSkpSfHy8kpKSdO2112rBggVKTk5WYmKibrjhBmVnZzdpZT3QXlALQCNqAW7XrEbnd7/7nSRp0qRJjvjKlSt19dVXS5Luu+8+RUVFafr06aqtrdWUKVP08MMPhyVZwC2oBaARtQC3a1aj05Qtd+Li4rR8+XItX27veQF4BbUANKIW4HZ8e3kz1CTHWrGz46pDzIy2Ii8fzXCMB822N3yytz4DOqZvbT5qxWLm2XVV3+LtTgGnv56bZsXG/OQ8x7hieJ01p9NhewPDQSs+s+eVHrJi/WucV5PxO6B1cGkDAADwLBodAADgWTQ6AADAs2h0AACAZ7EYGYDr+N7cZcVWVfa2Ylck2Is+jw7t4xjHeuzby9E6Gj7/woqlPPBX57iJx2LfYnfhjA4AAPAsGh0AAOBZNDoAAMCzaHQAAIBnsRi5GRJ3lVqxGz49z4qtSN/cFukAHcp9j8ywYlfccr8V63P7h47x5+XftQ+29W9hywuAu3FGBwAAeBaNDgAA8CwaHQAA4Fk0OgAAwLNYjNwMx/d/bMU+HWvP+6Gy2iAboGP51hNFVmzmtB9asdUDX3SMJy6+wpqT/K9JVqyhvOI0sgPgVpzRAQAAnkWjAwAAPItGBwAAeBZrdAC0Cw1HPrdiddN7WLEhv53jGO/OecSac8nga+0nYBNBwJM4owMAADyLRgcAAHgWjQ4AAPAsGh0AAOBZLEYG0G6FWqB85ixn7BKNCvFIFh4DHQVndAAAgGfR6AAAAM+i0QEAAJ7lujU6xhhJ0nHVSybCycA1jqte0j/eHx0BtYBQqIUIJwPXaGotuK7RqaqqkiS9oZcinAncqKqqSklJ9jdPexG1gG9CLQCNTlULPuOyPwsCgYBKSkqUkJCgqqoqpaenq7i4WImJiZFOrdkqKyvbbf5uy90Yo6qqKqWlpSkqqmN84uqVWnDbe6m53JZ/R64FY4wyMjJc82/RXG57LzWX2/Jvai247oxOVFSU+vbtK0ny+XySpMTERFf8UFuqPefvptw7yl+vX/NaLbTn3CV35d9Ra6GyslKSu/4tWoL8w6cptdAx/hwAAAAdEo0OAADwLFc3On6/X0uWLJHf7490Ki3SnvNvz7l7UXv+92jPuUvtP38vae//FuQfGa5bjAwAABAurj6jAwAAcDpodAAAgGfR6AAAAM+i0QEAAJ7l2kZn+fLl6t+/v+Li4jRmzBi99dZbkU4ppC1btujiiy9WWlqafD6f1q1b57jfGKPFixerT58+io+PV05Ojvbt2xeZZEPIy8vTqFGjlJCQoN69e2vatGkqKipyzKmpqVFubq569Oihrl27avr06SorK4tQxh0PtdA2qAX3oxZanxfrwJWNzurVq7VgwQItWbJEO3bs0PDhwzVlyhQdOnQo0qlZqqurNXz4cC1fvjzk/XfffbceeOABrVixQtu2bVOXLl00ZcoU1dTUtHGmoW3evFm5ubnaunWrXnnlFdXX12vy5Mmqrq4Ozrnpppv0wgsvaM2aNdq8ebNKSkp02WWXRTDrjoNaaDvUgrtRC23Dk3VgXGj06NEmNzc3OG5oaDBpaWkmLy8vglmdmiSzdu3a4DgQCJjU1FRzzz33BGPl5eXG7/ebp556KgIZntqhQ4eMJLN582ZjTGO+MTExZs2aNcE5u3fvNpJMQUFBpNLsMKiFyKEW3IVaiAwv1IHrzujU1dWpsLBQOTk5wVhUVJRycnJUUFAQwcyab//+/SotLXW8lqSkJI0ZM8a1r6WiokKSlJycLEkqLCxUfX294zUMHjxYGRkZrn0NXkEtRBa14B7UQuR4oQ5c1+gcOXJEDQ0NSklJccRTUlJUWloaoaxa5ut828trCQQCmj9/vsaPH69hw4ZJanwNsbGx6tatm2OuW1+Dl1ALkUMtuAu1EBleqQPXfXs5Iic3N1fvvfee3njjjUinAkQUtQB4pw5cd0anZ8+eio6OtlZwl5WVKTU1NUJZtczX+baH1zJv3jy9+OKLys/PV9++fYPx1NRU1dXVqby83DHfja/Ba6iFyKAW3IdaaHteqgPXNTqxsbHKysrSxo0bg7FAIKCNGzcqOzs7gpk1X2ZmplJTUx2vpbKyUtu2bXPNazHGaN68eVq7dq02bdqkzMxMx/1ZWVmKiYlxvIaioiJ98sknrnkNXkUttC1qwb2ohbbjyTqI8GLokJ5++mnj9/vNqlWrzAcffGBmz55tunXrZkpLSyOdmqWqqsrs3LnT7Ny500gy9957r9m5c6f5+OOPjTHG3HnnnaZbt25m/fr15m9/+5uZOnWqyczMNMeOHYtw5o3mzp1rkpKSzGuvvWYOHjwYvB09ejQ457rrrjMZGRlm06ZNZvv27SY7O9tkZ2dHMOuOg1poO9SCu1ELbcOLdeDKRscYYx588EGTkZFhYmNjzejRo83WrVsjnVJI+fn5RpJ1mzVrljGm8VLC22+/3aSkpBi/32/OP/98U1RUFNmk/0mo3CWZlStXBuccO3bMXH/99aZ79+6mc+fO5tJLLzUHDx6MXNIdDLXQNqgF96MWWp8X68BnjDGte84IAAAgMly3RgcAACBcaHQAAIBn0egAAADPotEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBodAADgWTQ6AADAs2h0AACAZ9HoAAAAz6LRAQAAnkWjAwAAPItGBwAAeBaNDgAA8CwaHQAA4Fk0OgAAwLNodJpo1apV8vl8OnDgQLMeN2nSJA0bNiysufTv319XX311WI8JNAV1ADSiFtoPGp0O7KuvvtKSJUt0wQUXKDk5WT6fT6tWrYp0WkCbev/99/WjH/1IZ5xxhjp37qyePXtqwoQJeuGFFyKdGtCmvPo7gUanAzty5IiWLl2q3bt3a/jw4ZFOB4iIjz/+WFVVVZo1a5buv/9+3X777ZKkSy65RI8++miEswPajld/J3SKdAKInD59+ujgwYNKTU3V9u3bNWrUqEinBLS5iy66SBdddJEjNm/ePGVlZenee+/V7NmzI5QZ0La8+juBMzottH79ev3gBz9QWlqa/H6/BgwYoF/96ldqaGgIOb+wsFDjxo1TfHy8MjMztWLFCmtObW2tlixZooEDB8rv9ys9PV0LFy5UbW1tq7wGv9+v1NTUVjk2OgYv1EEo0dHRSk9PV3l5eZs9J9o3L9SCV38ncEanhVatWqWuXbtqwYIF6tq1qzZt2qTFixersrJS99xzj2Pul19+qYsuukiXX365rrjiCj3zzDOaO3euYmNj9bOf/UySFAgEdMkll+iNN97Q7NmzNWTIEL377ru67777tHfvXq1bt+6kuQQCAX3xxRdNyjspKUkxMTEtft3AP/NSHVRXV+vYsWOqqKjQ888/r7/85S+aOXNm834g6LC8VAueY9AkK1euNJLM/v37jTHGHD161JozZ84c07lzZ1NTUxOMTZw40Ugyv/3tb4Ox2tpaM2LECNO7d29TV1dnjDHmiSeeMFFRUeb11193HHPFihVGknnzzTeDsX79+plZs2YFx/v37zeSmnTLz88P+frefvttI8msXLmymT8ZdCReroM5c+YE74+KijIzZswwX3zxRUt+TOgAvFwLxnjrdwJndFooPj4++P+rqqpUW1urc845R4888oj27NnjWMjVqVMnzZkzJziOjY3VnDlzNHfuXBUWFmrs2LFas2aNhgwZosGDB+vIkSPBueedd54kKT8/X+PGjQuZS2pqql555ZUm5e2lBWaIPC/Vwfz58zVjxgyVlJTomWeeUUNDg+rq6pp0PMBLteA1NDot9P777+u2227Tpk2bVFlZ6bivoqLCMU5LS1OXLl0csUGDBkmSDhw4oLFjx2rfvn3avXu3evXqFfL5Dh06dNJc4uLilJOT05KXAZwWL9XB4MGDNXjwYEnSVVddpcmTJ+viiy/Wtm3b5PP5WnxcdAxeqgWvodFpgfLyck2cOFGJiYlaunSpBgwYoLi4OO3YsUO33nqrAoFAs48ZCAR01lln6d577w15f3p6+kkf29DQoMOHDzfpeZKTkxUbG9vs/IATeb0OZsyYoTlz5mjv3r369re/3aTjomPyei20dzQ6LfDaa6/p888/13PPPacJEyYE4/v37w85v6SkRNXV1Y4Ofu/evZIad7SUpAEDBuidd97R+eef3+y/HouLi5WZmdmkufn5+Zo0aVKzjg+E4vU6OHbsmCT7r3HgRF6vhfaORqcFoqOjJUnGmGCsrq5ODz/8cMj5x48f1yOPPKIFCxYE5z7yyCPq1auXsrKyJEmXX365XnrpJf3+97+39u04duyYAoGAdarza3wei0jwSh0cOnRIvXv3dtxfX1+vxx9/XPHx8frOd77TpGOi4/JKLXgVjU4LjBs3Tt27d9esWbN04403yufz6YknnnC8yf9ZWlqa7rrrLh04cECDBg3S6tWrtWvXLj366KPBy/p++tOf6plnntF1112n/Px8jR8/Xg0NDdqzZ4+eeeYZvfzyyxo5cmTI45/O57EPPfSQysvLVVJSIkl64YUX9Omnn0qSbrjhBiUlJbXouPA+r9TBnDlzVFlZqQkTJuhb3/qWSktL9V//9V/as2ePfvvb36pr167NPiY6Fq/UguTR3wmRvOSrPTnxUsI333zTjB071sTHx5u0tDSzcOFC8/LLL1uX602cONEMHTrUbN++3WRnZ5u4uDjTr18/89BDD1nPUVdXZ+666y4zdOhQ4/f7Tffu3U1WVpb55S9/aSoqKoLzTryU8HT069fvpJcdfv1aga95sQ6eeuopk5OTY1JSUkynTp1M9+7dTU5Ojlm/fv1pHxve5cVa+PpYXvud4DPmJC0nAABAO8dXQAAAAM+i0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPCsVmt0li9frv79+ysuLk5jxozRW2+91VpPBbgatQBQB4icVrm8fPXq1brqqqu0YsUKjRkzRsuWLdOaNWtUVFRk7UB6okAgoJKSEiUkJPBFeggyxqiqqkppaWmKimo/JyKpBYRbe6yF06kDiVpAaE2uhdbYnGf06NEmNzc3OG5oaDBpaWkmLy/vlI8tLi4+6WZF3LgVFxe3xlu21VAL3Frr1p5q4XTqwBhqgds3305VC2H/Coi6ujoVFhZq0aJFwVhUVJRycnJUUFBwyscnJCRIks7WReqkmHCnh3bquOr1hl4Kvj/aA2oBraG91cLp1oFELSC0ptZC2BudI0eOqKGhQSkpKY54SkqK9uzZY82vra1VbW1tcFxVVfW/icWok483NP6Xafyf9nTamlpAq2hntdDcOpCoBTRRE2sh4h/w5uXlKSkpKXhLT0+PdEpARFALQCNqAeEU9kanZ8+eio6OVllZmSNeVlam1NRUa/6iRYtUUVERvBUXF4c7JSAiqAWg+XUgUQsIr7A3OrGxscrKytLGjRuDsUAgoI0bNyo7O9ua7/f7lZiY6LgBXkAtAM2vA4laQHiFfY2OJC1YsECzZs3SyJEjNXr0aC1btkzV1dW65pprWuPpANeiFgDqAJHVKo3OzJkzdfjwYS1evFilpaUaMWKENmzYYC1GA7yOWgCoA0RWq2wYeDoqKyuVlJSkSZrK6noEHTf1ek3rVVFR0WFOY1MLCIVaoBbQqKm1EPGrrgAAAFoLjQ4AAPAsGh0AAOBZNDoAAMCzaHQAAIBn0egAAADPotEBAACeRaMDAAA8i0YHAAB4Fo0OAADwrFb5riuc2t6VWVZs/5THrNi9X5xhxV69fKQVa/hgb3gSAwDAQzijAwAAPItGBwAAeBaNDgAA8CwaHQAA4FksRm4j0UO/7RivP3e5NafexFix3O5FVuzZ7062YgkfnEZyQBvyZQ11jAOx9n+GPpvUxYq9f8PDVqzeNIQvsRDOf2+GY9xl6kFrTqCmplVzQMfh8/ut2NELh1ux7/7fd6zYvlG1rZKTF3BGBwAAeBaNDgAA8CwaHQAA4Fk0OgAAwLNYjNxWPit1DG/c+2NryitD/9xW2QBhZ7LtRZP7ro61Yved95RjHOM7bs3Jia+yYvXG/rssoEBzUmy2V4Y94xiPeOJn1pzMuSVWrOHI562WE7wruldPK5a/fIUVe73G/tV9T+bFVuz4/o/Dk1g7xxkdAADgWTQ6AADAs2h0AACAZ7FGp400lFc4xh9/eqY9aagdAtoLc8cXVmzP4OcikEnr2TXuD1ZsypjrrZj/v1mjg9ZzTpy9ru3XGclWLIo1OpI4owMAADyMRgcAAHgWjQ4AAPAsGh0AAOBZLEZuI9EpvR3jc4bsjVAmQOv47LV0Ozj41I8rqLG/sflnL/0fe6IvxIPNqY8/9vt2ra3s/z+nfiDgUtE+zlE0Bz8tAADgWTQ6AADAs2h0AACAZ9HoAAAAz2IxcltJ6OIYXpT8dosPdSjLXpXZ7W+DHOOGD1jsjLaVced2K3bpM1ec8nG+unordub+bWHJSZLKe/awYq9uTbBiob4x/UTnvTvTiiXmv2/FWvc71dHRNRj7HVbf2f51bi/z75g4owMAADyLRgcAAHhWsxudLVu26OKLL1ZaWpp8Pp/WrVvnuN8Yo8WLF6tPnz6Kj49XTk6O9u3bF658AVegDoBG1ALcrtmNTnV1tYYPH67ly5eHvP/uu+/WAw88oBUrVmjbtm3q0qWLpkyZopqamtNOFnAL6gBoRC3A7Zq9GPnCCy/UhRdeGPI+Y4yWLVum2267TVOnTpUkPf7440pJSdG6dev04x//+PSybccaPtzvGN/2gr2ocfoVof9DcaL3//UBK/a9ip87xuksRm5V1IHN1NdZsYaiDyOQiVPZZYOs2Fmx60PMPPXSzZKSZCvW9ehHLUnLM6gFdziUFWPF0v8SgURcKKxrdPbv36/S0lLl5OQEY0lJSRozZowKCgrC+VSAa1EHQCNqAW4Q1svLS0tLJUkpKSmOeEpKSvC+E9XW1qq2tjY4rqysDGdKQJtrSR1I1AK8h1qAG0T8qqu8vDwlJSUFb+npIb4YEOgAqAWgEbWAcApro5OamipJKisrc8TLysqC951o0aJFqqioCN6Ki4vDmRLQ5lpSBxK1AO+hFuAGYf3oKjMzU6mpqdq4caNGjBghqfGU47Zt2zR37tyQj/H7/fL7O97+jQNu2WoHT72JLNqBltSB1HFrIdwOz812jAdfuceakxLdsp/zkIX7rVhDi47UMVALzWPq7V3C99bbV6cNiomzYscy7YsB0KjZjc5XX32lDz/8x5UU+/fv165du5ScnKyMjAzNnz9fd9xxh84880xlZmbq9ttvV1pamqZNmxbOvIGIog6ARtQC3K7Zjc727dt17rnnBscLFiyQJM2aNUurVq3SwoULVV1drdmzZ6u8vFxnn322NmzYoLg4uwMF2ivqAGhELcDtmt3oTJo0ScaYk97v8/m0dOlSLV269LQSA9yMOgAaUQtwO7693EVifNFWrP7k//0AOpRD88ZZsVlzX7JiVyb+xjFOiIpt8XP+6vD3HWNTyzoItJ6GskNW7Ma/25vLbhgcasNLnEzELy8HAABoLTQ6AADAs2h0AACAZ9HoAAAAz2IxsovUG3vrsYACEcgEaL7ood+2Ynuv6W7FJp79XouO/2L6g1YsdH2cevHxh/XHrdjM391sxTLWOnf0DVT9/ZTHBuAunNEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBYjA2g2M36EFbt65VorNrXLkTA+a/j+LrvxQ3u32W/d9VcrxjeTo73omnw00im4Fmd0AACAZ9HoAAAAz6LRAQAAnkWjAwAAPIvFyADCIlrGikWF8W+pGF+0Fau3n7JJNgyxF06f85NcK5b0X1tb9gRAG/vz939vxW7Q+Ahk4j6c0QEAAJ5FowMAADyLRgcAAHgWjQ4AAPAsFiO7yOkstkwcdyjM2QAn53tzlxV7bNoFVuz/u7qHFct4uc4xjj52PGx5SdK+a2Mc4z0X/C6sxwfaUvEb6XZwcNvn0Z5xRgcAAHgWjQ4AAPAsGh0AAOBZrNFxkXpjf1dyQIEmPXbz8Kcc40vGXmtP2vq3FuUFNEXDB3ut2BkL2z6PIft6OQP20iGg3eha3LSFmgk+e170dwY5xqFqtCPgjA4AAPAsGh0AAOBZNDoAAMCzaHQAAIBnsRjZRQZv+jcr9sF5j7boWHtnx1qxQXwRMzqAsssGRjoFIGyimrifZrTPZ8UC8TEhZnY8nNEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBYju4h/b7wdPK/t80DH5vP7HePyH33PmtN9/ftWLFBV1Wo5nczBm8dZsfU33n1CxG/NAdqL7qsKrNiKhf2s2HVJH1uxfTc5L0oZeGX48mpPOKMDAAA8i0YHAAB4VrManby8PI0aNUoJCQnq3bu3pk2bpqKiIsecmpoa5ebmqkePHurataumT5+usrKysCYNRBq1ADSiFuB2zWp0Nm/erNzcXG3dulWvvPKK6uvrNXnyZFVXVwfn3HTTTXrhhRe0Zs0abd68WSUlJbrsssvCnjgQSdQC0IhagNv5jDFN+w74EA4fPqzevXtr8+bNmjBhgioqKtSrVy89+eSTmjFjhiRpz549GjJkiAoKCjR27NhTHrOyslJJSUmapKnq5GNXxyv2lFixnyQcPOXjYnzRVuzCC6+wYoF3drcssTZ23NTrNa1XRUWFEhMTI52Opb3WQs3Fo61Y0i2fOMZrBr5gzbn0PPu91FD0Ydjy6tQn1Yp9NuMMK/bUzb+xYmfEnPpnVdZQa8V+Mv9mK9b5uW2nPFZboxb4vbD3DyOt2KvnL7Ni874zxTEO/FPz6QVNrYXTWqNTUVEhSUpOTpYkFRYWqr6+Xjk5OcE5gwcPVkZGhgoK7JXjklRbW6vKykrHDWhvqAWgEbUAt2lxoxMIBDR//nyNHz9ew4YNkySVlpYqNjZW3bp1c8xNSUlRaWlpyOPk5eUpKSkpeEtPT29pSkBEUAtAI2oBbtTiRic3N1fvvfeenn766dNKYNGiRaqoqAjeiouLT+t4QFujFoBG1ALcqEUbBs6bN08vvviitmzZor59+wbjqampqqurU3l5uaN7LysrU2qq/Zm7JPn9fvn9bOh1Mqs+sTdEu2LomlM+rr7FK6/QHO29Fqb8erMVu7nHe6d83J5fhPg8/Ksx4UhJkvTjcfZHGut6/7cVC+jU6zVmHZhixT5c+W0r1uO50B+joGnaey20dw0K8e3lx2oikIn7NOuMjjFG8+bN09q1a7Vp0yZlZmY67s/KylJMTIw2btwYjBUVFemTTz5RdnZ2eDIGXIBaABpRC3C7Zp3Ryc3N1ZNPPqn169crISEh+PlqUlKS4uPjlZSUpGuvvVYLFixQcnKyEhMTdcMNNyg7O7tJK+uB9oJaABpRC3C7ZjU6v/vd7yRJkyZNcsRXrlypq6++WpJ03333KSoqStOnT1dtba2mTJmihx9+OCzJAm5BLQCNqAW4XbManaZsuRMXF6fly5dr+fLlLU4KcDtqAWhELcDt+PZyl6tdFWKx3j1tnwfwz3bnPBKBZ7WXFBbU2AtW/8+2qxzjgf9nnzWnRzULj+EtAzrFW7HPr3FuCNrjsY75vudLPQEAgGfR6AAAAM+i0QEAAJ5FowMAADyLxcgu133XF1Zs+Zf2rq653YvaIh14zKYbx1uxx693LmB8Z/wfWjWHP1Xa32N0sL6bFfvDDjvXgb9vsGJnvLnLMQ60ODPAnVZOtGvyy8AxK9bzb185xh11w3zO6AAAAM+i0QEAAJ5FowMAADyLRgcAAHgWi5FdruGDvVbs5WGJdkyjmnC03WHICF4S/doOK5b5VmfHOOvGn1tz/jhnmRUbFuuzYue9O9OKVbzm3O273+rPrDnH939sxc5UoRUDOqJ/3z3Dis3ot9OKRVXXOsb20v2OgTM6AADAs2h0AACAZ9HoAAAAz6LRAQAAnsViZAAOgaNHHeNv3flXa84v7hxtxULpqo9OGTvejNwASMk/tC9S2aQuIWba8zoizugAAADPotEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBodAADgWTQ6AADAs2h0AACAZ9HoAAAAz6LRAQAAnkWjAwAAPItGBwAAeJbrvtTTGCNJOq56yUQ4GbjGcdVL+sf7oyOgFhAKtRDhZOAaTa0F1zU6VVVVkqQ39FKEM4EbVVVVKSkpKdJptAlqAd+EWgAanaoWfMZlfxYEAgGVlJQoISFBVVVVSk9PV3FxsRITEyOdWrNVVla22/zdlrsxRlVVVUpLS1NUVMf4xNUrteC291JzuS3/jlwLxhhlZGS45t+iudz2Xmout+Xf1Fpw3RmdqKgo9e3bV5Lk8/kkSYmJia74obZUe87fTbl3lL9ev+a1WmjPuUvuyr+j1kJlZaUkd/1btAT5h09TaqFj/DkAAAA6JBodAADgWa5udPx+v5YsWSK/3x/pVFqkPeffnnP3ovb879Gec5faf/5e0t7/Lcg/Mly3GBkAACBcXH1GBwAA4HTQ6AAAAM+i0QEAAJ5FowMAADzLtY3O8uXL1b9/f8XFxWnMmDF66623Ip1SSFu2bNHFF1+stLQ0+Xw+rVu3znG/MUaLFy9Wnz59FB8fr5ycHO3bty8yyYaQl5enUaNGKSEhQb1799a0adNUVFTkmFNTU6Pc3Fz16NFDXbt21fTp01VWVhahjDseaqFtUAvuRy20Pi/WgSsbndWrV2vBggVasmSJduzYoeHDh2vKlCk6dOhQpFOzVFdXa/jw4Vq+fHnI+++++2498MADWrFihbZt26YuXbpoypQpqqmpaeNMQ9u8ebNyc3O1detWvfLKK6qvr9fkyZNVXV0dnHPTTTfphRde0Jo1a7R582aVlJTosssui2DWHQe10HaoBXejFtqGJ+vAuNDo0aNNbm5ucNzQ0GDS0tJMXl5eBLM6NUlm7dq1wXEgEDCpqanmnnvuCcbKy8uN3+83Tz31VAQyPLVDhw4ZSWbz5s3GmMZ8Y2JizJo1a4Jzdu/ebSSZgoKCSKXZYVALkUMtuAu1EBleqAPXndGpq6tTYWGhcnJygrGoqCjl5OSooKAggpk13/79+1VaWup4LUlJSRozZoxrX0tFRYUkKTk5WZJUWFio+vp6x2sYPHiwMjIyXPsavIJaiCxqwT2ohcjxQh24rtE5cuSIGhoalJKS4oinpKSotLQ0Qlm1zNf5tpfXEggENH/+fI0fP17Dhg2T1PgaYmNj1a1bN8dct74GL6EWIodacBdqITK8Ugeu+/ZyRE5ubq7ee+89vfHGG5FOBYgoagHwTh247oxOz549FR0dba3gLisrU2pqaoSyapmv820Pr2XevHl68cUXlZ+fr759+wbjqampqqurU3l5uWO+G1+D11ALkUEtuA+10Pa8VAeua3RiY2OVlZWljRs3BmOBQEAbN25UdnZ2BDNrvszMTKWmpjpeS2VlpbZt2+aa12KM0bx587R27Vpt2rRJmZmZjvuzsrIUExPjeA1FRUX65JNPXPMavIpaaFvUgntRC23Hk3UQ4cXQIT399NPG7/ebVatWmQ8++MDMnj3bdOvWzZSWlkY6NUtVVZXZuXOn2blzp5Fk7r33XrNz507z8ccfG2OMufPOO023bt3M+vXrzd/+9jczdepUk5mZaY4dOxbhzBvNnTvXJCUlmddee80cPHgweDt69GhwznXXXWcyMjLMpk2bzPbt2012drbJzs6OYNYdB7XQdqgFd6MW2oYX68CVjY4xxjz44IMmIyPDxMbGmtGjR5utW7dGOqWQ8vPzjSTrNmvWLGNM46WEt99+u0lJSTF+v9+cf/75pqioKLJJ/5NQuUsyK1euDM45duyYuf7660337t1N586dzaWXXmoOHjwYuaQ7GGqhbVAL7kcttD4v1oHPGGNa95wRAABAZLhujQ4AAEC40OgAAADPotEBAACeRaMDAAA8i0YHAAB4Fo0OAADwLBodAADgWTQ6AADAs2h0AACAZ9HoAAAAz6LRAQAAnkWjAwAAPOv/B13kJ1MKpA2eAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Visualize Dataset\n", "import matplotlib.pyplot as plt\n", "\n", "grid = 3\n", "fig, axes = plt.subplots(grid, grid, figsize=(6, 6))\n", "for i in range(grid):\n", " for j in range(grid):\n", " axes[i][j].imshow(x_train[i * grid + j])\n", " axes[i][j].set_title(f\"label={y_train[i * grid + j]}\")\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Build Model\n", "We will use the Keras 3.0 sequential API to build a simple CNN." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Model: \"sequential\"\n",
       "
\n" ], "text/plain": [ "\u001b[1mModel: \"sequential\"\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓\n",
       "┃ Layer (type)                    ┃ Output Shape              ┃    Param # ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩\n",
       "│ conv2d (Conv2D)                 │ (None, 26, 26, 32)        │        320 │\n",
       "├─────────────────────────────────┼───────────────────────────┼────────────┤\n",
       "│ conv2d_1 (Conv2D)               │ (None, 24, 24, 32)        │      9,248 │\n",
       "├─────────────────────────────────┼───────────────────────────┼────────────┤\n",
       "│ conv2d_2 (Conv2D)               │ (None, 22, 22, 32)        │      9,248 │\n",
       "├─────────────────────────────────┼───────────────────────────┼────────────┤\n",
       "│ global_average_pooling2d        │ (None, 32)                │          0 │\n",
       "│ (GlobalAveragePooling2D)        │                           │            │\n",
       "├─────────────────────────────────┼───────────────────────────┼────────────┤\n",
       "│ dense (Dense)                   │ (None, 10)                │        330 │\n",
       "└─────────────────────────────────┴───────────────────────────┴────────────┘\n",
       "
\n" ], "text/plain": [ "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩\n", "│ conv2d (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m26\u001b[0m, \u001b[38;5;34m26\u001b[0m, \u001b[38;5;34m32\u001b[0m) │ \u001b[38;5;34m320\u001b[0m │\n", "├─────────────────────────────────┼───────────────────────────┼────────────┤\n", "│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m24\u001b[0m, \u001b[38;5;34m24\u001b[0m, \u001b[38;5;34m32\u001b[0m) │ \u001b[38;5;34m9,248\u001b[0m │\n", "├─────────────────────────────────┼───────────────────────────┼────────────┤\n", "│ conv2d_2 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m22\u001b[0m, \u001b[38;5;34m22\u001b[0m, \u001b[38;5;34m32\u001b[0m) │ \u001b[38;5;34m9,248\u001b[0m │\n", "├─────────────────────────────────┼───────────────────────────┼────────────┤\n", "│ global_average_pooling2d │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", "│ (\u001b[38;5;33mGlobalAveragePooling2D\u001b[0m) │ │ │\n", "├─────────────────────────────────┼───────────────────────────┼────────────┤\n", "│ dense (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m330\u001b[0m │\n", "└─────────────────────────────────┴───────────────────────────┴────────────┘\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 Total params: 19,146 (74.79 KB)\n",
       "
\n" ], "text/plain": [ "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m19,146\u001b[0m (74.79 KB)\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 Trainable params: 19,146 (74.79 KB)\n",
       "
\n" ], "text/plain": [ "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m19,146\u001b[0m (74.79 KB)\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 Non-trainable params: 0 (0.00 B)\n",
       "
\n" ], "text/plain": [ "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "NUM_CLASSES = 10\n", "INPUT_SHAPE = (28, 28, 1)\n", "\n", "\n", "def initialize_model():\n", " return keras.Sequential(\n", " [\n", " keras.Input(shape=INPUT_SHAPE),\n", " keras.layers.Conv2D(32, kernel_size=(3, 3), activation=\"relu\"),\n", " keras.layers.Conv2D(32, kernel_size=(3, 3), activation=\"relu\"),\n", " keras.layers.Conv2D(32, kernel_size=(3, 3), activation=\"relu\"),\n", " keras.layers.GlobalAveragePooling2D(),\n", " keras.layers.Dense(NUM_CLASSES, activation=\"softmax\"),\n", " ]\n", " )\n", "\n", "\n", "model = initialize_model()\n", "model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train Model (Default Callback)\n", "We will fit the model on the dataset, using MLflow's `mlflow.keras.MlflowCallback` to log metrics during training." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "BATCH_SIZE = 64 # adjust this based on the memory of your machine\n", "EPOCHS = 3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Log Per Epoch\n", "An epoch defined as one pass through the entire training dataset." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m30s\u001b[0m 34ms/step - accuracy: 0.5922 - loss: 1.2862 - val_accuracy: 0.9427 - val_loss: 0.2075\n", "Epoch 2/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m28s\u001b[0m 33ms/step - accuracy: 0.9330 - loss: 0.2286 - val_accuracy: 0.9348 - val_loss: 0.2020\n", "Epoch 3/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m28s\u001b[0m 33ms/step - accuracy: 0.9499 - loss: 0.1671 - val_accuracy: 0.9558 - val_loss: 0.1491\n" ] } ], "source": [ "model = initialize_model()\n", "\n", "model.compile(\n", " loss=keras.losses.SparseCategoricalCrossentropy(),\n", " optimizer=keras.optimizers.Adam(),\n", " metrics=[\"accuracy\"],\n", ")\n", "\n", "run = mlflow.start_run()\n", "model.fit(\n", " x_train,\n", " y_train,\n", " batch_size=BATCH_SIZE,\n", " epochs=EPOCHS,\n", " validation_split=0.1,\n", " callbacks=[mlflow.keras.MlflowCallback(run)],\n", ")\n", "mlflow.end_run()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Log Results\n", "The callback for the run would log **parameters**, **metrics** and **artifacts** to MLflow dashboard.\n", "\n", "![run page](https://i.imgur.com/YLGFDJEl.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Log Per Batch\n", "Within each epoch, the training dataset is broken down to batches based on the defined `BATCH_SIZE`. If we set the callback to not log based on epochs with `log_every_epoch=False`, and to log every 5 batches with `log_every_n_steps=5`, we can adjust the logging to be based on the batches." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m30s\u001b[0m 34ms/step - accuracy: 0.6151 - loss: 1.2100 - val_accuracy: 0.9373 - val_loss: 0.2144\n", "Epoch 2/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m29s\u001b[0m 34ms/step - accuracy: 0.9274 - loss: 0.2459 - val_accuracy: 0.9608 - val_loss: 0.1338\n", "Epoch 3/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m28s\u001b[0m 34ms/step - accuracy: 0.9477 - loss: 0.1738 - val_accuracy: 0.9577 - val_loss: 0.1454\n" ] } ], "source": [ "model = initialize_model()\n", "\n", "model.compile(\n", " loss=keras.losses.SparseCategoricalCrossentropy(),\n", " optimizer=keras.optimizers.Adam(),\n", " metrics=[\"accuracy\"],\n", ")\n", "\n", "with mlflow.start_run() as run:\n", " model.fit(\n", " x_train,\n", " y_train,\n", " batch_size=BATCH_SIZE,\n", " epochs=EPOCHS,\n", " validation_split=0.1,\n", " callbacks=[mlflow.keras.MlflowCallback(run, log_every_epoch=False, log_every_n_steps=5)],\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Log Results\n", "\n", "If we **log per epoch**, we will only have three datapoints, since there are only 3 epochs:\n", "\n", "![log per epoch](https://i.imgur.com/rFDj8SHl.png)\n", "\n", "By **logging per batch**, we can get more datapoints, but they can be noisier:\n", "\n", "![log per batch](https://i.imgur.com/ZCYXLqll.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "class MlflowCallbackLogPerBatch(mlflow.keras.MlflowCallback):\n", " def on_batch_end(self, batch, logs=None):\n", " if self.log_every_n_steps is None or logs is None:\n", " return\n", " if (batch + 1) % self.log_every_n_steps == 0:\n", " self.metrics_logger.record_metrics(logs, self._log_step)\n", " self._log_step += self.log_every_n_steps" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m29s\u001b[0m 34ms/step - accuracy: 0.5645 - loss: 1.4105 - val_accuracy: 0.9187 - val_loss: 0.2826\n", "Epoch 2/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m29s\u001b[0m 34ms/step - accuracy: 0.9257 - loss: 0.2615 - val_accuracy: 0.9602 - val_loss: 0.1368\n", "Epoch 3/3\n", "\u001b[1m844/844\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m29s\u001b[0m 34ms/step - accuracy: 0.9456 - loss: 0.1800 - val_accuracy: 0.9678 - val_loss: 0.1037\n" ] } ], "source": [ "model = initialize_model()\n", "\n", "model.compile(\n", " loss=keras.losses.SparseCategoricalCrossentropy(),\n", " optimizer=keras.optimizers.Adam(),\n", " metrics=[\"accuracy\"],\n", ")\n", "\n", "with mlflow.start_run() as run:\n", " model.fit(\n", " x_train,\n", " y_train,\n", " batch_size=BATCH_SIZE,\n", " epochs=EPOCHS,\n", " validation_split=0.1,\n", " callbacks=[MlflowCallbackLogPerBatch(run, log_every_epoch=False, log_every_n_steps=5)],\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluation\n", "Similar to training, you can use the callback to log the evaluation result." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 4ms/step - accuracy: 0.9541 - loss: 0.1487\n" ] } ], "source": [ "with mlflow.start_run() as run:\n", " model.evaluate(x_test, y_test, callbacks=[mlflow.keras_core.MlflowCallback(run)])" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" } }, "nbformat": 4, "nbformat_minor": 2 }