{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercise 4.3\n", "## Classification\n", "In the following tasks, we will repeatedly use some basic functions (e.g., the softmax function or the cross-entropy) of the [Keras](https://keras.io/) Library. To familiarize with them, we will implement the most important of them ourselves in this task.\n", "\n", "Suppose we want to classify some data (4 samples) into 3 distinct classes: 0, 1, and 2.\n", "We have set up a network with a pre-activation output z in the last layer.\n", "Applying softmax will give the final model output.\n", "\n", "input X ---> some network --> z
\n", "--> y_model = softmax(z)\n", "\n", "We quantify the agreement between truth (y) and model using categorical cross-entropy.\n", "\n", "$$J = - \\sum_i (y_i * \\log(y_\\mathrm{model}(x_i))$$\n", "\n", "In the following you are to implement softmax and categorical cross-entropy\n", "and evaluate them values given the values for z." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Data: 4 samples with the following class labels (input features X irrelevant here)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "y_cl = np.array([0, 0, 2, 1])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### output of the last network layer before applying softmax" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "z = np.array([\n", " [4, 5, 1],\n", " [-1, -2, -3],\n", " [0.1, 0.2, 0.3],\n", " [-1, 17, 1]\n", " ]).astype(np.float32)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Task 1)\n", "Write a function that turns any class labels y_cl into one-hot encodings y.\n", "\n", "0 --> (1, 0, 0)
\n", "1 --> (0, 1, 0)
\n", "2 --> (0, 0, 1)
\n", "\n", "Make sure that np.shape(y) = (4, 3) for np.shape(y_cl) = (4).\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Task 2)\n", "Write a function that returns the softmax of the input z along the last axis" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Task 3)\n", "Compute the categorical cross-entropy between data and model" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Task 4)\n", "Determine which calsses are predicted by the model (maximum prediction)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Task 5)\n", "Estimate how many samples are classified correctly (accuracy)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 4 }