{ "cells": [ { "cell_type": "markdown", "id": "efec26f5", "metadata": {}, "source": [ "# ๐Ÿง Machine Learning Homework: Classify the Penguins\n", "\n", "Welcome. In the lectures you watched machine learning. Here you will do it yourself, on a small\n", "and friendly set of data where you can see everything that happens.\n", "\n", "We follow the same path that any data scientist follows:\n", "\n", "> the data, then preparing the data, then running the algorithms, then judging the result\n", "\n", "You will use, in the same order as the course, every method that you were taught:\n", "Decision Tree, then SVM, then Gradient Descent, then Neural Network, then Evaluation.\n", "\n", "### How to use this worksheet\n", "* Some cells have blank spaces marked `___` and `# TODO`. Fill them in, then run the cell.\n", "* Every task ends with a short footer:\n", " * โœ… **Check yourself:** what your result should look like, so you know it is correct.\n", " * ๐Ÿ”ฌ **Why it matters:** what this idea is used for in normal life. No special knowledge is needed.\n", " * ๐ŸŽ‰ **Fun fact:** how this idea was used in one of the biggest moments in particle physics.\n", " * ๐Ÿ’ก **Go further:** an extra challenge, if you want to push yourself.\n", "* Keep the ๐Ÿ† Scoreboard near the end updated. Watch your models compete.\n", "\n", "You do not need any physics or any earlier machine learning set up to do this (suggested to be done on goole collab). If you finish the โœ… tasks,\n", "you have done the homework. The ๐Ÿ’ก challenges are extra.\n" ] }, { "cell_type": "markdown", "id": "7bcb5262", "metadata": {}, "source": [ "## Setup\n", "Run this one time. Everything here is already available in Google Colab." ] }, { "cell_type": "code", "execution_count": null, "id": "19268b23", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "from sklearn.model_selection import train_test_split, cross_val_score\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.tree import DecisionTreeClassifier\n", "from sklearn.svm import SVC\n", "from sklearn.metrics import accuracy_score, classification_report, confusion_matrix\n", "\n", "# A place to record each model score, so we can compare them at the end.\n", "scoreboard = {}\n", "print(\"Setup complete\")" ] }, { "cell_type": "markdown", "id": "19b2f30c", "metadata": {}, "source": [ "## Part 0. Meet the data ๐Ÿ” (about 5 min)\n", "\n", "We use the Palmer Penguins data. It has body measurements of penguins from 3 species\n", "(Adelie, Chinstrap, Gentoo). Our job is to predict the species from its body measurements.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "935c62af", "metadata": {}, "outputs": [], "source": [ "penguins = sns.load_dataset(\"penguins\")\n", "\n", "# TODO: show the first 5 rows of the table\n", "penguins.___()" ] }, { "cell_type": "code", "execution_count": null, "id": "9b8c4ba3", "metadata": {}, "outputs": [], "source": [ "# How many penguins of each species do we have? This is what we want to predict.\n", "print(penguins[\"species\"].value_counts())\n", "\n", "# TODO: draw a scatter plot of the two bill measurements, coloured by species\n", "sns.scatterplot(data=penguins, x=\"bill_length_mm\", y=\"___\", hue=\"___\")\n", "plt.title(\"Can you separate the species by eye?\")\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "bd9fc891", "metadata": {}, "source": [ "โœ… **Check yourself:** you should see 344 penguins (about 152 Adelie, 124 Gentoo,\n", "68 Chinstrap) and three colour clouds that overlap. That overlap is what makes this a real\n", "problem, because no straight line can separate them perfectly.\n", "\n", "๐Ÿ”ฌ **Why it matters:** looking at your data first is the habit that separates good analysts from\n", "bad ones. It is true whether you are finding bank fraud, reading medical scans, or studying\n", "particles. A model can only be as good as your understanding of what goes into it.\n", "\n", "๐ŸŽ‰ **Fun fact:** the biggest particle collider in the world, the Large Hadron Collider at CERN,\n", "makes about one billion particle collisions every second. Scientists cannot save them all, so\n", "the very first job is always to look at the data and decide what matters.\n", "\n", "๐Ÿ’ก **Go further:** which two species overlap the most? Which single measurement do you think best\n", "separates the species? Write your guess, and you will be able to test it soon.\n" ] }, { "cell_type": "markdown", "id": "67d9b069", "metadata": {}, "source": [ "## Part 1. Prepare the data ๐Ÿงน (about 8 min)\n", "\n", "Real data is messy. Before any algorithm we must handle missing values, pick our features (X)\n", "and our target (y), and split the data into a training set and a test set. This preparation step\n", "is where most of the real work in machine learning happens.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3ea9cc62", "metadata": {}, "outputs": [], "source": [ "# Some measurements are missing. Count them first.\n", "print(\"Missing values in each column:\")\n", "print(penguins.isna().sum())\n", "\n", "# In the Decision Tree lecture we saw that scikit learn cannot use rows with missing values,\n", "# so the simplest fix is to remove those rows.\n", "# TODO: remove every row that has a missing value, then reset the index\n", "data = penguins.___().reset_index(drop=True)\n", "print(\"\\nRows before:\", len(penguins), \"and after cleaning:\", len(data))" ] }, { "cell_type": "code", "execution_count": null, "id": "aca5c974", "metadata": {}, "outputs": [], "source": [ "# Features (X) are the two bill measurements. The target (y) is the species.\n", "features = [\"bill_length_mm\", \"bill_depth_mm\"]\n", "X = data[features].values\n", "\n", "# Models work with numbers, so we turn the species names into the numbers 0, 1 and 2.\n", "species_names = sorted(data[\"species\"].unique())\n", "name_to_number = {name: i for i, name in enumerate(species_names)}\n", "y = data[\"species\"].map(name_to_number).values\n", "print(\"Species to number:\", name_to_number)\n", "\n", "# TODO: split into 80 percent for training and 20 percent for testing.\n", "# Use random_state=42 so that everyone gets the same split.\n", "X_train, X_test, y_train, y_test = train_test_split(\n", " X, y, test_size=___, random_state=42)\n", "print(\"Train:\", X_train.shape, \"and Test:\", X_test.shape)" ] }, { "cell_type": "markdown", "id": "ae9f06b6", "metadata": {}, "source": [ "โœ… **Check yourself:** cleaning takes the data from 344 rows down to 333 rows. You should\n", "have about 266 penguins for training and 67 for testing. The species map to\n", "Adelie 0, Chinstrap 1, Gentoo 2.\n", "\n", "๐Ÿ”ฌ **Why it matters:** choosing and cleaning features is the real science. Professionals often\n", "spend more time preparing data than running the clever algorithms, because a model can only be as\n", "good as the numbers you give it. The features you picked are the whole view of the world for the\n", "model.\n", "\n", "๐ŸŽ‰ **Fun fact:** to study one tiny particle, physicists take many raw measurements and combine\n", "them into a few smart numbers, called features. One of these numbers, the mass, is how they have\n", "found brand new particles hiding inside the data.\n", "\n", "๐Ÿ’ก **Go further:** the data also has two more number columns (`flipper_length_mm` and\n", "`body_mass_g`) and two text columns (`island` and `sex`). How would you turn the text column\n", "`sex` into a number that a model can use?\n" ] }, { "cell_type": "markdown", "id": "60e9fc20", "metadata": {}, "source": [ "## Part 2. Decision Tree ๐ŸŒณ (about 7 min)\n", "\n", "A decision tree asks a series of yes or no questions (for example, is the bill longer than\n", "45 mm?) to sort each penguin. It is the simplest real classifier, and it is the easiest one to\n", "understand.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "8638e5f4", "metadata": {}, "outputs": [], "source": [ "# TODO: create a tree with max_depth=3, train it, then score it on the TEST set\n", "tree = DecisionTreeClassifier(max_depth=___, random_state=42)\n", "tree.fit(X_train, y_train)\n", "\n", "tree_acc = accuracy_score(y_test, tree.predict(X_test))\n", "print(f\"Decision Tree test accuracy: {tree_acc:.3f}\")\n", "scoreboard[\"Decision Tree\"] = tree_acc" ] }, { "cell_type": "markdown", "id": "3a7a6cd3", "metadata": {}, "source": [ "โœ… **Check yourself:** you should get about 0.91, which means about 91 out of every 100\n", "penguins are correct. That is good for just a few yes or no questions.\n", "\n", "๐Ÿ”ฌ **Why it matters:** decision trees are used everywhere. Banks use bigger versions of them to\n", "decide who gets a loan, and hospitals use them to flag patients who may be at risk. The same\n", "simple idea grows all the way up to the largest experiments in science.\n", "\n", "๐ŸŽ‰ **Fun fact:** a stronger version of the decision tree, called a boosted decision tree, helped\n", "scientists at CERN find the Higgs boson in 2012. The Higgs boson helps explain why matter has\n", "mass, and the discovery won a Nobel Prize.\n", "\n", "๐Ÿ’ก **Go further:** try every `max_depth` from 1 to 10 and draw a plot of test accuracy against\n", "depth. It goes up, then flattens, and can even drop. That drop is called overfitting, where the\n", "tree memorises the training penguins instead of learning the real pattern. Then try\n", "`cross_val_score(tree, X, y, cv=5).mean()` for a more honest score.\n" ] }, { "cell_type": "markdown", "id": "51f5e51a", "metadata": {}, "source": [ "## Part 3. Support Vector Machine โœ‚๏ธ (about 8 min)\n", "\n", "A support vector machine draws the widest possible boundary between the classes. It cares about\n", "feature scaling. If one measurement uses much bigger numbers than another, it takes over unless\n", "we rescale. Let us prove this with a small experiment.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "c8ca878b", "metadata": {}, "outputs": [], "source": [ "# First without scaling:\n", "svm_raw = SVC(kernel=\"rbf\").fit(X_train, y_train)\n", "acc_raw = accuracy_score(y_test, svm_raw.predict(X_test))\n", "\n", "# Now with scaling. TODO: fit the scaler on the TRAIN set, then transform BOTH sets.\n", "scaler = StandardScaler().fit(___)\n", "X_train_s = scaler.transform(X_train)\n", "X_test_s = scaler.transform(___)\n", "\n", "svm = SVC(kernel=\"rbf\").fit(X_train_s, y_train)\n", "svm_acc = accuracy_score(y_test, svm.predict(X_test_s))\n", "\n", "print(f\"SVM without scaling: {acc_raw:.3f}\")\n", "print(f\"SVM with scaling: {svm_acc:.3f}\")\n", "scoreboard[\"SVM\"] = svm_acc" ] }, { "cell_type": "markdown", "id": "fcaae0b6", "metadata": {}, "source": [ "โœ… **Check yourself:** scaling should push the accuracy up, from about 0.94 to about 0.97.\n", "The data and the model are the same. The only change is that we measured the features on a fair\n", "scale.\n", "\n", "๐Ÿ”ฌ **Why it matters:** scaling your features sounds boring, but forgetting it is one of the most\n", "common reasons that real models quietly fail. It matters any time your measurements cover very\n", "different ranges, for example a shopping website that mixes number of clicks with money spent.\n", "The fix you just used is part of almost every serious project.\n", "\n", "๐ŸŽ‰ **Fun fact:** support vector machines have been used at particle colliders to find very rare\n", "events, such as the top quark, which is the heaviest particle that we know. The machine had to\n", "pick these rare events out of a huge pile of ordinary ones.\n", "\n", "๐Ÿ’ก **Go further:** build `X` again using all four number measurements and run this test once more.\n", "The gap between scaled and not scaled becomes very large (about 0.69 up to 1.00). You can also\n", "compare the linear kernel with the rbf kernel.\n" ] }, { "cell_type": "markdown", "id": "4a1c356e", "metadata": {}, "source": [ "## Part 4. Gradient Descent ๐Ÿ“‰ (about 7 min)\n", "\n", "So far, a single call to `.fit()` trained each model for you. But how does training actually work\n", "inside? Almost always by gradient descent. You start somewhere, and then you take small steps\n", "downhill to make the error smaller. Let us write that core idea in three lines, exactly as in the\n", "lecture.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3f12859e", "metadata": {}, "outputs": [], "source": [ "# A simple bowl shaped function that we want to make as small as possible, and its slope.\n", "def f(theta): return theta ** 2\n", "def grad(theta): return 2 * theta # the slope of theta squared\n", "\n", "theta = 5.0 # start far from the lowest point, which sits at 0\n", "lr = 0.1 # learning rate is how big each downhill step is\n", "path = [theta]\n", "\n", "for step in range(25):\n", " # TODO: take one step downhill. Move theta against its slope.\n", " theta = theta - lr * ___\n", " path.append(theta)\n", "\n", "print(f\"Start value is 5.0. After 25 steps theta is {theta:.4f}. The target is 0.\")\n", "plt.plot(path, marker=\"o\"); plt.xlabel(\"step\"); plt.ylabel(\"theta\")\n", "plt.title(\"Gradient descent rolling downhill\"); plt.show()" ] }, { "cell_type": "markdown", "id": "e65d82e8", "metadata": {}, "source": [ "โœ… **Check yourself:** theta should slide from 5.0 down toward about 0.0 (about 0.02 after\n", "25 steps), and the curve should look like a ball rolling into a valley. Try `lr = 0.9` (it jumps\n", "around) or `lr = 0.01` (too slow to arrive).\n", "\n", "๐Ÿ”ฌ **Why it matters:** gradient descent, moving numbers downhill to make the error smaller, is the\n", "engine behind almost all modern artificial intelligence. The same three lines that you just wrote\n", "are, deep down, how every large neural network learns, including the chat assistants that many\n", "people use today.\n", "\n", "๐ŸŽ‰ **Fun fact:** gradient descent is how the learning models across physics are trained. This\n", "includes the models that help physicists/researchers decide, in real time, which collisions to keep\n", "and which ones to throw away.\n", "\n", "๐Ÿ’ก **Go further:** change the start value from 5.0 to a negative number. Does it still reach the\n", "lowest point? What happens with a very big learning rate like `lr = 1.1`?\n" ] }, { "cell_type": "markdown", "id": "a428760d", "metadata": {}, "source": [ "## Part 5. Neural Network ๐Ÿง  (about 10 min)\n", "\n", "Now the star of the course. A neural network stacks simple parts together to learn curved\n", "boundaries. We build a tiny one in PyTorch (already installed in Colab). It is trained by the\n", "same gradient descent that you just met.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ea925ddb", "metadata": {}, "outputs": [], "source": [ "import torch\n", "from torch import nn, optim\n", "\n", "torch.manual_seed(0)\n", "\n", "# Turn our scaled number arrays into torch tensors\n", "Xtr = torch.tensor(X_train_s, dtype=torch.float32)\n", "ytr = torch.tensor(y_train, dtype=torch.long)\n", "Xte = torch.tensor(X_test_s, dtype=torch.float32)\n", "\n", "# TODO: build a network with 2 inputs, then 16 hidden units with ReLU, then 3 outputs\n", "model = nn.Sequential(\n", " nn.Linear(2, ___),\n", " nn.ReLU(),\n", " nn.Linear(___, 3),\n", ")\n", "print(model)" ] }, { "cell_type": "code", "execution_count": null, "id": "04e6da14", "metadata": {}, "outputs": [], "source": [ "loss_fn = nn.CrossEntropyLoss()\n", "optimizer = optim.Adam(model.parameters(), lr=0.05)\n", "\n", "# TODO: train for 150 rounds (epochs). In each round: clear the old slopes, predict,\n", "# measure the loss, go backward, then take one step.\n", "losses = []\n", "for epoch in range(___):\n", " optimizer.zero_grad()\n", " predictions = model(Xtr)\n", " loss = loss_fn(predictions, ytr)\n", " loss.backward()\n", " optimizer.step()\n", " losses.append(loss.item())\n", "\n", "plt.plot(losses); plt.xlabel(\"epoch\"); plt.ylabel(\"loss\")\n", "plt.title(\"The loss should go down\"); plt.show()\n", "\n", "nn_pred = model(Xte).argmax(dim=1).numpy()\n", "nn_acc = accuracy_score(y_test, nn_pred)\n", "print(f\"Neural Network test accuracy: {nn_acc:.3f}\")\n", "scoreboard[\"Neural Network\"] = nn_acc" ] }, { "cell_type": "markdown", "id": "7a40a388", "metadata": {}, "source": [ "โœ… **Check yourself:** the loss curve should go down and settle at a low value. The test\n", "accuracy should be around 0.95. If the loss stays flat or shows `nan`, check the learning rate.\n", "\n", "๐Ÿ”ฌ **Why it matters:** you just built a neural network. To feel the size of this idea: networks\n", "made from these same parts, only much bigger, power voice assistants, translation apps, and self\n", "driving cars. And a small one, almost the same as yours, placed on a special chip, helps the\n", "biggest physics experiment in the world look through 40 million events every second and decide,\n", "in a few billionths of a second, what to keep.\n", "\n", "๐ŸŽ‰ **Fun fact:** neural networks help the IceCube experiment, a giant detector buried in the ice\n", "of Antarctica, find ghostly particles called neutrinos. In 2017 this work led to a famous result.\n", "A single neutrino was traced back to a galaxy far across the universe.\n", "\n", "๐Ÿ’ก **Go further:** add a second hidden layer, or change 16 to 4 and then to 64. More units make\n", "the network more flexible, but watch for overfitting. Does the test accuracy really improve, or\n", "only the training loss?\n" ] }, { "cell_type": "markdown", "id": "516f9a53", "metadata": {}, "source": [ "## Part 6. Judge the results โš–๏ธ (about 7 min)\n", "\n", "Accuracy is a single number. A confusion matrix shows which species get mixed up. That is the\n", "real story behind the number.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "7a736307", "metadata": {}, "outputs": [], "source": [ "# TODO: build the confusion matrix that compares the true labels with the network predictions\n", "cm = confusion_matrix(y_test, ___)\n", "\n", "plt.imshow(cm, cmap=\"Blues\")\n", "plt.xticks([0, 1, 2], species_names); plt.yticks([0, 1, 2], species_names)\n", "plt.xlabel(\"Predicted\"); plt.ylabel(\"Actual\"); plt.title(\"Confusion Matrix\")\n", "for i in range(3):\n", " for j in range(3):\n", " plt.text(j, i, cm[i, j], ha=\"center\", va=\"center\")\n", "plt.colorbar(); plt.show()" ] }, { "cell_type": "code", "execution_count": null, "id": "a300e616", "metadata": {}, "outputs": [], "source": [ "# Scoreboard: every model on the same test set\n", "print(\"SCOREBOARD\")\n", "for name, acc in sorted(scoreboard.items(), key=lambda kv: -kv[1]):\n", " print(f\" {name:22s} {acc:.3f}\")" ] }, { "cell_type": "markdown", "id": "fb685b9e", "metadata": {}, "source": [ "โœ… **Check yourself:** the diagonal line (the correct predictions) should be much darker\n", "than the rest. Most of the mix up is between the two species that overlapped in your scatter plot\n", "in Part 0. Your scoreboard should show the models within a few percent of each other.\n", "\n", "๐Ÿ”ฌ **Why it matters:** a single accuracy number can hide the truth. A medical test that is 99\n", "percent accurate can still be dangerous if it misses the 1 percent of people who are truly sick.\n", "So professionals always look at which mistakes a model makes, not only how many. Scientists are so\n", "careful that, before they announce a discovery, they ask for proof so strong that it would happen\n", "by pure luck less than 1 time in 3.5 million.\n", "\n", "๐ŸŽ‰ **Fun fact:** before scientists at CERN announced the Higgs boson, two separate teams each had\n", "to reach this very strict level of proof, called five sigma. Only then did they trust that the new\n", "particle was real and not a lucky accident.\n", "\n", "๐Ÿ’ก **Go further:** which model won on your run? Would it still win if you changed the\n", "`random_state` in Part 1? Print `classification_report(y_test, nn_pred)` to see how well each\n", "single species was predicted.\n" ] }, { "cell_type": "markdown", "id": "3841798e", "metadata": {}, "source": [ "## ๐ŸŽ“ Very well done\n", "\n", "You just built a complete machine learning pipeline from start to finish. You looked at raw data,\n", "cleaned it, chose features, trained four different kinds of models, and judged them in an honest\n", "way. This is the exact process that professionals use every day.\n", "\n", "More importantly, you now understand what each method actually does, not only how to call it. That\n", "understanding is the whole point of this exercise, and it is what makes the next and harder\n", "problems feel possible.\n", "\n", "The best way to make it stick is to go back and change things. Break the models on purpose. Every\n", "time you ask what happens if I do this, you turn a fact that you memorised into an understanding\n", "that you own.\n", "\n", "Did you finish early and improve something? Clean notebooks can be added back to the course, with\n", "your name credited. That is a nice thing to have to your name.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 5 }