{ "cells": [ { "cell_type": "markdown", "id": "0", "metadata": {}, "source": [ "---\n", "title: \"Chapter 4: Classes and patterns\"\n", "---" ] }, { "cell_type": "markdown", "id": "1", "metadata": {}, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sambaiga/ds-mlops-path/blob/main/tutorials/01-python-basics/04-classes.ipynb) [![Download Notebook](https://img.shields.io/badge/Download-Notebook-blue.svg?logo=jupyter&logoColor=white)](https://raw.githubusercontent.com/sambaiga/ds-mlops-path/main/tutorials/01-python-basics/04-classes.ipynb)" ] }, { "cell_type": "markdown", "id": "2", "metadata": {}, "source": [ "A student record is not three separate variables; it is one thing. A class lets you treat it that way: one `__repr__()`, one place where the rules about valid scores live, one factory method that builds it from a raw CSV row.\n", "\n", "Chapter 3 built functions: the logic. A function only holds logic; the data it operates on lives somewhere else, as loose variables the function accepts and returns. A class bundles data and the functions that operate on it under one name.\n", "\n", "This chapter adds the tools that Chapter 3 could not carry: classes, dataclasses, NamedTuples, TypedDicts, exceptions, and file I/O. Chapter 5 (`05-math-statistics.ipynb`) uses these patterns throughout as it introduces the standard library math and statistics modules." ] }, { "cell_type": "markdown", "id": "3", "metadata": {}, "source": [ "::: {.callout-note collapse=\"true\" icon=false}\n", "## Learning objectives\n", "\n", "By the end of Chapter 4 you will be able to:\n", "\n", "| # | Skill | Covered in |\n", "|---|---|---|\n", "| 1 | Define classes with `__init__`, `__repr__`, `@property`, and `@classmethod` | Sec. 1 |\n", "| 2 | Use inheritance and abstract base classes | Sec. 1 |\n", "| 3 | Create dataclasses and frozen dataclasses | Sec. 2 |\n", "| 4 | Use NamedTuple for immutable typed records | Sec. 3 |\n", "| 5 | Use TypedDict and type aliases for structured dicts | Sec. 4 |\n", "| 6 | Handle exceptions with `try`/`except`/`finally` and custom exception classes | Sec. 5 |\n", "| 7 | Read and write files with `pathlib`, `csv`, and `json` | Sec. 6 |\n", "| 8 | Recognise and avoid common Python gotchas | Sec. 7 |\n", ":::" ] }, { "cell_type": "markdown", "id": "4", "metadata": {}, "source": [ "## 1. Classes\n", "\n", "A **class** is a blueprint for creating objects. Each object (instance) bundles its own data (attributes) with the functions (methods) that operate on it. You create the blueprint once with `class`, then stamp out as many instances as you need.\n", "\n", "Three things make a class useful rather than just a namespace:\n", "- `__init__`: sets the initial state of each instance\n", "- `__repr__`: returns a readable string when you print or inspect the object\n", "- **methods**: functions that always have access to the instance's data via `self`" ] }, { "cell_type": "markdown", "id": "5", "metadata": {}, "source": [ "![A Python class bundles data (stored in `__init__`) and the functions that operate on it (`@property`, `@classmethod`, `__repr__`). Each instance owns its own data; all instances share the methods.](figs/class-anatomy.svg){fig-alt=\"Left: class StudentRecord box showing __init__, @property, @classmethod, __repr__ methods with colored role indicators. Right: alice instance showing attribute values after construction. Arrow from class to instance labeled instantiate.\"}" ] }, { "cell_type": "markdown", "id": "6", "metadata": {}, "source": [ "### `__init__` and `__repr__`\n", "\n", "Every class needs two methods before it's useful in interactive work. `__init__` runs the moment you create an instance: it receives the raw data and stores it on `self`. `__repr__` is what Python prints when you inspect the object in the REPL or call `print()` on it.\n", "\n", "Without `__repr__`, Python shows something like `<__main__.StudentRecord object at 0x7f3a2c1d4e50>`. That address tells you nothing. With a good `__repr__` you can see the name, program, and average in one glance, which makes debugging feel like reading, not guessing.\n", "\n", "The rule: `__repr__` should produce a string that is both human-readable and, where possible, copy-pasteable back into Python to re-create the object." ] }, { "cell_type": "code", "execution_count": null, "id": "7", "metadata": {}, "outputs": [], "source": [ "class StudentRecord:\n", " \"\"\"A single student's academic record.\n", "\n", " Args:\n", " name: Student's full name.\n", " scores: List of numeric assessment scores (0-100).\n", " program: Degree program name.\n", " \"\"\"\n", "\n", " def __init__(\n", " self,\n", " name: str,\n", " scores: list[float] | None = None,\n", " program: str = \"Undeclared\",\n", " ) -> None:\n", " self.name = name\n", " self.scores: list[float] = scores if scores is not None else []\n", " self.program = program\n", "\n", " def __repr__(self) -> str:\n", " avg = sum(self.scores) / len(self.scores) if self.scores else 0.0\n", " return f\"StudentRecord(name={self.name!r}, program={self.program!r}, avg={avg:.1f})\"" ] }, { "cell_type": "code", "execution_count": null, "id": "8", "metadata": {}, "outputs": [], "source": [ "alice = StudentRecord(\n", " name=\"Alice Kamau\",\n", " scores=[88.0, 92.0, 85.0, 90.0],\n", " program=\"Computer Science\",\n", ")\n", "bob = StudentRecord(\n", " name=\"Bob Mwangi\",\n", " scores=[62.0, 70.0, 58.0, 74.0],\n", " program=\"Mathematics\",\n", ")\n", "\n", "print(alice) # __repr__ in action: no print formatting needed\n", "print(bob)" ] }, { "cell_type": "markdown", "id": "9", "metadata": {}, "source": [ "### `@property`: computed attributes\n", "\n", "You could store the average as an attribute set in `__init__`: `self.average = sum(scores) / len(scores)`. The problem is stale state. If someone appends a score later, `student.scores.append(95.0)`, the stored average is now wrong, and nothing tells you.\n", "\n", "A `@property` computes the value fresh every time you access it. From the caller's side it looks exactly like reading a plain attribute: `student.average`, no parentheses. Under the hood Python calls the method. You get the right answer regardless of when scores were added.\n", "\n", "Use `@property` for any value that is derived from other attributes. Store it explicitly only if the computation is genuinely expensive and you have a caching strategy." ] }, { "cell_type": "code", "execution_count": null, "id": "10", "metadata": {}, "outputs": [], "source": [ "class StudentRecord:\n", " \"\"\"A single student's academic record with computed grade properties.\n", "\n", " Args:\n", " name: Student's full name.\n", " scores: List of numeric assessment scores (0-100).\n", " program: Degree program name.\n", " \"\"\"\n", "\n", " def __init__(\n", " self,\n", " name: str,\n", " scores: list[float] | None = None,\n", " program: str = \"Undeclared\",\n", " ) -> None:\n", " self.name = name\n", " self.scores: list[float] = scores if scores is not None else []\n", " self.program = program\n", "\n", " def __repr__(self) -> str:\n", " return f\"StudentRecord(name={self.name!r}, program={self.program!r}, avg={self.average:.1f})\"\n", "\n", " @property\n", " def average(self) -> float:\n", " \"\"\"Mean of all scores; 0.0 if no scores recorded.\"\"\"\n", " return sum(self.scores) / len(self.scores) if self.scores else 0.0\n", "\n", " @property\n", " def letter_grade(self) -> str:\n", " \"\"\"Letter grade derived from current average.\"\"\"\n", " avg = self.average\n", " if avg >= 90:\n", " return \"A\"\n", " if avg >= 80:\n", " return \"B\"\n", " if avg >= 70:\n", " return \"C\"\n", " if avg >= 60:\n", " return \"D\"\n", " return \"F\"" ] }, { "cell_type": "code", "execution_count": null, "id": "11", "metadata": {}, "outputs": [], "source": [ "alice = StudentRecord(\"Alice Kamau\", [88.0, 92.0, 85.0], \"Computer Science\")\n", "print(f\"Before: avg={alice.average:.1f} grade={alice.letter_grade}\")\n", "\n", "# Add a new score: the property recomputes on the fly\n", "alice.scores.append(95.0)\n", "print(f\"After : avg={alice.average:.1f} grade={alice.letter_grade}\")\n", "\n", "print(alice)" ] }, { "cell_type": "markdown", "id": "12", "metadata": {}, "source": [ "### `@classmethod` factory methods\n", "\n", "A `@classmethod` is an alternative constructor: it builds an instance from a different kind of input without requiring the caller to know what `__init__` expects.\n", "\n", "The pattern is useful whenever your data arrives in a format different from what `__init__` expects. A CSV row is a `dict[str, str]` where every value is a string. Putting the parsing logic inside a `@classmethod` keeps `__init__` clean and makes the intent explicit at the call site: `StudentRecord.from_dict(row)` reads like documentation." ] }, { "cell_type": "code", "execution_count": null, "id": "13", "metadata": {}, "outputs": [], "source": [ "class StudentRecord:\n", " \"\"\"A student record with factory method support.\n", "\n", " Args:\n", " name: Student's full name.\n", " scores: List of numeric assessment scores (0-100).\n", " program: Degree program name.\n", " \"\"\"\n", "\n", " def __init__(\n", " self,\n", " name: str,\n", " scores: list[float] | None = None,\n", " program: str = \"Undeclared\",\n", " ) -> None:\n", " self.name = name\n", " self.scores: list[float] = scores if scores is not None else []\n", " self.program = program\n", "\n", " def __repr__(self) -> str:\n", " return f\"StudentRecord(name={self.name!r}, program={self.program!r}, avg={self.average:.1f})\"\n", "\n", " @property\n", " def average(self) -> float:\n", " \"\"\"Mean of all scores; 0.0 if no scores recorded.\"\"\"\n", " return sum(self.scores) / len(self.scores) if self.scores else 0.0\n", "\n", " @property\n", " def letter_grade(self) -> str:\n", " \"\"\"Letter grade derived from current average.\"\"\"\n", " avg = self.average\n", " if avg >= 90:\n", " return \"A\"\n", " if avg >= 80:\n", " return \"B\"\n", " if avg >= 70:\n", " return \"C\"\n", " if avg >= 60:\n", " return \"D\"\n", " return \"F\"\n", "\n", " @classmethod\n", " def from_dict(cls, record: dict[str, object]) -> \"StudentRecord\":\n", " \"\"\"Construct a StudentRecord from a raw CSV row dict.\n", "\n", " Args:\n", " record: Dict with keys 'name', 'scores' (comma-separated string\n", " or list of floats), and 'program'.\n", "\n", " Returns:\n", " A new StudentRecord instance.\n", " \"\"\"\n", " raw_scores = record.get(\"scores\", \"\")\n", " if isinstance(raw_scores, str):\n", " scores = [float(s.strip()) for s in raw_scores.split(\",\") if s.strip()]\n", " else:\n", " scores = list(raw_scores) # type: ignore[arg-type]\n", " return cls(\n", " name=str(record.get(\"name\", \"\")),\n", " scores=scores,\n", " program=str(record.get(\"program\", \"Undeclared\")),\n", " )" ] }, { "cell_type": "code", "execution_count": null, "id": "14", "metadata": {}, "outputs": [], "source": [ "# Simulated rows as they would arrive from csv.DictReader\n", "raw_rows: list[dict[str, object]] = [\n", " {\"name\": \"Carol Osei\", \"scores\": \"91.0, 95.0, 89.0\", \"program\": \"Data Science\"},\n", " {\"name\": \"Dan Kimani\", \"scores\": \"74.0, 68.0, 80.0\", \"program\": \"Mathematics\"},\n", "]\n", "\n", "records = [StudentRecord.from_dict(row) for row in raw_rows]\n", "for r in records:\n", " print(r)" ] }, { "cell_type": "markdown", "id": "15", "metadata": {}, "source": [ "### Inheritance and abstract base classes\n", "\n", "Inheritance is often introduced as a code-reuse tool: put shared logic in a parent class. That is true, but it misses the more important use case in ML engineering: enforcing a contract.\n", "\n", "When you define an Abstract Base Class (ABC), you're saying: any class that claims to be a `BaseMetric` must implement `compute(actual_labels, predicted_labels)`. Python will refuse to instantiate a subclass that forgets to implement it. This isn't a convention or a code review comment, it's a runtime guarantee.\n", "\n", "This is how scikit-learn's `BaseEstimator` enforces that every estimator has `.fit()` and `.predict()`, and how PyTorch's `nn.Module` enforces `forward()`. You will implement the same pattern here for evaluation metrics." ] }, { "cell_type": "markdown", "id": "16", "metadata": {}, "source": [ "The inheritance chain makes the contract visible. `BaseMetric` declares what every metric must implement; concrete classes fill in the details. Python refuses to instantiate `BaseMetric` directly, you can only create `AccuracyMetric` or `F1Metric`." ] }, { "cell_type": "markdown", "id": "17", "metadata": {}, "source": [ "![Class hierarchy: BaseMetric(ABC) at the top with an abstract compute method, inherited by AccuracyMetric and F1Metric below, each implementing their own compute.](figs/abc-inheritance.svg){fig-alt=\"Three boxes connected by arrows. Top: BaseMetric(ABC) in blue with abstractmethod compute. Bottom-left: AccuracyMetric in green. Bottom-right: F1Metric in green.\"}" ] }, { "cell_type": "code", "execution_count": null, "id": "18", "metadata": {}, "outputs": [], "source": [ "from abc import ABC, abstractmethod\n", "\n", "\n", "class BaseMetric(ABC):\n", " \"\"\"Abstract base class for all evaluation metrics.\n", "\n", " Subclasses must implement `compute`; Python will refuse to instantiate\n", " any subclass that does not.\n", " \"\"\"\n", "\n", " @abstractmethod\n", " def compute(self, actual_labels: list[int], predicted_labels: list[int]) -> float:\n", " \"\"\"Compute the metric value.\n", "\n", " Args:\n", " actual_labels: Ground-truth labels.\n", " predicted_labels: Predicted labels.\n", "\n", " Returns:\n", " Scalar metric value.\n", " \"\"\"\n", "\n", "\n", "class AccuracyMetric(BaseMetric):\n", " \"\"\"Fraction of predictions that match the ground truth.\"\"\"\n", "\n", " def compute(self, actual_labels: list[int], predicted_labels: list[int]) -> float:\n", " if not actual_labels:\n", " return 0.0\n", " correct = sum(t == p for t, p in zip(actual_labels, predicted_labels, strict=False))\n", " return correct / len(actual_labels)\n", "\n", "\n", "class F1Metric(BaseMetric):\n", " \"\"\"Binary F1 score: harmonic mean of precision and recall.\"\"\"\n", "\n", " def compute(self, actual_labels: list[int], predicted_labels: list[int]) -> float:\n", " tp = sum(t == 1 and p == 1 for t, p in zip(actual_labels, predicted_labels, strict=False))\n", " fp = sum(t == 0 and p == 1 for t, p in zip(actual_labels, predicted_labels, strict=False))\n", " fn = sum(t == 1 and p == 0 for t, p in zip(actual_labels, predicted_labels, strict=False))\n", " if tp == 0:\n", " return 0.0\n", " precision = tp / (tp + fp)\n", " recall = tp / (tp + fn)\n", " return 2 * precision * recall / (precision + recall)" ] }, { "cell_type": "code", "execution_count": null, "id": "19", "metadata": {}, "outputs": [], "source": [ "actual_labels = [1, 0, 1, 1, 0, 1, 0, 0]\n", "predicted_labels = [1, 0, 1, 0, 0, 1, 1, 0]\n", "\n", "metrics: list[BaseMetric] = [AccuracyMetric(), F1Metric()]\n", "for metric in metrics:\n", " score = metric.compute(actual_labels, predicted_labels)\n", " print(f\"{metric.__class__.__name__:<18}: {score:.4f}\")\n", "\n", "# Demonstrate the ABC contract: Python refuses to instantiate BaseMetric directly\n", "try:\n", " _ = BaseMetric() # type: ignore[abstract]\n", "except TypeError as exc:\n", " print(f\"\\nABC contract enforced: {exc}\")" ] }, { "cell_type": "markdown", "id": "20", "metadata": {}, "source": [ "
Key Concept: class vs @dataclass: when to use which

Both are classes. The difference is how much Python generates for you.

NeedReach for
Store fields, get free __repr__ and __eq__@dataclass
Computed properties (@property)class
Alternative constructors (@classmethod)class
Enforce an interface (ABC)class
Immutable, hashable record@dataclass(frozen=True)

They can coexist: a @dataclass can have @property methods and @classmethod factories. Start with @dataclass when you only need structured storage; graduate to a plain class when you need the full toolkit shown in this section.
" ] }, { "cell_type": "markdown", "id": "21", "metadata": {}, "source": [ "
\n", " Activity 1: CourseRecord class

\n", "Define a CourseRecord class (not a dataclass) that stores code: str, semester: str, instructor: str, and students: list[str]. Add a @property enrolment that returns the student count, a __repr__ that prints CourseRecord(CS101 / 2024-S1), and a @classmethod from_dict(cls, row) that builds a record from a raw dict. Instantiate two records and print their enrolment counts.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "22", "metadata": {}, "outputs": [], "source": [ "# TODO: define CourseRecord\n", "...\n", "\n", "# Quick test\n", "row = {\n", " \"code\": \"CS101\",\n", " \"semester\": \"2024-S1\",\n", " \"instructor\": \"Dr. Mwangi\",\n", " \"students\": [\"Alice\", \"Bob\", \"Carol\", \"Dan\"],\n", "}\n", "# cr = CourseRecord.from_dict(row)\n", "# print(cr) # CourseRecord(CS101 / 2024-S1)\n", "# print(cr.enrolment) # 4" ] }, { "cell_type": "markdown", "id": "23", "metadata": {}, "source": [ "## 2. Dataclasses\n", "\n", "Writing `__init__` by hand for every class is repetitive when the class is just a container for data. The `@dataclass` decorator generates `__init__`, `__repr__`, and `__eq__` automatically from the field annotations." ] }, { "cell_type": "markdown", "id": "24", "metadata": {}, "source": [ "
\n", " Key Concept: @dataclass vs a plain class

\n", "Use @dataclass when the class is primarily a data container: a fixed set of typed fields with no complex initialisation logic. Use a plain class when you need a non-trivial __init__, inheritance, or methods that go well beyond reading and writing fields.

\n", "Rule of thumb: if the __init__ would be nothing but self.x = x assignments, reach for @dataclass.\n", "
" ] }, { "cell_type": "markdown", "id": "25", "metadata": {}, "source": [ "A `CourseConfig` holds the settings for one course. Without `@dataclass` you would write twelve lines of boilerplate. With it, you write the fields:" ] }, { "cell_type": "code", "execution_count": null, "id": "26", "metadata": {}, "outputs": [], "source": [ "from dataclasses import dataclass, field\n", "\n", "\n", "@dataclass\n", "class CourseConfig:\n", " \"\"\"Configuration for a single course.\"\"\"\n", "\n", " code: str\n", " semester: str\n", " pass_mark: float = 50.0\n", " max_attempts: int = 2\n", " tags: list[str] = field(default_factory=list)" ] }, { "cell_type": "markdown", "id": "27", "metadata": {}, "source": [ "`@dataclass` generates `__init__`, `__repr__`, and `__eq__`. Fields with `field(default_factory=list)` get a fresh list per instance, which is the fix for the mutable default problem from Chapter 3:" ] }, { "cell_type": "code", "execution_count": null, "id": "28", "metadata": {}, "outputs": [], "source": [ "cfg = CourseConfig(code=\"CS101\", semester=\"2024-S1\", pass_mark=55.0)\n", "cfg.tags.append(\"core\")\n", "cfg.tags.append(\"required\")\n", "\n", "print(cfg)\n", "print(f\"pass mark : {cfg.pass_mark}\")\n", "print(f\"tags : {cfg.tags}\")" ] }, { "cell_type": "markdown", "id": "29", "metadata": {}, "source": [ "### frozen=True: immutable and hashable\n", "\n", "`frozen=True` prevents field mutation after creation. A frozen dataclass is hashable, so it can be used as a dict key or put in a set:" ] }, { "cell_type": "code", "execution_count": null, "id": "30", "metadata": {}, "outputs": [], "source": [ "from dataclasses import dataclass\n", "\n", "\n", "@dataclass(frozen=True)\n", "class GradeScale:\n", " \"\"\"Immutable grade boundary configuration.\"\"\"\n", "\n", " pass_mark: float = 50.0\n", " distinction_mark: float = 75.0\n", " max_score: float = 100.0\n", "\n", " def __post_init__(self) -> None:\n", " if not (0 <= self.pass_mark < self.distinction_mark <= self.max_score):\n", " raise ValueError(f\"Invalid grade boundaries: {self.pass_mark} / {self.distinction_mark} / {self.max_score}\")" ] }, { "cell_type": "markdown", "id": "31", "metadata": {}, "source": [ "`__post_init__` runs automatically after `__init__` and is the right place for cross-field validation:" ] }, { "cell_type": "code", "execution_count": null, "id": "32", "metadata": {}, "outputs": [], "source": [ "standard = GradeScale(pass_mark=50.0, distinction_mark=75.0)\n", "strict = GradeScale(pass_mark=60.0, distinction_mark=80.0)\n", "\n", "# Frozen dataclasses are hashable: use as dict keys\n", "cohort_results: dict[GradeScale, list[str]] = {\n", " standard: [\"Alice\", \"Carol\"],\n", " strict: [\"Dan\"],\n", "}\n", "\n", "print(standard)\n", "print(f\"Standard scale cohort: {cohort_results[standard]}\")" ] }, { "cell_type": "markdown", "id": "33", "metadata": {}, "source": [ "
\n", " Activity 2: ExamResult dataclass

\n", "Define a frozen ExamResult dataclass with fields: student_id: str, course: str, score: float, semester: str. Add a __post_init__ that raises ValueError if score is outside 0-100. Instantiate a valid result and one that should fail, catching the error.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "34", "metadata": {}, "outputs": [], "source": [ "from dataclasses import dataclass\n", "\n", "\n", "@dataclass(frozen=True)\n", "class ExamResult:\n", " \"\"\"Immutable record of one exam attempt.\"\"\"\n", "\n", " student_id: str\n", " course: str\n", " score: float\n", " semester: str\n", "\n", " def __post_init__(self) -> None: ... # TODO: validate score in range 0-100\n", "\n", "\n", "# Tests\n", "r = ExamResult(\"S1042\", \"CS101\", 88.5, \"2024-S1\")\n", "print(r)\n", "try:\n", " ExamResult(\"S1043\", \"CS101\", 105.0, \"2024-S1\") # should raise\n", "except ValueError as e:\n", " print(f\"Caught: {e}\")" ] }, { "cell_type": "markdown", "id": "35", "metadata": {}, "source": [ "## 3. NamedTuple\n", "\n", "A `NamedTuple` is a tuple whose fields have names. It gives you a lightweight, immutable record: you access fields by name instead of by index, which makes the code self-documenting." ] }, { "cell_type": "markdown", "id": "36", "metadata": {}, "source": [ "
\n", " Key Concept: NamedTuple vs dataclass

\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "37", "metadata": {}, "source": [ "Define a NamedTuple using the class syntax. Fields get names and type annotations:" ] }, { "cell_type": "code", "execution_count": null, "id": "38", "metadata": {}, "outputs": [], "source": [ "from typing import NamedTuple\n", "\n", "\n", "class StudentGrade(NamedTuple):\n", " \"\"\"Immutable record of one assessed outcome.\"\"\"\n", "\n", " student_id: str\n", " course: str\n", " score: float\n", " grade: str" ] }, { "cell_type": "markdown", "id": "39", "metadata": {}, "source": [ "NamedTuples support both attribute access and positional unpacking. They also compare equal when all fields match:" ] }, { "cell_type": "code", "execution_count": null, "id": "40", "metadata": {}, "outputs": [], "source": [ "g = StudentGrade(student_id=\"S1042\", course=\"CS101\", score=88.5, grade=\"B\")\n", "\n", "print(g)\n", "print(f\"score : {g.score}\")\n", "\n", "# Unpack like a plain tuple\n", "sid, course, score, grade = g\n", "print(f\"{sid} scored {score} in {course}\")\n", "\n", "# Equal if all fields match\n", "same = StudentGrade(\"S1042\", \"CS101\", 88.5, \"B\")\n", "print(f\"Equal: {g == same}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "41", "metadata": {}, "outputs": [], "source": [ "grades: list[StudentGrade] = [\n", " StudentGrade(\"S1042\", \"CS101\", 88.5, \"B\"),\n", " StudentGrade(\"S1043\", \"CS101\", 91.0, \"A\"),\n", " StudentGrade(\"S1044\", \"CS101\", 74.0, \"C\"),\n", " StudentGrade(\"S1045\", \"CS101\", 55.5, \"D\"),\n", "]\n", "\n", "# Sort by score descending\n", "top = sorted(grades, key=lambda g: g.score, reverse=True)\n", "for g in top:\n", " print(f\" {g.student_id} {g.score:5.1f} {g.grade}\")" ] }, { "cell_type": "markdown", "id": "42", "metadata": {}, "source": [ "## 4. TypedDict and type aliases\n", "\n", "Raw dicts arrive from JSON files, CSV rows, and API responses. A `TypedDict` defines a typed schema for a dict: it tells your type checker which keys to expect and what type each value should be, without changing runtime behaviour at all." ] }, { "cell_type": "markdown", "id": "43", "metadata": {}, "source": [ "
\n", " Key Concept: TypedDict

\n", "A TypedDict is purely a type-checker hint. At runtime, a TypedDict is still an ordinary dict: no __init__, no field validation, no overhead. Use it when you receive raw dicts from external sources and want your editor to catch missing or misspelled keys before the code runs.\n", "
" ] }, { "cell_type": "markdown", "id": "44", "metadata": {}, "source": [ "Define a schema for the raw student rows that arrive from a CSV or API:" ] }, { "cell_type": "code", "execution_count": null, "id": "45", "metadata": {}, "outputs": [], "source": [ "from typing import TypedDict\n", "\n", "\n", "class StudentRow(TypedDict):\n", " \"\"\"Schema for a single row from the student CSV.\"\"\"\n", "\n", " student_id: str\n", " name: str\n", " gpa: float\n", " major: str" ] }, { "cell_type": "markdown", "id": "46", "metadata": {}, "source": [ "The TypedDict annotates the variable. The dict itself is still a plain dict at runtime:" ] }, { "cell_type": "code", "execution_count": null, "id": "47", "metadata": {}, "outputs": [], "source": [ "row: StudentRow = {\n", " \"student_id\": \"S1042\",\n", " \"name\": \"Alice\",\n", " \"gpa\": 3.95,\n", " \"major\": \"CS\",\n", "}\n", "\n", "print(row[\"name\"])\n", "print(type(row)) # still a plain dict at runtime" ] }, { "cell_type": "markdown", "id": "48", "metadata": {}, "source": [ "### Type aliases\n", "\n", "A type alias gives a long type expression a short, meaningful name. Use `type` (Python 3.12+) for new code:" ] }, { "cell_type": "code", "execution_count": null, "id": "49", "metadata": {}, "outputs": [], "source": [ "type ScoreList = list[float]\n", "type CohortData = dict[str, ScoreList]\n", "\n", "cohort: CohortData = {\n", " \"CS101\": [88.0, 91.0, 74.0, 55.5],\n", " \"MATH201\": [72.0, 84.0, 67.0, 90.0],\n", "}\n", "\n", "for course, scores in cohort.items():\n", " mean = sum(scores) / len(scores)\n", " print(f\"{course}: mean={mean:.1f}\")" ] }, { "cell_type": "markdown", "id": "50", "metadata": {}, "source": [ "## 5. Exception handling\n", "\n", "An **exception** is an error that occurs while your program is running. By default, Python stops immediately and prints a traceback. Exception handling lets you catch the error, respond to it gracefully, and keep the program running.\n", "\n", "```python\n", "# Without handling - program crashes:\n", "int(\"abc\") # ValueError: invalid literal for int() with base 10: 'abc'\n", "\n", "# With handling - program continues:\n", "try:\n", " int(\"abc\")\n", "except ValueError:\n", " print(\"That was not a number, skipping\")\n", "```\n", "\n", "In data pipelines and ML training loops, unhandled exceptions can discard hours of computation. Always handle errors at system boundaries (user input, file I/O, APIs).\n", "\n", "
\n", " Key Concept: try / except / else / finally

\n", "\n", "Catch the most specific exception you can. Bare except: or except Exception: hides bugs and silences keyboard interrupts.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "51", "metadata": {}, "outputs": [], "source": [ "def parse_score(raw: str) -> float:\n", " \"\"\"Parse a score string and validate it is in [0, 100].\n", "\n", " Args:\n", " raw: String representation of a numeric score.\n", "\n", " Returns:\n", " Validated float score.\n", "\n", " Raises:\n", " ValueError: If raw is not numeric or out of range.\n", " \"\"\"\n", " try:\n", " value = float(raw)\n", " except ValueError:\n", " raise ValueError(f\"{raw!r} is not a valid number\") from None\n", "\n", " if not 0 <= value <= 100:\n", " raise ValueError(f\"Score {value} is out of range [0, 100]\")\n", "\n", " return value" ] }, { "cell_type": "markdown", "id": "52", "metadata": {}, "source": [ "Test `parse_score()` against a range of inputs: valid numbers, out-of-range values, and non-numeric strings. The `else` clause runs only on the **success** path:" ] }, { "cell_type": "code", "execution_count": null, "id": "53", "metadata": {}, "outputs": [], "source": [ "# Test parse_score against valid and invalid inputs\n", "test_inputs: list[str] = [\"87.5\", \"105\", \"abc\", \"-3\", \"72\"]\n", "\n", "for raw in test_inputs:\n", " try:\n", " score = parse_score(raw)\n", " except ValueError as exc:\n", " print(f\" {raw!r:<8} -> ERROR: {exc}\")\n", " else:\n", " print(f\" {raw!r:<8} -> OK: {score}\")" ] }, { "cell_type": "markdown", "id": "54", "metadata": {}, "source": [ "`finally` runs regardless of whether an exception occurred or was handled. Use it for cleanup code (closing files, releasing connections) that must execute either way. This example illustrates the pattern using explicit `open`/`close`; in practice, always use `with open(...) as fh` instead (shown in Sec. 7):" ] }, { "cell_type": "code", "execution_count": null, "id": "55", "metadata": {}, "outputs": [], "source": [ "# else runs ONLY when try succeeds; finally ALWAYS runs (cleanup guarantee)\n", "def load_scores(filepath: str) -> list[float]:\n", " \"\"\"Load numeric scores from a text file, one per line.\"\"\"\n", " fh = None\n", " try:\n", " fh = open(filepath, encoding=\"utf-8\") # noqa: SIM115, PTH123\n", " lines = fh.readlines()\n", " except FileNotFoundError:\n", " print(f\"File not found: {filepath!r}\")\n", " return []\n", " except PermissionError as exc:\n", " print(f\"Permission denied: {exc}\")\n", " return []\n", " else:\n", " print(f\"Loaded {len(lines)} lines successfully\")\n", " return [float(line.strip()) for line in lines if line.strip()]\n", " finally:\n", " if fh is not None:\n", " fh.close()\n", " print(\"File handle closed\")" ] }, { "cell_type": "markdown", "id": "56", "metadata": {}, "source": [ "`finally` guarantees the file handle is closed even when the file doesn't exist: no resource leak is possible:" ] }, { "cell_type": "code", "execution_count": null, "id": "57", "metadata": {}, "outputs": [], "source": [ "# NOTE: prefer `with open(...) as fh` in practice (shown in Sec. 7).\n", "# This example uses explicit open/close to make else/finally visible.\n", "result = load_scores(\"nonexistent.txt\")\n", "print(f\"Result: {result}\")" ] }, { "cell_type": "markdown", "id": "58", "metadata": {}, "source": [ "### Custom exception classes\n", "Subclass a built-in exception to give callers a **specific type** to catch. Store the structured context as instance attributes for programmatic access:" ] }, { "cell_type": "code", "execution_count": null, "id": "59", "metadata": {}, "outputs": [], "source": [ "# Custom exception classes give callers something specific to catch\n", "class DataValidationError(ValueError):\n", " \"\"\"Raised when a data record fails validation.\"\"\"\n", "\n", " def __init__(self, field: str, value: object, reason: str) -> None:\n", " self.field = field\n", " self.value = value\n", " self.reason = reason\n", " super().__init__(f\"Validation failed for {field!r}={value!r}: {reason}\")" ] }, { "cell_type": "markdown", "id": "60", "metadata": {}, "source": [ "Define a validator that raises `DataValidationError` with field-level context, then test it. The `except DataValidationError` clause catches only your custom type, not any accidental `ValueError` from elsewhere in the code:" ] }, { "cell_type": "code", "execution_count": null, "id": "61", "metadata": {}, "outputs": [], "source": [ "def validate_student(record: dict[str, object]) -> None:\n", " \"\"\"Validate a student record dict against required field constraints.\"\"\"\n", " gpa = record.get(\"gpa\")\n", " if not isinstance(gpa, int | float):\n", " raise DataValidationError(\"gpa\", gpa, \"must be numeric\")\n", " if not 0.0 <= float(gpa) <= 4.0:\n", " raise DataValidationError(\"gpa\", gpa, \"must be in [0.0, 4.0]\")\n", "\n", " name = record.get(\"name\", \"\")\n", " if not isinstance(name, str) or not name.strip():\n", " raise DataValidationError(\"name\", name, \"must be a non-empty string\")" ] }, { "cell_type": "markdown", "id": "62", "metadata": {}, "source": [ "Test against valid and invalid records. The custom exception prints exactly which field failed and why:" ] }, { "cell_type": "code", "execution_count": null, "id": "63", "metadata": {}, "outputs": [], "source": [ "test_records: list[dict[str, object]] = [\n", " {\"name\": \"Alice\", \"gpa\": 3.95}, # valid\n", " {\"name\": \"Bob\", \"gpa\": 5.0}, # GPA out of range\n", " {\"name\": \"\", \"gpa\": 3.5}, # empty name\n", "]\n", "\n", "for rec in test_records:\n", " try:\n", " validate_student(rec)\n", " print(f\" {rec['name']!r:<10} -> valid\")\n", " except DataValidationError as exc:\n", " print(f\" {rec.get('name')!r:<10} -> {exc}\")" ] }, { "cell_type": "markdown", "id": "64", "metadata": {}, "source": [ "
\n", " Activity 4: safe batch parser

\n", "Write parse_batch(rows) that returns (valid, errors): a list of successfully parsed floats and a list of error messages.\n", "
rows = ['85.0', '92', 'n/a', '-5', '78.5', '110', '63']\n",
    "\n",
    "valid, errors = parse_batch(rows)\n",
    "# valid  = [85.0, 92.0, 78.5, 63.0]\n",
    "# errors = [\"'n/a' isn't a valid number\",\n",
    "#           \"'-5' out of range [0, 100]\",\n",
    "#           \"'110' out of range [0, 100]\"]
\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "65", "metadata": {}, "outputs": [], "source": [ "def parse_batch(rows: list[str]) -> tuple[list[float], list[str]]:\n", " \"\"\"Parse a batch of score strings, separating valid from invalid.\n", "\n", " Args:\n", " rows: List of raw score strings.\n", "\n", " Returns:\n", " Tuple of (valid_scores, error_messages).\n", " \"\"\"\n", " valid: list[float] = []\n", " errors: list[str] = []\n", " # TODO: iterate rows, use parse_score from above, collect results\n", " return valid, errors\n", "\n", "\n", "rows: list[str] = [\"85.0\", \"92\", \"n/a\", \"-5\", \"78.5\", \"110\", \"63\"]\n", "valid, errors = parse_batch(rows)\n", "print(f\"valid = {valid}\")\n", "print(f\"errors = {errors}\")" ] }, { "cell_type": "markdown", "id": "66", "metadata": {}, "source": [ "## 6. File I/O with pathlib\n", "\n", "**File I/O** (Input/Output) means reading data from files on disk and writing results back. Almost every data science workflow starts by loading a CSV, JSON, or Parquet file and ends by saving results somewhere.\n", "\n", "`pathlib.Path` is the modern Python way to work with file paths. It is cross-platform (works on Windows, macOS, and Linux without changes) and composable:\n", "\n", "```python\n", "from pathlib import Path\n", "\n", "data_dir = Path('tutorials') / 'data' # / joins path parts\n", "csv_file = data_dir / 'students.csv'\n", "print(csv_file) # tutorials/data/students.csv\n", "```\n", "\n", "
\n", " Key Concept: pathlib.Path, the modern way to handle paths

\n", "Since Python 3.4, pathlib.Path is the standard for file-system work. It is cross-platform, composable with /, and carries methods for existence checks, directory creation, and reading/writing, all in one object.

Always use with open(...) as fh (context manager) so the file is closed automatically, even if an exception occurs.\n", "
\n", "\n", "
\n", " Common Mistake: Bare string paths

\n", "open('data/file.csv') works but gives you no path-manipulation methods and is fragile on Windows vs. macOS/Linux. Use Path('data') / 'file.csv' instead.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "67", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "\n", "# Path composition: / operator joins parts, cross-platform\n", "project_root: Path = Path()\n", "data_dir: Path = project_root / \"data\"\n", "output_file: Path = data_dir / \"results\" / \"run_001.json\"\n", "\n", "print(f\"data_dir : {data_dir}\")\n", "print(f\"output_file : {output_file}\")" ] }, { "cell_type": "markdown", "id": "68", "metadata": {}, "source": [ "Every `Path` object knows its own parts: no string slicing to extract a filename or extension. `mkdir(exist_ok=True)` is the safest way to create a directory (no error if it already exists):" ] }, { "cell_type": "code", "execution_count": null, "id": "69", "metadata": {}, "outputs": [], "source": [ "# Path properties: inspect parts of a path without string slicing\n", "p = Path(\"tutorials/03-data-analysis/data/primary.csv\")\n", "print(f\"p.name : {p.name}\") # 'primary.csv'\n", "print(f\"p.stem : {p.stem}\") # 'primary'\n", "print(f\"p.suffix : {p.suffix}\") # '.csv'\n", "print(f\"p.parent : {p.parent}\") # 'tutorials/03-data-analysis/data'\n", "print(f\"p.parts : {p.parts}\")\n", "\n", "# Safe directory creation: no error if it already exists\n", "tmp_dir = Path(\"tmp_activity\")\n", "tmp_dir.mkdir(exist_ok=True)\n", "print(f\"\\ntmp_dir exists: {tmp_dir.exists()}\")" ] }, { "cell_type": "markdown", "id": "70", "metadata": {}, "source": [ "### Reading and writing files\n", "Always use the `with` statement. It closes the file automatically, even if an exception occurs. `DictWriter` writes rows as dicts keyed by column name:" ] }, { "cell_type": "code", "execution_count": null, "id": "71", "metadata": {}, "outputs": [], "source": [ "import csv\n", "from pathlib import Path\n", "\n", "tmp = Path(\"tmp_activity\")\n", "tmp.mkdir(exist_ok=True)\n", "\n", "csv_path = tmp / \"students.csv\"\n", "rows: list[dict[str, object]] = [\n", " {\"name\": \"Alice Kamau\", \"gpa\": 3.95, \"major\": \"CS\"},\n", " {\"name\": \"Bob Mwangi\", \"gpa\": 3.45, \"major\": \"Math\"},\n", " {\"name\": \"Carol Osei\", \"gpa\": 3.88, \"major\": \"CS\"},\n", "]\n", "\n", "with csv_path.open(\"w\", newline=\"\", encoding=\"utf-8\") as fh:\n", " writer = csv.DictWriter(fh, fieldnames=[\"name\", \"gpa\", \"major\"])\n", " writer.writeheader()\n", " writer.writerows(rows)\n", "\n", "print(f\"Wrote: {csv_path}\")" ] }, { "cell_type": "markdown", "id": "72", "metadata": {}, "source": [ "`DictReader` reads each row back as a `dict` keyed by header names, with no positional index access needed:" ] }, { "cell_type": "code", "execution_count": null, "id": "73", "metadata": {}, "outputs": [], "source": [ "import csv\n", "from pathlib import Path\n", "\n", "csv_path = Path(\"tmp_activity\") / \"students.csv\"\n", "\n", "with csv_path.open(encoding=\"utf-8\") as fh:\n", " reader = csv.DictReader(fh)\n", " loaded: list[dict[str, str]] = list(reader)\n", "\n", "for row in loaded:\n", " print(f\" {row['name']:<15} GPA={row['gpa']} {row['major']}\")" ] }, { "cell_type": "markdown", "id": "74", "metadata": {}, "source": [ "For single-document JSON files, `Path.write_text()` + `json.dumps()` and `Path.read_text()` + `json.loads()` is the most concise round-trip:" ] }, { "cell_type": "code", "execution_count": null, "id": "75", "metadata": {}, "outputs": [], "source": [ "import json\n", "from pathlib import Path\n", "\n", "tmp = Path(\"tmp_activity\")\n", "json_path = tmp / \"run_result.json\"\n", "run_data = {\"run_id\": \"exp-001\", \"accuracy\": 0.923, \"tags\": [\"baseline\"]}\n", "\n", "# Write: write_text is the cleanest one-liner for JSON\n", "json_path.write_text(json.dumps(run_data, indent=2), encoding=\"utf-8\")\n", "print(f\"Wrote: {json_path}\")\n", "\n", "# Read: read_text + json.loads\n", "reloaded: dict[str, object] = json.loads(json_path.read_text(encoding=\"utf-8\"))\n", "print(f\"Read back: {reloaded}\")" ] }, { "cell_type": "markdown", "id": "76", "metadata": {}, "source": [ "
Activity 2: experiment log

Write a function save_experiment(result: dict, log_dir: Path) -> Path that saves a result dict as a timestamped JSON file (run_20240101_120000.json) inside log_dir, creating the directory if it doesn't exist. Return the path to the written file. Then read it back and verify the round-trip.

Hint: Use datetime.now(tz=UTC).strftime('%Y%m%d_%H%M%S') for the timestamp and Path.mkdir(parents=True, exist_ok=True) for the directory.
" ] }, { "cell_type": "code", "execution_count": null, "id": "77", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "\n", "# TODO: implement save_experiment\n", "..." ] }, { "cell_type": "markdown", "id": "78", "metadata": {}, "source": [ "### Finding files\n", "`Path.iterdir()` yields the immediate children of a directory. `Path.rglob(pattern)` searches the entire subtree recursively:" ] }, { "cell_type": "code", "execution_count": null, "id": "79", "metadata": {}, "outputs": [], "source": [ "tmp = Path(\"tmp_activity\")\n", "print(\"Files in tmp_activity:\")\n", "for f in sorted(tmp.iterdir()):\n", " size = f.stat().st_size\n", " print(f\" {f.name:<30} {size:>6} bytes\")" ] }, { "cell_type": "markdown", "id": "80", "metadata": {}, "source": [ "`rglob('*.ipynb')` finds all matching files at any depth. After exploring, clean up the temporary directory with `shutil.rmtree()`:" ] }, { "cell_type": "code", "execution_count": null, "id": "81", "metadata": {}, "outputs": [], "source": [ "import shutil\n", "\n", "# rglob: recursive search by pattern\n", "notebooks = list(Path(\"tutorials\").rglob(\"*.ipynb\"))\n", "print(f\"Notebooks found: {len(notebooks)}\")\n", "for nb in sorted(notebooks)[:5]:\n", " print(f\" {nb}\")\n", "\n", "# Clean up tmp directory\n", "tmp = Path(\"tmp_activity\")\n", "shutil.rmtree(tmp)\n", "print(f\"\\nCleaned up: {tmp} exists = {tmp.exists()}\")" ] }, { "cell_type": "markdown", "id": "82", "metadata": {}, "source": [ "### Creating and checking directories\n", "\n", "`Path.mkdir()` creates directories; `Path.exists()` and `Path.is_dir()` check state without raising an error. Always prefer `mkdir(parents=True, exist_ok=True)` over conditionally calling `os.makedirs()`:" ] }, { "cell_type": "code", "execution_count": null, "id": "83", "metadata": {}, "outputs": [], "source": [ "results_dir = Path(\"results\") / \"experiment_001\"\n", "print(f\"Exists before : {results_dir.exists()}\")\n", "\n", "# parents=True creates any missing parent directories\n", "# exist_ok=True is silent if the directory already exists\n", "results_dir.mkdir(parents=True, exist_ok=True)\n", "print(f\"Exists after : {results_dir.exists()}\")\n", "print(f\"Is directory : {results_dir.is_dir()}\")\n", "\n", "# Write a file into the new directory\n", "log_file = results_dir / \"metrics.txt\"\n", "log_file.write_text(\"accuracy=0.923\\nval_loss=0.218\\n\")\n", "print(f\"Log file size : {log_file.stat().st_size} bytes\")\n", "\n", "# Clean up\n", "log_file.unlink()\n", "results_dir.rmdir()\n", "results_dir.parent.rmdir()\n", "print(\"Cleaned up\")" ] }, { "cell_type": "markdown", "id": "84", "metadata": {}, "source": [ "
\n", " Activity 5: experiment logger

\n", "Write a function that appends an experiment result as a JSON line to a log file, then reads and prints all logged runs.\n", "
log_experiment(Path('runs.jsonl'), run_id='run-001', accuracy=0.901, loss=0.312)\n",
    "log_experiment(Path('runs.jsonl'), run_id='run-002', accuracy=0.923, loss=0.218)\n",
    "\n",
    "# runs.jsonl contents:\n",
    "# {\"run_id\": \"run-001\", \"accuracy\": 0.901, \"loss\": 0.312}\n",
    "# {\"run_id\": \"run-002\", \"accuracy\": 0.923, \"loss\": 0.218}
\n", "Hint: JSONL (JSON Lines), one JSON object per line, is the standard format for streaming experiment logs. Use mode='a' to append.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "85", "metadata": {}, "outputs": [], "source": [ "import json\n", "from pathlib import Path\n", "\n", "\n", "def log_experiment(log_path: Path, **metrics: object) -> None:\n", " \"\"\"Append an experiment result as a JSON line to log_path.\n", "\n", " Args:\n", " log_path: Path to the .jsonl log file (created if absent).\n", " **metrics: Any metric name/value pairs to record.\n", " \"\"\"\n", " ... # TODO\n", "\n", "\n", "log_path = Path(\"runs.jsonl\")\n", "if log_path.exists():\n", " log_path.unlink() # start fresh for this activity\n", "\n", "log_experiment(log_path, run_id=\"run-001\", accuracy=0.901, loss=0.312)\n", "log_experiment(log_path, run_id=\"run-002\", accuracy=0.923, loss=0.218)\n", "\n", "# Read back and print all runs\n", "if log_path.exists():\n", " print(\"Logged runs:\")\n", " for line in log_path.read_text(encoding=\"utf-8\").splitlines():\n", " run = json.loads(line)\n", " print(f\" {run}\")\n", " log_path.unlink()\n", "else:\n", " print(\"(implement log_experiment above to see output)\")" ] }, { "cell_type": "markdown", "id": "86", "metadata": {}, "source": [ "## 7. Python gotchas\n", "\n", "You have seen these bugs. A function returns something different on the second call than on the first, even though you passed the same arguments. A list you never touched has changed. Division gives the wrong answer in exactly the edge case your test missed.\n", "\n", "These are not random bugs: they follow five reproducible patterns. Every experienced Python developer has been bitten by each one. This section names them so you can recognise them on sight." ] }, { "cell_type": "markdown", "id": "87", "metadata": {}, "source": [ "
\n", " Key Concept: Python's most dangerous bugs don't crash, they silently give wrong answers

\n", "Three gotchas account for most silent data bugs in Python:
1. Mutable default arguments: def f(x=[]):, the list is created once and shared across all calls. Use None as the sentinel instead.
2. is vs ==: is tests object identity (same object in memory); == tests value equality. Use == for values, is only for None.
3. Late-binding closures: a lambda inside a loop captures the variable, not its value at that moment. The loop ends, the variable has its final value, and every lambda returns the same thing.\n", "
" ] }, { "cell_type": "markdown", "id": "88", "metadata": {}, "source": [ "
\n", " Common Mistake: late binding in closures

\n", "When a closure captures a loop variable, it captures the variable by reference, not by value. By the time the closure runs, the loop has finished and the variable holds its final value.

\n", "Fix: use a default argument to bind the current value at definition time: lambda i=i: ...\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "89", "metadata": {}, "outputs": [], "source": [ "# Late binding: i is looked up when called, not when defined\n", "closures_bad = [lambda: i for i in range(4)] # noqa: B023\n", "print([f() for f in closures_bad]) # [3, 3, 3, 3] not [0, 1, 2, 3]\n", "\n", "# Fix: bind i at definition time via a default argument\n", "closures_ok = [lambda i=i: i for i in range(4)]\n", "print([f() for f in closures_ok]) # [0, 1, 2, 3]" ] }, { "cell_type": "markdown", "id": "90", "metadata": {}, "source": [ "
\n", " Common Mistake: class-level vs instance-level mutable attributes

\n", "A mutable attribute defined at class level (outside __init__) is shared across all instances. Appending to it from one instance affects every other.

\n", "Fix: always initialise mutable attributes inside __init__, or use field(default_factory=list) in a dataclass.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "91", "metadata": {}, "outputs": [], "source": [ "class Cohort:\n", " students: list[str] = [] # shared across ALL instances\n", "\n", "\n", "c1 = Cohort()\n", "c2 = Cohort()\n", "c1.students.append(\"Alice\")\n", "print(c2.students) # ['Alice'], c2 was not touched\n", "\n", "\n", "# Fix: initialise inside __init__\n", "class CohortFixed:\n", " def __init__(self) -> None:\n", " self.students: list[str] = [] # fresh list per instance\n", "\n", "\n", "c3 = CohortFixed()\n", "c4 = CohortFixed()\n", "c3.students.append(\"Alice\")\n", "print(c4.students) # []" ] }, { "cell_type": "code", "execution_count": null, "id": "92", "metadata": {}, "outputs": [], "source": [ "# GOTCHA 1: Mutable default argument\n", "# The default [] is created ONCE at function definition time - shared across all calls!\n", "def append_score_bad(score: float, history: list[float] = []) -> list[float]: # noqa: B006\n", " history.append(score)\n", " return history\n", "\n", "\n", "print(\"Bad default: the list leaks between calls:\")\n", "print(append_score_bad(82.0)) # [82.0] expected\n", "print(append_score_bad(91.0)) # [82.0, 91.0] WRONG: previous call leaked in!" ] }, { "cell_type": "markdown", "id": "93", "metadata": {}, "source": [ "The fix: use `None` as the default sentinel and create a fresh list inside the function body on each call:" ] }, { "cell_type": "code", "execution_count": null, "id": "94", "metadata": {}, "outputs": [], "source": [ "def append_score(score: float, history: list[float] | None = None) -> list[float]:\n", " if history is None:\n", " history = [] # new list created on every call where history is not provided\n", " history.append(score)\n", " return history\n", "\n", "\n", "print(\"Fixed: independent list each time:\")\n", "print(append_score(82.0)) # [82.0]\n", "print(append_score(91.0)) # [91.0] fresh list\n", "\n", "# Rule: never use a mutable object (list, dict, set) as a default argument value.\n", "# With @dataclass use field(default_factory=list) instead (shown in Sec. 4)." ] }, { "cell_type": "markdown", "id": "95", "metadata": {}, "source": [ "**Gotcha 2: assignment isn't a copy.** `b = a` creates a second name for the **same** list. Shallow `.copy()` creates a new outer container but inner objects are still shared. Use `copy.deepcopy()` for fully independent nested structures:" ] }, { "cell_type": "code", "execution_count": null, "id": "96", "metadata": {}, "outputs": [], "source": [ "# GOTCHA 2: Assignment is NOT a copy\n", "# For nested structures, .copy() is a SHALLOW copy: inner objects are still shared.\n", "\n", "import copy\n", "\n", "original: list[list[int]] = [[1, 2], [3, 4]]\n", "\n", "ref = original # same object\n", "shallow_copy = original.copy() # new outer list, shared inner lists\n", "deep_copy = copy.deepcopy(original) # completely independent\n", "\n", "original[0].append(99)\n", "print(f\"original : {original}\") # [[1, 2, 99], [3, 4]]\n", "print(f\"ref : {ref}\") # [[1, 2, 99], [3, 4]] : same object\n", "print(f\"shallow_copy : {shallow_copy}\") # [[1, 2, 99], [3, 4]] : inner list shared!\n", "print(f\"deep_copy : {deep_copy}\") # [[1, 2], [3, 4]] : fully independent" ] }, { "cell_type": "markdown", "id": "97", "metadata": {}, "source": [ "## Capstone exercise\n", "\n", "This exercise ties together Classes (Sec. 1), Exception Handling (Sec. 2), and File I/O (Sec. 3)." ] }, { "cell_type": "markdown", "id": "98", "metadata": {}, "source": [ "
Capstone exercise: student registry

Build a StudentRegistry class that:
  1. Stores a list of StudentRecord objects (from Sec. 1).
  2. Has a method add(record: StudentRecord) -> None.
  3. Has a method top_n(n: int = 3) -> list[StudentRecord] that returns the top n students by average.
  4. Has a @classmethod from_csv(cls, path: Path) -> 'StudentRegistry' that reads a CSV file and populates the registry using StudentRecord.from_dict(), raising a DataLoadError (custom exception) if the file doesn't exist.
  5. Has a method save_report(path: Path) -> None that writes a JSON summary of all students to disk.
Test it by creating a CSV file from the university dataset, loading the registry, printing the top 3 students, and saving a report.
" ] }, { "cell_type": "code", "execution_count": null, "id": "99", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "\n", "# TODO: define DataLoadError and StudentRegistry\n", "...\n", "\n", "# registry = StudentRegistry.from_csv(Path('data/students.csv'))\n", "# print(registry.top_n(3))\n", "# registry.save_report(Path('data/report.json'))" ] }, { "cell_type": "markdown", "id": "100", "metadata": {}, "source": [ "## Further reading\n", "\n", "| Resource | Why it matters |\n", "|---|---|\n", "| [Python Data Model](https://docs.python.org/3/reference/datamodel.html) | Complete reference for `__repr__`, `__eq__`, and all dunder methods |\n", "| [ABC module docs](https://docs.python.org/3/library/abc.html) | `abstractmethod`, `ABCMeta`, and `@abstractproperty` |\n", "| Gamma, E. et al. (1994). *Design Patterns*. | Factory Method and Template Method patterns underpinning classmethod factories and ABCs |\n", "| [Python exceptions hierarchy](https://docs.python.org/3/library/exceptions.html) | Full exception tree; shows which exceptions to catch and at what level |\n", "| [pathlib docs](https://docs.python.org/3/library/pathlib.html) | Complete reference for Path; covers `glob`, `walk`, `chmod`, and more |\n" ] }, { "cell_type": "markdown", "id": "101", "metadata": {}, "source": [ "## Summary\n", "\n", "| Concept | Key rule |\n", "|---|---|\n", "| `__init__` | Receives raw data and stores it on `self`; never compute derived values here |\n", "| `__repr__` | Return a human-readable, ideally re-creatable string; it's your first debugging tool |\n", "| `@property` | Compute derived values fresh each access; avoids stale state |\n", "| `@classmethod` | Alternative constructor for different input formats; keeps `__init__` clean |\n", "| ABC | Enforce a contract at instantiation time, not at code review time |\n", "| Exceptions | `except SpecificError` not bare `except`; `else` for success path; `finally` for cleanup |\n", "| `pathlib.Path` | Cross-platform paths; compose with `/`; read with `.read_text()`, write with `.write_text()` |\n", "| Context manager | `with open(...) as fh` always; file closes automatically |\n", "| Gotchas | Mutable defaults, `=` is not copy, late binding, class-level mutable attributes |\n", "\n", "**Next:** `05-math-statistics.ipynb`, arrays, broadcasting, and vectorised operations.\n" ] } ], "metadata": { "kernelspec": { "display_name": "ark", "language": "python", "name": "ark" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.12" } }, "nbformat": 4, "nbformat_minor": 5 }