{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"---\n",
"title: \"Part 15: Type Annotations\"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "1",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/sambaiga/ds-mlops-path/blob/main/tutorials/02-dev-tools/03-type-annotations.ipynb) [](https://raw.githubusercontent.com/sambaiga/ds-mlops-path/main/tutorials/02-dev-tools/03-type-annotations.ipynb)\n"
]
},
{
"cell_type": "markdown",
"id": "2",
"metadata": {},
"source": [
"**DS-MLOps Dev Tools**\n",
"\n",
"**Python 3.12+ | Author: Anthony Faustine**\n",
"\n",
"## Before you begin\n",
"\n",
"This notebook assumes you have completed [Part 13: Project Setup with uv](01-uv-project-setup.qmd) and [Part 14: Code Quality with ruff](02-code-quality-ruff.qmd). The `grade-predictor` project from those chapters is the codebase we annotate here.\n",
"\n",
"This is the one notebook in Part 3, because type annotations are pure Python: running annotated functions live shows the gap between what Python accepts at runtime and what a type checker would flag statically. Every example is executable.\n",
"\n",
"> Callout markers used throughout this notebook are explained on the [book cover page](../../index.qmd#callout-guide).\n"
]
},
{
"cell_type": "markdown",
"id": "3",
"metadata": {},
"source": [
"::: {.callout-note collapse=\"true\" icon=false}\n",
"## Learning Objectives\n",
"\n",
"By the end of Part 15 you will be able to:\n",
"\n",
"| # | Skill | Covered in |\n",
"|---|---|---|\n",
"| 1 | Explain why type annotations matter in DS code | Sec. 1 |\n",
"| 2 | Write annotated function signatures with basic types | Sec. 2 |\n",
"| 3 | Annotate numpy arrays with `NDArray` and pandas DataFrames | Sec. 3 |\n",
"| 4 | Use `TypeAlias` and `Protocol` for complex DS types | Sec. 4 |\n",
"| 5 | Interpret `ty check` output and fix type errors | Sec. 5 |\n",
"| 6 | Apply gradual typing: where to start and what to skip | Sec. 6 |\n",
":::\n"
]
},
{
"cell_type": "markdown",
"id": "4",
"metadata": {},
"source": [
"## 1. Why Type Annotations Matter"
]
},
{
"cell_type": "markdown",
"id": "5",
"metadata": {},
"source": [
"Two versions of the same function:\n",
"\n",
"```python\n",
"# Without annotations\n",
"def compute_grade(midterm, final, project, weights):\n",
" ...\n",
"\n",
"# With annotations\n",
"def compute_grade(\n",
" midterm: float,\n",
" final: float,\n",
" project: float,\n",
" weights: tuple[float, float, float] = (0.30, 0.45, 0.25),\n",
") -> float:\n",
" ...\n",
"```\n",
"\n",
"The annotated version is self-documenting: any editor with a type checker installed will warn you the moment you pass `\"82\"` instead of `82.0`. The unannotated version silently computes `\"82\" * 0.30 = \"82828282828282828282828282828282828282828282828282828282828282\"`. That is a real Python behavior, not a hypothetical.\n",
"\n",
"Python does not enforce annotations at runtime. That is the job of a static type checker. The annotation is documentation that a machine can check.\n"
]
},
{
"cell_type": "markdown",
"id": "6",
"metadata": {},
"source": [
"\n",
" Key Concept: Annotations are documentation a machine can check
\n",
"They tell collaborators and your future self what a function expects and returns, without writing a word of prose. A type checker like ty reads them and flags type mismatches before the code runs.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "7",
"metadata": {},
"source": [
"## 2. Basic Annotations"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8",
"metadata": {},
"outputs": [],
"source": [
"# The basic scalar types used in grade-predictor\n",
"from __future__ import annotations # enables newer type syntax on Python 3.10+\n",
"\n",
"\n",
"def compute_grade(\n",
" midterm: float,\n",
" final: float,\n",
" project: float,\n",
" weights: tuple[float, float, float] = (0.30, 0.45, 0.25),\n",
") -> float:\n",
" if abs(sum(weights) - 1.0) > 0.001:\n",
" raise ValueError(f\"weights must sum to 1, got {sum(weights):.3f}\")\n",
" return midterm * weights[0] + final * weights[1] + project * weights[2]\n",
"\n",
"\n",
"compute_grade(80.0, 85.0, 90.0)"
]
},
{
"cell_type": "markdown",
"id": "9",
"metadata": {},
"source": [
"Python runs `compute_grade(\"82\", 85.0, 90.0)` without raising an error. The annotation is a contract, not a runtime check:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10",
"metadata": {},
"outputs": [],
"source": [
"# Python does NOT enforce annotations at runtime\n",
"result = compute_grade(\"82\", 85.0, 90.0) # no error from Python\n",
"print(result) # \"82\" * 0.30 -> TypeError: can't multiply sequence by non-int of type 'float'\n",
"# Actually raises TypeError here, but only because float multiplication fails on str\n",
"# With an int weight it would silently produce wrong output"
]
},
{
"cell_type": "markdown",
"id": "11",
"metadata": {},
"source": [
"The full set of basic types used in DS function signatures:\n",
"\n",
"| Type | Use for |\n",
"|---|---|\n",
"| `int` | counts, indices |\n",
"| `float` | scores, rates, measurements |\n",
"| `str` | labels, column names, IDs |\n",
"| `bool` | flags, binary outcomes |\n",
"| `int or float` | either, when both are valid |\n",
"| `float or None` | an optional numeric value |\n",
"| `list[float]` | a sequence of floats |\n",
"| `tuple[float, float, float]` | a fixed-length sequence |\n",
"| `dict[str, float]` | a mapping from string keys to float values |\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12",
"metadata": {},
"outputs": [],
"source": [
"def grade_to_letter(average: float) -> str:\n",
" if average >= 85:\n",
" return \"A\"\n",
" elif average >= 70:\n",
" return \"B\"\n",
" elif average >= 55:\n",
" return \"C\"\n",
" elif average >= 45:\n",
" return \"D\"\n",
" return \"F\"\n",
"\n",
"\n",
"def flag_at_risk(score: float | None, threshold: float = 50.0) -> bool:\n",
" if score is None:\n",
" return True # missing score is treated as at-risk\n",
" return score < threshold\n",
"\n",
"\n",
"def grade_summary(midterm: float, final: float, project: float) -> dict[str, float]:\n",
" avg = compute_grade(midterm, final, project)\n",
" return {\"average\": avg, \"midterm\": midterm, \"final\": final, \"project\": project}\n",
"\n",
"\n",
"grade_summary(80.0, 85.0, 90.0)"
]
},
{
"cell_type": "markdown",
"id": "13",
"metadata": {},
"source": [
"\n",
"
Activity 1 - Annotate Three Functions\n",
"
Goal: Annotate these three signatures. Include one with a
float | None parameter for a nullable score, one that returns
dict[str, float], and one that takes a
list[str] of column names.\n",
"
def normalize_score(raw, min_val, max_val): ...\n",
"def compute_cohort_summary(scores): ... # returns dict\n",
"def select_columns(df, columns): ... # columns is list[str]\n",
"
\n",
" Pro Tip: pd.DataFrame is practical; pandera adds column types
\n",
"pd.DataFrame is a useful annotation even though it carries no column information. The next step is pandera.typing.DataFrame[Schema], which encodes column names and dtypes at the type level. Start with pd.DataFrame and graduate to pandera when you need column-level guarantees in a data pipeline.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "21",
"metadata": {},
"source": [
"\n",
"
Activity 2 - Annotate Array and DataFrame Functions\n",
"
Goal: Write and annotate two functions: one that takes
NDArray[np.float64] and returns a normalized array, and one that takes a
pd.DataFrame and returns a filtered
pd.DataFrame. Confirm both run correctly on the
sample DataFrame above.\n",
"
def normalize_features(X: NDArray[np.float64]) -> NDArray[np.float64]: ...\n",
"def filter_passing(df: pd.DataFrame, threshold: float = 50.0) -> pd.DataFrame: ...\n",
"
\n",
" Key Concept: Protocol over import
\n",
"evaluate(model: Predictor, ...) accepts any object with predict and fit methods: sklearn's LinearRegression, XGBRegressor, a custom class. No import of sklearn needed in the type signature. This is structural subtyping, and it keeps your utility functions independent of any specific ML library.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "29",
"metadata": {},
"source": [
"## 5. Running ty"
]
},
{
"cell_type": "markdown",
"id": "30",
"metadata": {},
"source": [
"Install `ty` and run it on the `grade-predictor` source:\n",
"\n",
"```bash\n",
"uv add --optional dev ty\n",
"uv run ty check src/\n",
"```\n",
"\n",
"Reading the output: each line is `file:line:col: error[code] message`. Errors must be fixed. Warnings are suggestions.\n",
"\n",
"Common errors in DS code:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "31",
"metadata": {},
"outputs": [],
"source": [
"# Simulate what ty would flag:\n",
"\n",
"# 1. Return type mismatch\n",
"def get_threshold() -> float:\n",
" return \"50.0\" # str, not float -- ty flags this\n",
"\n",
"\n",
"# 2. Argument type mismatch\n",
"def double_score(score: float) -> float:\n",
" return score * 2\n",
"\n",
"\n",
"result = double_score(\"82\") # str passed as float -- ty flags this\n",
"\n",
"\n",
"# 3. Optional not handled\n",
"def safe_grade(score: float | None) -> str:\n",
" return grade_to_letter(score) # score might be None -- ty flags this\n",
"\n",
"\n",
"# Correct version\n",
"def safe_grade_fixed(score: float | None) -> str:\n",
" if score is None:\n",
" return \"N/A\"\n",
" return grade_to_letter(score)\n",
"\n",
"\n",
"safe_grade_fixed(None)"
]
},
{
"cell_type": "markdown",
"id": "32",
"metadata": {},
"source": [
"Configure `ty` in `pyproject.toml`:\n",
"\n",
"```toml\n",
"[tool.ty]\n",
"python-version = \"3.12\"\n",
"```\n",
"\n",
"The `--ignore-missing-imports` flag suppresses errors from third-party packages that lack type stubs. Pandas stubs are partial; great-tables has no stubs. Use it when third-party noise hides real errors in your own code.\n"
]
},
{
"cell_type": "markdown",
"id": "33",
"metadata": {},
"source": [
"\n",
"
Activity 3 - Run ty and Fix Errors\n",
"
Goal: Add full type annotations to
core.py. Run
uv run ty check src/. Fix every error (not warning) that ty reports in your own code. Confirm the output is clean before moving on.\n",
"
uv run ty check src/\n",
"# Fix each error line by line\n",
"uv run ty check src/ # should report 0 errors\n",
"
\n",
" Common Mistake: Annotating everything at once
\n",
"Trying to annotate a 2000-line codebase in a single session produces two outcomes: you give up halfway, or you annotate things badly and introduce incorrect type information that misleads the checker. Start with the five most-called public functions. Get them clean. Move on.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "39",
"metadata": {},
"source": [
"## Capstone: Fully Annotate core.py\n",
"\n",
"Bring the `grade-predictor/src/grade_predictor/core.py` to zero type errors.\n"
]
},
{
"cell_type": "markdown",
"id": "40",
"metadata": {},
"source": [
"\n",
"
Capstone - Zero ty Errors\n",
"
Goal:\n",
"
\n",
"- Annotate every function in
core.py: compute_grade, grade_to_letter, flag_at_risk, add_average_marks \n",
"- Use
NDArray[np.float64] for any numpy array parameters \n",
"- Use
pd.DataFrame and pd.Series for pandas types \n",
"- Run
uv run ty check src/ and bring it to zero errors \n",
"- Commit:
git commit -m \"feat(types): fully annotate core.py\" \n",
"
\n",
"
uv run ty check src/\n",
"# Fix all errors\n",
"uv run ty check src/ # zero errors\n",
"