{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"---\n",
"title: \"Chapter 21: Type annotations\"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "1",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/sambaiga/ds-mlops-path/blob/main/tutorials/02-dev-tools/21-type-annotations.ipynb) [](https://raw.githubusercontent.com/sambaiga/ds-mlops-path/main/tutorials/02-dev-tools/21-type-annotations.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "2",
"metadata": {},
"source": [
"Your `grade-predictor` pipeline has been linted with Ruff and its style is consistent. But there is a failure Ruff cannot catch: you call `compute_grade(\"incomplete\", 85, 90)` and the function accepts it. Python does not complain. The string multiplies by 0.30, produces a concatenation, and the computation crashes three function calls later with a message that does not point back to the original mistake.\n",
"\n",
"The problem is intent: the function expects a float, the caller passed a string, and nothing in the code communicates that mismatch until runtime.\n",
"\n",
"Type annotations are that communication. Annotate `compute_grade(midterm: float, final: float, project: float) -> float:` and the IDE flags the wrong call the moment you type it. The error moves from a 2 am production crash to a red underline at edit time.\n",
"\n",
"Pydantic in Chapter 25 enforces these contracts at runtime when external data arrives. Type annotations are their static-time complement."
]
},
{
"cell_type": "markdown",
"id": "3",
"metadata": {},
"source": [
"::: {.callout-note collapse=\"true\" icon=false}\n",
"## Learning objectives\n",
"\n",
"By the end of this chapter you will be able to:\n",
"\n",
"| # | Skill | Covered in |\n",
"|---|---|---|\n",
"| 1 | Explain why type annotations matter in DS code | Sec. 1 |\n",
"| 2 | Write annotated function signatures with basic types | Sec. 2 |\n",
"| 3 | Annotate numpy arrays with `NDArray` and pandas DataFrames | Sec. 3 |\n",
"| 4 | Use `TypeAlias` and `Protocol` for complex DS types | Sec. 4 |\n",
"| 5 | Interpret `ty check` output and fix type errors | Sec. 5 |\n",
"| 6 | Apply gradual typing: where to start and what to skip | Sec. 6 |\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "4",
"metadata": {},
"source": [
"The diagram below shows the key shift: without type annotations, a wrong argument reaches the function and fails at runtime with a cryptic error; with them, the IDE underlines the mistake the moment you type it."
]
},
{
"cell_type": "markdown",
"id": "5",
"metadata": {},
"source": [
"{fig-alt=\"Two-column comparison. Left: compute_grade called with a string, Python accepts it, TypeError at runtime. Right: same call, IDE underlines the wrong argument type immediately.\"}"
]
},
{
"cell_type": "markdown",
"id": "6",
"metadata": {},
"source": [
"## 1. Why Type Annotations Matter"
]
},
{
"cell_type": "markdown",
"id": "7",
"metadata": {},
"source": [
"Two versions of the same function:\n",
"\n",
"```python\n",
"# Without annotations\n",
"def compute_grade(midterm, final, project, weights):\n",
" ...\n",
"\n",
"# With annotations\n",
"def compute_grade(\n",
" midterm: float,\n",
" final: float,\n",
" project: float,\n",
" weights: tuple[float, float, float] = (0.30, 0.45, 0.25),\n",
") -> float:\n",
" ...\n",
"```\n",
"\n",
"The annotated version is self-documenting: any editor with a type checker installed will warn you the moment you pass `\"82\"` instead of `82.0`. The unannotated version silently computes `\"82\" * 0.30 = \"82828282828282828282828282828282828282828282828282828282828282\"`. That is a real Python behavior, not a hypothetical.\n",
"\n",
"Python does not enforce annotations at runtime. That is the job of a static type checker. The annotation is documentation that a machine can check."
]
},
{
"cell_type": "markdown",
"id": "8",
"metadata": {},
"source": [
"\n",
" Key Concept: Annotations are documentation a machine can check
\n",
"They tell collaborators and your future self what a function expects and returns, without writing a word of prose. A type checker like ty reads them and flags type mismatches before the code runs.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "9",
"metadata": {},
"source": [
"## 2. Basic Annotations"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10",
"metadata": {},
"outputs": [],
"source": [
"# The basic scalar types used in grade-predictor\n",
"from __future__ import annotations # enables newer type syntax on Python 3.10+"
]
},
{
"cell_type": "markdown",
"id": "11",
"metadata": {},
"source": [
"Annotate the function with return type and parameter types. The IDE now knows `midterm` must be `float`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12",
"metadata": {},
"outputs": [],
"source": [
"def compute_grade(\n",
" midterm: float,\n",
" final: float,\n",
" project: float,\n",
" weights: tuple[float, float, float] = (0.30, 0.45, 0.25),\n",
") -> float:\n",
" if abs(sum(weights) - 1.0) > 0.001:\n",
" raise ValueError(f\"weights must sum to 1, got {sum(weights):.3f}\")\n",
" return midterm * weights[0] + final * weights[1] + project * weights[2]"
]
},
{
"cell_type": "markdown",
"id": "13",
"metadata": {},
"source": [
"Call it with valid values to confirm the annotation does not change behaviour:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"metadata": {},
"outputs": [],
"source": [
"compute_grade(80.0, 85.0, 90.0)"
]
},
{
"cell_type": "markdown",
"id": "15",
"metadata": {},
"source": [
"Python runs `compute_grade(\"82\", 85.0, 90.0)` without raising an error. The annotation is a contract, not a runtime check:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "16",
"metadata": {},
"outputs": [],
"source": [
"# Python does NOT enforce annotations at runtime\n",
"# The annotation says midterm: float, but we pass str: Python does not catch this\n",
"try:\n",
" result = compute_grade(\"82\", 85.0, 90.0)\n",
" print(result)\n",
"except TypeError as e:\n",
" print(f\"TypeError (not from the annotation): {e}\")"
]
},
{
"cell_type": "markdown",
"id": "17",
"metadata": {},
"source": [
"The full set of basic types used in DS function signatures:\n",
"\n",
"| Type | Use for |\n",
"|---|---|\n",
"| `int` | counts, indices |\n",
"| `float` | scores, rates, measurements |\n",
"| `str` | labels, column names, IDs |\n",
"| `bool` | flags, binary outcomes |\n",
"| `int or float` | either, when both are valid |\n",
"| `float or None` | an optional numeric value |\n",
"| `list[float]` | a sequence of floats |\n",
"| `tuple[float, float, float]` | a fixed-length sequence |\n",
"| `dict[str, float]` | a mapping from string keys to float values |"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "18",
"metadata": {},
"outputs": [],
"source": [
"def grade_to_letter(average: float) -> str:\n",
" if average >= 85:\n",
" return \"A\"\n",
" elif average >= 70:\n",
" return \"B\"\n",
" elif average >= 55:\n",
" return \"C\"\n",
" elif average >= 45:\n",
" return \"D\"\n",
" return \"F\"\n",
"\n",
"\n",
"def flag_at_risk(score: float | None, threshold: float = 50.0) -> bool:\n",
" if score is None:\n",
" return True # missing score is treated as at-risk\n",
" return score < threshold\n",
"\n",
"\n",
"def grade_summary(midterm: float, final: float, project: float) -> dict[str, float]:\n",
" avg = compute_grade(midterm, final, project)\n",
" return {\"average\": avg, \"midterm\": midterm, \"final\": final, \"project\": project}\n",
"\n",
"\n",
"grade_summary(80.0, 85.0, 90.0)"
]
},
{
"cell_type": "markdown",
"id": "19",
"metadata": {},
"source": [
"\n",
"
Activity 1: annotate Three Functions\n",
"Annotate these three signatures. Include one with a
float | None parameter for a nullable score, one that returns
dict[str, float], and one that takes a
list[str] of column names.\n",
"
def normalize_score(raw, min_val, max_val): ...\n",
"def compute_cohort_summary(scores): ... # returns dict\n",
"def select_columns(df, columns): ... # columns is list[str]\n",
"
\n",
" Pro Tip: pd.DataFrame is practical; pandera adds column types
\n",
"pd.DataFrame is a useful annotation even though it carries no column information. The next step is pandera.typing.DataFrame[Schema], which encodes column names and dtypes at the type level. Start with pd.DataFrame and graduate to pandera when you need column-level guarantees in a data pipeline.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "31",
"metadata": {},
"source": [
"\n",
"
Activity 2: annotate Array and DataFrame Functions\n",
"Write and annotate two functions: one that takes
NDArray[np.float64] and returns a normalized array, and one that takes a
pd.DataFrame and returns a filtered
pd.DataFrame. Confirm both run correctly on the
sample DataFrame above.\n",
"
def normalize_features(X: NDArray[np.float64]) -> NDArray[np.float64]: ...\n",
"def filter_passing(df: pd.DataFrame, threshold: float = 50.0) -> pd.DataFrame: ...\n",
"
\n",
" Key Concept: Protocol over import
\n",
"evaluate(model: Predictor, ...) accepts any object with predict and fit methods: sklearn's LinearRegression, XGBRegressor, a custom class. No import of sklearn needed in the type signature. This is structural subtyping, and it keeps your utility functions independent of any specific ML library.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "39",
"metadata": {},
"source": [
"## 5. Running ty"
]
},
{
"cell_type": "markdown",
"id": "40",
"metadata": {},
"source": [
"Install `ty` and run it on the `grade-predictor` source:\n",
"\n",
"```bash\n",
"uv add --optional dev ty\n",
"uv run ty check src/\n",
"```\n",
"\n",
"Reading the output: each line is `file:line:col: error[code] message`. Errors must be fixed. Warnings are suggestions.\n",
"\n",
"Common errors in DS code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "41",
"metadata": {},
"outputs": [],
"source": [
"# Simulate what ty would flag:\n",
"\n",
"# 1. Return type mismatch\n",
"def get_threshold() -> float:\n",
" return \"50.0\" # str, not float: ty flags this\n",
"\n",
"\n",
"# 2. Argument type mismatch\n",
"def double_score(score: float) -> float:\n",
" return score * 2\n",
"\n",
"\n",
"result = double_score(\"82\") # str passed as float: ty flags this\n",
"\n",
"\n",
"# 3. Optional not handled\n",
"def safe_grade(score: float | None) -> str:\n",
" return grade_to_letter(score) # score might be None: ty flags this\n",
"\n",
"\n",
"# Correct version\n",
"def safe_grade_fixed(score: float | None) -> str:\n",
" if score is None:\n",
" return \"N/A\"\n",
" return grade_to_letter(score)\n",
"\n",
"\n",
"safe_grade_fixed(None)"
]
},
{
"cell_type": "markdown",
"id": "42",
"metadata": {},
"source": [
"Configure `ty` in `pyproject.toml`:\n",
"\n",
"```toml\n",
"[tool.ty]\n",
"python-version = \"3.12\"\n",
"```\n",
"\n",
"The `--ignore-missing-imports` flag suppresses errors from third-party packages that lack type stubs. Pandas stubs are partial; great-tables has no stubs. Use it when third-party noise hides real errors in your own code."
]
},
{
"cell_type": "markdown",
"id": "43",
"metadata": {},
"source": [
"\n",
"
Activity 3: run ty and Fix Errors\n",
"Add full type annotations to
core.py. Run
uv run ty check src/. Fix every error (not warning) that ty reports in your own code. Confirm the output is clean before moving on.\n",
"
uv run ty check src/\n",
"# Fix each error line by line\n",
"uv run ty check src/ # should report 0 errors\n",
"
\n",
" Common Mistake: Annotating everything at once
\n",
"Trying to annotate a 2000-line codebase in a single session produces two outcomes: you give up halfway, or you annotate things badly and introduce incorrect type information that misleads the checker. Start with the five most-called public functions. Get them clean. Move on.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "49",
"metadata": {},
"source": [
"## Capstone: Fully Annotate core.py\n",
"\n",
"Bring the `grade-predictor/src/grade_predictor/core.py` to zero type errors."
]
},
{
"cell_type": "markdown",
"id": "50",
"metadata": {},
"source": [
"\n",
"
Capstone - Zero ty Errors\n",
"\n",
"
\n",
"- Annotate every function in
core.py: compute_grade, grade_to_letter, flag_at_risk, add_average_marks \n",
"- Use
NDArray[np.float64] for any numpy array parameters \n",
"- Use
pd.DataFrame and pd.Series for pandas types \n",
"- Run
uv run ty check src/ and bring it to zero errors \n",
"- Commit:
git commit -m \"feat(types): fully annotate core.py\" \n",
"
\n",
"
uv run ty check src/\n",
"# Fix all errors\n",
"uv run ty check src/ # zero errors\n",
"