# Chapter 0: From Basketball Stats to micrograd

Four Jupyter notebooks that walk from **realistic (hand-made) basketball stats**, through a
hand-written training loop, all the way to a tiny autograd engine — a
**"Chapter 0"** to do *before* Andrej Karpathy's
[micrograd](https://github.com/karpathy/micrograd) /
[*Neural Networks: Zero to Hero*](https://github.com/karpathy/nn-zero-to-hero).

This is an independent, unofficial educational companion. It is not affiliated
with or endorsed by Andrej Karpathy. The tiny autograd and neural-network code
in notebook 04 is adapted from micrograd; see
[Third-party notices](THIRD_PARTY_NOTICES.md) for attribution and license terms.

---

## Who this is for

This repo is for you if:

- You've tried watching Karpathy's micrograd / "spelled-out intro to neural networks" and still felt a little lost.
- You can write basic Python but ML math (loss, gradients, autograd) feels like black magic.
- You want to see **concrete features, targets, and numbers** before any abstractions.
- You want to understand **why** an autograd engine like micrograd exists, not just how to type in the code.

No PyTorch, no frameworks, no hidden magic. Just Python, Jupyter, and one
concrete question: **how many points will a basketball team score?**

The notebooks are designed to be run **top to bottom, in order**. Every output
you see was produced by actually running the code, not pasted in. Re-running
the notebooks reproduces the same learning results, although final decimal
places and rendered plot images can vary slightly across Python, Matplotlib,
and platform versions because of ordinary floating-point and rendering
differences.

### Data provenance

The basketball rows in these notebooks are entirely **synthetic and hand-made
for teaching**. They are not sourced from a league, team, player, or statistics
provider, and this project is not intended for sports forecasting or analysis
of real teams.

---

## The four ideas you'll build

Every neural network — and Karpathy's whole series — is built from four ideas.
You'll build each one by hand, on the same basketball dataset:

1. **A model** — a rule with tunable knobs that turns stats into a prediction.
2. **A loss** — one number measuring how wrong the model currently is.
3. **A gradient** — for each knob, "if I nudge it, does the loss go up or down, and how fast?"
4. **Gradient descent** — nudge every knob a little downhill, repeat, watch the loss fall.

---

## What's inside

The four notebooks live in the repo root. Run them in order.

### `01_basketball_linear_basics.ipynb`

Predicts a team's points from three stats (`off_rating`, `pace`, `opp_def_rating`).

- Starts with a **single knob** so the whole training loop fits in your head, then generalizes to all four.
- Defines a linear model and a mean-squared-error **loss**.
- Plots the loss as a **bowl** and shows that the **gradient is just its slope**.
- Estimates gradients **numerically** (nudge a knob, re-measure the loss).
- Runs **gradient descent** and watches the loss roll to the bottom.
- Checks one gradient against an **exact formula** and motivates why we'll need autograd.
- Bonus: **feature centering** — why we measure stats relative to a typical value.

### `02_derivatives_and_chain_rule.ipynb`

The small calculus toolkit that makes everything else click — built from zero and
**verified by nudging**, so nothing is taken on faith.

- What a **derivative** is: slope = rate of change; zero at the bottom of a valley.
- The handful of **rules** you need — the **power rule** (`d/dx x² = 2x`, the mysterious `2×`) and the **`tanh` derivative** (`1 − tanh²`).
- **Partial derivatives** = "nudge one variable"; the **gradient** is all of them together.
- **The chain rule** (`dy/dx = dy/du · du/dx`) — the heart of backpropagation.
- Closes by showing **backprop = the chain rule run backward** through a network.

This is the exact math Karpathy assumes. After it, his formulas stop looking like magic.

### `03_basketball_hidden_layer_pain.ipynb`

Extends the model with a tiny **hidden layer**: 3 inputs → 3 hidden units (`tanh`) → 1 output.

- Counts how the parameter count jumps from 4 to **16**.
- Trains it with the same numerical-gradient recipe (it still works!).
- Derives one hidden-layer weight's gradient **by hand** using notebook 02's chain rule — and shows the bookkeeping a *single* weight takes.
- Makes the point viscerally: hand-deriving gradients does **not** scale. We need a machine.

### `04_basketball_tiny_autograd.ipynb`

Builds a tiny scalar autograd engine in the spirit of micrograd:

- A `Value` class holding `data`, `grad`, its `_prev` parents, and a `_backward` rule (the chain rule, locally).
- Operator overloads for `+`, `*`, `**`, `tanh`; a `backward()` that topologically sorts the graph and flows gradients from the loss to every parameter.
- Verifies against the numerical "nudge" test, then rebuilds the basketball model with `Value`s — `loss.backward()` reproduces notebook 01's result with **zero** hand-derived gradients, and retrains notebook 03's "hopeless" net for free.
- Wraps it all into `Neuron` / `Layer` / `MLP` classes — the same classes Karpathy builds in his video.

The payoff: you'll have built, from a basketball dataset, a working miniature
of the engine Karpathy builds in his video.

---

## Getting started

### Prerequisites

- Python 3.10+
- A terminal (macOS, Linux, or Windows)

### Setup

1. Create and activate a virtual environment:

   ```bash
   python -m venv .venv

   # macOS / Linux:
   source .venv/bin/activate

   # Windows:
   .venv\Scripts\activate
   ```

2. Install dependencies:

   ```bash
   pip install -r requirements.txt
   ```

### Run the notebooks

From the project root (with the virtualenv activated):

```bash
jupyter notebook
```

- Your browser opens at `http://localhost:8888`.
- Open `01_basketball_linear_basics.ipynb` and run the cells top to bottom.
- Then `02_…`, `03_…`, and `04_…` in order.

---

## How this relates to Karpathy's micrograd / Zero-to-Hero

This repo is an unofficial **prequel** to Andrej Karpathy's
[*Neural Networks: Zero to Hero*](https://github.com/karpathy/nn-zero-to-hero)
course, the [micrograd project](https://github.com/karpathy/micrograd), and the
original
["spelled-out intro" video](https://www.youtube.com/watch?v=VMj-3S1tku0).

Use it like this:

1. Work through `01` until you're comfortable with parameters, loss, numerical gradients, and gradient descent on concrete numbers.
2. Work through `02` to get the derivative and the chain rule — the calculus Karpathy assumes.
3. Work through `03` to feel why manual gradients don't scale.
4. Work through `04` to build and use an autograd engine — including the `Neuron`/`Layer`/`MLP` classes — on the same task.

Then jump into Karpathy's video — it builds the same `Value` class and `MLP` and
scales them up to real neural nets. It'll make far more sense once you've seen
autograd motivated from a grounded example and the calculus demystified.

Notebook 04's `Value`, `Neuron`, `Layer`, and `MLP` implementation follows and
adapts the structure of Karpathy's MIT-licensed micrograd implementation. The
basketball progression, explanations, plots, and notebook composition are this
project's original educational material. The applicable upstream copyright and
license notices are preserved in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).

---

## Regenerating the notebooks

The notebooks are authored as plain-Python builder scripts in `build/`, then
executed to embed real outputs. To rebuild from scratch:

```bash
source .venv/bin/activate
python build/build_01.py && jupyter nbconvert --to notebook --execute --inplace 01_basketball_linear_basics.ipynb
python build/build_02.py && jupyter nbconvert --to notebook --execute --inplace 02_derivatives_and_chain_rule.ipynb
python build/build_03.py && jupyter nbconvert --to notebook --execute --inplace 03_basketball_hidden_layer_pain.ipynb
python build/build_04.py && jupyter nbconvert --to notebook --execute --inplace 04_basketball_tiny_autograd.ipynb
```

---

## Status

- [x] `01_basketball_linear_basics.ipynb` — linear model, numerical gradients, gradient descent
- [x] `02_derivatives_and_chain_rule.ipynb` — derivatives + the chain rule, from zero, verified by nudging
- [x] `03_basketball_hidden_layer_pain.ipynb` — hidden layer and the gradient-by-hand explosion
- [x] `04_basketball_tiny_autograd.ipynb` — micrograd-style engine + `Neuron`/`Layer`/`MLP` on basketball

Contributions, issues, and suggestions are welcome.

---

## License

MIT — see [LICENSE](LICENSE).