{ "cells": [ { "cell_type": "markdown", "id": "0", "metadata": {}, "source": [ "---\n", "title: \"Chapter 15: Grammar of graphics with lets-plot\"\n", "---" ] }, { "cell_type": "markdown", "id": "1", "metadata": {}, "source": [ "[](https://colab.research.google.com/github/sambaiga/ds-mlops-path/blob/main/tutorials/01-python-basics/15-lets-plot.ipynb) [](https://raw.githubusercontent.com/sambaiga/ds-mlops-path/main/tutorials/01-python-basics/15-lets-plot.ipynb)" ] }, { "cell_type": "markdown", "id": "2", "metadata": {}, "source": [ "You want to colour the scatter points by course. In matplotlib that means adding `c=results['course']` to `ax.scatter()`, but scatter expects a numeric array, not strings, so you need a colour map. The legend does not appear automatically, so you add `ax.legend()`. Now you manage the colour assignment yourself. Adding one data dimension turned a three-line chart into fifteen lines of housekeeping.\n", "\n", "The grammar of graphics solves this differently. You describe *what the data means* and let the library handle the rendering. `aes(color='course')` says 'colour represents course': the library picks the palette, maps it, and adds the legend automatically. The trend line is just `+ geom_smooth()`: one more layer, no manual computation.\n", "\n", "The tool is lets-plot, a Python port of R's ggplot2. By the end you will build any chart by composing three parts: data, aesthetics, and geometry. Chapter 16 (`16-data-storytelling.ipynb`) then applies a house style to both libraries.\n", "\n", "> Callout markers: [book cover page](../../index.qmd#callout-guide)." ] }, { "cell_type": "markdown", "id": "3", "metadata": {}, "source": [ "## Why the grammar of graphics?\n", "\n", "There is another way to think about it. A chart isn't a list of drawing commands: it's a **mapping** from data columns to visual channels (position, colour, shape, size). State that mapping once, and the library figures out how to draw it. Add a layer (a trend line, a rug, a label) and you extend the mapping rather than calling a new drawing function. This is the Grammar of Graphics, introduced by Leland Wilkinson in 1999 and popularised by Hadley Wickham's **ggplot2** for R.\n", "\n", "**Lets-Plot** ([lets-plot.org](https://lets-plot.org)) is JetBrains' Python implementation of the same grammar. Its API mirrors ggplot2 so closely that R users can read Lets-Plot code without a translation guide. It renders to HTML in Jupyter, to PNG for reports, and (in its Pro edition) to interactive Datalore dashboards.\n", "\n", "### Alternatives that use the same grammar\n", "\n", "| Library | Language | Notes |\n", "| --- | --- | --- |\n", "| **ggplot2** ([ggplot2.tidyverse.org](https://ggplot2.tidyverse.org)) | R | The original; Lets-Plot mirrors it deliberately |\n", "| **plotnine** ([plotnine.org](https://plotnine.org)) | Python | ggplot2 port; similar API, Matplotlib backend |\n", "| **Lets-Plot** ([lets-plot.org](https://lets-plot.org)) | Python / Kotlin | HTML-first output, fast, maintained by JetBrains |\n", "| **Vega-Altair** ([altair-viz.github.io](https://altair-viz.github.io)) | Python | Different grammar (Vega-Lite), interactive |\n", "\n", "### Already in your environment\n", "\n", "```bash\n", "uv add lets-plot # for a standalone project\n", "```" ] }, { "cell_type": "markdown", "id": "4", "metadata": {}, "source": [ "::: {.callout-note collapse=\"true\" icon=false}\n", "## Learning objectives\n", "\n", "By the end of Chapter 15 you will be able to:\n", "\n", "| # | Skill | Covered in |\n", "|---|---|---|\n", "| 1 | Explain the difference between declarative and imperative plotting | Sec. 1 |\n", "| 2 | Build a chart from `ggplot()`, `aes()`, and a `geom_*()` | Sec. 2 |\n", "| 3 | Distinguish mapping a variable from setting a fixed value | Sec. 3 |\n", "| 4 | Layer a statistical summary on top of raw data | Sec. 4 |\n", "| 5 | Facet one chart into many panels instead of looping over subplots | Sec. 5 |\n", "| 6 | Add titles, axis labels, and control colours with `labs()` and scale functions | Sec. 6 |\n", "| 7 | Name stat, position, and coord as grammar components, not one-off options | Sec. 7 |\n", ":::\n" ] }, { "cell_type": "markdown", "id": "5", "metadata": {}, "source": [ "## 1. Declarative vs. imperative\n", "\n", "In Chapter 14, building a scatter plot meant calling `ax.scatter()` directly: you were the one deciding which function draws which shape. The grammar of graphics flips this around. You describe **what the data means** (this column is the x position, this column is the colour) and the library decides how to draw it. The same description works whether you add one point or one million, and stays valid even if you later change the chart type entirely." ] }, { "cell_type": "code", "execution_count": null, "id": "6", "metadata": {}, "outputs": [], "source": [ "from lets_plot import LetsPlot\n", "\n", "LetsPlot.setup_html()" ] }, { "cell_type": "markdown", "id": "7", "metadata": {}, "source": [ "Rebuild the same dataset from Chapter 14: final scores across three courses and three Fall cohorts." ] }, { "cell_type": "code", "execution_count": null, "id": "8", "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "\n", "df = pd.read_csv(\"data/university_analytics.csv\")\n", "\n", "# Three courses share the same Fall track and run across three real\n", "# cohorts: Fall 2022, Fall 2023, and Fall 2024.\n", "courses = [\"Machine Learning\", \"Data Structures\", \"Python Programming\"]\n", "semesters = [\"Fall 2022\", \"Fall 2023\", \"Fall 2024\"]\n", "\n", "results = df[df[\"course\"].isin(courses) & df[\"semester\"].isin(semesters)].copy()\n", "results.head()" ] }, { "cell_type": "markdown", "id": "9", "metadata": {}, "source": [ "Here is the Chapter 14 scatter plot (study hours vs. final score) again, this time declaratively. `ggsize(width, height)` sets a fixed pixel size instead of leaving it to the browser: every chart in this notebook uses the same `500, 320` so they sit consistently in the page:" ] }, { "cell_type": "code", "execution_count": null, "id": "10", "metadata": {}, "outputs": [], "source": [ "from lets_plot import aes, geom_point, ggplot, ggsize\n", "\n", "ggplot(results, aes(x=\"study_hours\", y=\"final_score\")) + geom_point(alpha=0.4) + ggsize(500, 320)" ] }, { "cell_type": "markdown", "id": "11", "metadata": {}, "source": [ "
+: ggplot(data, aes(...)) declares the dataset and which columns map to which visual property, and one or more geom_*() layers say what shape to draw with that mapping. Change the geom_*() and the same aes() mapping produces a completely different chart type, often without touching anything else.\n",
"results DataFrame from this notebook.ggplot(results, aes(x='study_hours', y='final_score')).+ geom_point(alpha=0.5).+ geom_smooth(se=False) for a trend line.ggplot(data, aes(...)) owns the data and the default aesthetic mappings. aes() maps DataFrame columns to visual channels (x, y, color, size). geom_*() decides the shape (point, line, bar, histogram). Every chart you build in lets-plot is some combination of exactly these three.\n",
"final_score using geom_histogram(). Map x=\"final_score\" in aes(), and pass bins=20 to geom_histogram().\n",
"ggplot(results, aes(x=\"final_score\")) + geom_histogram(bins=20, fill=\"#4477AA\")\n", "
geom_lollipop() to reduce ink on ranked comparisonsgeom_bar() with geom_lollipop(): it draws a dot at the value and a thin stem from the baseline, making individual values easier to read when bars would crowd together.\n",
"geom_lollipop() bought you by comparing it against the bar chart it replaced.geom_lollipop(size=4, color=\"#0369A1\") with geom_bar(stat=\"identity\", fill=\"#0369A1\").aes(color='course') maps the course column to colour: the library assigns one colour per unique value and generates the legend automatically. geom_point(color='#0369A1') sets the colour to a fixed value for all points. Never put a constant inside aes(): that is the most common lets-plot mistake.\n",
"aes()aes(color=\"#4477AA\") tries to map the literal string \"#4477AA\" as if it were a column name. Lets-plot will either error or, worse, silently treat it as a constant category and draw a one-entry legend that says #4477AA. A fixed value is a setting and belongs as a plain keyword argument to the geom_*(), outside aes(). A column name that should vary per row is a mapping and belongs inside aes().\n",
"study_hours vs final_score where color is mapped to 'semester'.geom_point(color='#0369A1').'#0369A1' inside aes(color='#0369A1') and note what goes wrong.geom_point() + geom_smooth() draws points first, then the trend line on top. Reversing the order draws the line first and the points on top of it. This matters most when a layer has a solid fill that would otherwise hide whatever was drawn before it.\n",
"ggmarginal()ggmarginal() wraps any scatter plot with histograms or density curves along the x and y margins, letting you see both the bivariate relationship and the individual distributions without a separate figure:\n",
"\n",
"from lets_plot import ggmarginal\n",
"\n",
"p = (\n",
" ggplot(results, aes(x=\"study_hours\", y=\"final_score\", color=\"course\"))\n",
" + geom_point(alpha=0.4)\n",
" + geom_smooth(method=\"lm\", se=False)\n",
")\n",
"ggmarginal(p, type=\"density\")\n",
"\n",
"type accepts \"density\" (KDE curves), \"histogram\", or \"boxplot\". The marginals automatically inherit the color mapping so each course gets its own density curve.\n",
"ggplot(results, aes(x='study_hours', y='final_score')).+ geom_point(alpha=0.3, size=2).+ geom_density2d(color='#DC2626') as a second layer.+ geom_smooth(se=True, color='#0369A1').+ geom_point(tooltips=layer_tooltips().line('score: @final_score')).facet_wrap(facets=\"course\") splits the data by the course column and draws one panel per group, using the exact same aes() and geom_*() for every panel. There is no loop to write and no risk of accidentally giving one panel a different scale than the others, the bug from Chapter 14's Common Mistake callout. Add a second facet variable with facet_grid() for a full grid of panels.\n",
"study_hours vs. final_score, coloured by course, faceted into one panel per semester.\n",
"ggplot(results, aes(x=\"study_hours\", y=\"final_score\", color=\"course\")) \\\n",
" + geom_point(alpha=0.5) \\\n",
" + facet_wrap(facets=\"semester\")\n",
"labs() renames anything auto-generated from a column namelabs(title=..., x=..., y=..., color=...) is one more + layer. The named argument matches the aesthetic: use color= when you have aes(color=\"course\") and fill= when you have aes(fill=\"course\"). scale_color_manual(values=[...]) takes over from the default palette entirely, one hex code per group, in the order the groups appear. Chapter 16 applies one shared palette and theme to every chart in the book; here, you are choosing colours by hand, one chart at a time.\n",
"geom_density() chart from Section 4 and add a proper title, axis labels, and legend title with labs().\n",
"ggplot(results, aes(x=\"final_score\", fill=\"course\")) + geom_density(alpha=0.4) \\\n",
" + labs(title=..., x=..., y=..., fill=...)\n",
"data + aes + geom, optionally refined by stat (how raw rows become the numbers drawn), position (what happens when marks overlap), coord (the space they are drawn into), and facet (how many panels). Swap any one piece and the rest of the chart definition does not change: that is the whole idea of the grammar of graphics.\n",
"ggplot(results, aes(x='course', y='final_score')).+ stat_summary(fun='median', geom='bar', fill='#4477AA') to plot the median instead of the mean.+ coord_flip() so the course names are easier to read.groupby().mean() version?final_score, faceted by course, with geom_vline() marking the overall mean score so each course's panel can be compared against it.\n",
"overall_mean = results[\"final_score\"].mean()\n",
"\n",
"ggplot(results, aes(x=\"final_score\")) \\\n",
" + geom_histogram(bins=15, fill=\"#4477AA\") \\\n",
" + geom_vline(xintercept=overall_mean, color=\"#EE6677\", linetype=\"dashed\") \\\n",
" + facet_wrap(facets=\"course\") \\\n",
" + ggsize(700, 250)\n",
"Hint: geom_vline() takes a fixed xintercept, a setting, not a mapping, so it goes outside aes() just like the fixed colours in Sec. 3.\n",
"