{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"---\n",
"title: \"Chapter 17: The storytelling capstone\"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "1",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/sambaiga/ds-mlops-path/blob/main/tutorials/01-python-basics/17-storytelling-capstone.ipynb) [](https://raw.githubusercontent.com/sambaiga/ds-mlops-path/main/tutorials/01-python-basics/17-storytelling-capstone.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "2",
"metadata": {},
"source": [
"The department head has ten minutes before her next meeting. You have three charts and twelve numbers, and a strong opinion about which courses need help. She asks one question: *\"So what do I do?\"* Your charts are correct. None of them answer that.\n",
"\n",
"Chapters 14 through 16 taught you how to build a chart and how to make it honest. This chapter is about the step before any of that: knowing who the chart is for and what they need to decide, diagnosing why a draft chart is not landing, and recognising the moment you stop exploring and start explaining. It closes the Data Visualization part by putting every judgment call from the last three chapters, and both libraries, to work on one real case, start to finish. Chapter 18 picks up the engineering practices, uv, ruff, git, testing, that make analysis like this reproducible.\n",
"\n",
"> Callout markers: [book cover page](../../index.qmd#callout-guide)."
]
},
{
"cell_type": "markdown",
"id": "3",
"metadata": {},
"source": [
"::: {.callout-note collapse=\"true\" icon=false}\n",
"## Learning objectives\n",
"\n",
"By the end of Chapter 17 you will be able to:\n",
"\n",
"| # | Skill | Covered in |\n",
"|---|---|---|\n",
"| 1 | Name a chart's audience, decision, and medium, and let medium decide the library | Sec. 1 |\n",
"| 2 | Diagnose a failing chart with the Trifecta Checkup: content, story, or visual form | Sec. 2 |\n",
"| 3 | Decide explore vs explain and change how you build a chart accordingly | Sec. 3 |\n",
"| 4 | Rebuild one explanatory chart in matplotlib and lets-plot for two different mediums | Sec. 4 |\n",
"| 5 | Produce one explanatory chart, backed by a one-paragraph brief, from a real dataset | Capstone |\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "4",
"metadata": {},
"source": [
"## 1. Audience, decision, medium\n",
"\n",
"Every chart decision so far assumed the audience was you: fast defaults, all the data, nothing hidden. The moment someone else needs to act on what you found, three questions come before any chart type: who is this for, what do they need to decide, and how will they see it? Skip these and even a perfectly honest, perfectly decluttered chart can still miss."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"df = pd.read_csv(\"data/university_analytics.csv\")\n",
"\n",
"# Same three-course, three-Fall-cohort slice as Chapters 14 to 16: the\n",
"# department head's question is about these students specifically.\n",
"courses = [\"Machine Learning\", \"Data Structures\", \"Python Programming\"]\n",
"semesters = [\"Fall 2022\", \"Fall 2023\", \"Fall 2024\"]\n",
"results = df[df[\"course\"].isin(courses) & df[\"semester\"].isin(semesters)].copy()\n",
"results.head()"
]
},
{
"cell_type": "markdown",
"id": "6",
"metadata": {},
"source": [
"\n",
" Key Concept: Audience, decision, medium
\n",
"State three things before opening a plotting library: who will read this chart, what decision they need to make after seeing it, and how they will see it (a live presentation, a slide they read alone, a printed appendix). The same numbers, shown to a department head deciding whether to fund a study-hall program, need a different chart than the same numbers shown to a student checking their own grade.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "7",
"metadata": {},
"source": [
"Medium has a second consequence beyond how the chart is framed: it decides which library you reach for. matplotlib (Chapter 14) renders to a static image, the right choice for a printed report or a slide someone reads without you in the room. lets-plot (Chapter 15) renders to interactive HTML by default, the right choice for a live meeting or a notebook the reader scrolls through and hovers over themselves. Neither library is more correct: the medium decides, the same way it decided in Chapter 16, Section 5."
]
},
{
"cell_type": "markdown",
"id": "8",
"metadata": {},
"source": [
"\n",
" Activity 1 - Name the audience before you plot
\n",
"The department head is deciding whether to fund a mandatory study-hall program next term.
1. Fill in a dict: brief = {\"audience\": ..., \"decision\": ..., \"medium\": ...}.
2. audience: who reads this chart.
3. decision: the one thing they do differently depending on what the chart shows.
4. medium: how they will see it (live meeting, slide, printed report).
Expected: three short, specific answers, not \"everyone\" or \"management\".\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9",
"metadata": {},
"outputs": [],
"source": [
"# TODO: fill in the brief\n",
"brief = {\"audience\": ..., \"decision\": ..., \"medium\": ...}\n",
"brief"
]
},
{
"cell_type": "markdown",
"id": "10",
"metadata": {},
"source": [
"## 2. The Trifecta Checkup\n",
"\n",
"Here is a first attempt at answering the department head's question. Something about it is not working, but \"make it prettier\" is rarely the actual fix:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "11",
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"course_scores = results.groupby(\"course\")[\"final_score\"].mean()\n",
"\n",
"fig, ax = plt.subplots(figsize=(5, 3))\n",
"ax.bar(course_scores.index, course_scores.values, color=\"#4477AA\")\n",
"ax.set_ylabel(\"Mean final score\")\n",
"ax.set_title(\"Final score by course\")\n",
"fig.tight_layout()"
]
},
{
"cell_type": "markdown",
"id": "12",
"metadata": {},
"source": [
"\n",
" Key Concept: The Trifecta Checkup
\n",
"Berinato's Trifecta Checkup diagnoses a failing chart with three questions, asked in order: (1) Content: is the underlying analysis right, is this even the correct number to show? (2) Story: is there one clear message, could a reader state it in one sentence? (3) Visual form: is the chart type and design right for that message? Fixing (3) before checking (1) is polishing the wrong number. The chart above actually has real content: Machine Learning students do score noticeably lower than Python Programming students. It still fails the content check, for a subtler reason: \"course\" is not something the department head can act on. She is deciding whether to fund a study-hall program, an intervention aimed at how students study, not at which courses exist. Correct content and useful content are two different questions.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "13",
"metadata": {},
"source": [
"Fixed for content: not because the course numbers were wrong, but because they were not actionable. Group students by how much they studied instead of by which course they took, since study hours is the variable a study-hall program can actually change:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"metadata": {},
"outputs": [],
"source": [
"results[\"study_bucket\"] = pd.qcut(results[\"study_hours\"], 3, labels=[\"Low\", \"Mid\", \"High\"])\n",
"score_by_bucket = results.groupby(\"study_bucket\", observed=True)[\"final_score\"].mean()\n",
"\n",
"fig, ax = plt.subplots(figsize=(5, 3))\n",
"ax.bar(score_by_bucket.index, score_by_bucket.values, color=\"#4477AA\")\n",
"ax.set_ylabel(\"Mean final score\")\n",
"ax.set_ylim(0, 100)\n",
"ax.set_title(\"Final score by study-hours group\")\n",
"fig.tight_layout()"
]
},
{
"cell_type": "markdown",
"id": "15",
"metadata": {},
"source": [
"Content is fixed. Visual form has its own, independent failure mode: the same correct numbers, drawn dishonestly. Here is the same chart with nothing changed except the y-axis limits:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "16",
"metadata": {},
"outputs": [],
"source": [
"fig, ax = plt.subplots(figsize=(5, 3))\n",
"ax.bar(score_by_bucket.index, score_by_bucket.values, color=\"#4477AA\")\n",
"ax.set_ylabel(\"Mean final score\")\n",
"ax.set_ylim(50, 68)\n",
"ax.set_title(\"Final score by study-hours group\")\n",
"fig.tight_layout()"
]
},
{
"cell_type": "markdown",
"id": "17",
"metadata": {},
"source": [
"\n",
" Common Mistake: A visual-form problem can hide behind correct content
\n",
"Both charts above plot the exact same three numbers. The second one starts its y-axis at 50, the same truncation lie from Chapter 16, and it makes the Low-to-High gap look roughly three times larger than it is. Content and visual form are independent checks: this chart passes (1) and still fails (3). Run all three questions every time, even when an earlier one already passed.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "18",
"metadata": {},
"source": [
"Content and visual form both check out on the honest, zero-baseline version. Now the story check: does this chart state one message, or does the reader have to find it? Right now, nothing points at the answer: the department head still has to read three bar heights and work out the gap herself."
]
},
{
"cell_type": "markdown",
"id": "19",
"metadata": {},
"source": [
"\n",
" Activity 2 - Run the Trifecta Checkup
\n",
"Diagnose the truncated-y-axis chart from Chapter 16's capstone using the three questions.
1. Content: is the number plotted the right one for the question being asked?
2. Story: could you state the message in one sentence?
3. Visual form: is the chart type and axis honest?
Write one short sentence answering each question.
Expected: that chart's problem is visual form (the y-axis), not content or story: the checkup should say so explicitly, in order.\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "20",
"metadata": {},
"outputs": [],
"source": [
"# TODO: one sentence per question\n",
"content_check = \"...\"\n",
"story_check = \"...\"\n",
"visual_form_check = \"...\""
]
},
{
"cell_type": "markdown",
"id": "21",
"metadata": {},
"source": [
"## 3. Explore vs explain\n",
"\n",
"Every chart in Chapters 14 to 16 so far has been exploratory: fast, all the data, default styling, built to help you understand. The department head does not need to explore. She needs the one message, explained."
]
},
{
"cell_type": "markdown",
"id": "22",
"metadata": {},
"source": [
"\n",
" Key Concept: Explore, then explain
\n",
"An exploratory chart is for you: fast, shows everything, default styling, built to find a pattern. An explanatory chart is for someone else: filtered to the one message, branded, annotated, built to make a decision easy. The transition point is the moment you can write the \"so what\" in one sentence. Until you can write that sentence, you are still exploring, no matter how polished the chart looks.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "23",
"metadata": {},
"source": [
"The study-hours chart passed content and visual form. What is missing is the explanatory layer: the branding from Chapter 16 and an annotation that states the message, not just shows it. The department head asked for a printed handout, so matplotlib is the right tool here:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "24",
"metadata": {},
"outputs": [],
"source": [
"from ark.plot.matplot_theme import configure_matplotlib_style\n",
"\n",
"configure_matplotlib_style(font_size=10, fig_width=5)\n",
"\n",
"gap = score_by_bucket[\"High\"] - score_by_bucket[\"Low\"]\n",
"\n",
"fig, ax = plt.subplots(figsize=(6, 3.5))\n",
"colors = [\"#DC2626\" if b == \"High\" else \"#ADB5BD\" for b in score_by_bucket.index]\n",
"ax.bar(score_by_bucket.index, score_by_bucket.values, color=colors)\n",
"ax.set_ylabel(\"Mean final score\")\n",
"ax.set_title(f\"Students who study most score {gap:.0f} points higher\")\n",
"for i, v in enumerate(score_by_bucket.values):\n",
" ax.text(i, v + 1, f\"{v:.0f}\", ha=\"center\")\n",
"fig.tight_layout()"
]
},
{
"cell_type": "markdown",
"id": "25",
"metadata": {},
"source": [
"\n",
" Activity 3 - Write the \"so what\"
\n",
"Write the one-sentence takeaway the chart above is making explicit.
1. State the point gap between the High and Low study-hours groups.
2. Say what the department head should do next, in the same sentence.
Expected: one sentence a reader could act on without looking at the chart at all.\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26",
"metadata": {},
"outputs": [],
"source": [
"so_what = \"...\"\n",
"so_what"
]
},
{
"cell_type": "markdown",
"id": "27",
"metadata": {},
"source": [
"## 4. One message, two tools\n",
"\n",
"The chart above is right for a printed handout. Now the department head says she would rather review it live, in the meeting, hovering over each bar to see the exact numbers. Same audience, same decision, same message, a different medium, so a different library: lets-plot instead of matplotlib."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "28",
"metadata": {},
"outputs": [],
"source": [
"from lets_plot import LetsPlot, aes, geom_bar, ggplot, ggsize, labs, scale_fill_manual\n",
"\n",
"from ark.plot.theme import modern_theme\n",
"\n",
"LetsPlot.setup_html()\n",
"\n",
"score_by_bucket_df = score_by_bucket.reset_index()\n",
"\n",
"(\n",
" ggplot(score_by_bucket_df, aes(x=\"study_bucket\", y=\"final_score\", fill=\"study_bucket\"))\n",
" + geom_bar(stat=\"identity\")\n",
" + scale_fill_manual(values=[\"#ADB5BD\", \"#ADB5BD\", \"#DC2626\"])\n",
" + labs(\n",
" title=f\"Students who study most score {gap:.0f} points higher\",\n",
" x=\"\",\n",
" y=\"Mean final score\",\n",
" )\n",
" + modern_theme()\n",
" + ggsize(500, 320)\n",
")"
]
},
{
"cell_type": "markdown",
"id": "29",
"metadata": {},
"source": [
"\n",
" Key Concept: The judgment transfers, the tool does not have to match
\n",
"Every decision that made the matplotlib chart explanatory, one message, one colour reserved for emphasis, a title that states the finding, is unchanged here. Only the + layers changed. This is the payoff of Chapter 15's grammar: scale_fill_manual() and labs() work the same way regardless of which chart they are attached to, so switching libraries for a medium change costs a rewrite of the drawing code, not a rethink of the message.\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "30",
"metadata": {},
"source": [
"## Capstone: the department head's chart\n",
"\n",
"Everything from this chapter, applied once, on one real question: does study time predict performance strongly enough to justify funding a study-hall program, and does attendance tell the same story?"
]
},
{
"cell_type": "markdown",
"id": "31",
"metadata": {},
"source": [
"\n",
"
Capstone exercise - The full judgment loop\n",
"Produce one explanatory chart and a written brief, start to finish:\n",
"
\n",
"- Audience, decision, medium: reuse the
brief dict from Activity 1. Decide which medium fits: a live meeting or a printed handout. \n",
"- Content: bucket
attendance_pct into three groups the same way as study_bucket, and compute mean final_score per group. Is the gap as large as the study-hours gap? \n",
"- Story: write the one-sentence \"so what\" before touching a plotting library.
\n",
"- Visual form: build one chart, branded, that makes the message in step 3 the only thing a reader can take away. Use
configure_matplotlib_style() for a printed medium, or lets-plot with modern_theme() for a live one, matching the medium from step 1. \n",
"- Caption: print the brief and the \"so what\" sentence beneath the chart.
\n",
"
\n",
"
Hint: pd.qcut(results[\"attendance_pct\"], 3, labels=[\"Low\", \"Mid\", \"High\"]) mirrors the
study_bucket code from Section 2.\n",
"