{ "cells": [ { "cell_type": "markdown", "id": "6a7af110", "metadata": {}, "source": [ "---\n", "title: \"Pitching Hypothesis-Driven Data Investigations\"\n", "pagetitle: \"Pitching Investigations\"\n", "description-meta: \"Experiment checklist\"\n", "description-title: \"Experiment checklist\"\n", "author: \"Leon Yin\"\n", "author-meta: \"Leon Yin\"\n", "date: \"08-13-2023\"\n", "bibliography: references.bib\n", "execute: \n", " enabled: false\n", "keywords: pitching stories, experiment planning\n", "twitter-card:\n", " title: \"Pitching Hypothesis-Driven Data Investigations\"\n", " description: \"Answer these questions to bullet-proof your story\"\n", " image: assets/inspect-element-logo.jpg\n", "open-graph:\n", " title: \"Pitching Hypothesis-Driven Data Investigations\"\n", " description: \"Answer these questions to bullet-proof your story\"\n", " locale: us_EN\n", " site-name: Inspect Element\n", " image: assets/inspect-element-logo.jpg\n", "href: checklist\n", "---" ] }, { "cell_type": "markdown", "id": "89685dd1", "metadata": {}, "source": [ "This document asks essential questions to plan data experiments. Revisit these questions throughout your reporting, and use them to communicate your intentions and limitations with your editor. It will help determine if a story is worth pursuing by giving an estimation of time, complexity, and impact.\n", "\n", "As a side benefit, these questions form the backbone of a methodology to get reviewed by experts, as well as the target of your investigation.\n", "\n", "::: {.callout-note}\n", "Copy the checklist as text [below](#checklist), or as a public [Google Doc](https://docs.google.com/document/d/19tft98L90zUq4tn6TBXgaFacdbaI3Kr_/).\n", ":::\n", "\n", "\n", "## Checklist\n", "\n", "**1. What is the hypothesis of the story?**\n", "
\n", "_This is a testable claim that can be proven or disproven. The hypothesis is revised and honed in the early stages of an investigation._\n", "\n", "\n", "**2. Who is being harmed and at what scale?**\n", "
\n", "_Give us a sense of who is being harmed, and whether outcomes are proportionate?_\n", "\n", "\n", "**3. Who is causing the harm and what is the accountability angle?**\n", "
\n", "_What frameworks exist– legal, company policies, sworn testimonies, to hold them accountable?_\n", "\n", "\n", "**4. What is the evidence (anecdotal or otherwise) you’ve gathered that leads you to think you have a viable hypothesis?**\n", "
\n", "_If there is existing work on this topic (journalistic, academic, lived experience) how will you build off that work?_\n", "\n", "\n", "**5. What is a viability study you can perform?**\n", "
\n", "_How can you begin to collect and analyze data as a proof of concept?_\n", "\n", "\n", "**6. What data will you need to run an analysis? How will you gather the data?**\n", "
\n", "_Web scraping, public records requests, using open data? How will you decide what makes up your sample? Specificity is your friend._\n", "\n", "\n", "**7. How complicated is the data collection?**\n", "
\n", "_Will you be able to do this alone? Do you need to use proxies, cloud instances? Do you need to collect data over time?_\n", "\n", "\n", "**8. Will you need to filter out records from that data?**\n", "
\n", "_What records will you throw out and why?_\n", "\n", "\n", "**9. What are the limitations of the dataset(s) you are proposing to use? How will you test its accuracy?**\n", "
\n", "_Every dataset is imperfect and carries inherent assumptions. How will you assure what you’re writing is aligned with your dataset? How will you bulletproof that dataset?_\n", "\n", "\n", "**10. Do you need to classify the data for your experiment? If so, please describe how you propose doing that. Are there outside classifications or experts you can lean on? What are the limitations of your classification method?**\n", "
\n", "_Seldom is the outcome you want to measure already encoded in a column. Instead, you need to make that assessment._\n", "\n", "\n", "**11. How will you analyze the data? What statistical tests will run? Please list any limitations to your proposed method and any alternatives.**\n", "
\n", "_Start with simplicity._\n", "\n", "\n", "**12. What specific sentences will you be able to write based on your findings? What’s the lede? What’s the nutgraph?**\n", "
\n", "_Use this space to brainstorm how you’ll frame your findings. This helps make your findings tangible. I use TK’s at this step._\n", "\n", "\n", "**13. Can you imagine the charts or other visualizations this data will produce?**\n", "
\n", "_How will you communicate your findings visually?_\n" ] }, { "cell_type": "markdown", "id": "623e934f", "metadata": {}, "source": [ "## Acknowledgements\n", "\n", "This checklist is adapted from a checklist used by my editors Julia Angwin and Evelyn Larrubia at The Markup. Jeremy Singer-Vine provided feedback on the adapted list.\n" ] }, { "cell_type": "raw", "id": "d692e403", "metadata": {}, "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 5 }