{ "cells": [ { "cell_type": "markdown", "id": "accepting-radiation", "metadata": { "graffitiCellId": "id_m4c3phf" }, "source": [ "# How to make your research paper reproducible" ] }, { "cell_type": "markdown", "id": "civilian-saudi", "metadata": { "graffitiCellId": "id_30kvwou" }, "source": [ "## Abstract\n", "\n", "A reproducible paper is one in which every table and figure can be recreated by a reader or a reviewer from your data sets, using software on their own computer or using web-based tools.\n", "You may want to use a Jupyter notebook to write your next paper and make the entire document reproducible.\n", "\n", "It is assumed that the reader has a basic knowledge of the Python programming language and is familiar with using the terminal, and – if you wish to try this from scratch on your own computer – has the ability to install `pip` and `docker` on their computer.\n", "For the purposes of this tutorial, however, you only need a browser and a stable internet connection.\n", "The document is made interactive thanks to [Binder](https://mybinder.org/).\n", "You can also optionally archive the reproducible version of your paper on Zenodo.\n", "\n", "1. To start with, choose to perform your analysis in [one of the programming languages supported by Jupyter](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels).\n", "1. Create and launch a new Jupyter notebook.\n", "1. Enter your text in a “text cell”. You can also use LaTeX for any equations you want to include.\n", "1. Import your data in a “code cell”. You can store the data in the same folder as your notebook, or fetch data that was uploaded to Zenodo.\n", "1. Instead of preparing your tables and figures separately and placing them in your paper, include the code for each in a “code cell”. Then, “run” each code cell to generate the table or figure.\n", "1. When your paper is ready, upload the Jupyter notebook to a code-sharing platform like GitHub. Optionally, upload the file to Zenodo.\n", "1. Follow the instructions for Binder to allow your paper to be recreated in the cloud.\n", "1. Share the Binder link to your reproducible paper!\n", "\n", "As a bonus, using an experimental tool called “Jupyter Graffiti”, you can can watch a recording of this document in action right within it. Just click here!" ] }, { "cell_type": "markdown", "id": "interstate-diploma", "metadata": { "graffitiCellId": "id_dly0qyg" }, "source": [ "## Introduction\n", "\n", "Consider a piece of research, the output of which can be showcased by a single plot.\n", "For example, by analysing the spectrum of the \"invariant\" masses of two muons recorded by the [Compact Muon Solenoid](https://cms.cern) particle detector at the [Large Hadron Collider](https://home.cern/science/accelerators/large-hadron-collider), it is possible to detect the presence of a Z boson (with an invariant mass of ~91GeV/c²), as seen in the following image." ] }, { "cell_type": "markdown", "id": "protected-hierarchy", "metadata": { "graffitiCellId": "id_bue0pbr" }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "peaceful-forestry", "metadata": { "graffitiCellId": "id_1lj737o" }, "source": [ "Instead of simply including this image, you can provide the data that produced it, along with the code that prepared the was used in the process." ] }, { "cell_type": "markdown", "id": "aggressive-walker", "metadata": { "graffitiCellId": "id_ndrkocb" }, "source": [ "## Preparing the reproducible document\n", "\n", "You can install `jupyter` on your computer and create a notebook for your paper locally.\n", "However, since your final document will reside online and be made interactive via [Binder](https://mybinder.org/), you can work with the same environment provided by Binder by using [`repo2docker`](https://repo2docker.readthedocs.io/).\n", "You will need to install [`docker`](https://www.docker.com/) before proceeding.\n", "\n", "Once you have installed `repo2docker`, open a terminal and navigate to a new folder/directory where you would like to assemble the contents of your paper.\n", "Create a file called `requirements.txt` containing a list of the Python libraries to be used in your analysis.\n", "See [the file used in this tutorial](./requirements.txt).\n", "Save the file and then run the following command:\n", "\n", "```sh\n", "repo2docker -E .\n", "```\n", "\n", "The `-E` flag stands for “editable” and tells `repo2docker` to allow any changes made in your browser to be written to the directory you are working in.\n", "The `.` means “current directory”.\n", "\n", "The first time you run this command, `repo2docker` will build a “Docker container” with all of the software needed to run a Jupyter notebook.\n", "However, once this container has been built once, the program will run faster on subsequent launches.\n", "\n", "Some time after you run the above command, you will be presented with a URL in the same terminal window that you can copy and paste into a new browser tab.\n", "The Jupyter interface will greet you." ] }, { "cell_type": "markdown", "id": "specialized-leadership", "metadata": { "graffitiCellId": "id_lnmfyx9" }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "thrown-forum", "metadata": { "graffitiCellId": "id_yus1tmj" }, "source": [ "Now, click on the “New” button on the right side, and select a Python3 notebook.\n", "In your new notebook, you can add individual “cells” for (a) plain text prose formatted as [markdown](https://en.wikipedia.org/wiki/Markdown) or (b) code for performing your analysis." ] }, { "cell_type": "markdown", "id": "twenty-september", "metadata": { "graffitiCellId": "id_hes8zfl" }, "source": [ "Doubleclick on this text to enter the edit mode.\n", "You can make any changes in the text, if you wish, and exit the edit mode for a markdown cell by pressing Ctrl+Enter (or Command+Enter on macOS)." ] }, { "cell_type": "markdown", "id": "adjustable-feature", "metadata": { "graffitiCellId": "id_qzvls1d" }, "source": [ "### Setup\n", "\n", "Next, start preparing your analysis by importing the libraries you installed in the `requirements.txt` file and setting any configuration options as needed.\n", "The next cell is a code cell, and you can run the code in it by selecting the cell and hitting Ctrl/Command+Enter or by clicking on the “Run” button in the menu above.\n", "\n", "Go ahead, select the cell and click on “Run”." ] }, { "cell_type": "code", "execution_count": null, "id": "portuguese-consequence", "metadata": { "graffitiCellId": "id_o2ijekh" }, "outputs": [], "source": [ "# pandas is for working with data structures and performing data analysis\n", "import pandas as pd \n", "\n", "# numpy is for scientific computing\n", "import numpy as np\n", "\n", "# matplotlib is for plotting\n", "import matplotlib.pyplot as plt\n", "\n", "# Set the DPI for the images to make them larger than the default setting\n", "plt.rcParams['figure.dpi'] = 300" ] }, { "cell_type": "markdown", "id": "pressed-runner", "metadata": { "graffitiCellId": "id_19pwm5a" }, "source": [ "The blank space next to the cell in `In [ ]` will now show the number `1`.\n", "This shows that the code in the (input) cell has been executed and assigned a number.\n", "If you run the same cell again, the number will change to `2`." ] }, { "cell_type": "markdown", "id": "medieval-video", "metadata": { "graffitiCellId": "id_8rv90bg" }, "source": [ "### Import data\n", "\n", "You will need to connect the document with the data used in your analysis.\n", "The data can reside in the same directory as your paper or a sub-directory of the main directory.\n", "However, this will increase the size of your working directory and will make it more problematic to share your paper with others.\n", "\n", "Therefore, consider hosting the final dataset on an online data repository.\n", "In this tutorial, the data we will “analyse” are from the [CMS collaboration](https://cms.cern) at CERN, served from the CERN Open Data portal, in CSV format: https://opendata.web.cern.ch/record/545/.\n", "The next cell has the data-import step.\n", "\n", "Now, select this cell and click on “Run” as before." ] }, { "cell_type": "code", "execution_count": null, "id": "sealed-homeless", "metadata": { "graffitiCellId": "id_54go4i1" }, "outputs": [], "source": [ "data = pd.read_csv('https://opendata.cern.ch/record/545/files/Zmumu.csv')" ] }, { "cell_type": "markdown", "id": "appropriate-circular", "metadata": { "graffitiCellId": "id_dp9erdd" }, "source": [ "You can also show the reader that the data have been imported correctly by showing the first five rows (the “head”) of the CSV.\n", "When you run the next cell, an output block will appear below the input cell, carrying the same number as the input cell." ] }, { "cell_type": "code", "execution_count": null, "id": "circular-flight", "metadata": { "graffitiCellId": "id_g6eg6oi" }, "outputs": [], "source": [ "data.head()" ] }, { "cell_type": "markdown", "id": "structured-premium", "metadata": { "graffitiCellId": "id_rm8qm8c" }, "source": [ "### Describing and performing the analysis\n", "\n", "You can also use $\\LaTeX$ in your markdown prose.\n", "Here, I will quote from the [example provided by the CMS collaboration at CERN](https://github.com/cms-opendata-education/cms-jupyter-materials-english/blob/master/Exercises-with-open-data/Warming-up/Calculate-invariant-mass.ipynb).\n", "\n", "> Let's use the following expression for the invariant mass $M$ in the calculation:\n", "> \n", "> $M = \\sqrt{2p_{t1}p_{t2}(\\cosh(\\eta_{1}-\\eta_{2}) - \\cos(\\phi_{1}-\\phi_{2}))}$\n", "> \n", "> In the expression $p_t$ is the component of the momentum which is perpendicular to the beam axis, $\\eta$ is the pseudorapidity (angle) and $\\phi$ the azimuthal angle.\n", "> \n", "> In the calculation below we will use the `numpy` module which was named as `np` in the first code cell.\n", "With `numpy` it is possible to use mathematical commands like `sqrt` and `cosh` by calling first the name of the module (`np`) and then the command separated by a dot.\n", "So for example the square root could be called by writing `np.sqrt( )`.\n", ">\n", "> The labels _pt1_, _pt2_, _eta1_, _eta2_, _phi1_ and _phi2_ refer to the columns of the data.\n", "> In the code, you have to declare where the values will be taken.\n", "> So if you want to get the column _pt1_, you have to write `data.pt1` in the code.\n", "> \n", "> Now we are ready to calculate the values of the invariant masses for the different events.\n", "> `Numpy` will automatically calculate the values for all of the events when we give the calculation in the following form.\n", "> So the equation given is calculated for all of the rows.\n", "\n", "Once you have described the calculation, you can perform it in a code cell, for transparency.\n", "The next code cell calculates the “invariant” mass from the other data available.\n", "We can also display the head of the newly generated data." ] }, { "cell_type": "code", "execution_count": null, "id": "sharp-cooking", "metadata": { "graffitiCellId": "id_x8z6eaz" }, "outputs": [], "source": [ "invariant_mass = np.sqrt(2*data.pt1*data.pt2*(np.cosh(data.eta1-data.eta2) - np.cos(data.phi1-data.phi2)))" ] }, { "cell_type": "code", "execution_count": null, "id": "rotary-guest", "metadata": { "graffitiCellId": "id_m1izfrh" }, "outputs": [], "source": [ "invariant_mass.head()" ] }, { "cell_type": "markdown", "id": "sapphire-shell", "metadata": { "graffitiCellId": "id_8qemq9q" }, "source": [ "### Displaying plots\n", "\n", "Now that you have your data all ready for plotting, go ahead and write the code for doing so.\n", "The following code cell (which has comments to aide the reader) generates the same plot we saw earlier in the introduction." ] }, { "cell_type": "code", "execution_count": null, "id": "respected-printing", "metadata": { "graffitiCellId": "id_yihqkju" }, "outputs": [], "source": [ "# Make a histogram using the invariant mass calculated above.\n", "# N.B.: You can change the value for bins and re-run the same code.\n", "plt.hist(invariant_mass, bins=500)\n", "\n", "# Name the axises and give the title.\n", "plt.xlabel('Invariant mass [GeV]')\n", "plt.ylabel('Number of events')\n", "plt.title('The histogram of the invariant masses of two muons \\n')\n", "\n", "# Show the plot.\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "catholic-receptor", "metadata": { "graffitiCellId": "id_ypnmewy" }, "source": [ "Et voilà ! Your reproducible paper is ready.\n", "If a reader wants to make a change to your plotting code – say, changing the bin size – they can do so in the code cell above and re-run it.\n", "Go ahead, try changing the `bins` to a different number, like 50.\n", "\n", "Before you share it, make sure the document is clean.\n", "If you want to share it with all of the code already executed, simply click on `Kernel` and `Restart & Run All`. This will ensure that all of the code cells are run once only, and in the right order.\n", "If you want to share the paper without all of the code executed, click on `Kernel` and `Restart & Clear Output`.\n", "This will allow the reader to run the code on a clean slate, using Binder." ] }, { "cell_type": "markdown", "id": "brazilian-aurora", "metadata": { "graffitiCellId": "id_dj59jzu" }, "source": [ "## Sharing the paper\n", "\n", "Since your paper is compatible with Binder, you are all set.\n", "If you are familiar with `git` and use services such as GitHub, upload your entire directory to GitHub as a new repository.\n", "For example, this paper resides on Github at .\n", "You can get a “Badge” to put in your `README` file, that links directly to the executible version of your paper, like the following button that links to this page:\n", "\n", "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/RaoOfPhysics/reproducible-paper/main?filepath=index.ipynb)\n", "\n", "Once you upload your paper to GitHub, go to and get a Binder link to your paper as well. Share this link with your readers.\n", "\n", "Optionally, or if you aren’t familiar with `git`, upload the entire contents of your working directory to Zenodo and then point Binder to the Zenodo record." ] }, { "cell_type": "markdown", "id": "instrumental-homeless", "metadata": { "graffitiCellId": "id_uxze6qi" }, "source": [ "## Conclusion\n", "\n", "With a little preparation, you can make your next research paper more open, transparent and reproducible." ] }, { "cell_type": "markdown", "id": "vulnerable-vacuum", "metadata": { "graffitiCellId": "id_yxrdy58" }, "source": [ "## Acknowledgements\n", "\n", "This tutorial relies on example code produced by members of the CMS Collaboration.\n", "The code samples can be found in the following notebooks:\n", "\n", "- [Quick start to CMS open data](https://github.com/cms-opendata-education/cms-online-notebooks-for-binder/blob/master/quick-start-to-CMS-open-data.ipynb)\n", "- [Calculate invariant mass](https://github.com/cms-opendata-education/cms-jupyter-materials-english/blob/master/Exercises-with-open-data/Warming-up/Calculate-invariant-mass.ipynb)\n", "- [Invariant mass histogram](https://github.com/cms-opendata-education/cms-jupyter-materials-english/blob/master/Exercises-with-open-data/Warming-up/Invariant-mass-histogram.ipynb)\n", "- [Zmumu example by Tom McCauley](https://github.com/tpmccauley/cmsopendata-jupyter/blob/master/Zmumu.ipynb)\n", "- [Ymumu example by Tom McCauley](https://github.com/tpmccauley/cmsopendata-jupyter/blob/master/Ymumu.ipynb)\n", "\n", "### Python libraries and tools used\n", "\n", "- [`numpy`](https://numpy.org/)\n", "- [`scipy`](https://www.scipy.org/)\n", "- [`matplotlib`](https://matplotlib.org/)\n", "- [`jupyter`](https://jupyter.org/)\n", " - [`repo2docker`](https://repo2docker.readthedocs.io/)\n", " - [`jupyter-contrib-nbextensions`](https://jupyter-contrib-nbextensions.readthedocs.io/) for the table of contents\n", " - [`jupyter_nbextensions_configurator`](https://pypi.org/project/jupyter_nbextensions_configurator/)\n", " - [`jupytergraffiti`](https://github.com/willkessler/jupytergraffiti) for the narration" ] } ], "metadata": { "graffiti": { "firstAuthorId": "dev", "id": "id_5ceewci", "language": "EN" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": true, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 5 }