{ "cells": [ { "cell_type": "markdown", "id": "075ad3e8", "metadata": {}, "source": [ "# Vertical integration: RNA + ATAC\n", "\n", "Vertical integration combines modalities measured in the same cells. This tutorial is for RNA and ATAC, as in 10x Multiome. It runs scMM and VIMCCA on `D27mini_vertical`, 5,000 cells of the benchmark dataset `D27`, and then on data in your own format.\n", "\n", "Methods run on Linux. scMM has no Windows environment, and scMM and VIMCCA have no macOS environment. On Windows or macOS, [open this notebook in Colab](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_atac.ipynb): sections 1 to 4 take about 11 minutes there." ] }, { "cell_type": "markdown", "id": "33026021", "metadata": {}, "source": [ "## 1. Install\n", "\n", "pip may then print a note about restarting the kernel: no restart is needed, because the pins keep the numpy and pandas already installed." ] }, { "cell_type": "code", "execution_count": null, "id": "ec12bcdc", "metadata": {}, "outputs": [], "source": [ "import numpy, pandas\n", "pins = f\"numpy=={numpy.__version__} pandas=={pandas.__version__}\"\n", "%pip install -q \"multibench-sc>=0.7.4\" {pins}" ] }, { "cell_type": "markdown", "id": "019bb96e", "metadata": {}, "source": [ "## 2. Download the data and the environments\n", "\n", "`mtb.data.fetch` downloads `D27mini_vertical` (24 MB) once. `mtb.env.install` downloads the environments for scMM and VIMCCA. The download is 1.2 GB." ] }, { "cell_type": "code", "execution_count": null, "id": "e0067de3", "metadata": {}, "outputs": [], "source": [ "import multibench as mtb\n", "\n", "mtb.data.fetch(\"D27mini_vertical\")\n", "METHODS = [\"scMM\", \"VIMCCA\"]\n", "envs = mtb.env.install(METHODS)" ] }, { "cell_type": "markdown", "id": "588f9534", "metadata": {}, "source": [ "## 3. Run the methods\n", "\n", "`run` runs each method on `D27mini_vertical` and scores its output with the [scIB metrics](https://dsichang.github.io/scMultiBench/tutorials/evaluate/#what-each-metric-measures). The clustering metrics score how well the embedding's clusters and neighbourhoods match the cell types, and how separate the clusters are. Higher is better for every metric.\n", "\n", "
\n", "Details\n", "\n", "`params={\"Method\": {\"key\": value}}` sets a method's parameters, and `mtb.params_for` lists them.\n", "\n", "`mtb.load_run(\"out/D27mini_vertical\")` reloads these results later.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "4f2a9189", "metadata": {}, "outputs": [], "source": [ "res = mtb.run(\"D27mini_vertical\", \"vertical\", methods=METHODS,\n", " out_dir=\"out/D27mini_vertical\")\n", "res.summary" ] }, { "cell_type": "markdown", "id": "4e02d9ab", "metadata": {}, "source": [ "## 4. Plot\n", "\n", "`res.plot()` draws the scores as a bubble table. Circle size ranks the rows within each column and colour shows the value scaled per column, so a small gap can look large. `res.summary` has the values.\n", "\n", "
\n", "Details\n", "\n", "Metrics are grouped by family: blue for clustering. Each family starts with an `Overall` bar: each row's mean rank over the family's metrics, scaled so that the lowest gets the shortest bar.\n", "\n", "A column with fewer than two different values is drawn grey. A figure of one method is grey everywhere.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "78bb5543", "metadata": {}, "outputs": [], "source": [ "res.plot()" ] }, { "cell_type": "markdown", "id": "51dab3a0", "metadata": {}, "source": [ "## 5. Your own data\n", "\n", "Your data needs raw RNA counts, an ATAC matrix of the same cells and a cell type for each cell, as two `.h5ad` files.\n", "\n", "Recipes cover [a 10x `.h5` file](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-10x-h5), [labels in a CSV file](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-labels-csv), [no labels](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-no-labels) and [several donors or samples](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-donors).\n", "\n", "
\n", "Details: export\n", "\n", "A 10x Multiome MuData goes in with one call. Here the labels are in `mdata.obs`. For labels in `mdata[\"rna\"].obs`, write `labels=\"rna:celltype\"`.\n", "\n", "```python\n", "mtb.io.export_dataset(mdata, \"mydata/\" + NAME, rna=\"rna\", atac=\"atac\",\n", " atac_kind=\"peak\", labels=\"obs:celltype\",\n", " category=\"vertical\")\n", "```\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "031f4a0f", "metadata": {}, "source": [ "The next cell writes two small example files, `my_rna.h5ad` and `my_atac.h5ad`, from 60% of the cells of `D27mini_vertical`. Skip it for your own data." ] }, { "cell_type": "code", "execution_count": null, "id": "3fde3151", "metadata": {}, "outputs": [], "source": [ "mtb.data.write_example(\"D27mini_vertical\")" ] }, { "cell_type": "markdown", "id": "8b8ed959", "metadata": {}, "source": [ "Read your files. For your own data, change every file name marked `# <-`. On Colab, upload the files first: click the folder icon on the left, then the upload icon." ] }, { "cell_type": "code", "execution_count": null, "id": "b443ee9c", "metadata": {}, "outputs": [], "source": [ "import anndata as ad\n", "\n", "rna = ad.read_h5ad(\"my_rna.h5ad\") # <- your RNA file\n", "atac = ad.read_h5ad(\"my_atac.h5ad\") # <- your ATAC file, the same cells\n", "print(rna)\n", "print(atac)" ] }, { "cell_type": "markdown", "id": "8cccf705", "metadata": {}, "source": [ "Write the dataset folder. Change each name in quotes marked `# <-` to one the printout above lists for your files, with its prefix, as `\"obs:celltype\"`." ] }, { "cell_type": "code", "execution_count": null, "id": "312a51e9", "metadata": {}, "outputs": [], "source": [ "NAME = \"MYMULTIOME\" # <- a new name for your own data\n", "mtb.io.export_dataset(rna, \"mydata/\" + NAME,\n", " rna=\"X\", # <- where the RNA counts are:\n", " # \"X\" or \"layer:counts\"\n", " atac=atac, # the ATAC file, peak counts in X\n", " atac_kind=\"peak\", # the methods here read peaks\n", " labels=\"obs:celltype\", # <- the cell-type column\n", " overwrite=True)\n", "mtb.scan(\"mydata/\" + NAME, \"vertical\", methods=METHODS)" ] }, { "cell_type": "code", "execution_count": null, "id": "49e5fd6e", "metadata": {}, "outputs": [], "source": [ "mine = mtb.run(\"mydata/\" + NAME, \"vertical\", methods=METHODS,\n", " out_dir=\"out/\" + NAME)\n", "mine.summary" ] }, { "cell_type": "code", "execution_count": null, "id": "8eecb918", "metadata": {}, "outputs": [], "source": [ "mine.plot()" ] }, { "cell_type": "markdown", "id": "d12cfbf5", "metadata": {}, "source": [ "## 6. Every method of this task\n", "\n", "14 methods run this task. [This notebook](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_atac_all.ipynb) runs them all. Its 13 environments download 31.8 GB on a computer without a GPU and 37.6 GB on one with a GPU." ] }, { "cell_type": "markdown", "id": "4612e92e", "metadata": {}, "source": [ "## Troubleshooting\n", "\n", "`res.summary[res.summary.status != \"scored\"]` lists each method that failed or was skipped. Its `reason` column ends with the method's error output.\n", "\n", "
\n", "Details\n", "\n", "| symptom | fix |\n", "|---|---|\n", "| a method fails on a small dataset | `mtb.scan` names what the method needs from the data's size in its `caveat` column |\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "a1db1476", "metadata": {}, "source": [ "## Next steps\n", "\n", "- [Run a method](https://dsichang.github.io/scMultiBench/tutorials/run/), [Score an embedding](https://dsichang.github.io/scMultiBench/tutorials/evaluate/), [Load and plot results](https://dsichang.github.io/scMultiBench/tutorials/plot/) and [Discover methods](https://dsichang.github.io/scMultiBench/tutorials/discover/).\n", "- [Look at an embedding](https://dsichang.github.io/scMultiBench/tutorials/evaluate/#look-at-an-embedding) draws a method's embedding as a UMAP.\n", "- The [interactive explorer](https://shiny.maths.usyd.edu.au/scMultiBench/) has the full benchmark's rankings, with no install needed." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.10" } }, "nbformat": 4, "nbformat_minor": 5 }