{
"cells": [
{
"cell_type": "markdown",
"id": "075ad3e8",
"metadata": {},
"source": [
"# Vertical integration: RNA + ATAC\n",
"\n",
"Vertical integration combines modalities measured in the same cells. This tutorial is for RNA and ATAC, as in 10x Multiome. It runs scMM and VIMCCA on `D27mini_vertical`, 5,000 cells of the benchmark dataset `D27`, and then on data in your own format.\n",
"\n",
"Methods run on Linux. scMM has no Windows environment, and scMM and VIMCCA have no macOS environment. On Windows or macOS, [open this notebook in Colab](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_atac.ipynb): sections 1 to 4 take about 11 minutes there."
]
},
{
"cell_type": "markdown",
"id": "33026021",
"metadata": {},
"source": [
"## 1. Install\n",
"\n",
"pip may then print a note about restarting the kernel: no restart is needed, because the pins keep the numpy and pandas already installed."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ec12bcdc",
"metadata": {},
"outputs": [],
"source": [
"import numpy, pandas\n",
"pins = f\"numpy=={numpy.__version__} pandas=={pandas.__version__}\"\n",
"%pip install -q \"multibench-sc>=0.7.4\" {pins}"
]
},
{
"cell_type": "markdown",
"id": "019bb96e",
"metadata": {},
"source": [
"## 2. Download the data and the environments\n",
"\n",
"`mtb.data.fetch` downloads `D27mini_vertical` (24 MB) once. `mtb.env.install` downloads the environments for scMM and VIMCCA. The download is 1.2 GB."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0067de3",
"metadata": {},
"outputs": [],
"source": [
"import multibench as mtb\n",
"\n",
"mtb.data.fetch(\"D27mini_vertical\")\n",
"METHODS = [\"scMM\", \"VIMCCA\"]\n",
"envs = mtb.env.install(METHODS)"
]
},
{
"cell_type": "markdown",
"id": "588f9534",
"metadata": {},
"source": [
"## 3. Run the methods\n",
"\n",
"`run` runs each method on `D27mini_vertical` and scores its output with the [scIB metrics](https://dsichang.github.io/scMultiBench/tutorials/evaluate/#what-each-metric-measures). The clustering metrics score how well the embedding's clusters and neighbourhoods match the cell types, and how separate the clusters are. Higher is better for every metric.\n",
"\n",
"\n",
"Details
\n",
"\n",
"`params={\"Method\": {\"key\": value}}` sets a method's parameters, and `mtb.params_for` lists them.\n",
"\n",
"`mtb.load_run(\"out/D27mini_vertical\")` reloads these results later.\n",
"\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4f2a9189",
"metadata": {},
"outputs": [],
"source": [
"res = mtb.run(\"D27mini_vertical\", \"vertical\", methods=METHODS,\n",
" out_dir=\"out/D27mini_vertical\")\n",
"res.summary"
]
},
{
"cell_type": "markdown",
"id": "4e02d9ab",
"metadata": {},
"source": [
"## 4. Plot\n",
"\n",
"`res.plot()` draws the scores as a bubble table. Circle size ranks the rows within each column and colour shows the value scaled per column, so a small gap can look large. `res.summary` has the values.\n",
"\n",
"\n",
"Details
\n",
"\n",
"Metrics are grouped by family: blue for clustering. Each family starts with an `Overall` bar: each row's mean rank over the family's metrics, scaled so that the lowest gets the shortest bar.\n",
"\n",
"A column with fewer than two different values is drawn grey. A figure of one method is grey everywhere.\n",
"\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "78bb5543",
"metadata": {},
"outputs": [],
"source": [
"res.plot()"
]
},
{
"cell_type": "markdown",
"id": "51dab3a0",
"metadata": {},
"source": [
"## 5. Your own data\n",
"\n",
"Your data needs raw RNA counts, an ATAC matrix of the same cells and a cell type for each cell, as two `.h5ad` files.\n",
"\n",
"Recipes cover [a 10x `.h5` file](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-10x-h5), [labels in a CSV file](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-labels-csv), [no labels](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-no-labels) and [several donors or samples](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-donors).\n",
"\n",
"\n",
"Details: export
\n",
"\n",
"A 10x Multiome MuData goes in with one call. Here the labels are in `mdata.obs`. For labels in `mdata[\"rna\"].obs`, write `labels=\"rna:celltype\"`.\n",
"\n",
"```python\n",
"mtb.io.export_dataset(mdata, \"mydata/\" + NAME, rna=\"rna\", atac=\"atac\",\n",
" atac_kind=\"peak\", labels=\"obs:celltype\",\n",
" category=\"vertical\")\n",
"```\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"id": "031f4a0f",
"metadata": {},
"source": [
"The next cell writes two small example files, `my_rna.h5ad` and `my_atac.h5ad`, from 60% of the cells of `D27mini_vertical`. Skip it for your own data."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3fde3151",
"metadata": {},
"outputs": [],
"source": [
"mtb.data.write_example(\"D27mini_vertical\")"
]
},
{
"cell_type": "markdown",
"id": "8b8ed959",
"metadata": {},
"source": [
"Read your files. For your own data, change every file name marked `# <-`. On Colab, upload the files first: click the folder icon on the left, then the upload icon."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b443ee9c",
"metadata": {},
"outputs": [],
"source": [
"import anndata as ad\n",
"\n",
"rna = ad.read_h5ad(\"my_rna.h5ad\") # <- your RNA file\n",
"atac = ad.read_h5ad(\"my_atac.h5ad\") # <- your ATAC file, the same cells\n",
"print(rna)\n",
"print(atac)"
]
},
{
"cell_type": "markdown",
"id": "8cccf705",
"metadata": {},
"source": [
"Write the dataset folder. Change each name in quotes marked `# <-` to one the printout above lists for your files, with its prefix, as `\"obs:celltype\"`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "312a51e9",
"metadata": {},
"outputs": [],
"source": [
"NAME = \"MYMULTIOME\" # <- a new name for your own data\n",
"mtb.io.export_dataset(rna, \"mydata/\" + NAME,\n",
" rna=\"X\", # <- where the RNA counts are:\n",
" # \"X\" or \"layer:counts\"\n",
" atac=atac, # the ATAC file, peak counts in X\n",
" atac_kind=\"peak\", # the methods here read peaks\n",
" labels=\"obs:celltype\", # <- the cell-type column\n",
" overwrite=True)\n",
"mtb.scan(\"mydata/\" + NAME, \"vertical\", methods=METHODS)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "49e5fd6e",
"metadata": {},
"outputs": [],
"source": [
"mine = mtb.run(\"mydata/\" + NAME, \"vertical\", methods=METHODS,\n",
" out_dir=\"out/\" + NAME)\n",
"mine.summary"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8eecb918",
"metadata": {},
"outputs": [],
"source": [
"mine.plot()"
]
},
{
"cell_type": "markdown",
"id": "d12cfbf5",
"metadata": {},
"source": [
"## 6. Every method of this task\n",
"\n",
"14 methods run this task. [This notebook](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_atac_all.ipynb) runs them all. Its 13 environments download 31.8 GB on a computer without a GPU and 37.6 GB on one with a GPU."
]
},
{
"cell_type": "markdown",
"id": "4612e92e",
"metadata": {},
"source": [
"## Troubleshooting\n",
"\n",
"`res.summary[res.summary.status != \"scored\"]` lists each method that failed or was skipped. Its `reason` column ends with the method's error output.\n",
"\n",
"\n",
"Details
\n",
"\n",
"| symptom | fix |\n",
"|---|---|\n",
"| a method fails on a small dataset | `mtb.scan` names what the method needs from the data's size in its `caveat` column |\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"id": "a1db1476",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"- [Run a method](https://dsichang.github.io/scMultiBench/tutorials/run/), [Score an embedding](https://dsichang.github.io/scMultiBench/tutorials/evaluate/), [Load and plot results](https://dsichang.github.io/scMultiBench/tutorials/plot/) and [Discover methods](https://dsichang.github.io/scMultiBench/tutorials/discover/).\n",
"- [Look at an embedding](https://dsichang.github.io/scMultiBench/tutorials/evaluate/#look-at-an-embedding) draws a method's embedding as a UMAP.\n",
"- The [interactive explorer](https://shiny.maths.usyd.edu.au/scMultiBench/) has the full benchmark's rankings, with no install needed."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}