{ "cells": [ { "cell_type": "markdown", "id": "132050f8", "metadata": {}, "source": [ "# Vertical integration: RNA + ADT + ATAC\n", "\n", "Vertical integration combines modalities measured in the same cells. This tutorial is for RNA, surface protein and ATAC from the same cells. It runs Matilda and UINMF on `D22mini`, 3,000 cells of the benchmark dataset `D22`, and then on data in your own format.\n", "\n", "Matilda and UINMF run on Linux and macOS (Apple silicon), and UINMF has no Windows environment. On Windows, [open this notebook in Colab](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_adt_atac.ipynb): sections 1 to 4 take about 6 minutes there." ] }, { "cell_type": "markdown", "id": "8b9b4de8", "metadata": {}, "source": [ "## 1. Install\n", "\n", "pip may then print a note about restarting the kernel: no restart is needed, because the pins keep the numpy and pandas already installed." ] }, { "cell_type": "code", "execution_count": null, "id": "63127d6d", "metadata": {}, "outputs": [], "source": [ "import numpy, pandas\n", "pins = f\"numpy=={numpy.__version__} pandas=={pandas.__version__}\"\n", "%pip install -q \"multibench-sc>=0.7.4\" {pins}" ] }, { "cell_type": "markdown", "id": "12212525", "metadata": {}, "source": [ "## 2. Download the data and the environments\n", "\n", "`mtb.data.fetch` downloads `D22mini` (8 MB) once. `mtb.env.install` downloads the environments for Matilda and UINMF. On Linux the download is 1.5 GB on a computer without a GPU and 4.0 GB on one with a GPU. On macOS it is 1.0 GB." ] }, { "cell_type": "code", "execution_count": null, "id": "2da0842c", "metadata": {}, "outputs": [], "source": [ "import multibench as mtb\n", "\n", "mtb.data.fetch(\"D22mini\")\n", "METHODS = [\"Matilda\", \"UINMF\"]\n", "envs = mtb.env.install(METHODS)" ] }, { "cell_type": "markdown", "id": "163fa598", "metadata": {}, "source": [ "## 3. Run the methods\n", "\n", "`run` runs each method on `D22mini` and scores its output with the [scIB metrics](https://dsichang.github.io/scMultiBench/tutorials/evaluate/#what-each-metric-measures). The clustering metrics score how well the embedding's clusters and neighbourhoods match the cell types, and how separate the clusters are. Higher is better for every metric.\n", "\n", "
\n", "Details\n", "\n", "`params={\"Method\": {\"key\": value}}` sets a method's parameters, and `mtb.params_for` lists them.\n", "\n", "`mtb.load_run(\"out/D22mini\")` reloads these results later.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "f4c4d6f8", "metadata": {}, "outputs": [], "source": [ "res = mtb.run(\"D22mini\", \"vertical\", methods=METHODS, out_dir=\"out/D22mini\")\n", "res.summary" ] }, { "cell_type": "markdown", "id": "60d46607", "metadata": {}, "source": [ "## 4. Plot\n", "\n", "`res.plot()` draws the scores as a bubble table. Circle size ranks the rows within each column and colour shows the value scaled per column, so a small gap can look large. `res.summary` has the values.\n", "\n", "
\n", "Details\n", "\n", "Metrics are grouped by family: blue for clustering. Each family starts with an `Overall` bar: each row's mean rank over the family's metrics, scaled so that the lowest gets the shortest bar.\n", "\n", "A column with fewer than two different values is drawn grey. A figure of one method is grey everywhere.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "3ed5b47e", "metadata": {}, "outputs": [], "source": [ "res.plot()" ] }, { "cell_type": "markdown", "id": "a0232c8a", "metadata": {}, "source": [ "## 5. Your own data\n", "\n", "Your data needs raw counts for RNA and protein (ADT) in one `.h5ad` file, an ATAC peak matrix of the same cells in another, and a cell type for each cell.\n", "\n", "Recipes cover [a 10x `.h5` file](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-10x-h5), [labels in a CSV file](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-labels-csv), [no labels](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-no-labels) and [several donors or samples](https://dsichang.github.io/scMultiBench/tutorials/run/#recipe-donors).\n", "\n", "
\n", "Details: export\n", "\n", "The example keeps the protein names in `adata.uns[\"protein_names\"]`. A DataFrame in `adata.obsm[\"protein\"]` brings its own names in its columns.\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "19fb2e0a", "metadata": {}, "source": [ "The next cell writes two small example files, `my_file.h5ad` and `my_atac.h5ad`, from 60% of the cells of `D22mini`. Skip it for your own data." ] }, { "cell_type": "code", "execution_count": null, "id": "d55d9f7a", "metadata": {}, "outputs": [], "source": [ "mtb.data.write_example(\"D22mini\")" ] }, { "cell_type": "markdown", "id": "340da20d", "metadata": {}, "source": [ "Read your files. For your own data, change every file name marked `# <-`. On Colab, upload the files first: click the folder icon on the left, then the upload icon." ] }, { "cell_type": "code", "execution_count": null, "id": "8f61e727", "metadata": {}, "outputs": [], "source": [ "import anndata as ad\n", "\n", "adata = ad.read_h5ad(\"my_file.h5ad\") # <- your RNA and protein file\n", "atac = ad.read_h5ad(\"my_atac.h5ad\") # <- your ATAC file, the same cells\n", "print(adata)\n", "print(atac)" ] }, { "cell_type": "markdown", "id": "16bc2b5d", "metadata": {}, "source": [ "Write the dataset folder. Change each name in quotes marked `# <-` to one the printout above lists for your files, with its prefix, as `\"obs:celltype\"` or `\"obsm:protein\"`." ] }, { "cell_type": "code", "execution_count": null, "id": "ec1d2220", "metadata": {}, "outputs": [], "source": [ "NAME = \"MYTRI\" # <- a new name for your own data\n", "mtb.io.export_dataset(adata, \"mydata/\" + NAME,\n", " rna=\"X\", # <- where the RNA counts are:\n", " # \"X\" or \"layer:counts\"\n", " adt=\"obsm:protein\", # <- where the proteins (ADT) are\n", " atac=atac, # the ATAC file, peak counts in X\n", " atac_kind=\"peak\", # the methods here read peaks\n", " labels=\"obs:celltype\", # <- the cell-type column\n", " overwrite=True)\n", "mtb.scan(\"mydata/\" + NAME, \"vertical\", methods=METHODS)" ] }, { "cell_type": "code", "execution_count": null, "id": "475981c0", "metadata": {}, "outputs": [], "source": [ "mine = mtb.run(\"mydata/\" + NAME, \"vertical\", methods=METHODS,\n", " out_dir=\"out/\" + NAME)\n", "mine.summary" ] }, { "cell_type": "code", "execution_count": null, "id": "0e6335b0", "metadata": {}, "outputs": [], "source": [ "mine.plot()" ] }, { "cell_type": "markdown", "id": "df0987dc", "metadata": {}, "source": [ "## 6. Every method of this task\n", "\n", "5 methods run this task. [This notebook](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_adt_atac_all.ipynb) runs them all. Its 5 environments download 12.7 GB on a computer without a GPU and 18.5 GB on one with a GPU." ] }, { "cell_type": "markdown", "id": "0dd823e1", "metadata": {}, "source": [ "## Troubleshooting\n", "\n", "`res.summary[res.summary.status != \"scored\"]` lists each method that failed or was skipped. Its `reason` column ends with the method's error output.\n", "\n", "
\n", "Details\n", "\n", "| symptom | fix |\n", "|---|---|\n", "| a method fails on a small dataset | `mtb.scan` names what the method needs from the data's size in its `caveat` column |\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "7a2d11d9", "metadata": {}, "source": [ "## Next steps\n", "\n", "- [Run a method](https://dsichang.github.io/scMultiBench/tutorials/run/), [Score an embedding](https://dsichang.github.io/scMultiBench/tutorials/evaluate/), [Load and plot results](https://dsichang.github.io/scMultiBench/tutorials/plot/) and [Discover methods](https://dsichang.github.io/scMultiBench/tutorials/discover/).\n", "- [Look at an embedding](https://dsichang.github.io/scMultiBench/tutorials/evaluate/#look-at-an-embedding) draws a method's embedding as a UMAP.\n", "- The [interactive explorer](https://shiny.maths.usyd.edu.au/scMultiBench/) has the full benchmark's rankings, with no install needed." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.10" } }, "nbformat": 4, "nbformat_minor": 5 }