{
"cells": [
{
"cell_type": "markdown",
"id": "7d0ea12c",
"metadata": {},
"source": [
"# scMultiBench API quickstart (Colab)\n",
"\n",
"This notebook installs `multibench-sc`, looks up methods and draws the stored benchmark results. It runs no method, so it downloads nothing else. The integration tutorials install the method environments and run the methods."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e070fd6a",
"metadata": {},
"outputs": [],
"source": [
"import numpy, pandas\n",
"pins = f\"numpy=={numpy.__version__} pandas=={pandas.__version__}\"\n",
"%pip install -q \"multibench-sc>=0.7.4\" {pins}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2922017a",
"metadata": {},
"outputs": [],
"source": [
"import multibench as mtb\n",
"\n",
"print(len(mtb.find_methods()), \"methods in the package\")\n",
"mtb.find_methods(category=\"vertical\")"
]
},
{
"cell_type": "markdown",
"id": "5ad33a4d",
"metadata": {},
"source": [
"## Inspect a method\n",
"\n",
"`method_info` returns what the package knows about a method. `find_methods` filters methods by what your data has.\n",
"\n",
"\n",
"Details
\n",
"\n",
"A method matches when one of its variants meets every filter: category, modalities and `needs_labels`. A variant is one set of input files that a method accepts.\n",
"\n",
"The modality `\"atac\"` matches every method that reads ATAC. `\"atac_peak\"` keeps the methods that read peaks, and `\"atac_gas\"` the methods that read gene activity.\n",
"\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9baa9536",
"metadata": {},
"outputs": [],
"source": [
"info = mtb.method_info(\"Matilda\")\n",
"keys = (\"id\", \"language\", \"env\", \"needs_labels\", \"notes\", \"repo_url\", \"reference\",\n",
" \"supports\")\n",
"{k: info[k] for k in keys}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0c882f91",
"metadata": {},
"outputs": [],
"source": [
"mtb.find_methods(category=\"vertical\", modalities=[\"rna\", \"adt\"], needs_labels=False)"
]
},
{
"cell_type": "markdown",
"id": "0707807f",
"metadata": {},
"source": [
"## Draw the stored results\n",
"\n",
"The package ships stored results, so these figures draw without running anything.\n",
"\n",
"\n",
"Details
\n",
"\n",
"The stored results are the package's own runs of the methods, one dataset per category: D11, D28, D45 and D52.\n",
"\n",
"`mtb.plot.bar` draws each method's overall score on the dataset as one bar, best on top.\n",
"\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c06a8574",
"metadata": {},
"outputs": [],
"source": [
"stored = mtb.load_results(\"vertical\", dataset=\"D11\")\n",
"mtb.plot.bubble(stored)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61c585a3",
"metadata": {},
"outputs": [],
"source": [
"mtb.plot.bar(mtb.load_results(\"diagonal\", dataset=\"D28\"))"
]
},
{
"cell_type": "markdown",
"id": "99a3e816",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"- The integration tutorials install the environments, run methods and repeat the run on your own data: [vertical RNA + ADT](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_adt.ipynb), [vertical RNA + ATAC](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_atac.ipynb), [vertical RNA + ADT + ATAC](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_vertical_rna_adt_atac.ipynb), [diagonal [RNA, ATAC]](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_diagonal_rna_atac.ipynb), [diagonal [multiple RNA, multiple ATAC]](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_diagonal_multi.ipynb), [mosaic [RNA, RNA + ADT, ADT]](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_mosaic_rna_adt.ipynb), [mosaic [RNA, RNA + ATAC, ATAC]](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_mosaic_rna_atac.ipynb), [mosaic Mixed, with shared modality](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_mosaic_shared.ipynb), [mosaic Mixed, without shared modality](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_mosaic_unshared.ipynb), [cross Multiple RNA + ADT](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_cross_rna_adt.ipynb), [cross Multiple RNA + ATAC](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_cross_rna_atac.ipynb), [cross Multiple ADT + ATAC](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_cross_adt_atac.ipynb), [cross Multiple RNA + ADT + ATAC](https://colab.research.google.com/github/DSichang/scMultiBench/blob/main/notebooks/tutorial_cross_rna_adt_atac.ipynb).\n",
"- The [interactive explorer](https://shiny.maths.usyd.edu.au/scMultiBench/) has the full benchmark's rankings."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}