# Schema
Contact: brianraymor@chanzuckerberg.com
Document Status: _Approved_
Version: 6.0.0
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED" "MAY", and "OPTIONAL" in this document are to be interpreted as described in [BCP 14](https://tools.ietf.org/html/bcp14), [RFC2119](https://www.rfc-editor.org/rfc/rfc2119.txt), and [RFC8174](https://www.rfc-editor.org/rfc/rfc8174.txt) when, and only when, they appear in all capitals, as shown here.
## Schema versioning
The CELLxGENE schema version is based on [Semantic Versioning](https://semver.org/).
**Major version** is incremented when schema updates are incompatible with the AnnData data encoding or CELLxGENE API(s). Examples include:
* Renaming metadata fields
* Deprecating metadata fields
* Changing the type or format of a metadata field
**Minor version** is incremented when schema updates may require changes only to the `cellxgene-schema` CLI or the curation process. Examples include:
* Adding metadata fields
* Updating pinned ontologies or gene references
* Changing the validation requirements for a metadata field
**Patch version** is incremented for editorial updates and when adding organisms that do not require new metadata fields.
All changes are documented in the schema [Changelog](#appendix-a-changelog).
## Background
CELLxGENE aims to support the publication, sharing, and exploration of single-cell datasets. Building on those published datasets, CELLxGENE seeks to create references of the phenotypes and composition of cells that make up human tissues.
Creating references from multiple datasets requires some harmonization of metadata and features, but if that harmonization is too onerous, it will burden the goal of rapid data sharing. CELLxGENE balances publishing and reference creation needs by requiring datasets hosted by CELLxGENE Discover to include a small set of metadata readily available from data submitters.
This document describes the schema, a type of contract, that CELLxGENE requires all datasets to adhere to so that it can enable searching, filtering, and integration of datasets it hosts.
Note that the requirements in the schema are just the minimum required information. Datasets often have additional metadata, which is preserved in datasets submitted to CELLxGENE Discover.
## Overview
This schema supports multiple assay types. Each assay takes the form of one or more two-dimensional matrices whose values are quantitative measures of the phenotypes of cells.
The schema additionally describes how the dataset, genes, and cells are annotated to describe the biological and technical characteristics of the data.
This document is organized by:
* [General requirements](#general-requirements)
* [`X` (Matrix layers)](#x-matrix-layers), which describe the data required for different assays
* [`obs` (Cell metadata)](#obs-cell-metadata), which describe each cell in the dataset
* [`obsm` (Embeddings)](#obsm-embeddings), which describe each embedding in the dataset
* [`obsp`](#obsp), which describe pairwise annotation of observations
* [`var` and `raw.var` (Gene metadata)](#var-and-rawvar-gene-metadata), which describe each gene in the dataset
* [`varm`](#varm), which describe multi-dimensional annotation of variables/features
* [`varp`](#varp), which describe pairwise annotation of variables/features
* [`uns` (Dataset metadata)](#uns-dataset-metadata), which describe the dataset as a whole
## General Requirements
**AnnData.** The canonical data format for CELLxGENE Discover is HDF5-backed [AnnData](https://anndata.readthedocs.io/en/latest) as written by AnnData version 0.8.0 or greater. The on-disk format must be [AnnData specification (v0.1.0)](https://anndata.readthedocs.io/en/latest/fileformat-prose.html#anndata-specification-v0-1-0). Part of the rationale for selecting this format is to allow CELLxGENE to access both the data and metadata within a single file. The schema requirements and definitions for the AnnData `X`, `obs`, `var`, `raw.var`, `obsm`, and `uns` attributes are described below.
**Reserved Names**. The names of metadata fields MUST NOT start with `"__"`. The names of the metadata fields specified by the schema are **reserved** for the purposes and specifications described in the schema.
**Unique Names**. The names of schema and data submitter metadata fields in `obs` and `var` MUST be unique. For example, duplicate `"feature_biotype"` keys in AnnData `var` are not allowed.
Reserved Names from previous schema versions that have since been deprecated MUST NOT be present in datasets:
| Reserved Name |
AnnData |
Deprecated in |
| organism |
obs |
6.0.0 |
| organism_ontology_term_id |
obs |
6.0.0 |
| ethnicity |
obs |
3.0.0 |
| ethnicity_ontology_term_id |
obs |
3.0.0 |
| X_normalization |
uns |
3.0.0 |
| default_field |
uns |
2.0.0 |
| layer_descriptions |
uns |
2.0.0 |
| tags |
uns |
2.0.0 |
| version |
uns |
2.0.0 |
| contributors |
uns |
1.1.0 |
| preprint_doi |
uns |
1.1.0 |
| project_description |
uns |
1.1.0 |
| project_links |
uns |
1.1.0 |
| project_name |
uns |
1.1.0 |
| publication_doi |
uns |
1.1.0 |
|
|
|
**Redundant Metadata**. It is STRONGLY RECOMMENDED to avoid multiple metadata fields containing identical or similar information.
**No Personal Identifiable Information (PII)**. This is not strictly enforced by validation because it is difficult for software to predict what is and is not PII; however, curators MUST agree to the data submission policies of CELLxGENE Discover on behalf of data submitters which includes this requirement:
> It is my responsibility to ensure that this data is not identifiable. In particular, I commit that I will remove any [direct personal identifiers](https://docs.google.com/document/d/1sboOmbafvMh3VYjK1-3MAUt0I13UUJfkQseq8ANLPl8/edit) in the metadata portions of the data, and that CZI may further contact me if it believes more work is needed to de-identify it.
This includes names, emails, or other PII for researchers or curators involved in the data generation and submission.
#### *Note on types*
The types below are python3 types. Note that a python3 `str` is a sequence of Unicode code points, which is stored null-terminated and UTF-8-encoded by AnnData.
## `X` (Matrix Layers)
The data stored in the `AnnData.X` data matrix is the data that is viewable in CELLxGENE Explorer. For `AnnData.X`, `AnnData.raw.X`, and all layers, if a data matrix contains 50% or more values that are zeros, it MUST be encoded as a [`scipy.sparse.csr_matrix`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html) with zero values encoded as implicit zeros.
CELLxGENE's matrix layer requirements are tailored to optimize data reuse. Because each assay has different characteristics, the requirements differ by assay type. In general, CELLxGENE requires submission of "raw" data suitable for computational reuse when a standard raw matrix format exists for an assay. It is STRONGLY RECOMMENDED to also include a "normalized" matrix with processed values ready for data analysis and suitable for visualization in CELLxGENE Explorer. So that CELLxGENE's data can be provided in download formats suitable for both R and Python, the schema imposes the following requirements:
* All matrix layers MUST have the same shape, and have the same cell labels and gene labels.
* Because it is impractical to retain all barcodes in raw and normalized matrices, any cell filtering MUST be applied to both.
By contrast, those wishing to reuse datasets require access to raw gene expression values, so genes SHOULD NOT be filtered from either dataset.
Summarizing, any cell barcodes that are removed from the data MUST be filtered from both raw and normalized matrices and genes SHOULD NOT be filtered from the raw matrix.
* Any genes that publishers wish to filter from the normalized matrix MAY have their values replaced by zeros and MUST be flagged in the column [`feature_is_filtered`](#feature_is_filtered) of [`var`](#var-and-rawvar-gene-metadata), which will mask them from exploration.
* Additional layers provided at author discretion MAY be stored using author-selected keys, but MUST have the same cells and genes as other layers. It is STRONGLY RECOMMENDED that these layers have names that accurately summarize what the numbers in the layer represent (e.g. `"counts_per_million"`, `"SCTransform_normalized"`, or `"RNA_velocity_unspliced"`).
### Definitions for scATAC-seq assays
paired assay. `assay_ontology_term_id` is a descendant of both "EFO:0010891" for scATAC-seq and "EFO:0008913" for single-cell RNA sequencing. A gene expression matrix (RNA data) is required.
unpaired assay. `assay_ontology_term_id` is "EFO:0010891" for scATAC-seq or a descendant and is not a descendant of "EFO:0008913" for single-cell RNA sequencing. A gene activity matrix and not a peak matrix is required.
Also see the requirements for [scATAC-seq assets](#scatac-seq-assets).
The following table describes the matrix data and layers requirements that are **assay-specific**. If an entry in the table is empty, the schema does not have any other requirements on data in those layers beyond the ones listed above.
| Assay | "raw" required? | "raw" location | "normalized" required? | "normalized" location |
|-|-|-|-|-|
| scRNA-seq (UMI, e.g. 10x multiome, 10x v3, Slide-seqV2) | REQUIRED. Values MUST be de-duplicated molecule counts. Each cell MUST contain at least one non-zero value. All non-zero values MUST be positive integers stored as `numpy.float32`. Any two cells MUST NOT contain identical values for all their features. | `AnnData.raw.X` unless no "normalized" is provided, then `AnnData.X` | STRONGLY RECOMMENDED | `AnnData.X` |
| Visium Spatial (e.g. V1, CytAssist) | REQUIRED. Values MUST be de-duplicated molecule counts. All non-zero values MUST be positive integers stored as `numpy.float32`.
If uns['spatial']['is_single'] is False then each cell MUST contain at least one non-zero value.
If uns['spatial']['is_single'] is True then the unfiltered feature-barcode matrix (raw_feature_bc_matrix) MUST be used. See Space Ranger Feature-Barcode Matrices.
if assay_ontology_term_id is "EFO:0022860" for Visium CytAssist Spatial Gene Expression, 11mm, this matrix MUST contain 14336 rows; otherwise, this matrix MUST contain 4992 rows.
If the obs['in_tissue'] value is 1, then the cell MUST contain at least one non-zero value and any two cells MUST NOT contain identical values for all their features.
If any obs['in_tissue'] values are 0, then at least one cell corresponding to a obs['in_tissue'] with a value of 0 MUST contain a non-zero value.| `AnnData.raw.X` unless no "normalized" is provided, then `AnnData.X` | STRONGLY RECOMMENDED | `AnnData.X` |
| scRNA-seq (non-UMI, e.g. SS2) | REQUIRED. Values MUST be one of read counts (e.g. FeatureCounts) or estimated fragments (e.g. output of RSEM). Each cell MUST contain at least one non-zero value. All non-zero values MUST be positive integers stored as `numpy.float32`. Any two cells MUST NOT contain identical values for all their features. | `AnnData.raw.X` unless no "normalized" is provided, then `AnnData.X` | STRONGLY RECOMMENDED | `AnnData.X` |
| unpaired Accessibility (e.g. ATAC-seq, mCT-seq) | NOT REQUIRED | | REQUIRED | `AnnData.X` | STRONGLY RECOMMENDED |
|||||
## Integration Metadata
CELLxGENE requires ontology terms to enable search, comparison, and integration of data.
Ontology terms for cell metadata MUST use [OBO-format identifiers](http://www.obofoundry.org/id-policy.html), meaning a CURIE (prefixed identifier) of the form **Ontology:Identifier**.
For example, [EFO:0000001](https://www.ebi.ac.uk/ols4/ontologies/efo/classes?obo_id=EFO%3A0000001) is a term in the Experimental Factor Ontology (EFO).
The most accurate ontology term MUST always be used. If an exact or approximate ontology term is not available, a new term may be requested:
- For the [Cell Ontology], data submitters may [suggest a new term](https://github.com/obophenotype/cell-ontology/issues/new?assignees=bvarner-ebi&labels=new+term+request%2C+cellxgene&template=a_adding_term_cellxgene.md&title=%5BNTR-cxg%5D) and [notify the curation team](mailto:cellxgene@chanzuckerberg.com) of the pending term request, so that the datasets can be updated once the term is available.
To meet CELLxGENE schema requirements, the most accurate available CL term MUST be used until the new term is available. For example if `cell_type_ontology_term_id` describes a relay interneuron, but the most accurate available term in the CL ontology is [CL:0000099](https://www.ebi.ac.uk/ols4/ontologies/cl/classes?obo_id=CL%3A0000099) for *interneuron*, then the interneuron term can be used to fulfill this requirement and ensures that users searching for "neuron" are able to find these data. If no appropriate term can be found (e.g. the cell type is unknown), then `"unknown"` MUST be used. Users will still be able to access more specific cell type annotations that have been submitted with the dataset (but aren't required by the schema).
- For all other ontologies, data submitters may submit a [request to the curation team](mailto:cellxgene@chanzuckerberg.com) during the submission process.
Terms documented as obsolete in an ontology MUST NOT be used. For example, [EFO:0009310](https://www.ebi.ac.uk/ols4/ontologies/efo/classes/http%253A%252F%252Fwww.ebi.ac.uk%252Fefo%252FEFO_0009310) for *obsolete_10x v2* was marked as obsolete in EFO version 3.31.0 and replaced by [EFO:0009899](https://www.ebi.ac.uk/ols4/ontologies/efo/classes?obo_id=EFO%3A0009899) for *10x 3' v2*.
### Required Ontologies
The following ontology dependencies are *pinned* for this version of the schema.
| Ontology | OBO Prefix | Release | Download |
|:--|:--|:--|:--|
| [C. elegans Development Ontology] | WBls | [2025-04-01 WS297](https://github.com/obophenotype/c-elegans-development-ontology/releases/tag/v2025-04-01) | [wbls.owl](https://github.com/obophenotype/c-elegans-development-ontology/blob/v2025-04-01/wbls.owl) |
| [C. elegans Gross Anatomy Ontology] | WBbt | [2025-03-26 WS297](https://github.com/obophenotype/c-elegans-gross-anatomy-ontology/releases/tag/v2025-03-26) | [wbbt.owl](https://github.com/obophenotype/c-elegans-gross-anatomy-ontology/blob/v2025-03-26/wbbt.owl) |
| [Cell Ontology] | CL | [2025-04-10](https://github.com/obophenotype/cell-ontology/releases/tag/v2025-04-10) | [cl.owl](https://github.com/obophenotype/cell-ontology/releases/download/v2025-04-10/cl.owl)|
| [Drosophila Anatomy Ontology] | FBbt | [2025-03-27](https://github.com/FlyBase/drosophila-anatomy-developmental-ontology/releases/tag/v2025-03-27)| [fbbt.owl](https://github.com/FlyBase/drosophila-anatomy-developmental-ontology/releases/download/v2025-03-27/fbbt.owl) |
| [Drosophila Development Ontology] | FBdv | [2025-03-26](https://github.com/FlyBase/drosophila-developmental-ontology/releases/tag/v2025-03-26) | [fbdv.owl](https://github.com/FlyBase/drosophila-developmental-ontology/releases/download/v2025-03-26/fbdv.owl) |
| [Experimental Factor Ontology] | EFO | [2025-05-15 EFO 3.78.0](https://github.com/EBISPOT/efo/releases/tag/v3.78.0) | [efo.owl](https://github.com/EBISPOT/efo/releases/download/v3.78.0/efo.owl) |
| [Human Ancestry Ontology] | HANCESTRO | [3.0](https://github.com/EBISPOT/hancestro/releases/tag/3.0) | [hancestro-base.owl](https://github.com/EBISPOT/hancestro/blob/3.0/hancestro-base.owl) |
| [Human Developmental Stages] | HsapDv | [2025-01-23](https://github.com/obophenotype/developmental-stage-ontologies/releases/tag/v2025-01-23) | [hsapdv.owl](https://github.com/obophenotype/developmental-stage-ontologies/releases/download/v2025-01-23/hsapdv.owl) |
| [Mondo Disease Ontology] | MONDO | [2025-05-06](https://github.com/monarch-initiative/mondo/releases/tag/v2025-05-06) | [mondo.owl](https://github.com/monarch-initiative/mondo/releases/download/v2025-05-06/mondo.owl) |
| [Mouse Developmental Stages]| MmusDv | [2025-01-23](https://github.com/obophenotype/developmental-stage-ontologies/releases/tag/v2025-01-23) | [mmusdv.owl](https://github.com/obophenotype/developmental-stage-ontologies/releases/download/v2025-01-23/mmusdv.owl) |
| [NCBI organismal classification] | NCBITaxon | [2025-03-13](https://github.com/obophenotype/ncbitaxon/releases/tag/v2025-03-13) | [ncbitaxon.owl](https://github.com/obophenotype/ncbitaxon/releases/download/v2025-03-13/ncbitaxon.owl.gz) |
| [Phenotype And Trait Ontology] | PATO | [2025-05-14](https://github.com/pato-ontology/pato/releases/tag/v2025-05-14) | [pato.owl](https://github.com/pato-ontology/pato/blob/v2025-05-14/pato.owl) |
| [Uberon multi-species anatomy ontology] | UBERON | [2025-05-28](https://github.com/obophenotype/uberon/releases/tag/v2025-05-28) | [uberon.owl](https://github.com/obophenotype/uberon/releases/download/v2025-05-28/uberon.owl) |
| [Zebrafish Anatomy Ontology] | ZFA
ZFS | [2025-01-28](https://github.com/ZFIN/zebrafish-anatomical-ontology/releases/tag/v2025-01-28) | [zfa.owl](https://github.com/ZFIN/zebrafish-anatomical-ontology/blob/v2025-01-28/zfa.owl) |
| | | | |
[C. elegans Development Ontology]: https://obofoundry.org/ontology/wbls.html
[C. elegans Gross Anatomy Ontology]: https://obofoundry.org/ontology/wbbt.html
[Cell Ontology]: http://obofoundry.org/ontology/cl.html
[Drosophila Anatomy Ontology]: https://obofoundry.org/ontology/fbbt.html
[Drosophila Development Ontology]: https://obofoundry.org/ontology/fbdv.html
[Experimental Factor Ontology]: http://www.ebi.ac.uk/efo
[Human Ancestry Ontology]: http://www.obofoundry.org/ontology/hancestro.html
[Human Developmental Stages]: http://obofoundry.org/ontology/hsapdv.html
[Mondo Disease Ontology]: http://obofoundry.org/ontology/mondo.html
[Mouse Developmental Stages]: http://obofoundry.org/ontology/mmusdv.html
[NCBI organismal classification]: http://obofoundry.org/ontology/ncbitaxon.html
[Phenotype And Trait Ontology]: http://www.obofoundry.org/ontology/pato.html
[Uberon multi-species anatomy ontology]: http://www.obofoundry.org/ontology/uberon.html
[Zebrafish Anatomy Ontology]: https://obofoundry.org/ontology/zfa.html
### Required Gene Annotations
ENSEMBL identifiers are required for genes and [External RNA Controls Consortium (ERCC)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4978944/) identifiers for [RNA Spike-In Control Mixes] to ensure that all datasets measure the same features and can therefore be integrated.
The following gene annotation dependencies are *pinned* for this version of the schema.
| NCBITaxon | Source | Required version | Download |
|:--|:--|:--|:--|
| NCBITaxon:6239
for Caenorhabditis elegans | [ENSEMBL](https://www.ensembl.org/Caenorhabditis_elegans/Info/Index) | WBcel235 (GCA_000002985.3)
Ensembl 113 | [Caenorhabditis_elegans.WBcel235.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/caenorhabditis_elegans/Caenorhabditis_elegans.WBcel235.113.gtf.gz) |
| NCBITaxon:9483
for Callithrix jacchus | [ENSEMBL](https://www.ensembl.org/Callithrix_jacchus/Info/Index) | mCalJac1.pat.X
(GCA_011100555.1)
Ensembl 113 | [Callithrix_jacchus.mCalJac1.pat.X.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/callithrix_jacchus/Callithrix_jacchus.mCalJac1.pat.X.113.gtf.gz) |
| NCBITaxon:7955
for Danio rerio | [ENSEMBL](https://www.ensembl.org/Danio_rerio/Info/Index) | GRCz11 (GCA_000002035.4)
Ensembl 113 | [Danio_rerio.GRCz11.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/danio_rerio/Danio_rerio.GRCz11.113.gtf.gz) |
| NCBITaxon:7227
for Drosophila melanogaster| [ENSEMBL](https://www.ensembl.org/Drosophila_melanogaster/Info/Index) | BDGP6.46 (GCA_000001215.4)
Ensembl 113 | [Drosophila_melanogaster.BDGP6.46.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/drosophila_melanogaster/Drosophila_melanogaster.BDGP6.46.113.gtf.gz) |
| NCBITaxon:9595
for Gorilla gorilla gorilla | [ENSEMBL](https://www.ensembl.org/Gorilla_gorilla/Info/Index) | gorGor4
(GCA_000151905.3)
Ensembl 113 | [Gorilla_gorilla.gorGor4.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/gorilla_gorilla/Gorilla_gorilla.gorGor4.113.gtf.gz) |
| NCBITaxon:9606
for Homo sapiens | [GENCODE](https://www.gencodegenes.org/human/) | Human reference GRCh38.p14
(GENCODE v44/Ensembl 110) | [gencode.v44.primary_assembly.annotation.gtf](https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_44/gencode.v44.primary_assembly.annotation.gtf.gz) |
| NCBITaxon:9541
for Macaca fascicularis | [ENSEMBL](https://www.ensembl.org/Macaca_fascicularis/Info/Index) | Macaca_fascicularis_6.0
(GCA_011100615.1)
Ensembl 113 | [Macaca_fascicularis.Macaca_fascicularis_6.0.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/macaca_fascicularis/Macaca_fascicularis.Macaca_fascicularis_6.0.113.gtf.gz) |
| NCBITaxon:9544
for Macaca mulatta | [ENSEMBL](https://www.ensembl.org/Macaca_mulatta/Info/Index) | Mmul_10
(GCA_003339765.3)
Ensembl 113 | [Macaca_mulatta.Mmul_10.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/macaca_mulatta/Macaca_mulatta.Mmul_10.113.gtf.gz) |
| NCBITaxon:30608
for Microcebus murinus | [ENSEMBL](https://www.ensembl.org/Microcebus_murinus/Info/Index) | Mmur_3.0
(GCA_000165445.3)
Ensembl 113| [Microcebus_murinus.Mmur_3.0.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/microcebus_murinus/Microcebus_murinus.Mmur_3.0.113.gtf.gz) |
| NCBITaxon:10090
for Mus musculus | [GENCODE](https://www.gencodegenes.org/mouse/) | Mouse reference GRCm39
(GENCODE vM33/Ensembl 110) | [gencode.vM33.primary_assembly.annotation.gtf](https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M33/gencode.vM33.primary_assembly.annotation.gtf.gz) |
| NCBITaxon:9986
for Oryctolagus cuniculus | [ENSEMBL](https://www.ensembl.org/Oryctolagus_cuniculus/Info/Index) | OryCun2.0
(GCA_000003625.1)
Ensembl 113 | [Oryctolagus_cuniculus.OryCun2.0.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/oryctolagus_cuniculus/Oryctolagus_cuniculus.OryCun2.0.113.gtf.gz) |
| NCBITaxon:9598
for Pan troglodytes | [ENSEMBL](https://www.ensembl.org/Pan_troglodytes/Info/Index) | Pan_tro_3.0
(GCA_000001515.5)
Ensembl 113 | [Pan_troglodytes.Pan_tro_3.0.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/pan_troglodytes/Pan_troglodytes.Pan_tro_3.0.113.gtf.gz) |
| NCBITaxon:10116
for Rattus norvegicus | [ENSEMBL](https://www.ensembl.org/Rattus_norvegicus/Info/Index) | mRatBN7.2
(GCA_015227675.2)
Ensembl 113| [Rattus_norvegicus.mRatBN7.2.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/rattus_norvegicus/Rattus_norvegicus.mRatBN7.2.113.gtf.gz) |
| NCBITaxon:2697049
for SARS-CoV-2 | [ENSEMBL](https://covid-19.ensembl.org/index.html) | SARS-CoV-2 reference (ASM985889v3) | [Sars\_cov\_2.ASM985889v3.101.gtf](https://ftp.ensemblgenomes.org/pub/viruses/gtf/sars_cov_2/Sars_cov_2.ASM985889v3.101.gtf.gz) |
| NCBITaxon:9823
for Sus scrofa | [ENSEMBL](https://www.ensembl.org/Sus_scrofa/Info/Index) | Sscrofa11.1
(GCA_000003025.6)
Ensembl 113 | [Sus_scrofa.Sscrofa11.1.113.gtf](https://ftp.ensembl.org/pub/release-113/gtf/sus_scrofa/Sus_scrofa.Sscrofa11.1.113.gtf.gz) |
| NCBITaxon:32630
for synthetic construct | [ThermoFisher ERCC
Spike-Ins] | ThermoFisher ERCC RNA Spike-In Control Mixes (Cat # 4456740, 4456739) | [cms_095047.txt] |
|||||
[RNA Spike-In Control Mixes]: https://www.thermofisher.com/document-connect/document-connect.html?url=https%3A%2F%2Fassets.thermofisher.com%2FTFS-Assets%2FLSG%2Fmanuals%2Fcms_086340.pdf&title=VXNlciBHdWlkZTogRVJDQyBSTkEgU3Bpa2UtSW4gQ29udHJvbCBNaXhlcyAoRW5nbGlzaCAp
[ThermoFisher ERCC
Spike-Ins]: https://www.thermofisher.com/order/catalog/product/4456740#/4456740
[cms_095047.txt]: https://assets.thermofisher.com/TFS-Assets/LSG/manuals/cms_095047.txt
## `obs` (Cell Metadata)
`obs` is a [`pandas.DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).
Curators MUST annotate the following columns in the `obs` dataframe:
### index of pandas.DataFrame
| Key |
index of pandas.DataFrame |
| Annotator |
Curator MUST annotate. |
| Value |
str. The index of the pandas.DataFrame MUST contain unique identifiers for observations.
|
### array_col
| Key |
array_col |
| Annotator |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
int. This MUST be the value of the column coordinate for the corresponding spot from the array_col field in tissue_positions_list.csv or tissue_positions.csv. See Space Ranger Spatial Outputs.
| For Assay |
Value MUST be in the range between |
Visium Spatial Gene Expression V1 [EFO:0022857] |
0 and 127 |
Visium CytAssist Spatial Gene Expression, 6.5mm [EFO:0022859] |
0 and 127 |
Visium CytAssist Spatial Gene Expression, 11mm [EFO:0022860] |
0 and 223 |
|
### array_row
| Key |
array_row |
| Annotator |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
int. This MUST be value of the row coordinate for the corresponding spot from the array_row field in in tissue_positions_list.csv or tissue_positions.csv. See Space Ranger Spatial Outputs.
| For Assay |
Value MUST be in the range between |
Visium Spatial Gene Expression V1 [EFO:0022857] |
0 and 77 |
Visium CytAssist Spatial Gene Expression, 6.5mm [EFO:0022859] |
0 and 77 |
Visium CytAssist Spatial Gene Expression, 11mm [EFO:0022860] |
0 and 127 |
|
### assay_ontology_term_id
| Key |
assay_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be an EFO term and either:
-
the most accurate descendant of
"EFO:0002772" for assay by molecule excluding "EFO:0010961" for Visium Spatial Gene Expression while allowing its descendants
-
the most accurate descendant of
"EFO:0010183" for single cell library construction excluding "EFO:0010961" for Visium Spatial Gene Expression while allowing its descendants
If assay_ontology_term_id is either a descendant of "EFO:0010961" for Visium Spatial Gene Expression or "EFO:0030062" for Slide-seqV2 then all observations MUST contain the same value.
If assay_ontology_term_id is either "EFO:0010891" for scATAC-seq or its descendants, there are additional requirements for separate fragments file assets documented in scATAC-seq assets.
An assay based on 10X Genomics products SHOULD be the most accurate descendant of "EFO:0008995" for 10x technology. An assay based on SMART (Switching Mechanism at the 5' end of the RNA Template) or SMARTer technology SHOULD either be "EFO:0010184" for Smart-like or preferably its most accurate descendant.
Recommended values for specific assays:
|
### cell_type_ontology_term_id
| Key |
cell_type_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be "unknown" when:
-
no appropriate term can be found (e.g. the cell type is unknown)
-
assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression,
uns['spatial']['is_single'] is True, and the corresponding value of in_tissue is 0
The following CL terms MUST NOT be used:
Otherwise, for all other organisms this MUST be a CL term.
|
### development_stage_ontology_term_id
| Key |
development_stage_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. If unavailable, this MUST be "unknown".
Otherwise, for all other organisms this MUST be the most accurate descendant of UBERON:0000105 for life cycle stage, excluding UBERON:0000071 for death stage.
|
### disease_ontology_term_id
| Key |
disease_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be one of:
"PATO:0000461" for normal or healthy.
- one or more MONDO terms in ascending lexical order separated by the delimiter
" || " with no duplication of terms. For example, if the terms are "MONDO:1030008", "MONDO:0800349", "MONDO:0004604", and "MONDO:0043004" then the value MUST be "MONDO:0004604 || MONDO:0043004 || MONDO:0800349 || MONDO:1030008".
MONDO terms MUST be either:
|
### donor_id
| Key |
donor_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be free-text that identifies a unique individual that data were derived from. It is STRONGLY RECOMMENDED that this identifier be designed so that it is unique to:
- a given individual within the collection of datasets that includes this dataset
- a given individual across all collections in CELLxGENE Discover
It is STRONGLY RECOMMENDED that "pooled" be used for observations from a sample of multiple individuals that were not confidently assigned to a single individual through demultiplexing.
It is STRONGLY RECOMMENDED that "unknown" ONLY be used for observations in a dataset when it is not known which observations are from the same individual.
|
### in_tissue
| Key |
in_tissue |
| Annotator |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
int. This MUST be the value for the corresponding spot from the in_tissue field in tissue_positions_list.csv or tissue_positions.csv which is either 0 if the spot falls outside tissue or 1 if the spot falls inside tissue. See Space Ranger Spatial Outputs.
|
### is_primary_data
| Key |
is_primary_data |
| Annotator |
Curator MUST annotate. |
| Value |
bool. This MUST be False if uns['spatial']['is_single'] is False. This MUST be True if this is the canonical instance of this cellular observation and False if not. This is commonly False for meta-analyses reusing data or for secondary views of data.
|
### self_reported_ethnicity_ontology_term_id
| Key |
self_reported_ethnicity_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. If
organism_ontology_term_id is
"NCBITaxon:9606" for Homo sapiens, then the value MUST be one or more HANCESTRO terms in ascending lexical order separated by the delimiter " || " with no duplication of terms or "unknown" if unavailable.
For example, if the terms are "HANCESTRO:0014 and HANCESTRO:0005" then the value of self_reported_ethnicity_ontology_term_id MUST be "HANCESTRO:0005 || HANCESTRO:0014".
The following terms MUST NOT be used:
Otherwise, for all other organisms the
str value MUST be "na".
|
### sex_ontology_term_id
| Key |
sex_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be "unknown" if unavailable. If organism_ontology_term_id is "NCBITaxon:6239" for Caenorhabditis elegans, this MUST be "PATO:0000384" for male or "PATO:0001340" for hermaphrodite; otherwise, this MUST be one of:
|
### suspension_type
| Key |
suspension_type |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be "cell", "nucleus", or "na".
This MUST be the correct type for the corresponding assay:
If the assay does not appear in this table, the most appropriate value MUST be selected and the curation team informed during submission so that the assay can be added to the table.
|
### tissue_ontology_term_id
| Key |
tissue_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. If tissue_type is "cell culture" this MUST follow the requirements for cell_type_ontology_term_id.
If tissue_type is "tissue" or "organoid" then:
For all other organisms, this MUST be the most accurate descendant of UBERON:0001062 for anatomical entity.
|
### tissue_type
| Key |
tissue_type |
| Annotator |
Curator MUST annotate. |
| Value |
categorical with str categories. This MUST be "tissue", "organoid", or "cell culture".
|
When a dataset is uploaded, CELLxGENE Discover MUST automatically add the matching human-readable name for the corresponding ontology term to the `obs` dataframe. Curators MUST NOT annotate the following columns.
### assay
| Key |
assay |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be the human-readable name assigned to the value of assay_ontology_term_id.
|
### cell_type
| Key |
cell_type |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be "unknown" if the value of cell_type_ontology_term_id is "unknown"; otherwise, this MUST be the human-readable name assigned to the value of cell_type_ontology_term_id.
|
### development_stage
| Key |
development_stage |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be "unknown" if the value of development_stage_ontology_term_id is "unknown"; otherwise, this MUST be the human-readable name assigned to the value of development_stage_ontology_term_id.
|
### disease
| Key |
disease |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be one or more human-readable names for the terms in disease_ontology_term_id in the same order separated by the delimiter " || ".
For example, if the value of disease_ontology_term_id is "MONDO:0004604 || MONDO:0043004 || MONDO:0800349 || MONDO:1030008" then the value MUST be "Hodgkin's lymphoma, lymphocytic-histiocytic predominance || Weil's disease || atrial fibrillation, familial, 16 || mitral valve insufficiency".
|
When a dataset is uploaded, CELLxGENE Discover MUST annotate a unique observation identifier for each cell. Curators MUST NOT annotate the following column.
### observation_joinid
| Key |
observation_joinid |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
str
|
### self_reported_ethnicity
| Key |
self_reported_ethnicity |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be "na" if the value of self_reported_ethnicity_ontology_term_id is "na". This MUST be "unknown" if the value of self_reported_ethnicity_ontology_term_id is "unknown". Otherwise, this MUST be one or more human-readable names for the terms in self_reported_ethnicity_ontology_term_id in the same order separated by the delimiter " || ".
For example, if the value of self_reported_ethnicity_ontology_term_id is "HANCESTRO:0005 || HANCESTRO:0014" then the value of self_reported_ethnicity MUST be "European || Hispanic or Latin American".
|
### sex
| Key |
sex |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be "unknown" if the value of sex_ontology_term_id is "unknown"; otherwise, this MUST be the human-readable name assigned to the value of sex_ontology_term_id.
|
### tissue
| Key |
tissue |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be the human-readable name assigned to the value of tissue_ontology_term_id.
|
## `obsm` (Embeddings)
The value for each `str` key MUST be a `numpy.ndarray` of shape `(n_obs, m)`, where `n_obs` is the number of rows in `X` and `m >= 1`.
To display a dataset in CELLxGENE Explorer, Curators MUST annotate **one or more** embeddings of at least two-dimensions (e.g. tSNE, UMAP, PCA, spatial coordinates) as `numpy.ndarrays` in `obsm`.
### spatial
| Key |
spatial |
| Annotator |
Curator MUST annotate if uns['spatial']['is_single'] is True.
Curator MAY annotate if uns['spatial']['is_single'] is False.
Otherwise, this key MUST NOT be present. |
| Value |
numpy.ndarray with the following requirements
If assay_ontology_term_idis a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True, the array MUST be created from the corresponding pxl_row_in_fullres and pxl_col_in_fullres fields from tissue_positions_list.csv or tissue_positions.csv. See Space Ranger Spatial Outputs.
|
### X_{suffix}
| Key |
X_{suffix} with the following requirements:
- {suffix} MUST be at least one character in length.
- The first character of {suffix} MUST be a letter of the alphabet and the remaining characters MUST be alphanumeric characters,
'_', '-', or '.' (This is equivalent to the regular expression pattern "^[a-zA-Z][a-zA-Z0-9_.-]*$".)
- {suffix} MUST NOT be
"spatial".
{suffix} is presented as text to users in the Embedding Choice selector in CELLxGENE Explorer so it is STRONGLY RECOMMENDED that it be descriptive.
See also default_embedding in uns. |
| Annotator |
Curator MUST annotate if assay_ontology_term_id is neither a descendant of "EFO:0010961" for Visium Spatial Gene Expression nor "EFO:0030062" for Slide-seqV2.
Curator MAY annotate if assay_ontology_term_id is either a descendant of "EFO:0010961" for Visium Spatial Gene Expression or "EFO:0030062" for Slide-seqV2. |
| Value |
numpy.ndarray with the following requirements
|
## `obsp`
The size of the ndarray stored for a key in `obsp` MUST NOT be zero.
## `uns` (Dataset Metadata)
`uns` is a ordered dictionary with a `str` key. The data stored as a value for a key in `uns` MUST be `True`, `False`, `None`, or its size MUST NOT be zero.
Curators MUST annotate the following keys and values in `uns`:
### organism_ontology_term_id
| Key |
organism_ontology_term_id |
| Annotator |
Curator MUST annotate. |
| Value |
str. One of the following terms MUST be used:
|
### spatial
| Key |
spatial |
| Annotator |
Curator MUST annotate if assay_ontology_term_id is descendant of "EFO:0010961" for Visium Spatial Gene Expression or "EFO:0030062" for Slide-seqV2; otherwise, this key MUST NOT be present. |
| Value |
dict. The requirements for the key-value pairs are documented in the following sections:
- spatial['is_single']
- spatial[library_id]
- spatial[library_id]['images']
- spatial[library_id]['images']['fullres']
- spatial[library_id]['images']['hires']
- spatial[library_id]['scalefactors']
- spatial[library_id]['scalefactors']['spot_diameter_fullres']
- spatial[library_id]['scalefactors']['tissue_hires_scalef']
Additional key-value pairs MUST NOT be present.
|
#### is_single
| Key |
is_single |
| Annotator |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression or "EFO:0030062" for Slide-seqV2; otherwise, this key MUST NOT be present. |
| Value |
bool. This MUST be True:
- if
assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and the dataset represents one Space Ranger output for a single tissue section
- if
assay_ontology_term_id is "EFO:0030062" for Slide-seqV2 and the dataset represents the output for a single array on a puck
Otherwise, this MUST be False.
|
#### spatial[_library_id_]
| Key |
Identifier for the Visium library |
| Annotation |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
dict. There MUST be only one library_id. |
#### spatial[_library_id_]['images']
| Key |
images |
| Annotation |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
dict
|
#### spatial[_library_id_]['images']['fullres']
| Key |
fullres |
| Annotation |
Curator MAY annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
The full resolution image MUST be converted to anumpy.ndarray with the following requirements:
- The length of
numpy.ndarray.shape MUST be 3
- The
numpy.ndarray.dtype MUST be numpy.uint8
- The
numpy.ndarray.shape[2] MUST be either 3 (RGB color model for example) or 4 (RGBA color model for example)
|
#### spatial[_library_id_]['images']['hires']
| Key |
hires |
| Annotation |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
tissue_hires_image.png MUST be converted to anumpy.ndarray with the following requirements:
- The length of
numpy.ndarray.shape MUST be 3
- The
numpy.ndarray.dtype MUST be numpy.uint8
- If
assay_ontology_term_id is "EFO:0022860" for Visium CytAssist Spatial Gene Expression, 11mm, the largest dimension in numpy.ndarray.shape[:2] MUST be 4000pixels; otherwise, the largest dimension in numpy.ndarray.shape[:2] MUST be 2000pixels. See Space Ranger Spatial Outputs
- The
numpy.ndarray.shape[2] MUST be either 3 (RGB color model for example) for 4 (RGBA color model for example)
|
#### spatial[_library_id_]['scalefactors']
| Key |
scalefactors |
| Annotation |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
dict
|
#### spatial[_library_id_]['scalefactors']['spot_diameter_fullres']
| Key |
spot_diameter_fullres |
| Annotation |
Curator MUST annotate if assay_ontology_term_id is a descendant "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
float. This must be the value of the spot_diameter_fullres field from scalefactors_json.json. See Space Ranger Spatial Outputs. |
#### spatial[_library_id_]['scalefactors']['tissue_hires_scalef']
| Key |
tissue_hires_scalef |
| Annotation |
Curator MUST annotate if assay_ontology_term_id is a descendant of "EFO:0010961" for Visium Spatial Gene Expression and uns['spatial']['is_single'] is True; otherwise, this key MUST NOT be present. |
| Value |
float. This must be the value of the tissue_hires_scalef field from scalefactors_json.json. See Space Ranger Spatial Outputs.
|
### title
| Key |
title |
| Annotator |
Curator MUST annotate. |
| Value |
str. This text describes and differentiates the dataset from other datasets in the same collection. It is displayed on a page in CELLxGENE Discover that also has the collection name. To illustrate, the first dataset name in the Cells of the adult human heart collection is "All — Cells of the adult human heart".
It is STRONGLY RECOMMENDED that each dataset title in a collection is unique and does not depend on other metadata such as a different assay to disambiguate it from other datasets in the collection.
|
Curators MAY also annotate the following optional keys and values in `uns`. If the key is present, then its value MUST NOT be empty.
### batch_condition
| Key |
batch_condition |
| Annotator |
Curator MAY annotate. |
| Value |
list[str]. str values MUST refer to cell metadata keys in obs. Together, these keys define the batches that a normalization or integration algorithm should be aware of. For example if "patient" and "seqBatch" are keys of vectors of cell metadata, either ["patient"], ["seqBatch"], or ["patient", "seqBatch"] are valid values.
|
### {column}_colors
| Key |
{column}_colors where {column} MUST be the name of a category data type column in obs that is annotated by the data submitter or curator. The following columns that are annotated by CELLxGENE Discover MUST NOT be specified as {column}:
- assay
- cell_type
- development_stage
- disease
- self_reported_ethnicity
- sex
- tissue
Instead annotate {column}_ontology_term_id_colors for these columns such as assay_ontology_term_id.
|
| Annotator |
Curator MAY annotate. |
| Value |
numpy.ndarray. This MUST be a 1-D array of shape (, c), where c is greater than or equal to the number of categories in the {column} as calculated by:
anndata.obs.{column}.cat.categories.size
The color code at the Nth position in the ndarray corresponds to the Nth category of anndata.obs.{column}.cat.categories.
For example, if cell_type_ontology_term_id includes two categories:
anndata.obs.cell_type_ontology_term_id.cat.categories.values
array(['CL:0000057', 'CL:0000115'],
dtype='object')
then cell-type_ontology_term_id_colors MUST contain two or more colors such as:
['aqua' 'blueviolet']
where 'aqua' is the color assigned to 'CL:0000057' and 'blueviolet' is the color assigned to 'CL:0000115'.
All elements in the ndarray MUST use the same color model, limited to:
| Color Model |
Element Format |
|
Named Colors
|
str. MUST be a case-insensitive CSS4 color name with no spaces such as "aliceblue"
|
|
Hex Triplet
|
str. MUST start with "#" immediately followed by six case-insensitive hexadecimal characters as in "#08c0ff" |
|
### default_embedding
| Key |
default_embedding |
| Annotator |
Curator MAY annotate. |
| Value |
str. The value MUST match a key to an embedding in obsm for the embedding to display by default in CELLxGENE Explorer.
|
### X_approximate_distribution
| Key |
X_approximate_distribution |
| Annotator |
Curator MAY annotate. |
| Value |
str. CELLxGENE Discover runs a heuristic to detect the approximate distribution of the data in X so that it can accurately calculate statistical properties of the data. This field enables the curator to override this heuristic and specify the data distribution explicitly. The value MUST be "count" (for data whose distributions are best approximated by counting distributions like Poisson, Binomial, or Negative Binomial) or "normal" (for data whose distributions are best approximated by the Gaussian distribution.)
|
Curators MUST NOT annotate the following keys and values in `uns`.
When a dataset is uploaded, CELLxGENE Discover MUST automatically add the `citation` key and set its value.
### citation
| Key |
citation |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
str. Its format MUST use the following template:
| Citation Element |
Value |
"Publication: " |
Publication DOI url for the collection
This element MUST only be present if a Publication DOI is defined for the collection; otherwise, it MUST NOT be present. |
"Dataset Version: " |
Permanent url to this version of the dataset |
" curated and distributed by CZ CELLxGENE Discover in Collection: " |
Permanent url to the collection |
A citation for a H5AD dataset with a Publication DOI:
"Publication: https://doi.org/10.1126/science.abl4896 Dataset Version: https://datasets.cellxgene.cziscience.com/dbd8b789-3efa-4a63-9243-90cff64f2045.h5ad curated and distributed by CZ CELLxGENE Discover in Collection: https://cellxgene.cziscience.com/collections/e5f58829-1a66-40b5-a624-9046778e74f5"
A citation for a RDS dataset without a Publication DOI:
"Dataset Version: https://datasets.cellxgene.cziscience.com/08ea16dc-3f4e-4c84-8692-74d70be22d12.rds curated and distributed by CZ CELLxGENE Discover in Collection: https://cellxgene.cziscience.com/collections/10bf5c50-8d85-4c5f-94b4-22c1363d9f31"
|
When a dataset is uploaded, CELLxGENE Discover MUST automatically add the `organism` key and set its value to the matching human-readable name for the corresponding ontology term.
### organism
| Key |
organism |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
categorical with str categories. This MUST be the human-readable name assigned to the value of organism_ontology_term_id.
|
When a dataset is uploaded, CELLxGENE Discover MUST automatically add the `schema_reference` key and set its value to the permanent URL of this document.
### schema_reference
| Key |
schema_reference |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
This MUST be "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/6.0.0/schema.md".
|
---
When a dataset is uploaded, CELLxGENE Discover MUST automatically add the `schema_version` key and its value to `uns`.
### schema_version
| Key |
schema_version |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
This MUST be "6.0.0".
|
## `var` and `raw.var` (Gene Metadata)
`var` and `raw.var` are both of type [`pandas.DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).
Curators MUST annotate the following columns in the `var` dataframe and if present, the `raw.var` dataframe.
### index of pandas.DataFrame
| Key |
index of pandas.DataFrame |
| Annotator |
Curator MUST annotate. |
| Value |
str. The index of the pandas.DataFrame MUST contain unique identifiers for features. If present, the index of raw.var MUST be identical to the index of var.
If the feature is a RNA Spike-In Control Mix then the value MUST be an ERCC Spike-In identifier (e.g. "ERCC-0003").
If the feature is a gene then the value MUST be the gene_id attribute from the corresponding gene reference documented in Required Gene Annotations for either the organism_ontology_term_id or NCBITaxon:2697049 for SARS-CoV-2.
Version numbers MUST be removed from the gene_id if it is prefixed with "ENS" for Ensembl stable identifier. See I have an Ensembl ID, what can I tell about it from the ID? For example, if the gene_id is “ENSG00000186092.7”, then the value MUST be “ENSG00000186092”.
|
Curators MUST annotate the following column only in the `var` dataframe. This column MUST NOT be present in `raw.var`:
### feature_is_filtered
| Key |
feature_is_filtered |
| Annotator |
Curator MUST annotate. |
| Value |
bool. When a raw matrix is not present, the value for all features MUST be False.
When both a raw and normalized matrix are present, this MUST be True if the feature was filtered out in the normalized matrix (X) but is present in the raw matrix (raw.X). The value for all cells of the given feature in the normalized matrix MUST be 0. If a feature contains all zeroes in the normalized matrix, then either the corresponding feature in the raw matrix MUST be all zeroes or the value MUST be True.
|
|
Curators MUST NOT annotate the following columns in the `var` dataframe and if present, the `raw.var` dataframe.
When a dataset is uploaded, CELLxGENE Discover MUST automatically add the matching human-readable name for the corresponding feature biotype, identifier, and the NCBITaxon term for the reference organism to the `var` and `raw.var` dataframes. In addition, it MUST
add the feature length and type.
### feature_biotype
| Key |
feature_biotype |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
This MUST be "gene" or "spike-in".
|
### feature_length
### feature_name
| Key |
feature_name |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
str. If the feature_biotype is "spike-in" then this MUST be the ERCC Spike-In identifier appended with " (spike-in control)".
If the feature_biotype is "gene" and a gene_name attribute is assigned to the var.index feature identifier in its corresponding gene reference, this MUST be the value of the gene_name. If a gene_name attribute is not assigned, then this MUST default to the var.index feature identifier.
|
### feature_reference
| Key |
feature_reference |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
str. This MUST be the reference organism for a feature:
|
### feature_type
| Key |
feature_type |
| Annotator |
CELLxGENE Discover MUST annotate. |
| Value |
str. If the feature_biotype is "gene" then this MUST be the gene type assigned to the feature identifier in var.index. If the feature_biotype is "spike-in" then this MUST be "synthetic".
See GENCODE and Ensembl references.
|
## `varm`
The size of the ndarray stored for a key in `varm` MUST NOT be zero.
## `varp`
The size of the ndarray stored for a key in `varp` MUST NOT be zero.
---
## scATAC-seq assets
### Requirements
A Dataset MUST meet all of the following requirements to be eligible for scATAC-seq assets:
* assay_ontology_term_id values MUST be either all paired assays or unpaired assays
* is_primary_data values MUST be all True
* organism_ontology_term_id value MUST be either "NCBITaxon:9606" for Homo sapiens or "NCBITaxon:10090" for Mus musculus or one of its descendants. The value determines the required Chromosome Table.
If the assay_ontology_term_id values are all paired assays then the Dataset MAY have a fragments file asset.
If the assay_ontology_term_id values are all unpaired assays then the Dataset MUST have a fragments file asset.
## scATAC-seq Asset: Submitted Fragment File
This MUST be a gzipped tab-separated values (TSV) file.
The curator MUST annotate the following header-less columns. Additional columns and header lines beginning with `#` MUST NOT be included. Each row MUST represent a unique fragment.
### first column
| Annotator |
Curator MUST annotate. |
| Value |
str. This MUST be the reference genome chromosome the fragment is located on.
If the value of organism_ontology_term_id in the associated Dataset is "NCBITaxon:9606" for Homo sapiens then the first column value MUST be a value from the Chromosome column in the Human Chromosome Table.
If the value of organism_ontology_term_id in the associated Dataset is "NCBITaxon:10090" for Mus musculus then the first column value MUST be a value from the Chromosome column in the Mouse Chromosome Table.
|
### second column
| Annotator |
Curator MUST annotate. |
| Value |
int. This MUST be the 0-based start coordinate of the fragment.
|
### third column
| Annotator |
Curator MUST annotate. |
| Value |
int. This MUST be the 0-based end coordinate of the fragment. The end position is exclusive, representing the position immediately following the fragment interval. The value MUST be greater than the start coordinate specified in the second column and less than or equal to the Length of the Chromosome specified in the first column, as specified in the appropriate Chromosome Table.
|
### fourth column
| Annotator |
Curator MUST annotate. |
| Value |
str. This MUST be an observation identifier from the obs index of the associated Dataset. Every obs index value of the associated Dataset MUST appear at least once in this column.
|
### fifth column
| Annotator |
Curator MUST annotate. |
| Value |
int. This MUST be the total number of read pairs associated with this fragment. The value MUST be 1 or greater.
|
## scATAC-seq Asset: Processed Fragments File
From every submitted fragments file asset, CELLxGENE Discover MUST generate {artifact_id}-fragments.tsv.gz, a tab-separated values (TSV) file position-sorted and compressed by bgzip.
## scATAC-seq Asset: Fragments File index
From every processed fragments file asset, CELLxGENE Discover MUST generate {artifact_id}-fragments.tsv.gz.tbi, a tabix index of the fragment intervals from the fragments file.
## Chromosome Tables
Chromosome Tables are determined by the reference assembly for the gene annotation versions pinned in this version of the schema. Only chromosomes or scaffolds that have at least one gene feature present are included.
### Human (GRCh38.p14)
| Chromosome |
Length |
| chr1 |
248956422 |
| chr2 |
242193529 |
| chr3 |
198295559 |
| chr4 |
190214555 |
| chr5 |
181538259 |
| chr6 |
170805979 |
| chr7 |
159345973 |
| chr8 |
145138636 |
| chr9 |
138394717 |
| chr10 |
133797422 |
| chr11 |
135086622 |
| chr12 |
133275309 |
| chr13 |
114364328 |
| chr14 |
107043718 |
| chr15 |
101991189 |
| chr16 |
90338345 |
| chr17 |
83257441 |
| chr18 |
80373285 |
| chr19 |
58617616 |
| chr20 |
64444167 |
| chr21 |
46709983 |
| chr22 |
50818468 |
| chrX |
156040895 |
| chrY |
57227415 |
| chrM |
16569 |
| GL000009.2 |
201709 |
| GL000194.1 |
191469 |
| GL000195.1 |
182896 |
| GL000205.2 |
185591 |
| GL000213.1 |
164239 |
| GL000216.2 |
176608 |
| GL000218.1 |
161147 |
| GL000219.1 |
179198 |
| GL000220.1 |
161802 |
| GL000225.1 |
211173 |
| KI270442.1 |
392061 |
| KI270711.1 |
42210 |
| KI270713.1 |
40745 |
| KI270721.1 |
100316 |
| KI270726.1 |
43739 |
| KI270727.1 |
448248 |
| KI270728.1 |
1872759 |
| KI270731.1 |
150754 |
| KI270733.1 |
179772 |
| KI270734.1 |
165050 |
| KI270744.1 |
168472 |
| KI270750.1 |
148850 |
### Mouse (GRCm39)
| Chromosome |
Length |
| chr1 |
195154279 |
| chr2 |
181755017 |
| chr3 |
159745316 |
| chr4 |
156860686 |
| chr5 |
151758149 |
| chr6 |
149588044 |
| chr7 |
144995196 |
| chr8 |
130127694 |
| chr9 |
124359700 |
| chr10 |
130530862 |
| chr11 |
121973369 |
| chr12 |
120092757 |
| chr13 |
120883175 |
| chr14 |
125139656 |
| chr15 |
104073951 |
| chr16 |
98008968 |
| chr17 |
95294699 |
| chr18 |
90720763 |
| chr19 |
61420004 |
| chrX |
169476592 |
| chrY |
91455967 |
| chrM |
16299 |
| GL456210.1 |
169725 |
| GL456211.1 |
241735 |
| GL456212.1 |
153618 |
| GL456219.1 |
175968 |
| GL456221.1 |
206961 |
| GL456239.1 |
40056 |
| GL456354.1 |
195993 |
| GL456372.1 |
28664 |
| GL456381.1 |
25871 |
| GL456385.1 |
35240 |
| JH584295.1 |
1976 |
| JH584296.1 |
199368 |
| JH584297.1 |
205776 |
| JH584298.1 |
184189 |
| JH584299.1 |
953012 |
| JH584303.1 |
158099 |
| JH584304.1 |
114452 |
---
## Appendix A. Changelog
### schema v6.0.0
* General Requirements
* Added organism in obs to Reserved Names
* Added organism_ontology_term_id in obs to Reserved Names
* Required Ontologies
* Updated C. elegans Development Ontology (WBls) to release 2025-04-01 WS297
* Updated C. elegans Gross Anatomy Ontology (WBbt) to release 2025-03-26 WS297
* Updated Cell Ontology (CL) to release 2025-04-10
* Updated Drosophila Anatomy Ontology (FBbt) to release 2025-03-27
* Updated Drosophila Development Ontology (FBdv) to release 2025-03-26
* Updated Expermental Factor Ontology (EFO) to release 2025-05-15 EFO 3.78.0
* Updated Mondo Disease Ontology (MONDO) to release 2025-05-06
* Updated NCBI organismal classification (NCBITaxon) to release 2025-03-13
* Updated Phenotype And Trait Ontology (PATO) to release 2025-05-14
* Updated Uberon multi-species anatomy ontology (UBERON) to release 2025-05-28
* X (Matrix Layers)
* Updated requirements in table to not allow duplicate `obs`by raw counts
* obs (Cell metadata)
* Updated disease to address multiple labels
* Updated disease_ontology_term_id to allow multiple terms
* Updated self_reported_ethnicity delimiter requirements
* Updated self_reported_ethnicity_ontology_term_id delimiter requirements
* Deprecated organism
* Deprecated organism_ontology_term_id
* uns (Dataset Metadata)
* Updated `{column}_colors` requirements to not reference organism
* Added organism
* Added organism_ontology_term_id
* Updated `schema_reference` to "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/6.0.0/schema.md"
* Updated `schema_version` to "6.0.0"
* var and raw.var (Gene Metadata)
* Updated index of pandas.DataFrame requirements to remove version suffixes from Ensembl stable identifiers
* Clarified feature_is_filtered requirements
* Updated feature_name requirements to use feature identifier if a gene_name is unavailable
* scATAC-seq assets
* Updated requirements for *one organism per dataset*
### schema v5.3.0
* Schema Versioning
* **Patch versions** may be used to add organisms that do not require new metadata fields.
* General Requirements
* Required Ontologies
* Added C. elegans Development Ontology (WBls) release 2025-01-04 WS296
* Added C. elegans Gross Anatomy Ontology (WBbt) release 2025-01-02 WS296
* Updated Cell Ontology (CL) to the 2025-02-13 release
* Added Drosophila Anatomy Ontology (FBbt) release 2025-02-13
* Added Drosophila Development Ontology (FBdv) release 2025-02-12
* Updated Expermental Factor Ontology (EFO) to release 2025-02-17 EFO 3.75.0
* Updated Human Developmental Stages (HsapDv) to release 2025-01-23
* Updated Mondo Disease Ontology (MONDO) to release 2025-02-04
* Updated Mouse Developmental Stages (MmusDv) to release 2025-01-2
* Updated NCBI organismal classification (NCBITaxon) to release 2024-11-25
* Updated Phenotype And Trait Ontology (PATO) to release 2025-02-01
* Updated Uberon multi-species anatomy ontology (UBERON) to release 2025-01-15
* Added Zebrafish Anatomy Ontology (ZFA+ZFS) release 2025-01-28
* Required Gene Annotations
* Refactored table to include NCBI Taxon for supported organisms
* Added *Caenorhabditis elegans* WBcel235 (GCA_000002985.3) Ensembl 113
* Added *Callithrix jacchus* mCalJac1.pat.X (GCA_011100555.1) Ensembl 113
* Added *Danio rerio* GRCz11 (GCA_000002035.4) Ensembl 113
* Added *Drosophila melanogaster* BDGP6.46 (GCA_000001215.4) Ensembl 113
* Added *Gorilla gorilla gorilla* gorGor4 (GCA_000151905.3) Ensembl 113
* Added *Macaca fascicularis* Macaca_fascicularis_6.0 (GCA_011100615.1) Ensembl 113
* Added *Macaca mulatta* Mmul_10 (GCA_003339765.3) Ensembl 113
* Added *Microcebus murinus* Mmur_3.0 (GCA_000165445.3) Ensembl 113
* Added *Oryctolagus cuniculus* OryCun2.0 (GCA_000003625.1) Ensembl 113
* Added *Pan troglodytes* Pan_tro_3.0 (GCA_000001515.5) Ensembl 113
* Added *Rattus norvegicus* mRatBN7.2 (GCA_015227675.2) Ensembl 113
* Added *Sus scrofa* Sscrofa11.1 (GCA_000003025.6) Ensembl 113
* X (Matrix Layers)
* Updated _Visium Spatial Gene Expression_ table row to _Descendants of Visium Spatial Gene Expression_
* Added matrix requirements for _Visium CytAssist Spatial Gene Expression, 11mm_.
* Updated the STRONGLY RECOMMENDED requirement to a MUST. A matrix with 50% or more values that are zeros MUST be encoded as `scipy.sparse.csr_matrix`.
* obs (Cell metadata)
* Updated the requirements for `array_col`:
* MUST be annotated if the `assay_ontology_term_id` is a descendant of _Visium Spatial Gene Expression_
* Added ranges for _Visium CytAssist Spatial Gene Expression, 6.5mm_ and _Visium CytAssist Spatial Gene Expression, 11mm_
* Updated the requirements for `array_row`:
* MUST be annotated if the `assay_ontology_term_id` is a descendant of _Visium Spatial Gene Expression_
* Added ranges for _Visium CytAssist Spatial Gene Expression, 6.5mm_ and _Visium CytAssist Spatial Gene Expression, 11mm_
* Updated the requirements for `assay_ontology_term_id`:
* For _Visium Spatial Gene Expression_, only its descendants are allowed. All observations must contain the same value.
* For _scATAC-seq and its descendants_, added reference to scATAC-seq fragment asset requirements
* Added more recommended terms for assays
* Updated the requirements for `cell_type_ontology_term_id` to include descendants of _Visium Spatial Gene Expression_.
* Updated the ontology requirements for `cell_type_ontology_term_id` to include:
* WBbt for *Caenorhabditis elegans*
* ZFA for *Danio rerio*
* FBbt for *Drosophila melanogaster*
* Updated the ontology requirements for `development_stage_ontology_term_id` to include:
* WBls for *Caenorhabditis elegans*
* ZFS for *Danio rerio*
* FBdv for *Drosophila melanogaster*
* Updated `development_stage_ontology_term_id` to include descendants of Mus musclus
* Updated the requirements for `in_tissue` to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for `organism_ontology_term_id` to limit its values to a list of NCBITaxon terms
* Updated the requirements for `sex_ontology_term_id`:
* For *Caenorhabditis elegans*, values are limited to hermaphrodite, male, or `"unknown"`
* For all other organisms, values are limited to female, hermaphrodite, male, or `"unknown"`
* Updated the ontology requirements for `tissue_ontology_term_id` to include:
* WBbt for *Caenorhabditis elegans*
* ZFA for *Danio rerio*
* FBbt for *Drosophila melanogaster*
* obsm (Embeddings)
* Updated the requirements for `spatial` to include descendants of _Visium Spatial Gene Expression_ and to prohibit 'Not a Number' values.
* Updated the requirements for `X_{suffix}` to include descendants of _Visium Spatial Gene Expression_.
* uns (Dataset Metadata)
* Updated `schema_reference` to "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.3.0/schema.md"
* Updated `schema_version` to "5.3.0"
* Updated the requirements for `spatial` to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial['is_single'] to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial[library_id] to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial[library_id]['images'] to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial[library_id]['images']['fullres'] to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial[library_id]['images']['hires'] to include descendants of _Visium Spatial Gene Expression_. Added requirements for Visium CytAssist Spatial Gene Expression, 11mm.
* Updated the requirements for spatial[library_id]['scalefactors'] to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial[library_id]['scalefactors']['spot_diameter_fullres'] to include descendants of _Visium Spatial Gene Expression_.
* Updated the requirements for spatial[library_id]['scalefactors']['tissue_hires_scalef'] to include descendants of _Visium Spatial Gene Expression_.
* var and raw.var (Gene Metadata)
* Updated `feature_reference` to include MUST requirements for:
* *Caenorhabditis elegans*
* *Danio rerio*
* *Drosophila melanogaster*
* *Macaca fascicularis*
* *Microcebus murinus*
* *Oryctolagus cuniculus*
* *Rattus norvegicus*
* Updated `feature_reference` to include STRONGLY RECOMMENDED requirements and warning for current organisms with orthologous gene references:
* *Callithrix jacchus*
* *Gorilla gorilla gorilla*
* *Macaca mulatta*
* *Pan troglodytes*
* *Sus scrofa and its descendants*
* Added **scATAC-seq assets** section
### schema v5.2.0
* General Requirements
* Updated AnnData from version 0.8.0 to version 0.8.0 or greater
* Required Ontologies
* Updated CL to the 2024-08-16 release
* Updated EFO to the 2024-08-15 EFO 3.69.0 release
* Updated HsapDv to the 2024-05-28 release
* Updated MONDO to the 2024-08-06 release
* Updated MmusDv to the 2024-05-28 release
* Updated UBERON to the 2024-08-07 release
* obs (Cell metadata)
* Updated requirements for `development_stage_ontology_term_id` to require the most accurate descendant of _life cycle_.
* If organism_ontology_term_id is "NCBITaxon:9606" for Homo sapiens, this MUST be the most accurate descendant of `HsapDv:0000001` for _life cycle_
* If organism_ontology_term_id is "NCBITaxon:10090" for Mus musculus, this MUST be the most accurate descendant of `MmusDv:0000001` for _life cycle_
* Updated requirements for `suspension_type`
* Added mCT-seq
* Added MERFISH
* Added ScaleBio single cell RNA sequencing
* Added sci-RNA-seq3
* Removed CITE-seq and its descendants
* Removed smFISH and its descendants
* Removed snmC-seq
* Removed spatial proteomics and its descendants
* Replaced snmC-seq2 with methylation profiling by high throughput sequencing and its descendants
* uns (Dataset metadata)
* Updated `schema_reference` to "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.2.0/schema.md"
* Updated `schema_version` to "5.2.0"
* var and raw.var (Gene metadata)
* Updated the requirements for `feature_length`. All `feature_biotypes` are now included. The calculation of the value changed from the merged length of isoforms to the median of the lengths of isoforms.
* Added `feature_type`
### schema v5.1.0
* All references to "child" and "children" have been changed to "descendant" and "descendants" for accuracy.
* Required Ontologies
* Updated CL to the 2024-04-05 release
* Updated EFO to the 2024-04-15 EFO 3.65.0 release
* Updated MONDO to the 2024-05-08 release
* Updated UBERON to the 2024-03-22 release
* X (Matrix Layers)
* Added _Visium Spatial Gene Expression_ to the table of assays
* obs (Cell metadata)
* Added `array_col` for _Visium Spatial Gene Expression_ when uns['spatial']['is_single'] is True
* Added `array_row` for _Visium Spatial Gene Expression_ when uns['spatial']['is_single'] is True
* Updated the requirements for `assay_ontology_term_id` for _Visium Spatial Gene Expression_ and _Slide-seqV2_. All observations must contain the same value.
* Updated the requirements for `cell_type_ontology_term_id` for _Visium Spatial Gene Expression_ when uns['spatial']['is_single'] is True. The value must be `"unknown"` if the corresponding value of `in_tissue` is `0`.
* Added `in_tissue` for _Visium Spatial Gene Expression_ when uns['spatial']['is_single'] is True
* Updated the requirements for `is_primary_data` for _Visium Spatial Gene Expression_. The value must be Falsewhen uns['spatial']['is_single'] is False.
* Updated the requirements for `self_reported_ethnicity_ontology_term_id`. There must be no duplication of terms.
* obsm (Embeddings)
* Restored v3.1.0 requirement allowing only `numpy.ndarray` values with specific shapes due to Seurat conversion failures
* Added `spatial` for _Visium Spatial Gene Expression_ and _Slide-seqV2_
* Updated requirements for `X_{suffix}`. {suffix} MUST NOT be `"spatial"`.
* uns (Dataset metadata)
* Updated `{column}_colors` instructions
* Updated `schema_reference` to "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.1.0/schema.md"
* Updated `schema_version` to "5.1.0"
* Added `spatial` for _Visium Spatial Gene Expression_ and _Slide-seqV2_, including scale factors and underlay images for _Visium Spatial Gene Expression_.
### schema v5.0.0
* General Requirements
* Updated requirements to prohibit duplicate data submitter metadata field names in `obs` and `var`
* Required Ontologies
* Updated CL to the 2024-01-04 release
* Updated EFO to the 2024-01-15 EFO 3.62.0 release
* Updated MONDO to the 2024-01-03 release
* Updated UBERON to the 2024-01-18 release
* Required Gene Annotations
* Updated GENCODE (Human) to Human Reference GRCh38.p14 (GENCODE v44/Ensembl 110)
* Updated GENCODE (Mouse) to Mouse reference GRCm39 (GENCODE vM33/Ensembl 110)
* obs (Cell metadata)
* Updated the requirements for `assay_ontology_term_id` to not allow the parent terms `EFO:0002772` for _assay by molecule_ and `EFO:0010183` for _single cell library construction_. Their most accurate children are still valid.
* **Breaking change**. Updated the requirements for `cell_type` to annotate `"unknown"` as the label when the `cell_type_ontology_term_id` value is `"unknown"`.
* **Breaking change**. Updated the requirements for `cell_type_ontology_term_id` to replace `"CL:0000003"` for *native cell* with `"unknown"` to indicate that the cell type is unknown.
* Updated the requirements for `disease_ontology_term_id` to restrict MONDO terms to the most accurate child of `"MONDO:0000001"` for _disease_ or `"MONDO:0021178"` for _injury_ or preferably its most accurate child.
* obsm (Embeddings)
* Updated requirements for `X_{suffix}` to change the regular expression pattern from `"^[a-zA-Z][a-zA-Z0-9]*$"` to `"^[a-zA-Z][a-zA-Z0-9_.-]*$"`
* uns (Dataset metadata)
* Updated requirements. The data stored as a value for a key in `uns` MUST be `True`, `False`, `None`, or its size MUST NOT be zero.
* Updated schema_reference to "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.0.0/schema.md"
* Updated schema_version to "5.0.0"
### schema v4.0.0
* Required Ontologies
* Updated CL to the 2023-08-24 release
* Updated EFO to the 2023-08-15 EFO 3.57.0 release
* Updated HANCESTRO to the 3.0 release
* Updated MONDO to the 2023-08-02 release
* Updated UBERON to the 2023-09-05 release
* obs (Cell metadata)
* Updated the requirements for `cell_type_ontology_term_id`
* Added `index`
* Added `observation_joinid`
* Updated the requirements for `self_reported_ethnicity`
* Updated the requirements for `self_reported_ethnicity_ontology_term_id`
* Added `tissue_type`
* Updated the requirements for `tissue`
* Updated the requirements for `tissue_ontology_term_id`
* obsm (Embeddings)
* Prohibited ndarrays with a size of zero
* Updated requirements for `X_{suffix}`
* obsp
* Added section and prohibited ndarrays with a size of zero
* uns (Dataset metadata)
* Added `citation`
* Added `{column}_colors`
* Added `schema_reference`
* Updated the requirements for `schema_version` to be annotated by CELLxGENE Discover and not the curator
* var and raw.var (Gene metadata)
* Added `feature_length`
* varm
* Added section and prohibited ndarrays with a size of zero
* varp
* Added section and prohibited ndarrays with a size of zero
* X (Matrix Layers)
* Updated requirements for raw matrices
### schema v3.1.0
* Added section for Schema versioning
* Required Ontologies
* Updated CL to the 2023-07-20 release
* Updated EFO to the 2023-07-17 EFO 3.56.0 release
* Updated MONDO to the 2023-07-03 release
* Updated NCBITaxon to the 2023-06-20 release
* Updated PATO to the 2023-05-18 release
* Updated UBERON to the 2023-06-28 release
* obs (Cell metadata)
* `assay_ontology_term_id`
* Added Visium Spatial Gene Expression to recommended values
* Removed Smart-seq from recommended values
* `suspension_type`
* Added MARS-seq
* Added BD Rhapsody Whole Transcriptome Analysis
* Added BD Rhapsody Targeted mRNA
* Added inDrop
* Added STRT-seq
* Added TruDrop
* Added GEXSCOPE technology
* Added SPLiT-seq
* Changed spatial transcriptomics by high-throughput sequencing [EFO:0030005] and its children to spatial transcriptomics [EFO:0008994] and its children
* Updated Seq-Well [EFO:0008919] to Seq-Well [EFO:0008919] and its children
* uns (Dataset metadata)
* `schema_version`
* Must be annotated by CELLxGENE Discover and not the Curator.
### schema v3.0.0
* Updated AnnData version 0.7 to version 0.8.0
* All references to the "final" matrix has been replaced with "normalized" for clarity.
* General Requirements
* Reserved Names from previous schema versions that have since been deprecated MUST NOT be present.
* Updated *pinned* ontologies to require the most recent version
* obs (Cell metadata)
* Removed guidance in `assay_ontology_term_id` that allowed clarifying text enclosed in parentheses if there was not an exact match for an assay.
* Added `donor_id`
* Renamed `ethnicity_ontology_term_id` to `self_reported_ethnicity_ontology_term_id`. Added `"multiethnic"` value.
* Renamed `ethnicity` to `self_reported_ethnicity`. Added `"multiethnic"` value.
* Added `suspension_type`
* var and raw.var (Gene metadata)
* `feature_biotype` must be annotated by CELLxGENE Discover and not the Curator.
* uns (Dataset metadata)
* Updated `schema_version`
* Deprecated `X_normalization`
### schema v2.0.0
schema v2.0.0 substantially *remodeled* schema v1.1.0:
* "must", "should", and select other words have a defined, standard meaning.
* Curators are responsible for annotating ontology and gene identifiers. CELLxGENE Discover adds the assigned human-readable names for all identifiers.
* Documented and *pinned* the required versions of ontologies and gene annotations used in schema validation.
* General Requirements
* AnnData is now the canonical data format. The schema outline and descriptions are AnnData-centric.
* Metazoan multi-organism data is accepted by CELLxGene Discover. For data that is neither Human, Mouse, nor SARS-COV-2, features MUST be translated into orthologous genes from the Human and Mouse gene annotations.
* Policies for reserved names and redundant metadata are documented.
* [#45](https://github.com/chanzuckerberg/single-cell-curation/issues/45) Updated reference to new PII content
* X (matrix layers)
* Added guidance for sparse matrices
* Clarified matrix requirements by assay
* obs (cell metadata)
* Empty ontology fields are no longer permitted.
* Moved organism from uns to obs
* Clarified requirements and added detailed guidance for assays, tissue, and development stages
* Added ontology for mouse development stages
* Added ontology for sex
* Added `is_primary_data`
* var
* Replaced HGNC gene **symbols** as `var.index` with ENSEMBL or ERCC spike-in **identifiers**
* Added `feature_name`, `index`, and `feature_reference`
* Added `feature_is_filtered`
* Added requirements for `raw.var` which must be identical to `var`
* uns
* Added `batch_condition`
* Added `X_approximate_distribution`
* Replaced `layer_descriptions` with `X_normalization`
* Replaced `version` which included `corpora_schema_version` and `corpora_encoding_version` with `schema_version`
* Deprecated `tags` and `default_field` presentation metadata
* Removed obs_column_colors