Model Card Ontology
type: Ontology
version IRI: mcro.owl
imports: iao.owl
imports: core
imports: prov-o-20130430
imports: swo import final.owl
contributor: Degui Zhi
contributor: Kirk Roberts
contributor: Licong Cui
contributor: Xiaqian Jiang
creator: Tuan Amith and Cui Tao
description: An ontology representing the model card structure
license: https://creativecommons.org/licenses/by/4.0/
version info: 2023-03-07

has version
type: ObjectProperty

citation text
type: DatatypeProperty
sub property of: model card text
range: Literal
IAO_0000115: additional information abou the citation associated with the model

confidence interval value
type: DatatypeProperty
sub property of: model card text
range: decimal
IAO_0000115: value of the confidence interval attributed to the Performance Metric concept

contact text
type: DatatypeProperty
sub property of: model card text
range: Literal
IAO_0000115: Data pertaining to the contact information of the model's contact information. This is associated to the Owner Information class of the Model Card Ontology

custom_text
type: DatatypeProperty
sub property of: model card text
range: Literal
IAO_0000115: Additional textual information attributed to the licensing information.

documentation
type: DatatypeProperty
sub property of: model card text
range: Literal
IAO_0000115: Detailed textual information of the model card that is attributed to specific concepts of the Model Card Ontology. This contrasts with the "overview" data property as the "overview" has a summarized version of "documentation"

model card text
type: DatatypeProperty
range: Literal
IAO_0000115: Model Card Report Ontology specific data properties that provide attributes to its concepts.

name text
type: DatatypeProperty
sub property of: model card text
range: Literal
IAO_0000115: Name of the owner of the model. This is linked to the Owner Information concept of the Model Card Ontology

overview
type: DatatypeProperty
sub property of: model card text
type: FunctionalProperty
range: Literal
IAO_0000115: Summary textual information for the concepts of the Model Card Ontology

performance metric value
type: DatatypeProperty
sub property of: model card text
type: FunctionalProperty
range: decimal
IAO_0000115: general numeric value associated with the Performance Metric concepts

sensitive data
type: DatatypeProperty
sub property of: model card text
domain: Sensitive Data Information Section
range: Literal
IAO_0000115: "A description of any sensitive data that may be present in adataset. Be sure to note PII information such as names, addresses, phone numbers, etc. Preferably, such info should be scrubbed from a dataset if possible. Note that even non-identifying information, such as zip code, age, race, and gender, can be used to identify individuals when aggregated. Please describe any such fields here" Google LLC

slice
type: DatatypeProperty
sub property of: model card text
range: Literal
IAO_0000115: This is data property is specific to Perfomance Metric concepts. It attributes information about the data partition used to experiment on the model.

threshold value
type: DatatypeProperty
sub property of: model card text
range: decimal
IAO_0000115: This is a value pertaining to a threshold value that is attributed to the Performance Metric concept for the Model Card ontology.

Model Card Report
type: Class
subclass of: IAO_0000088
subclass of: Restriction
  on property: IAO_0000136
  some values from: IAO_0000064
subclass of: Restriction
  on property: BFO_0000051
  all values from: Consideration Information Section
subclass of: Restriction
  on property: BFO_0000051
  all values from: Model Detail Section
subclass of: Restriction
  on property: BFO_0000051
  all values from: Model Parameter Section
subclass of: Restriction
  on property: BFO_0000051
  all values from: Quantative Analysis Section
IAO_0000115: The Model Card (MC) is the document designed for transparent reporting of AI model provenance, usage, and ethics-informed evaluation. A report that details characteristics of machine learning model to supplement its release to the public.
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Accuracy Information Section
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: Section content in this section should describe information in "how close or far off a given set of measurements (observations or readings) are to their true value"
IAO_0000119: https://en.wikipedia.org/wiki/Accuracy_and_precision

Area Under the Curve Information Section
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: The content describes information pertaining to the area under the ROC curve.
alternative label: AUC

Citation Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000051
  some values from: IAO_0000301
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: information relating to the citation for the targetted model. This is part of the model details component of the model card report

Consideration Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000051
  some values from: Ethical Consideration Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Limitation Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Trade-off Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Use Case Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: User Information Section
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: This section pertains to "Factors" of the model card report by Mitchell et al., 2019. The considerations section includes qualitative information about your model, including some analysis of its risks and limitations. As such, this section usually requires careful consideration, and conversations with many relevant stakeholders, including other model developers, dataset producers, and downstream users likely to interact with your model, or be affected by its outputs.
alternative label: Factors

Dataset Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000050
  some values from: Model Parameter Section
subclass of: Restriction
  on property: BFO_0000050
  some values from: Quantative Analysis Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: IAO_0000622
subclass of: Restriction
  on property: BFO_0000051
  some values from: Graphic Collection Section
subclass of: Restriction
  on property: IAO_0000136
  some values from: IAO_0000100
IAO_0000115: Information about the data that was used to develop the model

Ethical Consideration Section
type: Class
subclass of: Risk Information Section
IAO_0000115: Content information that is a type of risk information that is specific to ethical consideration if using the model.

Evaluation Data Information Section
type: Class
subclass of: Dataset Information Section
IAO_0000115: The section is devoted in describing the evaluation dataset. "Evaluation datasets should include datasets that are publicly available for third-party use. These could be existing datasets or new ones provided alongside the model card analyses to enable further benchmarking... the evaluation datasets should not only be representative of the model’s typical use cases but also anticipated test scenarios and challenging cases." Details on the dataset(s) used for the quantitative analyses in the card. - Datasets - Motivation - Preprocessing" (Mitchell et al., 2019)
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

F1-Score Information Section
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: The F1 score is the harmonic mean of the precision and recall. This section describes information relating to the F1 score revealed by the performance of the machine learning model.
IAO_0000119: https://en.wikipedia.org/wiki/F-score

False Discovery Rate Information Section
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: The section describes information about the false discovery rate of the machine learning model. "The false discovery rate (FDR) is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons. FDR-controlling procedures are designed to control the FDR, which is the expected proportion of "discoveries" (rejected null hypotheses) that are false (incorrect rejections of the null)". quivalently, the FDR is the expected ratio of the number of false positive classifications (false discoveries) to the total number of positive classifications (rejections of the null). The total number of rejections of the null include both the number of false positives (FP) and true positives (TP). Simply put, FDR = FP / (FP + TP). FDR-controlling procedures provide less stringent control of Type I errors compared to familywise error rate (FWER) controlling procedures (such as the Bonferroni correction), which control the probability of at least one Type I error. Thus, FDR-controlling procedures have greater power, at the cost of increased numbers of Type I errors."
IAO_0000119: https://en.wikipedia.org/wiki/False_discovery_rate

False Ommission Rate Section Information
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: This section describes information relating to false ommision rate of the performance of the machine learning model. FOR is the probability that the true value is positive.
alternative label: FOR

False Positive Rate Information Section
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: This content describes the information pertaining to the model's false postive rate. "A false positive error, or false positive, is a result that indicates a given condition exists when it does not. For example, a pregnancy test which indicates a woman is pregnant when she is not, or the conviction of an innocent person. A false positive error is a type I error where the test is checking a single condition, and wrongly gives an affirmative (positive) decision."
IAO_0000119: https://en.wikipedia.org/wiki/False_positives_and_false_negatives#False_positive_error

Format Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: IAO_0000136
  some values from: IAO_0000098
IAO_0000115: This content summarizes the input and output data format for the model.

Graphic Collection Section
type: Class
subclass of: IAO_0000314
subclass of: Collection
subclass of: Restriction
  on property: BFO_0000050
  some values from: IAO_0000622
IAO_0000115: A section describing the set of graphics accompanying the model card's dataset

Input Format Information Section
type: Class
subclass of: Format Information Section
IAO_0000115: This secton content serves to describe the inputed data's format for the machine learning model.

License Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: IAO_0000136
  some values from: SWO_0000002
subclass of: Restriction
  on property: custom_text
  some values from: string
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: Section for information about the licensing for the machine learning model.

Limitation Information Section
type: Class
subclass of: IAO_0000314
IAO_0000115: Content section relating to the technical limits of the machine learning model (e.g. trained on ungeneralized data).

Mitigation Strategy Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000050
  some values from: Risk Information Section
IAO_0000115: Content section of the model card that addresses the risk or ethical consideration of the model.

Model Architecture Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: IAO_0000136
  some values from: IAO_0000064
subclass of: Restriction
  on property: IAO_0000136
  some values from: SWO_0000001
IAO_0000115: Content section summarizing the architecture of the model (software used, algortihm, etc.)

Model Card Graphic
type: Class
subclass of: IAO_0000038
IAO_0000115: a graph content specifically for a model card report

Model Detail Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000051
  some values from: Citation Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: License Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Model Architecture Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Owner Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Reference Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Version Information Section
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: Basic information about the model that includes licensing information, owner information, the architecture of the model (algorthim employed), references (cited papers), and versioning information.
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Model Parameter Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000051
  some values from: Dataset Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Input Format Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Model Architecture Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Output Format Information Section
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: Section summarizing the content relating to the parameters for the development of the model.

Out of Scope Use Case Section Information
type: Class
subclass of: Use Case Information Section
IAO_0000112: "for use on black-and-white images only; please consider our research group’s full-color-image classifier for color images.”
IAO_0000112: "not for use on text examples shorter than 100 tokens"
IAO_0000115: "Here, the model card should highlight technology that the model might easily be confused with, or related contexts that users could try to apply the model to. This section may provide an opportunity to recommend a related or similar model that was designed to better meet that particular need, where possible. This section is inspired by warning labels on food and toys, and similar disclaimers presented in electronic datasheets." Content that is a type of use case information but specific to use cases that are out of scope of the intended use case.
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Output Format Information Section
type: Class
subclass of: Format Information Section
IAO_0000115: This secton content serves to describe the outputted data's format for the machine learning model.

Owner Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: IAO_0000136
  some values from: Class
    union of: Organization
    union of: Person
subclass of: Restriction
  on property: contact text
  some values from: Literal
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: name text
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: Section for the information about the individuals involved in the development of the machine learning model. This includes contact information, the names of the individuals, etc.

Percision-Recall Curve Information Section
type: Class
subclass of: Performance Metric Information Section
IAO_0000115: This section is for the description of the precision and recall information of the machine learning model's performance.

Performance Metric Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: confidence interval value
  some values from: decimal
subclass of: Restriction
  on property: performance metric value
  some values from: decimal
subclass of: Restriction
  on property: slice
  some values from: Literal
subclass of: Restriction
  on property: threshold value
  some values from: decimal
IAO_0000115: Section content here summarizing the information about the metrics (precision, recall, f1 score, etc.) that measured the performance of the model. One can report the values and portion of the data (slice).

Personally Identifiable Data Information Section
type: Class
subclass of: Sensitive Data Information Section
IAO_0000115: personally identifiable information as "any information about an individual maintained by an agency, including (1) any information that can be used to distinguish or trace an individual's identity, such as name, social security number, date and place of birth, mother's maiden name, or biometric records; and (2) any other information that is linked or linkable to an individual, such as medical, educational, financial, and employment information." The section should specificially describe information about the personally identifiable datasets.
IAO_0000119: https://en.wikipedia.org/wiki/Personal_data
IAO_0000119: https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-122.pdf

Primary Intended Use Case Information Section
type: Class
subclass of: Use Case Information Section
IAO_0000115: "This section details whether the model was developed with general or specific tasks in mind (e.g., plant recognition worldwide or in the Pacific Northwest). The use cases may be as broadly or narrowly defined as the developers intend. For example, if the model was built simply to label images, then this task should be indicated as the primary intended use case."
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Primary Intended User Information Section
type: Class
subclass of: User Information Section
IAO_0000115: This section describes the primary intended user for the machine learning model. "For example, was the model developed for entertainment purposes, for hobbyists, or enterprise solutions? This helps users gain insight into how robust the model may be to different kinds of inputs."
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Quantative Analysis Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: BFO_0000051
  some values from: IAO_0000622
subclass of: Restriction
  on property: BFO_0000051
  some values from: Dataset Information Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Graphic Collection Section
subclass of: Restriction
  on property: BFO_0000051
  some values from: Performance Metric Information Section
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: content overview of the quantatitve analysis of the model, specifically the performance of the model
editorial note: need to connect evaluation data information and training data information
editorial note: should their be a specified graphic collection for Quantative Analysis?
preferred label: Quantative Analysis

Reference Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: wasDerivedFrom
  some values from: IAO_0000310
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: information for any references related to the machine learning model (research papers, website, etc.)

Risk Information Section
type: Class
subclass of: IAO_0000314
IAO_0000115: Content information relating to potential uncertainity about utilizing the model

Sensitive Data Information Section
type: Class
subclass of: Dataset Information Section
subclass of: Restriction
  on property: sensitive data
  some values from: Literal
IAO_0000115: Information relating to sensitive datasets that contain info like personal information that was used to develop the machine learning model

Synthetic Evaluation Data Information Section
type: Class
subclass of: Evaluation Data Information Section
IAO_0000115: The section is devoted in descrbing the synthetic evaluation datasets used for a machine learning model. Synthetic evaluation datasets are a type of evaluation dataset. "It is often difficult to find datasets that represent populations outside of the initial domain used in training. In some of these situations, synthetically generated datasets may provide representation for use cases that would otherwise go unevaluated" (Mitchell et al., 2019). This section should described the aforementioned.
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Trade-off Information Section
type: Class
subclass of: IAO_0000314
IAO_0000115: Content involving known trade-offs for the use of the model. Trade offs might include performance or technical or ethical. "This section should list additional concerns that were not covered in the previous sections. For example, did the results suggest any further testing? Were there any relevant groups that were not represented in the evaluation dataset? Are there additional recommendations for model use? What are the ideal characteristics of an evaluation dataset for this model?"
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596
alternative label: Caveats
alternative label: Recommendations

Training Data Information Section
type: Class
subclass of: Dataset Information Section
IAO_0000115: The section is for information about the training data used for evaluation of a machine learning model.
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596

Use Case Information Section
type: Class
subclass of: IAO_0000314
IAO_0000115: The content here involves information about intential uses of the model. "This section should allow readers to quickly grasp what the model should and should not be used for, and why it was created. It can also help frame the statistical analysis presented in the rest of the card, including a short description of the user(s), use-case(s), and context(s) for which the model was originally developed."
IAO_0000119: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596
alternative label: Intended Use

User Information Section
type: Class
subclass of: IAO_0000314
IAO_0000115: Content information pertaining to intended users of the model. This includes individuals that may be impacted as an outcome of the use of the model.
comment: target user for the model
alternative label: Intended User Information

Version Information Section
type: Class
subclass of: IAO_0000314
subclass of: Restriction
  on property: IAO_0000136
  some values from: IAO_0000129
subclass of: Restriction
  on property: documentation
  some values from: Literal
subclass of: Restriction
  on property: overview
  some values from: Literal
IAO_0000115: This content section describes any versioning information for the machine learning model.

member
deprecated: true