An ontology representing the model card structure
additional information abou the citation associated with the model
value of the confidence interval attributed to the Performance Metric concept
Data pertaining to the contact information of the model's contact information. This is associated to the Owner Information class of the Model Card Ontology
Additional textual information attributed to the licensing information.
Detailed textual information of the model card that is attributed to specific concepts of the Model Card Ontology. This contrasts with the "overview" data property as the "overview" has a summarized version of "documentation"
Model Card Report Ontology specific data properties that provide attributes to its concepts.
Name of the owner of the model. This is linked to the Owner Information concept of the Model Card Ontology
Summary textual information for the concepts of the Model Card Ontology
general numeric value associated with the Performance Metric concepts
"A description of any sensitive data that may be present in adataset. Be sure to note PII information such as names, addresses, phone numbers, etc. Preferably, such info should be scrubbed from a dataset if possible. Note that even non-identifying information, such as zip code, age, race, and gender, can be used to identify individuals when aggregated. Please describe any such fields here" Google LLC
This is data property is specific to Perfomance Metric concepts. It attributes information about the data partition used to experiment on the model.
This is a value pertaining to a threshold value that is attributed to the Performance Metric concept for the Model Card ontology.
The Model Card (MC) is the document designed for transparent reporting of AI model provenance, usage, and ethics-informed evaluation. A report that details characteristics of machine learning model to supplement its release to the public.
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru. 2019. Model Cards for Model Reporting. In FAT* ’19: Conference on Fairness, Accountability, and Transparency, January 29–31, 2019, Atlanta, GA, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3287560. 3287596
Section content in this section should describe information in "how close or far off a given set of measurements (observations or readings) are to their true value"
The content describes information pertaining to the area under the ROC curve.
Area Under the Curve Information Section
information relating to the citation for the targetted model. This is part of the model details component of the model card report
This section pertains to "Factors" of the model card report by Mitchell et al., 2019. The considerations section includes qualitative information about your model, including some analysis of its risks and limitations. As such, this section usually requires careful consideration, and conversations with many relevant stakeholders, including other model developers, dataset producers, and downstream users likely to interact with your model, or be affected by its outputs.
Information about the data that was used to develop the model
Content information that is a type of risk information that is specific to ethical consideration if using the model.
The section is devoted in describing the evaluation dataset. "Evaluation datasets should include datasets that are publicly available for third-party use. These could be existing datasets or new ones provided alongside the model card analyses to enable further benchmarking... the evaluation datasets should not only be representative of the model’s typical use cases but also anticipated test scenarios and challenging cases." Details on the dataset(s) used for the quantitative analyses in the card. - Datasets - Motivation - Preprocessing" (Mitchell et al., 2019)
The F1 score is the harmonic mean of the precision and recall. This section describes information relating to the F1 score revealed by the performance of the machine learning model.
The section describes information about the false discovery rate of the machine learning model. "The false discovery rate (FDR) is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons. FDR-controlling procedures are designed to control the FDR, which is the expected proportion of "discoveries" (rejected null hypotheses) that are false (incorrect rejections of the null)". quivalently, the FDR is the expected ratio of the number of false positive classifications (false discoveries) to the total number of positive classifications (rejections of the null). The total number of rejections of the null include both the number of false positives (FP) and true positives (TP). Simply put, FDR = FP / (FP + TP). FDR-controlling procedures provide less stringent control of Type I errors compared to familywise error rate (FWER) controlling procedures (such as the Bonferroni correction), which control the probability of at least one Type I error. Thus, FDR-controlling procedures have greater power, at the cost of increased numbers of Type I errors."
False Discovery Rate Information Section
This section describes information relating to false ommision rate of the performance of the machine learning model. FOR is the probability that the true value is positive.
False Ommission Rate Section Information
This content describes the information pertaining to the model's false postive rate. "A false positive error, or false positive, is a result that indicates a given condition exists when it does not. For example, a pregnancy test which indicates a woman is pregnant when she is not, or the conviction of an innocent person. A false positive error is a type I error where the test is checking a single condition, and wrongly gives an affirmative (positive) decision."
This content summarizes the input and output data format for the model.
A section describing the set of graphics accompanying the model card's dataset
This secton content serves to describe the inputed data's format for the machine learning model.
Section for information about the licensing for the machine learning model.
Content section relating to the technical limits of the machine learning model (e.g. trained on ungeneralized data).
Content section of the model card that addresses the risk or ethical consideration of the model.
Content section summarizing the architecture of the model (software used, algortihm, etc.)
a graph content specifically for a model card report
Basic information about the model that includes licensing information, owner information, the architecture of the model (algorthim employed), references (cited papers), and versioning information.
Section summarizing the content relating to the parameters for the development of the model.
"for use on black-and-white images only; please consider our research group’s full-color-image classifier for color images.”
"not for use on text examples shorter than 100 tokens"
"Here, the model card should highlight technology that the model might easily be confused with, or related contexts that users could try to apply the model to. This section may provide an opportunity to recommend a related or similar model that was designed to better meet that particular need, where possible. This section is inspired by warning labels on food and toys, and similar disclaimers presented in electronic datasheets." Content that is a type of use case information but specific to use cases that are out of scope of the intended use case.
Out of Scope Use Case Section Information
This secton content serves to describe the outputted data's format for the machine learning model.
Section for the information about the individuals involved in the development of the machine learning model. This includes contact information, the names of the individuals, etc.
This section is for the description of the precision and recall information of the machine learning model's performance.
Percision-Recall Curve Information Section
Section content here summarizing the information about the metrics (precision, recall, f1 score, etc.) that measured the performance of the model. One can report the values and portion of the data (slice).
personally identifiable information as "any information about an individual maintained by an agency, including (1) any information that can be used to distinguish or trace an individual's identity, such as name, social security number, date and place of birth, mother's maiden name, or biometric records; and (2) any other information that is linked or linkable to an individual, such as medical, educational, financial, and employment information." The section should specificially describe information about the personally identifiable datasets.
Personally Identifiable Data Information Section
"This section details whether the model was developed with general or specific tasks in mind (e.g., plant recognition worldwide or in the Pacific Northwest). The use cases may be as broadly or narrowly defined as the developers intend. For example, if the model was built simply to label images, then this task should be indicated as the primary intended use case."
Primary Intended Use Case Information Section
This section describes the primary intended user for the machine learning model. "For example, was the model developed for entertainment purposes, for hobbyists, or enterprise solutions? This helps users gain insight into how robust the model may be to different kinds of inputs."
Primary Intended User Information Section
content overview of the quantatitve analysis of the model, specifically the performance of the model
need to connect evaluation data information and training data information
should their be a specified graphic collection for Quantative Analysis?
information for any references related to the machine learning model (research papers, website, etc.)
Content information relating to potential uncertainity about utilizing the model
Information relating to sensitive datasets that contain info like personal information that was used to develop the machine learning model
The section is devoted in descrbing the synthetic evaluation datasets used for a machine learning model. Synthetic evaluation datasets are a type of evaluation dataset. "It is often difficult to find datasets that represent populations outside of the initial domain used in training. In some of these situations, synthetically generated datasets may provide representation for use cases that would otherwise go unevaluated" (Mitchell et al., 2019). This section should described the aforementioned.
Synthetic Evaluation Data Information Section
Content involving known trade-offs for the use of the model. Trade offs might include performance or technical or ethical. "This section should list additional concerns that were not covered in the previous sections. For example, did the results suggest any further testing? Were there any relevant groups that were not represented in the evaluation dataset? Are there additional recommendations for model use? What are the ideal characteristics of an evaluation dataset for this model?"
The section is for information about the training data used for evaluation of a machine learning model.
The content here involves information about intential uses of the model. "This section should allow readers to quickly grasp what the model should and should not be used for, and why it was created. It can also help frame the statistical analysis presented in the rest of the card, including a short description of the user(s), use-case(s), and context(s) for which the model was originally developed."
Content information pertaining to intended users of the model. This includes individuals that may be impacted as an outcome of the use of the model.
This content section describes any versioning information for the machine learning model.