GPAI Training Transparency

The table below shows an overview evaluated public summaries with A+ as the highest grade score and F the lowest. A separate list is provided further below with published summaries currently being evaluated, and another list for models which require a public summary is missing despite being necessary. You can click the model name to go to the detailed evaluation page which has more information, a link to the summary, and our evaluation notes. You can also see a detailed overview of scores for each section of the public summary. You can also click on the columns to sort the table by name or by score.

Why we started this project:

• We contend that compliance cannot be fait accompli, and that the public summaries are a key factor in creating transparency and enabling rights enforcement. Towards this, our work also acts as a guide for providers who are yet to publish their summaries to consider how to do so with the highest possible quality and utility.
• Compliance also invites practices that are intentionally or unintentionally deficient in achieving the goals. Our work serves as a useful tool for describing how and where and why certain practices are 'bad', e.g., where they use obfuscation, do not provide stated information. Using this, we can detect trends or patterns in whether the same issues occur in many summaries, and if so, how they can be collectively addressed through guidance, or enforced with priority.
• The largest challenge in undertaking this work has been finding public summaries as there is no consistent format or practice for how they should be provided. For this, we provide recommendations.
• The template for public summaries provided by the AI Office is intended to be revised with time to improve the state of documentation as well as to better guide the providers. We also provide recommendations for these to improve the quality and accessibility of the public summaries.