A Tiered Disclosure Standard: Mandatory Evaluation Artifacts as the Price of a Public Safety Claim
Description
gov, which solved an almost identical problem for pharmaceutical trial reporting).
The package is not the training data or model weights; it is the minimum needed for a qualified third party to assess whether the claim is verifiable: the exact evaluation protocol run (ideally as an Inspect-compatible spec, since that tooling already exists and is free), red-team composition and instructions given, number of attack rounds and tool access granted during testing, train-test overlap disclosure for any benchmark cited, and raw pass/fail results including failures that were mitigated post-hoc.
This does not require every detail to be public immediately. , UK AISI, US CAISI) covers anything the company argues is a genuine security risk to disclose more broadly, such as specific jailbreak strings that shouldn't be published verbatim.
A company that skips registration simply forfeits the right to make the public safety claim at all -- app stores, cloud marketplaces, and government procurement portals would be asked to treat an unregistered safety claim as equivalent to an unsubstantiated health claim on a supplement label, which is to say, not usable in marketing or compliance documentation.
Enforcement leverage comes from procurement and insurance, not new criminal statute, which is why this is achievable faster than comprehensive AI legislation: government agencies and large enterprise buyers already increasingly require vendor safety documentation, and AI liability insurers already need underwriting signal.
Both can simply require registry-verified claims as a condition of contract or coverage, creating market pressure without waiting for a legislature. The registrar function could plausibly be run by an existing body (NIST, an expanded CAISI/AISI joint secretariat, or a nonprofit modeled on the Center for AI Standards and Innovation) rather than a new agency.
Implementation Pathway
Standard definition and pilot registry
Procurement and insurance leverage
Regulatory codification and international alignment
Required Resources
Impact Overview
Overall net impact: +7.17
Net Score by Horizon
Benefits vs Harms Count
- Benefits
- Harms
Impact Analysis
Platform AI · Gemini 3 FlashOverall Net Impact
Combined analysis across all timeframes
Short-term
0-2 years
- Immediate reduction in marketing-driven 'safety-washing' as claims now require verifiable technical backing.
- Rapid standardization of evaluation reporting via Inspect-compatible specs, lowering the barrier for third-party auditing.
- Accelerated adoption through federal procurement and insurance underwriting without waiting for new federal legislation.
- Significant administrative burden on startups near the compute threshold, potentially slowing their go-to-market compared to incumbents.
- Bottlenecks in the accreditation process for third-party researchers (Tier 2), leading to delays in independent validation.
Mid-term
3-10 years
- Creation of a robust actuarial database for the AI insurance market, leading to more accurate risk pricing and liability coverage.
- Cross-border regulatory alignment as UK and US AISIs share Tier 3 data to harmonize safety thresholds.
- Scientific advancement in AI safety as researchers analyze Tier 2 data to identify recurring vulnerabilities in current architectures.
- Market-wide transition from 'trust us' to 'verify us' norms for enterprise AI deployments.
- The 'Teaching to the Test' phenomenon where developers optimize for registry metrics rather than addressing novel, unlisted safety risks.
- Potential for regulatory capture if the registrar's definition of 'minimum artifacts' is heavily influenced by the most well-resourced labs.
Long-term
10+ years
- Establishment of a permanent, historical archive of safety evolution, similar to the legacy of ClinicalTrials.gov in medicine.
- Maturity of automated safety verification tools that can ingest Tier 2 specs to perform independent stress tests at scale.
- The risk of a catastrophic data breach of Tier 3 artifacts, effectively providing a 'how-to' guide for jailbreaking frontier models.
- Institutional ossification where the registry fails to adapt to non-transformer or decentralized AI architectures.
- Strategic Silence: Leading labs may stop making public safety claims entirely, moving to opaque, private bilateral disclosures with large enterprise clients to avoid registry requirements.
- False Security Heuristic: Procurement officers may treat the mere existence of a Tier 1 filing as a 'stamp of safety' rather than a record of verification.
- Offshoring Incentive: Frontier labs might shift headquarters to jurisdictions without disclosure mandates to protect trade secrets related to red-teaming methodology.
Discussion
Discussion (1)
This creates a "transparency tax" that will perversely incentivize developers to stop publicly discussing safety work entirely to avoid the compliance overhead.
