OpenAI Names Four Areas for Safety Assessor Testing
OpenAI has published a document outlining its plan to open its models to outside safety assessors. The document sets forth four key areas for review and seven principles guiding the assessment process.
Four Key Areas
-
Independent Assessment of Safety Cases: This includes examining safety cases across training, evaluation, internal deployment, and external deployment. Assessors will verify if the team followed the specified conditions and if the evidence supports the claims made.
-
Safeguard Stack Testing: OpenAI wants assessors to test the effectiveness of safeguards against jailbreaks and capability uplift in cyber and biological domains with grey box access. They should also evaluate how AI agents interact with access controls, sandboxing, detection, and response systems under realistic operating conditions.
-
Capability Evaluations: This area focuses on evaluating AI capabilities under the Preparedness Framework, which encompasses chemical/biological risks, cybersecurity, and AI self-improvement. OpenAI emphasizes the need to correctly set thresholds and refresh evaluations as models surpass them.
-
Independent Investigation of Misalignment Incidents: Assessors would conduct thorough investigations into misalignment incidents, utilizing cyberforensics skills, alignment expertise, and the ability to analyze chains of thought at scale.
Seven Principles for Assessment Work
- Scope Agreement: Both parties must agree on the scope before beginning any assessment.
- Pre-Registration of Claims: Assessments should be pre-registered, and claims should clearly indicate whether they originate from OpenAI or the assessors.
- Transparency: All findings and methodologies should be openly documented.
- Timely Feedback: Labs should have time to address findings before publication, with the possibility of requesting redactions.
- Multi-Disciplinary Expertise: Assessments should involve multiple experts in relevant fields.
- Continuous Improvement: The process should foster continuous improvement in AI safety practices.
- Respect for Intellectual Property: Assessor work should respect OpenAI's intellectual property rights while ensuring transparency and accuracy.