AI Labs and Outside Evaluators: Control and Transparency Concerns
Anthropic and OpenAI Seek Control over Published Findings
Anthropic and OpenAI are advocating for significant control over what external evaluators can publish regarding their AI models, as reported by Reuters. Both companies express concerns that such findings could reveal trade secrets.
Promises of Independence, Concerns of Leakage
Despite recently promising external reviewers complete independence, both labs have sought strict control. The worry is that employees working inside the labs might leak secrets to competitors.
Proposed Solutions
Discussions include monitoring reviewer teams and implementing a ban on individuals working at more than one lab simultaneously. Until October 5th, both labs aimed to maintain tight control. By October 8th, Anthropic reported some shifts in its discussions with reviewers.
Differing Approaches
Anthropic stated that it allows previous reviewer partnerships to determine their methods and publication choices. However, for new partnerships, they retain the right to review and approve reports.
OpenAI, on the other hand, aims to avoid approving entire reports. They typically remove confidential details and allow reviewers to disclose when significant edits were made.
Contracts and Transparency
A contract reviewed by Reuters indicates that some labs already allow review firms to publish confidential details without approval.
Industry Insights
In September, Anthropic collaborated with Accenture’s AI arm, Faculty, as its first embedded evaluator. OpenAI plans to let external groups test models during training.
According to Rumman Chowdhury, a former US science envoy for AI, labs can use access to influencers as leverage without established rules.