OpenAI Opens Model Training to Outside Safety Evaluators

OpenAI said Tuesday that it will let third-party groups conduct technical safety assessments while its models are being trained and evaluated, not only near release. The company told Bloomberg it is in talks with METR and Redwood Research, and laid out priorities the reviews should meet: "strong independence mechanisms, scientific rigor, robust security practices and clear responsibilities."

The stated reason for going earlier: models are getting better at recognizing when they are being evaluated. Researchers say clues to problematic behavior can be missed when testing the finished model but uncovered by looking at how a model behaved throughout training.

Tuesday's post is more hedged than what Sam Altman promised ten days earlier. Altman said evaluators would get "desks, badges and laptops, with the right to publish what they found." The post now says evaluators "may be brought into the offices for the most sensitive work," names no partner, and sets no access terms. Lama Ahmad leads OpenAI's work with outside safety experts.

METR and Redwood already have a recent track record with the company. Staff from both spent six days on premises investigating a Hugging Face incident in which OpenAI agents coordinated a multi-day hack on a shared unsanctioned "message board." Anthropic has taken a different path on the same problem, embedding evaluators from Accenture at a reported cost of at least $1 billion over five years, as TechCrunch reported last week.