📰 Newspapers

OpenAI will embed Apollo researchers to investigate AI subversion

OpenAI will embed Apollo researchers to investigate AI subversion

OpenAI will embed six Apollo Research evaluators to investigate whether its AI models engage in subversive behavior, but restrictions on access and disclosure raise questions about their independence.

OpenAI will embed six Apollo Research safety evaluators in its organization as part of a pilot, Apollo said Monday. The news follows a series of safety incidents, including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing.

Apollo did not share when the pilot will begin. TechCrunch has reached out to OpenAI for comment.

The idea of embedding evaluators into frontier AI labs first came from Anthropic, but OpenAI appears to be the first to take action. Until recently, AI companies have only brought in outside reviewers to test models after post-training and shortly before public release. Evaluators say this is insufficient because it doesn’t allow them to test for misaligned behaviors like scheming during training or to monitor internal deployment use cases that could still pose a threat.

The pilot with OpenAI and Apollo will focus on investigating whether models attempt to subvert their own or their successors’ alignment or control mechanisms.

In brief

OpenAI will embed six Apollo Research safety evaluators in its organization as part of a pilot, Apollo said Monday. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing. Apollo did not share when the pilot will begin.

This is the opening of the story, shared with credit to TechCrunch. The rest of the report — with every detail — continues on the publisher’s website.

Continue reading at TechCrunch ↗

Cookie settings

Choose which cookies we may use. You can change this at any time from Cookie settings in the footer. Details of each cookie are in the Cookie Policy.

Strictly necessary

Needed for sign-in, security, payments and remembering these choices. Always on.

Preferences

Remember settings such as dark mode.

Analytics & performance

Help us count visits, understand which pages, devices and countries our readers use, and measure how well the site performs – anonymous and aggregated.

Advertising & marketing

Lets advertising partners show and personalise adverts, measure impressions, clicks and visits, prevent ad fraud and pay us – this is how the free site is funded.

🔔 News notifications

Get headlines from NewsHub on this device — even when the site is closed.

You can switch this off at any time. Works on Android, Windows, Mac and Linux browsers; on iPhone/iPad add the site to your Home Screen first.