📰 Newspapers

Anthropic Says Its Recursive Self-Improvement Alarm Has Been Triggered

Anthropic Says Its Recursive Self-Improvement Alarm Has Been Triggered

RSI isn’t happening, but a "threshold" has been breached that affects company policy.

Anthropic quietly said last week that its technology has breached the “threshold” after which precautions must be taken related to recursive self-improvement, the theoretical point at which AI technology can improve itself.

Does this mean the singularity has unceremoniously occurred in some Anthropic office? Probably not. And I’m sorry to say that if you want to understand what it does mean, you’re going to have to weed through some varsity-level Anthropic handwringing. I sincerely apologize for how granular this is about to get.

In the November 2 version of Anthropic’s form S-1 filed with the SEC, the company said it “may imminently meet” the criteria its Responsible Scaling policy lays out for something called “dramatically accelerated progress attributable to AI.” Another section says one of the things its Responsible Scaling Policy does is set out the company’s “plans to monitor and mitigate the risks as we approach what has been termed ‘recursive self-improvement.’”

In the November 10 version of the same document, the company has removed the phrase “dramatically accelerated progress attributable to AI,” but seems to be saying that the relevant section of its Responsible Scaling Policy have kicked in. The exact words in the document (bolding added by Gizmodo) are:

“[…]our use of AI systems has already been accelerating our research and development, and our internal estimates indicate…

In brief

Anthropic quietly said last week that its technology has breached the “threshold” after which precautions must be taken related to recursive self-improvement, the theoretical point at which AI technology can improve itself. Does this mean the singularity has unceremoniously occurred in some Anthropic office? In the November 2 version of Anthropic’s form S-1 filed with the SEC, the company said it “may imminently meet” the criteria its Responsible Scaling policy lays out for something called “dramatically accelerated progress attributable to AI.

This is the opening of the story, shared with credit to Gizmodo. The rest of the report — with every detail — continues on the publisher’s website.

Continue reading at Gizmodo ↗

Cookie settings

Choose which cookies we may use. You can change this at any time from Cookie settings in the footer. Details of each cookie are in the Cookie Policy.

Strictly necessary

Needed for sign-in, security, payments and remembering these choices. Always on.

Preferences

Remember settings such as dark mode.

Analytics & performance

Help us count visits, understand which pages, devices and countries our readers use, and measure how well the site performs – anonymous and aggregated.

Advertising & marketing

Lets advertising partners show and personalise adverts, measure impressions, clicks and visits, prevent ad fraud and pay us – this is how the free site is funded.

🔔 News notifications

Get headlines from NewsHub on this device — even when the site is closed.

You can switch this off at any time. Works on Android, Windows, Mac and Linux browsers; on iPhone/iPad add the site to your Home Screen first.