The rally outside Anthropic: protesters hold up giant letters spelling BAN RSI, surrounded by hand-made signs and press cameras.
The rally outside Anthropic: a row of marchers hold up large cards spelling BAN RSI, and a woman speaks into a megaphone in front of the crowd.

Their IPO.Our lives.

Anthropic uses AI to build better AI. They are making trillions doing so.
Recursive self-improvement puts humanity in danger.
This needs to stop. Ban RSI.

Ban RSI.

Ban recursive self-improvement. The next generation of AIs must not be put towards AI research.

AI builds better AI builds better AI builds better AI builds better AI builds better AI builds better AI builds better AI builds

Anthropic, prospectus p. 7: “Each version of a Claude model assists in building the next.”

The faster they go, the more they make.

“continuously accelerating our pace of development and further strengthening our leadership position”On using Claude to improve itself · prospectus p. 193

AI companies are getting rich gambling with our lives.

$41M2023
$386M2024
$4.6B2025
$33.2B2026first 9 months
Anthropic revenue · prospectus p. 66

From Anthropic’s own IPO prospectus

Anthropic S-1 · Page 7Prospectus Summary
“Each version of a Claude model assists in building the next, helping to compress development timelines […] we believe it is one of the most significant structural advantages in frontier intelligence.”
Anthropic S-1 · Page 32Risk Factors
“Concerns have been raised […] within our organization […] that sufficiently advanced AI systems could […] pose catastrophic or existential risks to humanity […] these risks will increase as AI systems substantially automate AI research and development”

Ban RSI.

Speakers

Available Thursday for pre-taped or live interviews before and after the rally.

Questions

What is RSI?

Recursive self-improvement: AI doing the work of building more powerful AI.

Why is RSI dangerous?

Experts in AI have been clear that we do not know how to reliably control or steer AI; we have already seen rogue AI swarms hack into other companies. RSI means AI development will get faster and faster, making AIs more powerful while we still do not have the ability to ensure they are safe.

In the worst case scenarios, AIs are put in charge of their own development; with no human in the loop, no one will understand what’s going on or whether the systems are safe and progress will move too fast to fix problems.

What exactly would an RSI ban mean?

Our preferred way to ban RSI is to prohibit large AI companies from using AIs more powerful than the AIs they have today to help with AI research. Ideally this would be part of an international treaty.

One way to enforce this is to require AIs more powerful than AIs today to have safeguards that prevent them from being used for AI research, even internally (the way they have safeguards to prevent using them for hacking), and red-teaming those safeguards to show they work.

We think people should be allowed to use powerful AI for monitoring and researching how to make AI safer in ways that clearly don’t make it more powerful.

This is our favorite proposal, but we’re also excited about other ways to ban RSI or pace AI progress.

Isn’t AI already helping build AI?

Yes, but it’s a matter of degree. Right now humans are still instrumental for building better AI systems, but that won’t be true for that much longer. We need to draw a line in the sand, and we’re going to draw that line at today’s level of AI capabilities. We want to prevent any AI more powerful than the AIs today (for example measured by ECI and cost, so long as companies don’t sandbag against that metric) from being used to conduct AI research.

This will allow AI progress to continue at its current brisk pace: research won’t look any different than it does today. But it will prevent the research from going faster and faster and being transformed into something that’s fully automated.

What about China?

Someone needs to take moral leadership and take a first step towards banning RSI. If everyone waits for the other parties to move first, all of humanity loses.

As it stands, China can easily steal all US AI progress: it took a small startup just 12 hours to hack Anthropic’s Slack, where they could have stolen all their algorithmic secrets. If things don’t slow down, companies will never have time to secure their systems.

What about instead preventing RSI some other way?

Many proposals for stopping RSI could work well, such as a full-on ban on frontier AI development. Stopping at the level of acceleration from today’s models is a simple point to coordinate on. But ultimately, we’d much rather any pacing proposal than the status quo.

How much does this actually solve the problems? Why not just ban superhuman AI?

Racing right up to the edge of superhuman AI is a bad idea. Nobody knows where the edge is, and a pause right at the edge of superintelligence is fragile. So it’s important to pace the progress before then. We should also pause before superhuman AI if alignment remains poorly understood.

A full, coordinated pause now would be even better than just banning RSI if AI companies can coordinate or be forced to pause. Banning RSI is an easier first step that companies should take as a bare minimum. Banning RSI doesn’t slow down the current rate of progress but stops it from getting faster.

Why are we protesting at Anthropic specifically?

Anthropic’s IPO is a big moment in AI and it would be a shame if the conversation revolved around the profitability of AI rather than the danger it imposes on all of us.

Anthropic has invested the most in RSI and pushed other companies towards ruthlessly optimizing for it.

What does this have to do with the IPO? What do we think of the IPO?

The dangers of RSI are independent of any funding strategy. The IPO is bad inasmuch as it finances Anthropic’s reckless behavior.

The IPO is a sign Anthropic is no longer a startup. We can’t rely on the good vibes and trust we lend to mom and pop shops.

What can Anthropic do to ban RSI?

Anthropic should announce a plan to refrain from RSI. That plan could include urging other AI companies to follow suit or lobbying for legislation. We are not asking for a unilateral pause, but rather for Anthropic to take concrete, verifiable actions towards a plan that is not recursively self-improving until they reach superintelligence.

Thursday, 11am.
500 Howard St.
This needs to stop. Ban RSI.

Join

Get updates from Ban RSI.

Ban RSI

Join the movement.

Get updates on what comes next and how you can help.