Debate series: should we push for a pause on the development of AI?

Will someone write about the symbolic importance of the ask for a pause?

Right now, most of what has been written here on this seems focused on the techno-economics as if a pause were only about slowing down a technical process.

Asking for a pause is also a signal that one is extremely serious about the risk (and by the same token, not asking for a pause but speaking about existential risks seems very hard to communicate to a broader public).

It's definitely good to think about whether a pause is a good idea. Together with Joep from PauseAI, I wrote down my thoughts on the topic here.

Since then, I have been thinking a bit on the pause and comparing it to a more frequently mentioned option, namely to apply model evaluations (evals) to see how dangerous a model is after training.

I think the difference between the supposedly more reasonable approach of evals and the supposedly more radical approach of a pause is actually smaller than it seems. Evals aim to detect dangerous capabilities. What will need to happen when those evals find that, indeed, a model has developed such capabilities? Then we'll need to implement a pause. Evals or a pause is mostly a choice about timing, not a fundamentally different approach.

With evals, however, we'll move precisely to the brink, look straight into the abyss, and then we plan to halt at the last possible moment. Unfortunately, though, we're in thick mist and we can't see the abyss (this is true even when we apply evals, since we don't know which capabilities will prove existentially dangerous, and since an existential event may already occur before running the evals).

And even if we would know where to halt: we'll need to make sure that the leading labs will practically succeed in pausing themselves (there may be thousands of people working there), that the models aren't getting leaked, that we'll implement the policy that's needed, that we'll sign international agreements, and that we gain support from the general public. This is all difficult work that will realistically take time.

Pausing isn't as simple as pressing a button, it's a social process. No one knowns how long that process of getting everyone on the same page will take, but it could be quite a while. Is it wise to start that process at the last possible moment, namely when the evals turn red? I don't think so. The sooner we start, the higher our chance of survival.

Also, there's a separate point that I think is not sufficiently addressed yet: we don't know how to implement a pause beyond a few years duration. If hardware and algorithms improve, frontier models could democratize. While I believe this problem can be solved by international (peaceful) regulation, I also think this will be hard and we will need good plans (hardware or data regulation proposals) for how to do this in advance. We currently don't have these, so I think working on them should be a much higher priority.

I'm really happy to see this happening.

In fact, I'd like to see more things along these kinds of lines.

While there's a lot of good discussion on this forum, we aren't always going to end up discussing the most important topics organically. So I think it's often helpful for CEA/the mods to occasionally direct the attention of the forum towards the discussion topics that will move us forward as a community.

If we wanted to go beyond this, then I think it would be quite valuable to find two people with opposite views to work together in order to produce a high-quality distillation of any such debates.

Thanks for noticing something you thought should happen (or having it flagged to you) and making it happen!

Love this idea, thanks for organizing this.

PSA: the term "compute overhang" or "hardware overhang" has been used in many ways. Today it seems to most often (but far from always) mean amount labs can quickly scale up the size of the largest training run (especially because a ban on large training runs ends). When you see it or use it, make sure everyone knows what it means.

(It will come up often in this debate.)

Any thoughts on using https://www.kialo.com/ or a similar tool specailized on debate?

Great initiative! 

Perhaps it'd be good to ask people who are doing some public-facing campaigning to contribute? For example, the team at the Existential Risk Observatory or those behind PauseAI. I might be wrong, but I don't think anyone on the list of agreed contributors represents that specific theory of change. 

I think a public-facing campaign is important to think about if we want to reduce the likelihood of articles such as 'How Silicon Valley doomers are shaping Rishi Sunak’s AI plans' being written.

Great idea, thanks for organizing!

I'm a long-time lurker but I registered to make this comment: While these posts are incredibly high quality, they are legally naive. There is no mention of the First Amendment or Bernstein v. Department of Justice, which is a significant gap. Yes, let's have the discussion about pausing AI and part of that should include its legality. 

This is a great idea, and I look forward to reading the diverse views on the wisdom of an AI pause.

I do hope that the authors contributing to this discussion take seriously the idea that an 'AI pause' doesn't need to be fully formalizable at a political, legal, or regulatory level. Rather, its main power can come from promoting an informal social consensus about the serious risks of AGI development, among the general public, journalists, politicians, and the more responsible people in the AI industry. 

In other words, the 'Pause AI' campaign might get most of its actual power and influence from helping to morally stigmatize reckless AI development, as I argued here.  

Thus, the people who argue that pausing AI isn't feasible, or realistic, or legal, or practical, may be missing the point. 'Pause AI' can function as a Schelling point, or focal point, or coordination mechanism, or whatever you want to call it, with respect to public discourse about the ethics of AI development.

I include links to my two old posts arguing for keeping AI development:

First I argued that AI is a necesary (almost irrepleaceable) tool to deal with the other existential risks (mainly nuclear war):

https://forum.effectivealtruism.org/posts/6j6qgNa3uGmzJEMoN/artificial-intelligence-as-exit-strategy-from-the-age-of

Then, that currently AI risk is simply "too low to be measured", and we need to be closer to AGI to develop realistic alignment work:

https://forum.effectivealtruism.org/posts/uHeeE5d96TKowTzjA/world-and-mind-in-artificial-intelligence-arguments-against

I think the idea of debates like this is great, good work team!

Is there any plan to expand this to non-AI cause areas?

So the final post will be released on September 27th?