Frontier Model Forum

By Zach Stein-Perlman @ 2023-07-26T14:30 (+40)

This is a linkpost to https://blog.google/outreach-initiatives/public-policy/google-microsoft-openai-anthropic-frontier-model-forum/

This is a crosspost, probably from LessWrong. Try viewing it there.


SammyDMartin @ 2023-07-26T16:00 (+11)

This seems overall very good at first glance, and then seems much better once I realized that Meta is not on the list. There's nothing here that I'd call substantial capabilities acceleration (i.e. attempts to collaborate on building larger and larger foundation models, though some of this could be construed as making foundation models more useful for specific tasks). Sharing safety-capabilities research like better oversight or CAI techniques is plausibly strongly net positive even if the techniques don't scale indefinitely. By the same logic, while this by itself is nowhere near sufficient to get us AI existential safety if alignment is very hard (and could increase complacency), it's still a big step in the right direction.

adversarial robustness, mechanistic interpretability, scalable oversight, independent research access, emergent behaviors and anomaly detection. There will be a strong focus initially on developing and sharing a public library of technical evaluations and benchmarks for frontier AI models.

The mention of combating cyber threats is also a step towards explicit pTAI

BUT, crucially, because Meta is frozen out we can know both that this partnership isn't toothless, represents a commitment to not do the most risky and antisocial things Meta presumably doesn't want to give up, and the fact that they're the only major AI company in the US to not join will be horrible PR for them as well. 

Zach Stein-Perlman @ 2023-07-26T16:07 (+6)

(Briefly-- I of course agree that Meta AI is currently bad at safety, but I think a more constructive and less adversarial approach to them is optimal. And it doesn't seem that they're "frozen out"; I hope they improve their safety and join the FMF in the future.)

SammyDMartin @ 2023-07-26T16:17 (+10)

Yeah I didn't mean to imply that it's a good idea to keep them out permanently, but the fact that they're not in right now is a good sign that this is for real. If they'd just joined and not changed anything about their current approach I'd suspect the whole thing was for show

Andreas P @ 2023-07-27T03:33 (+7)

For someone new to looking at AI concerns, can either of you briefly explain why Meta is worse than the others? The biggest difference I'm aware of is that Meta is open source vs the others that are not

Zach Stein-Perlman @ 2023-07-27T06:44 (+9)

Good question. Yeah, Meta AI tends to share their research and model weights while OpenAI, Google DeepMind, and Anthropic seem to be becoming more closed. But more generally, those three labs seem to be concerned about catastrophic risk from AI while Meta does not. Those three labs have alignment plans (more or less), they do alignment research, they are working toward good red-teaming and model evals, they tend to support strong regulation that might be able to prevent dangerous AI from being trained or deployed, their leadership talks about catastrophic risks, and a decent chunk of their staff is concerned about catastrophic risks.

Sorry I don't have time to provide sources for all these claims.

Andreas P @ 2023-07-29T08:09 (+2)

Not a problem, that's a good starting point for me to effectively jump into the different reasons and find sources. I appreciate it!

Zach Stein-Perlman @ 2023-07-28T23:19 (+4)

Separately from the other thread-- the little evidence I'm aware of (bing chat, sparks of AGI, absence of evidence on safety) suggests that Microsoft is bad on safety. I'm surprised they were included.

Edit: and I weakly think their capabilities aren't near the frontier, except for their access to OpenAI's stuff.