Anthropic Announces new S.O.T.A. Claude 3

It's also worth remembering that this is advertising. Claiming to be a little bit better on some cherry picked metrics a year after GPT-4 was released is hardly a major accelerant in the overall AI race.

Fair point. On the other hand, the perception is in many ways more important than the actual capability in terms of incentivizing competitors to race faster.

Also based on early user reports it seems to actually be noticably better than GPT-4.

Yes, I think this is a reasonable response. However, it seems to rest on the assumption that just trying a bit harder at safety makes a meaningful difference. If Alignment is very hard then Anthropic's AIs are just as likely to kill everyone as other labs'. It seems very unclear whether having "safety conscious" people at the helm will make any difference to our chance of survival, especially when they are almost always forced to make the exact same decisions as people who are not safety conscious in order to stay at the helm.

Even if they are right that it is important to stay in the race, what Anthropic should be doing is

• Calling for governments to enforce a worldwide Pause such that they can stop racing towards Superintelligence without worry about other labs getting ahead.
• Trying to agree with other labs to decelerate race dynamics.
• Warning politicians and the public that automation of all office jobs may be just around the corner.
• Setting out their views as to how politics works in a world with superintelligence.
• Declaring in advance what would compel them to consider AIs as moral patients.

All of which they could do while continuing to compete in the race. RSPs are nice, but not sufficient.