Third-wave AI safety needs sociopolitical thinking

Just to respond to a narrow point because I think this is worth correcting as it arises: Most of the US/EU GDP growth gap you highlight is just population growth. In 2000 to 2022 the US population grew ~20%, vs. ~5% for the EU. That almost exactly explains the 55% vs. 35% growth gap in that time period on your graph; 1.55 / 1.2 * 1.05 = 1.36.

This shouldn't be surprising, because productivity in the 'big 3' of US / France / Germany track each other very closely and have done for quite some time. (Edit: I wasn't expecting this comment to blow up, and it seems I may have rushed this point. See Erich's comment below and my response.) Below source shows a slight increase in the gap, but of <5% over 20 years. If you look further down my post the Economist has the opposing conclusion, but again very thin margins. Mostly I think the right conclusion is that the productivity gap has barely changed relative to demographic factors.

I'm not really sure where the meme that there's some big / growing productivity difference due to regulation comes from, but I've never seen supporting data. To the extent culture or regulation is affecting that growth gap, it's almost entirely going to be from things that affect total working hours, e.g. restrictions on migration, paid leave, and lower birth rates^[1], not from things like how easy it is to found a startup.

https://www.economist.com/graphic-detail/2023/10/04/productivity-has-grown-faster-in-western-europe-than-in-america

But in aggregate, western Europeans get just as much out of their labour as Americans do. Narrowing the gap in total GDP would require additional working hours, either via immigration or by raising the amount of time citizens spend on the job.

1. ^{^}

   Fertility rates are actually pretty similar now, but the US had much higher fertility than Germany especially around 1980 - 2010, converging more recently, so it'll take a while for that to impact the relative sizes of the working populations. 

Erich_Grunewald 🔸 @ 2025-04-04T12:18 (+23)

This is weird because other sources do point towards a productivity gap. For example, this report concludes that "European productivity has experienced a marked deceleration since the 1970s, with the productivity gap between the Euro area and the United States widening significantly since 1995, a trend further intensified by the COVID-19 pandemic".

Specifically, it looks as if, since 1995, the GDP per capita gap between the US and the eurozone has remained very similar, but this is due to a widening productivity gap being cancelled out by a shrinking employment rate gap:

This report from Banque de France has it that "the EU-US gap has narrowed in terms of hours worked per capita but has widened in terms of GDP per hours worked", and that in France at least this can be attributed to "producers and heavy users of IT technologies":

The Draghi report says 72% of the EU-US GDP per capita gap is due to productivity, and only 28% is due to labour hours:

Part of the discrepancy may be that the OWID data only goes until 2019, whereas some of these other sources report that the gap has widened significantly since COVID? But that doesn't seem to be the case in the first plot above (it still shows a widening gap before COVID).

Or maybe most of the difference is due to comparing the US to France/Germany, versus also including countries like Greece and Italy that have seen much slower productivity growth. But that doesn't explain the France data above (it still shows a gap between France and the US, even before COVID).

AGB 🔸 @ 2025-04-04T18:52 (+15)

Thanks for this. I already had some sense that historical productivity data varied, but this prompted me to look at how large those differences are and they are bigger than I realised. I made an edit to my original comment.

TL;DR: Current productivity people mostly agree about. Historical productivity they do not. Some sources, including those in the previous comment, think Germany was more productive than the US in the past, which makes being less productive now more damning compared to a perspective where this has always been the case. 

*** 

For simplicity I'm going to focus on US vs. Germany in the first three bullets:

1. I struggle to find sources claiming a large gap in current productivity between the US and Germany.
   1. Our World in Data shows 7%
   2. The Banque France report you linked shows 8%
   3. Wikipedia gives two sources:
      1. The OECD, which shows 1%
      2. The ILO, which shows 2%
   4. The Bergeund report you linked to shows what appears to be ~5%^[1]
2. However, historical productivity estimates vary much more significantly.
   1. Some, like Our World in Data, show German productivity below US productivity near-continuously, e.g. 6% lower in 1995.
   2. Some sources claim Germany had higher productivity 20+ years ago, and combined with the slightly lower present productivity that everybody agrees on that can imply noticeably lower growth:
      1. If you look at the charts in the Banque France report you can see an example of this, with German productivity given as 3% higher in 2000.
      2. A striking example is the OECD, which puts German productivity >10% higher in 1995.^[2]
      3. The Bergeund report, which is also the source for a chart from Draghi's report - shown below - has around 8% higher German productivity in 1995.
3. So there are two competing stories, which I don't know how to adjudicate between:
   1. German productivity used to be slightly higher than US productivity, but is now slightly lower.
   2. German productivity has always been slightly lower than US productivity.
4. German productivity is higher than EU productivity as a whole, by a factor of roughly 1.15x.
   1. So you can substitute 'slightly higher' for 'slightly lower' and 'slightly lower' for 'significantly lower' in (3), producing two parallel competing stories for the EU as a whole.
   2. This link, with chart copied below as 'Figure 3', is an example of the 'EU productivity has always been significantly lower than US productivity' story.
   3. Note also that this chart agrees with Bergeund/Draghi - chart also below - about the current state; they both show EU-wide productivity at around 82% of US productivity in the present day. It's the 90s that they sharply disagree about, where one graph shows 74% and the other shows 95%, almost a 1.3x gap.

****

Where does that leave the conversation about European regulation? This is just my $0.02, but:

In my opinion the large divergences of opinion about the 90s, while academically interesting, are only indirectly relevant to the situation today. The situation today seems broadly accepted to be as follows:

• Western and Northern EU countries - Germany, Austria, France, Netherlands, Belgium, Luxembourg, Denmark, Norway, Sweden - have very similar productivity to the US.
• Eastern EU countries - mostly ex-USSR countries - have much lower productivity, but are catching up fast.
• Southern EU countries, e.g. Italy/Spain/Greece, have been languishing.
  • (Incidentally, so has the UK. I started to look at this when trying to understand how UK productivity compared to other countries, and was surprised to learn that the gap vs. nearby European countries was very similar to the gap vs. the US.)
• Creating an average productivity of around 82% of the US level

I think that when Americans think about European regulations, they are mostly thinking about the Western and Northern countries. For example, when I ask Claude which EU countries have the strongest labour rights, the list of countries it gives me is entirely a subset of those countries. But unless you think replacing those regulations with US-style regulations would allow German productivity to significantly exceed US productivity, any claim that this would close the GDP per capita gap between the US and Germany - around 1.2x - without more hours being worked is not very reasonable. Let alone the GDP gap, which layers on the US' higher population growth. 

Digging into Southern Europe and figuring out why e.g. Italy and Germany have failed to converge seems a lot more reasonable. Maybe regulation is part of that story. I don't know.

So I land pretty much where the Economist article is, which is why I quoted it:

But in aggregate, western Europeans get just as much out of their labour as Americans do. Narrowing the gap in total GDP would require additional working hours, either via immigration or by raising the amount of time citizens spend on the job.

1. ^{^}

   I am eyeballing at page 66 and adding together 'TFP' and 'capital deepening' factors. I think that amounts to labour productivity, and indeed the report does say "labour productivity...ie the product of TFP and capital deepening". Less confident about this than the other figures though.

2. ^{^}

   Unhelpfully, the data is displayed as % of 2015 productivity. I'm getting my claim from (a) OECD putting German 1995 productivity at 80% of 2015 levels, vs. the US being at 70% of 2015 levels and (b) 2022 productivity being 107% vs. 106% of 2022 levels. Given the OECD has 2022 US/German productivity virtually identical, I think the forced implication is that they think German productivity was >10% higher in 1995. 

NickLaing @ 2025-04-03T16:01 (+11)

Most changed mind votes in history of EA comments? This blew my mind a bit, I feel like I've read so much about American productivity outpacing Europe, think this deserves a full length article.

David Mathers🔸 @ 2025-04-04T08:07 (+3)

If productivity is so similar, how come the US is quite a bit richer per capita? Is that solely accounted for by workers working longer hours? 

Davik @ 2025-04-07T20:17 (+1)

Two factors come to mind.

It isn't. Prices are just inflated creating false growth. Real gdp, adjusted for variable inflation, shows dead even growth. 

Inequality. There are more billionaires in the US siphoning money from across the globe due to the lack of effective capital regulation. Hence what effects do persist are partially explained as capital flight to the USA inflating the top of the per capital pyramid without in any way increasing actual worker productivity. 

David Mathers🔸 @ 2025-04-10T10:18 (+2)

"Real gdp, adjusted for variable inflation, shows dead even growth. "    I asked about gdp per capita right now, not growth rates over time. Do you have a source showing that the US doesn't actually have higher gdp per capita?

Inequality is probably part of the story, but I had a vague sense median real wages are higher in the US. Do you have a source saying that's wrong? Or that it goes away when you adjust for purchasing power? 

Because you are so strongly pushing a particular political perspective on twitter-tech right=good roughly, I worry that your bounties are mostly just you paying people to say things you already believe about those topics. Insofar as you mean to persuade people on the left/centre of the community to change their views on these topics, maybe it would be better to do something like make the bounties conditional on people who disagree with your takes finding the investigations move their views in your direction.

I also find the use of the phrase "such controversial criminal justice policies" a bit rhetorical dark artsy and mildly incompatible with your calls for high intellectual integrity. It implies that a strong reason to be suspicious of Open Phil's actions has been given. But you don't really think the mere fact that a political intervention on an emotive, polarized topic is controversial is actually particularly informative about it. Everything on that sort of topic is controversial, including the negation of the Open Phil view on the US incarceration rate. The phrase would be ok if you were taking a very general view that we should be agnostic all political issues where smart, informed people disagree. But you're not doing that, you take lots of political stances in the piece: de-regulatory libertarianism, the claim that environmentalism has been net negative and Dominic Cummings can all accurately be described as "highly controversial".

Maybe I am making a mountain out of a molehill here. But I feel like rationalists themselves often catastrophise fairly minor slips into dark arts like this as strong evidence that someone lacks integrity. (I wouldn't say anything as strong as that myself; everyone does this kind of thing sometimes.) And I feel like if the NYT referred to AI safety as "tied to the controversial rationalist community" or to "highly controversial blogger Scott Alexander" you and other rationalists would be fairly unimpressed.

More substantively (maybe I should have started with this as it is a more important point), I think it is extremely easy to imagine the left/Democrat wing of AI safety becoming concerned with AI concentrating power, if it hasn't already. The entire techlash anti "surveillance" capitalism, "the algorithms push extremism" thing from left-leaning tech critics is ostensibly at least about the fact that a very small number of very big companies have acquired massive amounts of unaccountable power to shape political and economic outcomes. More generally, the American left has, I keep reading, been on a big anti-trust kick recently. The explicit point of anti-trust is to break up concentrations of power. (Regardless of whether you think it actually does that, that is how its proponents perceive it. They also tend to see it as "pro-market"; remember that Warren used to be a libertarian Republican before she was on the left.) In fact, Lina Khan's desire to do anti-trust stuff to big tech firms was probably one cause of Silicon Valley's rightward shift.

 It is true that most people with these sort of views are currently very hostile to even the left-wing of AI safety, but lack of concern about X-risk from AI isn't the same thing as lack of concern about AI concentrating power. And eventually the power of AI will be so obvious that even these people have to concede that it is not just fancy autocorrect.

It is not true that all people with these sort of concerns only care private power and not the state either. Dislike of Palantir's nat sec ties is a big theme for a lot of these people, and many of them don't like the nat sec-y bits of the state very much either. Also a relatively prominent part of the left-wing critique of DOGE is the idea that it's the beginning of an attempt by Elon to seize personal effective control of large parts of the US federal bureaucracy, by seizing the boring bits of the bureaucracy that actually move money around. In my view people are correct to be skeptical that Musk will ultimately choose decentralising power over accumulating it for himself.

Now strictly speaking none of this is inconsistent with your claim that the left-wing of AI safety lacks concern about concentration of power, since virtually none of these anti-tech people are safetyists. But I think it still matters for predicting how much the left wing of safety will actually concentrate power, because future co-operation between them and the safetyists against the tech right and the big AI companies is a distinct possibility.  

I find it pretty difficult to see how to get broad engagement on this when being so obviously polemical / unbalanced in the examples.

As someone who is publicly quite critical of mainstream environmentalism, I still find the description here so extreme that it is hard to take seriously as more than a deeply partisan talking point.

The "Environmental Protection Act" doesn't exist, do you mean the "National Environmental Policy Act" (NEPA)?

Neither is it true that environmentalists are single-handedly responsible for nuclear declining and clearly modern environmentalism has done a huge amount of good by reducing water and air pollution.

I think your basic point -- that environmentalism had a lot more negative effects than commonly realized and that we should expect similar degrees of unintended negative effects for other issues -- is probably true (I certainly believe it).

But this point can be made with nuance and attention to detail that makes it something that people with different ideological priors can read and engage with constructively. I think the current framing comes across as "owning the libs" or "owning the enviros" in a way that makes it very difficult for those arguments to get uptake anywhere that is not quite right-coded.

I responded well to Richard's call for More Co-operative AI Safety Strategies, and I like the call toward more sociopolitical thinking, since the Alignment problem really is a sociological one at heart (always has been). Things which help the community think along these lines are good imo, and I hope to share some of my own writing on this topic in the future.

Whether or not I agree with Richard's personal politics or not is kinda beside the point to this as a message. Richard's allowed to have his own views on things and other people are allowed to criticse this (I think David Mathers' comment is directionally where I lean too). I will say that not appreciating arguments from open-source advocates, who are very concerned about the concentration of power from powerful AI, has lead to a completely unnecessary polarisation against the AI Safety community from it. I think, while some tensions do exist, it wasn't inevitable that it'd get as bad as it is now, and in the end it was a particularly self-defeating one. Again, by doing the kind of thinking Richard is advocating for (you don't have to co-sign with his solutions, he's even calling for criticism in the post!), we can hopefully avoid these failures in the future.

On the bounties, the one that really interests me is the OpenAI board one. I feel like I've been living in a bizarro-world with EAs/AI Safety People ever since it happened because it seemed such a collosal failure, either of legitimacy or strategy (most likely both), and it's a key example of the "un-cooperative strategy" that Richard is concerned about imo. The combination of extreme action and ~0 justification either externally or internally remains completely bemusing to me and was big wake-up call for my own perception of 'AI Safety' as a brand. I don't think people can underestimate the second-impact effect this bad on both 'AI Safety' and EA, coming about a year after FTX.

I broadly agree with the thesis that AI safety needs to incorporate sociopolitical thinking rather than shying away from it. No matter what we do on the technical side, the governance side will always run up against moral and political questions that have existed for ages. That said, I have some specific points about this post to talk about, most of which are misgivings:

• The claim that environmentalism caused climate change is ....reaching. To say that first wave environmentalism's opposition to nuclear energy slowed our mitigation of climate change is a slightly more defensible claim, but at the time, (1) nuclear waste was still a less solved problem, (2) fossil fuel companies were actively contributing to disinformation campaigns to ensure their continued dominance of the energy market, and (3) even without the opposition to nuclear from the early environmentalist movement, which was far from large enough to effect a systemic change to the entire planet and be held responsible for it, there still would likely have not been enough nuclear plants to overtake fossil fuels in some miraculous, instantaneous energy transition for the same reasons that that doesn't happen now: nuclear energy is capital-intensive, and it takes a long time in human-years to break even.
• Also, I am suspicious of framing "opposition to geoengineering" as bad -- this, to me, is a red flag that someone has not done their homework on uncertainties in the responses of the climate system to large-scale interventions like albedo modification. Geoengineering the planet wrong is absolutely an X-risk.

•  the "Left wants control, Right wants decentralization" dichotomy here seems not only narrowly focused on Republican - Democrat dynamics, but also wholly incorrect in terms of what kinds of political ideology actually leads one to support one versus the other. Many people on the left would argue uncompromisingly for decentralization due to concerns about imperfect actors cementing power asymmetries through AI. Much of the regulation that the center-left advocates for is aimed at preventing power accumulation by private entities, which could be viewed as a safeguard against the emergence of some well-resourced, corporate network-state that is more powerful than many nation-states. I think Matrice nailed it above in that we are all looking at the same category-level, abstract concerns like decentralization versus control, closed-source versus open-source, competition versus cooperation, etc. but once we start talking about object-level politics -- Republican versus Democratic policy, bureaucracy and regulation as they actually exist in the U.S. -- it feels, for those of us on the left, like we have just gotten off the bus to crazy town. The simplification of "left == big government, right == small government" is not just misleading; it is entirely the wrong axis for analyzing what left and right are (hence the existence of the still oversimplified, but one degree less 1-dimensional, Political Compass...). It seems to me that it is important for us to all step outside of our echo chambers and determine precisely what our concerns and priorities are so we can determine areas where we align and can act in unison.

   

•  I sense a pretty clear Silicon Valley style right-wing echo-chamber aura from this post in general, and although I will always assume best intentions, I find it worrying that there are more nods to network-states and ideas that tread closely to the neoreactionary ideas of Curtis Yarvin and Nick Land than there are nods to post-capitalist or left-wing visions of the future, such as AI enabling decentralized planned economies or more direct democracies. We need to be careful that the discussions we are having about AI safety as they relate to politics do not end up reflecting, for instance, a lack of diversity of opinion within the Bay Area tech community that is doing the discussing. 

Some points I liked / agree with:

• I agree we need to rethink political structures in a much bigger, broader way as it concerns AI. If we bring AGI into the picture, entirely new kinds of economies become possible, so it doesn't make all that much sense to focus on Republican / Democrat dynamics except for pragmatic actions in the near-term. I actually think we should be focusing way more on what the best ways to handle these things in the abstract would be to avoid muddy waters of current events and the political milieu.
• AI as a policy and moral advisor is among my favorite ideas. I would love to see an AI that can provide an evidence-based case for a given policy over another having taken in the entire corpus of information available on it independent of echo chambers. I do worry about who might build such an AI, though, as if it were not open-source and interpretable, it could be rigged to provide a veneer of rigorous justification for any draconian policy a government prefers to pass.

Anybody else in addition to Fukuyama, Friedman, and Cummings who you think is good at doing the kind of thinking you're pointing to here?

Are there alternatives to your two factor model that seem particularly plausible to you?

a) The link to your post on defining alignment research is broken
b) "Governing with AI opens up this whole new expanse of different possibilities" - Agreed. This is part of the reason why my current research focus is on wise AI advisors.
c) Regarding malaria vaccines, I suspect it's because the people who focused on high-quality evidence preferred bet nets and folk who were interested in more speculative interventions were drawn more towards long-termism.

Thanks for posting this. It makes me think -"Oh, someone is finally mentioning this".

Observation : I think that your model rests on hypotheses that are the hypotheses I expect someone from Silicon Valley to suggest, using Silicon-Valley-originating observations. I don't think of Silicon Valley as 'the place' for politics, even less so epistemically accurate politics (not evidence against your model of course, but my inner simulator points at this feature as a potential source of confusion)

We might very well need to use a better approach than our usual tools for thinking about this. I'm not even sure current EAs are better at this than the few bottom-lined social science teachers I met in the past -being truth-seeking is one thing, knowing the common pitfalls of (non socially reflexive) truth-seeking in political thinking is another. 

For some reasons that I won't expand on, I think people working on hierarchical agency are really worth talking to on this topic and tend to avoid the sort of issues 'bayesian' rationalists will fall into.

Thanks for this. Is there a place where I can see the sources you are using here?

I am particularly interested in the source for this:
"The other graph here is an interesting one. It's the financial returns to IQ over two cohorts. The blue line is the older cohort, it's from 50 years ago or something. It's got some slope. And then the red line is a newer cohort. And that's a much steeper slope. What that means is basically for every extra IQ point you have, in the new cohort you get about twice as much money on average for that IQ point. "

Executive summary: As EA and AI safety move into a third wave of large-scale societal influence, they must adopt virtue ethics, sociopolitical thinking, and structural governance approaches to avoid catastrophic missteps and effectively navigate complex, polarized global dynamics.

Key points:

1. Three-wave model of EA/AI safety: The speaker describes a historical progression from Wave 1 (orientation and foundational ideas), to Wave 2 (mobilization and early impact), to Wave 3 (real-world scale influence), each requiring different mindsets—consequentialism, deontology, and now, virtue ethics.
2. Dangers of scale: Operating at scale introduces risks of causing harm through overreach or poor judgment; environmentalism is used as a cautionary example of well-intentioned movements gone wrong due to inadequate thinking and flawed incentives.
3. Need for sociopolitical thinking: Third-wave success demands big-picture, historically grounded, first-principles thinking to understand global trends and power dynamics—not just technical expertise or quantitative reasoning.
4. Two-factor world model: The speaker proposes that modern society is shaped by (1) technology increasing returns to talent, and (2) the expansion of bureaucracy. These create opposing but compounding tensions across governance, innovation, and culture.
5. AI risk framings are diverging: One faction views AI risk as anarchic threat requiring central control (aligned with left/establishment), while another sees it as concentrated power risk demanding decentralization (aligned with right/populists); AI safety may mirror broader political polarization unless deliberately bridged.
6. Call to action: The speaker advocates for governance “with AI,” rigorous sociopolitical analysis, moral framework synthesis, and truth-seeking leadership—seeing EA/AI safety as “first responders” helping humanity navigate an unprecedented future.

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Crossposting this comment from LW, because I think there is some value here:

https://www.lesswrong.com/posts/6YxdpGjfHyrZb7F2G/third-wave-ai-safety-needs-sociopolitical-thinking#HBaqJymPxWLsuedpF

The main points are that value alignment will be way more necessary for ordinary people to survive, no matter the institiutions adopted, that the world hasn't yet weighed in that much on AI safety and plausibly never will, but we do need to prepare for a future in which AI safety may become mainstream, that Bayesianism is fine actually, and many more points in the full comment.

Love this, thanks!