How I hope the EA community will respond to the AI bubble popping

I agree that the OP is too confident/strongly worded, but IMO this

which is more than enough to justify EA efforts here. 

could be dangerously wrong. As long as AI safety consumes resources that might have counterfactually gone to e.g. nuclear disarmament, stronger international relations, it might well be harmful in expectation.

This is doubly true for warlike AI 'safety' strategies like Aschenbrenner's call to intentionally arms race China, Hendrycks, Schmidt and Wang's call to 'sabotage' countries that cross some ill-defined threshold, and Yudkowsky calling for airstrikes on data centres. I think such 'AI safety' efforts are very likely increasing existential risk.

I don't think it's clear, absent further argument, that there has to be a 10% chance of full AGI in the relatively near future to justify the currently high valuations of tech stocks. New, more powerful models could be super-valuable without being able to do all human labour. (For example, if they weren't so useful working alone, but they made human workers in most white collar occupations much more productive.) And you haven't actually provided evidence that most experts think there's a 10% chance current paradigm will lead to AGI. Though the latter point is a bit of a nitpick if 24% of experts think it will, since I agree the latter is likely enough to justify EA money/concern. (Maybe the survey had some don't knows though?).

76% of experts saying it's "unlikely" the current paradigm will lead to AGI leaves ample room for a majority thinking there's a 10%+ chance it will . . . .

 . . . . and the field are still mostly against you (at the 10% threshold). 

I agree that the "unlikely" statistic leaves ample room for the majority of the field thinking there is a 10%+ chance, but it does not establish that the majority actually thinks that.

I would like to bring back more of the pre-ChatGPT disposition where people were more comfortable emphasizing their uncertainty, but standing by the expected value of AI safety work.

I think there are at least two (potentially overlapping) ways one could take the general concern that @Yarrow Bouchard 🔸 is identifying here. One, if accepted, leads to the substantive conclusion that EA individuals, orgs, and funders shouldn't be nearly as focused on AI because the perceived dangers are just too remote. An alternative framing doesn't necessarily lead there. It goes something like there has been a significant and worrisome decline in the quality of epistemic practices surrounding AI in EA since the advent of ChatGPT. If it -- but not the other -- framing is accepted, it leads in my view to a different set of recommended actions.

I flag that since I think the relevant considerations for assessing the alternative framing could be significantly different.

Yarrow Bouchard 🔸 @ 2025-12-08T14:20 (+3)

One not need choose between the two because they both point toward the same path: re-examine claims with greater scrutiny. There is no excuse for the egregious flaws in works like "Situational Awareness" and AI 2027. This is not serious scholarship. To the extent the EA community gets fooled by stuff like this, its reasoning process, and its weighing of evidence, will be severely impaired.

If you get rid of all the low-quality work and retrace all the steps of the argument from the beginning, might the EA community end up in basically the same place all over again, with a similar estimation of AGI risk and a similar allocation of resources toward it? Well, sure, it might. But it might not.

If your views are largely informed by falsehoods and ridiculous claims, half-truths and oversimplifications, greedy reductionism and measurements with little to no construct validity or criterion validity, and, in some cases, a lack of awareness of countervailing ideas or the all-too-eager dismissal of inconvenient evidence, then you simply don’t know what your views would end up being if you started all over again with more rigour and higher standards. The only appropriate response is to clear house. Put the ideas and evidence into a crucible and burn away what doesn’t belong. Then, start from the beginning and see what sort of conclusions can actually be justified with what remains.

A large part of the blame lies at the feet of LessWrong and at the feet of all the people in EA who decided, in some important cases quite early on, to mingle the two communities. LessWrong promotes skepticism and suspicion of academia, mainstream/institutional science, traditional forms of critical thinking and scientific skepticism, journalism, and society at large. At the same time, LessWrong promotes reverence and obsequence toward its own community, positioning itself as an alternative authority to replace academia, science, traditional critical thought, journalism, and mainstream culture. Not innocently. LessWrong is obsessed with fringe thinking. The community has created multiple groups that Ozy Brennan describes as "cults". Given how small the LessWrong community is, I’d have to guess that the rate at which the community creates cults must be multiple orders of magnitude higher than the base rate for the general population.

LessWrong is also credulous about racist pseudoscience, and, in the words of a former Head of Communications at the Centre for Effective Altruism, is largely "straight-up racist". One of the admins of LessWrong and co-founders of Lightcone Infrastructure once said, in the context of a discussion about the societal myth that gay people are evil or malicious and a danger to children:

I think... finding out (in the 1950s) that someone maintained many secret homosexual relationships for many years is actually a signal the person is fairly devious, and is both willing and capable of behaving in ways that society has strong norms about not doing.

It obviously isn't true about homosexuals once the norm was lifted, but my guess is that it was at the time accurate to make a directional bayesian update that the person had behaved in actually bad and devious ways.

Such statements make "rationalist" a misnomer. (I was able to partially dissuade of him of this nonsense by showing him some of the easily accessible evidence he could have looked up for himself, but the community did not seem to particularly value my intervention.)

I don’t know that the epistemic practices of the EA community can be rescued as long the EA community remains interpenetrated with LessWrong to a major degree. The purpose of LessWrong is not to teach rationality, but to disable one’s critical faculties until one is willing to accept nonsense. Perhaps it is futile to clamour for better-quality scholarship when such a large undercurrent of the EA community is committed to the idea that normal ideas of what constitutes good scholarship are wrong and that the answers to what constitutes actually good scholarship lie with Eliezer Yudkowsky, an amateur philosopher with no relevant qualifications or achievements in any field, who frequently speaks with absolute confidence and is wrong, who experts often find non-credible, who has said he literally sees himself as the smartest person on Earth, and who rarely admits mistakes (despite making many) or issues corrections. If Yudkowsky is your highest and more revered authority, if you follow him in rejecting academia, institutional science, mainstream philosophy, journalism, normal culture, and so on, then I don’t know what could possibly convince you that the untrue things you believe are untrue, since your fundamental epistemology comes down to whether Yudkowsky says something is true or not, and he’s told you to reject all other sources of truth. 

To the extent the EA community is under LessWrong’s spell, it will probably remain systemically irrational forever. Only within the portions of the EA community who have broken that spell, or never come under it in the first place, is there the hope for academic standards, mainstream scientific standards, traditional critical thinking, journalistic fact-checking, culturally evolved wisdom, and so on to take hold. It would be like expecting EA to be rational about politics while 30% of the community is under the spell of QAnon, or to be rational about global health while a large part of the community is under the spell of anti-vaccination pseudoscience. It’s just not gonna happen.

But maybe my root cause analysis is wrong and the EA community can course correct without fundamentally divorcing LessWrong. I don’t know. I hope that, whatever is the root cause, whatever it takes to fix it, the EA community’s current low standards for evidence and argumentation pertaining to AGI risk get raised significantly. 

I don’t think it’s a brand new problem, by the way. Around 2016, I was periodically arguing with people about AI on the main EA group on Facebook. One of my points of contention was that MIRI’s focus on symbolic AI was a dead-end and that machine learning had empirically produced much better results, and was where the AI field was now focused. (MIRI took a long time before they finally hired their first researcher to focus on machine learning.) I didn’t have any more success convincing people about that back then than I’ve been having lately with my current points of contention.

I agree though that the situation seems to have gotten much worse in recent years, and ChatGPT (and LLMs in general) probably had a lot to do with that. 

David Mathers🔸 @ 2025-12-08T15:37 (+16)

I don't think EAs AI focus is a product only of interaction with Less Wrong,-not claiming you said otherwise-but I do think people outside the Less Wrong bubble tend to be less confident AGI is imminent, and in that sense less "cautious".  

I think EAs AI focus is largely a product of the fact that Nick Bostrom knew Will and Toby when they were founding EA, and was a big influence on their ideas. Of course, to some degree this might be indirect influence from Yudkowsky since he was always interacting with Nick Bostrom, but it's hard to know in what direction the influenced flowed here. I was around in Oxford during the embryonic stages of EA, and while I was not involved-beyond being a GWWC member, I did have the odd conversation with people who were involved, and my memory is that even then, people were talking about X-risk from AI as a serious contender for the best cause area, as early as at least 2014, and maybe a bit before that. They -EDIT: by "they" here I mean, "some people in Oxford, I don't remember who"; don't know when Will and Toby specifically first interacted with LW folk-were involved in discussion with LW people, but I don't think they got the idea FROM LW. Seems more likely to me they got it from Bostrom and the Future of Humanity Institute, who were just down the corridor. 

What is true is that Oxford people have genuinely expressed much more caution about timelines. I.e. in What We Owe the Future, published as late as 2022, Will is still talking about how AGI might be more than 50 years, away but also "it might come soon-within the next fifty or even twenty years." (If you're wondering what evidence he cites, it's the Cotra bioanchors report.) His discussion primarily emphasizes uncertainty about exactly when AGI will arrive, and how we can't be confident it's not close. He cites a figure from an Open Phil report guessing an 8% chance of AGI by 2036*. I know you're view is that this is all wildly wrong still, but it's quite different from what many-not all-Less Wrong people say, who tend to regard 20 years as a long time line. (Maybe Will has updated to shorter timelines since of course.) 

I think there is something of a divide between people who believe strongly in a particular set of LessWrong derived ideas about the imminence of AGI, and another set of people who are mainly driven by something like "we should take positive EV bets with a small chance of paying off, and doing AI stuff just in case AGI arrives soon". Defending the point about taking positive EV bets with only a small chance of pay-off is what a huge amount of the academic work on Longtermism at the GPI in Oxford was about. (This stuff definitely has been subjected to-severe-levels of peer reviewed scrutiny, as it keeps showing up in top philosophy journals with rejection rates of like, 90%.) 

*This is more evidence people were prepared to bet big on AI risk long before the idea that AGI is actually imminent became as popular as it is now. I think people just rejected the idea that useful work could only be done when AGI was definitely near, and we had near-AGI models. 

Linch @ 2025-12-08T19:21 (+12)

eh, I think the main reason EAs believe AGI stuff is reasonably likely is because this opinion is correct, given the best available evidence^[1]. 

Having a genealogical explanation here is sort of answering the question on the wrong meta-level, like giving a historical explanation for "why do evolutionists believe in genes" or telling a touching story about somebody's pet pig for "why do EAs care more about farmed animal welfare than tree welfare." 

Or upon hearing "why does Google use ads instead of subscriptions?" answering with the history of their DoubleClick acquisition. That history is real, but it's downstream of the actual explanation: the economics of internet search heavily favor ad-supported models regardless of the specific path any company took. The genealogy is epiphenomenal.

The historical explanations are thus mildly interesting but they conflate the level of why. 

EDIT: man I'm worried my comment will be read as a soldier-mindset thing that only makes sense if you presume the "AGI likely soon" is already correct. Which does not improve on the conversation. Please only upvote it iff a version of you that's neutral on the object-level question would also upvote this comment.

1. ^{^}

   Which is a different claim from whether it's ultimately correct. Reality is hard.

David Mathers🔸 @ 2025-12-08T19:32 (+3)

Yeah, it's fair objection that even answer the why question like I did presupposes that EAs are wrong, or at least, merely luckily right. (I think this is a matter of degree, and that EAs overrated the imminence of AGI and the risk of takeover on average, but it's still at least reasonable to believe AI safety and governance work can have very high expected value for roughly the reasons EAs do.) But I was responding to Yarrow who does think that EAs are just totally wrong, so I guess really I was saying that "conditional on a sociological explanation being appropriate, I don't think it's as LW-driven as Yarrow thinks", although LW is undoubtedly important.)

Linch @ 2025-12-08T21:01 (+4)

presupposes that EAs are wrong, or at least, merely luckily right

Right, to be clear I'm far from certain that the stereotypical "EA view" is right here. 

I guess really I was saying that "conditional on a sociological explanation being appropriate, I don't think it's as LW-driven as Yarrow thinks", although LW is undoubtedly important.

Sure that makes a lot of sense! I was mostly just using your comment to riff on a related concept. 

I think reality is often complicated and confusing, and it's hard to separate out contingency vs inevitable stories for why people believe what they believe. But I think the correct view is that EAs' belief on AGI probability and risk (within an order of magnitude or so)  is mostly not contingent (as of the year 2025) even if it turns out to be ultimately wrong.

The Google ads example was the best example I could think of to illustrate this. I'm far from certain that Google's decision to use ads was actually the best source of long-term revenue (never mind being morally good lol). But it still seemed like the internet as we understand it meant it was implausible that Google ads was counterfactually due to their specific acquisitions.

Similarly, even if EAs ignored AI before for some reason, and never interacted with LW or Bostrom, it's implausible that, as of 2025, people who are concerned with ambitious, large-scale altruistic impact (and have other epistemic, cultural, and maybe demographic properties characteristic of the movement) would not think of AI as a big deal. AI is just a big thing in the world that's growing fast. Anybody capable of reading graphs can see that.

That said, specific micro-level beliefs (and maybe macro ones) within EA and AI risk might be different without influence from either LW or the Oxford crowd. For example there might be a stronger accelerationist arm. Alternatively, people might be more queasy with the closeness with the major AI companies, and there will be a stronger and more well-funded contingent of folks interested in public messaging on pausing or stopping AI. And in general if the movement didn't "wake up" to AI concerns at all pre-ChatGPT I think we'd be in a more confused spot.

Yarrow Bouchard 🔸 @ 2025-12-08T23:49 (+2)

How many angels can dance on the head on a pin? An infinite number because angels have no spatial extension? Or maybe if we assume angels have a diameter of ~1 nanometre plus ~1 additional nanometre of diameter for clearance for dancing we can come up with a ballpark figure? Or, wait, are angels closer to human-sized? When bugs die do they turn into angels? What about bacteria? Can bacteria dance? Are angels beings who were formerly mortal, or were they "born” angels?^[1]

AI is just a big thing in the world that's growing fast. Anybody capable of reading graphs can see that.

Well, some of the graphs are just made-up, like those in "Situational Awareness", and some of the graphs are woefully misinterpreted to be about AGI when they’re clearly not, like the famous METR time horizon graph.^[2] I imagine that a non-trivial amount of EA misjudgment around AGI results from a failure to correctly read and interpret graphs.

And, of course, when people like titotal examine the math behind some of these graphs, like those in AI 2027, they are sometimes found to be riddled with major mistakes.

What I said elsewhere about AGI discourse in general is true about graphs in particular: the scientifically defensible claims are generally quite narrow, caveated, and conservative. The claims that are broad, unqualified, and bold are generally not scientifically defensible. People at METR themselves caveat the time horizons graph and note its narrow scope (I cited examples of this elsewhere in the comments on this post). Conversely, graphs that attempt to make a broad, unqualified, bold claim about AGI tend to be complete nonsense.

Out of curiosity, roughly what probability would you assign to there being an AI financial bubble that pops sometime within the next five years or so? If there is an AI bubble and if it popped, how would that affect your beliefs around near-term AGI?

1. ^{^}

   How is correctness physically instantiated in space and time and how does it physically cause physical events in the world, such as speaking, writing, brain activity, and so on? Is this an important question to ask in this context? Do we need to get into this?
   
   You can take an epistemic practice in EA such as "thinking that Leopold Aschenbrenner's graphs are correct" and ask about the historical origin of that practice without making a judgement about whether the practice is good or bad, right or wrong. You can ask the question in a form like, "How did people in EA come to accept graphs like those in 'Situational Awareness' as evidence?" If you want to frame it positively, you could ask the question as something like, "How did people in EA learn to accept graphs like these as evidence?" If you want to frame it negatively, you could ask, "How did people in EA not learn not to accept graphs like these as evidence?" And of course you can frame it neutrally.
   
   The historical explanation is a separate question from the evaluation of correctness/incorrectness and the two don't conflict with each other. By analogy, you can ask, "How did Laverne come to believe in evolution?" And you could answer, "Because it's the correct view," which would be right, in a sense, if a bit obtuse, or you could answer, "Because she learned about evolution in her biology classes in high school and college", which would also be right, and which would more directly answer the question. So, a historical explanation does not necessarily imply that a view is wrong. Maybe in some contexts it insinuates it, but both kinds of answers can be true. 
   
   But this whole diversion has been unnecessary.

2. ^{^}

   Do you know a source that formally makes the argument that the METR graph is about AGI? I am trying to pin down the series of logical steps that people are using to get from that graph to AGI. I would like to spell out why I think this inference is wrong, but first it would be helpful to see someone spell out the inference they’re making. 

Denkenberger🔸 @ 2025-12-20T08:06 (+2)

Yeah, and there are lots of influences. I got into X risk in large part due to Ray Kurzweil's The Age of Spiritual Machines (1999) as it said "My own view is that a planet approaching its pivotal century of computational growth - as the Earth is today - has a better than even chance of making it through. But then I have always been accused of being an optimist."

I think the question is not 'whether there should be EA efforts and attention devoted to AGI' but whether the scale of these efforts is justified - and more importantly, how do we reach that conclusion.

You say no one else is doing effective work on AI and AGI preparedness but what I see in the world suggests the opposite. Given the amount of attention and investment of individuals, states, universities, etc in AI, statistically there's bound to be a lot of excellent people working on it, and that number is set to increase as AI becomes even more mainstream. 

Teenagers nowadays have bought in to the idea that 'AI is the future' and go to study it at university, because working in AI is fashionable and well-paid. This is regardless of whether they've had deep exposure to Effective Altruism. If EA wanted this issue to become mainstream, well... it has undoubtedly succeeded. I would tend to agree that at this moment in time, a redistribution of resources and 'hype' towards other issues would be justified and welcome (but again, crucially, we have to agree on a transparent method to decide whether this is the case, as OP I think importantly calls for).

 

What percentage chance would you put on an imminent alien invasion and what amount of resources would say is rational to allocate for defending against it? The astrophysicist Avi Loeb at Harvard is warning that there is a 30-40% chance interstellar alien probes have entered our solar system and pose a threat to human civilization. (This is discussed at length in the Hank Green video I linked and embedded in the post.)

It’s possible we should start investing in R&D now that can defend against advanced autonomous space-based technologies that might be used by a hostile alien intelligence. Even if you think there’s only a 1% chance of this happening, it justifies some investment.

And most of what EAs are working on is determining whether we're in that world

As I see it, this isn’t happening — or just barely. Everything flows from the belief that AGI is imminent, or at least that’s there’s a very significant, very realistic chance (10%+) that it’s imminent, and whether that’s true or not is barely ever questioned.

Extraordinary claims require extraordinary evidence. Most of the evidence cited by the EA community is akin to pseudoscience — Leopold Aschenbrenner’s "Situational Awareness" fabricates graphs and data; AI 2027 is a complete fabrication and the authors openly admit that, but it’s not foregrounded enough in the presentation such that it’s misleading. Most stuff is just people reasoning philosophically based on hunches. (And in these philosophical discussions, people in EA even invent their own philosophical terms like "epistemics" and "truth-seeking" that have no agreed-upon definition that anyone has written down — and which don’t come from academic philosophy or any other academic field.)

It’s very easy to make science come to whatever conclusion you want when you can make up data or treat personal subjective guesses plucked from thin air as data. Very little EA "research" on AGI would clear the standards for publication in a reputable peer-reviewed journal, and the research that would clear that standard (and has occasionally passed it, in fact) tends to make much more narrow, conservative, caveated claims than the beliefs that people in EA actually hold and act on. The claims that people in EA are using to guide the movement are not scientifically defensible. If they were, they would be publishable.

There is a long history of an anti-scientific undercurrent in the EA community. People in EA are frequently disdainful of scientific expertise. Eliezer Yudkowsky and Nate Soares seem to call for the rejection of the "whole field of science" in their new book, which is a theme Yudkowsky has promoted in his writing for many years.

The overarching theme of my critique is that the EA approach to the near-term AGI question is unscientific and anti-scientific. The treatment of the question of whether it’s possible, realistic, or likely in the first place is unscientific and anti-scientific. It isn’t an easy out to invoke the precautionary principle because the discourse/"research" on what to do to prepare for AGI is also unscientific and anti-scientific. In some cases, it seems incoherent.

If AGI will require new technology and new science, and we don’t yet know what that technology or science is, then it’s highly suspect to claim that we can do something meaningful to prepare for AGI now. Preparation depends on specific assumptions about the unknown science and technology that can’t be predicted in advance. The number of possibilities is far too large to prepare for them all, and most of them we probably can’t even imagine.

Are you presupposing that good practical reasoning involves (i) trying to picture the most-likely future, and then (ii) doing what would be best in that event (while ignoring other credible possibilities, no matter their higher stakes)?

No, of course not.

I have written about this at length before, on multiple occasions (e.g. here and here, to give just two examples). I don’t expect everyone who reads one of my posts for the first time to know all that context and background — why would they? — but, also, the amount of context and background I have to re-explain every time I make a new post is already high because if I don’t, people will just raise the obvious objections I didn’t already anticipate and respond to in the post.

But, in, short: no.

I'm just very dubious of the OP's apparent assumption that losing such a bet ought to trigger deep "soul-searching". It's just not that easy to resolve deep disagreements about what priors / epistemic practices are reasonable.

I agree, but I didn’t say the AI bubble popping should settle the matter, only that I hoped it would motivate people to revisit the topic of near-term AGI with more open-mindedness and curiosity, and much less hostility toward people with dissenting opinions, given that there are already clear, strong objections — and some quite prominently made, as in the case of Toby Ord’s post on RL scaling — to the majority view of the EA community that seem to have mostly escaped serious consideration.

You don’t need an external economic event to see that the made-up graphs in "Situational Awareness" are ridiculous or that AI 2027 could not rationally convince anyone of anything who is not already bought-in to the idea of near-term AGI for other reasons not discussed in AI 2027. And so on. And if the EA community hasn’t noticed these glaring problems, what else hasn’t it noticed?

These are examples that anyone can (hopefully) easily understand with a few minutes of consideration. Anyone can click on one of the "Situational Awareness" graphs and very quickly see that the numbers and lines are just made-up, or that the y-axis has an ill-defined unit of measurement (“effective compute”, which is relative the tasks/problems compute is used for) or no unit of measurement (just “orders of magnitude”, but orders of magnitude of what?) and also no numbers. Plus other ridiculous features, such as claiming that GPT-4 is an AGI.

With AI 2027, it takes more like 10-20 minutes to see that the whole thing is just based on a few guys’ gut intuitions and nothing else. There are other glaring problems in EA discourse around AGI that take more time to explain, such as objections around benchmark construct validity or criterion validity. Even in cases where errors are clear, straightforward, objective, and relatively quick and simple to explain (see below), people often just ignore it when someone points them out. More complex or subtle errors will probably never be considered, even if they are consequential. 

The EA community doesn’t have any analogue of peer review — or it just barely does — where people play the role of rigorously scrutinizing work to catch errors and make sure it meets a certain quality threshold. Some people in the community (probably a minority, but a vocal and aggressive minority) are disdainful of academic science in general and peer review in particular, and don’t think peer review or an analogue of it would actually be helpful. This makes things a little more difficult.

I recently caught two methodological errors in a survey question asked by the Forecasting Research Institute. Pointing them out was an absolutely thankless task and was deeply unpleasant. I got dismissed and downvoted, and if not for titotal’s intervention one of the errors probably never would have gotten fixed. This is very discouraging.

I’m empathetic to the fact that producing research or opinion writing and getting criticized to death also feels deeply unpleasant and thankless, and I’m not entirely sure on the nuances of how to make both sides of the coin feel rewarded rather than punished, but surely there must be a way. I’ve seen it work out well before (and it’s not like this is a new problem no one has dealt with before).

The FRI survey is one example, but one of many. In my observation, people in the EA community are not receptive to the sort of scrutiny that is commonplace in academic contexts. This could be anything from correcting someone on a misunderstanding of the definitions of technical terms used in machine learning or pointing out that Waymo vehicles still have a human in the loop (Waymo calls it "fleet response"). The community pats itself on the back for "loving criticism". I don’t think anybody really loves criticism — only rarely — and maybe the best we can hope for is to begrudgingly accept criticism. But that involves setting up a social and maybe even institutional process of criticism that currently doesn’t exist in the EA community.

When I say "not receptive", I don’t just mean that people hear the scrutiny and just disagree — that’s not inherently problematic, and could be what being receptive to scrutiny looks like — I mean that, for example, they downvote posts/comments and engage in personal insults or accusations (e.g. explicit accusations of "bad faith", of which there is one in the comments on this very post), or other hostile behaviour that discourages the scrutiny. Only my masochism allows me to continue posting and commenting on the EA Forum. I honestly don’t know if I have the stomach to do this long-term. It's probably a bad idea to try.

The Unjournal seems like it could be a really promising project in the area of scrutiny and sober second thought. I love the idea of commissioning outside experts to review EA research. I think for organizations with the money to pay for this, this should be the default.

Some supporters of AI Safety may overestimate the imminence of AGI. It's not clear to me how much of a problem that is? 

It seems plausible that there could be significant adverse effects on AI Safety itself. There's been an increasing awareness of the importance of policy solutions, whose theory of impact requires support outside the AI Safety community. I think there's a risk that AI Safety is becoming linked in the minds of third parties with a belief in AGI imminence in a way that will seriously if not irrevocably damage the former's credibility in the event of a bubble / crash.

One might think that publicly embracing imminence is worth the risk, of course. For example, policymakers are less likely to endorse strong action for anything that is expected to have consequences many decades in the future. But being perceived as crying wolf if a bubble pops is likely to have some consequences.

Working on AI isn't the same as doing EA work on AI to reduce X-risk. Most people working in AI are just trying to make the AI more capable and reliable. There probably is a case for saying that "more reliable" is actually EA X-risk work in disguise, even if unintentionally, but it's definitely not obvious this is true. 

Thanks, Vasco. 

"AI as normal technology" is a catchy phrase, and could be a useful one, but I was so confused and surprised when I dug in deeper to what the "AI as a normal technology" view actually is, as described by the people who coined the term. 

I think "normal technology" is a misnomer, because they seem to think some form of transformative AI or AGI will be created sometime over the next several decades, and in the meantime AI will have radical, disruptive economic effects.

They should come up with some other term for their view like "transformative AI slow takeoff" because "normal technology" just seems inaccurate.

This is directly answered in the post. Edit: Can you explain why you don’t find what is said about this in the post satisfactory?