The Prospect of an AI Winter

Algorithmic progress on ImageNet seems to effectively halve compute requirements every 4 to 25 months (Erdil and Besiroglu 2022); assume that the doubling time is 50% longer for transformers.

I think it's important not to take the trend in algorithmic progress too literally. At the moment, we only really know the rate for computer vision, which might be very different than for other tasks. The confidence interval is also quite wide, as you mentioned (the 5th percentile is 4 months and 95th percentile is 25 months). And algorithmic progress is plausibly driven by increasing algorithmic experimentation over time, which might become bottlenecked after either the relevant pool of research talent is exhausted or we reach hardware constraints. For these reasons, I have wide uncertainty regarding the rate of general algorithmic progress in the future.

In my experience, fast algorithmic progress is often the component that yields short timelines in compute-centric models. And yet, both the rate and the mechanism behind algorithmic progress is very poorly understood. Extrapolating this rate naively gives a false impression of high confidence in the future, in my opinion. Assuming that the rate is exogenous gives the arguably false impression that we can't do much to change it. I would be very careful before interpreting these results.

Nicely written, these make a lot of sense to me. My case for AI winter would focus on two tailwinds that will likely cease by the end of the decade: money and data. 

1. Money can't continue scaling like this. Spending on training runs has gone up by about an order of magnitude every two years over the last decade. By 2032 this trend would put us at $100B training runs, which would be 3x Google's entire R&D budget. If TAI doesn't emerge before then, spending growth will need to slow down, likely slowing AI progress as well. 
2. Maybe data can't either. Full analysis here, but basically high quality language data will run out well before 2030, possibly within the next year or two. But there are other kinds of data that could continue scaling. For example, the Epoch report discusses low quality language data like private texts and emails as one possibility. I would look more to the transition from language models to multimodal vision-and-language models as an important trend not only because vision is a useful modality, but because it would allow data scaling to continue.

I'd like to have a better view of questions about the continuation of Moore's Law. Without a full writeup, the claims about Moore's Law ending seem more credible now than in the past. I would be really interested in an aggregation of historical forecasts about Moore's Law to see whether the current doomsaying is any different from the long run trend. 

First of all, great post, thanks for exploring this topic! 

So I'm a little confused about the definition here:

AI winter is operationalised as a drawdown in annual global AI investment of ≥50%

 I would guess that the burst of the dot-com bubble meets this definition? But I wouldn't exactly call 2002-2010 an "internet winter": useage kept growing and growing, just with a better understanding of what you can and can't profit from. I think theres a good chance (>30%) of this particular definition of "AI winter" occurring, but I also reckon if it happens, people will feel like it's unfair to characterize it as such. 

I think a more likely outcome is a kind of "AI autumn": Investment keeps coming at a steady rate, and lots and lots of people are using AI for the things it's good at, but the number of advancements slows significantly, and certain problems prove intractable, and the hype dies down. I think we've already seen this process happen for Autonomous Vehicles. I think this scenario is very likely. 

Nice post, found this pretty well written and convincing (though I already shared the bottom line, just less firmly). 

Random thoughts:

A severe extreme geopolitical tail event, such as a great power conflict between the US and China, may occur.

What type of great power conflict do you have in mind here? "Extreme tail event" makes it sound like you're thinking of a fairly large scale war, but great power conflict seems to refer to any military confrontation. E.g. I haven't at all wrapped my head around a military confrontation between China and the US over Taiwan yet, and Metaculus is at ~20% for 

Will armed conflicts between the Republic of China (Taiwan) and the People's Republic of China (PRC) lead to at least 100 deaths before 2026?

Also, I wonder if you have considered any potential craziness that happens after conditional on development of TAI before 2030. E.g. say TAI is developed in 2027, maybe the plausible set of scenarios for 2028 and 2029 include sufficiently many scenarios where we see a >50% decrease in AI funding such that I might want to increase your bottom line forecast?