What are the numbers in mind for the super-short AGI timelines so many long-termists are alarmed about?
By Evan_Gaensbauer @ 2022-04-19T21:09 (+41)
The proportion of long-termists in effective altruism who are expressing confident convictions that the timeline for smarter-than-human AI is much shorter than has previously been predicted been increasing at an accelerating rate in the last year. This appears to be a shift in perspective among several hundred long-termists. Among the dozens I've read opinions from, numbers are almost never provided.
I don't check the platforms for the best AI forecasting or whatever much, so maybe there are models or predictions that make clear the quantitative estimates for these timelines. Please comment or answer with such a resource if you're aware of one.
Otherwise, based on the way different people are talking about it, I wouldn't be surprised if they thought the timeline is 10-20 years, or 5-10 years, or even 2-3 years. I've talked to others who are also concerned and open-minded to this or that short AI timeline but haven't done the research themselves yet, or had much opportunity to learn from those who have. We want to understand better but basic information crucial to understanding more like numbers for different models or timelines aren't being presented. We want to know and need to know help better. What are the numbers?
nicolenohemi @ 2022-04-20T04:37 (+81)
Disclaimer: Be careful about definitions and interpreting metaculus questions. The latter involves resolution criteria on defining AGI that does not align with my own (e.g. meeting the named criteria would not replace all human tasks). Also, there has been an inflow of additional forecasters based on recent developments which should be factored in.
I listed some of my current sources down below. I hope this helps!
- Metaculus forecasts:
- Date of general AI: lower 25%: 2032, Median: 2045, upper 75%: 2067
- Date Weakly General AI is Publicly Known: lower 25%: 2026, median: 2032, upper 75%: 2044
- Date AIs Capable of Developing AI Software: lower 25%: 2026 median: 2032, upper 75%: 2045
- Other:
- AI Progress Essay Contest (lists relevant questions)
- More x-risk relevant prediction market question suggestions to look out for
- General accuracy evaluation of recently resolved Metaculus community predictions on AI timelines by Marius Hobbahn
- Shane Legg (DeepMind Co-founder): 50%: 2030, some chance in next 10-30y
- Demis Hassabis: 10-20y from now
- Eliezer & Paul’s IMO challenge bet: “Paul at <8%, Eliezer at >16% for AI made before the IMO is able to get a gold (under time controls etc. of grand challenge) in one of 2022-2025. Separately, we have Paul at <4% of an AI able to solve the "hardest" problem under the same conditions.”
- Eliezer:
“My probability is at least 16% [on the IMO grand challenge falling], though I'd have to think more and Look into Things, and maybe ask for such sad little metrics as are available before I was confident saying how much more.” - Paul Christiano:
"I'd put 4% on "For the 2022, 2023, 2024, or 2025 IMO an AI built before the IMO is able to solve the single hardest problem"
I think the IMO challenge would be significant direct evidence that powerful AI would be sooner, or at least would be technologically possible sooner. I think this would be fairly significant evidence, perhaps pushing my 2040 TAI probability up from 25% to 40% or something like that."
- Eliezer:
- Ajeya's "When required computation may be affordable" (from ACT):
- Ajeya created a function to evaluate the predicted annual investments on giant AI projects (upward sloping curve), vs. the likely cost of training a human-level AI (downward sloping curve).
- Eventually, these curves meet, representing the first trained human-level AI.
- You can play around with the spreadsheet here.
- Ajeya’s values: 20% neural net, short horizon, 30% neural net, medium horizon, 15% neural net, long horizon, 5% human lifetime as training data, 10% evolutionary history as training data, 10% genome as parameter number
- Just for fun: Annual Reddit singularity predictions