Questions on databases of AI Risk estimates

By Froolow @ 2022-10-02T09:12 (+24)

I was hoping to write something for the Future Fund contest and - being entirely a one-trick pony - was going to look at uncertainty analysis in AI Catastrophe predictions. 

I've done a review of the forums and my conclusion is that predictions around AI Catastrophe are very heavily focussed on when AI will be invented and the overall top-level probability that AI will be a catastrophe if it is invented. Other than that predictions about AI Risk are quite sparse. For example, few people seem to have offered a numerical prediction about whether they think the AI Alignment Problem is solvable in principle, few people have offered a numerical prediction about the length of time we could contain a misaligned AI and so on. The only end-to-end model of AI Risk with numerical prediction I have found is Carlsmith (2021): https://arxiv.org/abs/2206.13353. 

In my review I came across the 'Database of Existential Risk Estimates' - link here: https://forum.effectivealtruism.org/posts/JQQAQrunyGGhzE23a/database-of-existential-risk-estimates. This seems to contain many estimates of exactly what I am looking for - predictions of specific events which will occur on the path to an AI catastrophe, rather than the overall risk of catastrophe itself.

Thanks so much!


Erich_Grunewald @ 2022-10-02T10:05 (+7)

Am I right that Carlsmith (2021) is the only end-to-end model of AI Risk with numerical predictions at each stage (by end-to-end I mean there are steps in between 'AI invented' and 'AI catastrophe' which are individually predicted)? Any other examples would be really helpful so I can scope out the community consensus on the microdynamics of AI risk.

This spreadsheet (found here) has estimates on the propositions in Carlsmith by (some of?) the reviewers of that paper.

Froolow @ 2022-10-02T17:34 (+3)

This is absolutely incredible - can't believe I missed it! Thank you so much

Erich_Grunewald @ 2022-10-02T18:33 (+1)

I'm excited to see what you come up with!

NunoSempere @ 2022-10-04T12:13 (+6)

See also: Michael Aird's database of existential risk estimates, searchable @ https://metaforecast.org/?forecastingPlatforms=xrisk 

NunoSempere @ 2022-10-04T12:12 (+6)

You might also find use in David Manheim's Modeling Transformative AI Risk (MTAIR) sequence. And you might also want to ask the LessWrong.com forum  this same question.

Froolow @ 2022-10-04T13:11 (+1)

Thank you so much for the links - the Manheim work in this particular one is absolutely spectacular

NunoSempere @ 2022-10-04T12:21 (+2)

I've spammed you with a few sources. But even though they do exist, they are fairly scattered and my sense is still that your impression is right that there aren't many such models. 

David Manheim would know better, so I recommend you check with him.

NunoSempere @ 2022-10-04T12:18 (+2)

Se also: <https://www.openphilanthropy.org/research/semi-informative-priors-over-ai-timelines/>

NunoSempere @ 2022-10-04T12:17 (+2)

Se also: <https://www.openphilanthropy.org/research/semi-informative-priors-over-ai-timelines/>

NunoSempere @ 2022-10-04T12:14 (+2)

You might also want to check AI impacts' work on AI timelines https://aiimpacts.org/category/ai-timelines/, particularly their surveys: https://aiimpacts.org/category/ai-timelines/predictions-of-human-level-ai-timelines/ai-timeline-surveys/ 

Misha_Yagudin @ 2022-10-02T18:25 (+2)

Nicole Noemi gathers some forecasts about AI risk (a) from Metaculus, Deepmind co-founders, Eliezer Yudkowsky, Paul Christiano, and Aleja Cotra's report on AI timelines.

h/t Nuño

Froolow @ 2022-10-03T13:31 (+1)

Thank you, really appreciate the information