Thoughts on this $16.7M "AI safety" grant?

Disclosure: I have past, present, and potential future affiliations with MIT FutureTech. These views are my own.

Thank you for this post. I think it would be helpful for readers if you explained the context a little more clearly; I think the post is a little misleading at the moment. 

These were not “AI Safety” grants; they were for “modeling the trends and impacts of AI and computing” which is what Neil/the lab does. Obviously that is important for AI Safety/x-risk reduction, but it is not just about AI Safety/x-risk reduction and somewhat upstream.

Importantly, the awarded grants were to be disbursed over several years for an academic institution, so much of the work which was funded may not have started or been published. Critiquing old or unrelated papers doesn't accurately reflect the grant's impact.

You claim to have read 'most of their research' but only cite two papers, neither funded by Open Philanthropy. This doesn't accurately represent the lab's work.

Your criticisms of the papers lack depth, e.g., 'This paper has many limitations (as acknowledged by the author)' without explaining why this is problematic. Why are so many people citing that 2020 paper, if it is not useful? Do you do research in this area, or are you assuming that you know what is useful/good research here? (genuine question - I honestly don't know).

By asking readers to evaluate '$16.7M for this work', you imply that the work you've presented was what was funded, which is not the case.

Could you please update your post to address these issues and provide a more accurate representation of the grants and the lab's work?

Now, to answer your question, I personally think the work being done by the lab deserves significant funding. Some reasons:

• I think modeling the trends and impacts of AI and computing is very important and that it is valuable for OP to be able to fund very rigorous work to reduce their related uncertainties. 
• I think that it is very valuable to have respected researchers and institutions producing rigorous and credible work; I think that the impact of research scales superlinearly based on the credibility and rigor of the researchers. 
• The lab is growing very rapidly and attracting a lot of funding and interest from many sources.
• The work is widely cited in policy documents, including, for instance, the 2024 Economic Report of the President.
• The work is widely covered in the media.
• Neil seems to be well respected by those who know him. I joined the lab after I spoke to a range of people I respect about their experiences working with him. Everyone I spoke with was very positive about Neil and the importance of his work. My experiences at the lab have reinforced my perspective. 
• Many of the new and ongoing projects (which I cannot discuss) seem quite neglected and important (e.g., they respond to requests from funders and I don't know of other research on them). I expect they will be very valuable once they are released.
• The lab is interdisciplinary and has a very broad, balanced and integrative approach to AI trends and impacts. Neil has a broad background and knowledge across many domains. This is reflected by how the lab functions; we hire and engage with people across many areas of the AI landscape, from people working on hardware and algorithms, to those working directly on AI risks reduction and evaluation. For instance, see the wide range of attendees at our AI Scaling Workshop (and the agenda). This seems rare and valuable (especially in a place like MIT CSAIL). 

I think it's worth noting that the two papers linked (which I agree are flawed and not that useful from an x-risk viewpoint) don't acknowledge OpenPhil funding, and so maybe the OpenPhil funding is going towards other projects within the lab. 

I think that Neil Thompson has some work which is pretty awesome from an x-risk perspective (often in collaboration with people from Epoch):

• Algorithmic progress in language models
• Economic impacts of AI-augmented R&D
• The growing influence of industry in AI research

From skimming his Google Scholar, a bunch of other stuff seems broadly useful as well.

In general, research forecasting AI progress and economic impacts seems great, and even better if it's from someone academically legible like Neil Thompson. 
 

defun @ 2024-07-17T16:25 (+5)

• Algorithmic progress in language models

This paper is from Epoch. Thompson is a "Guest author".

• Economic impacts of AI-augmented R&D
• The growing influence of industry in AI research

I think this paper and this article are interesting but I'd like to know why you think they are "pretty awesome from an x-risk perspective".

---

Epoch AI has received much less funding from Open Philanthropy ($9.1M), yet they are producing world-class work that is widely read, used, and shared.

PeterSlattery @ 2024-07-17T17:35 (+7)

This seems misleading. Some of the authors are from Epoch, but there are authors from two other universities on the paper. 

Also, where does it say that he is a guest author? Neil is a research advisor for Epoch and my understanding is that he provides valuable input on a lot of their work. 
 

defun @ 2024-07-17T17:52 (+3)

Sorry, I should have attached this in my previous message.

where does it say that he is a guest author?

Here.

PeterSlattery @ 2024-07-17T18:07 (+11)

Thanks. My impression is that they are using 'Guest author' on their blog post to differentiate who works for Epoch or is external. As far as I can tell, that usage implies nothing about the contribution of the authors to the paper.

Rebecca @ 2024-07-18T06:00 (+2)

Yeah this seems the most straightforward interpretation

calebp @ 2024-07-17T19:10 (+4)

I think it's worth noting that the two papers linked (which I agree are flawed and not that useful from an x-risk viewpoint)

 

I haven't read the papers but I am surprised that you don't think they are useful from an x-risk perspective. The second paper "A Model for Estimating the Economic Costs of Computer Vision Systems that use Deep Learning" seems highly relevant to forecasting AI progress which imo is one of the most useful AIS interventions.

The OP's claim 

This paper has many limitations (as acknowledged by the author), and from an x-risks point of view, it seems irrelevant.

Seems overstated and I'd guess that many people working on AI safety would disagree with them. 

 

defun @ 2024-07-17T19:39 (+5)

Hi calebp.

If you have time to read the papers, let me know if you think they are actually useful.

Zach Stein-Perlman @ 2024-07-16T21:07 (+4)

Setting aside whether Neil's work is useful, presumably almost all of the grant is for his lab. I failed to find info on his lab.

peterbarnett @ 2024-07-16T21:19 (+5)

I would guess grants made to Neil's lab are referring to the MIT FutureTech group, which he's the director of. FutureTech says on its website that it has received grants from OpenPhil and the OpenPhil website doesn't seem to mention a grant to FutureTech anywhere, so I assume the OpenPhil FutureTech grant was the grant made to Neil's lab. 

defun @ 2024-07-16T21:12 (+5)

It's MIT FutureTech: https://futuretech.mit.edu/

Zach Stein-Perlman @ 2024-07-16T21:30 (+4)

Thanks.

I notice they have few publications.

I really liked MIT FutureTech's recent paper, "Beyond AI Exposure: Which Tasks are Cost-Effective to Automate with Computer Vision?" I think it's among the 10 best economics of AI papers I've read from the last few years. It proposes an economic model of the circumstances under which companies would automate human labor with AI. 

Previously, some of the most cited papers on the potential impacts of AI automation used an almost embarrassingly simple methodology: surveys. They take a list of jobs or tasks, and survey people about whether they think AI could someday automate that job or task. That's it. For validation, they might cross reference different people's estimates. Their conclusion would be something like "according to our survey, people think AI could automate X% of jobs." This methodology has been employed by some of the highest profile papers on the potential economic impact of AI, including this paper in Science and this paper with >10K citations. 

(Other papers ignore the micro-level of individuals tasks and firms, and instead model the macroeconomic adoption of AI. For example, Tom Davidson, Epoch, Phil Trammell, Anton Korinek, William Nordhaus, and Chad Jones have done research where they suppose that it's cost-effective for AI to automate a certain fraction of tasks. This macro-level modeling is also valuable, but by ignoring the choices of individual firms to automate individual tasks, they assume away a lot of real world complexity.) 

The MIT FutureTech paper significantly improves upon the survey method by creating a mathematical model of what it would cost for a firm to automate a task with AI. The basic premise is that a firm will automate human labor with AI if the human labor is more expensive than AI automation would be. To estimate the cost of AI automation, they break down the costs of automation into the following parts:

Then they estimate distributions for each of these parameters, and come up with an overall distribution for the cost of AI automation. They compare the distribution of AI automation costs to the distribution of human wages in tasks that could be automated, and thereby estimate which tasks it would be cost-effective to automate. This allows them to make conclusions like "X% of tasks would be cost-effective to automate." 

There's a lot of detail to the paper, and there are plenty of reasonable critiques one could make of it. I don't mean to endorse the other sections or the bottom-line conclusions. But I think this is clearly the state of the art methodology for estimating firm-level adoption of AI automation, and I would be excited to see future work that refines this model or applies it to other domains. 

More broadly, I've found lots of Neil Thompson's research informative, and I think FutureTech is one of the best groups working on the economics of AI. I am surprised at the size of the grant, as I'd tend to think economics research is pretty cheap to fund, but I don't know the circumstances here. 

(Disclosure: Last summer I applied for an internship at MIT FutureTech.) 

...huh, I usually disagree with posts like this, but I'm quite surprised by the 2022 and 2023 grants.