Metaculus Questions Suggest Money Will Do More Good in the Future

It's worth pointing out that these questions apply specifically to global health and development, but could be very different in other cause areas.

I don't think question 1 provides evidence that money will do more good in the future. It might even suggest the opposite: As you point out, malaria prevention and deworming might run out of room for more funding, and to me this seems more likely than the discovery of a more cost-effective option that is also highly scalable (beyond >$30 million per year).

Relevant: https://www.metaculus.com/questions/1332/will-global-malaria-mortality-rates-be-reduced-by-90-when-compared-with-2015-rates-by-2030/

Metaculus users give a 52% chance that global malaria deaths will be 90% lower in 2030 than they were in 2015. If that is the case, then I would expect their now highly rated malaria charities to be less cost effective than they are now.

For the first question, I was one of the forecasters who gave close to the current Metaculus median answer (~30%). I can't remember my exact reasoning, but roughly:

1. Outside view on how frequently things have changed + some estimates on how likely things are to change in the future, from an entirely curve fitting perspective.

2. Decent probability that the current top charities will go down in effectiveness as the problems become less neglected/we've had stronger partial solutions for them/we discover new evidence about them. Concretely:

Malaria: CRISPR or vaccines.  But also I place decent probability on bednet production and distribution being fully solved by states/international actors/large NGOs.
Deworming:  Possibilities include a) we uncover new evidence that suggests deworming is less effective than we previously thought, especially at 2021/2030 worm loads* or b) mass deworming decreases the total amount of worms, decreasing marginal value, or c) both.
Cash: At GiveWell scale, I don't think direct cash transfers is ever competitive with your current best guess for top health/development interventions under naive cost-effectiveness unless you apply a strong robustness penalty to everything else. 
Vitamin A: haven't thought much about it tbqh, so retreat to priors.
 

*The evidentiary base for deworming was always shaky to begin with, I think (within the randomista paradigm) it's reasonable to model deworming as a relatively high-risk high-reward economic intervention, and we may eventually discover evidence that conclusively demonstrates that deworming isn't economically that effective.

This could happen for two reasons: either the 2019 recommended charities run out of room for more funding, or GiveWell finds a charity that's better than any of the 2019 recommendations.

I think the third reason that you mention later is more likely, namely that "Perhaps instead Metaculus expects GiveWell's estimate is currently too pessimistic, and it will converge on the true answer by 2031."

This seems most likely to me since my guess is that the reason why GiveWell's estimate changed from $890 in 2016 to $436 in 2019 is because their methodology or something else changed (e.g. 2019-GiveWell thought 2016-GiveWell's way of estimating the "cost to get an outcome as good as saving a life" over-estimated the true cost). I don't think they actually believe the best giving opportunity was twice as cost-effective in 2019 as 2016. (I don't actually know this for sure, so if you or someone else knows better, please correct my guess.)

This at least weakly suggests that Metaculus users expect GiveWell to improve its recommendations over time.

Does it? If "the 2019 recommended charities run out of room for more funding" this could be because e.g. governments or billionaires have stepped in to make sure that every child who needs one has a bed net and all deworming efforts have been fully funded forever. It seems to me that this would mean that the marginal philanthropist no longer has access to this low-hanging fruit. Since these opportunities would be fully-funded, GiveWell would then have to recommend the next best thing, which would be less cost-effective in expectation, not more.

While my comments reveal that I disagree with the inferences you make in this post, I just want to say that I think this sort of analysis is really cool and I appreciate your thoughtfulness with coming up with these questions and writing this post. Your writings have influenced my thinking and changed my view on several occasions and I look forward to reading more of what you have to say in the future.

Cool questions! I am a bit hesitant to update much:

• they don’t seem to be too active, e.g. few comments, interest count at around 10 (can you see the number of unique forecasters somehow?)
• the people doing the forecasts are probably EA adjacent and if they did something akin to a formal analysis, they would share it with the EA community, or at least in the comments, as it seems relatively useful to contribute this

Is there some kind of up-to-date dashboard or central source for GiveWell's main "cost-per-expected-life" figure? 

• The Metaculus question mentioned in this post cites values like $890 in 2016,  $823 in 2017, $617 in 2018 and $592 in 2019, and I can't find the field they refer to in the resolve condition (?!)
• This 80K article lists the value as $2300 in 2020.
• This GiveWell summary sheet from 2016 has a minimum value of $901
• GiveWell's Top Charities page lists $3000-$5000 to save a life for Malaria Consortium, Against Malaria Foundation, New Incentives, and Hellen Keller International.

If such a thing does not exist, I'll probably reach out to GiveWell and see what they think about implementing one. There are so many numbers floating around that are hard to verify and differ dramatically.

I mistakenly submitted this as a question instead of as a post. Is there any way to convert it to a post?

For comparison, in 2019, GiveWell estimated that its most cost-effective charity spends $592 per life saved equivalent. (These figures are adjusted for inflation.)

I think you said in 2020 the $592 number was a mistake:

Correction: I copy/pasted the cost per life saved numbers from the previous question, but the $592 number for 2019 is as of the middle of the year. The end-of-year estimate (adjusted to 2015 dollars) is $436.

$436 is a lot closer to $430 than $592.

Meta question: Is there a reason you made this post a Question post rather than a regular post? Seems like it should be a regular post.

Givewell charities are expected to increase local GDP, and have other effects which are expected to improve wellbeing over the long run. Starting these 2nd order effects sooner can offset the additional cost of doing them now (if that exists).

I bought the Metaculus predictions (using my Tachyon points on Metaculus) for each of the two questions in your post as well as the malaria question Charles Dillon linked to.

For those unfamiliar, the Metaculus prediction has a better track record than the community prediction (public to everyone, which Michael cited in his post as what Metaculus thinks) over questions on the Metaculus site, however is not generally visible until forecasts on a question are closed.

Knowing the Metaculus predictions does not seem very insightful to me. Nevertheless, if anyone would care to know them, feel free to DM me.

Metaculus estimates that GiveWell's top charity in 2031 will need to spend $430 per life saved equivalent (according to GiveWell's own analysis).

Assuming I'm not mistaken about how Metaculus works, $430 is the median of recent forecasters' median forecasts on this question. But don't we really care more about the mean expected cost-effectiveness than the median? (I could be wrong about this since after all (I think) GiveWell's estimate is the median output of their staff members' inputs into their cost-effectiveness model.)

Metaculus doesn't give the mean value, but looking at the shape of the probability density distribution on the Metaculus question, it looks to be roughly symmetrical about the median and given that the axis is logarithmic, I think this means the mean value is greater than the median.

So assuming the mean is what we care about, then it seems that Metaculus community prediction is greater than GiveWell's 2019 estimate, suggesting that money (given to GiveWell charities) will do less good in the future rather than more.