In current EA, scalability matters

By Peter Wildeford @ 2022-03-03T14:42 (+108)

Summary: A less cost-effective opportunity that is more scalable can be better than a more cost-effective but less scalable opportunity.

That is, in the current effective altruism movement with lots of funding, to maximize the total effectiveness of the entire EA portfolio, scalability should be prioritized on the margin and cost-effectiveness should become more of a bar to meet than something to maximize.

This is what motivates a lot of the current focus on highly scalable projects (megaprojects).

The Case

We're now in a world where effective altruism funding is definitely very plentiful[1]. At least for now, total available funding seems to currently exceed total available fundable opportunities[2].

This implies two things:

(1) When looking for new opportunities, a less cost-effective (in terms of social good per dollar spent) opportunity that is more scalable (in terms of total dollars that can be spent to achieve the target cost-effectiveness) can sometimes be more exciting and more helpful to the overall EA portfolio than a more cost-effective but less scalable opportunity.

(2) Cost-effectiveness still matters, but requires us either to threshold fund everything above a certain bar (e.g., everything that can be about as good as Against Malaria Foundation[3]) or identify very scalable opportunities that can take billions of dollars on the margin at a higher bar.

My sense is that the earlier effective altruism movement of 2010-2014 spent all their time aiming to find opportunities that maximized cost-effectiveness per dollar without caring much about scalability (e.g., "how to do the most good with your limited money"), whereas if the above is right we need to shift to caring about scalability much more and use cost-effectiveness more as a threshold (e.g., identify very scalable opportunities that are as good as AMF or better)[4].

An Example

Imagine that we had these five projects (and only these projects) in the EA portfolio:

EA as of 2010-2014, with relatively fewer resources (we didn't have $100M to spend), would've been ecstatic about Alpha because it only costs $100 to buy one unit of impact, which is much better than Beta's $500 per unit, GiveDeltaly's $2000 per unit, or Gamma's $3333.33 per unit.

But "modern" EA, with lots of money and a shortage of opportunities to spend it on would gladly buy Alpha first but would be more excited by Beta because it allows us to deploy more of our portfolio at a better effectiveness.

Note though that no one in "modern EA" would be excited by Gamma - even though it's a huge megaproject and very scalable, it doesn't beat our baseline of GiveDeltaly.

~

...Now let's think of things as allocating an EA bank account and use Research. What should we use Research for? Early EA would want us to focus our research efforts on finding another opportunity like Alpha since it is very cost-effective! But modern EA would rather we look for opportunities like Beta - even though it is less effective than Alpha, it can use up 1000x more funds!

Like say we have an EA bank account with $2,000,000,000. If we followed modern EA advice and bought Alpha, bought Beta, bought Research and used it to find another Beta, and bought the second Beta, and then put the remainder into GiveDeltaly[5], we'd have 1,350,350 units of impact.

But if we followed Early EA advice and bought Alpha, bought Beta, bought Research and used it to find another Alpha, and bought the second Alpha, and then put the remainder into GiveDeltaly, we'd have 1,151,800 units of impact. Lower total impact even though we used research to find a more cost-effective intervention!

This implies the scalability of the projects we identify can matter just as much, if not more than, the cost-effectiveness of the project! I think this scalability mindset is often missed by people who focus mainly on cost-effectiveness and is the main reason IMO to think more about megaprojects.

But this does also imply that scalability isn't the only thing that matters - no one wants to spend a dollar on Gamma even though it is very scalable.

Notes

This post was originally this comment. I'm turning it into a top-level post and expanding upon it briefly.

This is just my personal opinion and not necessarily the opinion of anyone else at Rethink Priorities. This post was not seen by anyone else at Rethink Priorities prior to me posting.

On March 3 at 12:36PM CT I added a few edits to improve clarity, including adding a rephrasing suggested by Stefan Schubert.

  1. I don't think we have properly internalized how much funding is currently available. Ben Todd in "Is effective altruism growing? An update on the stock of funding vs. people" quotes EA as having $46B and growing 37% per year, and this is now actually looking like it could be an underestimate. A $46B portfolio would be enough to launch two Manhattan Project-level initiatives at the level of spending of the original Manhattan Project (after adjusting for inflation)! ↩︎

  2. Though I will stop short of saying that everything that should get funded does get funded, since there do appear to be some gaps that still need to be ironed out. ↩︎

  3. It's not actually all that clear yet where we should set the bar. GiveWell is currently aiming for >=8x cash transfers (e.g., GiveDirectly). It makes sense our bar should be better than GiveDirectly since otherwise it does seem like we literally could spend billions of dollars on GiveDirectly if we wanted to. This bar is a little below the "Against Malaria Foundation or better" bar but the AMF bar is a bit easier to talk about and reason about. It's possible the actual bar could be even lower, given GiveWell has currently only identified $400 million in 8x or better opportunities and we have billions to give. I think this bar also holds even if you include animal welfare and longtermist opportunities - which GiveWell currently does not - since there are not yet multi-billion dollar opportunities in either of those areas. ↩︎

  4. One personal hot take I am currently saving for those of you who read footnotes but that I'd like to think more about - I think this implies that longtermist funders who mainly want to improve the long-run future still should fund a ton of animal welfare and global health/development (GHD) stuff because doing so still meets any plausible bar that EA can set given the tremendous amount of capital we have. That is, there just aren't enough longtermist opportunities to meet all this funding. I'd understand longtermists passing on animal/GHD stuff if funding was more constrained or if we do identify longtermist stuff that is just massively scalable such that there's nothing left over for animal/GHD stuff - and maybe it is reasonable to wait a year or more and save money to see if that does become the case. ↩︎

  5. Of course not in this example is the opportunity to save money for the future after we do more research. This likely could make the value of the portfolio higher, even though it would take longer to deploy. I leave this out now for simplicity. ↩︎

Ardenlk @ 2022-03-04T16:14 (+27)

FWIW the way I conceptualise this situation is that cost effectiveness is still king, but: spending a dollar is a lot less expensive in terms of 'true cost' than it used to be, because it implies the inability to fund another thing to a greatly reduced extent (which is the real cost of spending money).

This in turn means that spending time/labour to find new opportunities is relatively more expensive than it used to be vs. the true cost of spending a dollar, which is why we want to take opportunities that have a much larger dollar spend:labor/time ratio than we used to.

If an opportunity is not scalable, that means it has a lot of labour/time costs that are hidden, because once you use up the opportunity you have to find another one before you can keep having impact, which costs labour/time, whereas scalable opportunities don't have that. Therefore they're cheaper in true cost, therefore more cost effective at the same level of effectiveness.

I don't think I'm disagreeing with you -- but this feels like the conceptually cleaner way of thinking about it for my brain.

omernevo @ 2022-03-03T19:05 (+25)

I generally agree with the idea and appreciate the clarity of this post.

 

One related thought which I think is potentially useful both for thinking of which projects to fund or to start:

Projects usually need to be scalable at advanced stages, but not at the start. It's ok (and even recommended* in many cases) to start doing things in non-scalable ways that aren't cost-effective.
A lot of times, the value in information \ experience \ growth is high enough that it's worth starting out doing things that you won't be able to sustain as you grow.


Obviously, there should be a plan (or at least ideas on how) to become more scalable later. I'd be looking for projects that have reasonable path(s) to being very scalable down the road.

 

* This link is advice for for-profit startups. It's only partially relevant for our context but the point I'm making is made there in more detail.

Peter Wildeford @ 2022-03-03T19:09 (+4)

This is an important clarification - thanks!

Michael_Wiebe @ 2022-03-03T17:24 (+13)

When looking for new opportunities, a less cost-effective (in terms of social good per dollar spent) opportunity that is more scalable (in terms of total dollars that can be spent to achieve the target cost-effectiveness) can sometimes be more exciting and more helpful to the overall EA portfolio than a more cost-effective but less scalable opportunity.

In the ITC framework, this is captured by diminishing returns. To optimally allocate resources, you give your next dollar to the intervention with the highest marginal utility per dollar. This means funding the low-scale intervention until its MU/$ is below that of the high-scale intervention, and then switching to allocating your next dollar to the high-scale intervention.

Restating your point: if you have a huge budget, then you need to have scalable opportunities (ie. with low diminishing returns) in order to spend your whole budget. There might be a bunch of small interventions (ie. fully funding them would use up 0.0000001% of your budget) with the highest MU/$, but if there are transaction costs to identifying and funding them, it could be optimal to ignore them and focus on more scalable interventions.

evelynciara @ 2022-03-03T20:12 (+11)

I think two things would help clarify the thought experiment:

Stefan_Schubert @ 2022-03-04T10:24 (+8)

We are trying to maximize the area under the curve.

There's another criterion also: the cost-effectiveness has to be over the bar. You could have, e.g. a dashed or coloured line for the bar.

Peter Wildeford @ 2022-03-04T02:29 (+5)

That's a really good idea, but I don't think I should prioritize doing that. If you or someone else wanted to supply the relevant images though, I'd be happy to include them.

SjirH @ 2022-03-09T21:08 (+7)

I would like to push back a bit, as I don't think it's true that scalability per se matters more now than it did in the past.

Instead, I think the availability of more funding has pushed down the cost-effectiveness bar for funding opportunities, thereby "unlocking" some new worthy funding opportunities, including some very scalable ones.

To see this, consider that the added value of discovering/creating any new funding opportunity for the community is roughly given by (not accounting for diminishing returns when spending at bar level):

"value created by adding a new funding opportunity" = ("average cost-effectiveness of the opportunity" - "current cost-effectiveness bar") * "room for funding of the opportunity"

I.e. what you're effectively doing by adding a new opportunity is improving the cost-effectiveness of money that would have otherwise been spent at the bar level.

This implies that any opportunity that is above the current bar in terms of its cost-effectiveness can be worth discovering if it's scalable enough. But that is nothing new: it was true as much in 2010-2014 as it is now. It's just that the bar was higher, so some very scalable but below-bar opportunities weren't worth discovering back then but are now.

SjirH @ 2022-03-09T21:09 (+6)

I also think it's worth stressing that the best alternative to finding a great (above-bar) option to spend money on now is not to spend on options below the bar, but to wait / keep looking and spend it at an above-bar opportunity later (and ideally invest to give while you're at it).

In your example, this cashes out (roughly) in us using Research multiple times to find as many Alpha-like projects as possible and fund those, and to only start looking for and funding Beta-like projects when there are no more Alpha-like projects to find. Even if there is only one extra Alpha and one extra Beta to find, it's better (with the parameters as provided in your example) to find and fund that Alpha and find and fund Beta, than to find at fund only one of the two.

Cases somewhat akin to "you can only use Research for either Alpha or Beta" can occur, but only under very specific conditions, e.g. when opportunities are time-sensitive and/or when there is a very tight bottleneck on research resources (=strongly increasing marginal costs to doing research), which might in fact be the case currently.

(As a side point: given the option of investing to give, it's important to "set" the bar taking into account our expectations of how cost-effective future opportunities will be, investment benefits one can achieve in the meantime, value drift and expropriation risks etc.)