Why we look at the limiting factor instead of the problem scale

By Joey 🔸 @ 2019-01-28T19:12 (+64)

Scale, or importance, is held as one of the 3 criteria to consider when evaluating an intervention for promisingness. With the idea being that large scale problems might suggest area area will be more effective to work, on assuming it also scores well on the other criteria. Some interventions are predicated on very strong scale arguments, such as far future or wild animal suffering. However, we (Charity Entrepreneurship) have found that scale specifically is quite a poor indicator of the promisingness of an area.

Organizations tend to be incentivised  to scale both from an impact perspective and from a personal perspective. However, all organizations, eventually, hit a limiting factor that makes it hard to scale faster. For some organizations, it might be the total scale of the issue. For example, perhaps when working on a nearly eradicated disease, the limiting factor might just be how much more of the problem there is left to deal with. This would be a case when the traditional use of scale comes to almost the same result as the limiting factor model. Both models suggest this would not be a great area to work on due to the problem scale being quite small (currently).

However, there are many times when it diverges. For example, certain issues might have a massive problem scale, but be quickly limited by some other factor. Givewell has talked about surgeries being limited by the supply of surgeons - this is not a scale of problem issue specifically, but it is a limiting factor. A scale model might suggest that if there is a ton of surgeries still to be done then this is a worthwhile issue to focus on. But a limiting factor model would suggest that it would be quickly capped by the number of surgeons. Below is a very simplified comparison.

Scale model

Limiting factor model

As you can see, the results end up being quite different: surgeries are limited by the talent pool far earlier than vaccinations become limited by anything else. The magnitudes of the categories were set to be more cross-comparable (e.g. 1 million compares to 10 full-time staff). And both get limited several times before their problem size limit. In this case, it would not even matter if surgeries had 10 times the scale (say, 200 million people affected) if they will be stopped by talent, logistics, and funding before they can even help 20 million people. These numbers are estimates, but given our work and research in these areas we are confident both of these will hit a limiting factor far below their problem size (or what is often referred to as ‘Scale or importance’).

A common response is that scale is only one of the factors considered when evaluating an intervention. However, things like ‘logistical limit’ fall under the radar of tractability/solvability, and there are two problems with this. First, tractability is not really currently used in this way. Right now, lots of claims are being made along the lines of “cause X should be focused on more due to it having a huge problem size” with no further reference to tractability. Secondly, tractability is often seen as the speed at which you will make progress as opposed to a specific factor that will stop growth from occurring. For example, an intervention could be very shovel ready, but only at a small scale before its limiting factor comes into play. If we go back the surgery charity example, it could be that 3 surgeons want to start a charity, and for them it is very tractable and shovel ready, since their marginal effort is great and, at a very limited scale, their solvability rate is very high. The endeavour will run into specific “scale” issues involving a limiting factor of ‘hiring other surgeons’, And this ends up feeling more like a scale issue rather than a tractability one.

Another common concern might be funding limits, which on their own is are not hard limits like problem size. However, while something like a funding limit can be improved upon with more fundraising and field building, this is not an easy task. For the money ranking, I do not think you would just put down the amount you have fundraised for the area, but a reasonable bound for how much could be fundraised, while taking into account the current donor space and a reasonable amount of time (e.g. 2-5 years). This can change over time, but so can the size of the problem: factory farming is a growing problem and global poverty is a shrinking one, but that does not change the importance of having sense of their scale.

A claim that I hear a lot in the animal space is that Wild animal suffering is such a huge scale problem that we should seriously consider working on it. Many people would suggest that Wild animal suffering is an issue with a much larger scale than something like vaccinations based on the pure number of beings affected. This claim is definitely true, since there are trillions of wild animals and only ~20 million people in need of any single vaccination. But if we look more closely at its limiting factors, I think this claim is pretty misleading.

The problem size limit is indeed huge. In fact, it has to be cropped, or otherwise it would go all the way to the top of this post and the other sections would become impossible to see. However, that is not really what matters. Even if wild animal suffering is a huge problem and even if there is only a very limited amount of funding and talent that wants to work in the area, you will bump into problems with scaling far before it starts to matter if there are a billion wild animals or a billion billion. In the practical cases, when comparing these interventions, if a charity were founded in both of these areas, the vaccination charity would be able to get to a much larger scale than the wild animal suffering-focused charity could. A claim that “we should work on intervention X due to its massive problem scale” seems quite inaccurate. These sorts of arguments are extremely common for wild animals specifically and more broadly in EA.

Researching to determine the causes’ limiting factors generally ends up with more time being spent on considering the funder and talent space compared to, for example, mapping out the specific number of animals affected by any given intervention. And it usually ends up with a set of fairly different interventions looking promising.

Of course, the perspective changes depending on what you are looking to do in a given area. For example, when considering donating to a charity, the main thing examined by Givewell is room for funding. They are looking for an intervention for which its limiting factor to doing more good is funding. Some interventions might be very promising, but due to talent, logistical problems, or the size of the problem limitations, even if Givewell gave them more funding, they would not necessarily be able to create more impact. Organizations with room for more funding are generally stopped not by other factors but by ‘room for funding’ itself. Due to Givewell being a funder, it makes sense for them to take a careful look at room for funding as that is the limiting factor they can improve. As a charity entrepreneur, what the funding space looks like for a given organization should play a big role in what charity to found. As an employee at an NGO, you mostly have to consider how much of a limiting factor talent is for the organization.

These models are really simplified on both the scale and the limiting factor side, and I think there are other possible ways to use scale differently (or use tractability in a different way to cover some of the same concerns). I do not think 100% of EAs use a simple scale-based way of looking at problems, but I do think a large % of EAs use a fairly simple “size of the problem” based way of considering scale without thought to the % of the problem they can solve, given the first limiting factor that will stop growth/progress.