Thoughts on the "Meta Trap"

By Rohin Shah @ 2016-12-20T21:36 (+10)

Cross-posted to my blog. Thanks to Ajeya Cotra and Jeff Kaufman for feedback on a draft of this post. Any remaining errors are my own. Comments on this post may be copied to my blog.

Last year, Peter Hurford wrote a post titled ‘EA risks falling into a "meta trap". But we can avoid it.’ Ben Todd wrote a followup that clarified a few points. I have a few more meta traps to add to the list.

What do I mean by "meta"?

In the original two posts, I believe "meta" means "promoting effective altruism in abstract, with the hope that people do good object level projects in the future". This does not include cause prioritization research, which is typically also lumped under "meta". In this post, I'll use "meta" in much the same way -- I'm talking about work that is trying to promote effective altruism in the abstract and/or fundraising for effective charities. Example organizations include most of the Centre for Effective Altruism (Giving What We Can, EA Outreach, EA Global, Chapters team), 80,000 Hours, most of .impact, Raising for Effective Giving, Charity Science, Center for Applied Rationality, Envision, Students for High Impact Charity and local EA groups. However, it does not include GiveWell or the Foundational Research Institute.

What is this post not about?

I am not suggesting that we should put less money into meta; actually I think most meta-charities are doing great work and should get more money than they have. I am more worried that in the future, meta organizations will be funded more than they should be because of leverage ratio considerations that don't take into account all of the potential downsides.

Potential Biases

Most of the work I do falls under "meta"—I help coordinate across local EA groups, and I run a local group myself. As a result, I've become pretty familiar with the landscape of certain meta organizations, which means that I've thought much more about meta than other cause areas. I expect that if I spent more time looking into other cause areas, I would find issues with them as well that I don't currently know about.

I have become most familiar with local EA groups, the CEA chapters team, Giving What We Can, and 80,000 Hours, so most of my examples focus on those organizations. I'm not trying to single them out or suggest that they suffer most from the issues I'm highlighting—it's just easiest for me to use them as examples since I know them well.

Summary of considerations raised before

In Peter Hurford’s original post, there were several points made:

 
1. Meta Orgs Risk Not Actually Having an Impact. Since meta organizations are often several levels removed from direct impact, if even one of the "chains of impact" between levels fails to materialize, the meta org will not have any impact.
 
2. Meta Orgs Risk Curling In On Themselves. This is the failure mode where meta organizations are optimized to spread the meta-movement but fail to actually have object-level impact.
 
3. You Can’t Use Meta as an Excuse for Cause Indecisiveness. Donating to meta organizations allows you to skip the work of deciding which object-level cause is best, which is not good for the health of the movement.
 
4. At Some Point, You Have to Stop Being Meta. This is the worry that we do too much meta work and don't get around to doing object-level work soon enough.
 
5. Sometimes, Well Executed Object-Level Action is What Best Grows the Movement. For example, GiveWell built an excellent research product without focusing on outreach, but still ended up growing the movement.

It seems to me that meta trap #4 is simply a consequence of several other traps -- there are many meta traps that will cause us to overestimate the value of meta work, and as a result we may do more meta work than is warranted, and not do enough object-level work. So, I will be ignoring meta trap #4 for the rest of this post.

I'm also not worried about meta trap #2, because I think that meta organizations will always have a “chain of impact” that bottoms out with object-level impact. (Either you start a meta organization that helps an object-level organization, or you start a meta organization that helps some other organization that inductively eventually helps an object-level organization.) To fall into meta trap #2, at some point in the chain one of the organizations would have to change direction fairly drastically to not have any object-level impact at all. However, the magnitude of the object-level impact may be much smaller than we expect. In particular, this "chain of impact" is what causes meta trap #1.

The Chain of Impact

Let's take the example of the Chapters team at the Centre for Effective Altruism (CEA), which is one of the most meta organizations -- they are aimed at helping and coordinating local EA groups, which are helping to spread EA, which is both encouraging more donations to effective charities and encouraging more effective career choice, which ultimately causes object-level impact. At each layer, there are several good metrics which are pretty clear indicators that object-level impact is happening:

 
• The Chapters team could use "number of large local EA groups" or "total GWWC pledges/significant plan changes from local groups"
 
• Each local group could use "number of GWWC pledges", "number of significant plan changes", and "number of graduated members working at EA organizations"
 
• GWWC and 80,000 Hours themselves have metrics to value a pledge and a significant plan change, respectively.
 
• The object-level charities that GWWC recommends have their own metrics to evaluate their object-level impact.

Like I said above, this “chain of impact” makes it pretty unlikely that meta organizations will end up curling in on themselves and become detached from the object level. However, a long chain does mean that there is a high risk of having no impact (meta trap #1). In fact, I would generalize meta trap #1:

Meta Trap #1. Meta orgs amplify bad things too

The whole point of meta organizations is to have large leverage ratios through a "chain of impact", which is usually operationalized using a "chain of metrics". However, this sort of chain can lead to other amplifications as well:

Meta Trap #1a. Probability of not having an impact

This is the point made in the original post -- the more meta an organization is, the more likely that one of the links in the chain fails and you don't have any impact at all.

One counterargument is that often, meta organizations help many organizations one level below them (eg. GWWC). In this case, it is extremely unlikely that none of these organizations have any impact, and so the risk is limited to just the risk that the meta organization itself fails at introducing enough additional efficiency to justify its costs.

Meta Trap #1b. Overestimating impact

It is plausible that organizations systematically overestimate their impact (something like the overconfidence effect). If most or all of the organizations along the chain overestimate their impact, then the organization at the top of the chain will have a vastly overestimated object-level impact. Note that if an organization phrases its impact in terms of its impact on the organization directly below them in the chain, this does not apply to that number. It only applies to the total estimated object-level impact of the organization.

You can get similar problems with selection effects, where the people who start meta organizations are more likely to think that meta organizations are worthwhile. Each selection effect leads to more overconfidence in the approach, and once again the negative effects of the bias grow as you go further up the chain.

Meta Trap #1c. Issues with metrics

A commonly noted issue with metrics is that an organization will optimize to do well on its metrics, rather than what we actually care about. In this way, badly chosen metrics can make us far less effective.

As a concrete example, let's look at the CEA Chapters Team chain of metrics:

 
• One of many plausible metrics for the CEA Chapters team is "number of large local groups". (I'm not sure what metrics they actually use.) They may focus on getting smaller local groups to grow, but this may result in local groups having large speaker events and counting vaguely interested students as "members" that results in a classification of "large" even though not much has actually changed with that local group.
 
• Local groups themselves often use "number of GWWC pledges" as a metric. They may start to promote the pledge very widely with external incentives (eg. free food for event attendees, followed by social pressure to sign the pledge). As a result, more students may take the pledge, but they may be much more likely to drop out quickly.
 
• GWWC has three metrics on its home page -- number of pledges, amount of money already donated, and amount of money pledged. They may preferentially build a community for pledge takers who already make money, since they will donate sooner and increase the amount of money already donated. Students may lose motivation due to the lack of community and so would forget about the pledge, and wouldn’t donate when the time came. GWWC would still be happy to have such pledges, because they increase both the number of pledges and the amount of money pledged.
 
• The various object-level organizations that GWWC members donate to could have similar problems. For example, perhaps the Against Malaria Foundation would focus on areas where bednets can be bought and distributed cheaply, giving a better “cost per net” metric, rather than spending slightly more to distribute nets in regions with much higher malaria burdens, where the money would save more lives.

If all of these were true, you'd have to question whether the CEA Chapters team is having much of an impact at all. On the other hand, even though the metrics for AMF lead to suboptimal behavior, you can still be quite confident that they have a significant impact.

I don't think that these are true, but I could believe weaker versions (in particular, I worry that GWWC pledges from local groups are not as good as the average GWWC pledge).

In addition to the 5 traps that Peter Hurford mentions, I believe there are other meta traps that tend to make us overestimate the impact of meta work:

Meta Trap #6. Marginal impact may be much lower than average impact.

The GiveWell top charities generally focus on implementing a single intervention. As a result, we can roughly expect that the marginal impact of a dollar donated is about the same as the average impact of a dollar, since both dollars go to the same intervention. It's still likely lower (for example, AMF may start funding bednets in areas with lower malaria burden, reducing marginal impact), but not that much lower.

However, meta organizations typically have many distinct activities for the same goal. These activities can have very different cost-effectiveness. The marginal dollar will typically fund the activity with the lowest (estimated) cost-effectiveness, and so will likely be significantly less impactful than the average dollar.

Note that this assumes that the activities are not symbiotic -- that is, the argument only works if stopping one of the activities would not significantly affect the cost-effectiveness of other activities. As a concrete example of such activities in a meta organization, see this comment about all the activities that get pledges for GWWC.

The numbers that are typically publicized are for average impact. For example, on average, GWWC claims a 104:1 multiplier, or 6:1 for their pessimistic calculation, and claims that the average pledge is worth $73,000. On average the cost to 80,000 Hours of a significant plan change is £1,667. (These numbers are outdated and will probably be replaced by newer ones soon.) What about the marginal impact of the next dollar? I have no idea, except that it's probably quite a bit worse than the average. I would not be surprised if the marginal dollar didn't even achieve a 1:1 multiplier under GWWC's assumptions for their pessimistic impact calculation. (But their pessimistic impact calculation is really pessimistic.)   To be fair to meta organizations, marginal impact is a lot harder to assess than average impact. I myself focus on average impact when making the case for local EA groups because I actually have a reasonable estimate for those numbers.

One counterargument against this general argument is that each additional activity further shares fixed costs, making everything more cost-effective. For example, GWWC has to maintain a website regardless of which activities it runs. If it adds another activity to get more pledges that relies on the website, then the cost of maintaining the website is spread over the new activity as well, making the other activities more cost effective. Similar considerations could apply to office space costs, legal fees, etc. I would guess that this is much less important than the inherent difference in cost effectiveness of different activities.

Meta Trap #7. Meta suffers more from coordination problems.

As the movement grows and more people and meta organizations become a part of it, it becomes more important to consider the group as a whole, rather than the impact of just your actions. This 80,000 Hours post applies this idea to five different situations.

Especially in the area of promoting effective altruism in the abstract, there are a lot of organizations working toward the same goal and targeting the same people. For example, perhaps Alice learns about EA from a local EA group, goes to a CFAR workshop, starts a company because of 80,000 Hours and takes the Founder's Pledge and GWWC pledge. We now have five different organizations that can each claim credit for Alice's impact, not to mention Alice herself. In addition, from a "single player counterfactual analysis", it is reasonable for all the organizations to attribute nearly all the impact to themselves -- if Alice was talking to these organizations while making her decision, each organization could separately conclude that Alice would not have made these life changes without them, and so counterfactually they get all the credit. (And this could be a reasonable conclusion by each organization.) However, the total impact caused would then be smaller than the sum of the impacts each organization thinks they had.

Imagine that Alice will now have an additional $2,000 of impact, and each organization spent $1,000 to accomplish this. Then each organization would (correctly) claim a leverage ratio of 2:1, but the aggregate outcome is that we spent $5,000 to get $2,000 of benefit, which is clearly suboptimal. These numbers are completely made up for pedagogical purposes and not meant to be actual estimates. In reality, even in this scenario I suspect that the ratio would be better than 1:1, though it would be smaller than the ratio each organization would compute for itself.

Note that the recent changes at CEA have helped with this problem, but it still matters.

Meta Trap #8. Counterfactuals are harder to assess.

It's very unclear what would happen in the absence of meta organizations -- I would expect the EA movement to grow anyway simply by spreading through word of mouth, but I don't know how much. If Giving What We Can didn’t exist, perhaps EA would grow at the same rate and EAs would publicize their donations on the EA Hub. If 80,000 Hours didn’t exist, perhaps some people would still make effective career choices by talking to other EAs about their careers. It is hard to properly estimate the counterfactual for impact calculations -- for example, GWWC asks pledge takers to self-report their counterfactual donations, which is fraught with uncertainties and biases, and as far as I know 80,000 Hours does not try to estimate the impact that a person would have had before their plan change.

This isn't a trap in and of itself -- it becomes a trap when you combine it with biases that lead us to overestimate how much counterfactual impact we have by projecting the counterfactual as worse than it actually would have been. We should care about this for the same reasons that we care about robustness of evidence.

I think that with most object-level causes this is less of an issue. When RCTs are conducted, they eliminate the problem, at least in theory (though you do run into problems when trying to generalize from RCTs to other environments). I think that this is a problem in far future areas (would the existential risk have happened, or would it have been solved anyway?), but people are aware of the problem and tackle it (research into the probabilities of various existential risks, looking for particularly neglected existential risks such as AI risk). I haven't seen anything similar for meta organizations.

What should we do?

Like I mentioned before, I’m not worried about meta traps #2 and #4. I agree that meta traps #3 (cause indecisiveness) and #5 (object-level action as a meta strategy) are important but I don't have concrete suggestions for them, other than making sure that people are aware of them.

For meta trap #1, I agree with Peter's suggestion: The more steps away from impact an EA plan is, the more additional scrutiny it should get. In addition, I like 80,000 Hours’ policy of publishing some specific, individual significant plan changes that they have caused. Looking at the details of individual cases makes it clear what sort of impact the organization has underneath all the metrics, and ideally would directly show the object-level impact that the organization is causing (even if the magnitude is unclear). It seems to me that most meta organizations can do some version of this.

I worry most about meta trap #6 (marginal vs. average impact). It also applies to animal welfare organizations, but I'm less worried there because Animal Charity Evaluators (ACE) does a good job of thinking about that consideration deeply, creating cost-effectiveness estimates for each activity, and basing its recommendations on that. We could create "Meta Charity Evaluators", but I'm not confident that this is actually worthwhile, given that there are relatively few meta organizations, and not much funding flows to them. However, this is similar to the case for ACE, so there's some reason to do this. This could also help with meta trap #7 (coordination problems), if it took on coordination as an explicit goal.

I would guess that we don't need to do anything about meta trap #8 (hard counterfactuals) now. I think most meta organizations have fairly strong cases for large counterfactual impact. I would guess that we could make good progress by research into what the counterfactuals are, but again since there is not much funding for meta organizations now, this does not seem particularly valuable.