Objections and Concerns About Our New Direction

By Holden Karnofsky @ 2012-06-22T15:59 (+2)

This is a linkpost to https://www.openphilanthropy.org/blog/objections-and-concerns-about-our-new-direction

Note: The Open Philanthropy Project was formerly known as GiveWell Labs. Before the launch of the Open Philanthropy Project Blog, this post appeared on the GiveWell Blog. Uses of “we” and “our” in the below post may refer to the Open Philanthropy Project or to GiveWell as an organization. Additional comments may be available at the original post.

GiveWell has recently been taking on activities that may seem to represent a pretty substantial change of direction, especially for those who think of us as a “charity evaluator focused on saving the most lives per dollar spent.”

• Within global health and nutrition, we’re considering restricted funding for specific projects, not just recommendations of particular charities.
• We’re also exploring other causes that are extremely different from global health and may be far less amenable to measurement and “cost per life saved” type calculations, such as meta-research.

When discussing these activities, we’ve lately been encountering a couple of different objections and concerns; this post discusses the objections and our responses. In a nutshell:

• Some are concerned that we’ll lose our objectivity if we get involved in providing restricted funding: we’ll be tempted to rank the groups following our plans ahead of the groups following their own plans, and we’ll thus lose the quality of being a disinterested third-party evaluator. We believe we can draw a meaningful line between “charities we recommend for unrestricted funding” and “plans we have designed,” leaving individual donors to decide whether they’d rather take our recommendation unconditionally or only follow our advice in the areas where we’re disinterested; we also believe that being open to providing restricted funding is necessary and important, and justifies the resources we’ll be investing. More
• Some are concerned that by going into new causes, we’ll be spreading ourselves too thin. Understanding global health is already an ambitious and difficult goal; it’s been suggested that we should “stick to our knitting.” We feel that sticking to global health, when we see other causes as potentially more promising, would be out of line with our fundamental mission and value-added as an organization that seeks to help people do as much good as possible. More
• Some are concerned specifically about new causes that don’t lend themselves to measurement and cost-effectiveness calculations (such as meta-research). It may be difficult to remain systematic and transparent about how we make decisions in these more speculative areas. We recognize this concern, but feel that we can remain systematic and transparent even where measurement is difficult or impossible; furthermore, we feel that we must find a way to do this if we are to have a strong case that philanthropy as a whole (not just sub-sectors of it) should be more systematic and transparent. More

Despite the concerns and risks above, we feel that the benefits of our new direction outweigh them. A major input into this view is the feeling that sticking to our old process would be extremely unlikely to result in finding more outstanding giving opportunities within a reasonable period of time; this is something we will be writing more about.

That said, we do recognize the concerns and risks, and we are interested in others’ thoughts on them.

The risk of losing our objectivity

To date, all of GiveWell’s recommendations have involved unrestricted support to existing organizations. Because of this, we can be pointed to as a “neutral third party” that recommends organizations based exclusively on impact-related criteria. But we’re now contemplating doing what a lot of major funders do and helping to set the agenda for a funded organization, through the mechanism of restricted funding. If we did this, we might have difficulty being neutral between (a) projects that we help design and (b) charities that are simply asking for unrestricted funds, not contracting with us. In fact, we might be tempted to eschew (b) entirely and focus exclusively on designing - rather than finding - giving opportunities.

One important principle here is that we will draw a clear line between organizations we recommend for unrestricted funding and projects designed by GiveWell. We don’t know exactly how the visual presentation will work yet, but we have agreed on the principle that there will be a clear distinction - including on our higher-level and frequently-accessed pages - between GiveWell-designed projects and recommended charities.

Of course, there is still a risk that recommendations for unrestricted funding will have “soft conditions” (i.e., that it will be clear to charities what activities they have to carry out in order to earn or maintain recommendations); this is something that has always been true, though I think the situation is somewhat mitigated by the nature of the room for more funding analysis we perform. (Our analysis asks for predicted charity activities based on total unrestricted funding, not based on GiveWell-specific funding. The expectation is that if GiveWell-directed funding falls short of expectations and the gap is made up by other funding, the activities will still be as outlined; this hopefully provides charities an incentive to project the activities they would most like to carry out, rather than projecting the activities they hope will most appeal to GiveWell specifically.)

Even with a clear distinction, there could still be a reasonable concern that GiveWell will over-allocate resources (in terms of investigative capacity) to designing its own projects, as opposed to finding great organizations. We recognize this concern, but wish to note that - philosophically - we greatly prefer unrestricted to restricted funding, and greatly prefer a “hands-off” to a “hands-on” approach. We don’t have the capacity to actively manage projects ourselves, and we believe projects are likely to work out better when they are run by people who fully buy into them (as opposed to people who are fulfilling the requirements of restricted funding).

It’s partly because of this philosophy that we’ve stayed away from restricted funding to date, and we remain highly cautious about it. We would prefer to stick to unrestricted funding and may never in fact deal in restricted funding.

Yet it is worth noting why we are considering restricted funding now in a way that we haven’t before. Our impression is that major funders frequently make extensive use of restricted funding; as a result, the existing landscape consists of many charities whose agendas are set partly or fully by external funders.

• We’ve been surprised by the disconnect we’ve observed in which there is a large number of promising interventions but few charities that focus on these interventions (in a way such that additional dollars will mean additional execution).
• More generally, we’ve been surprised that in the majority of conversations in which we ask an organization what it would do with more unrestricted funding, it has no clear answer, and prefers instead to tailor its answer to our priorities.

Practically speaking, charities have to focus on what they can fund; and in today’s world, it seems possible that agendas are largely set by funders. Our ideal role would be to “free” great organizations from restricted funding, allowing them to carry out promising projects that they can’t fund otherwise. However, it seems possible that there are too few charities for whom funding would make this sort of a difference, and there is thus some argument for our taking the sort of active role that other funders do.

Finally, by being open to restricted funding, we’ve come across some opportunities that are similar to “unrestricted funding” in most relevant ways, but that structurally involve restrictions and that we couldn’t have come across using our former approach. For example, we’re currently considering the idea of funding particular parts of UNICEF that work on particular interventions that we’re interested in. This wouldn’t involve laying out our own plan, and it would involve getting money to a specific team and leaving the use of the funds at their discretion; however, we could not find this sort of giving opportunity by talking to general UNICEF representatives and asking what they would do with more unrestricted funding. In some sense it may be appropriate to think of UNICEF (and other organizations like it) as a coalition of teams with their own priorities rather than as a single team with a single set of priorities; so in this case a gift that is formally restricted may have many of the desirable qualities of an unrestricted gift. To avoid confusion, we will still distinguish any recommendations along these lines from purely unrestricted gifts, as laid out above.

The risk of spreading ourselves too thin

We still have a lot to learn about global health and nutrition (as indicated by, among other things, our continued learning from VillageReach’s progress). It has been suggested that we should “stick to our knitting,” focusing on the areas of giving in which (a) we’ve built up the brand we have; (b) data and feedback loops tend to be unusually good for the nonprofit world, facilitating learning.

In response, I’d observe:

• GiveWell is still a young organization. I believe we have attracted attention more for “bringing a different perspective and approach to giving” than for “being experts in global health” (the latter certainly does not describe us). We recognize that we’re taking some level of risk in moving into new areas, but we also believe that taking risks and staying open to new approaches is a major part of what makes GiveWell what it is and that part of “sticking to our knitting” is retaining this quality. We believe that GiveWell and the donors who use our research will be best served by our continuing to do whatever we believe will lead to the best giving opportunities, continuing to change course as much as necessary to facilitate this, and continuing to bring a different perspective and approach to giving - not continuing to focus on global health.
• While we currently believe that global health is the most promising cause given the information available, we are not confident in this conclusion. We believe that other causes are potentially promising as well, and if we never investigate them, we will be failing in our mission of finding the best giving opportunities possible.
• We are currently expanding our staff; we expect that we will invest at least as much time in global health over the next few years as over the last few (while also investing time in other causes).

The risk of losing transparency and systematicity as we move away from highly measurable interventions

We have written before that the cause of “global health and nutrition” seems unusually well-suited to meaningful measurement and metrics (by the standards of the nonprofit sector). When working within this cause, we have been able to be relatively clear about our process and about what distinguishes a recommended from a non-recommended charity. There is some risk that as we tackle other causes, such as meta-research, we will have less of an evidence base to go off of; our goals will be further out; we will have to use more intuition and may therefore become less systematic and transparent.

We believe this is a real risk. However, we also believe that (a) the best opportunities for good giving don’t necessarily lie in the domains with the highest measurability (though there is something to be said for measurability, all else equal); (b) we have reached the point where we feel we can explore causes such as meta-research in a way that - while not as systematic as our work on global health - will still include a great deal of public discussion of how we’re thinking, why we recommend what we do, what the key assumptions are in our thinking and recommendations, and how our projects progress over time.

We have long advocated that philanthropists should be more systematic and transparent in their work. If our own systematicity and transparency applies only to the cause where measurement is easiest, we won’t have a very strong case; if, however, we can consistently bring an unusually level of systematicity and transparency to every cause we examine (even those that are less prone to measurement), we will have much more potential to change philanthropy broadly rather than just a single sector of it.

The benefits of our new direction

The above discussion addresses potential concerns over our new direction. We have previously discussed the substantial benefits: finding the best giving opportunities possible and reaching the largest donors possible, both of which are core to our mission. Dealing with the above issues - keeping a focus on recommending unrestricted funding when possible, covering new causes without overly detracting from continued progress on the causes we know well, and remaining systematic and transparent - will be a challenge, but we feel that it is well worth it, especially because we feel we are reaching the limits (for the moment) of our old approach. (We went through a large number of charities in 2011 and are skeptical that we will find new contenders for our top charities, using that basic methodology, anytime in the near future.)

We welcome further comments and criticisms regarding our new approach.