Can we apply start-up investing principles to non-profits?

By Peter Wildeford @ 2017-06-27T03:16 (+40)

How do you find the best non-profits to donate to? This is an important question that is critical to effective altruism.

One suggestion comes from Holden Karnofsky at the Open Philanthropy Project, who describes a strategy called “hits-based giving”. In this framework, you make a number of investments, some of which are very counter-intuitive and against expert consensus, with the understanding that many will not amount to much but those that work will generate excess returns to make the overall portfolio have a high altruistic return on philanthropic investment.

This strategy originates from YCombinator. In the essay “Black Swan Farming”, Paul Graham argues that funding for-profit startups is the art of hunting for the one deal that will make it big. You have a lot of “misses” when you invest, but the one time you make a “hit”, it will hit big and repay all your losses and then some. In order to guess right, you have to make many gambles. YCombinator has been working on this problem since 2005, and has since invested over $170M into over 1400 different start-ups. The combined valuation of their current start-up batch is stated to now be over $80B.      

“Black swan farming” seems to work well for YCombinator. But does it apply well when donating to non-profits? Does hits-based giving work?  Since writing that post on April 2016, OpenPhil has already allocated over $197M according to this philosophy. YCombinator is also applying hits-based giving to their own batch of non-profits, to which they have donated $3M.

The “Start Up” Approach

Separately, Ben Todd outlines that many donors concerned with effectiveness judge organizations based on their short-term marginal impact. For example, as Todd mentions, GiveWell had returns lower than its costs for the first four years, but then quickly exploded in its fundraising ratio, doubling money moved from 2012 to 2013, again from 2013 to 2014, and moving more money in 2015 than twice as much raised in 2013 and 2014 combined. An impact assessment focused solely on short-run fundraising ratios in 2011 would have missed GiveWell as an incredibly valuable investment.

In contrast, Todd argues for evaluating early “start-up” non-profits with standard start-up metrics, such as making sure they have a high-quality product, a large addressable market, and the ability to “sell” to this market at scale. Similarly, the organization should have a good growth rate and the team should ideally demonstrate competence and have a track record. For example, GiveWell had a superior research product with the ability to scale to millions of small donors plus dozens of interested large-scale foundations. While the team did not have much of a prior track record, they showed their competence through their early research and early traction with donors.

Lastly, Todd implies that upfront, early investments in rigorous cost-effectiveness analyses are premature, as they draw attention away from growing the core product in quality and scale, and they likely focus too much on the short-run impact, ignoring long-run opportunities.

Venture Capital vs. Hedge Funds

While the terminology applied is very loose and hard to generalize, the arguments by Karnosfky and Todd seem to compare non-profit donations to “start up” investments via venture capital -- doing what Graham and Thiel suggest and making hundreds of guesses to find the few diamonds in the rough that provide outsized returns.

However, this is not the only form of for-profit investing. One might also consider the approach of hedge funds, which appear to relatively employ less of a “hits-based” approach and more of an upfront investment in analytics. While VCs do some due diligence, hedge funds often employ very complex risk modeling when making investments.

This means if we could successfully generalize and compare venture capital versus hedge funds and see if one strategy generates superior returns compared to the other we could have some preliminary evidence for whether it is better to be “hits-based” or “evidence-based”.

Frustratingly, it is very difficult to compare the average returns of venture capital and hedge funds because the intra-group variation between individual firms is massive and dwarfs comparisons between the two groups. Also, the private nature of firms and selection bias in reporting makes finding accurate, systematic summary statistics quite hard.

A literature review of relevant research on VC firms finds that the average returns of VC are roughly equivalent to that of the stock market though with significant variation and methodological uncertainty (Rin, Hellmann, & Puri, 2011, p78-80). Furthermore, choices of sampling periods and methodology can dramatically change whether venture capital is determined to be more or less profitable than private equity on average (Rin, Hellmann, & Puri, 2011, p90).

Overall, high variation between individual firms in the same general category, the looseness of category definitions, the highly privatized nature of individual firm strategies, and the high uncertainty in results means that we unfortunately cannot draw firm conclusions from this line of investigation. This might mean that either approach is fine, but varies a lot more based on management and circumstances than approach to investing, but it is very hard to generalize this to non-profits.

How Similar Are For-Profit and Non-Profit Investments?

However, even comparing non-profit donations to hedge funds implies that making a donation is like for-profit investing. This is a view that many impact-interested donors appear to hold -- the prevalence of the phrase “impact investing” as a term for efficient giving drives this analogy home. However, the standard advice for individual for-profit investors is to avoid trying to “beat the market” by searching for investment opportunities on one’s own and instead to invest in index funds. Does this mean that non-profit investors should be advised to donate to altruistic “index funds” as well?

How similar is for-profit and non-profit investing? It appears to me like there are numerous key differences:

• Non-profit investing affords you the opportunity to be far more risk-neutral than you can in for-profit investing, which changes your options. Index funds are typically chosen less because the diversification increases average returns, but rather because the diversification decreases the variance of the investment, exposing you to less risk. A risk-neutral for-profit investor might be pursuing variance increasing strategies instead, like leverage. However, altruistic investments are not used with the intention of saving for one’s own future, which allows the altruist to be more risk-neutral to chase higher expected returns.
  
  

• The prices of non-profit investments don’t instantly change when they’re identified as more valuable, allowing good deals to be available much longer. If analysis shows that a particular stock is very hot, say offering the opportunity to invest $5 to get $50, according to the efficient market hypothesis, that analysis will nearly instantly be priced into the stock and the stock will quickly become worth ~$50. However, if a donation opportunity allows you to donate $3500 to save a life, arguably worth about $9.1M, the donation opportunity does not suddenly get bid up to $9.1M per life saved. Instead, the donation opportunity is used up until diminishing marginal returns mean it no longer exists, which happens significantly more slowly than a hot stock changes price. Therefore we should expect good giving to be hard, but not nearly as hard as finding a hot stock.
  
  

• For-profit investing typically does not have massive negative returns, but non-profit investing can. Unless you’re investing with leverage, breaking the law, are a massive “too big to fail” financial institution, or otherwise are doing something weird, the for-profit investments you make will typically not lose any more money than you put in. If you invest $1M, the worst that can happen is that you lose $1M. However, with non-profit investing, when you donate $1M, you run the risk of the non-profit being somehow net negative in confusing ways. Being able to guard against these risks is important in non-profit investing, but not in for-profit investing.
  
  

• Non-profit investing lets you arbitrage based on your values, whereas for-profit investing does not. Many foundations spending millions of dollars spend it based on values that are different than EA principles of being neutral toward the location or species of those that you help, or being neutral toward taking very far-future bets. The more your values depart from the global mainstream, the easier it should be to find good giving opportunities (e.g., donating or starting something yourself) that maximize those values, because the good ones won’t have been taken yet (unless your values are so weird that no one is willing to help you make progress on them).
  
  

• More people are trying a lot harder to “beat you” in for-profit investing than non-profit investing. In the for-profit world, your quest to find a hot stock with amazing ROI is going up against hundreds of thousands of incredibly well-funded analysts collectively working billions of hours a year to outcompete you. On the other hand, in the non-profit world, while some sharp non-profit investors are buying up the best opportunities already, it seems like most people don’t care, and the total community of people chasing optimal donations is a few thousand people perhaps collectively spending at most 2M hours a year. This makes it many times easier to outcompete the market in non-profit investing.
  
  

• Non-profit investing isn’t even a competition and analysts will share their best opportunities with you for free. Unlike effective for-profit investing, effective altruism isn’t a competition and since great donation opportunities take years to go away, they can be shared with you for free. If every single for-profit hedge fund gave you instant, free access to their advice in an easily summarized form, I imagine for-profit investing would slant away from index funds too.
  
  

• The returns for for-profit funds are relatively clear, but non-profit returns require a lot of work to understand. While there might be issues of applying the correct methodology, you can generally look at how much cash you get back for how much cash you put in. With non-profit investing, there is no clear measure of your return on investment. Instead, you have to use complex analysis to assess your return and some investments will never be able to show a conclusive return even if they do have one.
  
  

• Non-profit investors do not have a clear “index fund”. An index fund tries to diversify as much as possible by investing in a wide variety of stocks from a wide variety of markets. The S&P 500 is basically like investing a tiny bit in every large company in the US. On the other hand, investing in GiveWell’s top charities or in Effective Altruism Funds is a lot more like investing in an actively managed fund that has no expense ratio -- a fund that looks at many possible stocks but selects only the few that they think will beat the average. A true altruistic “index fund” would instead invest a small amount in every single charity and diversify as much as possible. The fact that this sounds like such a bad idea (and conversely that investing everything in only one stock sounds like a bad idea for personal finance) shows the difference between for-profit and non-profit investing.
  
  

Therefore, while pursuing higher returns at the chance of higher risk can be a good strategy for both start-up investing and optimal donating, there are also important differences between these two activities. For-profit investors have to exploit their insider knowledge and connections via start-up investing to beat the market, but in the non-profit world, the differences in pricing, the pooling of wisdom, and the relative lack of competition means that high returns might be found through an evidence-based approach. Moreover, the difficulty of understanding non-profit returns would suggest that non-profit investors would have to collect a lot of evidence just to understand how well their portfolios are doing.

The Incubation Approach

The difficulty of understanding non-profit returns, the ability to widely disseminate impact analysis, plus the lack of quickly diminishing returns, places a high premium on the value of collecting information and communicating it with the rest of the impact-interested community. Arguably, the effective altruism community currently under-invests in exploration, and this analysis provides some additional theoretic reasons why exploration could be so highly valuable.

The mere presence of large “hits” combine with the possibility of missing them is not, in itself, persuasive -- that’s just FOMO. For example, the possibility of there being “hit” lottery tickets does not suggest it is a good idea to do “hits-based” lottery ticket buying. Indeed, any investing strategy has both a false positive and a false negative rate, and care needs to be paid to both. If this comes at the cost of occasionally missing out on a big “hit”, that doesn’t mean your strategy is wrong. Instead, we would want to ascertain whether we have properly balanced our false positive and false negative rates to produce the highest expected returns. While I know Karnofsky and Todd have thought about this a lot and are not solely driven by FOMO, I do not think there has been enough published analysis of this type.

On the other hand, I agree with Todd that many EAs overemphasize the false negative rate with too much desire for rigor. I agree it’s important to have principles that would allow for taking risk on innovative ideas, and would have allowed you to fund organizations like The Against Malaria Foundation in the beginning, before they began to show signs of impact.

Another non-profit investing framework I think is worth considering is represented by GiveWell’s incubation grants, the Global Innovation Fund, Charity Science’s search for co-founders for future exceptional global poverty charities, and maybe EA Grants (though I'm not sure yet). While I don’t understand the exact process for vetting and approving these grants from these orgs, these grants seem like a great way to buy information.

My idealized version of the vetting process might go something like this:

1. Get a large pool of candidate projects and project teams, whether by soliciting applications and/or waiting for people to apply.
   
   

1. Following Todd’s approach, evaluate each project based on the quality of the team members, the quality of the core service or product the non-profit aims to provide, the upside opportunity of what the non-profit could potentially achieve, and the likelihood of achieving this scale. This could be done with a few interviews and some guesswork. It’s important to acknowledge there are many ways in which this evaluation may not accurately predict impact.
   
   

1. Identify the top projects you are willing to fund that meet the cut based on the criteria in step 2. Challenge these projects to come up with a plan to “prove” their model within a short but reasonable timeframe (e.g., 1-3 years). Offer them funding to cover all of their costs while they prove themselves, re-evaluating after each year. If they don’t appear to make the bar, require them to try something else.
   
   

1. When an organization does demonstrate their cost-effectiveness to an adequate degree, give them all the funding they need to scale up (e.g., by being a GiveWell top charity and receiving millions in funding).
   
   

1. Whether the organization succeeds or fails, write up and publicly publish copious notes and retrospectives, both qualitative and quantitative, on why the organization succeeded or failed.
   
   

My understanding is that this approach has two key differences from the approach employed by Karnofsky and Todd. First, step three requires each team to be producing information in a very tangible way -- they either succeed and demonstrate a successful program for scale-up or they fail and we learn from their failure. Second, step two could include additional selection criteria, such as being on a list of priority programs or generating information relevant to identifying priority programs.

This contrasts with Todd’s view that thorough evaluative work is not worth doing within the first few years of a new non-profit, since it takes a lot of work to know whether what you’re doing is working. This also elaborates on Karnofsky’s view that while you may not “require a strong evidence base before funding something”, you still should aim toward building that evidence base.

I cannot claim that this process would properly balance false positives and false negatives, but it does look good that this process avoids the dual traps of continuing a program that appears to work but doesn’t actually work (c.f., PlayPumps and many many many others) while also not falling into the short-sightedness that Todd warns us about by being able to fund an early GiveWell, AMF, Charity Science Outreach, Giving What We Can, or Google.

Can We Fund the Future?

A larger concern would be whether the process could risk falling into the narrow-mindedness that Karnofsky and Todd warn us about -- would we be able to recognize and fund work with long payoffs, like the Green Revolution, Peter Singer's early animal rights work, LGBTQ rights work in the '60s, or civil rights work in the '30s? Certainly you can’t do a randomized controlled trial to see whether MIRI is actually reducing existential risk, but would that mean they could never get a grant under this incubation approach?

Accounting for long payoffs is possible, but would require a lot more domain expertise, including a few breakthroughs, in how we measure and evaluate charities. This may involve investing in more fundamental research to understand, e.g. protesting, political influencing, or science R&D, before making concrete-level grants in very long-run areas.

Perhaps individual donors with sharp domain knowledge in a particular field may feel comfortable that they can identify hits without waiting for more fundamental research. I see that as the best argument for “hits-based giving”. Whether or not making these type of long-term bets with high domain knowledge would outdo mid-range bets or short-term marginal improvements is, naturally, unclear.

Either way, I’d encourage transparent grantmaking with a process that generates as much useful information for other future grantmakers. This incubation process seems quite promising to me, and I’d love to see it scaled up to expand to other cause areas beyond global poverty, with the large-scale funding and transparency needed to find demonstrably good opportunities across many cause areas.