How we use back-of-the-envelope calculations in our grantmaking

By Open Philanthropy @ 2025-05-28T23:22 (+75)

Emma Buckland, Program Associate in Farm Animal Welfare, drafts a quick back-of-the-envelope calculation for her colleagues.

This post was written by Open Philanthropy Global Health and Wellbeing Staff. Original blog post here.

At Open Philanthropy, our mission is to help others as much as we can with the resources available to us. When making tough calls about which grants will help the most, we rely on a tool that helps clarify expected impact: back-of-the-envelope calculations, or BOTECs.

BOTECs are rough quantitative models that estimate potential grants’ social return on investment (SROI). Open Phil program staff use them to compare a grant's expected benefits to its estimated costs.

Despite the name, most BOTECs wouldn’t actually fit on the back of an envelope. They often involve dozens of assumptions, nested calculations, and scenario planning — especially when we’re modeling larger grants or entire focus areas.

Staff across our Global Health and Wellbeing (GHW) focus areas use BOTECs to help answer a key question: Will this grant likely clear our bar for GHW grants?[1] That bar is currently set at a return of slightly over 2,000x in “Open Phil dollars.”[2] 

A key purpose of the BOTEC exercise is to figure out: what would have to be true for this grant to meet or exceed the bar? This helps us focus our attention on the most decision-relevant assumptions.

What BOTECs can (and can’t) tell us

BOTECs are flexible tools, and their structure varies depending on the type of grant.

Some BOTECs are backward-looking, based on a grantee’s track record. Others are forward-looking, built around a grantee’s theory of change and how likely we think it is to succeed.

To explore the range of possible outcomes, program staff often build multiple versions of a BOTEC — using optimistic and pessimistic inputs to see how sensitive our estimates are to certain parameters.[3] This isn’t about reaching a precise “answer,” but about stress-testing our thinking under different plausible scenarios.

We don’t rely on BOTECs alone to make grant decisions; they are one piece of a broader picture. We also weigh the grantee’s leadership and track record, strategic considerations the BOTEC can’t easily quantify, and potential for “unknown upside” or unusual leverage.[4] Grantmaking is always uncertain, and BOTECs don’t remove that uncertainty — but they do help us navigate it.

Sample BOTECs

Below are four sample BOTECs, simplified a bit for readability. We use pseudonyms to preserve confidentiality, but all draw from real grants we made across vastly different focus areas. We are sharing these examples to give a sense of how we think, where we (frequently!) debate, and how we try to allocate resources to help others as much as we can. As always, we welcome your feedback here.

Trialing statins for treating tuberculosis

This grant falls within our focus area of global health R&D.

A recent Phase IIa/b trial tested atorvastatin as an additional treatment for tuberculosis (TB), which kills more than a million people each year.[5] The trial found that atorvastatin helped to reduce TB infection levels more quickly and showed promising improvements in reducing lung damage, based on chest X-rays.

This grant would support a Phase IIc trial to evaluate atorvastatin as an addition to standard TB treatments.[6] The proposed trial would investigate whether different dosing strategies of atorvastatin improve survival rates, accelerate the reduction of TB infection levels, and reduce long-term damage to the lungs. The grant would also support efforts to develop a novel sweat-based diagnostic test for TB.

Benefits of the grant

Costs: The current trial would cost $320,000. However, it is only a Phase IIc trial, so there would be additional research and rollout costs to consider.[9] We roughly estimate these costs to be $7 million.[10] 

Probability of success: We estimate a 40% probability that Phase II and Phase III trials will succeed.

Final SROI: The grant’s SROI is the benefit ($OP 54 billion) divided by the all-in program costs ($7 million), all multiplied by the estimated chance of success (40%). This yields ~3,000x SROI, well above our ~2,000x bar. That is, we expect the grant to generate 3,000 times as much value — in Open Phil dollars — as it costs.

Some limitations of this BOTEC

This BOTEC leaves out some additional benefits — such as the potential value of a new TB diagnostic tool — as well as any improvements to quality of life beyond preventing deaths.

In addition, we only have very rough estimates of what subsequent trials/studies will cost, and we don’t include the possibility that newer TB treatments in development reduce the TB burden beyond the current downward trend.

Updates since making the grant

More than a year has passed since we made this grant, and during that time, we’ve continued to learn about the wider TB landscape. If we were modeling it today, here’s what we’d change about the BOTEC:

While the lower chance of success and longer time to impact would reduce the SROI, the larger speedup would substantially increase it. Taken together, these changes would increase the final SROI.

Reducing the cost of pXRF screening

This grant falls within our focus area of global public health policy.

Current methods for testing lead in paint are resource- and skill-intensive. Portable XRF-based measurement (pXRF) uses a handheld device to quickly and cheaply detect lead in paint and other materials. If successful, the device would allow inspectors to more easily identify lead in paint and prevent it from causing harm.

This grant would fund the nonprofit organization Lead the Way[11] to:

The key assumptions in the theory of change[12] are:

The expected value of this grant is ($OP 2.3 billion in benefit) / (total cost of project [~$350,000]) = ~6,500x SROI in expectation. For any one parameter, changing the best guess to the more pessimistic end of the confidence range still yields an SROI above the ~2,000x bar.

Supporting a fundraising organization for effective charities

This grant falls within our focus area of effective giving & careers.

Giving Ground is the only professional organization promoting effective giving in its country. With just 3.5 full-time employees, it raises money almost exclusively for GiveWell's top charities.

In many cases, Open Philanthropy program staff use backward-looking BOTECs to estimate future impact — so to assess a potential new grant, we created a BOTEC based on Giving Ground’s past performance:

After the 15% adjustment, our best guess is that over the next two years, Giving Ground will raise $2.27 for GiveWell for every $1 it spends: $800,000 x (1-.15 [counterfactuality discount]) / ($300,000 [operating expenses]) = 2.27x.[17] Our bar for effective giving interventions is a 2x ratio,[18] so this clears our bar.

Improving conditions for broiler chickens

This grant falls within our focus area of farm animal welfare. Note that the FAW team uses a different bar than the rest of our GHW focus areas.

Feather Forward runs national campaigns to convince corporations to improve their animal welfare policies. The campaigns focus on stopping the use of caged hens to produce eggs and on improving the welfare of chickens raised for meat (called “broiler chickens”).

Just like the grant above, this is a backward-looking BOTEC. When considering a new grant to Feather Forward, we modeled their impact under their previous grant as follows:

Given these adjustments, we estimated that Feather Forward’s work improved the welfare of 5.4 million animals alive at any time.[21] To determine the full value of this impact, we apply additional considerations that include:

After incorporating these factors, we calculated that this grant easily exceeded our internal bar for farm animal welfare interventions. While our FAW BOTECs incorporate different moral weights and welfare improvement assessments than our other focus areas, the core approach remains consistent.

  1. ^

     The ~2,000x bar means that for every dollar we spend, we now aim to create as much value as giving ~$2,000 to someone earning $50,000/year in the U.S. We value improvements in income using a logarithmic utility function, so that an x% improvement in income is valued at $50,000 * ln(1+x). For health improvements, we use $100,000 as the value of a disability-adjusted life year (DALY) averted; an intervention that averted 3 DALYs would be valued at $300,000. See here for more on these methods.

  2. ^

     Our fundamental unit of benefits is what we call the “Open Phil dollar” ($OP), which is equivalent to giving $1 to someone earning $50,000/year in the U.S. We value a DALY at $100,000 in Open Phil dollars ($OP 100,000).

  3. ^

     In rare cases, we model our uncertainty more explicitly, specifying distributions for key input parameters and using Monte Carlo simulations to model the distribution of possible outcomes. We sometimes use carlo.app for this.

  4. ^

     For example: funding projects in emerging fields that build critical infrastructure or develop expertise, where direct impacts may only follow years later and be hard to attribute to any project.

  5. ^

     Phase IIa/b trials assess a treatment’s efficacy and safety. Atorvastatin is a widely used cholesterol-lowering drug (i.e. a statin). TB is a leading global health concern that primarily affects the lungs.

  6. ^

     Phase IIc trials are late-stage exploratory trials that build on earlier Phase II results to further confirm dosing, safety, and efficacy in a larger or more specific patient population. They aim to reduce uncertainties before progressing to larger, definitive Phase III trials.

  7. ^

     Our framework for R&D grants is that we do not create technologies which would never have existed in the absence of our grant. Rather, we assume that our funding allows researchers to develop the technology earlier than they would have without our funding. The exact number of years we would speed up a desired outcome versus a counterfactual is highly debatable; most of our R&D BOTECs assume between 2 and 7 years.

  8. ^

     Our estimate of DALYs per death from TB takes into account the expected years of healthy life remaining for the average TB patient.

  9. ^

     Including one or more Phase III trials, guideline changes, research dissemination, and potential validation studies.

  10. ^

     To avoid overestimating SROI, we look at the all-in program costs rather than our own contribution. For this grant, you could think of $OP 54 billion as “the benefits our funding would generate if we put in $7 million.” If we put in ~4.5% of that funding, we should only “credit” ourselves with ~4.5% of the benefits.

  11. ^

     All organizations mentioned in this piece are pseudonymous to preserve confidentiality.

  12. ^

     For some of these points, we show the range of potential outcomes our investigator thought was plausible, as well as the figure they used in the actual calculation. So “5% (range: 5-20%)” implies that the plausible odds were thought to range from 5% to 20%, and the figure actually used was 5%.

  13. ^

     By “formal,” we mean a licensed manufacturer operating within the regulatory framework of the country. By “informal,” we mean those operating outside of it (and often selling at cheaper price points).

  14. ^

     $3,900 in Open Phil dollars per 1 ug/dL per birth is a value we use across many lead grants. It is calculated here and is based on effects of lead in a child’s blood on later life income and health.

  15. ^

     The other 90% of the credit goes to the regulators who implement the method, the advocates who push for its adoption, etc.

  16. ^

     There is also a second theory of change: the cheaper, easier method reduces measurement costs that would have otherwise been borne by either philanthropic funds or governments. We assume that potential impact from speeding up enforcement (as above) dwarfs the cost reduction factor, so we haven’t factored this second theory of change into our SROI.

  17. ^

     If a significant share of the funding had gone toward charities not recommended by GiveWell, we’d also have applied a “cost-effectiveness adjustment,” since we assume the average charity (even in the global health space) is somewhat less impactful than one of GiveWell’s top charities. For example, some charities might get a 50% adjustment, meaning we think each dollar to those charities has roughly the same impact as $0.50 going to a top charity. (Since all funding went to GiveWell top charities in this case, we didn’t apply any cost-effectiveness adjustment.)

  18. ^

     For giving opportunities that act as money multipliers, we typically look for a return of twice our funding (even though this conceptually is higher than our normal ROI bar since we think effective giving is >0.5 times as effective as our bar). The extent of this higher bar is subjective, but we believe a higher bar is warranted for a few reasons:

    1. We think the way we model "meta" funding opportunities, which are more distant from impact, is more likely to overestimate cost-effectiveness by failing to account for additional efforts required by other actors to achieve impact. (If we give money directly to GiveWell, we’re responsible for 100% of that. If our grantee spends $1,000 and raises $2,000 for GiveWell, our $1,000 may be only partly responsible — for example, the donors reached by the grantee may also have been influenced by GiveWell’s own advertising, or by efforts from other actors in the effective giving world.)
    2. Our model is likely to overestimate the impact from additional donations to “high-impact” charities because it assumes that any funding attributed to the grantee would otherwise not have gone to charity at all (rather than going to a less impactful charity).
    3. Intuitively, we aren't excited about supporting opportunities that could spend an additional $1 to generate only slightly more than that for high-impact charities, even if all donations are counterfactual. Supporting these opportunities would mean that small errors in our calculations (or unforeseen expenses) could result in negative impact (e.g. spending $1 to raise $0.90), and potentially send us into a meta trap.
  19. ^

     We estimate that 34 million animals are alive at any time in the supply chains of the companies that have made commitments. In most cases, we lack data on companies’ animal product usage and the number of animals in companies’ supply chains. When individual company data is not available, we use data from a comparable company (adjusted for revenues or outlets) or use national and sector animal product usage and company sector market share (e.g. how many chickens are sold in retail, and the share of the retail market the company accounts for).

  20. ^

     Outside of advocacy groups, no other institutional actors are pushing corporations to adopt these commitments. So while it is relatively easy to attribute almost all of the credit to advocacy groups, crediting specific groups involves highly uncertain judgment calls.

  21. ^

     Because broiler chickens are typically slaughtered after less than two months, well over 5.4 million animals are affected over the course of a year.


Knight Lee @ 2025-05-30T07:51 (+6)

Can you do a back-of-the-envelope calculation, on the costs and benefits of doing more back-of-the-envelope calculations? E.g. getting a different person to independently replicate a back-of-the-envelope calculation, in order to average out errors and biases specific to one individual and make it more robust.

SummaryBot @ 2025-05-29T16:09 (+1)

Executive summary: Open Philanthropy explains how it uses back-of-the-envelope calculations (BOTECs) to estimate the cost-effectiveness of grants across focus areas like global health, lead exposure reduction, animal welfare, and effective giving, illustrating their approach through detailed examples and emphasizing both the utility and limitations of these rough but decision-critical models.

Key points:

  1. BOTECs clarify expected impact by estimating a grant’s social return on investment (SROI), helping Open Phil determine whether a grant clears its cost-effectiveness threshold — currently ~2,000x in “Open Phil dollars” for Global Health and Wellbeing grants.
  2. The models vary by grant type — DALYs averted for health, suffering reduced for animals, or funds raised for effective charities — and may be forward- or backward-looking depending on available data and theory of change.
  3. BOTECs guide but don’t dictate decisions; qualitative factors like leadership, track record, and unusual upside are also considered, and multiple BOTEC versions test the robustness of conclusions across different scenarios.
  4. Examples illustrate application and nuance: A tuberculosis R&D grant modeled to avert nearly 20,000 deaths annually showed a 3,000x SROI; a lead detection method grant had an expected 6,500x SROI; an effective giving org cleared a 2x bar for fundraising ROI; and a broiler welfare campaign surpassed the animal welfare team's separate bar.
  5. Open Phil adjusts BOTECs over time as new information arises — for example, reassessing speedup timelines or success probabilities post-grant — and openly acknowledges uncertainties, estimation challenges, and speculative assumptions in modeling.
  6. The post invites community feedback and aims to demystify Open Phil’s quantitative thinking, while signaling that BOTECs are one tool among many in a broader evaluative process.

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.