My impact assessment of Giving What We Can

By Vasco Grilo🔸 @ 2023-04-15T06:59 (+40)

Summary

• I estimated the value of each of the Giving What We Can Pledges^[1] (GWWC Pledges) started between 2009 and 2022:
  • Using the fraction of reported donations, and counterfactuality coefficient estimated in GWWC’s impact evaluation.
  • Calculating future donations extrapolating the past annual donations of each pledge individually. In contrast, GWWC extrapolates the past annual donations per pledge, and then multiplies it by the number of pledges.
• I reckon:
  • GWWC caused 384 M$ (pessimistic to optimistic, 58.4 to 587) of donations due to its pledges in that period, with the caused donations per pledge being 46.6 k$ (7.09 to 71.3).
  • Half of the caused donations originated from 2.75 % of pledgers.
  • There is a correlation of -0.570 between the number of new pledges and donations caused by GWWC per new pledge.
• I also did a shallow cost-effectiveness analysis. Accounting for pledge and non-pledge donations going to the area of creating a better future only, and excluding GWWC’s impact besides donations, I estimate:
  • GWWC’s multiplier between 2020 to 2022 was 12.2 (0.797 to 19.1), which is 40.8 % GWWC’s best guess for the same period. Factoring in all donations, and using GWWC’s discount rate and effectiveness coefficient, I get a multiplier of 48.6 (4.87 to 68.3), which is 1.62 times GWWC’s best guess.
  • The non-marginal cost-effectiveness of GWWC from 2020 to 2022 was 12.6 (0.208 to 47.0) times the marginal one expected for longtermism and catastrophic risk prevention.

• I believe it would be important to study (by descending order of importance):
  • GWWC’s impact besides donations, which I believe may well be the driver of its overall impact.
  • The current marginal cost-effectiveness, which will tend to be lower than the (non-marginal) one I have estimated for 2020 to 2022 given diminishing marginal returns. These are supported by the negative correlation between the number of new pledges and donations caused by GWWC per new pledge.
  • The counterfactuality of the donations going to GWWC’s cause area of creating a better future, which in my view is the driver of the overall impact of the donations.

Acknowledgements

Thanks to Michael Townsend for feedback on the draft, and providing data.

Methods

Value of Giving What We Can Pledges

I wanted to estimate the value of Giving What We Can Pledges based on as much donation data as possible. So I considered the annual donations of each pledger individually, instead of those of all pledgers together, as GWWC did in their analysis here (which was announced here). Consequently, I can get values of the GWWC Pledges not only by year, but also by pledge.

I estimated a pessimistic, realistic, and optimistic value for each pledge. The realistic one is supposed to be the best proxy for the expected value, but I would have ideally worked with distributions instead of point estimates.

I calculated the pessimistic value from the donations caused by GWWC assuming no further donations, from the product between:

• Reported donations.
• All donations as a fraction of reported donations, which I set to 1.27, as considered by GWWC here.
• Donations caused by GWWC as a fraction of all donations, which I set to:
  • 0 for 3 major pledgers who explicitly said GWWC did not impact them (search for “excluded 3” here).
  • 26 % for all other cases, as considered by GWWC here^[2].

To determine the realistic and optimistic values, I started by estimating the future donations caused by GWWC, assuming:

• No change relative to the past (constant extrapolation). To do this:
  • I supposed the future annual donations caused by GWWC are equal to the ratio between the ones caused in the past (i.e. the pessimistic value of the pledge), and the pledge age^[3].
  • This corresponds to a common ratio of 1 between the donations of consecutive years.
• Exponential extrapolation (growth or decay). To do this:
  • I got the common ratio between the donations of consecutive years from the (exponential of the) slope of a linear regression of the logarithm of annual reported donations on year of the pledge^[4].
  • I weighted the data points by annual reported donations, such that large annual donations play a more important role in shaping the exponential.
  • I limited the common ratio to 1.03, given annual growth rate of global real GDP has been 3 % in the last decades^[5].
• For both cases:
  • A downwards adjustment to the common ratio to account for existential risk, multiplying it by 99.8 % (= (1 - 1/6)^(1/100)) in agreement with Toby Ord’s guess given in The Precipice of an existential risk of 1/6 between 2021 and 2120.
  • The donations to continue until the age of 65^[6], as considered by GWWC (search for “being 65” here).

Finally, I obtained the realistic and optimistic values of the pledge by adding the donations caused by GWWC assuming no further donations (the pessimistic value of the pledge) to the minimum and maximum of the above (future donations caused by GWWC assuming constant or exponential extrapolation). In practice, the realistic values respect the constant extrapolation, and the optimistic ones a 3 % annual growth rate. As reported donations for each pledger increase significantly over time^[7], my hard limit of 1.03 for the common ratio ends up being active for all donors except one.

Note:

• The donation data I used were retrieved from GWWC’s database on 22 February 2023. GWWC only shared anonymised data which was strictly necessary for me to carry out this analysis, and support my volunteering work.
• I adjusted all donations for inflation using these data from The World Bank for the years before 2022, and in2013dollars for 2022 and 2023.
• For each pledger, I ignored the donations made before 2009, when GWWC started.

Cost-effectiveness of Giving What We Can

I obtained the non-marginal cost-effectiveness of GWWC for the years between 2020 and 2022 from the product between:

• GWWC’s multiplier, which I computed from the product between:
  • Donations caused by GWWC regarding the GWWC Pledges started between 2020 and 2022, whose estimation is described in the previous section.
  • GWWC’s overall impact as a fraction of that of GWWC Pledges, which I assume to be 1.23 (= 1 + 19/83). This is based on GWWC’s estimates of 83 M$ for the donations caused by GWWC Pledges taken between 2020 and 2022, and 19 M$ for the non-pledge donations caused during the same period (see here). I have not modelled GWWC’s impact besides donations.
  • Reciprocal of GWWC’s overall costs (adjusted for inflation) as estimated by GWWC here, which are in the 1st table below.
  • Fraction of donations going to GWWC’s cause area of creating a better future, which is 13.3 % (= 11/(65 + 7 + 11)) based on the values provided here for the period between 2020 and 2022.
    • Feel free to check this post and this comment to get a sense of why I think the sign of the impact of donations going to the areas of improving human wellbeing and improving animal wellbeing is quite unclear.
    • I do not think excluding the donations going to such areas corresponds to an extreme position. From here, Benjamin Todd “would donate to the Long Term Future Fund over the global health fund, and would expect it to be perhaps 10-100x more cost-effective (and donating to global health is already very good)”. 80,000 Hours thinks global health is “important and underinvested in”, but it is not one of the 18 areas which are part of its list of the most pressing problems.
    • If I had set the (non-marginal) cost-effectiveness of donations to the area of creating a better future to 100 times that of improving human/animal wellbeing, I would have got similar results.
• Marginal cost-effectiveness in bp/G$ I estimated here for the cause area of longtermism and catastrophic risk prevention with method 3 with truncation. I used my 5th percentile, mean and 95th percentile estimates for the pessimistic, realistic and optimistic ones. The values are in the 2nd table below.

Some caveats:

• I have not used the effectiveness coefficient of 0.81 mentioned here by GWWC.
• I obtained the pessimistic/optimistic non-marginal cost-effectiveness multiplying the pessimistic/optimistic GWWC multiplier by the 5th/95th percentile marginal cost-effectiveness of longtermism and catastrophic risk prevention. So my pessimistic/optimistic non-marginal cost-effectiveness is more pessimistic/optimistic than my pessimistic/optimistic GWWC multiplier. Thanks to Edo for commenting on this.

The last point means I may be overestimating the uncertainty around GWWC’s cost-effectiveness. On the other hand, I have neglected the uncertainty of other factors, such as the reporting and counterfactuality coefficients, and the fraction of donations going to creating a better future.

Results

The data, calculations and full results are in this Sheet (see tab “TOC”).

Value of Giving What We Can Pledges

Cost-effectiveness of Giving What We Can

Discussion

Value of Giving What We Can Pledges

I estimate GWWC caused 384 M$ (pessimistic to optimistic, 58.4 to 587) of donations due to its pledges between 2009 and 2022, with the mean pledge being responsible for 46.6 k$ (7.09 to 71.3).

The 1st graph shows there is a negative correlation between the number of new pledges and donations caused by GWWC per new pledge. The correlation coefficient is -0.570 for my realistic estimates^[8]. In other words, starting more pledges has been associated with less donations per pledge, which makes sense given diminishing returns. This suggests cost-effectiveness will decrease as GWWC widens its reach, unless the costs per new pledge decreases enough due to economies of scale.

The 2nd graph illustrates the realistic donations caused by GWWC are quite heavy-tailed. Half of them come from only 2.75 % of pledgers, and the Gini coefficient is 0.149^[9]. This is qualitatively in agreement with GWWC’s finding that “less than 1% of our donors account for 50% of our recorded donations”. As a result, understanding well the extent to which GWWC caused the donations of top pledgers, and whether they have been accurately reported, is very important. Furthermore, the expansion strategy of GWWC should arguably be centred around these high return donors.

Cost-effectiveness of Giving What We Can

According to my results, GWWC’s multiplier between 2020 to 2022 was 12.2 (0.797 to 19.1), which is 40.8 % (= 12.2/30) GWWC’s best guess for the same period (see here). Nevertheless, factoring in all donations instead of just those going to the area of creating a better future, while using GWWC’s discount rate^[10] and effectiveness coefficient of 0.81 (see here), I get a multiplier to 48.6 (4.87 to 68.3), which is 1.62 (= 48.6/30) times GWWC’s best guess. I believe the difference is mostly explained by me calculating future donations extrapolating the past annual donations of each pledge individually, whereas GWWC extrapolates the past annual donations per pledge, and then multiplies it by the number of pledges.

I estimate the non-marginal cost-effectiveness of GWWC from 2020 to 2022 was 12.2 (0.208 to 46.8) times that expected for longtermism and catastrophic risk prevention. There is lots of uncertainty, with my optimistic estimate for that period being 225 (= 46.8/0.208) times my pessimistic one. However, the realistic estimate being one order of magnitude above 1 suggests GWWC is quite cost-effective.

The decrease in (non-marginal) cost-effectiveness of 76.1 % (= 1 - 4.98/20.8) from 2020 to 2022 can be explained by the reduction in GWWC’s multiplier. This is related to the increase of 305 % (= 1.34/0.331 - 1) in overall costs not being accompanied by a change in donations caused by GWWC, which decreased 3.08 % (= 1 - 41.0/42.3). Nonetheless, my realistic estimate will tend to underestimate the amount of donations^[11], because “[GWWC] found an increase in recorded GWWC Pledge donations with time” (see here).

I believe it would be important to study (by descending order of importance):

•  GWWC’s impact besides donations, which I believe may well be the driver of its overall impact.
• The current marginal cost-effectiveness, which will tend to be lower than the (non-marginal) one I have estimated for 2020 to 2022 given diminishing marginal returns. These are supported by the negative correlation between the number of new pledges and donations caused by GWWC per new pledge.
• The counterfactuality of the donations going to GWWC’s cause area of creating a better future, which in my view is the driver of the overall impact of the donations.

1. ^{^}

    These include The GWWC Pledge and The Trial Pledge.

2. ^{^}

    I could have used personalised parameters for the other surveyed pledgers, and the mean parameters for the others. However, since only a tiny minority of pledgers was surveyed, I opted to use the same parameters for every pledger besides the aforementioned 3 major ones.

3. ^{^}

    I got the pledge age from the difference between 2023 and the year the pledge started.

4. ^{^}

    If a pledge started in year S, year Y of the pledge corresponds to the year S + Y. So, for a pledge started in 2010, years 0 and 12 correspond to 2010 and 2022.

5. ^{^}

    Based on these data from The World Bank, it was 3 % (= (134.08/51.33)^(1/31) - 1) between 1990 and 2021.

6. ^{^}

    I set the age of the 2.21 % (= 1 - 8,196/8,381) of pledgers whose age was not available to the mean age of 32.5 years of the pledgers whose age was available.

7. ^{^}

    GWWC also “found an increase in recorded GWWC Pledge donations with time” (see here).

8. ^{^}

    Calculated in B3 of tab “New pledges”.

9. ^{^}

    For context, the lowest Gini coefficient respecting the income distribution within countries in 2019 was 0.308 for Albania, according to these data from Our World in Data.

10. ^{^}

     My annual discount factor of 99.8 % (derived from Toby Ord’s best guess for the existential risk from 2021 to 2120) is higher than the 96.5 % considered by GWWC (in line with the UK government’s green book; see here).

11. ^{^}

     Although my realistic and optimistic estimates consider donations until the age of 65.