Cost-effectiveness of Shrimp Welfare Project’s Humane Slaughter Initiative

By Vasco Grilo🔸 @ 2024-10-06T08:25 (+65)

The views expressed here are my own, not those of my employers.

Summary

Cost-effectiveness of HSI

The calculations are in this Sheet.

I estimate the past cost-effectiveness of HSI is 431 DALY/$, multiplying:

My assumptions for the pain intensities are guesses for my personal time trade-offs, and imply each of the following individually neutralise 1 day of fully healthy life:

My estimate for the past cost-effectiveness of HSI is:

In addition, I estimate the past cost-effectiveness of HSI linked to helping shrimps slaughtered via:

Consequently, holding the number of shrimps helped per $ constant, it is much better to target producers slaughtering shrimps via air asphyxiation. I assumed the same number of shrimps helped per $ for both methods, so the higher cost-effectiveness of helping shrimps slaughtered via air asphyxiation comes from the respective reduction in the disease burden per shrimp being 21.0 times as large. Given this large difference, I asked Aaron whether he had any estimates/guesses for the fraction of shrimps HSI helps which were being slaughtered via each of the 2 methods. Here is Aaron’s reply:

This is tricky because the answer is kind of both...

Most of the producers we work with are already implementing some kind of ice slurry, as they typically sell to the European market, and ice slurry improves the quality (for example sometimes sodium metabisulfite is added to the slurry mix, which prevents discolouration).

However, for ice slurry to function as a slaughter method, the shrimp has to be submerged for long enough (which we currently understand to be over 30 seconds), and typically instead the crates of shrimps are “dipped” into ice slurry for a few seconds, so in reality once removed they die of asphyxiation/crushing.

Though it’s important to note that practices can vary significantly by country and production systems (and producer).

I am currently planning to direct my next annual donations to SWP.

Cost-effectiveness of sludge removal

SWP also works on removing sludge from shrimp ponds, which:

Aaron noted he thinks most of the benefits come from the 1st point:

I also want to add that although we do ask farmers to lower their densities - densities in India are already pretty low anyway (at least in the remote villages we’re working in), so I don’t think most of the benefits are realised here, I do think it’s mostly water quality improvements (and primarily hydrogen sulphide and un-ionised ammonia).

I believe SWP’s work on removing sludge from shrimp ponds might be even more cost-effective than HSI, but it is not clear. I calculate high stocking density and un-ionised ammonia account for 5.32 and 7.72 times as much suffering as ice slurry slaughter. Nonetheless, Aaron caveated that:

Our current shrimps helped / $ / year on this is ~500 [i.e. 1/3 (= 500/(1.5*10^3)) of HSI’s] (though the India team keeps on optimising the process and bringing costs down, so this number is improving month after month - I’m hopeful this will eventually equal HSI).

Also worth noting that the scale here is much lower than that of HSI, each pond we work with typically stocks ~100,000 shrimps, meaning we're impacting in the range of 10s of millions of shrimps each year. Whereas with HSI, each producer we work with commits to stunning a minimum of 100 million shrimps. So even if cost-effectiveness here exceeded that of HSI, the number of shrimps helped will almost certainly be much higher with HSI.

It’s also tough to know here how to convert from per $/year to just per $, as sludge slowly re-accumulates, so the impact year after year is reduced (and the program hasn't been running for long enough for us to assess if farmers continue to remove sludge after our initial intervention).

Acknowledgements

Thanks to Michael Johnston for nudging me to Fermi estimate the cost-effectiveness of shrimp welfare interventions. Thanks to Aaron Boddy for feedback on some of the inputs. Thanks to Aaron and Michael for feedback on the draft.

Aaron Boddy🔸 @ 2024-10-07T09:30 (+38)

Thanks so much Vasco for your work on this! As with MHR in the past, we really appreciate folks doing in-depth analyses like this, and are very appreciative of the interest in our work :)
 

In the spirit of this week’s Forum theme, I wanted to provide some more context regarding SWP’s room for more funding.

Our overheads (i.e. salaries, travel/conferences) and program costs for the India sludge removal work, are currently covered by grants until the end of 2026. Meaning that any additional funds are put towards HSI. (For context, our secured grants do also cover the cost of some stunners, but HSI as a program is still able to absorb more funding).

Each stunner costs us $55k and we ask the producers we work with to commit to stunning a minimum of 120 million shrimps per annum. This results in a cost-effectiveness of ~2,000+ shrimps helped / $ / year (i.e. our marginal impact of additional dollars is higher than our historical cost-effectiveness).
 

We’re having our annual team retreat (which we call “Shrimposium”) next week, during which we hope to map out how we can deploy stunners in such a way as to catalyse a tipping point so that pre-slaughter stunning becomes the industry standard.

We’ve had some good indications recently that HSI does contribute to “locking-in” industry adoption, with Tesco and Sainsbury’s recently publishing welfare policies, building on similar wins in the past (such as M&S and Albert Heijn).

This has always been the Theory of Change for the HSI project. Although we’re very excited by how cost-effective it is in its own right, ultimately we want to catalyse industry-wide adoption - deploying stunners to the early adopters in order to build towards a tipping point that achieves critical mass. In other words, over the next few years we want to take the HSI program from Growth to Scale.

I would be surprised if post-Shrimposium our targets regarding HSI required less funding than our current projections. In other words, though I don’t currently have an exact sense of our room for more funding, I’m confident SWP is in a position to absorb significantly more funding to support our HSI work.
 

If anyone wants to reach out to me directly, you can contact me at aaron@shrimpwelfareproject.org. You can also donate to SWP through our website, or book a meeting with me via this link.

Angelina Li @ 2024-10-07T10:36 (+23)

Can I directly ask: Is SWP one of the orgs that has been affected by Good Ventures dropping support for some cause areas? (I couldn't parse all the context previously on a quick skim.)

I'm not sure how much they funded you previously, or if this is a concern. No worries if you can't or would prefer not to say :)

Aaron Boddy🔸 @ 2024-10-07T14:49 (+27)

Hey Angelina! Sure - happy to answer :) 
Yes we were affected by the Good Ventures announcement, and our current funding update is actually very similar to that of Wild Animal Initiative, as in:

  • The Navigation Fund (TNF) plans to fill SWP's funding gap left by OP, at least through the end of 2026.
  • We're keen on diversifying our funding, so as not to continue relying on a single funder

However we differ in that our budget is smaller than that of WAI, and the majority of it is put toward a single program (HSI)

Angelina Li @ 2024-10-07T17:45 (+16)

Wow, go TNF 🙌 💜 I didn't realize they were bailing out non wild animal welfare orgs as well, that's very cool -- and must have been a lift. That all makes sense. Congrats, and I'm wishing you success on the funding diversity front!

Vasco Grilo🔸 @ 2024-10-30T15:47 (+2)

Thanks for the great context, Aaron!

Is there any chance HSI may increase the number of shrimp? I guess it would tend to increase costs, and therefore decrease the number of shrimp. I ask because I estimate that moving from ice slurry to electrical stunning only increases welfare by 4.34 % (= 1 - 4.85/5.07). In this case, since I think farmed shrimp have negative lives (for any slaughter method), an increase of more than 4.34 % in the number of shrimp would make HSI harmful.

Aaron Boddy🔸 @ 2024-11-05T21:45 (+10)

Hey Vasco! Interesting question, unfortunately I don't know the answer...

My sense is no, as you say, the intervention increases costs without an increase in productivity for the producers. But ultimately an incentive here is continued market access, which I'm sure an economist could model whether or not this could lead to an increase in the number of shrimps (over time).

Another point to emphasise though - it's my sense that the intervention should be modelled as electrical stunning replaces air asphyxiation, rather than (perfectly implemented) ice slurry. Ice slurry slaughter is just a very difficult thing to do correctly in practice (and I’ve not seen it happen) - as even if at some point the shrimps are submerged in ice for a short period of time, it's often not long enough to kill them (~30seconds).

Vasco Grilo🔸 @ 2024-11-06T13:16 (+2)

Thanks, Aaron.

Another point to emphasise though - it's my sense that the intervention should be modelled as electrical stunning replaces air asphyxiation, rather than (perfectly implemented) ice slurry. Ice slurry slaughter is just a very difficult thing to do correctly in practice (and I’ve not seen it happen) - as even if at some point the shrimps are submerged in ice for a short period of time, it's often not long enough to kill them (~30seconds).

I accounted for badly implemented ice slurry slaughter. I assumed:

All of the shrimps helped transition to electrical stunning, 62.5 % (= 1 - 0.375) from air asphyxiation, and 37.5 % from ice slurry (= 0.75*0.5). I got these fractions assuming 75 % of the targeted producers use some form of ice slurry, half of those implement it properly, and the other half improperly to the point of it being practically equivalent to air asphyxiation. I made these assumptions having in mind Aaron’s comment at the end of this section.

In any case, based on my assumptions, it does not matter whether HSI is harmful for the 37.5 % of the affected shrimp which go from well implemented ice slurry to electrical stunning slaughter. The overall cost-effectiveness is dominated by making 62.5 % of the affected shrimp go from air asphyxiation to electrical stunning slaughter. I estimate 97.3 % (= 0.625*0.0447/0.0287) of the benefits come from helping shrimp slaughtered via air asphyxiation, and that the increase in welfare for these is 48.6 % (= 1 - 4.85/9.44), whereas the number of shrimp would very hardly increase by that. So the question of whether the number of shrimp increases is only relevant if a very small fraction of the helped shrimp is slaughtered via air asphyxiation (again, conditional on my assumptions).

MHR @ 2024-10-07T23:11 (+28)

Thanks for looking at this Vasco, it's always great to see others doing this kind of cost-effectiveness analysis. 

Your results indicate a substantially higher direct cost-effectiveness for SWP relative to the analysis I did last year. From looking at your methodology, I believe our primary difference comes from a difference in weighting the relative badness of different levels of pain. I used the same numbers as a 2023 RP report which weighted excruciating pain as 33 times worse than hurtful pain, while your weights put excruciating pain at 100000x worse than hurtful pain. 

I've updated towards thinking 33x is probably at least an order of magnitude too low (and more recent RP reports have used weights in the vicinity of 600x), but I would personally be skeptical of 100000x. 

Of course much of SWP's impact is through creating systemic change, so I don't want to over-emphasize the importance of these direct impact CEAs, as valuable as they are.

bruce @ 2024-10-07T23:38 (+12)

Without checking the assumptions / BOTEC in detail, I'll just flag that this implies you are indifferent between saving 1 human life (not just averting 1 malaria death) and helping ~1000-1900 shrimp die via electrical stunning instead of dying via air asphyxiation / ice slurry.[1]

(This is not an anti-SWP comment tbc!) 

  1. ^

    Depending on whether you are using $3000 to $5500 per life saved; 

    15000 * 3000 / 43500 = 1034.5
    15000 * 5500 / 43500 = 1896.6

Vasco Grilo🔸 @ 2024-10-08T07:58 (+2)

Thanks for the note, Bruce. Your estimate makes sense. Based on GiveWell's implicit valuation of saving a life of 51 DALY/life[1], and my 0.0288 DALYs averted per shrimp helped, helping 1.77 k (= 51/0.0288) shrimp is as good as saving a life.

  1. ^

    According to OP, “GiveWell uses moral weights for child deaths that would be consistent with assuming 51 years of foregone life in the DALY framework (though that is not how they reach the conclusion)”.

Angelina Li @ 2024-11-15T19:20 (+6)

Edit: I think I was just wrong about this, per Aaron's response here :) Whoops!

Small to medium nit: the costs reported in the SWP CEA you are using as a base don't seem to include operational overhead (e.g. see Aaron's comment that the stunners cost $55k, which is the same as the total cost reported in the CEA).

I understand SWP has fundraised for overhead separately, but IMO it should be best practice to include overhead costs in your denominator — obviously someone has to cover those costs, it's not like they don't exist!

MichaelStJules @ 2024-11-15T19:48 (+6)

If the overhead is already covered, the marginal cost-effectiveness, what we get from further donations, should just reflect what's not covered.

Angelina Li @ 2024-11-15T20:39 (+6)

Quick pushback: overhead is a part of the costs needed to make this product (good management is needed to keep systems running)[1] and ignoring that leads to weird conclusions.

e.g. I think under some versions of your view, if donor A pitches in $50k for a stunner, and then donor B pitches in last $5k, you give donor B all the credit. (Obligatory Nuno ref.) I might be misunderstanding you, apologies if so.

 

  1. ^

    There's a nearby thing about how "not all overhead costs are necessary program costs" or something, and I probably buy that multiplying by [total costs] / [program specific costs] might not be the right way to apportion overhead.

MichaelStJules @ 2024-11-16T01:43 (+4)

I agree that we should treat credit assignment differently, but when deciding what to fund, we should always be able to reduce the problem to "What maximizes?" or "What's the best that can be done with my $X?". I think the relevant question here is something like "If I donate $X to SWP, what happens?" And then you divide the (expected) difference in the counterfactuals by $X to get your cost-effectiveness.

I have three possibilities in mind. 1 is the scenario I expected to actually be the case here. 1 is mutually exclusive with 2 and 3, but 2 and 3 can happen together or either alone.

  1. If the overhead costs are fixed and already fully covered either way, then your donations just go towards non-overhead costs, and the marginal cost-effectiveness should not reflect overhead. This is the scenario I had in mind.
  2. If your donations influence how much overhead is necessary, e.g. because they need to hire more, and some of your donations go to that, then you need to include the extra overhead in the costs and make sure your impact reflects that.
  3. If your donations influence how much overhead costs are covered by the funders covering the overhead, then you need to worry about their counterfactuals, i.e. what they would have otherwise done with their money. That goes into the impact of your donations.

1 could still involve indirect effects to worry about, i.e. you might increase the probability that funders cover overhead in the future like this, for SWP or others, which actually means we get something like 3.

Henry Howard🔸 @ 2024-10-08T07:19 (+6)

I would discourage you from doing guesswork this rough and legitimising it by calling it a "Fermi estimate".

Most cells in your spreadsheet are commented "my guess".

Sometimes confidence intervals are so wide that they don't tell us anything. I think that's the case here.

Vasco Grilo🔸 @ 2024-10-08T08:26 (+2)

Thanks for looking into my calculations, Henry.

I would discourage you from doing guesswork this rough and legitimising it by calling it a "Fermi estimate".

Interesting. I was thinking that calling it a "Fermi estimate" would highlight it is a rough calculation instead of legitimising it. I also did not want to say something like "please note these numbers are super speculative, and you should not trust them at all", because I have only guessed parameters for which there is basically no data, so it seems hard to easily improve on my estimate. However, I definitely encourage people to make a copy of my sheet, and use their own numbers.

Most cells in your spreadsheet are commented "my guess".

Nitpick. Not "most", as only 30 % (= 6/20) of the inputs of the tab you were referring to say "My guess." in the notes. The fraction is lower considering all the cells in the tab, and even lower considering all the cells of all tabs.

The 6 cells in the tab you printed saying "My guess." in the notes refer to the following assumptions (which I mentioned in the post; I did not hide in the sheet any of my assumptions) for both air asphyxiation and electrical stunning slaughter:

  • Time in excruciating pain as a fraction of that in disabling pain equal to that of ice slurry.
  • Time in hurtful pain as a fraction of that in disabling pain equal to that of ice slurry.
  • Time in annoying pain as a fraction of that in hurtful pain equal to that of ice slurry.

I have now clarified these in the notes of the relevant cells in the tab. I made them because I only had data about the time in disabling pain for air asphyxiation and electrical stunning slaughter. Assuming these have the same proportion of time in pain across categories as ice slurry slaughter felt the most agnostic assumption to me (i.e. the one which would minimise over or underestimating cost-effectiveness). Do you have a better assumption?

Even if I assume air asphyxiation slaughter does not involve annoying, hurtful or excruciating pain to minimise the badness of the initial conditions, thus underestimating cost-effectiveness, I still conclude the past cost-effectiveness of HSI is 1.36 k times the marginal cost-effectiveness of GiveWell's top charities. @Henry Howard🔸[1], I encourage you to make a copy of the sheet, and try to get to a ratio lower than 1. I believe this is very hard, which makes me think HSI is robustly more cost-effective than GiveWell's top charities.

  1. ^

    I am tagging you here because I have added this sentence to the comment after posting it for the 1st time.

Henry Howard🔸 @ 2024-10-08T11:40 (+2)

The definition of Fermi estimate linked in this post defines a Fermi estimate as aiming to be within 1 magnitude of true. Given just the Rethink Priorities welfare range estimates span several magnitudes (infinite really, given lower bound is 0), this at least is incorrect.

This sort of chaining of EV calculations is common on this forum. I think it's counterproductive. Show the confidence intervals and it becomes clear that the result is as good as "I have no idea", which is a fine thing to say. Just say that.

Vasco Grilo🔸 @ 2024-10-08T12:16 (+4)

Could you clarify the argument you are making? I agree the 5th percentile past cost-effectiveness of HSI is 0 given this is RP's 5th percentile welfare range of shrimps. However, I think what matters is the expected cost-effectiveness. Are you suggesting one should disregard interventions whose 5th percentile cost-effectiveness is 0? Imagine one could pay 1 k$ to save 0 lives with 10 % probability, and 1 life with 90 % probability. The 5th percentile cost-effectiveness is 0 (the 5th percentile cost-effectiveness of deworming programs could also be super low?), but the expected cost-effectiveness is 0.9 life/k$, i.e. around 4.5 times the cost-effectiveness of GiveWell's top charities of 0.2 life/k$ (= 1/(5*10^3)).

Vasco Grilo🔸 @ 2024-10-17T15:05 (+4)
  • Disabling pain is 10 times as intense as fully healthy life.

I was previously assuming disabling pain to be 100 times as intense as fully healthy life, i.e. 10 (= 100/10) times as intense as I am assuming now. I updated after going through the studies discussed here, especially Wallenstein et. al (1980). According to this, it looks like disabling pain is 8.16 to 18.8 times as bad as annoying pain, whereas I was supposing disabling pain to be 1 k times as bad as annoying pain. Now I am assuming disabling pain is 100 times (= 10^2) as bad as annoying pain, which is still more intense than the suggested by the study, but not so much so.

My updated past cost-effectiveness of HSI is 431 DALY/$, which is 99.5 % (= 431/433) of my previous value of 433 DALY/$. There is basically no change because the cost-effectiveness is approximately proportional to the intensity of excruciating pain, which I have not updated.

Vasco Grilo🔸 @ 2024-10-21T21:37 (+2)

My estimate for the past cost-effectiveness of HSI is:

  • 28.7 times my estimate for the cost-effectiveness of corporate campaigns for chicken welfare of 15.0 DALY/$.
  • 43.4 k times my estimate for the cost-effectiveness of GiveWell’s top charities of 0.00994 DALY/$.

For HSI to be as cost-effective as GiveWell’s top charities, for example, one of the following would have to happen:

Vasco Grilo🔸 @ 2024-11-29T16:53 (+1)

My comparison between HSI and corporate campaigns for chicken welfare now relies on my updated cost-effectiveness analysis of the latter (instead of this one).