The person-affecting value of existential risk reduction

By Gregory Lewis🔸 @ 2018-04-13T01:44 (+64)

Introduction

The standard motivation for the far future cause area in general, and existential risk reduction in particular, is to point to the vast future that is possible providing we do not go extinct (see Astronomical Waste). One crucial assumption made is a 'total' or 'no-difference' view of population ethics: in sketch, it is just as good to bring a person into existence with a happy life for 50 years as it is to add fifty years of happy life to someone who already exists. Thus the 10lots of potential people give profound moral weight to the cause of x-risk reduction.

Population ethics is infamously recondite, and so disagreement with this assumption commonplace; many find at least some form of person affecting/asymmetrical view plausible: that the value of 'making happy people' is either zero, or at least much lower than the value of making people happy. Such a view would remove a lot of the upside of x-risk reduction, as most of its value (by the lights of the total view) is ensuring a great host of happy potential people exist.

Yet even if we discount the (forgive me) person-effecting benefit, extinction would still entail vast person-affecting harm. There are 7.6 billion people alive today, and 7.6 billion premature deaths would be deemed a considerable harm by most. Even fairly small (albeit non-pascalian) reductions in the likelihood of extinction could prove highly cost-effective.

To my knowledge, no one has 'crunched the numbers' on the expected value of x-risk reduction by the lights of person affecting views. So I've thrown together a guestimate as a first-pass estimate.

An estimate 

The (forward) model goes like this:

  1. There are currently 7.6 billion people alive on earth. The worldwide mean age is 38, and worldwide life expectancy is 70.5. 
  2. Thus, very naively, if 'everyone died tomorrow', the average number of life years lost per person is 32.5, and the total loss is 247 Billion life years.
  3. Assume the extinction risk is 1% over this century, uniform by year (i.e. the risk this year is 0.0001, ditto the next one, and so on.)
  4. Also assume the tractability of x-risk reduction is something like (borrowing from Millett and Snyder-Beattie) this: 'There's a project X that is expected to cost 1 billion dollars each year, and would reduce the risk (proportionately) by 1% (i.e. if we spent a billion each year this century, xrisk over this century declines from 1% to 0.99%).
  5. This gives a risk-reduction per year of around 1.3 * 10-6 , and so an expected value of around 330 000 years of life saved.

Given all these things, the model spits out a mean 'cost per life year' of $1500-$26000 (mean $9200). 

Caveats and elaborations

The limitations of this are nigh-innumerable, but I list a few of the most important below an approximately ascending order.

Zeroth: The model has a wide range of uncertainty, and reasonable sensitivity to distributional assumptions: you can modulate mean estimate and range by a factor of 2 or so by whether the distributions used are Beta, log normal, or tweaking their variance. 

First: Adjustment to give 'cost per DALY/QALY' would be somewhat downward, although not dramatically (a factor of 2 would imply everyone who continues to live does so with a disability weight of 0.5, in the same ballpark as those used for major depression or blindness).

Second, trends may have a large impact, although their importance is modulated by which person-affecting view is assumed. I deliberately set up the estimate to work in a 'one shot' single year case (i.e. the figure applies to a 'spend 1B to reduce extinction risk in 2018 from 0.0001 to 0.000099' scenario).

By the lights of a person-affecting view which considers only people who exist now, making the same investment 10 years from now (i.e. spent 1B to reduce extinction risk in 2028 from 0.0001 to 0.000099) is less attractive, as some of these people would have died, and the new people who have replaced them have little moral relevance. These views thus imply a fairly short time horizon, and are particularly sensitive to x-risk in the near future. Given the '1%' per century is probably not uniform by year, and plausibly lower now but higher later, this would imply a further penalty to cost-effectiveness.

Other person affecting views consider people who will necessarily exist (however cashed out) rather than whether they happen to exist now (planting a bomb with a timer of 1000 years is still accrues person-affecting harm). In a 'extinction in 100 years' scenario, this view would still count the harm of everyone alive then who dies, although still discount the foregone benefit of people who 'could have been' subsequently in the moral calculus. 

Thus the trends in factual basis become more salient. One example is the ongoing demographic transition, and the consequently older population give smaller values of life-years saved if protected from extinction in the future. This would probably make the expected cost-effectiveness somewhat (but not dramatically) worse.

A lot turns on the estimate for marginal 'x-risk reduction'. I think the numbers offered in terms of base rate, and how much it can be reduced for now much lean on the conservative side of the consensus of far-future EAs. Confidence in (implied) scale or tractability an order of magnitude impose commensurate increases on the risk estimate. Yet in such circumstances the bulk of disagreement is explained by empirical disagreement rather than a different take on the population ethics.

Finally, this only accounts for something like the (welfare) 'face value' of existential risk reduction. There would be some further benefits by the light of the person-affecting view itself, or ethical views which those holding a person affecting view are likely sympathetic to: extinction might impose other harms beyond years of life lost; there could be person affecting benefits if some of those who survive can enjoy extremely long and happy lives; and there could be non-welfare goods on an objective list which rely on non-extinction (among others). On the other side, those with non-deprivationists accounts of the badness of death may still discount the proposed benefits.

Conclusion

Notwithstanding these challenges, I think the model, and the result that the 'face value' cost-effectiveness of x-risk reduction is still pretty good, is instructive.

First, there is a common pattern of thought along the lines of, "X-risk reduction only matters if the total view is true, and if one holds a different view one should basically discount it". Although rough, this cost-effectiveness guestimate suggests this is mistaken. Although it seems unlikely x-risk reduction is the best buy from the lights of a person-affecting view (we should be suspicious if it were), given ~$10000 per life year compares unfavourably to best global health interventions, it is still a good buy: it compares favourably to marginal cost effectiveness for rich country healthcare spending, for example.  

Second, although it seems unlikely that x-risk reduction would be the best buy by the lights of a person affecting view, this would not be wildly outlandish. Those with a person-affecting view who think x-risk is particularly likely, or that the cause area has easier wins available than implied in the model, might find the best opportunities to make a difference. It may therefore supply reason for those with such views to investigate the factual matters in greater depth, rather than ruling it out based on their moral commitments.

Finally, most should be morally uncertain in matters as recondite as population ethics. Unfortunately, how to address moral uncertainty is similarly recondite. If x-risk reduction is 'good but not the best' rather than 'worthless' by the lights of person affecting views, this likely implies x-risk reduction looks more valuable whatever the size of the 'person affecting party' in one's moral parliament.

null @ 2018-04-13T11:09 (+16)

Thanks for writing this up! This does seem to be an important argument not made often enough.

To my knowledge this has been covered a couple of times before, although not as thoroughly.

Once by Oxford Prioritization Project however they approached it from the other end, instead asking "what absolute percentage x-risk reduction would you need to get for £10,000 for it to be as cost effective as AMF" and finding the answer of 4 x 10^-8%. I think your model gives £10,000 as reducing x-risk by 10^-9%, which fits with your conclusion of close but not quite as good as global poverty.

Note they use 5% before 2100 as their risk, also do not consider QALYs, instead only looking at 'lives saved' which is likely bias them against AMF, since it mostly saves children.

We also calculated this as part of the Causal Networks Model I worked on with Denise Melchin at CEA over the summer. The conclusion is mentioned briefly here under 'existential effectiveness'.

I think our model was basically the same as yours, although we were explicitly interested in the chance of existential risk before 2050, and did not include probabilistic elements. We also tried to work in QALYs, although most of our figures were more bullish than yours. We used by default:

Ignoring these second order effects then, our model suggested it would cost £5 billion to reduce x-risk by 1% point, which corresponds to a cost of about £2 per QALY. In retrospect this should be at least 1 or 2 orders of magnitude higher (increasing researcher cost and decreasing x-risk possibility by and order of magnitude each).

I find your x-risk chance somewhat low, I think 5% before 2100 seems more likely. Your cost-per-percent to reduce x-risk also works out as much higher than the one we used, but seems more justified (ours was just pulled from the air as 'reasonable sounding').

null @ 2018-04-20T06:07 (+6)

I also made a very rough estimate in this article: https://80000hours.org/articles/extinction-risk/#in-total-how-effective-is-it-to-reduce-these-risks Though this estimate is much better and I've added a link to it.

I also think x-risk over the century is over 1%, and we can reduce it much more cheaply than your guess, though it's nice to show it's plausible even with conservative figures.

CarlShulman @ 2018-04-13T18:43 (+13)

Other person affecting views consider people who will necessarily exist (however cashed out) rather than whether they happen to exist now (planting a bomb with a timer of 1000 years is still accrues person-affecting harm). In a 'extinction in 100 years' scenario, this view would still count the harm of everyone alive then who dies, although still discount the foregone benefit of people who 'could have been' subsequently in the moral calculus.

Butterfly effects change the identities of at least all yet-to-be conceived persons, so this would have to not be interested in particular people, but population sizes/counterparts.

null @ 2018-04-13T21:31 (+2)

+1. Navigating this is easier said than done, and one might worry about some sort of temporal parochialism being self-defeating (persons at t1-tn are all better off if they cooperate across cohort with future-regarding efforts instead of all concerning themselves with those who are morally salient at their corresponding t).

My impression is those with person-affecting sympathies prefer trying to meet these challenges rather than accept the moral character of destructive acts change with a (long enough) delay, or trying to reconcile this with the commonsensical moral importance of more normal future-regarding acts (e.g. climate change, town planning, etc.)

null @ 2018-04-13T17:07 (+10)

I've been saying to people that I wish there was a post series about all the practical implications of different philosophical positions (I often have the unflattering impression philosophy EAs like to argue about them just because it's their favourite nerd topic - and not because of the practical relevance).

So special thanks to you for starting it! ;-)

ElizabethBarnes @ 2018-04-13T13:57 (+6)

See also the models in https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5576214/ (cost-effectiveness of mitigating biorisk) and https://onlinelibrary.wiley.com/doi/full/10.1111/j.1539-6924.2007.00960.x (asteroid risk), which have estimates for the risk level, cost of reducing it, and cost per qualy for different future discount levels.

"If we ignore distant future generations by discounting, the benefits of reducing existential risk fall by between 3 and 5 orders of magnitude (with a 1% to 5% discount rate), which is still far more cost-effective than measures to reduce small-scale casualty events. Under our survey model (Model 1), the cost per life-year varies between $1,300 and $52,000 for a 5% discount rate and between $770 and $30,000 for a 1% discount rate. These costs are even competitive with first-world healthcare spending, where typically anything less than $100,000 per quality adjusted life-year is considered a reasonable purchase.29

This suggests that even if we are concerned about welfare only in the near term, reducing existential risks from biotechnology is still a cost-effective means of saving expected life if the future chance of an existential risk is anything above 0.0001 per year."

I think their model ought to include a category of catastrophic risk - they don't have anything between disaster (100,000 deaths) and extinction.

"Even if we expected humanity to become extinct within a generation, traditional statistical life valuations would warrant a 32 billion annual investment in asteroid defense (Gerrard & Barber, 1997). Yet the United States spends only $4 million per year on asteroid detection and there is no direct spending on mitigation."

null @ 2018-04-20T18:39 (+5)

I made a similar observation about AI risk reduction work last year:

"Someone taking a hard 'inside view' about AI risk could reasonably view it as better than AMF for people alive now, or during the rest of their lives. I'm thinking something like:

1 in 10 risk of AI killing everyone within the next 50 years. Spending an extra $1 billion on safety research could reduce the size of this risk by 1%.

$1 billion / (0.1 risk reduced by 1% 8 billion lives) = $125 per life saved. Compares with $3,000-7,000+ for AMF.

This is before considering any upside from improved length or quality of life for the present generation as a result of a value-aligned AI.

I'm probably not quite as optimistic as this, but I still prefer AI as a cause over poverty reduction, for the purposes of helping the present generation (and those remaining to be born during my lifetime)."

http://effective-altruism.com/ea/18u/intuition_jousting_what_it_is_and_why_it_should/amj

null @ 2018-04-15T16:05 (+4)

Three cheers for this. Two ways in which the post might understate the case for person-affecting focusing on ex risk

  1. Most actions to reduce ex risk would also reduce catastrophic non-ex risks. e.g. efforts to reduce the risk of an existential threat attack by an engineered pathogen would also reduce the risk of e.g. >100m people dying in an attack by an engineered pathogen. I would expect that the benefits from reducing GCRs as a side-effect of reducing ex risks would be significantly larger than the benefits accruing from preventing ex risks because the probability of GCRs is much much greater. I wouldn't be that surprised if that increased the EV of ex risk by an order of magntidue, thereby propelling ex risk reduction further into AMF territory.

  2. As I have noted before on this forum, most people advancing person-affecting views tend to opt for asymmetric versions where future bad lives matter but future good lives don't. If you're temporally neutral and aggregative, then you end up with a moral theory which is practically exactly the same as negative utiltiarianism (priorities one two three four etc are preventing future suffering).

It is in general good to reassert that there are numerous reasons to focus on ex risk aside from the total view, including neglectedness, political short-termism, the global public goods aspect, the context of the techologies we are developing, the tendency to neglect rare events etc

null @ 2018-04-15T22:42 (+4)

As I have noted before on this forum, most people advancing person-affecting views tend to opt for asymmetric versions where future bad lives matter but future good lives don't. If you're temporally neutral and aggregative, then you end up with a moral theory which is practically exactly the same as negative utiltiarianism (priorities one two three four etc are preventing future suffering

If someone did take an asymmetric view and really committed to it, I would think you should probably be in favour of increasing existential risk, as that removes the possibility of future suffering, rather trying to reduce existential risk. I suppose you might have some (not obviously plausible) story you had about how humanity's survival decreases future suffering: You could think humans will remove misery in surviving non-humans if humans dodge existential risk, but this misery wouldn't be averted if humans went extinct but other life keep living.

null @ 2018-04-16T09:02 (+2)

I think the argument is as you describe in the last sentence, though I haven't engaged much with the NUs on this.

null @ 2018-04-20T20:43 (+2)

thanks, gregory. it's valuable to have numbers on this but i have some concerns about this argument and the spirit in which it is made:

1) most arguments for x-risk reduction make the controversial assumption that the future is very positive in expectation. this argument makes the (to my mind even more) controversial assumption that an arbitrary life-year added to a presently-existing person is very positive, on average. while it might be that many relatively wealthy euro-american EAs have life-years that are very positive, on average, it's highly questionable whether the average human has life-years that are on average positive at all, let alone very positive.

2) many global catastrophic risks and extinction risks would affect not only humans but also many other sentient beings. insofar as these x-risks are risks of the extinction of not only humans but also nonhuman animals, to make a determination of the person-affecting value of deterring x-risks we must sum the value of preventing human death with the value of preventing nonhuman death. on the widely held assumption that farmed animals and wild animals have bad lives on average, and given the population of tens of billions of presently existing farmed animals and 10^13-10^22 presently existing wild animals, the value of the extinction of presently living nonhuman beings would likely swamp the (supposedly) negative value of the extinction of presently existing human beings. many of these animals would live a short period of time, sure, but their total life-years still vastly outnumber the remaining life-years of presently existing humans. moreover, most people who accept a largely person-affecting axiology also think that it is bad when we cause people with miserable lives to exist. so on most person-affecting axiologies, we would also need to sum the disvalue of the existence of future farmed and wild animals with the person-affecting value of human extinction. this may make the person-affecting value of preventing extinction extremely negative in expectation.

3) i'm concerned about this result being touted as a finding of a "highly effective" cause. $9,600/life-year is vanishingly small in comparison to many poverty interventions, let alone animal welfare interventions (where ACE estimates that this much money could save 100k+ animals from factory farming). why does $9,600/life-year suddenly make for a highly effective when we're talking about x-risk reduction, when it isn't highly effective when we're talking about other domains?

null @ 2018-04-20T21:26 (+5)

1) Happiness levels seem to trend strongly positive, given things like the world values survey (in the most recent wave - 2014, only Egypt had <50% of people reporting being either 'happy' or 'very happy', although in fairness there were a lot of poorer countries with missing data. The association between wealth and happiness is there, but pretty weak (e.g. Zimbabwe gets 80+%, Bulgaria 55%). Given this (and when you throw in implied preferences, commonsensical intuitions whereby we don't wonder about whether we should jump in the pond to save the child as we're genuinely uncertain it is good for them to extent their life), it seems the average human takes themselves to have a life worth living. (q.v.)

2) My understanding from essays by Shulman and Tomasik is that even intensive factory farming plausibly leads to a net reduction in animal populations, given a greater reduction in wild animals due to habitat reduction. So if human extinction leads to another ~100M years of wildlife, this looks pretty bad by asymmetric views.

Of course, these estimates are highly non-resilient even with respect to sign. Yet the objective of the essay wasn't to show the result was robust to all reasonable moral considerations, but that the value of x-risk reduction isn't wholly ablated on a popular view of population ethics - somewhat akin to how Givewell analysis on cash transfers don't try and factor in poor meat eater considerations.

3) I neither 'tout' - nor even state - this is a finding that 'xrisk reduction is highly effective for person-affecting views'. Indeed, I say the opposite:

Although it seems unlikely x-risk reduction is the best buy from the lights of the [ed: typo - as context suggests, meant 'person-affecting'] total view (we should be suspicious if it were), given $13000 per life year compares unfavourably to best global health interventions, it is still a good buy: it compares favourably to marginal cost effectiveness for rich country healthcare spending, for example.

null @ 2018-04-20T21:28 (+3)

I'm surprised by your last point, since the article says:

Although it seems unlikely x-risk reduction is the best buy from the lights of the total view (we should be suspicious if it were), given $13000 per life year compares unfavourably to best global health interventions, it is still a good buy: it compares favourably to marginal cost effectiveness for rich country healthcare spending, for example.

This seems a far cry from the impression you seem to have gotten from the article. In fact your quote of "highly effective" is only used once, in the introduction, as a hypothetical motivation for crunching the numbers. (Since, a-priori, it could have turned out the cost effectiveness was 100 times higher, which would have been very cost effective).

On your first two points, my (admittedly not very justified) impression is the 'default' opinons people typically have is that almost all human lives are positive, and that animal lives are extremely unimportant compared to humans. Whilst one can question the truth of these claims, writing an article aimed at the majority seems reasonable.

It might be that actually within EA the average opinion is closer to yours, and in any case I agree the assumptions should have been clearly stated somewhere, along with the fact he is taking the symmetric as opposed to asymmetric view etc.

null @ 2018-04-17T11:47 (+2)

Thanks for this. Other previous work includes my extension of the Oxford Prioritization Project AI model that actually showed better cost-effectiveness than AMF (near the bottom). And to Halstead's point, global catastrophic risk reduction is generally even higher cost effectiveness from the perspective of the present generation than existential risk reduction, such as this.

WilliamKiely @ 2021-07-01T20:47 (+1)

I believe there's a minor error in your Guesstimate model.  The "proportional x-risk reduction" should be 0.01, not 0.011, to correspond to the 1% proportional reduction mentioned in the post:

Edit: Actually, never mind. I see this is an artifact of inputting the reduction as an uncertain value from 0.005 to 0.02.

null @ 2018-04-17T15:14 (+1)

Thus the trends in factual basis become more salient. One example is the ongoing demographic transition, and the consequently older population give smaller values of life-years saved if protected from extinction in the future. This would probably make the expected cost-effectiveness somewhat (but not dramatically) worse.

I think this is largely compensated by a rise in average life-expectancy.

I'd also like to remark Bostrom's point in Astronomical Waste that extinction could prevent current people from living billions of years, and that this gives enough reason for person-affecting utilitarians to prioritize x-risk reduction.

From Bostrom (2003):

"[..] we ought to assign a non-negligible probability to some current people surviving long enough to reap the benefits of a cosmic diaspora. A so-called technological “singularity” might occur in our natural lifetime, or there could be a breakthrough in life-extension [..] Clearly, avoiding existential calamities is important, not just because it would truncate the natural lifespan of six billion or so people, but also [..] because it would extinguish the chance that current people have of reaping the enormous benefits of eventual colonization."

stijnbruers @ 2018-04-15T19:04 (+1)

Perhaps interesting in this context: my current population ethical view of variable critical level utilitarianism https://stijnbruers.wordpress.com/2018/02/24/variable-critical-level-utilitarianism-as-the-solution-to-population-ethics/

null @ 2018-04-13T11:31 (+1)

Edit: My comment is wrong - i had misread the price as £1 billion as a one-off, but it is £1 billion per year

I'm not quite able to follow what role annualising the risk plays in your model, since as far as I can tell you seem to calculate your final cost effectiveness in terms purely of the risk reduction in 1 year. This seems like it should undercount the impact 100-fold.

e.g. if I skip annualising entirely, and just work in century blocks I get:

I might well have misunderstood some important part of your model, or be making some probability-related mistake.

null @ 2018-04-13T11:58 (+2)

The mistake might be on my part, but I think where this may be going wrong is I assume the cost needs to be repeated each year (i.e. you spent 1B to reduce risk by 1% in 2018, then have to spend another 1B to reduce risk by 1% in 2019). So if you assume a single 1B pulse reduces x risk across the century by 1%, then you do get 100 fold better results.

I mainly chose the device of some costly 'project X' as it is hard to get a handle on (e.g.) whether 10^-10 reduction in xrisk/$ is a plausible figure or not. Given this, I might see if I can tweak the wording to make it clearer - or at least make any mistake I am making easier to diagnose.