In defence of fanaticism

By Global Priorities Institute @ 2021-01-01T14:33 (+13)

This is a linkpost to https://globalprioritiesinstitute.org/hayden-wilkinson-in-defence-of-fanaticism/

Abstract

Consider a decision between: 1) a certainty of a moderately good outcome, such as one additional life saved; 2) a lottery which probably gives a worse outcome, but has a tiny probability of some vastly better outcome (perhaps trillions of additional blissful lives created). Which is morally better? By expected value theory (with a plausible axiology), no matter how tiny its probability of the better outcome, (2) will be better than (1) as long as that better outcome is good enough. But this seems fanatical. So you may be tempted to abandon expected value theory.

But not so fast — denying all such fanatical verdicts brings serious problems. For one, you must reject either that moral betterness is transitive or even a weak principle of tradeoffs. For two, you must accept that judgements are either: inconsistent over structurally-identical pairs of lotteries; or absurdly sensitive to small probability differences. For three, you must accept that the practical judgements of agents like us are sensitive to our beliefs about far-off events that are unaffected by our actions. And, for four, you may also be forced to accept judgements which you know you would reject if you simply learned more. Better to accept fanaticism than these implications.

Introduction

Suppose you face the following moral decision.

Dyson’s Wager

You have $2,000 to use for charitable purposes. You can donate it to either of two charities.

The first charity distributes bednets in low-income countries in which malaria is endemic.^[1] With an additional $2,000 in their budget this year, they would prevent one additional death from malaria. You are certain of this.

The second charity does speculative research into how to do computations using ‘positronium’ — a form of matter which will be ubiquitous in the far future of our universe. If our universe has the right structure (which it probably does not), then in the distant future we may be able to use positronium to instantiate all of the operations of human minds living blissful lives, and thereby allow morally valuable life to survive indefinitely long into the future^[2].^[3] From your perspective as a good epistemic agent, there is some tiny, non-zero probability that, with (and only with) your donation, this research would discover a method for stable positronium computation and would be used to bring infinitely many blissful lives into existence.^[4]

What ought you do, morally speaking? Which is the better option: saving a life with certainty, or generating a tiny probability of bringing about infinitely many future lives?

A common view in normative decision theory and the ethics of risk — expected value theory — says that it’s better to donate to the speculative research. Why? Each option has some probability of bringing about each of several outcomes, and each of those outcomes has some value, specified by our moral theory. Expected value theory says that one option is better than another if and only if it has the greater probability-weighted sum of value — the greater expected value.^[5] Here, the option with the greater expected value is donating to the speculative research (at least on certain theories of value — more on those in a moment). So perhaps that is what you should do.

That verdict of expected value theory is counterintuitive to many. All the more counterintuitive is that it can still be better to donate to speculative research no matter how low the probability is (short of being 0)^[6], since there are so many blissful lives at stake. For instance, the odds of your donation actually making the research succeed could be 1 in 10 to the one hundredth power. (10 to the one hundredth power is greater than the number of atoms in the observable universe). The chance that the research yields nothing at all would be 99.99... percent, with another 96 nines after that. And yet expected value theory says that it is better to take the bet, despite it being almost guaranteed that it will actually turn out worse than the alternative; despite the fact that you will almost certainly have let a person die for no actual benefit. Surely not, says my own intuition. On top of that, suppose that $2,000 spent on preventing malaria would save more than one life. Suppose it would save a billion lives, or any enormous finite value. Expected value theory would say that it’s still better to fund the speculative research — expected value theory says that it would be better to sacrifice those billion or more lives for a minuscule chance at the infinitely many blissful lives (and likewise if the number of blissful lives were finite but still sufficiently many). But endorsing that verdict, regardless of how low the probability of success and how high the cost, seems fanatical. Likewise, even without infinite value at stake, it would also seem fanatical to judge a lottery with sufficiently tiny probability of arbitrarily high finite value as better than getting some modest value with certainty.

Fanatical verdicts depend on more than just our theory of instrumental rationality, expected value theory. They also depend on our theory of (moral) value, or axiology. Various plausible axiologies, in conjunction with expected value theory, deliver that fanatical conclusion. Foremost among them is totalism: that the ranking of outcomes is determined by the total aggregate of value of each outcome; and that this total value increases linearly, without bound, with the sum of value in all lives that ever exist. By totalism, the outcome containing infinitely many blissful lives is indeed a much better one than that in which one life is saved. And, as we increase the number of blissful lives, we can increase how much better it is without bound. No matter how low the probability of those many blissful lives, the expected total value of the speculative research is greater than that of malaria prevention. (Likewise, even if there are only finitely many blissful lives at stake, for any tiny probability there can be sufficiently many of them to make the risky gamble better than saving a life with certainty.) But this problem isn’t unique to totalism. When combined with expected value theory, analogous problems face most competing axiologies, including: averageism, critical-level views, prioritarianism, pure egalitarianism, maximin, maximax, and narrow person-affecting views. Those axiologies each allow possible outcomes to be unboundedly valuable, so it’s easy enough to construct cases like Dyson’s Wager for each.^[7] And some — namely, critical-level views and prioritarianism — already deliver the same result as totalism in the original Dyson’s Wager. In this paper, I’ll focus on totalism, both to streamline the discussion and because it seems to me far more plausible than the others.^[8] But suffice it to say that just about any plausible axiology can deliver fanatical verdicts when combined with expected value theory.

In general, we will sometimes be led to verdicts that seem fanatical if we endorse Fanaticism.^[9] Inversely, to succeed in avoiding fanatical verdicts, our theory of instrumental rationality and our axiology must not imply Fanaticism.

Fanaticism: For any tiny (finite) probability > 0, and for any finite value v, there is some finite V that is large enough that L risky is better than L safe (no matter which scale those cardinal values are represented on).

L risky: value V with probability; value 0 otherwise

L safe: value v with probability 1

The comparison of lotteries L risky and L safe resembles Dyson’s Wager: one option gives a slim chance of a potentially astronomical value V ; the other a certainty of some modest value v. But, here, V need not be infinite, in case you think infinite value impossible. And, with some minor assumptions (see Section 3), Fanaticism in this form implies the fanatical verdict in Dyson’s Wager. Likewise, to reject the fanatical verdict in Dyson’s Wager, we must reject Fanaticism.

Note that Fanaticism is quite a strong claim. As defined here, it requires that the ranking of L risky above L safe holds not only when the number of lives in each outcome are proportional to V, v,_ and 0. It must hold whenever outcomes can be cardinally represented with those values. Recall that cardinal representations of value^[10] are unique only up to positive affine transformations — two outcomes represented by 0 and v on one scale could instead be represented by 0 × a + b and v × a + b (for any positive a and real b). Conversely, an outcome that contains many happy lives might still be represented cardinally with value 0. So Fanaticism doesn’t apply only to risky lotteries in which some possible outcome contains zero valuable lives, or zero value on net. It also applies to lotteries that can be represented as L risky and L safe even though every one of their outcomes contains enormous numbers of blissful lives, or enormous amounts of suffering, as long the differences in value between those outcomes are proportional to 0, v, and V.

Given how strong and how counterintuitive Fanaticism is, you might think it easy to reject. And many philosophers and other thinkers have done so, rejecting either Fanaticism or similar principles. For instance, Bostrom (2009) presents a compelling reductio ad absurdum for fanatical verdicts in the prudential context. Bostrom (2011), Beckstead (2013), and Askell (2019) treat even a weak form of (moral) Fanaticism as a reductio for moral theories. Others propose theories of rationality with the expressed purpose of avoiding fanatical verdicts. For instance, some propose that we simply ignore outcomes with small enough probabilities (e.g., D’Alembert 1761; Buffon 1777; Smith 2014^[11]; Monton 2019). Others insist that we maximise not expected moral value but instead the expected utility of outcomes (given by some increasing function of an outcome’s value), and that the correct utility function is bounded above so as to keep the expectation of utility bounded as well (e.g., Arrow 1971: 64).

Meanwhile, there are few defenders of Fanaticism or, more broadly, of fanatical verdicts in cases similar to Dyson’s Wager. Notable examples include Pascal (1669) himself, Parfit (1984: §3.27), and H´ajek (2014). And even they most often endorse such verdicts only because they are a consequence of expected value theory, not because they see good independent justification for them. I suspect that even many diehard adherents of expected value theory are uncomfortable with the fanatical verdicts supplied by their theory

This situation is unfortunate. There are compelling arguments in favour of Fanaticism that do not rely on expected value theory, and so we have good reason to accept it even if we reject that particular theory. If we do not, we face disturbing implications.

The paper proceeds as follows. Section 2 addresses some common motivations for rejecting Fanaticism. Section 3 introduces the necessary formal framework for what follows. Sections 4 through 6 present arguments in favour of Fanaticism, each premised on weaker claims than expected value theory, and each (by my reckoning) more compelling than the last. The first is a basic continuum argument. The second is that, to deny Fanaticism, we must accept either what I’ll call ‘scale dependence’ or an absurd sensitivity to arbitrarily small differences in probability. And the final nails in the coffin are an updated version of Parfit’s classic Egyptology Objection and what I’ll call the Indology Objection, by which those who deny Fanaticism must make judgements which appear deeply irrational. Section 7 is the conclusion.

Read the rest of the paper