Arguments for utilitarianism are impossibility arguments under unbounded prospects

Thanks for the feedback and criticism!

Too much work done by citations.

Hmm, I didn't expect or intend for people to dig through the links, but it looks like I misjudged what things people would find cruxy for the rest of the arguments but not defended well enough, e.g. your concerns with infinite expected utility.

EDIT: I've rewritten the arguments for possibly unbounded impacts.

The arguments for infinite prospective utility didn't hold up for me. A spatially infinite universe doesn't give us infinite expectation from our action - even if the universe never ends, our light cone will always be finite.

But can you produce a finite upper bound on our lightcone that you're 100% confident nothing can pass? (It doesn't have to be tight.) If not, then you could consider a St Petersburg-like prospect, with for each $n$, has probability $1/2^n$  of size (or impact) $2^n$, in whatever units you're using. That's finite under every possible outcome, but it has an infinite expected value.

Re Oesterheld's paper, acausal influence seems an extremely controversial notion in which I personally see no reason to believe.

Section II from Carlsmith, 2021 is one of the best arguments for acausal influence I'm aware of, in case you're interested in something more convincing. (FWIW, I also thought acausal influence was crazy for a long time, and I didn't find Newcomb's problem to be a compelling reason to reject causal decision theory.)

EDIT: I've now cut the acausal stuff and just focus on unbounded duration.

It seemed like you just attempted to define these things and then asserted this - maybe I missed something in the definition?

This follows from the theorems I cited, but I didn't include proofs of the theorems here. The proofs are technical and tricky,^[1] and I didn't want to make my post much longer or spend so much more time on it. Explaining each proof in an intuitive way could probably be a post on its own.

I don't see how this makes sense. If all possible outcomes have me living a finite amount of time and generating finite utility per life-year, I don't see why expectation would be infinite.

How long you live could be distributed like a St Petersburg gamble, e.g. for each $n$, with probability $1/2^n$., you could live $2^n$ years. The expected value of that is infinite, even though you'd definitely only live a finite amount of time.

Too much emphasis on what you find 'plausible'. IMO philosophy arguments should just taboo that word.

Ya, I got similar feedback on an earlier draft for making it harder to read, and tried to cut some uses of the word, but still left a bunch. I'll see if I can cut some more.

1. ^{^}

   They work by producing some weird set of prospects. They then show that you can't order them in a way that satisfies the axioms, applying them one-by-one and then violating one of them or getting a contradiction.

Are there other violations of the principle of reflection that aren't avoidable? I'm not familiar with it

The case reminded me of one you get without countable additivity. Suppose you have two integers drawn with a fair chancy process that is as likely to result in any integer. What’s the probability the second is greater than the first? 50 50. Now what if you find out the first is 2? Or 2 trillion? Or any finite number? You should then think the second is greater.

I think Parfit's Hitchhiker poses a similar problem for everyone, though.

You're outside town and hitchhiking with your debit card but no cash, and a driver offers to drive you if you pay him when you get to town. The driver can also tell if you'll pay (he's good at reading people), and will refuse to drive if he predicts that you won't. Assuming you'd rather keep your money than pay conditional on getting into town, it would be irrational to pay then (you wouldn't be following your own preferences). So, you predict that you won't pay. And then the driver refuses to drive you, and you lose.

So, the thing to do here is to somehow commit to paying and actually pay, despite it violating your later preference to not pay when you get into town.

And we might respond the same way for A vs B-$100 in the money pump in the post: just commit to sticking with A (at least for high enough value outcomes) and actually do it, even though you know you'll regret it when you find out the value of A.

So maybe (some) money pump arguments prove too much? Still, it seems better to avoid having these dilemmas when you can, and unbounded utility functions face more of them.

On the other hand, you can change your preferences so that you actually prefer to pay when you get into town. Doing the same for the post's money pump would mean actually not preferring B-$100 over some finite outcome of A. If you do this in response to all possible money pumps, you'll end up with a bounded utility function (possibly lexicographic, possibly multi-utility representation). Or, this could be extremely situation-specific preferences. You don't have to prefer to pay drivers all the time, just in Parfit's Hitchhiker situations. In general, you can just have preferences specific to every decision situation to avoid money pumps. This violates the Independence of Irrelevant Alternatives, at least in spirit.

See also https://www.lesswrong.com/tag/parfits-hitchhiker

I personally think unbounded utility functions don't work, I'm not claiming otherwise here, the comment above is about the thought experiment.

One way to illustrate this point is with Turing machines,^[1] with finite but arbitrarily expandable tape to write on for memory. There are (finite) Turing machines that can handle arbitrarily large finite inputs, e.g. doing arithmetic with or comparing two arbitrarily large but finite integers. They only use a finite amount of tape at a time, so we can just feed more and more tape for larger and larger numbers. So, never actually infinite, but arbitrarily large and arbitrarily expandable. A similar argument might apply for more standard computer architectures, with expandable memory, but I'm not that familiar with how standard computers work.

You might respond that we need an actual infinite amount of possible tape (memory space) to be able to do this. Like there has to be an infinite amount of matter available to us to turn into memory space. That isn't true. The universe and amount of available matter for tape could be arbitrarily large but finite, and we could (in principle) need less tape than what could be available in all possible outcomes, and the amount of tape we'd need will scale with the "size" or value of the outcome. For example, if we want to represent how long you'll live in years or an upper bound for it, in case you might live arbitrarily long, the amount of tape you'd need would scale with how long you'd live. It could scale much slower, e.g. we could represent the log of the number of years, or the log of the log, or the log of the log of the log, etc.. Still, the length of the tape would have to be unbounded across outcomes.

So, I'd have concerns about:

1. there not being enough practically accessible matter available (even if we only ever need a finite amount), and
2.  the tape being too spread out spatially to work (like past the edge of the observable universe), or
3. the tape not being packable densely enough without collapsing into a black hole.

So the scenario seems unrealistic and physically impossible. But if it's impossible, it's for reasons that don't have to do with infinite value or infinitely large things (although black holes might involve infinities).

1. ^{^}

   For anyone unaware, it's a type of computer that you can actually build and run, but not the architecture we actually use.

On the other hand, if we can't rule out arbitrarily large finite brains with certainty, then the requirements of rationality (whatever they are) should still apply when we condition on it being possible.

Maybe we should discount some very low probabilities (or probability differences) to 0 (and I'm very sympathetic to this), but that would also be vulnerable to money pump arguments and undermine expected utility theory, because it also violates the standard finitary versions of the Independence axiom and Sure-Thing Principle.

I would guess that arbitrarily large but finite (extended) brains are much less realistic than infinite universes, though. I'd put a probability <1% on arbitrarily large brains being possible, but probability >80% on the universe being infinite. So, maybe actual infinities can make do with more conservative assumptions than the particular money pump argument in my post (but not necessarily unboundedness in general).

From my Responses section:

The hypothetical situations where irrational decisions would be forced could be unrealistic or very improbable, and so seemingly irrational behaviour in them doesn’t matter, or matters less. The money pump I considered doesn’t seem very realistic, and it’s hard to imagine very realistic versions. Finding out the actual value (or a finite upper bound on it) of a prospect with infinite expected utility conditional on finite actual utility would realistically require an unbounded amount of time and space to even represent. Furthermore, for utility functions that scale relatively continuously with events over space and time, with unbounded time, many of the events contributing utility will have happened, and events that have already happened can’t be traded away. That being said, I expect this last issue to be addressable in principle by just subtracting from B - $100 the value in A already accumulated in the time it took to estimate the actual value of A, assuming this can be done without all of A’s value having already been accumulated.

(Maybe I'm understating how unrealistic this is.)