Hi, I'm Luke Muehlhauser. AMA about Open Philanthropy's new report on consciousness and moral patienthood

We have!

Prior to beginning this investigation, we were on the fence about whether we should recommend grants related to fish welfare. As a result of initial findings from this report (long before publication), we decided to go ahead with investigating fish welfare grants:

A third key output from this investigation is that we decided (months ago) to begin investigating possible grants targeting fish welfare. This is largely due to my failure to find any compelling reason to “draw lines” in phylogeny (see previous section). As such, I could find little justification for suggesting that there is a knowably large difference between the probability of chicken consciousness and the probability of fish consciousness. Furthermore, humans harm and kill many, many more fishes than chickens, and some fish welfare interventions appear to be relatively cheap.

Since then, Lewis has recommended >$1 million in grants related to fish welfare: 1, 2, 3.

That said, keep in mind this caveat from the report:

Of course, this decision to investigate possible fish welfare grants could later be shown to have been unwise, even if the Open Philanthropy Project assumes my personal probabilities of consciousness in different taxa, and even if those probabilities don’t change. For example, I have yet to examine other potential criteria for moral patienthood besides consciousness, and I have not yet examined the question of moral weight (see above). The question of moral weight, especially, could eventually undermine the case for fish welfare grants, even if the case for chicken welfare grants remains robust. Nevertheless, and consistent with our strategies of hits-based giving and worldview diversification, we decided to seek opportunities to benefit fishes in case they should be considered moral patients with non-negligible weight.

Re: (1), I'll focus on the common fruit fly for concreteness. Before I began this investigation, I probably would've given fruit fly consciousness very low probability (perhaps <5%), and virtually all of that probability mass would've been coming from a perspective of "I really don't see how fruit flies could be conscious, but smart people who have studied the issue far more than I have seem to think it's plausible, so I guess I should also think it's at least a little plausible." Now, having studied consciousness a fair bit, I have more specific ideas about how it might turn out to be the case that fruit flies are conscious, even if I think they're relatively low probabilitiy, and of course I retain some degree of "and maybe my ideas about consciousness are wrong, and fruit flies are conscious via mechanisms that I don't currently find at all plausible." As reported in section 4.2, my current probability that fruit flies are conscious (as loosely defined in section 2.3.1 is 10%.

Re: (2). This question raises issues related to Pascal's Mugging. I don't pretend to have a solution, but some especially relevant discussions are Pascal's Muggle: Infinitesimal Priors and Strong Evidence, Why we can’t take expected value estimates literally (even when they’re unbiased), Modeling Extreme Model Uncertainty, and Sequence thinking vs. cluster thinking. As mentioned here, the Open Philanthropy Project remains eager to get more clarity on how we should think about making decisions under different kinds of uncertainty, and we expect to write more about this issue in the future.

In the meantime, the possibility of electron consciousness does not currently inform my actions.

Re: (3). You're right that Chalmers is "an interesting choice" for me to get feedback from. For example, Chalmers is a property dualist and panpsychist, which in some ways is about as different a position on consciousness from mine as you could find. On the other hand, Chalmers is (in my opinion) an unusually sharp thinker about consciousness, and also he has a deserved reputation for his ability to give critical feedback to others from the perspective of their own view. And, as hoped, Chalmers' feedback on an earlier draft of my report was very helpful, and I invested dozens of hours improving the draft in response to his feedback alone.

There were a few other consciousness researchers (whom I won't name) that I had hoped to get feedback from, but they weren't interested to give it. That's not surprising, since my report is so different from the type of work that consciousness researchers typically engage with.

My report makes so many claims (or at least, "guesses") that I have no doubt that if other consciousness experts gave extensive feedback on it, they would find plenty with which they disagree. In some cases, I know from their writing some specific things they would disagree with. But in many cases, I'm not sure where they would disagree, both because I haven't read all their works on consciousness, and because most consciosuness experts have only written about a tiny portion of the issues covered (at least briefly) in my report.

Re: (4). This question is beyond the scope of the intended purpose of this AMA, but I'll make a couple brief comments. It would take a lot of work for me to write a similar document that usefully complements Carl's, but I may do so one day. An old post of mine on this general topic is Scholarship: How to Do It Efficiently, but it's pretty narrow in scope.

Quick note: I agree with Benito that it's preferable to split different questions into separate comments, but in this case don't worry about doing so. I'll reply to all your questions as soon as I can, though I'm going to answer some other, quicker-to-answer questions first.

(Meta: It might be more helpful to submit individual questions as separate comments, so that people can up vote them separately and people's favourite questions (and associated answers) can rise to the top.)

I'm not sure what you mean by "objective definition" or "objectively correct answer," but I don't think I think of consciousness as being "objective" in your sense of the term.

The final question, for me, is "What should I care about?" I elaborate my "idealized" process for answering this question in section 6.1.2. Right now, my leading guess for what I'd conclude upon going through some approximation of that idealized process is that I'd care about beings with valenced conscious experience, albeit with different moral weights depending on a variety of other factors (early speculations in Appendix Z7).

But of course, I don't know quite what sense of "valenced conscious experience" I'd end up caring about upon undergoing my idealized process for making moral judgments, and the best I can do at this point is something like the definition by example (at least for the "consciousness" part) that I begin to elaborate in section 2.3.1.

Re: Type A physicalism, aka Type A materialism. As mentioned in section 2.3.2, I do think my current view is best thought of as "'type A materialism,' or perhaps toward the varieties of 'type Q' or 'type C' materialism that threaten to collapse into 'type A' materialism anyway…" (see the footnote after this phrase for explanations). One longer article that might help clarify how I think about "type A materialism" w.r.t. consciousness or other things is Mixed Reference: The Great Reductionist Project and its dependencies.

That said, I do think the "triviality" objection is a serious one (Ctrl+F the report for "triviality objection to functionalism"), and I haven't studied the issue enough to have a preferred answer for it, nor am I confident there will ever be a satisfying answer to it — at least, for the purposes of figuring out what I should care about. Brian wrote a helpful explainer on some of these issues: How to Interpret a Physical System as a Mind. I endorse many of the points he argues for there, though he and I end up with somewhat different intuitions about what we morally care about, as discussed in the notes from our conversation.

Among the PCIFs mentioned in section 3.2.2, I consider it to be one of the more interesting ones, principally because it may be indicative of certain kinds of "self-modeling," and the theoretical approaches to phenomenal consciousness I intuitively find most promising are those which involve certain kinds of self-modeling.

That said, I still give it very little weight, since it's very unclear what, exactly, different mirror-related behaviors imply about the self-modeling capacities of the animal, and how those relate to the self-modeling capacities that may be necessary for conscious experience. For example see the quotes and sources in footnote 109.

Unfortunately, the literature on consciousness is in even greater terminological disarray than many other fields, for understandable reasons: (1) to an unusual degree, we still don't know what we're talking about w.r.t. consciousness, and as a result, (2) consciousness studies is an extremely interdisciplinary "field," with contributions from scholars of nearly non-overlapping professional backgrounds. So, I'm not sure what a useful source on "useful terminology" would look like. Possibly a broad and shallow survey like my report or Weisberg (2014) is the best we can do at this stage.

I do suspect it’s possible for long-term meditation practitioners to have observations about the “internal” explananda of consciousness that non-practitioners (such as myself) can’t have. Susan Blackmore may be one such person. Of course, before taking any particular report seriously I’d want to check whether the reporting practitioner seems to be a reliable and clear reporter of their internal experience, whether their reports can be replicated by others in a certain way, etc.

As for determining the “true nature” of conscious experience: that’s more a matter of normal science, and especially of proposing cognitive algorithms that would instantiate the observed explananda of consciousness, and the meditation practitioners who are helpful for increasing our understanding of the explananda of consciousness might or might not happen to also be useful for doing the scientific work of coming up with strong explanations of those explananda.

These questions are beyond the intended scope of this AMA, but I’ll make a few quick remarks about the second question.

The calendar time spent is pretty uninformative, since large chunks of it passed by while I was working on other Open Phil projects that were (temporarily) more urgent, or waiting for feedback from someone, or for other reasons. As for total hours, my work logs suggest something like >800 hours on this report (by far my largest project thus far), though that includes lots of “getting up to speed” background reading on the various fields with which the report engages.

I'm not sure. My best guess would be something in the direction of the GDAK-inspired cognitive architecture discussed in section 6.2.4, but I don't yet have a clear picture of all the cognitive features that would need to be working together in a certain way for me to be even moderately confident the program is "conscious" in the sense defined in the report. I'm pretty sure none of the programs I described in the report are conscious (in the sense defined in the report).

For others' benefit, what I said in the report was:

I think I can make a weakly plausible case for (e.g.) Gazami crab consciousness, and I think I can make a weakly plausible case for chimpanzee non-consciousness.

By "weakly plausible" I meant that I think I can argue for a ~10% chance of Gazami crab consciousness, and separately for a ~10% chance of chimpanzee non-consciousness.

Such arguments would draw from considerations that are mentioned at least briefly somewhere in the report, but it would bundle them together in a certain way and elaborate certain points.

My argument for ~10% chance of chimpanzee non-consciousness would look something like an updated version of Macphail (1998), plus many of the considerations from Dennett (2017). Or, to elaborate that a bit: given the current state of evidence on animal cognition and behavior, and given what is achievable using relatively simple deep learning architectures (including deep reinforcement learning), it seems plausible (though far from guaranteed) that the vast majority of animal behaviors, including fairly sophisticated ones, are the product of fairly simple (unconscious) learning algorithms operating in environments with particular reward and punishment gradients, plus various biases in the learning algorithms "organized in advance of experience" via evolution. Furthermore, it seems plausible (though not likely, I would say) that phenomenal consciousness depends on a relatively sophisticated suite of reasoning and self-modeling capacities that humans possess and chimpanzees do not (and which may also explain why chimpanzees can't seem to learn human-like syntactically advanced language). I am pretty confident this conjunction of hypotheses isn't true, but I think something like this is "weakly plausible." There are other stories by which it could turn out that chimpanzees aren't conscious, but the story outlined above is (very loosely speaking) the "story" I find most plausible (among stories by which chimpanzees might not be conscious).

My case for a ~10% chance of Gazami crab consciousness would involve pulling together a variety of weak considerations in favor of the "weak plausibility" of Gazami crab consciousness. For example: (1) given the considerations from Appendix H, perhaps phenomenal consciousness can be realized by fairly simple cognitive algorithms, (2) even assuming fairly "sophisticated" cognition is required for consciousness (e.g. a certain kind of self-model), perhaps 100,000 neurons are sufficient for that, and (3) perhaps I'm confused about something fairly fundamental, and I should be deferring some probability mass to the apparently large number of consciousness scholars who are physicalist functionalists and yet think it's quite plausible that arthropods are conscious.

I'll use the terms "nociception" and "pain" as defined in Appendix D:

nociception is the encoding and processing of noxious stimuli, where a noxious stimulus is an actually or potentially body-damaging event (either external or internal, e.g. cutaneous or visceral)… Pain, in contrast to mere nociception, is an unpleasant conscious experience associated with actual or potential body damage (or akin to unpleasant experiences associated with noxious stimuli).

…Nociception can occur without pain, and pain can occur without nociception. Loeser & Treede (2008) provide examples: “after local anesthesia of the mandibular nerve for dental procedures, there is peripheral nociception without pain, whereas in a patient with thalamic pain [a kind of neuropathic pain resulting from stroke], there is pain without peripheral nociception.”

Often, it's assumed that if animals are conscious, then it's likely that they experience pain in conjunction with the types of nociceptive processing that, in humans, would be accompanied by conscious pain — or at least, this seems likely for the animals that are fairly similar to us in their neural architecture (e.g. mammals, or perhaps all vertebrates). And likewise, it seems that if we limit the nociception that occurs in their bodies, then this will also limit the conscious pain they experience if they are conscious at all.

Unfortunately, humans also experience pain without nociception, e.g. (as my report says) "neuropathic pain, and perhaps also… some cases of psychologically-created experiences of pain, e.g. when a subject is hallucinating or dreaming a painful experience." Assuming whichever animals are conscious are also capable of non-nociceptive pain, this means that conscious animals may be capable of suffering in ways that are difficult for us to detect.

Even still, I think we can study the mechanisms of nociception (and other triggers for conscious pain) independently of studying consciousness, and this could lead to interventions that (probably) address the usual causes of conscious pain even if we don't know whether a given animal is conscious or not.

However, I'm not sure this is quite what you meant to be asking; let me know if there was a somewhat different question you were hoping I'd answer.

I'm not sure whether the thing you're trying to say is compatible with what I'd say or not. The way I'd say it is this:

The 'weak illusionist' says that while many features of conscious experience can be 'explained away' as illusions, the 'something it's like'-ness of conscious experience is not (and perhaps cannot be) an illusion, and thus must be "explained" rather than "explained away." In contrast, the "strong illusionist" says that even the 'something it's like'-ness of conscious experience is an illusion.

But what might it be for the 'something it's like'-ness to be an illusion? Basically, that it seems to us that there is more to conscious experience than 'zero qualia', but in fact there are only zero qualia. E.g. it seems to us that there are 'diet qualia' that are more than 'zero qualia', but in fact 'diet qualia' have no distinctive features beyond 'zero qualia.'

Now in fact, I think there probably is more to 'zero qualia' than Frankish's "properties of experiences that dispose us to judge that experiences have introspectable qualitative properties that are intrinsic, ineffable, and subjective," but I don't think those additional properties will be difficult for the strong illusionst to adopt, and I don't think they'll vindicate a position according to which there is a distinctive 'diet qualia' option. Speaking of diet qualia vs. zero qualia is very rough: the true form of the answer will be classes of cognitive algorithms (on my view).

As mentioned in another comment, I'm perhaps most excited about the potential informativeness of (1) computational modeling of the sort I describe in section 6.2.4, (2) certain kinds of studies of human consciousness (Ctrl+F in the report for "perhaps the most promising path forward"), and (3) improvements to tools and techniques of human neuroscience that could help with (2) (Ctrl+F in the report for "we need fundamental breakthroughs").

But if I spent ~10 hours on each of the suggested research ideas from section 5, I'd probably have a better sense of each suggestion's cost and likely benefits, and my intuitions about prioritization would probably change.