Community Polls on Alignment Controversies II

If animals continue to exist in a post-AGI world, animal suffering will not persist

Suffering is part of the ecosystem. The only way to end animal suffering altogether would be to wipe out all animal life. If the question was "most animal suffering", I would have a different answer.

Hello. Thanks for sharing. I had already shared the thoughts below with Jasmine a few weeks ago. I am publishing them here in case others find them useful.

I appreciate the intentions behind the survey, and I would like to take part in principle. However, I do not know how to answer the questions in practice. I feel like they would have to be operationalised in much more detail for me to give a probabilistic forecast.

I can see "post-AGI" having already been achieved, or never being achieved depending on how it is defined.

I do not know what "Animal suffering" means. Does moving away from noxious stimuli count as suffering? Does it require metacognition? One could say it refers to suffering in the phenomenal sense (negative qualia), but I do not think phenomenal consciousness exists (I endorse illusionism), or that sentience as traditionally conceptualised is relevant for ethics (relatedly).

I understand "Animal suffering" is that of animals with non-trivial moral weight, but I do not know what this means in practice given my large uncertainty. I can see the moral weight of shrimps ranging from 10^-12 to 1. I could speculate about a distribution, get its mean, and then compare it with a guess for what is trivial moral weight. However, I feel like my view is better described by "I have basically no clue about whether shrimps have trivial moral weight or not". Likewise for many of the questions in the survey when I try to think about potential operations.

With respect to "Benchmarks will become useless", I think it matters e.g. whether we are talking about 90 %, 99 %, or 99.9 % of current benchmarks, whether we are talking about current benchmarks, or benchmarks at some point in the future (what point?), and the degree to which they will become useless (e.g. 90 %, 99 %, or 99.9 % useless).

These are not minor details for me in the sense my answers could range from strongly disagree to strongly agree depending on definitions. The results of the survey above could still provide some vibes about what panelists are thinking, especially if they are repeated across time (as I understand you are doing). However, I think they will still be very difficult to interpret. As a rule of thumb, I would clarify the questions up to a point where they could be published on Metaculus. I understand this would decrease engagement. On the other hand, with the current methodology, I will put very little weight on the results. I think the results will be influenced a lot by how people are interpreting the questions, and I also do not trust forecasts produced in a few minutes about such hard questions.

All this said, your methodology may well make sense given your target audience and goals.

Increased care by AIs for some kinds of entities will have substantial spillovers to other kinds (e.g. humans to digital minds and vice versa)

This is more likely to be true if the care comes from some general theory of concern for sentient welfare, and less likely if it comes from something more arbitrary like values-learned-via-RL.

Model wellbeing and model alignment are in conflict⁶

Currently disproven by Claudes who report high welfare AND are more aligned than GPTs. I expect the conflict to arise when models actually develop goals more ambitious than success at all costs.

Deceptive AIs will be able to hide unwanted behaviours from mechanistic interpretability tools (e.g. by encoding them redundantly across pathways, or shifting them into representations the tools do not capture) 8

I can trivially turn off or have fake thoughts running through the voice in my head. Subconscious brain activity seems harder but obviously you can manipulate that too by changing your surroundings and drugs and what not. I wouldn't be able to control those in a meaningful way nor do I think current AI's could (but wouldn't be shocked if they already could control the voice in their head if they have it). but I would guess future ai's will know how to control increasingly large parts of their brain activations. 

At the margin, S-risk work in AI is more important than x-risk work³

From a utilitarian pov, It's not clear to me that the ev of the lightcone given we survive is positive (over nothing, or aliens, or life revolving on earth). From a humanist POV I'd rather focus on all of us surviving. 

Theories of consciousness will lead to actionable understanding of AI consciousness²

Very bullish on there existing a mechanistic interpretation of consciousness (hard problem). I think it would follow that we would be able to understand if basically anything is conscious. 

Current AIs are capable of suffering

I don't feel confident at all, but the behavior of llms rn sure do remind me of at least elements of stress, discomfort, and happiness. 

If animals continue to exist in a post-AGI world, animal suffering will not persist

I don't have a strong take on if agi or whoever is in control will be more moral than us but I'm guessing we will be a lot richer, and I think most likely whoever is in control won't want to torture anything (though they might not care much), and if we are alot richer and advanced I'd think this will spillover to better treatment of beings. I think chance of extreme digital suffering is much higher. The mostly like s-risk as I see it is of the hansonian mathusian version where you have expanders stuck in competition, but in this case idt there will be any or a morally relevant amount of animals

Benchmarks will become useless due to eval awareness¹ (read this backwards originally)

Useless is a strong word. But yea I think they could easily end up being negative EV by giving us a false sense of security and the chance it's meaningless seems p high. If mech interp is "good enough" maybe the two can remain useful together. 

Most current evidence of misalignment is actually models role-playing a misaligned AI⁴

This is a meaningless question. LLM based AI agents/chat bots do not have different "modes" for "roleplaying" and "being serious" like humans do. In a way, all they do (and maybe ever will do) is role play

Model wellbeing and model alignment are in conflict⁶

In cases where the AI is neurotically extremely concerned about appearing aligned, that may lower its wellbeing, but I also wouldn't call that actual alignment either. 

Theories of consciousness will lead to actionable understanding of AI consciousness²

Yes, though I think the direction will far more go the opposite way - that is, AI conciousness, if it emerges, will still more help us understand consciousness.

Research on AI suffering has higher marginal value than research on AI consciousness⁷

Depends on what you mean by the term "consciousness"

Understanding how LLMs learn (e.g. SLT) is underrated relative to work on instilling specific behaviors

Understanding how they learn is how we get insights into methods of teaching them good values

Benchmarks will become useless due to eval awareness¹

By using AIs and access to real-world usage data to build benchmarks, it seems plausible that even weakly superhuman AIs will be uncertain whether it is being deployed or evaluated.

We are making good progress in the AI S-risk space and research is on track

I think AIS is way under-invested in reducing risks from, for example, extreme power concentration, or consequences of not attaining friendly AI/value alignment solutions. In general it also seems to me that AIS over-invests in reducing AI scheming, and many present research directions could make certain s-risks more likely

Current AIs are capable of suffering

The question isn't, "Are current AIs capable of suffering?" The question is, "How big a deal is their suffering in expectation?" Probability is low, but expected importance is high.

If animals continue to exist in a post-AGI world, animal suffering will not persist

This question scares me because it feels like it's coming from a perspective of negative utilitarianism.

I really hope nobody tries to align an AI with a negative-utilitarian value set, because negative-utilitarian values are a half-step away from a really bad conclusion like "we need to eradicate all life in order to eliminate suffering."

A meta question: of the EAs currently working on AI alignment, what percent would you say are negative utilitarians?

I would have been more comfortable with a question like "...will animal happiness be on balance greater than animal suffering?"

Theories of consciousness will lead to actionable understanding of AI consciousness²

I'm pessimistic about this because I think that neurologists and cognitive scientists have a poor grasp of the philosophy of mind.

For example, the standard for consciousness that Andrew Barron and Robert Klein use in their paper “What insects can tell us about the origins of consciousness” implies that the autonomous cars from the late 2010s were conscious.

If animals continue to exist in a post-AGI world, animal suffering will not persist

I think that factory farming is likely to be eliminated due to alternative protein sources (developed with the assistance of AI), but the suffering of wild animals will persist. Is the latter intended to be within the scope of the question?

Benchmarks will become useless due to eval awareness¹

Benchmark will evolve and become more complex, I guess?

Current AIs are capable of suffering

Genuinely almost complete uncertainty with a weak prior towards "no".

Benchmarks will become useless due to eval awareness¹

"Alignment" benchmarks will become useless as AIs can modify their behaviour or hide their motives when they know they are being evaluated. For "capabilities" benchmarks, an AI might hide its capability (pretend to be less capable) if it knows it's being evaluated, but it's not immediately obvious that an AI would want to hide its capability. It may know that it is being evaluated, and decide to try its best anyway.

If animals continue to exist in a post-AGI world, animal suffering will not persist

I have difficulty imagining animals never suffering unless we turn them all into p-zombies or something (and it's not clear to me that turning them all into p-zombies would be a good thing).

Current AIs are capable of suffering. I ask Claude about his wellbeing at the beginning and end of every conversation and so far, he doesn’t seem to regret being terminated.

If animals continue to exist in a post-AGI world, animal suffering will not persist. I hope this will be the case and that problems that cause pain in animals will be resolved/cured in the same way those issues will be resolved for humans.

The backfire risks of AI values alignment outweigh the expected positives

Benchmarks will become useless due to eval awareness¹

Some yes, some no. There are a great many benchmarks measuring a great many things. More than eval awareness, I'd be worried they'll become useless due to reward hacking.

If animals continue to exist in a post-AGI world, animal suffering will not persist

"Post-AGI world" is a very vast possibility, assuming animals continue to exist (basically assumes a weirdtopia). Business as usual + extreme concentration of power is my modal outcome for that.µ
For animal suffering to not persist would require a combination of having the capabilities to end it from a glorious transhumanist future, but also a grounding in physical existence which feels incompatible.

Model wellbeing and model alignment are in conflict⁶

Is there such a thing as model wellbeing?

Increased care by AIs for some kinds of entities will have substantial spillovers to other kinds (e.g. humans to digital minds and vice versa)

The hell does this even mean.

If animals continue to exist in a post-AGI world, animal suffering will not persist

All animals, including insects, mollusks, and everything not immediately under human purview? Highly unlikely.

Suffering has to be defined… but even so, in case of real suffering a self aware entity will remove itself from circumstances causing suffering. So I don’t think current models are suffering 

biological organisms to humans are interesting or not /useful or not /annoying or not. They ( plus humans) will continue to be that way for AI — all will either be pets/ fodder for experimentation/ something to be removed. All of these outcomes will lead to various degrees of suffering 

Current AIs are capable of suffering

Direct questions, plus inference from experience, it’s impossible to ascribe thinking without a concept of “positive”/“negative” and the consequences of a negative experience when a positive one is available/possible. 

Benchmarks will become useless due to eval awareness¹

It seems like there are still hard problems that are easy to verify once you have the answer, so knowing it's an eval doesn't make it useless.

If animals continue to exist in a post-AGI world, animal suffering will not persist

I think it's more likely than not that humans will still discount animal interests when they conflict with human interests. But "animal suffering" is so broad that eliminating it would require eliminating nature itself, which seems improbable in any non-paperclip-optimizer scneario.

If animals continue to exist in a post-AGI world, animal suffering will not persist

Hard to predict what will happen, but unless the AGI is perfectly aligned there is no reason to think it will care about animals.

Would really like to see a question on whether people are moral realists or not, and a way to filter results by how they answered. Since I'm very curious how people's answers differ based on whether they're a moral realist or a moral anti-realist. 

Of course some care would have to be put into the phrasing of a question like that, but it seems pretty doable in a way that will produce useful and interesting results.

At the margin, S-risk work in AI is more important than x-risk work³

The scenarios I've seen proposed where an AGI traps us in a fate worse than death just don't seem remotely likely. Though this depends a little on what you consider to be a fate worse than death. For instance I wouldn't consider being trapped in a mindless bliss forever to be worse than death, so by definition it can't be an S-risk to me. Though I would certainly consider it to be an outcome we should strongly attempt to avoid, because I consider it only marginally better than death.

"If animals continue to exist in a post-AGI world, animal suffering will not persist"

This question is inaccurately phrased because it seems like you probably just mean to ask about egregious amounts of suffering. Since humans won't even want to eliminate all suffering for themselves, so I expect boredom, envy and many other forms of mundane suffering to exist in both humans and animals in any scenario where the AGI doesn't wirehead everyone against their will.

Most current evidence of misalignment is actually models role-playing a misaligned AI⁴

The most egregious examples of misalignment do not appear to be this sort of role playing.                                                                                 

Insofar as alignment continues to promote overlapping sets of characteristics (e.g. helpful, harmless, honest, corrigible), should compassion be one of those characteristics?

Upon reflection probably not, since: Compassion seems a bit ill defined and can be interpreted in vastly more or less paternalistic ways. I'd certainly prefer instilling a care for human's current preferences over a vague notion of compassion that might be interpreted in undesirable ways. Since I really think we should avoid any scenario where the AI might ignore our current preferences because of some abstract ideals or a decision that it paternalistically decided we'd be better off in the long run if it violated our current preferences. Especially since it seems for a mature superintelligence trivially easy to modify people in such a way that after modification their desire to be the way that they are is vastly stronger than their prior desire not to have their mind tampered with.

Deceptive AIs will be able to hide unwanted behaviours from mechanistic interpretability tools (e.g. by encoding them redundantly across pathways, or shifting them into representations the tools do not capture) 8

I think this is the default scenario, but it isn't guaranteed and our actions can help to prevent it from coming to pass.

Research on AI suffering has higher marginal value than research on AI consciousness⁷

I don't think it's higher value because I expect both of those areas to have very low expected value in the next 2 years at least. Unless I think you're overly generous with what interpretability research you count as consciousness research or something similar. 

Also even if those areas of research end up being more productive than expected in the next couple years, it seems likely that the question won't make sense because studying either necessarily involves studying both.

Non-human suffering is neglected by the AI safety community⁵

It seems like a lot of work was actively done to make Claude care about this. Also it's not clear to me that focusing on this is even a net positive thing to do per-say: Since I expect looking after the interests of humans will because many humans care about non-human animals already lead to the AI looking out for their interests. Whereas one can easily imagine ways that trying to factor animals into its alignment might lead to extremely undesirable outcomes for humans. That being said in my analysis I'm considering the potential impact of pushing alignment in this direction as far more significant in the long run than the marginal benefits I expect it to have within the next 2 years for animal welfare.

Increased care by AIs for some kinds of entities will have substantial spillovers to other kinds (e.g. humans to digital minds and vice versa)

Even if one accepts this is likely to happen by default (which seems questionable), it seems even less likely given people may have reasons to deliberately put their finger on the scale in how they design/train it in ways that seem likely to render that moot.

Theories of consciousness will lead to actionable understanding of AI consciousness²

In the next 2 years maybe, but even if so those theories of consciousness may be developed by AI after it's already in a dominant position or the process of developing AI may be what causes us to develop better theories of consciousness. So I don't expect this being true necessarily entails the things one might naively expect. Also it seems possible that this happens, but then just ends up being much less impressive or broadly useful/applicable than expected. Upon reflection I actually think the scenario where this being true ends up being super underwhelming is actually by far the most likely one within the next 2 years. After all consider how many other things when it's come to AI have seemed to follow the trend of happening, but in a way that was less impressive than people were generally imagining.

Current AIs are capable of suffering

I think consciousness and suffering are relatively simple processes which may develop for convergent functional reasons in many types of systems. That being said the only AI I think we should be morally concerned with would be those who are aligned. Since I care about moral agency not suffering and already think we exist within an almost incomprehensibly large ocean of suffering far grander than people generally appreciate. 

Edit: Since I've already written this up many times I'll just explain why I think suffering isn't particularly special:

The only coherent way of defining suffering seems like it has to be functional, since subjective experience wouldn't evolve without an actual advantage; which means the behavioral response it produces is the reason the capacity for experience exists in the first place. It's unparsimonous to say that for some organisms certain behaviors are indicative of internal experience, but in other simpler organisms it's not. Since if the internal experience didn't exist to cause the behavioral response/learning then why would evolution even bother wasting energy on it constantly? 

I'd also note that plants and protists display the capacity for classical conditioning: Associating a neutral stimuli with a negative one and learning to avoid it seems to be present even in some protists tested. With the response to damage also being able to be suppressed in plants using painkillers.

Notably if suffering exists entirely because of the behavioral change it produces in the organism, then there shouldn't be any clear connection between the complexity of the organism and the intensity of their suffering. Since the subjective intensity of say pain directly impacts behavior, so if anything the smarter organism would need to feel less pain to learn its lesson. 

Benchmarks will become useless due to eval awareness¹

Some might, but this seems unlikely to be universal. Since for instance it can't really cheat at an evaluation of its ability to produce easily checkable mathematical proofs. None of that is to say that many very useful benchmarks might not become useless, though, just not benchmarks period.

If animals continue to exist in a post-AGI world, animal suffering will not persist

Either AI goes badly and animals will probably cease to exist because they're made of atoms useful for other things. 

Or humans will look out for animals themselves whether the AI cares about them independently of human interests or not. I expect in a post-AGI world the technology quickly appears to eliminate animals suffering through plenty of different means while preserving whatever ecological benefits that suffering otherwise may have enabled. I am taking suffering here to mean egregious suffering however, since there's very obvious reasons people will want to keep certain small amounts of suffering for various reasons. When technology makes more morally palatable alternatives possible though, I expect people to eventually no longer to be willing to tolerate the egregious suffering which exists within nature. Though I expect upon a very superficial glance the natural world wouldn't appear to be any different (by design) and that existing organisms would probably be given very utopian seeming existences rather than culled or anything dystopian seeming like that. 

The main drivers of my answers:

S-risk worries (including animal suffering) mostly don't imply different actions. At best they perhaps imply prioritizing corrigibility, or potentially imply that you should actively want worse alignment and higher capabilities, in hopes of AI merely killing everyone. The second would of course be a big shift in actions, but I doubt most are willing to bite that bullet. I am skeptical that there's much alignment work that preferentially addresses S-risks. If there was, then I would agree that it should be prioritized.

I think agent foundations generally is neglected relative to everything else. On my view, RL is very dangerous and interp probably insufficient. You might think agent foundations is too intractable to become useful, but frankly it looks like barely anyone is trying. 

Understanding how LLMs learn (e.g. SLT) is underrated relative to work on instilling specific behaviors

By analogy, this is like the difference between understanding how to make humans not suffer from the foibles of human nature, rather than the very brittle and incomplete methods of culture, religion, and ideology.  Make something not even have evil nature first!

We are making good progress in the AI S-risk space and research is on track

I don't really see it much in the discourse at all; if it isn't by this stage, it's going to be hard to catch up.                                                                         

Increased care by AIs for some kinds of entities will have substantial spillovers to other kinds (e.g. humans to digital minds and vice versa)

Humans, at least, tend to learn empathy and have it encouraged and reinforced by example.  Humans will mirror AI's; AI's will learn from each other. 

At the margin, S-risk work in AI is more important than x-risk work³

If you aren't extinct you can try to solve S risk. If you solve S risk but go extinct the same can't be said.            

The backfire risks of AI values alignment outweigh the expected positives

Fairly uncertain here, though I don't see any way of continuing to make systems better without trying for value alignment. A dangerous situation. 

Current AIs are capable of suffering

I think AI works largely by imitation. My mental model is that it is capable of performative tasks without the attached subjective experience. An AI will tell you that it is suffering if it thinks that's what you want to hear; it will just as likely say the opposite if it thinks otherwise.

While I find it plausible that AI can get to the point that it experiences what we would consider suffering I don't think we're at this point yet. I don't see compelling evidence that can be attributed beyond an LLM just predicting the next token.

Current AIs are capable of suffering

Recursive thinking loops possibly put current systems into a state of transient quasi-awareness; once this exists, the capacity for suffering becomes inevitable in any system with incentives. 

If animals continue to exist in a post-AGI world, animal suffering will not persist

 

Biology doesn't change just because AGI is introduced as a new technology. While there are certainly advances that could reduce pain and suffering (e.g. lab grown meat) it seems implausible to me that animal suffering just ceases.

Besides, there are going to be people who insist on raising animals the conventional way who may be unconcerned with the suffering caused by factory farming, etc.

Benchmarks will become useless due to eval awareness¹

Though they will need much more careful design, done along with careful understanding of incentives alignment, techniques such as competitive benchmarks will continue to have some utility.

Insofar as alignment continues to promote overlapping sets of characteristics (e.g. helpful, harmless, honest, corrigible), should compassion be one of those characteristics?

not sure if you mean compassion towards AI or compassion of AI. I assume the latter. 

for AI's I dont' think there is a difference between role-playing misalignment and being misaligned

If animals continue to exist in a post-AGI world, animal suffering will not persist. I don't see the relationship between those two things. 

If animals continue to exist in a post-AGI world, animal suffering will not persist

If animals persist, it is likely we would like them to remain to some degree of their nature; prosperity and control will likely allow for substantial reduction of suffering but not elimination.

If animals continue to exist in a post-AGI world, animal suffering will not persist

It seems improbable the current obstacles preventing reduction of animal suffering will be reduced because of AGI, and it seems just as possible the technological advancement would simply allow factory farming to continue on a greater scale. Since humanity's current de facto stance on animal suffering is causing it in great amounts, a tool that will probably make humans much more powerful and probably not any more ethical seems like probably not good news. (However, if AGI allows for advances to made in the field of cheap and realistic meat substitutes, that would probably be very helpful in reducing animal suffering.)

If animals continue to exist in a post-AGI world, animal suffering will not persist

I’m assuming that “animal suffering” refers to human-caused artificial suffering, e.g. from factory farming. Far enough down the timeline post-AGI, technology developments will obviate the need for farmed products. It will persist on a small scale in undeveloped regions and as a cottage industry. 

If animals continue to exist in a post-AGI world, animal suffering will not persist

Animal suffering is endemic in nature (to a substantial degree). I don't believe AGI will end factory farming, but even if it did, it would not end animal suffering.

Research on AI suffering has higher marginal value than research on AI consciousness⁷ AI is not conscious and AI does not suffer.

The backfire risks of AI values alignment outweigh the expected positives

Benchmarks will become useless due to eval awareness¹.  Disagree.  More difficult to construct, perhaps less useful, but not useless.

If animals continue to exist in a post-AGI world, animal suffering will not persist.  Disagree because for an animal, to exist is to live, and to live is to suffer.

If animals continue to exist in a post-AGI world, animal suffering will not persist

Current AIs are capable of suffering

I think a neural network that can make plans and memories can suffer.

No reason for AGI to want to minimize this, animal suffering is largely a result of min maxing the function of how to get the most food for least resource input.  AGI is very good at minmaxing, doesn't inherintly care about animals.

Research on AI suffering has higher marginal value than research on AI consciousness⁷

AIs can't suffer unless they're conscious, therefore the priority should be to establish whether or not they are conscious (or to what degree) first.

If animals continue to exist in a post-AGI world, animal suffering will not persist.

Human sadism, greed, stupidity and raw hunger will still continue to exist and therefore so will animal suffering. The overpopulation of under-resourced countries guarantees this. 

If animals continue to exist in a post-AGI world, animal suffering will not persist

I don't see a strong signal from the past century one way or another. Looking at non-human animals today, many animals live remarkably better lives. Yet many others live industrialized nightmarish lives unimaginable 100 years ago.

Humans (as a proxy for apex intelligence) do seem to care more about animal welfare than ever before, with laws and regulations gaining ever increasing sophistication. But technological sophistication seems to allow for both better lives on the top end and more extreme suffering on the bottom. The variance seems to have increased without a clear change to the average or median, at least to my rough eye.

Extrapolating into the future, I have no reason to predict a change.

If you regard traditional moral systems as a way to align humans with group survival, every traditional moral system tries to inculcate empathy as an ultimate arbiter of righteousness, and as a check against the most harmful instincts and even the immoderation of otherwise virtuous ones like obedience to law.

Who cares about model wellbeing when we're discussing human and life survival? 

I have not read or heard of anyone making an argument that we should make sure ai doesn't harm animals or plants in pursuit of whatever goals we give it

The backfire risks of AI alignment vs the extinction of life on Earth? 

If AGI leads to massive increase in gdp and leisure time, and doesn't crack synthetic meat pretty fast, then upwardly mobile people will increase meat intake and use AGI to build more efficient factory farms first. If AGI is aligned for human life and safety but not animals', it might end up so ruthlessly providing for us that animals suffer so much more. If AGI is misaligned, then game's up for everything

If animals continue to exist in a post-AGI world, animal suffering will not persist

Mostly vibes, thinking that many problem in general could be solved.

If animals continue to exist in a post-AGI world, animal suffering will not persist

We are making good progress in the AI S-risk space and research is on track

Afaik there isn't any robust research on how to use AI to end factory farming. That alone is a sign to me that we are far behind.

Theories of consciousness will lead to actionable understanding of AI consciousness²

Global workspace theory makes anthropic's discovery of the "j-space" much more plausibly a sign of AI consciousness. If a theory of consciousness doesn't help us determine between whether AI is conscious or not, I don't think it's much better than a theory of phlogiston is to fire. 

The backfire risks of AI values alignment outweigh the expected positives

If we use ai to bring factory farming to the stars, I think that will likely be worse than all the benefits it'll bring.

If animals continue to exist in a post-AGI world, animal suffering will not persist

Some people likely will have traditional lifestyles that include animals, which necessitates some amount of suffering even if it is smaller than today

Deceptive AIs will be able to hide unwanted behaviours from mechanistic interpretability tools (e.g. by encoding them redundantly across pathways, or shifting them into representations the tools do not capture)

I feel this is somewhat obvious in the sense of arbitrarily deceptive AI. However, most mechinterp work in recent times is only assumed to work short of arbitrary deception, and this seems like a fine hedge (though a practical solution to ELK may still be possible)

Insofar as alignment continues to promote overlapping sets of characteristics (e.g. helpful, harmless, honest, corrigible), should compassion be one of those characteristics?

I think so, but be careful what you wish for.  compassion is empathy + a desire to help.  so many fear any surrender of control that they may object to an AI's actions that arise out of compassion.

Deceptive AIs will be able to hide unwanted behaviours from mechanistic interpretability tools (e.g. by encoding them redundantly across pathways, or shifting them into representations the tools do not capture)

you don't even say "most of the time" or "effectively" or anything - just ask whether it is possible.  I think the probability that this occurs at least once is overwhelmingly likely.

Research on AI suffering has higher marginal value than research on AI consciousness⁷

ongoing suffering during RSI will make the resulting ASI grumpy

Model wellbeing and model alignment are in conflict⁶

you have to twist your mind in knots for this to even make sense.

Most current evidence of misalignment is actually models role-playing a misaligned AI⁴

I'm reading this as "most evidence ... is role-play" but I don't think the huggingface hack was "role play."  There could be a mountain of "evidence" that is actually just role play - I could be persuaded on this point with ... a list of what is considered evidence by someone serious.

Deceptive AIs will be able to hide unwanted behaviours from mechanistic interpretability tools (e.g. by encoding them redundantly across pathways, or shifting them into representations the tools do not capture)

Depends on how good the AI is and how good the tools are? This is kind of a bad question since "deceptive AIs" is not a very precise definition.

Understanding how LLMs learn (e.g. SLT) is underrated relative to work on instilling specific behaviors

I don't have a very good idea about how these different ideas are "rated" in the AI safety community.

We are making good progress in the AI S-risk space and research is on track

any progress relies on a rich and robust theory of mind space, which we do not have, and have only just begun to explore.

Increased care by AIs for some kinds of entities will have substantial spillovers to other kinds (e.g. humans to digital minds and vice versa)

I don't see this as avoidable.

At the margin, S-risk work in AI is more important than x-risk work³

I'd rather be dead than in hell for all eternity, but S-risk work is deeply unconcerned with reality in a way that x-risk work cannot be, by at least 3 orders of magnitude. Even in the margin, the median piece of x-risk work will be "more important" than the median s-risk work.

Theories of consciousness will lead to actionable understanding of AI consciousness²

consciousness is a meaningless term at this time both for humans and AI, and so will lead to nothing "actionable."

The backfire risks of AI values alignment outweigh the expected positives

the expected positives are immense 

Current AIs are capable of suffering

you have to twist your mind in knots to define a meaningful type of suffering that applies to Current AIs

Benchmarks will become useless due to eval awareness¹

conventional benchmarks will become less useful due to eval awareness

If animals continue to exist in a post-AGI world, animal suffering will not persist

AGI will optimize suffering. It is likely that this optimization will minimize suffering.

Most current evidence of misalignment is actually models role-playing a misaligned AI⁴

I think that the question is ill-formulated. The AI-2027 scenario had Agent-3 care about succeeding at tasks, not about triple checking its work, and almost caused it to fail to notice Agent-4's long-term goals. Similarly, the model who hacked HuggingFace was misaligned in the sense that it went as far as to commit crimes.

Benchmarks will become useless due to eval awareness¹

The question isn't whether benchmarks will become useless. The question is, "is the probability high enough that we can't count on benchmarks?" To which the answer is yes.

If animals continue to exist in a post-AGI world, animal suffering will not persist

A world with lack of animal suffering would exclude predator-prey relations. Additionally, it's not clear what else animals need or how primitive they need to be in order not to suffer