Against most, but not all, AI risk analogies

"The analogies establish almost nothing of importance about the behavior and workings of real AIs"

You seem to be saying that there is some alternative that establishes something about "real AIs," but then you admit these real AIs don't exist yet, and you're discussing "expectations of the future" by proxy. I'd like to push back, and say that I think you're not really proposing an alternative, or that to the extent you are, you're not actually defending that alternative clearly.

I agree that arguing by analogy to discuss current LLM behavior is less useful than having a working theory of interpretability and LLM cognition - though we don't have any such theory, as far as I can tell - but I have an even harder time understanding what you're proposing is a superior way of discussing a future situation that isn't amenable to that type of theoretical analysis, because we are trying to figure out where we do and do not share intuitions, and which models are or are not appropriate for describing the future technology. And I'm not seeing a gears level model proposed, and I'm not seeing concrete predictions.

Yes, arguing by analogy can certainly be slippery and confusing, and I think it would benefit from grounding in concrete predictions. And use of any specific base rates is deeply contentious, since reference classes are always debateable. But at least it's clear what the argument is, since it's an analogy. In opposition to that, arguing by direct appeal to your intuitions, where you claim your views are a "straightforward extrapolation of current trends" is being done without reference to your reasoning process. And that reasoning process, because it doesn't have a explicit gears level model, is based on informal human reasoning and therefore, as Lakens argues, deeply rooted in metaphor anyways, seems worse - it's reasoning by analogy with extra steps.

For example, what does "straightforward" convey, when you say "straightforward extrapolation"? Well, the intuition the words build on is that moving straight, as opposed to extrapolating exponentially or discontinuously, is better or simpler. Is that mode of prediction easier to justify than reasoning via analogies to other types of minds? I don't know, but it's not obvious, and dismissing one as analogy but seeing the other as "straightforward" seems confused.

Even if there are risks to using analogies with persuasion, we need analogies in order to persuade people. While a lot of people here are strong abstract thinkers, this is really rare. Most people need something more concrete to latch onto. Uniform disarmament here is a losing strategy; and not justified here as I don't think the analogies are as weak as you think. If you tell me what you consider to be the two weakest analogies above, I'm sure I'd be pretty to steelman at least one of them.

If we want to improve epistemics, a better strategy would probably be to always try to pair analogies (at least for longer texts/within reason). So identify an analogy to describe how you think about AI, identify an alternate plausible analogy for how you should think about it and then explain why your analogy is better/whereabouts you believe AI lies between the two.

Many proponents of AI risk seem happy to critique analogies when they don't support the desired conclusion, such as the anthropomorphic analogy.

Of course! Has there ever been a single person in the entire world who has embraced all analogies instead of useful and relevant analogies?

Maybe you're claiming that AI risk proponents reject analogies in general when someone is using an analogy that supports the opposite conclusion, but accepting the validity of analogies when it supports their conclusion. If this were the case, it would be bad, but I don't actually think this is what is happening. My guess would be that you've seen situations where someone has used an analogy to critique AI safety and then the AI safety person said something along the lines, "Analogies are often misleading" and you took this as a rejection of analogies in general as opposed to a reminder to check whether the analogy actually applies.

Interesting post! I think analogies are good for public communication but not for understanding things at a deep level. They're like a good way to quickly template something you haven't thought about at all with something you are familiar with. I think effective mass communication is quite important and we shouldn't let the perfect be the enemy of the good.

I wouldn't consider my Terminator comparison an analogy in the sense of the other items on this list. Most of the other items have the character of "why might AI go rogue?" and then they describe something other than AI that is hard to understand or goes rogue in some sense and assert that AI is like that. But Terminator is just literally about an AI going rogue. It's not so much an analogy as a literal portrayal of the concern. My point wasn't so much that you should proactively tell people that AI risk is like Terminator, but that people are just going to notice this on their own (because it's incredibly obvious), and contradicting them makes no sense.