Strawmen, steelmen, and mithrilmen: getting the principle of charity right

A lot of the work with mithrilmen is keeping an argument at a level of abstraction where it sounds sensible as a principle, but yet declining to interrogate it further, perhaps because venerated people hold that position.

From personal experience, I can tell you that really really smart people are wrong all the time. They're much more likely to be wrong when talking outside of their domain of expertise, but even a physics professor talking about physics will inevitably get stuff wrong in regular conversation. 

If someone says something that doesn't make sense, you should of course try and understand their argument and see if you're missing something. But "this person made a mistake" should always be a hypothesis under consideration, and it's often the most likely explanation. 

Michael - good points. 

It sounds like proper steelmanning is mostly applied to arguments, evidence, values, and reasons, whereas mithrilmanning is often applied more to specific influential individuals who tend to be associated with certain positions. (e.g. we might think 'Yann LeCun's machine learning research has been cited 600,000 times, so he must have some valid points when he expresses the view that we shouldn't worry about AI extinction risk - even though he sounds irrational and deranged on this topic.) The mithril armor is really being wrapped around some prestigious person who's making apparently weak arguments, more than around the apparently weak arguments themselves.

My suggestion for overcoming mithrilmanning is to find a less prestigious, but still reputable person, who makes the same arguments, and interrogate the validity of those arguments as if they're from a less influential source. (e.g. if some less-famous AI researchers makes basically the same arguments as LeCun, then dissect that less-famous person's arguments, rather than trying to face down LeCun, as if he's some Final Boss in a scary video game.) This is basically a social psychology hack to make us less intimidated by some famous person's reputation, so we can engage with the quality of their arguments, without getting misled by our  instincts to submit and defer to high-status individuals.

The problem with strawmanning and steelmanning isn't a matter of degree, and I don't think goldilocks can be found in that dimension at all. If you find yourself asking "how charitable should I be in my interpretation?" I think you've already made a mistake.

Instead, I'd like to propose a fourth category. Let's call it.. uhh.. the "blindman"! ^^

The blindman interpretation is to forget you're talking to a person, stop caring about whether they're correct, and just try your best to extract anything usefwl from what they're saying.^[1] If your inner monologue goes "I agree/disagree with that for reasons XYZ," that mindset is great for debating or if you're trying to teach, but it's a distraction if you're purely aiming to learn. If I say "1+1=3" right now, it has no effect wrt what you learn from the rest of this comment, so do your best to forget I said it.

For example, when I skimmed the post "agentic mess", I learned something I thought was exceptionally important, even though I didn't actually read enough to understand what they believe. It was the framing of the question that got me thinking in ways I hadn't before, so I gave them a strong upvote because that's my policy for posts that cause me to learn something I deem important--however that learning comes about.

Likewise, when I scrolled through a different post, I found a single sentence^[2] that made me realise something I thought was profound. I actually disagree with the main thesis of the post, but my policy is insensitive to such trivial matters, so I gave it a strong upvote. I don't really care what they think or what I agree with, what I care about is learning something.

1. ^{^}

   "What they believe is tangential to how the patterns behave in your own models, and all that matters is finding patterns that work."

   From a comment on reading to understand vs reading to defer/argue/teach.

2. ^{^}

   "The Waluigi Effect: After you train an LLM to satisfy a desirable property $P$, then it's easier to elicit the chatbot into satisfying the exact opposite of property $P$."

Thanks I really like this, and would appreciate some examples so I can get my head around this. It might be hard without being uncharitable, but I struggle to think of concrete examples at thte moment.

Thanks for writing this! Seems useful to have a term for excessive charitability. Being able to point at it succinctly might help mitigate information cascades.