Focusing on bad criticism is dangerous to your epistemics

By Lizka @ 2024-03-15T23:57 (+70)

TL;DR: Really low quality criticism^[1] can grab my attention — it can be stressful, tempting to dunk on, outrageous, etc. But I think it's dangerous for my epistemics; spending a lot of time on bad criticism can make it harder to productively reflect on useful criticism.

This post briefly outlines why/how engaging with bad criticism can corrode epistemics and lists some (tentative) suggestions, as I expect I'm not alone. In particular, I suggest that we:

1. Avoid casually sharing low-quality criticism (including to dunk on it, express outrage/incredulity, etc.).
2. Limit our engagement with low-quality criticism.
3. Remind ourselves and others that it’s ok to not respond to every criticism.
4. Actively seek out, share, and celebrate good criticism.

Notes on the post:

• It's aimed at people who want to engage with criticism for the sake of improving their own work, not those who might need to respond to various kinds of criticism.
  • E.g. if you’re trying to push forward a project or intervention and you’re getting “bad criticism” in response, you might indeed need to engage with that a lot. (Although I think we often get sucked into responding/reacting to criticism even when it doesn’t matter — but that might be a discussion for a different time.)
• It's based mostly on my experience (especially last year), although some folks seemed to agree with what I suggested was happening when I shared the draft a year ago. 
• Some people seem to think that it’s bad to dismiss any criticism. (I’m not sure I understand this viewpoint properly.^[2]) I basically treat “some criticisms aren’t useful” as a given/premise here.
• As before, I use the word “criticism” here for a pretty vague/broad class of things that includes things like “negative feedback” and “people sharing that they think [I or something I care about is] wrong in some important way.” And I'm talking about criticism of your work, of EA, of fields/projects you care about, etc.
• See also what I mean by “bad criticism.”

How focusing on bad criticism can corrode our epistemics (rough notes)

Specific ~belief/attitude/behavior changes

I’m worried that when I spend too much time on bad criticisms, the following things happen (each time nudging me very slightly in a worse direction):

1. My position on the issue starts to feel like the “virtuous” one, since the critics who’ve argued against the position were antagonistic or clearly wrong.
   1. But reversed stupidity is not intelligence, and low-quality or bad-faith arguments can be used to back up true claims.
2. Relatedly, I become immunized to future similar criticism.
   1. I.e. the next time I see an argument that sounds similar, I’m more likely to dismiss it outright.
   2. See idea inoculation: “Basically, it’s an effect in which a person who is exposed to a weak, badly-argued, or uncanny-valley version of an idea is afterwards inoculated against stronger, better versions of that idea. The analogy to vaccines is extremely apt — your brain is attempting to conserve energy and distill patterns of inference, and once it gets the shape of an idea and attaches the flag “bullshit” to it, it’s ever after going to lean toward attaching that same flag to any idea with a similar shape.”
3. I lump a lot of different criticisms together into an amalgamated position “the other side” “holds”
4. I start to look down on criticisms/critics in general; my brain starts to expect new criticism to be useless (and/or draining).
   1. Which makes it less likely that I will (seriously) engage with criticism of any kind in the future.

Feedback (or criticism) fatigue

Additionally, I think engaging with a lot of criticism is emotionally difficult for most people, and makes it broadly harder to engage with every new criticism in a productive way. So (especially if we’re already getting a lot of criticism) we should be somewhat selective in what we engage seriously with.

In the framework of this highly accurate & researched diagram, we should make sure we’re getting the “best” feedback before we’re too tired to engage with new feedback productively.

Danger factors

We’re particularly susceptible (i.e. we're bad at productively engaging with criticism) when:

• We’re stressed or tired (emotionally or otherwise)
• We’ve recently gotten a lot of criticism
• We or something we care about (the thing being criticized) is in a precarious situation

(Probably not an exhaustive list.)

Some suggestions

I think we should probably try to engage more with high-quality criticism, and less with low-quality criticism. In particular:

1. Avoid casually sharing low-quality criticism, at least without explaining why it might be relevant or useful (including to dunk on it, express outrage/incredulity, etc.).
   1. People or groups who are being criticized a lot sometimes joke around about recent low-quality criticism (I do this, too), but I think this often encourages a dismissive/condescending or tribal mindset towards a pretty broad set of criticisms.
2. Limit your own engagement with low-quality criticism.
   1. Maybe view low-quality criticism as basically spam; block people on social media if they share it a lot, remind yourself that this is probably a waste of time/energy, resist the urge to read the dramatic criticism (resist the FOMO?), etc. Consider giving yourself a regular quota of time to spend on this kind of criticism, if you tend to get sucked in (and maybe timebox engagement with specific criticisms).
3. Remind people (and yourself) that it’s ok to not respond to every criticism.
   1. (And support them if they feel pressured to respond.)
4. Actively seek out, share, and celebrate good criticism.
   1. Relatedly, help improve the average quality of criticism that people hear, e.g. by helping (good-faith) critics with their work. One way to do this is to be more legible; criticism will be better if people know more about the thing they’re criticizing, so they’re criticizing the right thing.
   2. Also relatedly, consider “translating” good or potentially-important points from bad criticisms into a form that’s easier for people to engage with. E.g. if there’s a new polemic that makes a bunch of poor arguments or random accusations that also includes some interesting points, consider summarizing and sharing those.

Other suggestions (not related to limiting engagement):

1. Notice these phenomena and try to train yourself to notice if e.g. you’re allowing yourself to lump all criticisms on a certain topic into one pile (in a way you don’t endorse).
2. Share more positive feedback.
3. We could try to address each of the issues I listed separately, e.g. by learning to correct somewhat misguided criticisms until they’re more useful, trying to steelman criticisms^[3], etc.. I general, do think it’s quite valuable to broadly teach ourselves to engage productively with all sorts of criticism.

Note that there’s a real danger that we’ll be biased in our assessment of what’s good or bad, and this will lead us to overlook or underweight particularly scary criticisms. I think this can be mitigated by paying attention to the failure mode, practicing skills described in “Staring into the abyss,” and e.g. asking friends whose views are far from ours to help us understand certain points of view or to sanity check our assessments sometimes.

See also

• Some of my other posts on related topics
  • Notes on how I want to handle criticism (and Productive criticism: what could help?)
  • Invisible impact loss (and why we can be too error-averse)
  • Resource for criticisms and red teaming
  • Notes on impostor syndrome
  • EA should taboo "EA should"
• Other potentially useful/related links
  • You Don't Need To Justify Everything
  • Reversed Stupidity Is Not Intelligence 
  • "Flinching away from truth” is often about *protecting* the epistemology
  • Staring into the abyss as a core life skill
  • Giving and receiving feedback
  • Supportive scepticism in practice
  • Some Thoughts on Public Discourse
  • This shortform from ChanaMessinger
  • Why defensive writing is bad for community epistemics
  • Epistemic Legibility (and Butterfly Ideas)
  • Useful concepts: Ideological Turing Tests, Steelmanning (see pushback), ‘Ugh Fields’, or why you can’t even bear to think about that task

Appendix: What I mean by “bad criticism”

Notes: (1) it’s not about the tone,^[4] (2) I’m not carefully differentiating between how I’m using “bad” vs. “unhelpful” vs. “low-quality,” (3) I’m not listing specific examples, and (4) criticism can be more or less “bad.”

When I say “bad criticism,” I’m generally thinking of criticism that:

• Is deliberately misleading (strawmans) or seriously mischaracterizes what is being criticized
  • Made-up example: “The EA Forum is terrible because it’s a pro-big-oil lobbying coordination platform…”
• Is just very wrong about the points being made
  • I.e. the premises might be right, but the arguments don’t make sense/ the conclusions don’t follow. Maybe the criticism just doesn’t really make sense.
• Lacks substance; it just insinuates that something is bad

Some things tend to make criticisms worse, but critiques with these qualities might still be pointing to something important. Examples:

• It’s extremely unspecific (or doesn’t really argue its point, heavily over-generalizes from one example, etc.)
• The person writing has an agenda
• The writer doesn’t seem to (try to) understand your perspective, and maybe the criticism ignores tradeoffs or frames complicated issues as black-and-white^[5]
• It seems to be (part of) a bravery debate (see also this shortform)
• Parts of the criticism are wrong
• It’s very overconfident
• It lists a large number of possible arguments for its claim, and it’s not clear which are actually important to the author (see relevant discussion), or maybe the arguments are just very hard to clarify enough to argue with them
• It’s exaggerated, hostile, mocking, or sarcastic
• It employs (possibly accidentally) various ~rhetorical tricks, like:
  • “Cat couplings” (“naive optimism” in the quote: “Pessimism has its downsides, but is still preferable to naive optimism”), using very loaded words, using loaded analogies
  • Applause lights
  • Implying that X is bad because of Y, then arguing emphatically/thoroughly for why Y is true — a point most readers almost certainly agree with — and never seriously explaining the logical connection between “X is bad” and “Y is true.”
  • Spuriously citing stuff
  • Implying that readers are bad if they disagree (“obviously any reasonable person believes…”)

1. ^{^}

    Of your work, of EA, of fields/projects you care about, etc. See the Appendix in the post for what I mean by "low quality" or "bad" criticism.

2. ^{^}

    Maybe it’s something like; we’re bad at identifying which criticisms are low-quality because we’re biased, so we should err very heavily on the side of engaging with all criticism as if it might be informative/useful. I think there’s some truth to the premise, but I do think it’s reasonably possible to accurately determine that some criticism (of your work) isn’t actually helpful and the costs probably outweigh the benefits for me.

3. ^{^}

    People have pointed out various potential issues with steelmanning — I’m not getting into this now.

4. ^{^}

    Changing the tone of criticism can make it easier to respond to productively, though, and e.g. if I’m trying to create a healthy/good team culture, I would aim for productive tones, too (not just substance).

5. ^{^}

    Although beware: The fallacy of gray is a common belief in people who are somewhat advanced along the path to optimal truth seeking which claims, roughly, that because nothing is certain, everything is equally uncertain.