Chesterton Fences and EA’s X-risks

By jehan @ 2022-09-02T11:56 (+78)

TL;DR: EA lacks the protective norms nearly universal in mature institutions, leaving it vulnerable to the two leading causes of organizational sudden death: corruption and sex scandals.
 

A foundational belief in EA is the importance of unintended negative consequences; a famous actionable heuristic around this is Chesterton’s Fence:

Let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.

It’s not to say that you can never take down fences, only that you recognize what you’re losing so that you can weigh it against what you stand to gain.  However, I believe EA has carelessly dismantled valuable fences while gaining comparatively little. At the very least there are low-hanging fruit fences that could have outsized returns with few costs.

1) Dismantled Fence: Undervaluing Outside Experience

This has been discussed extensively, so I won't comment further except to note that outside experience is a rite of passage in a wide range of communities.[1]

It may be that this general lack of outside experience has prevented EA from adopting hard-won norms found elsewhere:

2) Dismantled Fence: Excessive Fraternization

Meme in EA: All your friends, roommates and partners are EAs.

This is not a good thing. Many have commented, including Will MacAskill, that this is personally suboptimal and have tried to correct it. The EA-consistent reason for this seems to either be work-life balance or viewpoint diversity.

These reasons are valid, but not the reason nearly every successful organization has rules around this: because it's an existential risk for The Mission.

Not in some second-order way either, excessively close and incestuous relationships cause dysfunction directly and in particular leave it open to perhaps the modern era's leading cause of organizational sudden death: sex scandals.

Successful institutions have rules around fraternization and power relationships, EA seems to largely lack explicit norms here-if anything the norm seems to actually go the other way from the mainstream.

I understand that relationships are what makes a community, but bad relationships will make a bad community while also endangering the mission.[2]

3) Dismantled Fence: Lack of Explicit Anti-Corruption Rules

You may read “corruption” and think “graft”, but that's typically the last rung on the corruption ladder. It's not so overt or conscious at first, so more like self-dealing, conflict of interest, and nepotism.

Few explicit rules address the problems above, and there is at least the perception that within EA they exist (but this perception could be mitigated via visible norms).[3]


Mature institutions have rules preventing these sorts of failure modes. EA however is now combining fraternization with few rules around self-dealing and favoritism to create a massive opportunity for organizational death with no comparable upside.
 

Fences as an Immune system

These fences not only prevent existing members from inadvertently sliding into scandal, but also act as an immune system for bad actors.

EA now has an enormous amount of power, money, and influence. When it was small and unappealing it may have gotten away with security through obscurity, but it's now far too big of a prize to exist so immunocompromised.




(Cross-posted here: www.atvbt.com/eacrit)

 

  1. ^
  2. ^

    "But our people/intrinsic beliefs/other norms prevent this from being an issue!" is what pre-scandal institutions always say.

  3. ^

    Relatedly, I can say from having lived in China for many years that the perception of charities as corrupt is a major reason for comparatively low charitable giving. The potential collateral damage of an EA sex or corruption scandal should not be underestimated. 


Guy Raveh @ 2022-09-02T19:58 (+5)

Interesting post! I've just been talking to some friends about how there's not enough protection against sexual harassment taking place in EA spaces (i.e. Julia Wise can't be physically present in every conference and local group). Not to mention sex scandals.

But on one specific point - could you give an example of successful institutions and their rules about fraternisation?

Most of the Wikipedia article talks about the military, and there's some talk about education, while I think of EA more like a workplace, and for those the Wiki article mostly talks about legality of restrictions.

In my country it's extremely uncommon to have restrictions like these in a workplace. It's also a cultural thing - I've heard (correct me if I'm wrong) that in the US there are very strong social norms against talking to (or having lunch with) your manager's manager, while here in Israel there are none, and I don't see any damage from that.

But maybe I'm just misunderstanding, which is why I'm asking for examples.

jehan @ 2022-09-02T22:49 (+9)

Great question! When I write fraternization I mean any personal relationship, romantic or platonic. I realize the Wiki article is misleading in this regard, here's an example where the term is used to mean romantic relationships. Romantic interactions generally pose the most risk so policies around this are common. This is the lowest-hanging fruit for EA as far as I'm concerned.

You probably already know this though and are referring to platonic interactions?

I agree that having lunch here and there with someone is fine. The issue is when  close friends can influence EA activities (like access to events, grants, jobs etc.). Here Apple defines close friendships as a "significant personal relationship" and therefore "do not conduct Apple business" with them.
 

With appropriate conflict of interest norms, I don't think EA will need to do anything close to trying to regulate normal friendships. I'm mostly referring to the often intense social scene that emerges around EA communities and events. An example cited in this post "Open EA Global":

Waking up hungover, again, trying to make sense of the fact the most serious partying in my life now happens at conferences of people who talk obsessively about doing the most good.

This seems unprecedented for a mission-driven organization. The American Psychiatric Association probably doesn't have a policy around raging parties because there is a 0% chance that raging parties would be a fixture of APA conferences.

What I would like to see is for EA to professionalize a little bit and implement some movement-protecting norms around 1) romantic relationships and 2) certain social gatherings, which would be most of the relevant risk mitigation.

Sorry to be the fun-police, I just think the riskiest sorts of fun should happen a bit further from EA.

Guy Raveh @ 2022-09-03T00:08 (+3)

Thanks for the detailed reply! All those suggestions seem reasonable to me, after your clarifications.

Yonatan Cale @ 2022-09-04T21:26 (+2)

Maybe a start would be to state explicitly "when we put in these rules, we will also look back on what happened until then"

Anyway, +1