Reflections on Vox's "How effective altruism let SBF happen"

(EDIT: Added a follow-up comment after reading the original article)

A lot of this discourse feels like it's missing the point to me. FTX was not an EA org. (Alameda was founded mostly with EAs, but then most of them left, in part because of bad governance and lack of ethics!). FTX was not beholden to EAs, and EA and EA orgs didn't have any say in how FTX was governed. EA may have been well placed to blow the whistle on this, and maybe to say things to Sam about this, but it seems very off to say that the governance of EA orgs led to the bad governance of FTX.

(Also, Alameda was founded in ~2018, where the EA scene was very, very different and much less mature (and probably much worse governed). I expect many bad governance decisions are baked in when an org is founded)

I think the correct reference class for the governance of FTX is more like startups, esp crypto startups. I don't see that much of a causal link between, eg, how well OpenPhil or MIRI is governed, and how FTX was governed.

To be clear, I am not arguing that EA orgs are not badly governed, I just think these should be two separate conversations. And, even, that the fact that FTX was badly governed and this led to a disaster, is at best weak evidence that EA orgs governed in a similar way will also lead to a disaster, given how different the work is (it's harder to fuck up when you aren't managing >$10bn of customer funds!) (Though, to be clear, I think that if I discovered an EA org being governed in the same way as FTX it would be a concerning red flag. Just that it should have been a red flag regardless of the FTX blow up!)

I think that paragraph is quite misguided. "Becoming much more risk averse" is a great way to stop doing anything at all because it's passed through eight layers of garbage. On top of this, it's not like "literally becoming the US federal government" and "not having any accounting or governance at all" are your only two options; this creates a sad false dichotomy. SBF was actively and flagrantly ignoring governance, regulation, and accounting. This is not remotely common for EA orgs.

Like, for the last couple of decades we've been witnesssing over and over again how established, risk-averse institutions fail because they're unable to compete with new, scrappy, risk-tolerant ones (that is, startups).

"Good governance" and bureaucracy are, while correlated, emphatically not the same thing. EA turning into a movement that fundamentally values these over just doing good in the world as effectively as possible will be a colossal failure, because bureaucracy is a slippery slope and the Thing That Happens when you emulate the practices that have been used for centuries is that you end up not being able to do anything. I'd be very sad if this was our final legacy.

Follow-up to my previous comment: I was responding to the post written here, not the underlying article in the above comment. I've now read the article, and broadly agree with it (esp on the significant over-focus on philosophy and controversial ideas within EA, and how this is harmful), but think that the claims about governance are badly argued and don't hold up, even if the conclusions may be correct

The specific part of the article on the importance of good, professional governance in foundations, is something I feel more confused about, though Matthews' was not arguing that EA being better governed would have prevented FTX collapsing, more that it would have reduced general harm to EA + limited SBF's power.

Like, sure, I think SBF's various foundations being further from him would have been a good idea a priori (and even better in hindsight!). But I don't actually see how the FTX disaster is evidence for this, beyond being evidence of SBF's poor judgement and bad intentions. Like, the issue at hand is not that SBF funded a bunch of pandemic prevention etc! I don't think it's even that he was involved in the foundation's decisions ("foundation staff would sometimes cc SBF on a pitching email"). I think it's that funding was unstable and got pulled out unexpectedly/there's risks of clawbacks, it harmed the reputation of those that received it, that the funding was gained immorally (though evaluating this is confusing to me, since some of their money was gained legitimately, some was from stealing from customers), that it's harmed the reputation of the movement + causes he donated to, and that the giving plausibly allowed him to launder his reputation in a way that gained him more credibility and less suspicion to perpetuate the fraud. None of these are that related to how involved he was in the foundations!

And, finally, I'd argue that the Future Fund was actually much more de-centralised and democratic than eg OpenPhil, given their enormous regranter program. It wasn't quite just giving people money and letting them do whatever they wanted with it, but I'd say that it made giving much more diverse and accounting for many more different perspectives than any other foundation I'm aware of. (I think this comes with many downsides, to be clear, though I'm net a fan of the regranter program. Just that criticising it as SBF being too involved seems silly)

Sharing my reflections on the piece here (not directly addressing this particular post but my own reflections I shared with a friend.)

While I agree with lots of points the author makes and think he raises valuable critiques of EA, I don’t find his arguments related to SBF to be especially compelling.  My run-through of the perceived problems within EA that the author describes and my reactions:

1. The dominance of philosophy. I personally find parts of long-termism kooky and I'm not strongly compelled by many of its claims, but the Vox author doesn’t explain how this relates to SBF (or his misdeeds)... it feels more like shoehorning a critique of EA in to a piece on SBF? 
2. Porous boundaries between billionaires and their giving. So yes it sounds like SBF was very directly involved in the philanthropy his funds went toward but I don’t think that caused (much? any?) incremental reputational harm to EA vs. a world where he created the “SBF family foundation” and had other people running the organization. 

• If I wanted to rescue this argument, maybe I could say SBF’s behavior here is representative of a common trait of his (at FTX and in his charity) – SBF doesn’t even have the dignity to surround himself with yes-men; he insists on doing it all himself! And maybe that’s a red-flag RE cult of personality/genius and/or fraud that EA should have caught on to. 
• I will say, though, that the FTX Future Fund had a board/team that was fairly star-studded and ran a big re-granting program (i.e., let others make grants with their money). Which is to say I’m not sure how directly involved SBF actually was in the giving. [As an aside, I think it’s fine for billionaires to direct their own giving and am a lot more suspect of non-profit bloat and organizational incentives than the Vox author is.] 

1. 3. Utilitarianism free of guardrails. I agree a lack of guardrails is a problem, but: 

• a) On utilitarianism’s own account it seems to me you should recognize that if you commit massive fraud you’ll probably get caught and it will all be worthless (+ cause serious reputational harm to utilitarianism), so then committing the fraud is doing utilitarianism wrong. [I don’t think I’m no-true-Scotsman-ing here?] 
• b) More importantly… the author doesn't explain how unabashed utilitarianism led to SBF's actions - it's sort of vaguely hand-waving and trying to make a point by association vs. actual causal reasoning / proof, in the same vein as the dominance of philosophy point above? I guess the steelman is: SBF wanted to do the most good at any cost, and genuinely thought the best way to do so was to commit fraud (?) A bit tough for me to swallow. 

1. 4. Utilitarianism full of hubris. A rare reference to evidence (well, an unconfirmed account, but at least it’s something!) Comparing the St. Petersburg paradox to SBF figuring let’s double-or-nothing our way out of letting Alameda default is an interesting point to make, but SBF's take on this was so wild as to surprise other EA-ers. So it strikes me as a point in favor of “SBF has absurd viewpoints and his actions reflect that” vs. “EA enabled SBF.” Meanwhile the author moves directly from this anecdote to “This is not, I should say, the first time a consequentialist movement has made this kind of error” (emphasis added).  SBF != the movement and I think the consensus EA view is the opposite of SBF’s, so this feels misleading at best.

One EA critique in the piece that resonated with me - and I'm not sure I'd seen put so succinctly elsewhere is: 

“The philosophy-based contrarian culture means participants are incentivized to produce ‘fucking insane and bad’ ideas, which in turn become what many commentators latch to when trying to grasp what’s distinctive about EA." 

While not about SBF, it's a point I don't see us talking about often enough with regard to EA perceptions / reputation and I appreciated the author making it. 

TL;DR: I thought it was an interesting and thought-provoking piece with some good critiques of EA, but the author (or - perhaps more likely - editor who wrote the title / sub-headers) bit off more than they could chew in actually connecting  EA to SBF's actions.

The key passages (my emphasis):

Longtermism seems weird not because of its critics but because of its proponents: it’s expressed mainly by philosophers, and there are strong incentives in academic philosophy to carry out thought experiments to increasingly bizarre (and thus more interesting) conclusions.

This means that longtermism as a concept has been defined not by run-of-the-mill stuff like donating to nuclear nonproliferation groups, but by the philosophical writings of figures like Nick Bostrom, MacAskill, Greaves, and Nick Beckstead, figures who have risen to prominence in part because of their willingness to expound on extreme ideas.

These are all smart people, but they are philosophers, which means their entire job is to test out theories and frameworks for understanding the world, and try to sort through what those theories and frameworks imply. There are professional incentives to defend surprising or counterintuitive positions, to poke at widely held pieties and components of “common sense morality,” and to develop thought experiments that are memorable and powerful (and because of that, pretty weird).

This isn’t a knock on philosophy; it’s what I studied in college and a field from which I have learned a tremendous amount. It’s good for society to have a space for people to test out strange and surprising concepts. But whatever the boundary-pushing concepts being explored, it’s important not to mistake that exploration for practical decision-making.

[…]

The dominance of academic philosophers in EA, and those philosophers’ increasing attempts to apply these kinds of thought experiments to real life — aided and abetted by the sudden burst of billions into EA, due in large part to figures like Bankman-Fried — has eroded the boundary between this kind of philosophizing and real-world decision-making. Poets, as Percy Shelley wrote, may be the unacknowledged legislators of the world, but EA made the mistake of trying to turn philosophers into the actual legislators of the future. A good start would be more clearly stating that funding priorities, for now, are less “longtermist” in this galaxy-brained Bostrom sense and more about fighting specific existential risks — which is exactly what EA funders are doing in most cases. The philosophers can trod the cosmos, but the funders and advocates should be tethered closer to Earth.

[…]
 

The problem is utilitarianism free from any guardrails …

Sam Bankman-Fried is a hardcore, pure, uncut Benthamite utilitarian. His mother, Barbara Fried, is an influential philosopher known for her arguments that consequentialist moral theories like utilitarianism that focus on the actual results of individual actions are better suited for the difficult real-world trade-offs one faces in a complex society. Her son apparently took that insight very, very seriously.

Effective altruists aren’t all utilitarians, but the core idea of EA — that you should attempt to act in such a way to promote the greatest human and animal happiness and flourishing achievable — is shot through with consequentialist reasoning. The whole project of trying to do the most good you can implies maximizing, and maximizing of “the good,” and that is the literal definition of consequentialism.

It’s not hard to see the problem here: If you’re intent on maximizing the good, you better know what the good is — and that isn’t easy. “​​EA is about maximizing a property of the world that we’re conceptually confused about, can’t reliably define or measure, and have massive disagreements about even within EA,” Holden Karnofsky, the co-CEO of Open Philanthropy and a leading figure in the development of effective altruism, wrote in September. “By default, that seems like a recipe for trouble.”

Indeed it was. It looks increasingly likely that Sam Bankman-Fried appears to have engaged in extreme misconduct precisely because he believed in utilitarianism and effective altruism, and that his mostly EA-affiliated colleagues at FTX and Alameda Research went along with the plan for the same reasons.

 

[…]
 

I think taking a high-earning job with the explicit aim of donating the money still makes a lot of sense for most big-money options.

But what SBF did was not just quantitatively but qualitatively different from classic “earn to give.” You can make seven figures a year as a trader in a hedge fund, but unless you manage the whole fund, you probably won’t become a billionaire. Bankman-Fried very much wanted to be a billionaire — so he could have more resources to devote to EA giving, if we take him at his word — and to do that, he set up whole new corporations that never would’ve existed without him. Those corporations then engaged in incredibly risky business practices that never would’ve occurred if he and his team hadn’t entered the field. He was not one-for-one replacing another finance bro who would have used the earnings on sushi and strippers rather than altruistic causes. He was building a whole new financial world, with consequences that would be much grander in scale.

And in building this world, he acted like a vulgar utilitarian. Philosophers like to talk about “biting the bullet”: accepting an unsavory implication of a theory you’ve adopted, and arguing that this implication really isn’t that bad.

 

[…]
 

Bankman-Fried’s error was an extreme hubris that led him to bite bullets he never should have bitten. He famously told economist Tyler Cowen in a podcast interview that if faced with a game where “51 percent [of the time], you double the Earth out somewhere else; 49 percent, it all disappears,” he’d keep playing the game continually.

 

This is known as the St. Petersburg paradox, and it’s a confounding problem in probability theory, because it’s true that playing the game creates more happy human lives in expectation (that is, adjusting for probabilities) than not playing. But if you keep playing, you’ll almost certainly wipe out humankind. It’s an example of where normal rules of rationality seem to break down.

But Bankman-Fried was not interested in playing by the normal rules of rationality. Cowen notes that if Bankman-Fried kept this up, he’d almost certainly wipe out the Earth eventually. Bankman-Fried replied, “Well, not necessarily. Maybe you St. Petersburg paradox into an enormously valuable existence. That’s the other option.”

These are fun dorm room arguments. They should not guide the decision-making of an actual financial company, yet there is some evidence they did. An as-yet-unconfirmed account of an Alameda all-hands meeting describes CEO Caroline Ellison explaining to staff that she and Bankman-Fried faced a choice in early summer 2022: either to let Alameda default after some catastrophic losses, or to raid consumer funds at FTX to bolster Alameda. As the researcher David Dalrymple has noted, this was basically her and Bankman-Fried making a “double or nothing” coin flip: By taking this step, they reasoned they could either save Alameda and FTX or lose both (as wound up happening), rather than keep just FTX, as in a scenario where the consumer funds were not raided.

If one of the main motivations for effective altruism is to challenge traditional, ineffective ways of doing things, such as bureaucracy, mismanagement, passivity, and established procedures, and to differentiate itself from the world of charity and its connection to the establishment, especially in societies like the UK, then traditional institutional experience by older persons won't be enough. I would argue that a better goal would be identifying what is not working and must be worked on  in relation to learning about and fighting fraud, including developing better tools to do so.

To truly go against the traditional ineffective way of doing things and create differentiation, effective altruism needs to prioritize developing new tools and approaches for addressing issues like charity fraud. This could include using coordination technologies, artificial intelligence, and knowledge networks to identify and review potential frauds, as well as working on building a community of expertsand an academic network who can help develop and implement these solutions. By focusing on innovation and tackling these challenges head-on, effective altruism can continue to set itself apart and make a real impact.