Recent and upcoming media related to EA

By Emma Richter🔸 @ 2024-03-28T20:32 (+104)

I’m Emma from the Communications team at the Centre for Effective Altruism (CEA). I want to flag a few media items related to EA that have come out recently or will be coming out soon, given they’ll touch on topics—like FTX—that I expect will be of interest to Forum readers. 

I’ll add links for these episodes once they become available and plan to update this post as needed.

Habryka @ 2024-03-29T01:49 (+57)

The Washington Post article rings quite hollow. It claims that CEA and other EA organizations have taken FTX's downfall as an opportunity for "reflection and institutional reform", and cites the legal investigation CEA sponsored as evidence of this. 

However, as far as I can tell the primary goal of that legal investigation was mostly PR-related,[1] trying to clear CEA's name and prove to the outside world that no one knew the full extend of FTX's fraud, and was not aimed at facilitating an internal reflection process (or at the very least people I've talked to at CEA described it to me as something they did not at all expect to be helpful as part of a reflection process, and multiple described the constraints imposed by it as harmful). 

If-anything the legal investigation seems to have actually substantially interferred with CEA reflection processes, with Will MacAskill himself telling me that the EV board prevented him from publishing his reflections on FTX due to legal and PR concerns, and due to it maybe making the legal investigation seem less legitimate. 

Overall, the piece reads like a puff piece and not like something that displays real reflection. I mean, it's fine to write puff-pieces from time to time, but it rings hollow in a world where actual institutional reform as a result of FTX has been almost completely absent.

  1. ^

    Edit: I mean PR here not necessarily in a shallow sense. I think it's reasonable for CEA to want to clear its name. It just has basically nothing to do with an "internal reflection process" and "institutional reform" and claiming it does seems deceptive to me

Owen Cotton-Barratt @ 2024-03-29T11:03 (+72)

I think you're missing some important ground in between "reflection process" and "PR exercise".

I can't speak for EV or other people then on the boards, but from my perspective the purpose of the legal investigation was primarily about helping to facilitate justified trust. Sam had by many been seen as a trusted EA leader, and had previously been on the board of CEA US. It seemed it wouldn't be unreasonable if people in EA (or even within EV) started worrying that leadership were covering things up. Having an external investigation was, although not a cheap signal to send, much cheaper in worlds where there was nothing to hide compared to worlds where we wanted to hide something; and internal trust is extremely important. Between that and wanting to be able to credibly signal to external stakeholders like the Charity Commission, I think general PR was a long way down the list of considerations.

Habryka @ 2024-03-29T17:18 (+30)

Yep, I think the investigation did screen off some extreme scenarios in which people at CEA were colluding very actively with Sam about his biggest crimes. I at least had very little probability on this, and while it's good to clear up such an extreme scenario, I don't think it has much to do with actually preventing future things like FTX. 

I totally agree there is value in signaling to external stakeholders that CEA is not literally criminal, but I don't think this has anything to do with a claimed "internal reflection process" and "institutional reform". 

If the section had said "my organization has been working hard on trying to clarify that we were not literally colluding on crimes with Sam" then I wouldn't have objected, or even something weaker like "I do think people overestimate the degree to which EA knew about Sam's crimes, and a recent investigation showed that at CEA nobody knew of the major crimes that Sam committed before FTX collapsed". But that's not what the section claims. It directly claims that the investigation was part of an "internal reflection process" and "instutitional reform", and I have been shared on documents by CEA employees where the legal investigation was explicitly called out as not being helpful for facilitating a reflection process and institutional reform (which, to be clear, does not mean that others at CEA don't think it was important, but I think the case for that is quite weak, given the large costs it imposed on other people trying to do investigations, aggregating evidence, and the extreme narrowness of the questions investigated). 

Owen Cotton-Barratt @ 2024-03-29T18:12 (+37)

Sorry, I wasn't meaning to object to your critique overall. On my impression it's substantively correct that it's a puff piece and slightly disingenuous. (Maybe I should have owned that up-front. I still feel a bit bad about saying it baldly, because Zach is relatively new in a role that I think is often both difficult and thankless; I'd guess he's doing a good job overall and it seems bad if the feedback he gets skews too negative; I think it's probably good if he does more media. But I'm now worried that not saying it will be more confusing/costly, so I've just said it.)

But:

  • You seemed to be misstating what was going on with the external investigation, and I think that that obscured what was happening in a way that seemed unhelpful for readers, and wanted to correct that
  • I still think you're overstating the extent to which the things described aren't about institutional reform
    • I think they kind of are -- but it should be understood that the purpose of the reforms is more about improving legibility than about fundamentally reducing the risk of similar occurrences
Habryka @ 2024-03-29T19:43 (+34)

because Zach is relatively new in a role that I think is often both difficult and thankless

Yeah, I agree with this. 

I think Zach definitely has a very difficult job, and I would like to support him in what he is doing. I seem to have disagreements about how to go about it, but that doesn't change the fact that I think he is paying large costs in stress and associated suffering in the pursuit of aims that I do care deeply about, and I think it's important to recognize that despite any disagreements we might have about strategy and tactics. Man in the arena and all that. 

Habryka @ 2024-03-29T19:54 (+27)

I still think you're overstating the extent to which the things described aren't about institutional reform

I think they kind of are -- but it should be understood that the purpose of the reforms is more about improving legibility than about fundamentally reducing the risk of similar occurrences

So, I feel like you are using "improving legibility" as kind of a euphemism here. I don't currently believe that you think that the changes that CEA has made would make it easier for you or me or the external world to discover whether something like FTX was likely to happen again. At least I can't think of any changes that would help with that. 

To the contrary, my sense is the changes that have happened are much more directly optimizing for "making people believe that something like FTX won't happen again", but not in a way that is actually sensitive to the underlying facts. Like, I don't think CEA has made it easier for external parties to judge whether EA will have good or bad consequences for the world. 

I think in order to use the words "improving legibility" here, the key thing that needs to be the case is that CEA is doing things that would cause people to believe that future things like FTX are likely to happen, if they are likely to happen, and to cause people to believe that future things like FTX are unlikely to happen, if they are indeed are unlikely to happen. I don't currently see such things, and indeed my guess is the public's ability to judge what is going in inside of EA, and the consequences of our actions, has gotten worse, not better, as a result of recent changes.

I don't think the legal investigation helps very much here, and I am not sure whether it overall would make people's beliefs here more or less accurate (as I also think the similar OpenAI investigation has demonstrated) and the primary reason people do these kinds of legal investigations is to get a voice of authority on their side that makes people more hesitant to criticize them. I don't think it's completely useless, but it's pretty close to it (especially when realizing how much OpenAI seems to have been able to manipulate a similar investigation in whatever way they desired).

Owen Cotton-Barratt @ 2024-03-29T22:55 (+12)

(Sorry you're getting downvoted, this seems like a productive conversation to me.)

There's a big question about who things are being legible to. I don't think this would make things meaningfully more legible to you or people in your reference class of basically being high context on the org and having a lot of intersections in social circles.

I think there's a large crowd who are closer to "the general public" who only know about things via stuff that's written publicly, and will only ever read a small fraction of that. I think that these kinds of reform help to make legible to that crowd that the org isn't on some broad index-of-corruption where there is a culture of doing things for your mates, following the path of convenience (especially if that means getting more money), etc. That's not directly about "will there be another SBF?" (base rates are surely low), but pretty informative anyway for a bunch of questions about how excited people will be to interact with the org/community.

I think if EV had had a high corruption culture (even if they didn't literally know about any crimes), it would have been much more costly to have an external investigation (and such an investigation would likely have shaken out differently). This doesn't exclude the possibility that there would have been better/cheaper ways to signal this. And perhaps there are other effects I'm not tracking; I'd be interested if you wanted to elaborate on the view that it makes it harder for the public to tell what's going on inside EA.

Larks @ 2024-03-30T01:46 (+15)

I think if EV had had a high corruption culture (even if they didn't literally know about any crimes), it would have been much more costly to have an external investigation (and such an investigation would likely have shaken out differently).

This is an interesting argument, thanks for making it. Would you mind explaining in a bit more depth? I would have thought the structure of the investigation, especially including the fact that the results were not published, would limit the cost of the investigation. As far as I'm aware the only thing outside observers have to go on is this paragraph, which appears to be from this guy:

Finally, I want to add this point.  Over the last year, my team and I had the chance to meet dozens of people affiliated with the various EV projects.  We were consistently impressed by everyone’s genuine commitment and dedication to the EA mission, particularly during a very challenging year.  I wish you all the best in your future endeavors.  

Is there some significant difference you see between the EVF investigation and the OpenAI investigation?

Owen Cotton-Barratt @ 2024-03-30T02:04 (+8)

  1. In the world with high corruption, I'd have expected more individuals to be scared of what an investigation would turn up.

  2. If the investigation found major issues which weren't publicly reported on, it would increase the fragility of EV's position, since anyone who had seen the investigation could become a whistleblower with a relatively incontrovertible case.

These don't totally exclude "there's a big collusion not to share the issues", but "major internal collusion" without also "was colluding with SBF" is getting to be a narrower target, and I think it would still feel fragile (such that the colluders might rationally prefer to avoid an investigation).

(Not commenting on the comparison with the OpenAI investigation because I haven't understood it thoroughly enough to feel I have good takes.)

Jason @ 2024-03-30T16:13 (+72)

I have the opposite reaction.

Background

To recap, Zach wrote about four months ago:

Mintz found no evidence that anyone at EV (including employees, leaders of EV-sponsored projects, and trustees) was aware of the criminal fraud of which Sam Bankman-Fried has now been convicted.

While we are not publishing any additional details regarding the investigation because doing so could reveal information from people who have not consented to their confidences being publicized and could waive important legal privileges that we do not intend to waive, we recognize that knowledge of criminal activity isn’t the only concern. I plan to share other non-privileged information on lessons learned in the aftermath of FTX and encourage others to share their reflections as well. 

The information released from the investigation in this statement is exceedingly narrow -- we didn't have actual knowledge of SBF's criminal fraud. That's an exceedingly low bar to meet! That EVF won't be joining SBF on his Club Fed vacation is an awfully slender reed to talk about "justified trust."

General Reaction to Scope of Disclosure (and Likelihood of Possible Inferences)

Is there anything else that has been "credibly signal[ed]" through the investigation process? (quoting Owen's earlier comment). Although I am aware of the reasons to maintain legal privileges, the choice to do so does little to persuade that this is a world in which EVF has "nothing to hide compared to [one in which it] wanted to hide something" (quoting same comment). Indeed, worlds in which the investigation sponsor chooses to hide behind privilege as to all but one (or a few) of the report's ultimate conclusions seem very likely to be worlds in which there the sponsor "wanted to hide something." And the choice to exclude certain important questions from the investigation's scope would update me toward the view that the investigation sponsor didn't want to find out the answers or strongly suspected they would be unflattering.

If Mintz had concluded that there was no negligence or recklessness in relation to EVF's interactions with SBF, I am guessing that conclusion would have been disclosed! Likewise, if I wanted to convey to the CC that I was a trustworthy public charity, I would release a whole lot more than "no actual knowledge of a massive fraud" if I could do so.

Reacting to the Positive View of the Investigation

In the world with high corruption, I'd have expected more individuals to be scared of what an investigation would turn up.

But fear based on what an investigation would turn up only makes sense if you were concerned that the negative results would get out. So I think this is only correct insofar as those conducting the investigation were confident that it would find no actual knowledge of the criminal fraud.

If the investigation found major issues which weren't publicly reported on, it would increase the fragility of EV's position, since anyone who had seen the investigation could become a whistleblower with a relatively incontrovertible case.

But the number of people who would see the investigation report could be extremely limited. Presumably access to the report itself is extremely limited, given the rationale about not waiving privileges and protecting sources who spoke to the investigative firm. Outside counsel can't leak without counsel surrendering his law license, incurring massive liability, and watching his business crumble. It's plausible to me that no more than 10-15 people at EVF have seen the full report, and that these people are almost all insiders who are rather unlikely to leak.

How The Investigation Could Have Actually Rebuilt Lost Trust and Confidence

There was a more transparent / credible way to do this. EVF could have released, in advance, an appropriate range of specific questions upon which the external investigator was being asked to make findings of fact -- as well as a set of possible responses (on a scale of "investigation rules this out with very high confidence" to "investigation shows this is almost certain"). For example -- and these would probably have several subquestions each -- one could announce in advance that the following questions were in scope and that the investigator had committed to providing specific answers:

  • Did anyone associated with EVF ever raise concerns about SBF being engaged in fraudulent activity? Did they ever receive any such concerns?
  • Did anyone associated with EVF discourage, threaten, or seek to silence any person who had concerns about illegal, unethical, or fraudulent conduct by SBF? (cf. the "Will basically threatened Tara" report).
  • When viewed against the generally-accepted norms for donor vetting in nonprofits, was anyone associated with EVF negligent, grossly negligent, or reckless in evaluating SBF's suitability as a donor, failing to raise concerns about his suitability, or choosing not to conduct further investigation?

That kind of pre-commitment would have updated my faith in the process, and my confidence that the investigation reached all important topics. If EVF chose not to release the answers to those questions, it would have known that we could easily draw the appropriate inferences. Under those circumstances -- but not the actual circumstances -- I would view willingness to investigate as a valuable signal.

Owen Cotton-Barratt @ 2024-03-30T20:54 (+9)

Thanks, this makes sense. I hadn't thought about the possibility of committing to have the external investigation provide public answers on specific questions. (And the fact that I hadn't thought of it gives me something to reflect on about how to ideally act during crises.)

Habryka @ 2024-03-30T18:14 (+13)

Yeah, I think you are pointing towards something real here. 

Like, I do think a thing that drove my reaction to this was a perspective in which it was obvious that most people in EA didn't literally actively participate in the FTX fraud. I have encountered very extreme and obviously wrong opinions about this in the public (the comment section of the WaPo article provides many examples of this), and there is some value in engaging with that. 

But I do think that is engaging with a position that is extremely shallow, and the mechanism of it seems like its trying to attack shallowness with more shallowness, which I don't think is necessarily a strategy to rule out, but one that I am glad EA has avoided so far. 

The way I would like Zack to have done it would have been to make a well-scoped statement that is honest and says "we might still be substantially responsible for SBF, but if so, we were because of indirect effects, not because we directly helped with the fraud. We really didn't know how bad the fraud was, and we did not directly participate in it, and here is an investigation that shows that"

I'd be interested if you wanted to elaborate on the view that it makes it harder for the public to tell what's going on inside EA.

I mean, as I mentioned as a concrete example, Will was told by CEA board members and leadership to not write anything detailed publicly about his FTX experiences due to the ongoing investigation. This seems like it has been extremely costly in terms of my ability and others ability to understand what happened, and I don't see the corresponding payoff in the investigation results. To quote from Will 9 months ago: 

The independent investigation commissioned by EV is still ongoing, and the firm running it strongly preferred me not to publish posts on backwards-looking topics around FTX while the investigation is still in-progress. I don’t know when it’ll be finished, or what the situation will be like for communicating on these topics even after it’s done.

Will publishing backwards-looking posts on FTX seems like the obvious and most central thing that should happen as part of FTX reform. We have clear documentation that the investigation prevented that from happening (and as far as I can tell the reflections are still not published, so it seems like the investigations completely prevented this kind of information from entering the public record).

RobBensinger @ 2024-03-31T21:32 (+13)

It directly claims that the investigation was part of an "internal reflection process" and "instutitional reform", and I have been shared on documents by CEA employees where the legal investigation was explicitly called out as not being helpful for facilitating a reflection process and institutional reform. 

This seems like the headline claim to me. EAs should not be claiming false things in the Washington Post, of all things.

Habryka @ 2024-03-31T22:00 (+7)

I didn't intend the above to be a straightforward accusation of lying. My sense is that different people at CEA disagree here about how useful this investigation was for reform purposes (which I think is evidence of the claim being false, but not strong evidence of deception).

If I had to make a bet, I would say majority opinion at CEA on the question of "did this investigation obstruct or assist internal reform efforts at CEA (as opposed to e.g. address legal liability and reputational risks independent from their best guess about what appropriate reform is)?" would be "obstruct", but I don't know, I haven't done a survey. 

When I was talking to people about the investigation in the context of broader reform, the almost universal response was "oh, I don't think you should get your hopes up about this investigation" and "oh, I don't really think the investigation is going to help much with that". And that seemed reflected in some Google Docs I saw. But idk, maybe Zack thinks it was more part of it.

Jason @ 2024-03-30T16:27 (+58)

I mean, it's fine to write puff-pieces from time to time, but it rings hollow in a world where actual institutional reform as a result of FTX has been almost completely absent.

We really need to see the receipts on any claim that a significant degree of actual institutional reform as a result of FTX has occurred. I think that's part of why the op-ed comes off as a puff piece. Another reason is that we haven't seen the receipts to sufficiently establish that there was meaningful investigation of any question beyond whether anyone at EVF had actual knowledge of the fraud. 

The kinds of reform-with-receipts I'd be looking for might include:

  • A published policy against retaliation against any person on the basis that the person expressed concern about possible unethical, illegal, or fraudulent activity by a donor. For certain classes of protected persons (e.g., an employee), this should include a right to a hearing before an independent arbitrator (e.g., at the AAA) with the ability to order reinstatement, appropriate damages, and fees/costs.
  • A publicly-known staff member with express responsibility for making determinations on donor risk issues, whose judgment of excessive risk could only be overturned by formal board action, and whose employment contract sharply limited the staff member's exposure to adverse employment action (e.g., such action could only be taken for good cause shown to an independent arbitrator).[1]
  1. ^

    By analogy, administrative law judges in the US federal government enjoy very strong protections from adverse actions by their employing agencies. The costs of these protections (e.g., shielding ALJs who are just plain awful at their jobs) are seen as less important than promoting their independence from the agency that has an interest in their judicial decisions. 

Habryka @ 2024-03-30T18:08 (+33)

I agree with this overall sentiment, though want to mark that I don't think "donor risk" is the right way to model this. 

The key thing to IMO investigate is why SBF, substantially driven by EA principles, built FTX on a substantial basis of fraud. Even if FTX/Sam had donated nothing to CEA, the harm seems just as big, and the key thing to investigate is what role EA played in FTX coming into existence, not what EA could have done to avoid being associated with SBF.

Jason @ 2024-03-31T21:21 (+15)

I doubt we disagree much, if at all.

Examining "what role EA played in FTX coming into existence" seems desirable, but doing a good job on this still seems rather challenging as a practical matter at least for now. FTX is unlikely to cooperate (and doing so would consume revenue that morally belongs to the victims). SBF denies having done anything wrong and will appeal. The co-conspirators are not going to talk because of their deal with USAO/SDNY, and even once that is done, I'd probably be worried about state and foreign criminal exposure in their shoes.[1] Although EVF settled with the FTX estate, many actors and orgs still have legal exposure. EVF could have counsel investigate EVF and keep the results privileged, but a license to practice law doesn't give you the ability to investigate whatever you'd like and keep the results secret. 

One final issue is who should sponsor an investigation of "what role EA played in FTX coming into existence"; given the linkages between EVF, EVF insiders, and SBF, EVF would be somewhere in the vicinity of last place on my ideal preference list.

As for the reference to "donor risk" in the reform-with-receipts segment, I had examples of the policy response I'd like to see specifically from EVF in mind. But I do think donor-risk management is part of a defense-in-depth strategy. No one owns EA principles, and while taking steps to discourage people away from fraud-approving views, we're not going to find any way to absolutely preclude people from interpreting / applying those principles in a way that motivates them to do awful things.[2] 

That means we shouldn't put all the marbles into one basket. If a would-be fraudster knows ahead of time that EA orgs will reject his money if it seems too suspect, and that in fact the donor due diligence process might lead to discovery of the fraud --> life in a prison cell, then they are less likely to start fraud-to-give in the first place. There being no way to be 100.00% sure donor money is clean, I gestured to "donor risk issues" rather than overselling with words like assure, ensure, guarantee, etc.[3]

  1. ^

    It's usually not considered cool for a state to prosecute after the feds, but it's not against the rules. And it's likely that the co-conspirators violated criminal laws in a number of countries where victims lived. I would not assume that the US, which can be pretty aggressive on seeking extradition for financial crimes where the defendants never set foot on U.S. soil, would deny such a request.

  2. ^

    In the same way, no one owns any set of ideas, and almost all ideas worth having can be twisted to justify bad actions. Of course, I would definitely agree that EA ideas are unusually vulnerable to being transmuted into naive consequentialism.

  3. ^

    As someone who is significantly less than 100% a utilitarian, I also identify a moral obligation to take reasonable steps to address the possibility that an organization might be receiving funds tainted by fraud (among other things).

Habryka @ 2024-03-31T22:02 (+10)

One final issue is who should sponsor an investigation of "what role EA played in FTX coming into existence"; given the linkages between EVF, EVF insiders, and SBF, EVF would be somewhere in the vicinity of last place on my ideal preference list.

I think CEA or EV collaborating with an external but still within-EA trusted organization seems like the best choice to me here. Hiring someone who is broadly known to be independent (like, IDK, you could choose someone from Rethink, or Tyler Cowen, or someone else in that kind of reference class), seems like a good idea.

Jason @ 2024-03-31T23:07 (+2)

[Edit: With "collaboration" as defined in Habryka's response below, my question dissolves.]

Is there a reason to prefer CEA or EV collaborating with an investigator versus someone (or several someones) funding an investigator(s), taking a back seat once the proposal is accepted and funded, and deferring to the investigator(s) what to publish? [I may be reading too much into "collaborating" in your first sentence.]

Habryka @ 2024-03-31T23:16 (+24)

What I mean by collaboration is "is willing to share any information with them and allow staff to speak freely". The key obstacle I have faced in trying to do investigations is that nobody is willing to talk or say anything that goes on the record by organizational policy, which of course makes this kind of thing very hard to pull off. 

I also think it would help a lot if CEA were to lend some credibility to the investigation. People don't want to repeat the same thing hundreds of times, and it would IMO be good for CEA/EV to put some social capital on the line to encourage people to talk to the investigators.

Stephen Clare @ 2024-03-29T14:40 (+51)

Considering how much mud was being slung around the FTX collapse, "clearing CEA's name" and proving that no one there knew about the fraud seems not just like PR to me, but pretty important for getting the org back to a place where it’s able to meaningfully do its work.

Plus, that investigation is not the only thing mentioned in the reflection reform paragraph. The very next sentence also says CEA has "reinvested in donor due diligence, updated our conflict-of-interest policies and reformed the governance of our organization, replacing leadership on the board and the staff."

Nathan Young @ 2024-04-01T21:32 (+12)

I don't love this article but it's fine. In general many other articles about EA are too negative so it doesn't really seem worth writing a big correction when the median person who hears about EA probably hears about the right thing.

Specifically, are new readers gonna believe that EA has done a load of useful soul-searching because this articles says it? I doubt it. There are enough articles saying that EAs are a bunch of cynical psychopaths that many will probably assume this is the fluff piece (that it is). 

I don't really think this meta discussion is that high priority, the better question is, how does EA improve and focus back on effective altruism, but I don't have a great answer to that, other than write articles specifically focused at that.