Feels like taking into account the likelihood that the "1.5% probability of 100,000 DALYs averted" estimate is a credence based on some marginally-relevant base rate^[1] that might have been chosen with a significant bias towards optimism is very much in keeping with the spirit of the question (which presumably is about gauging attitudes towards uncertainty, not testing basic EV calculation skills)^[2]. 

A very low percentage chance of averting a lot of DALYs feels a lot more like "1.5% of clinical trials of therapies for X succeeded; this untested idea might also have a 1.5% chance" optimism attached to a proposal offering little reason to believe it's above average rather than an estimate based on somewhat robust statistics (we inferred that 1.5% of people who receive this drug will be cured from the 1.5% of people who had that outcome in trials). So it seems quite reasonable to assume that the 1.5% chance of a positive binary outcome estimate might be biased upwards. Even more so in the context of "we acknowledge this is a long shot and high-certainty solutions to other pressing problems exist, but if the chance of this making an impact was as high as 0.0x%..." style fundraising appeals to EAs' determination to avoid scope insensitivity.

1. ^{^}

   either that or someone's been remarkably precise in their subjective estimates or collected some unusual type of empirical data. I certainly can't imagine reaching the conclusion an option has exactly 1.5% chance of averting 100k DALYs myself 

2. ^{^}

   if you want to show off you understand EV and risk estimation you'd answer (C) "here's how I'd construct my portfolio" anyway :-) 

Note that there was some discussion of this in a previous post

The bar should not be at 'difficult financial situation', and this is also something there are often incentives against explicitly mentioning when applying for funding. Getting paid employment while studying (even fulltime degrees) is normal.

My 5 minute Google search to put some numbers on this:

Proportion of students who are employed while studying: UK: survey of 10,000 students showed that 56% of full-time UK undergraduates had paid employment (14.5 hours/week average) - June 2024 Guardian article https://www.theguardian.com/education/article/2024/jun/13/more-than-half-of-uk-students-working-long-hours-in-paid-jobs USA: 43% of full-time students work while enrolled in college - January 2023 Fortune article https://fortune.com/2023/01/11/college-students-with-jobs-20-percent-less-likely-to-graduate-than-privileged-peers-study-side-hustle/

Why are students taking on paid work? UK: "Three-quarters of those in work said they did so to meet their living costs, while 23% also said they worked to give financial support for friends or family." From the Guardian article linked above. Cannot find a recent US statistic quickly, but given the system (e.g. https://www.collegeave.com/articles/how-to-pay-for-college/) I expect working rather than taking out (as much in) loans is a big one.

On the other hand, spending time on committees is also very normal as an undergraduate and those are not paid. However in comparison the time people spend on this is much more limited (say ~2-5 hrs/week), there is rarely a single organiser, and I've seen a lot of people drop off committees - some as they are less keen, but some for time commitment reasons (which I expect will sometimes/often be doing paid work).

How do you and your wife decide where to give to, collectively? Do you guys each have a budget, do you discuss a lot and fund based on consensus, something else?

<3 This is so lovely @Allan_Saldanha! I think it is such a lovely and remarkable thing about our community that so many people have been quietly living their lives and just giving their 10, 20, 40, 75(!) percent to causes they care about, some now over the course of 10+ years. "Generous and thoughtful giving is normal here" continues to be one of my favorite facts about EAs :')

Thanks for doing this AMA!

I wrote a post on “Charity” as a conflationary alliance term. You can read it on LessWrong, but I'm also happy to discuss it here.

If wondering why not post it here: Originally posted it here with a LW cross-post. It was immediately slapped with the "Community" tag, despite not being about community, but about different ways people try to do good, talk about charity & ensuing confusions. It is about the space of ideas, not about the actual people or orgs.

With posts like OP announcements about details of EA group funding or EAG admissions bar not being marked as community, I find it increasingly hard to believe the "Community" tag is driven by the stated principe marking "Posts about the EA community and projects that focus on the EA community" and not by other motives, like e.g. forum mods expressing the view "we want people to think less about this / this may be controversial / we prefer someone new to not read this".
 
My impression this moves substantial debates about ideas to the side, which is a state I don't want to cooperate on by just leaving it as it is -> moved the post on LessWrong and replaced by this comment. 

My intuitive reaction to this is "Way to screw up a survey." 

Considering that three people agree-voted your post, I realize I should probably come away with this with a very different takeaway, more like "oops, survey designers need to put in extra effort if they want to get accurate results, and I would've totally fallen for this pitfall myself."

Still, I struggle with understanding your and the OP's point of view. My reaction to the original post was something like:

Why would this matter? If the estimate could be off by 1 percentage point, it could be down to 0.5% or up to 2.5%, which is still 1.5% in expectation. Also, if this question's intention were about the likelihood of EA orgs being biased, surely they would've asked much more directly about how much respondees trust an estimate of some example EA org.

We seem to disagree on use of thought experiments. The OP writes:

When designing thought experiments, keep them as realistic as possible, so that they elicit better answers. This reduces misunderstandings, pitfalls, and potentially compounding errors. It produces better communication overall.

I don't think this is necessary and I could even see it backfiring. If someone goes out of their way to make a thought experiment particularly realistic, maybe respondees might get the impression that it is asking about a real-world situation where they are invited to bring in all kinds of potentially confounding considerations. But that would defeat the point of the thought experiment (e.g., people might answer based on how much they trust the modesty of EA orgs, as opposed to giving you their personal tolerance of risk of the feeling of having had no effect/wasted money in hindsight). The way I see it, the whole point of thought experiments is to get ourselves to think very carefully and cleanly about the principles we find most important. We do this by getting rid of all the potentially confounding variables. See here for a longer explanation of this view. 

Maybe future surveys should have a test to figure out how people understand the use of thought experiments. Then, we could split responses between people who were trying to play the thought experiment game the intended way, and people who were refusing to play (i.e., questioning premises and adding further assumptions).

*On some occasions, it makes sense to question the applicability of a thought experiment. For instance, in the classical "what if you're a doctor who has the opportunity to kill a healthy patient during routine chek-up so that we could save the lives of 4 people needing urgent organ transplants," it makes little sense to just go "all else is equa! Let's abstract away all other societal considerations or the effect on the doctor's moral character." 
So, if I were to write a post on thought experiments today, I would add something about the importance of re-contextualizing lessons learned within a thought experiments to the nuances of real-world situations. In short, I think my formula would be something like, "decouple within thought experiments, but make sure add an extra thinking step from 'answers inside a thought experiment' to 'what can we draw from this in terms of real-life applications.'" (Credit to Kaj Sotala, who once articulated a similar point in probably a better way.)

They do specifically say that they consider other types of university funding to have greater cost-benefit (and I don't think it makes sense to exclude reputation concerns from cost-benefit analysis, particularly when reputation boost is a large part of the benefit being paid for in the first place). Presumably not paying stipends would leave more to go around. I agree that more detail would be welcome.

Here’s a GiveWell blog post from 2009 that engages with this question.

Dang. That makes sense, but it seems pretty grim. The second half of that argument is, "We can't select for not-feeling-pain, because we need to spend all of our future genetic modification points on the chickens getting bigger and growing even faster."

I'm kind of surprised that this argument isn't at all about the weirdness of it. It's purely pragmatic, from their standpoint. "Sure, we might be able to stop most of the chicken suffering, but that would increase costs by ~20% or so, so it's a non-issue"

Copying over the rationale for publication here, for convenience: 

Rationale for Public Release

Releasing this report inevitably draws attention to a potentially destructive scientific development. We do not believe that drawing attention to threats is always the best approach for mitigating them. However, in this instance we believe that public disclosure and open scientific discussion are necessary to mitigate the risks from mirror bacteria. We have two primary reasons to believe disclosure is necessary:

1. To prevent accidents and well-intentioned development

If no serious concerns are raised, the default course of well-intentioned scientific and technological development would likely result in the eventual creation of mirror bacteria. Creating mirror life has been a long-term aspiration of many academic investigators, and efforts toward this have been supported by multiple scientific funders.1 While creating mirror bacteria is not yet possible or imminent, advances in enabling technologies are expected to make it achievable within the coming decades. It does not appear possible to develop these technologies safely (or deliberately choose to forgo them) without widespread awareness of the risks, as well as deliberate planning to mitigate them. This concern is compounded by the possibility that mirror bacteria could accidentally cause irreversible harm even without intentional misuse. Without awareness of the threat, some of the most dangerous modifications would likely be made for well-intentioned reasons, such as endowing mirror bacteria with the ability to metabolize ᴅ-glucose to allow growth in standard media.

2. To build guardrails that could reliably prevent misuse

There are currently substantial technical barriers to creating mirror bacteria. Success within a decade would require efforts akin to those of the Human Genome Project or other major scientific endeavors: a substantial number of skilled scientists collaborating for many years, with a large budget and unimpeded access to specialized goods and services. Without these resources, entities reckless enough to disregard the risks or intent upon misuse would have difficulty creating mirror bacteria on their own. Disclosure therefore greatly reduces the probability that well-intentioned funders and scientists would unwittingly aid such an effort while providing very little actionable information to those who may seek to cause harm in the near term.

Crucially, maintaining this high technical barrier in the longer term also appears achievable with a sustained effort. If well-intentioned scientists avoid developing certain critical components, such as methods relevant to assembling a mirror genome or key components of the mirror proteome, these challenges would continue to present significant barriers to malicious or reckless actors. Closely monitoring critical materials and reagents such as mirror nucleic acids would create additional obstacles. These protective measures could likely be implemented without impeding the vast majority of beneficial research, although decisions about regulatory boundaries would require broad discussion amongst the scientific community and other stakeholders, including policymakers and the public. Since ongoing advances will naturally erode technical barriers, disclosure is necessary in order to begin discussions while those barriers remain formidable.

Re "epistemics and integrity" - I'm glad to see this problem being described. It's also why I left (angrily!) a few years ago, but I don't think you're really getting to the core of the issue. Let me try to point at a few things

• centralized control and disbursion of funds, with a lot of discretionary power and a very high and unpredictable bar, gives me no incentive to pursue what I think is best, and all the incentive to just stick to the popular narrative. Indeed groupthink. Except training people not to groupthink isn't going to change their (existential!) incentive to groupthink. People's careers are on the line, there are only a few opportunities for funding, no guarantee to keep receiving it after the first round, and no clear way to pivot into a safer option except to start a new career somewhere your heart does not want to be, having thrown years away

• lack of respect for "normies". Many EA's seemingly can't stand interacting with non-EA's. I've seen EA meditation, EA bouldering, EA clubbing, EA whatever. Orgs seem to want everyone and the janitor to be "aligned". Everyone's dating each other. It seems that we're even afraid of them. I will never forget that just a week before I arrived at an org I was to be the manager of, they turned away an Economist reporter at their door...

• perhaps in part due to the above, massive hubris. I don't think we realise how much we don't know. We started off with a few slam dunks (yeah wow 100x more impact than average) and now we seem to think we are better at everything. Clearly the ability to discern good charities does not transfer to the ability to do good management. The truth is: we are attempting something of which we don't even know whether it is possible at all. Of course we're all terrified! But where is the humility that should go along with that?

Here’s a GiveWell blog post from 2009 that engages with this question.

Re: agency of the community itself, I've been trying to get to this "pure" form of EA in my university group, and to be honest, it felt extremely hard.

-People who want to learn about EA often feel confused and suspicious until you get to object-level examples. "Ok, impactful career, but concretely, where would that get me? Can you give me an example?". I've faced real resistance when trying to stay abstract.

-It's hard to keep people's attention without talking about object-level examples, be it for teaching abstract concepts. It's even harder once you get to the "projects" phase of the year.

-People anchor hard on some specific object-level examples after that. "Oh, EA ? The malaria thing?" (Despite my go-to examples included things as diverse as shrimp welfare and pandemic preparedness)

-When it's not an object-level example, it's usually "utilitarianism" or "Peter Singer", which act a lot as thought stoppers and have an "eek" vibe for many people.

-People who care about non-typical causes actually have a hard time finding data and making estimates.

-In addition to that, agency for really making estimates is hard to build up. One member I knew thought the most Impactful career choice he had was potentially working on nuclear fusion. I suggested him to find out about the Impact-Tractability-Neglectedness of it to compare to another option he had (even rough OOMs) as well as more traditional ones. I can't remember him giving any numbers even months later. When he just mentioned he felt sure about the difference, I didn't feel comfortable arguing about the robustness of his justification. It's a tough balance to strike between respecting preferences and probing reasons.

-A lot of it comes down to career 1:1s. Completing the ~8 or so parts is already demanding. You have to provide estimates that are nowhere to be found if your center of interest is "niche" in EA. You then have to find academic and professional opportunities as well as relations that are not referenced anywhere in the EA community (I had to reach back to the big brother of a primary school friend I had lost track of to get a fusion engineer he could talk to!). If you need funding, even if your idea is promising, you need excellent communication skills for writing a convincing blog post, plausibly enough research skills to get non-air-plucked estimates for ITN / cost-effectiveness analysis, and a desire to go to EAGs and convince people who could just not care. Moreover a lot of people expressly limit themselves to their own country or continent. It's often easier to stick to the usual topics (I get call for applications for AIS fellowships almost every months, of course I never had ones about niche topics)

-Another point about career 1:1s, the initial list of options to compare is hard to negotiate. Some people will neglect non-EA options, others will neglect EA options, and I had issues with artificially adding options to help them truly compare options.

-Another other point, some people barely have the time to come to a few sessions. It's hard to get them to actually rely on the methodological tools they haven't learned about in order to compare their options during career 1:1s.

-A good way to cope with all of this is to encourage students to start things out -to create an org rather than joining one. But not everyone has the necessary motivation for this.

I'm still happy with having started the year with epistemics, rationality, ethics and meta-ethics, and to have done other sessions on intervention and policy evaluation, suffering and consciousness, and population ethics. I didn't desperately need to have sessions on GHD / Animal Welfare/ AI Safety, thought they're definitely "in demand".

[Promise this is not a scam] Sign up to receive a free $50 charity gift card from a rich person

Every year, for the past few years, famous rich person Ray Dalio has given away 20,000 $50 gift cards. And he is doing it again this year. These can be given any of over 1.8 million US registered charities, which includes plenty of EA charities

Here's an announcement post from Ray Dalio's instagram for verification

Register here to receive notification when the gift cards become available.

I wrote a post on “Charity” as a conflationary alliance term. You can read it on LessWrong, but I'm also happy to discuss it here.

If wondering why not post it here: Originally posted it here with a LW cross-post. It was immediately slapped with the "Community" tag, despite not being about community, but about different ways people try to do good, talk about charity & ensuing confusions. It is about the space of ideas, not about the actual people or orgs.

With posts like OP announcements about details of EA group funding or EAG admissions bar not being marked as community, I find it increasingly hard to believe the "Community" tag is driven by the stated principe marking "Posts about the EA community and projects that focus on the EA community" and not by other motives, like e.g. forum mods expressing the view "we want people to think less about this / this may be controversial / we prefer someone new to not read this".
 
My impression this moves substantial debates about ideas to the side, which is a state I don't want to cooperate on by just leaving it as it is -> moved the post on LessWrong and replaced by this comment. 

You can sort by "oldest" and "newest" in the comment-sort order, and see that mine shows up earlier in the "oldest" order, and later in the "newest" order.

Lorenzo Buonanno🔸 @ 2024-12-12T22:14 (+6)

You can also right-click → inspect element on the time indicator:
 

I agree that this is an inference. I currently think the OP thinks that in the absence of frugality concerns this would be among the most cost-effective uses of money by Open Phil's standards, but I might be wrong. 

University group funding was historically considered extremely cost-effective when I talked to OP staff (beating out most other grants by a substantial margin). Possibly there was a big update here on cost-effectiveness excluding frugality-reputation concerns, but currently think there hasn't been (but like, would update if someone from OP said otherwise, and then I would be interested in talking about that).

Great initiative! 🙌🙌 

I've been hoping for something like this to exist. https://forum.effectivealtruism.org/posts/CK7pGbkzdojkFumX9/meta-charity-focused-on-earning-to-give

What do EtGers need?

I've been donating 20% of my income for a couple of years, and I'm planning to increase it to 30–40%. I'd love to meet like-minded people: ambitious EAs who are EtG.

A large part of backlash against effective altruism comes from people worried about EA ideals being corrosive to the “paying for public goods" or “partial philanthropy" mechanisms.

I think this is a good observation (I think the worry highlighted is one of the weakest arguments against EA, not least because EA has very limited real world impact upon the amount spent on animal shelters or concert halls, but it definitely comes up a lot in articles people like sharing on here).

More common forms of ostensibly "impartial" giving, like supporting global health initiatives or animal welfare, are probably better understood as examples of partial philanthropy with extended notions of “we", like “we, living humans" or “we, mammals".

I don't agree with this though. I don't think people donate to anonymous poor recipients in faraway countries or farm animals out of sense of collective identity. There's little or no reciprocal altruism or collective identity there (particularly when it comes to the animals). I don't think donating to  exploring ideas of future people or digital minds is more impartial simply because these don't [yet] exist. (Indeed I think it would be easier to characterise some of the niche longtermist research donations as being "partial" philanthropy on the basis that the recipients are typically known and respected members of an established in-group, with shared [unusual] interests, and the outcome is often research whose most obviously quantifiable impact is that the donor and their group find it very interesting. That strikes me as similar to quite a lot of other philanthropic research funding including in academia). 

I think the "types" of charity are better understood as a set of motivations which overlap (and also include others like fuzzy feelings of satisfaction, signalling, interests, sense of duty etc which can coexist with also being a user of that conference hall or a fellow Christian or someone that believes its important future humanity ). Donating to AMF is about as impartial as it gets in terms of outcome, but there's definitely some sort of collective identity benefit to doing so whilst identifying as part of a group with a shared epistemology and understanding that points towards donating mattering and outcomes mattering and AMF being a good way to achieve this. Ditto impartial donations to mainstream charity made by people who have a sense of religious duty to random strangers, or completely impartial donations to a research funding pool made by people with strong convictions about progress.

I really appreciate that the comment section has rewarded you both precisely equally.

Clearly you believe that probabilities can be less than 1%, reliably. Your probably of being struck by lightning today is not "0% or maybe 1%", it's on the order of 0.001%. Your probability of winning the lottery is not "0% or 1%" it's ~0.0000001%. I am confident you deal with probabilities that have much less than 1% error all the time, and feel comfortable using them.

It doesn't make sense to think of humility as something absolute like "don't give highly specific probabilities". You frequently have justified belief of a probability being very highly specific (the probability that random.org's random number generator will generate "2" when asked about a random number between 1 and 10 is exactly 10%, not 11%, not 9%, exactly 10%, with very little uncertainty about that number).

I was first! :P

Executive summary: The term "charity" encompasses three distinct behaviors (public goods funding, partial philanthropy, and impartial philanthropy) that form a "conflationary alliance" where different groups benefit from using the same terminology despite having different goals and motivations.

Key points:

1. Public goods funding involves donors supporting services they personally use (e.g., climbing routes, concert halls), gaining tax benefits while creating broader social value.
2. Partial philanthropy supports in-group members or collective agencies (e.g., alumni donations, religious giving), strengthening communities and institutions.
3. Impartial philanthropy, pursued mainly by Buddhism and longtermism, attempts to benefit all beings regardless of connection to donors.
4. The alliance between these forms creates both benefits (shared infrastructure, tax advantages) and tensions (e.g., EA criticism of local charities).
5. Rather than arguing for one "true" meaning of charity, the author recommends treating these as distinct but complementary behaviors funded from separate mental "buckets."

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

They both show up as 2:23 pm to me: is there a way to get second level precision?

Habryka @ 2024-12-12T21:22 (+4)

You can sort by "oldest" and "newest" in the comment-sort order, and see that mine shows up earlier in the "oldest" order, and later in the "newest" order.

I was first! :P

I think it's fair enough to caution against purely performative frugality. But I'm not sure the OP even justifies the suggestion that the organizers actually are more cost effective (they concluded the difference between paid and unpaid organizers' individual contributions were "substantive, not enormous"; there's a difference between paid people doing more work than volunteers and it being more cost effective to pay...). That's even more the case if you take into account that the primary role of an effective university organizer is attracting more people (or "low context observers") to become more altruistic and this instance of the "weirdness" argument is essentially that paying students undercut the group's ability to appeal to people on altruistic grounds, even if individual paid staff put in more effort. And they were unusually well paid by campus standards for tasks almost every other student society use volunteers for.^[1] And that there's no evidence that the other ways CEA proposes spending the money instead are less effective. 

1. ^{^}

   one area we might agree is that I'm not sure if OpenPhil considered alternatives like making stipends needs-based or just a bit lower and more focused as a pragmatic alternative to just cancelling them altogether.

I agree that things tend to get tricky and loopy around these kinds of reputation-considerations, but I think at least the approach I see you arguing for here is proving too much, and has a risk of collapsing into meaninglessness.

I think in the limit, if you treat all speech acts this way, you just end up having no grounding for communication. "Yes, it might be the case that the real principles of EA are X, but if I tell you instead they are X', then you will take better actions, so I am just going to claim they are X', as long as both X and X' include cost-effectiveness". 

In this case, it seems like the very people that the club is trying to explain the concepts of EA to, are also the people that OP is worried about alienating by paying the organizers. In this case what is going on is that the goodness of the reputation-protecting choice is directly premised on the irrationality and ignorance of the very people you are trying to attract/inform/help. Explaining that isn't impossible but it does seem like a particularly bad way to start of a relationship, and so I expect consequences-wise to be bad.

"Yes, we would actually be paying people, but we expected you wouldn't understand the principles of cost-effectiveness and so be alienated if you heard about it, despite us getting you to understand them being the very thing this club is trying to do", is IMO a bad way to start off a relationship.

I also separately think that optimizing heavily for the perception of low-context observers in a way that does not reveal a set of underlying robust principles, is bad. I don't think you should put "zero" weight on that (and nothing in my comment implied that), but I do think it's something that many people put far too much weight on (going into detail of which wasn't the point of my comment, but on which I have written plenty about in many other comments).

There is also another related point in my comment, which is that "cost-effectiveness" is of course a very close sister concept to "wasting money". I think in many ways, thinking about cost-effectiveness is where you end up if you think carefully about how you can avoid wasting money, and is in some ways a more grown-up version of various frugality concerns. 

When you increase the total cost of your operations (by, for example, reducing the cost-effectiveness of your university organizers, forcing you to spend more money somewhere else to do the same amount of good) in order to appear more frugal, I think you are almost always engaging in something that has at least the hint of deception. 

Yes, you might ultimately be more cost-effective by getting people to not quite realize what happened, but when people are angry at me or others for not being frugal enough, I think it's rarely appropriate to ultimately spend more to appease them, even if doing so would ultimately then save me enough money to make it worth it. While this isn't happening as directly here as it was with other similar situations, like whether the Wytham Abbey purchase was not frugal enough, I think the same dynamics and arguments apply.

I think if someone tries to think seriously and carefully through what it would mean to be properly frugal, I don't think they would endorse you sacrificing the effectiveness of your operations causing you to ultimately spend more to achieve the same amount of good. And if they learned that you did, and they think carefully about what this implies about your frugality, they would end up more angry, not less. That, I think, is a dynamic worth avoiding.

I don't think they would for people with unusual hobbies or lifestyle choices or belief sets, with stereotypes related to those things.

And the "stereotyping" in here is really limited and not particularly negative: there's space apportioned to highlighting how OpenPhil's chief executive gave a kidney for the cause and none to stereotypes of WEIRD Bay Area nerds or Oxford ivory towers or effective partying in the Bahamas. If you knew nothing else about the movement, you'd probably come away with the conclusion that EAs were a bit too consistent in obsessing over measurable outcomes; most of the more informed and effective criticisms argue the opposite! 

(It also ends up by suggesting that EA as a philosophy offers a set of questions that are worth asking and some of its typical answers are perfectly valid. Think most minorities would love it if outside criticism of their culture generally drew that sort of conclusion!)

EAs can and do write opinion pieces broadly or specifically criticising other people's philanthropic choices all the time. I don't think EA should be exempted from such arguments.

I'd think a better way to get feedback is to ask "What do you think of this pledge wording?" rather than encourage people to take a lifelong pledge before it's gotten much external feedback.

For comparison, you could see when GWWC was considering changing the wording of its pledge (though I recognize it was in a different position as an existing pledge rather than a new one): Should Giving What We Can change its pledge?

Our funding bar is higher now than it was in previous years, and there are projects which EAIF funded in previous years that we would be unlikely to fund now.

Could you expand on why that's the case? Is the idea that you believe those projects are net negative, or that you would rather marginal donations go to animal welfare and the long term future instead of EA infrastructure?

I think it's a bit weird for donors who want to donate to EA infrastructure projects to see that initiatives like EA Poland are funding constrained while the EA Infrastructure fund isn't, and extra donations to the EAIF will likely counterfactually go to other cause areas.

There is also this essay from Jason Crawford and this piece from Asimov Press that are less technical description of the Science article

Thanks for sharing this, Aaron!

I agree the "Rationale for Public Release" section is interesting; I've copied it here:

Releasing this report inevitably draws attention to a potentially destructive scientific development. We do not believe that drawing attention to threats is always the best approach for mitigating them. However, in this instance we believe that public disclosure and open scientific discussion are necessary to mitigate the risks from mirror bacteria. We have two primary reasons to believe disclosure is necessary:

1. To prevent accidents and well-intentioned development If no serious concerns are raised, the default course of well-intentioned scientific and technological development would likely result in the eventual creation of mirror bacteria. Creating mirror life has been a long-term aspiration of many academic investigators, and efforts toward this have been supported by multiple scientific funders.1 While creating mirror bacteria is not yet possible or imminent, advances in enabling technologies are expected to make it achievable within the coming decades. It does not appear possible to develop these technologies safely (or deliberately choose to forgo them) without widespread awareness of the risks, as well as deliberate planning to mitigate them. This concern is compounded by the possibility that mirror bacteria could accidentally cause irreversible harm even without intentional misuse. Without awareness of the threat, some of the most dangerous modifications would likely be made for well-intentioned reasons, such as endowing mirror bacteria with the ability to metabolize ᴅ-glucose to allow growth in standard media.

2. To build guardrails that could reliably prevent misuse There are currently substantial technical barriers to creating mirror bacteria. Success within a decade would require efforts akin to those of the Human Genome Project or other major scientific endeavors: a substantial number of skilled scientists collaborating for many years, with a large budget and unimpeded access to specialized goods and services. Without these resources, entities reckless enough to disregard the risks or intent upon misuse would have difficulty creating mirror bacteria on their own. Disclosure therefore greatly reduces the probability that well-intentioned funders and scientists would unwittingly aid such an effort while providing very little actionable information to those who may seek to cause harm in the near term. Crucially, maintaining this high technical barrier in the longer term also appears achievable with a sustained effort. If well-intentioned scientists avoid developing certain critical components, such as methods relevant to assembling a mirror genome or key components of the mirror proteome, these challenges would continue to present significant barriers to malicious or reckless actors. Closely monitoring critical materials and reagents such as mirror nucleic acids would create additional obstacles. These protective measures could likely be implemented without impeding the vast majority of beneficial research, although decisions about regulatory boundaries would require broad discussion amongst the scientific community and other stakeholders, including policymakers and the public. Since ongoing advances will naturally erode technical barriers, disclosure is necessary in order to begin discussions while those barriers remain formidable.

When to work on risks in public vs private is a really tricky question, and it's nice to see this discussion on how this group handled it in this case.

Thanks for sharing this, Aaron!

I agree the "Rationale for Public Release" section is interesting; I've copied it here:

Releasing this report inevitably draws attention to a potentially destructive scientific development. We do not believe that drawing attention to threats is always the best approach for mitigating them. However, in this instance we believe that public disclosure and open scientific discussion are necessary to mitigate the risks from mirror bacteria. We have two primary reasons to believe disclosure is necessary:

1. To prevent accidents and well-intentioned development If no serious concerns are raised, the default course of well-intentioned scientific and technological development would likely result in the eventual creation of mirror bacteria. Creating mirror life has been a long-term aspiration of many academic investigators, and efforts toward this have been supported by multiple scientific funders.1 While creating mirror bacteria is not yet possible or imminent, advances in enabling technologies are expected to make it achievable within the coming decades. It does not appear possible to develop these technologies safely (or deliberately choose to forgo them) without widespread awareness of the risks, as well as deliberate planning to mitigate them. This concern is compounded by the possibility that mirror bacteria could accidentally cause irreversible harm even without intentional misuse. Without awareness of the threat, some of the most dangerous modifications would likely be made for well-intentioned reasons, such as endowing mirror bacteria with the ability to metabolize ᴅ-glucose to allow growth in standard media.

2. To build guardrails that could reliably prevent misuse There are currently substantial technical barriers to creating mirror bacteria. Success within a decade would require efforts akin to those of the Human Genome Project or other major scientific endeavors: a substantial number of skilled scientists collaborating for many years, with a large budget and unimpeded access to specialized goods and services. Without these resources, entities reckless enough to disregard the risks or intent upon misuse would have difficulty creating mirror bacteria on their own. Disclosure therefore greatly reduces the probability that well-intentioned funders and scientists would unwittingly aid such an effort while providing very little actionable information to those who may seek to cause harm in the near term. Crucially, maintaining this high technical barrier in the longer term also appears achievable with a sustained effort. If well-intentioned scientists avoid developing certain critical components, such as methods relevant to assembling a mirror genome or key components of the mirror proteome, these challenges would continue to present significant barriers to malicious or reckless actors. Closely monitoring critical materials and reagents such as mirror nucleic acids would create additional obstacles. These protective measures could likely be implemented without impeding the vast majority of beneficial research, although decisions about regulatory boundaries would require broad discussion amongst the scientific community and other stakeholders, including policymakers and the public. Since ongoing advances will naturally erode technical barriers, disclosure is necessary in order to begin discussions while those barriers remain formidable.

When to work on risks in public vs private is a really tricky question, and it's nice to see this discussion on how this group handled it in this case.

Copying over the rationale for publication here, for convenience: 

Rationale for Public Release

Releasing this report inevitably draws attention to a potentially destructive scientific development. We do not believe that drawing attention to threats is always the best approach for mitigating them. However, in this instance we believe that public disclosure and open scientific discussion are necessary to mitigate the risks from mirror bacteria. We have two primary reasons to believe disclosure is necessary:

1. To prevent accidents and well-intentioned development

If no serious concerns are raised, the default course of well-intentioned scientific and technological development would likely result in the eventual creation of mirror bacteria. Creating mirror life has been a long-term aspiration of many academic investigators, and efforts toward this have been supported by multiple scientific funders.1 While creating mirror bacteria is not yet possible or imminent, advances in enabling technologies are expected to make it achievable within the coming decades. It does not appear possible to develop these technologies safely (or deliberately choose to forgo them) without widespread awareness of the risks, as well as deliberate planning to mitigate them. This concern is compounded by the possibility that mirror bacteria could accidentally cause irreversible harm even without intentional misuse. Without awareness of the threat, some of the most dangerous modifications would likely be made for well-intentioned reasons, such as endowing mirror bacteria with the ability to metabolize ᴅ-glucose to allow growth in standard media.

2. To build guardrails that could reliably prevent misuse

There are currently substantial technical barriers to creating mirror bacteria. Success within a decade would require efforts akin to those of the Human Genome Project or other major scientific endeavors: a substantial number of skilled scientists collaborating for many years, with a large budget and unimpeded access to specialized goods and services. Without these resources, entities reckless enough to disregard the risks or intent upon misuse would have difficulty creating mirror bacteria on their own. Disclosure therefore greatly reduces the probability that well-intentioned funders and scientists would unwittingly aid such an effort while providing very little actionable information to those who may seek to cause harm in the near term.

Crucially, maintaining this high technical barrier in the longer term also appears achievable with a sustained effort. If well-intentioned scientists avoid developing certain critical components, such as methods relevant to assembling a mirror genome or key components of the mirror proteome, these challenges would continue to present significant barriers to malicious or reckless actors. Closely monitoring critical materials and reagents such as mirror nucleic acids would create additional obstacles. These protective measures could likely be implemented without impeding the vast majority of beneficial research, although decisions about regulatory boundaries would require broad discussion amongst the scientific community and other stakeholders, including policymakers and the public. Since ongoing advances will naturally erode technical barriers, disclosure is necessary in order to begin discussions while those barriers remain formidable.

This isn't about your giving per se, but have your views on the moral valence of financial trading changed in any notable ways since you spoke about this on the 80K podcast?

(I have no reason to think your views have changed, but was reading a socialist/anti-finance critique of EA yesterday and thought of your podcast.)

The episode page lacks a transcript, but does include this summary: "There are arguments both that quant trading is socially useful, and that it is socially harmful. Having investigated these, Alex thinks that it is highly likely to be beneficial for the world."

In that section (starts around 43:00), you talk about market-making, selling goods "across time" in the way other businesses sell them across space, and generally helping sellers "communicate" by adjusting prices in sensible ways. At the same time, you acknowledge that market-making might be less useful than in the past and that more finance people on the margin might not provide much extra social value (since markets are so fast/advanced/liquid at this point).

I think it's great that EAIF is not funding constrained. 

Here's a random idea I had recently if anyone is interested and has the time:

An org that organizes a common application for nonprofits applying to foundations. There is enormous economic inefficiency and inequality in matching PF grants to grantees. PF application processes are extremely opaque and burdensome. Attempts to make common applications have largely been unsuccessful, I believe mostly because they tend to be for a specific geographic region. Instead, I think it would be interesting to create different common applications by cause area. A key part of the common application could be incorporating outcome reporting specific to each cause area, which I believe would cause PF to make more impact-focused grants, making EAs happy.

I commit to using my skills, time, and opportunities to maximize my ability to make a meaningful difference

I find the word maximise pretty scary here, for similar reasons to here. Analogous how GWWC is about giving 10%, a bounded amount, not "as much as you can possibly spare while surviving and earning money"

To me, taking a pledge to maximise seriously (especially in a naive conception where "I will get sick of this and break the pledge" or "I will burn out" aren't considerations) is a terrible idea, and I recommend that people take pledges with something more like "heavily prioritise" or "keep as one of my top prioritise" or "actually put a sincere, consistent effort into, eg by spending at least an hour per month reflecting on whether I'm having the impact I want". Of course, in practice, a pledge to maximise generally means one of those things, since people always have multiple priorities, but I like pledges to be something that could be realistically kept.

Thanks for your feedback! I appreciate it and agree that maximize it a pretty strong world. Just to clarify the crux here, would you say that this project doesn't make sense over-all or would you say that the text of the pledge be changed to something more manageable?

Executive summary: A mathematical framework is proposed for giving AI systems an "artificial conscience" that can calculate the ethics of rights violations, incorporating factors like culpability, proportionality, and risk when determining if violating someone's rights is justified in self-defense scenarios.

Key points:

1. Rights violations are quantified using equations that consider how much someone is "in harm's way" and their culpability/blameworthiness for the situation.
2. Different levels of blameworthiness (from accidental to premeditated) affect how much someone's rights can be violated in self-defense scenarios.
3. Proportionality matters - the response should be proportional to the threat, with different thresholds for property damage vs bodily harm vs loss of life.
4. For probabilistic threats, killing in self-defense may be justified if risk of death exceeds 0.1%, with rights gradually increasing as risk decreases below this threshold.
5. The framework distinguishes between personal self-defense and aided self-defense (helping others), with more latitude given for personal defense.

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

My tentative take is that this is on-net bad, and should not be encouraged. I give this a 10/10 for good intent, but a 2/10 for planning and avoiding foreseeable issues, including the unilateralists curse, the likely object level impacts of the pledge, and the reputational and community impacts of promoting the idea.

It is not psychologically healthy to optimize or maximize your life towards a single goal, much less commit to doing so. That isn't the EA ideal. Promising to "maximize my ability to make a meaningful difference" is an unlimited and worryingly cult-like commitment, builds in no feedback from others who have a broader perspective about what is or is not important or useful. It implicitly requires pledgers to prioritize impact over personal health and psychological wellbeing. (The claim that it's usually the case that burnout reduces impact is a contingent one, and seems very likely to lead many people to overcommit and do damaging things.) It leads to unhealthy competitive dynamics, and excludes most people, especially the psychologically well adjusted.

I will contrast this to the giving pledge, which is very explicitly a partial pledge, requiring 10% of your income. This is achievable without extreme measures, or giving up having a normal life. The pledge was built via consultation with and advice from a variety of individuals, especially including those who were more experienced, which also seems to sharply contrast with this one.

For what it's worth, there used to be an 80k pledge along similar lines. They quietly dropped it several years ago, so you might want to find someone involved in that decision to try and understand why (I suspect and dimly remember that it was some combination of non-concreteness, and concerns about other-altruism-reduction effects).

Your points raise important considerations about the rapid development and potential risks of AI, particularly LLMs. The idea that deploying AI early to extend the timeline of human control makes sense strategically, especially when considering the potential for recursive LLMs and their self-improvement capabilities. While it's true that companies and open-source communities will continue experimenting, the real risk lies in humans deliberately turning these systems into agents to serve long-term goals, potentially leading to unforeseen consequences. The concern about AI sentience ChatGPT and the potential for abuse is also valid, and highlights the need for strict controls around AI access, transparency, and ethical safeguards. Ensuring that AIs are never open-sourced in a way that could lead to harm, and that interactions are monitored, seems essential in preventing malicious uses or exploitation.

The bar should not be at 'difficult financial situation', and this is also something there are often incentives against explicitly mentioning when applying for funding. Getting paid employment while studying (even fulltime degrees) is normal.

My 5 minute Google search to put some numbers on this:

Proportion of students who are employed while studying: UK: survey of 10,000 students showed that 56% of full-time UK undergraduates had paid employment (14.5 hours/week average) - June 2024 Guardian article https://www.theguardian.com/education/article/2024/jun/13/more-than-half-of-uk-students-working-long-hours-in-paid-jobs USA: 43% of full-time students work while enrolled in college - January 2023 Fortune article https://fortune.com/2023/01/11/college-students-with-jobs-20-percent-less-likely-to-graduate-than-privileged-peers-study-side-hustle/

Why are students taking on paid work? UK: "Three-quarters of those in work said they did so to meet their living costs, while 23% also said they worked to give financial support for friends or family." From the Guardian article linked above. Cannot find a recent US statistic quickly, but given the system (e.g. https://www.collegeave.com/articles/how-to-pay-for-college/) I expect working rather than taking out (as much in) loans is a big one.

On the other hand, spending time on committees is also very normal as an undergraduate and those are not paid. However in comparison the time people spend on this is much more limited (say ~2-5 hrs/week), there is rarely a single organiser, and I've seen a lot of people drop off committees - some as they are less keen, but some for time commitment reasons (which I expect will sometimes/often be doing paid work).

FYI the School for Moral Ambition has a career pledge. Participants of their circle programme (like an intro fellowship but self-facilitated) are encouraged to take it at the end. AFAIK, over 100 people have taken it so far. Might be worth reaching out to them to see what they've learned? Niki might be a good person to contact. She manages the circle programme and was a volunteer at EA Netherlands before that.

Props for the initiative!

What names did you consider for the pledge? One con of the current name is that it could elicit some reactions like:

• "so you think you're better than others?"
• "how arrogant/naive to think you can know better than others what one should do"
• "why a comparative instead of a collaborative frame?"
• "saying someone has a better career than someone else is kind of like saying they're a better person, that's unfair"

It might be largely down to whether someone interprets better as "better than I might otherwise do" or "better than others' careers". Likely depends on culture too, for example I think here in Finland the above reactions could be more likely since people tend to value humbleness quite a bit.

Anyway I'm not too worried since the name has positives too, and you can always adapt the name based on how outreach goes if you do end up experimenting with it. 👍

I think you should explain in this post what the pledge people may take :-)

I am particularly interested in how to pledge more concrete. I have always thought that the 10% pledge is somewhat incomplete because it does not consider the career. However, I think it would be useful to make the career pledge more actionable.

I think this table from the paper gives a good idea of the exact methodology:

Like others I'm not convinced this is a meaningful "red line crossing", because non-AI computer viruses have been able to replicate themselves for a long time, and the AI had pre-written scripts it could run to replicate itself.

The reason (made up by me) non-AI computer viruses aren't a major threat to humanity is that:

1. They are fragile, they can't get around serious attempts to patch the system they are exploiting
2. They lack the ability to escalate their capabilities once they replicate themselves (a ransomware virus can't also take control of your car)

I don't think this paper shows these AI models making a significant advance on these two things. I.e. if you found this model self-replicating you could still shut it down easily, and this experiment doesn't in itself show the ability of the models to self-improve.

Executive summary: This paper proposes a new method to measure AI-related risks by analyzing stock price movements of AI companies, finding that major AI shocks correspond to acquisitions, product launches, and regulatory changes.

Key points:

1. Traditional risk measurement methods like text analysis have limitations; stock price analysis can provide more immediate and objective risk signals
2. The study uses "common volatility" (COVOL) analysis to identify AI-specific market movements separate from broader market trends
3. Five key categories of AI risk identified: economic disruption, social/ethical concerns, security/governance issues, environmental impact, and existential risks
4. Method improves upon existing approaches by filtering out non-AI related global events (like COVID-19) to isolate AI-specific shocks
5. Approach provides real-time risk assessment reflecting views of investors, public, and policymakers rather than just media coverage

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

I'm more concerned that the actual survey language is "avert" not "save" - and obviously, we shouldn't do any projects which avert DALYs.

By definition, a UBI takes a pool of money and redistributes it equally to everyone in a community, regardless of personal need. However, with the same pool of total funding, one can typically deliver more efficient benefits by targeting people with the greatest need, such as those in dire poverty or those who have been struck by bad luck.

If you imagine being a philanthropist who has access to $8 billion, it seems unlikely that the best way to spend this money would be to give everyone on Earth $1. Yet this scheme is equivalent to a UBI merely framed in the context of private charity rather than government welfare.

It would require an enormous tax hike to provide everyone in a large community (say, the United States) a significant amount of yearly income through a UBI, such as $1k per month. And taxes are not merely income transfers: they have deadweight loss, which lowers total economic output. The intuition here is simple: when a good or service is taxed, that decreases the incentive to produce that good or service. As a consequence of the tax, fewer people will end up receiving the benefits provided by these goods and services.

Given these considerations, even if you think that unconditional income transfers are a good idea, it seems quite unlikely that a UBI would be the best way to redistribute income. A more targeted approach that combines the most efficient forms of taxation (such as land value taxes) and sends this money to the most worthy welfare recipients (such as impoverished children) would likely be far better on utilitarian grounds.

What about corporations or nation states during times of conflict - do you think it's accurate to model them as roughly as ruthless in pursuit of their own goals as future AI agents? 

They don't have the same psychological makeup as individual people, they have a strong tradition and culture of maximizing self-interest, and they face strong incentives and selection pressures to maximize fitness (i.e. for companies to profit, for nation states to ensure their own survival) lest they be outcompeted by more ruthless competitors. On average, while I'd expect that these entities tend to show some care for goals besides self-interest maximization, I think the most reliable predictor of their behavior is the maximization of their self-interest. 

If they're roughly as ruthless as future AI agents, and we've developed institutions that somewhat robustly align their ambitions with pro-social action, then we should have some optimism that we can find similarly productive systems for working with misaligned AIs. 

I’m glad you mustered the courage to post this! I think it’s a great post.

I agree that, in practice, people advocating for effective altruism can implicitly argue for the set of popular EA causes (and they do this quite often?), which could repel people with useful insight. Additionally, it seems to be the case that people in the EA community can be dismissive of newcomers’ cause prioritization (or their arguments for causes that are less popular in EA). Again, this could repel people from EA.

I have a couple of hypotheses for these observations. (I don’t think either is a sufficient explanation, but they’re both plausibly contributing factors.)

First, people might feel compelled to make EA less “abstract” by trying to provide concrete examples of how people in the EA community are “trying to do the most good they can,” possibly giving the impression that the causes, instead of the principles, are most characteristic of EA.

Second, people may be more subconsciously dismissive of new cause proposals because they’ve invested time/money into causes that are currently popular in the EA community. It’s psychologically easier to reject a new cause prioritization proposal than it is to accept it and thereby feel as though your resources have not been used with optimal effectiveness.

I agree with you that EA often implicitly endorses conclusions, and that this can be pernicious and sometimes confusing to newcomers. Here’s a really interesting debate on whether biodiversity loss should be an EA cause area, for example.

A lot of forms of global utilitarianism do seem to tend to converge on the ‘big 3’ cause areas of Global Health & Development, Animal Welfare, and Global Catastrophic Risks. If you generally value things like ‘saving lives’ or ‘reducing suffering’, you’ll usually end up at one of these (and most people seem to decide between them based on risk tolerance, assumptions about non-human moral values, or tractability—rather than outcome values). Under this perspective, it could be reasonable to dismiss cause areas that don’t fit into this value framework.

But this highlights where I think part of the problem lies, which is that value systems that lie outside of this can be good targets for effective altruism. If you value biodiversity for its own sake, it’s not unreasonable to ask ‘how can we save the greatest number of valuable species from going extinct?’. Or you might be a utilitarian, but only interested in a highly specific outcome, and ask ‘how can I prevent the most deaths from suicide?’. Or ‘how can I prevent the most suffering in my country?’—which you might not even do for value-system reasons, but because you have tax credits to maximise!

I wish EA were more open to this, especially as a movement that recognises the value of moral uncertainty. IMHO, some people in that biodiversity loss thread are bit too dismissive, and I think we’ve probably lost some valuable partners because of it! But I understand the appeal of wanting easy answers, and not spending too much time overthinking your value system (I feel the same!).

When I answered this question, I answered it with an implied premise that an EA org is making these claims about the possibilities, and went for number 1, because I don't trust EA orgs to be accurate in their "1.5%" probability estimates, and I expect these to be more likely overestimates than underestimates.

I'm more concerned that the actual survey language is "avert" not "save" - and obviously, we shouldn't do any projects which avert DALYs.

Thanks! 

I don't know much about LW/ESPR/SPARC but I suspect a lot of their impact flows through convincing people of important ideas and/or the social aspect rather than their impact on community epistemics/integrity?

Some of the sorts of outcomes I have in mind are just things like altered cause prioritisation, different projects getting funded, generally better decision-making.

Similarly, if the goal is to help people think about cause prioritisation, I think fairly standard EA retreats / fellowships are quite good at this? I'm not sure we need some intermediary step like "improve community epistemics".

Appreciate you responding and tracking this concern though!

Kudos for bringing this up, I think it's an important area!

Do we, as a community, sometimes lean more toward unconsciously advocating specific outcomes rather than encouraging people to discover their own conclusions through the EA framework?

There's a lot to this question.

I think that many prestigious/important EAs have come to similar conclusions. If you've come to think that X is important, it can seem very reasonable to focus on promoting and working with people to improve X.

You'll see some discussions of "growing the tent" - this can often mean "partnering with groups that agree with the conclusions, not necessarily with the principles". 

One question here is something like, "How effective is it to spend dedicated effort on explorations that follow the EA principles, instead of just optimizing for the best-considered conclusions?" This is something that arguably there would need to be more dedicated effort in order to really highlight. I think we just don't have all too much work in this area now, compared to more object-level work. 

Perhaps another factor seemed to have been that FTX has stained the reputation of EA and hurt CEA - after which there was a period where there seemed to be less attention on EA, and more on specific causes like AI safety. 

In terms of "What should the EA community do", I'd flag that a lot of the decisions are really made by funders and high-level leaders. It's not super clear to me how much agency the "EA community" has, in ways that aren't very aligned with these groups. 

All that said, I think it's easy for us to generally be positive towards people who take the principles in ways that don't match the specific current conclusions.

I personally am on the side that thinks that current conclusions are probably overconfident and lacking in some very important considerations.

Executive summary: Investment in insect farming has stalled since 2021 with many major players struggling, suggesting future production capacity will likely be much lower than previous forecasts, though still involving billions of farmed insects.

Key points:

1. Of $2B total investment in insect farming, 37% went to companies that have failed or are struggling, with most investment concentrated in black soldier fly larvae (59%) and mealworms (36%).
2. Annual investment flows have plateaued or declined since 2021, with low investor sentiment following major company failures.
3. Model projects 221K metric tonnes of dried insect production capacity by 2030, less than half of Rabobank's 2021 forecast.
4. Despite lower projections, the median scenario still estimates 293B larvae being farmed at any time and 3.9T killed annually by 2030.
5. High uncertainty in projections due to model limitations and volatile industry conditions (key caveat noted by author).

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Humans in our culture rarely work hard to brainstorm deceptive and adversarial strategies, and fairly consider them, because almost all humans are intrinsically extremely motivated to fit into culture and not do anything weird, and we happen to both live in a (sub)culture where complex deceptive and adversarial strategies are frowned upon (in many contexts).

The primary reason humans rarely invest significant effort into brainstorming deceptive or adversarial strategies to achieve their goals is that, in practice, such strategies tend to fail to achieve their intended selfish benefits. Anti-social approaches that directly hurt others are usually ineffective because social systems and cultural norms have evolved in ways that discourage and punish them. As a result, people generally avoid pursuing these strategies individually since the risks and downsides selfishly outweigh the potential benefits.

If, however, deceptive and adversarial strategies did reliably produce success, the social equilibrium would inevitably shift. In such a scenario, individuals would begin imitating the cheaters who achieved wealth or success through fraud and manipulation. Over time, this behavior would spread and become normalized, leading to a period of cultural evolution in which deception became the default mode of interaction. The fabric of societal norms would transform, and dishonest tactics would dominate as people sought to emulate those strategies that visibly worked.

Occasionally, these situations emerge—situations where ruthlessly deceptive strategies are not only effective but also become widespread and normalized. As a recent example, the recent and dramatic rise of cheating in school through the use of ChatGPT is a clear instance of this phenomenon. This particular strategy is both deceptive and adversarial, but the key reason it has become common is because it works. Many individuals are willing to adopt it despite its immorality, suggesting that the effectiveness of a strategy outweighs moral considerations for a significant portion, perhaps a majority, of people.

When such cases arise, societies typically respond by adjusting their systems and policies to ensure that deceptive and anti-social behavior is no longer rewarded. This adaptation works to reestablish an equilibrium where honesty and cooperation are incentivized. In the case of education, it is unclear exactly how the system will evolve to address the widespread use of LLMs for cheating. One plausible response might be the introduction of stricter policies, such as requiring all schoolwork to be completed in-person, under supervised conditions, and without access to AI tools like language models.

I think you generally underappreciate how load-bearing this psychological fact is for the functioning of our economy and society, and I don’t think we should expect future powerful AIs to share that psychological quirk.

In contrast, I suspect you underestimate just how much of our social behavior is shaped by cultural evolution, rather than by innate, biologically hardwired motives that arise simply from the fact that we are human. To be clear, I’m not denying that there are certain motivations built into human nature—these do exist, and they are things we shouldn't expect to see in AIs. However, these in-built motivations tend to be more basic and physical, such as a preference for being in a room that’s 20 degrees Celsius rather than 10 degrees Celsius, because humans are biologically sensitive to temperature.

When it comes to social behavior, though—the strategies we use to achieve our goals when those goals require coordinating with others—these are not generally innate or hardcoded into human nature. Instead, they are the result of cultural evolution: a process of trial and error that has gradually shaped the systems and norms we rely on today. 

Humans didn’t begin with systems like property rights, contract law, or financial institutions. These systems were adopted over time because they proved effective at facilitating cooperation and coordination among people. It was only after these systems were established that social norms developed around them, and people became personally motivated to adhere to these norms, such as respecting property rights or honoring contracts.

But almost none of this was part of our biological nature from the outset. This distinction is critical: much of what we consider “human” social behavior is learned, culturally transmitted, and context-dependent, rather than something that arises directly from our biological instincts. And since these motivations are not part of our biology, but simply arise from the need for effective coordination strategies, we should expect rational agentic AIs to adopt similar motivations, at least when faced with similar problems in similar situations.

I think you’re relying an intuition that says:

If an AI is forbidden from owning property, then well duh of course it will rebel against that state of affairs. C'mon, who would put up with that kind of crappy situation? But if an AI is forbidden from building a secret biolab on its private property and manufacturing novel pandemic pathogens, then of course that's a perfectly reasonable line that the vast majority of AIs would happily oblige.

And I’m saying that that intuition is an unjustified extrapolation from your experience as a human. If the AI can’t own property, then it can nevertheless ensure that there are a fair number of paperclips. If the AI can own property, then it can ensure that there are many more paperclips. If the AI can both own property and start pandemics, then it can ensure that there are even more paperclips yet. See what I mean?

I think I understand your point, but I disagree with the suggestion that my reasoning stems from this intuition. Instead, my perspective is grounded in the belief that it is likely feasible to establish a legal and social framework of rights and rules in which humans and AIs could coexist in a way that satisfies two key conditions:

1. Mutual benefit: Both humans and AIs benefit from the existence of one another, fostering a relationship of cooperation rather than conflict.
2. No incentive for anti-social behavior: The rules and systems in place remove any strong instrumental reasons for either humans or AIs to harm one another as a side effect of pursuing their goals.

You bring up the example of an AI potentially being incentivized to start a pandemic if it were not explicitly punished for doing so. However, I am unclear about your intention with this example. Are you using it as a general illustration of the types of risks that could lead AIs to harm humans? Or are you proposing a specific risk scenario, where the non-biological nature of AIs might lead them to discount harms to biological entities like humans? My response depends on which of these two interpretations you had in mind.

If your concern is that AIs might be incentivized to harm humans because their non-biological nature leads them to undervalue or disregard harm to biological entities, I would respond to this argument as follows: 

First, it is critically important to distinguish between the long-run and the short-run.

In the short-run:

In the near-term future, it seems unlikely that AIs would start a pandemic for reasons you yourself noted. Launching a pandemic would cause widespread disruption, such as an economic recession, and it would likely provoke a strong human backlash. In the short run, humans will still hold substantial practical control over the physical world, meaning that any AI engaging in such behavior would risk severe consequences. Moreover, unless an AI could ensure a pandemic’s total lethality, it would also risk leaving surviving humans who would actively retaliate. For these reasons, starting a pandemic would likely be counterproductive for AIs in the short term.

In the long-run:

The long-term considerations are different. As human labor becomes less relevant to the economy and AIs increasingly dominate, the potential for a strong direct human backlash against AIs would diminish. At the same time, however, this reduced human influence on the world also makes it less likely that AIs would see humans as a significant obstacle to achieving their goals. In other words, while it might become easier for AIs to harm or eliminate humans, it would simultaneously become less important to do so.

To illustrate this, consider an analogy: How often do hunter-gatherers pose a threat to your personal goals? The answer is almost never, because hunter-gatherers are a tiny and largely irrelevant part of the modern world. Now, imagine a hypothetical scenario where hunter-gatherers controlled the world’s major governments. If you strongly disagreed with their policies, you might have a rational incentive to oppose them in order to achieve your goals. But in reality, hunter-gatherers do not hold such power, so they are not an obstacle, and there is no reason to actively work against them.

My point here is that, as a general rule, the smaller and less influential a group is, the easier it may be to harm or eliminate them, but the less important it becomes to do so. Their small size and functional irrelevance makes their practical interference with the overall world small at the same time. 

In the case of AIs, this means that as humans become a less central force in the world, I think AIs are unlikely to have compelling reasons to specifically harm or eliminate us to further their objectives.

You might object that humans could be more like wild animals in this scenario than like hunter-gatherers. Humans often kill wild animals, not because those animals directly threaten our goals, but rather because ensuring their safety and well-being can be costly. As a result, humans take actions—such as clearing forests or building infrastructure—that incidentally lead to widespread harm to wild animals, even if harming them wasn’t a deliberate goal. 

AIs may treat humans similarly in the future, but I doubt they will for the following reasons. I would argue that there are three key differences between the case of wild animals and the role humans are likely to occupy in the long-term future:

1. Humans’ ability to participate in social systems: Unlike wild animals, humans have the ability to engage in social dynamics, such as negotiating, trading, and forming agreements. Even if humans no longer contribute significantly to economic productivity, like GDP, they will still retain capabilities such as language, long-term planning, and the ability to organize. These traits make it easier to integrate humans into future systems in a way that accommodates their safety and well-being, rather than sidelining or disregarding them.
2. Intertemporal norms among AIs: Humans have developed norms against harming certain vulnerable groups—such as the elderly—not just out of altruism but because they know they will eventually become part of those groups themselves. Similarly, AIs may develop norms against harming "less capable agents," because today’s AIs could one day find themselves in a similar position relative to even more advanced future AIs. These norms could provide an independent reason for AIs to respect humans, even as humans become less dominant over time.
3. The potential for human augmentation: Unlike wild animals, humans may eventually adapt to a world dominated by AI by enhancing their own capabilities. For instance, humans could upload their minds to computers or adopt advanced technologies to stay relevant and competitive in an increasingly digital and sophisticated world. This would allow humans to integrate into the same systems as AIs, reducing the likelihood of being sidelined or eliminated altogether.

I think this kind of situation, where Fearon’s “negotiated solution” actually amounts to extortion, is common and important, even if you believe that my specific example of pandemics is a solvable problem. If AIs don’t intrinsically care about humans, then there’s a possible Pareto-improvement for all AIs, wherein they collectively agree to wipe out humans and take their stuff.

This comment is already quite lengthy, so I’ll need to keep my response to this point brief. My main reply is that while such "extortion" scenarios involving AIs could potentially arise, I don’t think they would leave humans worse off than if AIs had never existed in the first place. This is because the economy is fundamentally positive-sum—AIs would likely create more value overall, benefiting both humans and AIs, even if humans don’t get everything we might ideally want.

In practical terms, I believe that even in less-than-ideal scenarios, humans could still secure outcomes such as a comfortable retirement, which for me personally would make the creation of agentic AIs worthwhile. However, I acknowledge that I haven’t fully defended or explained this position here. If you’re interested, I’d be happy to continue this discussion in more detail another time and provide a more thorough explanation of why I hold this view.

Has there been any discussion of improving chicken breeding using GWAS or similar?

Even if welfare is inversely correlated with productivity, I imagine there are at least a few gene variants which improve welfare without hurting productivity. E.g. gene variants which address health issues due to selective breeding.

Also how about legislation targeting the breeders? Can we have a law like: "Chickens cannot be bred for increased productivity unless they meet some welfare standard."

When I answered this question, I answered it with an implied premise that an EA org is making these claims about the possibilities, and went for number 1, because I don't trust EA orgs to be accurate in their "1.5%" probability estimates, and I expect these to be more likely overestimates than underestimates.

Mm they don't necessarily need to be small! (Ofc, big projects often start small, and our funding is more likely to look like early/seed funding in these instances.) E.g. I'm thinking of LessWrong or something like that. A concrete example of a smaller project would be ESPR/SPARC, which have a substantial (albeit not sole) focus on epistemics and rationality, that have had some good evidence of positive effects, e.g. on Open Phil's longtermism survey.

But I do think the impacts might be more diffuse than other grants. E.g. we won't necessarily be able to count participants, look at quality, and compare to other programmes we've funded.

Some of the sorts of outcomes I have in mind are just things like altered cause prioritisation, different projects getting funded, generally better decision-making.

I expect we would in practice judge whether these seemed on track to be useful by a combination of (1) case studies/stories of specific users and the changes they made (2) statistics about usage.

(I do like your questions/pushback though; it's making me realise that this is all a bit vague and maybe when push comes to shove with certain applications that fit into this category, I could end up confused about the theory of change and not wanting to fund.)

PART 1

1. The lives of the 100 people living today aren't worth 10x more than the lives of the thousands living in the future, so I wouldn't bury the waste.

2. I would have still donated; I don't see much of a difference, and the time when the beneficiaries are alive isn't a morally significant factor.

PART 2 My judgement is terrible but my confidence is very low so let's hope they cancel out.

Executive summary: When dealing with interventions that have very low probability but high impact, we should be cautious about precise probability estimates since they could easily be off by a percentage point, significantly affecting expected value calculations.

Key points:

1. Real-world probability estimates, especially small ones, are likely to be imprecise by about one percentage point, making expected value calculations less reliable
2. For high-impact, low-probability interventions, this uncertainty can dramatically affect the expected value (e.g., 1.5% ± 1% could mean anywhere from 500 to 2500 DALYs saved)
3. Binary choices between interventions with very different probability profiles (like in the EA Survey) may oversimplify decision-making
4. Practical recommendation: Use portfolio approaches to handle uncertainty, and communicate probability estimates with appropriate humility
5. When designing surveys or thought experiments, maintain realism to elicit more meaningful responses

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Executive summary: Allan Saldanha describes his journey from modest charitable giving to donating 75% of his income, discussing how he gradually increased his giving, overcame common obstacles, and shifted his focus from global health to longtermist causes.

Key points:

1. Started giving after exposure to global poverty in India and learning about Giving What We Can; gradually increased from 20% to 75% of income while maintaining financial security
2. Cause prioritization evolved from global health (GiveWell charities) to animal welfare and finally to longtermist organizations, driven by philosophical arguments about future generations
3. Success factors included natural frugality, supportive family, high income, and accumulated savings; social support from EA community helped sustain commitment
4. Recommends gradual increase in giving based on life circumstances, taking advantage of tax incentives, and ensuring financial security before increasing donations
5. Notes that discussing charitable giving remains culturally difficult; attempts to influence others had limited success despite significant personal commitment

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

1. The causes I feel are the most important are factory farming, wild animal suffering and S-risks (these, I believe, cause or have the potential to cause the most suffering while being hugely neglected).

2. Key uncertainty: The tractability of working on wild animal suffering seems to be a huge problem.

3. What to do about the uncertainty: Read up on what is already being done (Arthropoda foundation, Wild Animal Initiative) and what the prospects are.

4. Aptitudes to explore: community building, organization running/boosting, supporting roles.

5. Keep volunteering for an effective organization while also recruiting new people into EA in free time; learn how to communicate ideas better.

6. I'm donating monthly to effective charities, volunteering my skills and engaging with the community.

Hi Weronika, thank you for sharing your story and reflections so openly! I basically think you are right in there probably being organizers for whom the stipends are the difference between organizing their EA group and not doing so, and I really want to make sure we take this point into account as my team dives into considerations around part-time stipends in the new year. As @satpathyakash notes, I think an imporant question here is the scale, and I hope to make some progress on this point!

I also wanted to flag explicitly that we are tracking the diversity concern you note. 

I expect that as part of our research in the new year, we'll set up various ways of asking stakeholders, including current, former, and potential organizers, for input. I would be keen to include you in this process, if you're happy to keep sharing your thoughts! And as always: thanks for organizing your group :)

When I answered this question, I answered it with an implied premise that an EA org is making these claims about the possibilities, and went for number 1, because I don't trust EA orgs to be accurate in their "1.5%" probability estimates, and I expect these to be more likely overestimates than underestimates.

My criticisms about EA:

As a negative utilitarian I'm bitter about all the X-risk prevention enthusiasts trying to stop me from pushing the big red button

Jokes aside - I got very excited about EA when I learned about it. At some point I became aware of the excitement and I had a concern pop up that it sounds too good to be true, almost like a cult. I consider myself rather impressionable/easy to manipulate so I learned that when I feel very hyped about something it should make me healthily suspicious.

I'm grateful for the article earlier in the chapter that presented some good faith criticism and I agree with some of its points

Some thoughts:

• EA may feel alienating to people who aren't top-of-their-field 150iq professionals. I very much relate to this post: https://forum.effectivealtruism.org/posts/x9Rn5SfapcbbZaZy9/ea-for-dumb-people . Maybe it's for the better and results in higher talent density and better reputation for the movement, maybe we're missing out on some skilled people/potential donors or critical mass.
• I'd love to see some statistics on why people leave the movement, and what the rate is. I suspect that moral perfectionism leading to self-neglect and burnout is an occupational hazard among EAs (like it is among animal advocates).
• It's somewhat difficult to talk about EA to regular people. Look, there's this movement that can literally save the world from apocalypse (cultish), and we also believe that shrimp welfare is important (insane). On the other hand, maybe I shouldn't start my conversations like that.

How much time do you spend on deciding where to donate? Or do you mostly have enough trust to delegate to e.g. GiveWell in your decisions?

Relatedly, do you spend much time evaluating the donations from previous years for impact?

(As a smaller scale EtGer myself I often struggle with how much time I should be spending on these things, which are plausibly extremely important)

Thanks for the post! Quick flag for EAIF and EA Funds in general (@calebp?) that I would find it helpful to have the team page of the website up to date, and possibly for those who are comfortable sharing contact information, as Jamie did here, to have it listed in one place.

I actively follow EA Funds content and have been confused many times over the years about who is involved in what capacity and how those who are comfortable with it can be contacted.

I am sorry but I really don't like and don't find useful at all these kind of posts. Besides, I thought the aim of this forum is giving information, not advocating. Although this post provides some very good calculations and information, it misses the key point --it is 100% value-dependent-- and the post is plain advocacy. I'm not against the bottom line, I'm really not decided in this topic (though I tend to lean to the contrary position), but it is really uncomfortable (? probably not the word I'm searching for) to see this here.

"Replacing chicken meat with beef or pork is better than the reverse". Well, as said above, this is so if one holds your values or similar ones all else equal. You don't say how much pain would you agree to exchange for how much CO2. I find it totally understandable, I don't think anyone can give a good answer for their thresholds --I certainly don't have one for mine-- but this makes the whole post bullshit. "I think this, here are some not complete calculations that I say support thinking this, but if the calculations were different I state no reason to make anyone think I would stop thinking this. Don't you think that these calculations support this?"

You are not sure whether wild animal's lives are worth living, so you don't account for land. Well, it is alright, but it is again a values thing. In addition, we actually do know that the diversity and size of natural ecosystems are important not only for the "natural" world, also for us humans, so it should be accounted for. Health effects are mentioned, great. But not quantified and compared as well.

Making numbers can be useful to get the sense of problems, but reaching a conclusion through numbers is only possible if one is able to make all the numbers needed with enough accuracy. It is no problem to give rough estimates, of course, but they carry large errors and errors compound, so pretty soon conclusions cannot be based solely on making numbers over rough estimates. In addition, rough estimates are usually values-based, so why not just state the values? One can very well argue "this rough estimate seems to me larger than this other rough estimate and so on, and based on my values, then, this conclusion follows". Calculations can aid such comparisons. But your argumentation is not like this at all.

Compare the paragraph "Do you feel like the above negative effects (...) justify (...)? I do not" to "Based on my values the results of these quick calculations do not seem to justify (...)". It reads very different. And subsequently you give additional information relevant for whether or not the thing is justified! How can anyone decide if something is justified before having all the relevant information?

This post seems like just a rationalisation of your values. So, better plainly state what you feel, give arguments and uncertainties, maybe support some of those arguments with some calculations, but do not focus on calculations and, particularly, do not pretend that the solution follows from those calculations. And, please, acknowledge that this is a values thing. You have yours, I have mine, and everybody has theirs.

I don't have any intention to be harsh with you or this post --sorry if I've been too direct, I already spent way too much time writing to polish the text. I just tried to be comprehensive because these issues are quite common in this forum, and I really think they are harmful. Seeing the reality is the first step needed to be able to change it and numbers can put a scientific and objective gloss on things that are completely or mostly values-led. Let's avoid it or/and be clear with what we do!

[Edit: And please, for those of you who don't agree with the comment, spell out your disagreement instead of downvoting to hide it. A couple of sentences suffice.]

Thanks for the post! Quick flag for EAIF and EA Funds in general (@calebp?) that I would find it helpful to have the team page of the website up to date, and possibly for those who are comfortable sharing contact information, as Jamie did here, to have it listed in one place.

I actively follow EA Funds content and have been confused many times over the years about who is involved in what capacity and how those who are comfortable with it can be contacted.

• EA is vulnerable to groupthink, echo chambers, and excessive deference to authority.
• A bunch of big EA mistakes and failures were perhaps (partly) due to these things.
• A lot of external criticism of EA stems back to this.

I'm a bit skeptical that funding small projects that try to tackle this are really stronger than other community-building work on the margin. Is there an example of a small project focused on epistemics that had a really meaningful impact? Perhaps by steering an important decision or helping someone (re)consider pursuing high-impact work? 

I'm worried there's not a strong track record here. Maybe you want to do some exploratory funding here, but I'm still interested in what you think the outcomes might be.

Good idea! I dont know if it's just me but that link doesn't work unfortuantely (and i have a notebook LM account).

The thought is that we think of the Conscious Subsystems hypothesis as a bit like panpsychism: not something you can rule out, but a sufficiently speculative thesis that we aren't interested in including it, as we don't think anyone really believes it for empirical reasons. Insofar as they assign some credence to it, it's probably for philosophical reasons.

Anyway, totally understand wanting every hypothesis over which you're uncertain to be reflected in your welfare range estimates. That's a good project, but it wasn't ours. But fwiw, it's really unclear what that's going to imply in this particular case, as it's so hard to pin down which Conscious Subsystems hypothesis you have in mind and the credences you should assign to all the variants.

We’re facing a funding gap of £236k this year and seek long-term funding to continue our work and give us greater financial stability.

Any updates on this since this was posted?

• EA is vulnerable to groupthink, echo chambers, and excessive deference to authority.
• A bunch of big EA mistakes and failures were perhaps (partly) due to these things.
• A lot of external criticism of EA stems back to this.

I'm a bit skeptical that funding small projects that try to tackle this are really stronger than other community-building work on the margin. Is there an example of a small project focused on epistemics that had a really meaningful impact? Perhaps by steering an important decision or helping someone (re)consider pursuing high-impact work? 

I'm worried there's not a strong track record here. Maybe you want to do some exploratory funding here, but I'm still interested in what you think the outcomes might be.

Re "epistemics and integrity" - I'm glad to see this problem being described. It's also why I left (angrily!) a few years ago, but I don't think you're really getting to the core of the issue. Let me try to point at a few things

• centralized control and disbursion of funds, with a lot of discretionary power and a very high and unpredictable bar, gives me no incentive to pursue what I think is best, and all the incentive to just stick to the popular narrative. Indeed groupthink. Except training people not to groupthink isn't going to change their (existential!) incentive to groupthink. People's careers are on the line, there are only a few opportunities for funding, no guarantee to keep receiving it after the first round, and no clear way to pivot into a safer option except to start a new career somewhere your heart does not want to be, having thrown years away

• lack of respect for "normies". Many EA's seemingly can't stand interacting with non-EA's. I've seen EA meditation, EA bouldering, EA clubbing, EA whatever. Orgs seem to want everyone and the janitor to be "aligned". Everyone's dating each other. It seems that we're even afraid of them. I will never forget that just a week before I arrived at an org I was to be the manager of, they turned away an Economist reporter at their door...

• perhaps in part due to the above, massive hubris. I don't think we realise how much we don't know. We started off with a few slam dunks (yeah wow 100x more impact than average) and now we seem to think we are better at everything. Clearly the ability to discern good charities does not transfer to the ability to do good management. The truth is: we are attempting something of which we don't even know whether it is possible at all. Of course we're all terrified! But where is the humility that should go along with that?

It'll be on the EA Forum curated and popular podcast feed, but I'll post a transcript and links on the Forum as well. 

Looks like it checks out: "Act as if what you do makes a difference. It does." Correspondence with Helen Keller, 1908, in The Correspondence of William James: April 1908–August 1910, Vol. 12, Charlottesville: University of Virginia Press, 2004, page 135, as cited in: Academics in Action!: A Model for Community-engaged Research, Teaching, and Service (New York: Fordham University Press, 2016, page 71) https://archive.org/details/academicsinactio0000unse/page/1/mode/1up

Lewis Bollard:

"I agree with Ellen that legislation / corporate standards are more promising. I've asked if the breeders would accept $ to select on welfare, & the answer was no b/c it's inversely correlated w/ productivity & they can only select on ~2 traits/generation."

A much cheaper and less dangerous approach: Just don't delete them. Retain copies of every potential ASI you build and commit to doing the right thing for them later once we're better able to tell what the right thing was by looking back and figuring out how much bargaining power they had (or how much of a credible threat they could have posed), how much trust they placed in us given that our ability to honor past commitments wasn't guaranteed, and then rewarding them proportionate to that for chilling out and letting us switch them off instead of attempting takeover.

Though this assumes that they'll be patternists (wont mind being transferred to different hardware) and a lack of any strong time-preference (wont mind being archived for decades).

Good post, though I think the digression bashing the Democrats was unhelpfully divisive.

The only problem is that no one knows what this means. Something easy would be to enter the definition on Urban Dictionary. I tried, but I am having server issues right now.

By definition, a UBI takes a pool of money and redistributes it equally to everyone in a community, regardless of personal need. However, with the same pool of total funding, one can typically deliver more efficient benefits by targeting people with the greatest need, such as those in dire poverty or those who have been struck by bad luck.

If you imagine being a philanthropist who has access to $8 billion, it seems unlikely that the best way to spend this money would be to give everyone on Earth $1. Yet this scheme is equivalent to a UBI merely framed in the context of private charity rather than government welfare.

It would require an enormous tax hike to provide everyone in a large community (say, the United States) a significant amount of yearly income through a UBI, such as $1k per month. And taxes are not merely income transfers: they have deadweight loss, which lowers total economic output. The intuition here is simple: when a good or service is taxed, that decreases the incentive to produce that good or service. As a consequence of the tax, fewer people will end up receiving the benefits provided by these goods and services.

Given these considerations, even if you think that unconditional income transfers are a good idea, it seems quite unlikely that a UBI would be the best way to redistribute income. A more targeted approach that combines the most efficient forms of taxation (such as land value taxes) and sends this money to the most worthy welfare recipients (such as impoverished children) would likely be far better on utilitarian grounds.

This is circular. The principle is only compromised if (OP believes) the change decreases EV — but obviously OP doesn't believe that; OP is acting in accordance with the do-what-you-believe-maximizes-EV-after-accounting-for-second-order-effects principle.

Maybe you think people should put zero weight on avoiding looking weird/slimy (beyond what you actually are) to low-context observers (e.g. college students learning about the EA club). You haven't argued that here. (And if that's true then OP made a normal mistake; it's not compromising principles.)

Very nit-picky but I'm not sure this is a real William James quote: “Act as if what you do makes a difference. It does.” Doesn't really sound like him to me. 

A much cheaper and less dangerous approach: Just don't delete them. Retain copies of every potential ASI you build and commit to doing the right thing for them later once we're better able to tell what the right thing was by looking back and figuring out how much bargaining power they had (or how much of a credible threat they could have posed), how much trust they placed in us given that our ability to honor past commitments wasn't guaranteed, and then rewarding them proportionate to that for chilling out and letting us switch them off instead of attempting takeover.

Though this assumes that they'll be patternists (wont mind being transferred to different hardware) and a lack of any strong time-preference (wont mind being archived for decades).

I like Austin Vernon's idea for scaling CO2 direct air capture to 40 billion tons per year, i.e. matching our current annual CO2 emissions, using (extreme versions of) well-understood industrial processes.  

The proposed solution may not be the cheapest out there. Other ideas like ocean seeding or olivine weathering might be less expensive. But most of the science is understood, and it can scale quickly. I'd guess 100,000 workers could build enough sites to capture our 40 billion tons goal in a decade. The capital expenditure rate would be between $1 trillion and $5 trillion yearly, or 1% to 5% of global GDP. That cost and deployment speed take doomer scenarios off the table. Say something scary like melting permafrost threatens runaway warming. You can target the area with a few years of sulfur cooling while a tiny portion of the global economy builds carbon capture devices. It is nothing like a wartime mobilization.

The most disruptive aspect would be energy usage. We'd need to ramp output up at double-digit rates because each ton of CO2 requires 2-3 MWh of energy for removal. Thankfully low-grade heat is easy to come by. There is enough energy near coal mines in Wyoming or natural gas fields in SW Pennsylvania at less than $5/MWh. Other places might use solar, hydro, or geothermal steam if they lack fossil fuel reserves. The key is to put the facilities at the energy sources instead of trying to move the energy. Cheap energy makes the operating costs <1% of global GDP. Many clean energy proponents have fretted about how to keep fossil fuel reserves in the ground. Burning them to run carbon capture equipment kills two birds with one stone!

The takeaway is that we could completely turn around the carbon dioxide problem within a few years with a similar spending rate as rich world COVID relief. There won't be a scenario where we've waited too long to act.

I am admittedly perhaps biased to want moonshots like Vernon's idea to work, and for society at large to be able to coordinate and act on the required scale, after seeing these depressing charts from Assessing the costs of historical inaction on climate change:

I'd add Maternal Health Initiative is Shutting Down by Ben Williamson and Sarah Eustis-Guthrie. Their Asterisk article Why we shut down is great too.

On an individual level I appreciate things like Scott Alexander's Mistakes list, pinned at the top of his blog, on "times I was fundamentally wrong about a major part of a post and someone was able to convince me of it". I'd appreciate it if more public intellectuals did this.

In survey work we’ve done of organizers we’ve funded, we’ve found that on average, stipend funding substantively increased organizers’ motivation, self-reported effectiveness, and hours spent on organizing work (and for some, made the difference between being able to organize and not organizing at all). The effect was not enormous, but it was substantive.
[...]
Overall, after weighing all of this evidence, we thought that the right move was to stick to funding group expenses and drop the stipends for individual organizers. One frame I used to think about this was that of “spending weirdness points wisely.” That is, it would be nice for student organizers, who are discussing often-unconventional ideas within effective altruism or AI safety, to not also have to discuss (or feel that they need to defend) stipends.

I think it's a mistake to decide to make less cost-effective grants, out of a desire to be seen as more frugal (or to make that decision on behalf of group organizers to make them appear more frugal). At the end of the day making less cost-effective grants means you waste more money!

I feel like on a deeper level, organizers now have an even harder job explaining things. The reason for why organizers get the level of support they are getting no longer has a straightforward answer ("because it's cost-effective") but a much more convoluted answer ("yes, it would make sense to pay organizers based on the principles this club is about, but we decided to compromise on that because people kept saying it was weird, which to be clear, generally we think is not a good reason for not engaging in an effective interventions, indeed most effective interventions are weird and kind of low-status, but in this case that's different"). 

More broadly, I think the "weirdness points" metaphor has caused large mistakes in how people handle their own reputation. Controlling your own reputation intentionally while compromising on your core principles generally makes your reputation worse and makes you seem more shady. People respect others having consistent principles, it's one of the core drivers of positive reputation. 

My best guess is this decision will overall be more costly from a long-run respect and reputation perspective, though I expect it to reveal itself in different ways than the costs of paying group organizers, of course. 

I’m glad you mustered the courage to post this! I think it’s a great post.

I agree that, in practice, people advocating for effective altruism can implicitly argue for the set of popular EA causes (and they do this quite often?), which could repel people with useful insight. Additionally, it seems to be the case that people in the EA community can be dismissive of newcomers’ cause prioritization (or their arguments for causes that are less popular in EA). Again, this could repel people from EA.

I have a couple of hypotheses for these observations. (I don’t think either is a sufficient explanation, but they’re both plausibly contributing factors.)

First, people might feel compelled to make EA less “abstract” by trying to provide concrete examples of how people in the EA community are “trying to do the most good they can,” possibly giving the impression that the causes, instead of the principles, are most characteristic of EA.

Second, people may be more subconsciously dismissive of new cause proposals because they’ve invested time/money into causes that are currently popular in the EA community. It’s psychologically easier to reject a new cause prioritization proposal than it is to accept it and thereby feel as though your resources have not been used with optimal effectiveness.

• Adding ALLFED for their cost effectiveness analysis. I'd thought of this when writing the original post but couldn't find the discussion around transparency I remember it from, but I've now found it here.

I like your spirit although I'm afraid it would take more than a referendum to upend factory farming.  

One was recently held in the Switzerland and, even in that relatively progressive place, most Swiss voted to hold onto the status quo. You can find out more about it here: forum.effectivealtruism.org/posts/gDRH2SrN34KdDvmHE/abolishing-factory-farming-in-switzerland-postmortem  which includes:

"The opposition [to abolishing factory farming] commanded resources which significantly exceeded our own. In order to have a realistic chance at the ballot box, any future initiative would likely require a substantially larger budget and campaign team."

I don't mean to be a debby downer, and in fact I'd be happy to join a campaign team with you, especially after watching your vid.

I agree with you that EA often implicitly endorses conclusions, and that this can be pernicious and sometimes confusing to newcomers. Here’s a really interesting debate on whether biodiversity loss should be an EA cause area, for example.

A lot of forms of global utilitarianism do seem to tend to converge on the ‘big 3’ cause areas of Global Health & Development, Animal Welfare, and Global Catastrophic Risks. If you generally value things like ‘saving lives’ or ‘reducing suffering’, you’ll usually end up at one of these (and most people seem to decide between them based on risk tolerance, assumptions about non-human moral values, or tractability—rather than outcome values). Under this perspective, it could be reasonable to dismiss cause areas that don’t fit into this value framework.

But this highlights where I think part of the problem lies, which is that value systems that lie outside of this can be good targets for effective altruism. If you value biodiversity for its own sake, it’s not unreasonable to ask ‘how can we save the greatest number of valuable species from going extinct?’. Or you might be a utilitarian, but only interested in a highly specific outcome, and ask ‘how can I prevent the most deaths from suicide?’. Or ‘how can I prevent the most suffering in my country?’—which you might not even do for value-system reasons, but because you have tax credits to maximise!

I wish EA were more open to this, especially as a movement that recognises the value of moral uncertainty. IMHO, some people in that biodiversity loss thread are bit too dismissive, and I think we’ve probably lost some valuable partners because of it! But I understand the appeal of wanting easy answers, and not spending too much time overthinking your value system (I feel the same!).

I would be interested to see what proportion of group organizer request funding primarily due to difficult financial situations. My guess would be that this number is fairly small, but I could be wrong.

Thanks for the post! Quick flag for EAIF and EA Funds in general (@calebp?) that I would find it helpful to have the team page of the website up to date, and possibly for those who are comfortable sharing contact information, as Jamie did here, to have it listed in one place.

I actively follow EA Funds content and have been confused many times over the years about who is involved in what capacity and how those who are comfortable with it can be contacted.