One recent paper  suggests that an estimated additional $200–328 billion per year is required for the various measures of primary care and public health interventions from 2020 to 2030 in 67 low-income and middle-income countries and this will save 60 million lives. But if you look at just the amount needed in low-income countries for health care - $396B - and divide by the total 16.2 million deaths averted by that, it suggests an average cost-effectiveness of ~$25k/death averted. 

Other global health interventions can be similarly or more effective: a 2014 Lancet article estimates that, in low-income countries, it costs $4,205 to avert a death through extra spending on health^[22]. Another analysis suggests that this trend will continue and from 2015-2030 additional spending in low-income countries will avert a death for $4,000-11,000^[23].

For comparison, in high-income countries, the governments spend $6.4 million to prevent a death (a measure called “value of a statistical life”)^[24]. This is not surprising given the poorest countries spend less than $100 per person per year on health on average, while high-income countries almost spend $10,000 per person per year^[25].

GiveDirectly is a charity that can productively absorb very large amounts of donations at scale, because they give unconditional cash transfers to extremely poor people in low-income countries. A Cochrane review suggests that such unconditional cash-transfers “probably or may improve some health outcomes.^[21]  One analysis suggests that cash-transfers are roughly equivalent as effective as averting a death on the order of $10k .

So essentially  cost-effectiveness doesn't drop off sharply after Givewell's top charities are 'fully funded', and one could spend billions and billions at similar cost-effectiveness, Gates only has ~$100B and only spends~$5B a year.

I wrote a post about this 7 years ago! Still roughly valid.

This is a great question, and the same should be asked of governments (as in: "why doesn't the UK aid budget simply all go to mosquito nets?")

A likely explanation for why the Gates Foundation doesn't give to GiveWell's top charities is that those charities don't currently have much room for more funding (GiveWell had to rollover funding last year because they couldn't spend it all. A recent blog posts suggests they may have more room for funding soon https://blog.givewell.org/2022/07/05/update-on-givewells-funding-projections/)

A likely explanation for why the Gates Foundation doesn't give to GiveDirectly is that they don't see strong enough evidence yet for the effectiveness (particularly in the long-term or at the societal level) of unconditional cash transfers (A Cochrane review from this year suggests slight short-term benefits: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD011135.pub3/full)

Thanks for asking this. I can chime in, although obviously I can't speak for all the organizations listed, or for "EA leadership." Also, I'm writing as myself — not a representative of my organization (although I mention the work that my team does). 

1. I think the Forum team takes this worry seriously, and we hope that the Forum contributes to making the EA community more truth-seeking in a way that disregards status or similar phenomena (as much as possible). One of the goals for the Forum is to improve community norms and epistemics, and this (criticism of established ideas and entities) is a relevant dimension; we want to find out the truth, regardless of whether it's inconvenient to leadership. We also try to make it easy for people to share concerns anonymously, which I think makes it easier to overcome these barriers.
   1. I personally haven't encountered this problem (that there are reasons to be afraid of criticizing leadership or established norms) — no one ever hinted at this, and I've never encountered repercussions for encouraging criticism, writing some myself, etc. I think it's possible that this happens, though, and I also think it's a problem even if people in the community only think it's a problem, as that can still silence useful criticism. 
2. I can't speak about funding structures, but we did run the criticism contest in part to encourage criticism of the most established organizations and norms, and we explicitly encouraged criticism of the most important of those. 
3. I think lots of organizations have "mistakes" pages, and Ben linked to a question asking for examples of this kind of thing. Off the top of my head, I don't know of much else — this could be a good project for someone to undertake!

Are there publicly available compilations of times where EA leadership or major/public facing organizations have made meaningful changes as a result of public or private feedback?

Some examples here: Examples of someone admitting an error or changing a key conclusion.

Quoting Kelsey Piper:

If I tell you “I’m torturing an animal in my apartment,” do you go “well, if there are no other animals being tortured anywhere in the world, then that’s really terrible! But there are some, so it’s probably not as terrible. Let me go check how many animals are being tortured.”

(a minute later)

“Oh, like ten billion. In that case you’re not doing anything morally bad, carry on.”

I can’t see why a person’s suffering would be less morally significant depending on how many other people are suffering. And as a general principle, arbitrarily bounding variables because you’re distressed by their behavior at the limits seems risky.

Not a philosopher, but scope sensitivity follows from consistency (either in the sense of acting similarly in similar situations, or maximizing a utility function). Suppose you're willing to pay $1 to save 100 birds from oil; if you would do the same trade again at a roughly similar rate (assuming you don't run out of money) your willingness to pay is roughly linear in the number of birds you save.

Scope insensitivity in practice is relatively extreme; in the original study, people were willing to pay $80 for 2000 birds and $88 for 200,000 birds. So if you think this represents their true values, people were willing to pay $.04 per bird for the first 2000 birds but only $0.00004 per bird for the next 198,000 birds. This is a factor of 1000 difference; most of the time when people have this variance in price they are either being irrational, or there are huge diminishing returns and they really value something else that we can identify. For example if someone values the first 2 movie tickets at $1000 each but further movie tickets at only $1, maybe they really enjoy the experience of going with a companion, and the feeling of happiness is not increased by a third ticket. So in the birds example it seems plausible that most people value the feeling of having saved some birds.

Why should you be consistent? One reason is the triage framing, which is given in Replacing Guilt. Another reason is the money-pump; if you value birds at $1 per 100 and $2 per 1000, and are willing to make trades in either direction, there is a series of trades that causes you to lose both $ and birds.

All of this relies on you caring about consequences somewhat. If your morality is entirely duty-based or has some other foundation, there are other arguments but they probably aren't as strong and I don't know them.

Two empirical reasons not to take the extreme scope neglect in studies like the 2,000 vs 200,000 birds one as directly reflecting people's values.

First, the results of studies like this depend on how you ask the question. A simple variation which generally leads to more scope sensitivity is to present the two options side by side, so that the same people would be asked both about 2,000 birds and about the 200,000 birds (some call this "joint evaluation" in contrast to "separate evaluation"). Other variations also generally produce more scope sensitive results (this Wikipedia article seems uneven in quality but gives a flavor for some of those variations.) The fact that this variation exists means that just take people's answers at face value does not work as a straightforward approach to understanding people's values, and I think the studies which find more scope sensitivity often have a strong case for being better designed.

Second, there are variants of scope insensitivity which involve things other than people's values. Christopher Hsee has done a number of studies in the context of consumer choice, where the quantity is something like the amount of ice cream that you get or the number of entries in a dictionary, which find scope insensitivity under separate evaluation (but not under joint evaluation), and there is good reason to think that people do prefer more ice cream and more comprehensive dictionaries. Daniel Kahneman has argued that several different kinds of extension neglect all reflect similar cognitive processes, including scope neglect in the bird study, base rate neglect in the Tom W problem, and duration neglect in studies of colonoscopies. And superforecasting researchers have found that ordinary forecasters neglect scope in questions like (in 2012) "How likely is it that the Assad regime will fall in the next three months?" vs. "How likely is it that the Assad regime will fall in the next six months?"; superforecasters' forecasts are more sensitive to the 3 month vs. 6 month quantity (there's a passage in Superforecasting about this which I'll leave as a reply, and a paper by Mellers & colleagues with more examples). These results suggest that people's answers to questions about values-at-scale has a lot to do with how people think about quantities, that "how people think about quantities" is a fairly messy empirical matter, and that it's fairly common for people's thinking about quantities to involve errors/biases/limitations which make their answers less sensitive to the size of the quantity.

This does not imply that the extreme scope sensitivity common in effective altruism matches people's values; I think that claim requires more of a philosophical argument rather than an empirical one. Just that the extreme scope insensitivity found in some studies probably doesn't match people's values.

Hm, I think that most of the people who participated in this experiment: 

three groups of subjects were asked how much they would pay to save 2,000 / 20,000 / 200,000 migrating birds from drowning in uncovered oil ponds. The groups respectively answered $80, $78, and $88.¹ This is scope insensitivity or scope neglect: the number of birds saved—the scope of the altruistic action—had little effect on willingness to pay.

would agree after the results were shown to them that they were doing something irrational that they wouldn't endorse if aware of it. (Example taken from here: https://www.lesswrong.com/posts/2ftJ38y9SRBCBsCzy/scope-insensitivity )

There's also an essay from 2008 about the intuitions behind utilitarianism that you might find helpful for understanding why someone could consider scope insensitivity a bias instead of just the way human values work: 

https://www.lesswrong.com/posts/r5MSQ83gtbjWRBDWJ/the-intuitions-behind-utilitarianism 

I think scope insensitivity could be a form of risk aversion over the difference you make in the world (=difference-making) (scope insensitivity is related at least). I explain here why I think that risk aversion over the difference you make is irrational even though risk aversion over states of the world is not. 

I think they are basically not a bias in the way confirmation bias is, and anyone claiming otherwise is pre-supposing linear aggregation of welfare already. From a thing I wrote recently:

Scope neglect is not a cognitive bias like confirmation bias. I can want there to be ≥80 birds saved, but be indifferent about larger numbers: this does not violate the von Neumann-Morgenstern axioms (nor any other axiomatic systems that underlie alternatives to utility theory that I know of). Similarly, I can most highly value there being exactly 3 flowers in the vase on the table (less being too sparse, and more being too busy). The pebble-sorters of course go the extra mile.

Calling scope neglect a bias pre-supposes that we ought to value certain things linearly (or at least monotonically). This does not follow from any mathematics I know of. Instead it tries to sneak in utilitarian assumptions by calling their violation "biased".

Various social aggregation theorems (e.g. Harsanyi's) show that "rational" people must aggregate welfare additively.

(I think this is a technical version of Thomas Kwa's comment.)

To answer this question in short: It is so because it's innate. Like any other bias scope insensitivity comes from within, in the case of an individual as well as an organization run by individuals. We may generalize it as the product of human values because of the long-running history of constant 'Self-Value' teachings(not the spiritual ones). But there will always be a disparity when considering the ever-evolving nature of human values, especially in the current era.
 

--------

On the contrary, most of the time, I do consider scope insensitivity as the typical human way. One absurd reason I identified is the outward negligence towards any scope of sensitive issues. There's always this demand for a huge and attractive convincing, whenever there are multiple issues at hand. And the ones with the ability to convince often get listened to. The result: the insensitivity towards the scope of the issue posed by a commoner(less talented). 

This is just one case. If we somehow avert from pinning blame, we can say that there is a very real imbalance between the scope identifiers and the scope rectifiers.

There's a lot of interesting writing about the evolutionary biology and evolutionary psychology of genetic selfishness, nepotism, and tribalism, and why human values descriptively focus on the sentient beings that are more directly relevant to our survival and  reproductive fitness -- but that doesn't mean our normative or prescriptive values should follow whatever natural selection and sexual selection programmed us to value.

Looking at your profile I think you have a good idea of answers already, but for the benefit of everyone else who upvoted this question looking for an answer, here's my take:

Are there AI risk scenarios which involve narrow AIs?

Yes, a notable one being military AI i.e. autonomous weapons (there are plenty of related posts on the EA forum). There are also multipolar failure modes on risks from multiple AI-enabled superpowers instead of a single superintelligent AGI.

Why does most AI risk research and writing focus on artificial general intelligence?

A misaligned AGI is a very direct pathway to x-risk, where an AGI that pursues some goal in an extremely powerful way without having any notion of human values could easily lead to human extinction. The question is how to make an AI that's more powerful than us do what we want it to do. Many other failures modes like bad actors using tool (narrow) AIs seem less likely to lead directly to x-risk, and is also more of a coordination problem than a technical problem. 

What happens when we create AI companions for children that are more “engaging” than humans? Would children stop making friends and prefer AI companions?
What happens when we create AI avatars of mothers that are as or more “engaging” to babies than real mothers, and people start using them to babysit? How might that affect a baby’s development?
What happens when AI becomes as good as an average judge at examining evidence, arguments, and reaching a verdict?
 

1. "AGI" is largely an imprecisely-used initialism: when people talk about AGI, we usually don't care about generality and instead just mean about human-level AI. It's usually correct to implicitly substitute "human-level AI" for "AGI" outside of discussions of generality. (Caveat: "AGI" has some connotations of agency.)
2. There are risk scenarios with narrow AI, including catastrophic misuse, conflict (caused or exacerbated by narrow AI), and alignment failure. On alignment failure, there are some good stories. Each of these possibilities is considered reasonably likely by AI safety & governance researchers.

1. OpenPhil has commissioned various reviews of its work, e.g. on power-seeking AI.
2. Less formal, but there was this facebook debate between some big names in AI.

Overall, I think a) this would be cool to see more of and b) it would be a service to the community if someone collected all the existing examples together.

Not exactly what you're describing, but MIRI and other safety researchers did the MIRI conversations and also sort of debated at events. They were helpful and I would be excited about having more, but I think there are at least three obstacles to identifying cruxes:

• Yudkowsky just has the pessimism dial set way higher than anyone else (it's not clear that this is wrong, but this makes it hard to debate whether a plan will work)
• Often two research agendas are built in different ontologies, and this causes a lot of friction especially when researcher A's ontology is unnatural to researcher B. See the comments to this for a long discussion on what counts as inner vs outer alignment.
• Much of the disagreement comes down to research taste; see my comment here for an example of differences in opinion driven by taste.

That said, I'd be excited about debates between people with totally different views, e.g. Yudkowsky and Yann Lecun if it could happen...

The debate on this subject has been ongoing between individuals who are within or adjacent to the EA/LessWrong communities (see posts that other comments have linked and other links that are sure to follow). However, these debates often are highly insular and primarily are between people who share core assumptions about:

1. AGI being an existential risk with a high probability of occurring
2. Extinction via AGI having a significant probability of occurring within our lifetimes (next 10-50 years)
3. Other extinction risks (e.g pandemics or nuclear war) not likely manifesting prior to AGI and curtailing AI development such that AGI risk is no longer of relevance in any near-term timeline as a result
4. AGI being a more deadly existential risk than other existential risks (e.g pandemics or nuclear war)
5. AI alignment research being neglected and/or tractable
6. Current work on fairness and transparency improving methods for AI models not being particularly useful towards solving AI alignment

There are many other AI researchers and individuals from other relevant, adjacent disciplines that would disagree with all or most of these assumptions. Debates between that group and people within the EA/LessWrong community who would mostly agree with the above assumptions is something that is sorely lacking, save for some mud-flinging on Twitter between AI ethicists and AI alignment researchers.

There was a prominent debate between Eliezer Yudkowsky and Robin Hanson back in 2008 which is a part of the EA/rationalist communities' origin story, link here: https://wiki.lesswrong.com/index.php?title=The_Hanson-Yudkowsky_AI-Foom_Debate

Prediction is hard and reading the debate from the vantage point of 14 years in the future it's clear that in many ways the science and the argument has moved on, but it's also clear that Eliezer made better predictions than Robin Hanson did, in a way that inclines me to try and learn as much of his worldview as possible so I can analyze other arguments through that frame. 

What's directly relevant is not the level of existential risk, but how much we can affect it. (If existential risk was high but there was essentially nothing we could do about it, it would make sense to prioritize other issues.) Also relevant is how effectively we can do good in other ways. I'm pretty sure it costs less than 10 billion times as much (in expectation, on the margin) to save the world as to save a human life, which seems like a great deal. (I actually think it costs substantially less.) If it cost much more, x-risk reduction would be less appealing; the exact ratio depends on your moral beliefs about the future and your empirical beliefs about how big the future could be.

I'd also be very interested in the reverse of this. Is there anyone who has thought very hard about AI risk and decided to de-prioritise it?

I think Transformative AI is unusually powerful and dangerous relative to other things that can plausibly kill us or otherwise drastically affect human trajectories, and many of us believe AI doom is not inevitable. 

I think it's probably correct for EAs to focus on AI more than other things.

Other plausible contenders (some of which I've worked on) include global priorities research, biorisk mitigation, and moral circle expansion. But broadly a) I think they're less important or tractable than AI, b) many of them are entangled with AI (e.g. global priorities research that ignores AI is completely missing the most important thing).

Thank you for asking this! Some fascinating replies!

A related question:

Considering other existential risks like engineered pandemics, etc., is there an ethical case for continuing to escalate the advancement of AI development despite the possibly-pressing risk of unaligned AGI for addressing/mitigating other risks, such as developing better vaccines, increasing the rate of progress in climate technology research, etc.?

[I'll be assuming a consequentialist moral framework in this response, since most EAs are in fact consequentialists. I'm sure other moral systems have their own arguments for (de)prioritizing AI.]

Almost all the disputes on prioritizing AI safety are really epistemological, rather than ethical; the two big exceptions being a disagreement about how to value future persons, and one on ethics with very high numbers of people (Pascal's Mugging-adjacent situations).

I'll use the importance-tractability-neglectedness (ITN) framework to explain what I mean. The ITN framework is meant to figure out whether an extra marginal dollar to cause 1 will have more positive impact than a dollar to cause 2; in any consequentialist ethical system, that's enough reason to prefer cause 1. Importance is the scale of the problem, the (negative) expected value in the counterfactual world where nothing is done about it - I'll note this as CEV, for counterfactual expected value. Tractability is the share of the problem which can be solved with a given amount of resources; percent-solved-per-dollar, which I'll note as %/$. Neglectedness is comparing cause 1 to other causes with similar importance-times-tractability, and seeing which cause currently has more funding. In an equation:

$CE{V}_1\ast (\%/\${)}_1>CE{V}_2\ast (\%/\${)}_2?$

Now let's do ITN for AI risk specifically:

Tractability - This is entirely an epistemological issue, and one which changes the result of any calculations done by a lot. If AI safety is 15% more likely to be solved with a billion more dollars to hire more grad students (or maybe Terence Tao), few people who are really worried about AI risk would object to throwing that amount of money at it. But there are other models under which throwing an extra billion dollars at the problem would barely increase AI safety progress at all, and many are skeptical of using vast amounts of money which could otherwise help alleviate global poverty on an issue with so much uncertainty.

Neglectedness - Just about everyone agrees that if AI safety is indeed as important and tractable as safety advocates say, it currently gets less resources than other issues on the same or smaller scales, like climate change and nuclear war prevention.

Importance - Essentially, importance is probability-of-doom [p(doom)] multiplied by how-bad-doom-actually-is [v(doom)], which gives us expected-(negative)-value [CEV] in the counterfactual universe where we don't do anything about AI risk.

$p\left(doom\right)\ast v\left(doom\right)=CEV$

The obvious first issue here is an epistemological one; what is p(doom)? 10% chance of everyone dying is a lot different from 1%, which is in turn very different from 0.1%. And some people think p(doom) is over 50%, or almost nonexistent! All of these numbers have very different implications regarding how much money we should put into AI safety.

Alright, let's briefly take a step back before looking at how to calculate v(doom). Our equation now looks like this:

$p\left(doom\right)\ast v\left(doom\right)\ast (\%/\${)}_1>CE{V}_2\ast (\%/\${)}_2?$

Assuming that the right side of the equation is constant, we now have 3 variables that can move around: $p\left(doom\right),(\%/\${)}_1,$ and $v\left(doom\right)$. I've shown that the first two have a lot of variability, which can lead to multiple orders of magnitude difference in the results. 

The 'longtermist' argument for AI risk is, plainly, that $v\left(doom\right)$ is so unbelievably large that the variations in $p\left(doom\right)$ and $(\%/\${)}_1$ are too small to matter. This is based on an epistemological claim and two ethical claims.

Epistemological claim: the expected amount of people (/sentient beings) in the future is huge. An OWID article estimates it at between 800 trillion and 625 quadrillion given a stable population of 11 billion on Earth, while some longtermists, assuming things like space colonization and uploaded minds, go up to 10^53 or something like that. This is the Astronomical Value Thesis.

This claim, at its core, is based on an expectation that existential risk will effectively cease to exist soon (or at least drop to very very low levels), because of something like singularity-level technology. If x-risk stays at something like 1% per century, after all, it's very unlikely that we ever reach anything like 800 trillion people, let alone some of the higher numbers. This EA Forum post does a great job of explaining the math behind it.

Moral claim 1: We should care about potential future sentient beings; 800 trillion humans existing is 100,000 times better than 8 billion, and the loss of 800 trillion future potential lives should be counted as 100,000 times as bad as the loss of today's 8 billion lives. This is a very non-intuitive moral claim, but many total utilitarians will agree with it.

If we combine the Astronomical Value Thesis with moral claim 1, we get to the conclusion that $v\left(doom\right)$ is so massive that it overwhelms nearly everything else in the equation. To illustrate, I'll use the lowball estimate of 800 trillion lives:

$p\left(doom\right)\ast 800,000,000,000,000lives\ast (\%/\${)}_1>CE{V}_2\ast (\%/\${)}_2?$

You don't need advanced math to know that the side with that many zeroes is probably larger. But valid math is not always valid philosophy, and it has often been said that ethics gets weird around very large numbers. Some people say that this is in fact invalid reasoning, and that it resembles the case of Pascal's mugging, which infamously 'proves' things like that you should exchange 10$ for a one-in-a-quadrillion chance of getting 50 quadrillion dollars (after all, the expected value is $50).

So, to finish, moral claim 2: at least in this case, reasoning like this with very large numbers is ethically valid.

And there you have it! If you accept the Astronomical Value Thesis and both moral claims, just about any spending which decreases x-risk at all will be worth prioritizing. If you reject any of those three claims, it can still be entirely reasonable to prioritize AI risk, if your p(doom) and tractability estimates are high enough. Plugging in the current 8 billion people on the planet:

$p\left(doom\right)\ast 8,000,000,000lives\ast (\%/\${)}_1>CE{V}_2\ast (\%/\${)}_2?$

That's still a lot of zeroes!

Reasonable people think it has the most chance of killing all of us and ending future conscious life. Compared to other risks it is bigger, compared to other cause areas it will extinguish more lives.

Related questions:
-what's the best way to talk about EA on your personal social media?
-what's the best way to talk about it if you go viral on Twitter? (this happened to me today)
-what's the best way to talk about it to your parents and older family members? 
etc. 

I think, kind of, 'templates' about how to approach these situations risk seeming manipulative and being cringey, as 'scripts' always are if you don't make them your own, but I'd really enjoy reading a post collecting advice from EA community builders, communicators, marketers, content writers...etc on their experiences trying to talk about EA in various contexts, and what tends to make those conversations go well or badly. 

(If you'd be interested to produce something like this, I'd be happy to collaborate with you on it)

Here's an excellent resource for different ways of pitching EA (and sub-parts of EA). Disclaimer - I do not know who is the owner of this remarkable document. I hope sharing it here is acceptable! As far as I know, this is a public document. 

My contingently-favorite option:

Effective Altruism is a global movement that spreads practical tools and advice about prioritizing social action to help others the most. We aim that when individuals are about to invest time or money in helping others, they will examine their options and choose the one with the highest impact - whether they are looking to help through donations, career decisions, social entrepreneurship, and so on.

The movement mainly promotes these tools through fellowships, courses, lectures, one-on-one consultations, and discussion groups. It is also a thriving community where we research how we can help others the most, and then do it!

FYI I'm also interested in this. 
I do think it's consistent to be pro-choice and place a high value on future lives (both because people might be able to create more future lives by (eg) working on longtermist causes than by having kids themself, and because you can place a high value on lives but say that it is outweighed by the harm done by forcing someone to give birth. But I think that pro-natalist and non-person-affecting views do have implications for reproductive rights and the ethics of reproduction that are seldom noticed or made explicit.

Richard Chappell wrote this piece, though IMHO it doesn't really get to the heart of the tension.

I've only just stumbled upon this question and I'm not sure if you'll see this, but I wrote up some of my thoughts on the problems with the Total View  of population ethics (see "Abortion and Contraception" heading specifically). 

Personally, I think there is a tension there which does not seem to have been discussed much in the EA forum. 

Here is a good post on the side of being pro-life: https://forum.effectivealtruism.org/posts/ADuroAEX5mJMxY5sG/blind-spots-compartmentalizing

I have thought about this a lot, and I think pro-life might actually win out in terms of utility maximization if it doesn't increase existential risk.

1. See here. (Separating importance and neglectedness is often not useful; just thinking about cost-effectiveness is often better.)
2. No.

[My impression. I haven't worked on grantmaking for a long time.] I think this depends on the topic, size of the grant, technicality of the grant, etc. Some grantmakers are themselves experts. Some grantmakers have experts in house. For technical/complicated grants, I think non-expert grantmakers will usually talk to at least some experts before pulling the trigger but it depends on how clearcut the case for the grant is, how big the grant is, etc.

I might be missing the part of my brain that makes these concerns make sense, but this would roughly be my answer: Imagine that you and everyone in your household consume water with lead it in every day. You have the chance to learn if there is lead in the the water. If you learn that it does, you'll feel very bad but also you'll be able to change your source of water going forward. If you learn that it does not, you'll no longer have this nagging doubt about the water quality. I think learning about EA is kind of like this. It will be right or wrong to eat animals regardless of whether you think about it, but only if you learn about it can you change for the better. The only truly shameful stance, at least to me, is to intentionally put your head in the sand.

My secondary approach would be to say that you can't change your past but you can change your future. There is no use feeling guilt and shame about past mistakes if you've already fixed them going forward. Focus your time and attention on what you can control.

My two cents: I view EA as supererogatory, so I don't feel bad about my previous lack of donations, but feel good about my current giving.

Changing the "moral baseline" does not really change decisions: seeing "not donating" as bad and "donating" as neutral leads to the same choices as seeing "not donating" as neutral and "donating" as good.

Meta:

1. Seems like a more complicated question than [I could] solve with a comment
2. Seems like something I'd try doing one on one, talking with (and/or about) a real person with a specific worry, before trying to solve it "at scale" for an entire class
3. I assume my understanding of the problem from these few lines will be wrong and my advice (which I still will write) will be misguided
4. Maybe record a lesson for us and we can watch it?

Tools I like, from the CFAR handbook, which I'd consider using for this situation:

1. IDC (maybe listen to that part afraid you'll think of yourself as a bad person, maybe it is trying to protect you from something that matters. I wouldn't just push that feeling away)
2. homunculus (imagine you're waking up in your own body for the first time, in a brain that just discovered this EA stuff, and you check the memories of this body and discover it has been what you consider "bad" for all its life. You get to decide what to do from here, it's just the starting position for your "game", instead of starting from birth)

If you haven't come across it, a lot of EAs have found Nate Soares' Replacing Guilt series useful for this. (I personally didn't click with it but have lots of friends who did).

I like the way some of Joe Carlsmith's essays touch on this. 

• Wholehearted Choice and Morality as Taxes.
• Care and Demandingness.

What has helped me most is this quote from Seneca:

Even this, the fact that it [the mind] perceives the failings it was unaware of in itself before, is evidence for a change for the better in one's character.

That helped me feel a lot better about finding unnoticed flaws and problems in myself, which always felt like a step backwards before. 

I also sometimes tell myself a slightly shortened Litany of Gendlin:

What is true is already so.
Owning up to it doesn't make it worse.
Not being open about it doesn't make it go away.
People can stand what is true,
for they are already enduring it.

If X and Y make me a bad person, then...I'm already being a bad person. Owning up to it doesn't make it worse, and ignoring it doesn't make it disappear. Owning up to it is, in fact, a sign of moral progress - as bad as it feels, it means I'm actually a better person than I was previously.

My personal approach:

• I no longer think of myself as "a good person" or "a bad person", which may have something to do with my leaning towards moral anti-realism. I recognize that I did bad things in the past and even now, but refuse to label myself "morally bad" because of them; similarly, I refuse to label myself "morally good" because of my good deeds. 
  • Despite this, sometimes I still feel like I'm a bad person. When this happens, I tell myself: "I may have been a bad person, so what? Nobody should stop me from doing good, even if I'm the worst person in the world. So accept that you have been bad, and move on to do good stuff." (note that the "so what" here doesn't mean I'm okay with being bad; just that being bad has no implication on what I should do from now on)
• It doesn't mean it's okay to do bad things. I ask myself to do good things and not to do bad things, not because this makes me a better person, but because the things themselves are good or bad.
• The past doesn't matter (except in teaching you lessons), because it is already set in stone. The past is a constant term in your (metaphorical) objective function; go optimize for the rest (i.e. the present and the future).

I think You Don't Need To Justify Everything is a somewhat less related post (than others that have been shared in this thread already) that is nevertheless on point (and great).

I think it's okay to feel guilty, shame, remorse, rage, or even hopeless about our past "mistakes". These are normal emotions, and we can't or rather shouldn't purposely avoid or even bury them. It's analogous to someone being dumped by a beloved partner and feeling like the whole world is crumbling. No matter how much we try to comfort such a person, he/or she will feel heartbroken.

In fact, feeling bad about our past is a great sign of personal development because it means we realize our mistakes! We can't improve ourselves if we don't even know what we did wrong in the first place. Hence, we should burn these memories hard into our minds and apologize to ourselves for making such mistakes. Then we should promise to ourselves (or even better, make concrete plans) to prevent repeating the same mistakes or to repair the damages (e.g., eat less or no meat, be more prudent in spending or donate more to EA causes, etc.)

Nobody is born a saint, so keep learning and growing into a better person :)

In no particular order. I'll add to this if I think of extra books.

Precursors to thinking about existential risks and GCRs

• Reasons and Persons (1984) and On What Matters, Vol. 1 (2011)
• The End of the World: The Science and Ethics of Human Extinction (1996)
• The Fate of the Earth (1982)

Nuclear risk

• The Making of the Atomic Bomb (1986)
• The Doomsday Machine: Confessions of a Nuclear War Planner (2017)
• Command and Control: Nuclear Weapons, the Damascus Accident, and the Illusion of Safety (2013)
• A longer list is available here

Biosecurity

• Biosecurity Dilemmas: Dreaded Diseases, Ethical Responses, and the Health of Nations (2017)
• Biohazard: The Chilling True Story of the Largest Covert Biological Weapons Program in the World—Told from the Inside by the Man Who Ran It (2000)
• The Dead Hand: The Untold Story of the Cold War Arms Race and its Dangerous Legacy (2009)^[1]

AI safety

• Smart Than Us (2014)
• The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World (2019)

Recovery

• Feeding Everyone No Matter What: Managing Food Security After Global Catastrophe (2014)
• The Knowledge: How to Rebuild Our World from Scratch (2014)

Space / big-picture thinking

• Pale Blue Dot (1994), Cosmos (1980), and Billions and Billions^[2] (1997)
• Maps of Time: An Introduction to Big History (2004)

Fiction

• The Three-Body Problem (2006)
• The Andromeda Strain (1969)
• The Ministry for the Future (2020)
• Cat's Cradle (1963)

1. ^{^}

   This covers both nuclear security and biosecurity topics.

2. ^{^}

   Especially Chapter 8: 'The Environment: Where Does Prudence Lie?', which contains some remarkable precursors to metaphors and arguments in The Precipice etc.

3. I'm not sure they do as much bespoke advising as Longview, but I'd say GiveWell and Farmed Animal Funders. I think you could contact either one with the amount you're thinking of giving and they could tell you what kind of advising they can provide. 

I think parts of What We Owe the Future by Will MacAskill discuss this approach a bit.

The 80k podcast also has some potentially relevant episodes though they're prob not directly what you most want.

• https://80000hours.org/podcast/episodes/phil-trammell-patient-philanthropy/
• https://80000hours.org/podcast/episodes/will-macaskill-ambition-longtermism-mental-health/
  • Maybe especially the section on patient philanthropy.
• https://80000hours.org/podcast/episodes/will-macaskill-what-we-owe-the-future/
• https://80000hours.org/podcast/episodes/sam-bankman-fried-high-risk-approach-to-crypto-and-doing-good/
  • Some bits of this. E.g. some of the bits on political donations.

If "EA is a question," and that question is how to do the most good, I think Peter Singer will always consider himself an effective altruist.

However, he seems to disagree about whether the answer to that question entails a predominant focus on common longtermist topics. I suspect, while he will always see himself as an EA, it will be as an EA that has important differences in cause area prioritization. For more info, he discusses his views about longtermism here, perhaps captured best by the following quote:

When taking steps to reduce the risk that we will become extinct, we should focus on means that also further the interests of present and near-future people. If we are at the hinge of history, enabling people to escape poverty and get an education is as likely to move things in the right direction as almost anything else we might do; and if we are not at that critical point, it will have been a good thing to do anyway.

I don't see how the Simulation Hypothesis is a counterargument to EA, if you presume everyone else is still as "real" (I.e, simulated at the same level of detail) as you are. After all, you clearly have conscious experience, emotional valence,  and so on, despite being a simulation - so does everyone else, so we should still help them live their best simulated lives. After all, whether one is a simulation or not, we can clearly feel the things we call pleasure and pain, happiness and sorrow, freedom and despair, so we clearly have moral worth in my worldview. Though we should also probably be working on some simulation-specific research as well, I don't see how something like malaria nets would cease to be worthwhile.

It's the standard term used in impact evaluation. See e.g. https://en.wikipedia.org/wiki/Impact_evaluation#Counterfactual_evaluation_designs 

I think that it's very tractable. For example, I estimated that corporate campaigns improve 9 to 120 years of chicken life per dollar spent and this improvement seems to be very significant. It would likely cost hundreds or thousands of dollars to improve a life of one human to such a degree, even in developing countries. There are many caveats to this comparison that I can talk about upon request but I don't think that they change the conclusion.

Another way to see tractability is to look at the big wins for animal advocacy in 2021 or 2020. This progress is being achieved with only about $200 million spending per year (with a lot of it being non-EA money, I think).

I believe they are largely tractable, there's a variety of different intervention types (Policy, Direct work, Meta, Research), cause areas (Alt Proteins, Farmed Animals, Wild animal suffering, Insects), organisations and geographies to pursue them in. Of particular note may be potentially highly tractable and impactful work in LMIC (Africa, Asia, Middle East, Eastern Europe)

I will say animal welfare is a newer and less explored area than global health but that may mean that your donation can be more impactful and make more of a difference as there could be a snowball effect from funding new high-potential intervention or research.

If you are quite concerned about traceability, perhaps you could consider donating to organisations that are doing more research or meta-work to discover more tractable interventions

Either way, it's not entirely clear and highly depends on your philosophy, risk tolerance, knowledge and funding counterfactuals. 

The "alignment problem for advanced agents" or "AI alignment" is the overarching research topic of how to develop sufficiently advanced machine intelligences such that running them produces good outcomes in the real world.

Both 'advanced agent' and 'good' should be understood as metasyntactic placeholders for complicated ideas still under debate. The term 'alignment' is intended to convey the idea of pointing an AI in a direction--just like, once you build a rocket, it has to be pointed in a particular direction.

"AI alignment theory" is meant as an overarching term to cover the whole research field associated with this problem, including, e.g., the much-debated attempt to estimate how rapidly an AI might gain in capability once it goes over various particular thresholds.

Other terms that have been used to describe this research problem include "robust and beneficial AI" and "Friendly AI". The term "value alignment problem" was coined by Stuart Russell to refer to the primary subproblem of aligning AI preferences with (potentially idealized) human preferences.

https://arbital.com/p/ai_alignment/ 

Hi Jason, I'm the author of the aforementioned research into IUDs, artificial wombs, and legislative solutions, which is indeed very cursory. The research is included at the bottom of a [larger draft](https://docs.google.com/document/d/10VL9m-GW2f428WZSEs834kiDrHFxtfPNQzc6ljLwTyc/edit?usp=sharing) of an eventual EA forum post outlining reasons why EAs might oppose abortion and potential interventions in that regard.

The draft's philosophical arguments against abortion are much more mature than its section on potential interventions, partially because I've thought far more about the philosophical component than the practical component, and partially because I've been extremely time-poor since the time I dumped my thoughts into that draft.

I'd be more than happy to hear your thoughts / collaborate on a post on interventions to reduce the rate of abortion. Separating the philosophical post ("why might reducing the abortion rate be an EA priority?") from the practical post ("how could we effectively reduce the abortion rate?") might be an even better way to go. I'm sure commenters would have much to say on both topics :)

I sense the answer is yes. I seem to recall that someone looked into this. 

Also I guess the answer is technically yes since I wouldn't be surprised if some interventions already lower the rate of unwanted pregnancy.

Hi - I'm just curious what the rationale for this would be? 

There has been some very cursory research into things like IUDs, artificial wombs, legislative action etc., but I don't think the author ever finished or published it.

We should discount someone's life based on the probability of them not existing. This is not controversial. (But standard-economic constant-factor-per-year discounting is too crude to be useful.)

I'm not sure we need "published studies" but "proper studies" seem like a great idea. 

1. There's Giving what we can and A happier world. Also Kurzgesagt itself got two large grants for making EA videos, I think they're going to make many more in the future.
2. If you google for "effective altruism podcasts" you can find some. There's also a recent small grants program to start new ones.

Open Philanthropy has given grants to Kurzgesagt, who have made some videos on EA-related subjects. There is also some other EA-aligned YouTube creator whose name escapes me now.

The 80k podcast has wide reach, including outside EA I believe.

So there is some of this. I think part of the answer is that it’s really hard to reach a massive audience while retaining a high-fidelity message.

I think that's because the EA movement is just that, a global movement, not an organization.

You probably also haven't seen Fridays For Future ads, or the FFF podcast.

The EA Survey has info on cause prio changes over time. Summary is:

The clearest change in average cause ranking since 2015 is a steady decrease for global poverty and increases for AI risk and animal welfare.

Holden Karnofsky wrote Three Key Issues I’ve Changed My Mind About on the Open Philanthropy blog in 2016.

On AI safety, for example:

I initially guessed that relevant experts had strong reasons for being unconcerned, and were simply not bothering to engage with people who argued for the importance of the risks in question. I believed that the tool-agent distinction was a strong candidate for such a reason. But as I got to know the AI and machine learning communities better, saw how Superintelligence was received, heard reports from the Future of Life Institute’s safety conference in Puerto Rico, and updated on a variety of other fronts, I changed my view.

Maybe check out the term "light cone".

One common method is the Time trade-off, you can find other common methods at the bottom of that Wikipedia page.

For more details the answers in this thread might be interesting.

Also worth noting that (as far as I know) no EA group only uses QALYs.

For example, GiveWell uses their own researched "moral weights" https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/comparing-moral-weights 

1. As you said, they're (almost by definition) not neglected
2. The media picks topics based on some algorithm which is simply different from the EA algorithm. If that wouldn't be true, I guess we wouldn't really need EA

Magnify Mentoring is also relevant here. 

I wrote a bit about non-STEM inclusivity (list of project ideas, a post on language), and I think there are some active efforts to expand outside of the English-speaking world (things like conferences, translation projects, local and online groups, fellowships, camps, etc.) — but more would be good! 

This doesn't completely answer your question, but you may be interested in this page on CEA's website, particularly the "Our work" section.

Loosely related, but you might be interested in the topic "moral advocacy": https://forum.effectivealtruism.org/topics/moral-advocacy 

It's been a while since I read it but Joe Carlsmith's series on expected utility might help some. 

In roughly ascending order of plausibility: 

• Cluelessness. Maybe we can't knowably affect the future (I find this argument probably the most weak)
• Anthropics/Doomsday argument. It's extremely unlikely that we should observe ourselves as among the 100 billion earliest people, so we should update heavily towards thinking the future is inevitably doomed
• Simulation Hypothesis. (relatedly) If we're extremely certain we're in a simulation, then (under most theories of value and empirical beliefs) we should care more about the experiences of existing beings.
• You might not care. I think this is maybe the strongest argument. If morality is mind-independent, the fact that you don't care about astronomical waste even after careful consideration of all the argument is by itself maybe sufficient argument that it's not a key moral priority (to you).

Why don't EAs support basic research such as lab grown meat* as a means to displace the current state of factory farming?

That's news to me. 😕 Animal Charity Evaluators recommends several charities that promote alternative proteins, such as the Material Innovation Initiative and the Good Food Fund. Although no longer an ACE recommended charity, the Good Food Institute is one of the leading orgs that funds and promotes alternative protein innovation, and is often recommended by EAs.

There's a trend in EA and now more general circles to "offset your carbon footprint" but again this feels like a mitigation/reactionary way of spending your money. I would much rather my money go to nuclear fusion research b/c if it worked out, it would have orders of magnitude more impact than simply mitigating my own carbon footprint

I'm not familiar with carbon offsetting as being a 'trend in EA' - as far as I'm aware the canonical EA treatment of this is Claire's piece arguing against it.

Similarly, if I look at the EA forum wiki page for climate change, every single bullet point is about research, and the first one is 'innovative approaches to clean energy' which includes nuclear.

I'm personally pretty skeptical of lab-grown meat after looking into it for a while (see here, here, and here). I do think some investment into the space makes sense for reasons similar to your argument, but I'm personally a bit skeptical of "I think the science of my current approach doesn't work and will never work but there's a small chance I'm wrong so it might make sense to work on it anyway" as a way to do science.*

My guess is that the future replacement for meat will not look like lab-grown mammalian cells, and if it does, how we get there will look like radically different approaches than the way we're currently doing it (e.g. substantially more AI-assisted).

I do tentatively think something in the general shape of your critique is right, and more effort should be spent mapping out the theory of victory for farmed animal agriculture in general, and for technological solutions in particular (see also).

* (I intuitively think this type of probabilistic hedging mentality is more natural to threat analysis or political wins. This is an empirical question though, and I'm also confused here).

For the purpose of trying this thread, it would be nice to post questions as "Answers" to this post.^[1] Although you're welcome to post a question on the Forum if you think that's better: you can see a selection of those here. 

Not a stupid question! 

1. ^{^}

   The post is formatted as a "Question" post, which might have been a mistake on my part, as it means that I'm asking people to post questions in the form of "Answers" to the Question-post, and the terminology is super confusing as a result.

There is a recent post and talk on Measuring Good Better. I found it really interesting to see how different organizations use WELLBYs, "moral weights", or something different.

The most important principle is to maximize long-run utility. All else follows.

I don't think there's any single good definition of "highly engaged EAs". Giving What We Can lists 8,771 people that have signed their pledge https://www.givingwhatwecan.org/about-us/members 

Is there a good article and Youtube video that summarizes arguments for/against meat eating? Ie objective analysis that goes through the best arguments for all sides

I really liked this one https://slatestarcodex.com/2019/12/11/acc-is-eating-meat-a-net-harm/ , it's not perfect, and I personally strongly disagree with their estimates of animal suffering, but I don't know of anything better.

If the status quo you're defending was considered an extreme opinion, would you still argue for it?

I don't think EA is arguing for the status quo, could you elaborate a bit?

Definitely not an expert, but I think there is still no consensus on "slow takeoff vs fast takeoff" (fast takeoff is sometimes referred to as FOOM).

It's a very important topic of disagreement, see e.g. https://www.lesswrong.com/posts/hRohhttbtpY3SHmmD/takeoff-speeds-have-a-huge-effect-on-what-it-means-to-work-1 

Doesn't money also have diminishing marginal returns for helping other people so EAs should be somewhat risk averse when earning to give?

Personally, since it's very unlikely that I will become a mega-billionaire, I think that my contributions will be so marginal that returns can be considered linear. But you might be interested in https://forum.effectivealtruism.org/topics/risk-aversion 

Many people of sound mind choose assisted suicide in their old age and advanced illness.

The biggest challenge for humanity is to bring out the best in itself.
If we cannot inspire the public to be their best, we will not be able to get politicians, corporations, and governments to act in humanity's best interest.
This may not be as hard as some might think. "Just do it", "Keep calm and carry on", and the Can Do spirit are examples of how we inspire people to do better. Safe driving is an example of how we can promote altruistic and cooperative behavior on roads around the world. See my post for more. https://forum.effectivealtruism.org/posts/7srarHqktkHTBDYLq/bringing-out-the-best-in-humanity

Surviving the next 100 years and building a wonderful future.

Do you have a specific example in mind?

From what I see, there are many different kinds of EA grantmakers, and they seem to be using different processes, especially in the longtermist vs neartermist space.

I don't think there's a single general answer to "how do grantmakers update on expert judgment".

Do you have an example in mind? I don't think there are specific norms besides the standard forum norms.
I would say relevant quotes are: "assume good faith", "don't mislead", and "mistakes are expected and fine (please accept corrections, though)"

I think the name "karma" comes from reddit.com, caught on and was used by other internet forums to give a name to the "fake internet points" that users get for posting.

It's definitely not an accurate reflection of a persons input into discussions on the forum, but having a positive karma is a strong indicator that a user is not trolling/spamming.

Note that unilaterally using gene drives in this way is usually considered a really bad idea because of poisoning the well against further use. Not just by usual conservative bioethics types, but by the scientist who first proposed using CRISPR to affect wild populations like mosquitos: 
"Esvelt, whose work helped pave the way for Target Malaria’s efforts, is terrified, simply terrified, of a backlash between now and then that could derail it. This is hardly a theoretical concern. In 2002, anti-GMO hysteria led the government of Zambia to reject 35,000 tons of food aid in the middle of a famine out of fear it could be genetically modified. Esvelt knows that the CRISPR gene drive is a tool of overwhelming power. If used well, it could save millions of lives, help rescue endangered species, even make life better for farm animals.

If used poorly, gene drives could cause social harms that are difficult to reverse. What if gene drives get a bad rap? What if an irresponsible scientist moves too fast and prompts a strong political countermovement, like that which has stymied other genetically modified organisms since the 1990s? What if an irresponsible journalist — let’s call him Dylan Matthews — writes a bad article that misconstrues the issue and sends the project off the rails?

“To the extent that you or I say something or publish something that reduces the chance that African nations will choose to work with Target Malaria by 1 percent, thereby causing a 1 percent chance that project will be delayed by a decade, the expected cost of our action is 25,000 children dead of malaria,” Esvelt tells me. “That’s a lot of kids.”"