Frontpage posts - Effective Altruism forum viewer

A simple argument for trying less hard by Elias Schmied

Elias Schmied — Sat, 13 Jun 2026 18:13:16 +0000

People often make arguments against “trying hard” (working very hard, pushing yourself to the brink, being intensely goal-directed, and so on) by pointing to the risks of burnout or of losing some kind of wholesomeness^[1].

But there’s another, very simple argument against it that I have not seen anyone fully make explicit^[2], even though I think it’s very important. It goes like this:

We face a lot of uncertainty about the sign of our impact.
Therefore, we should be very vigilant about our epistemics to make sure that we are not having a negative impact in expectation.
But trying hard deeply distorts our epistemics—it makes us more prone to motivated reasoning about what we’re doing, and leaves us with less slack to reflect on it.
Therefore, all else being equal, we should try less hard.

Crucially, this argument applies much more strongly to people working in “longtermist areas”—which other critiques of trying hard generally don’t do. For example, global health EAs whose terminal value is short-term welfare also face uncertainty about the impact of their actions—but much less (especially about the sign) than people trying to improve the long-term future. So this argument suggests that it’s especially dangerous for longtermists to try very hard.^[3]^[4]

I’ll go through the steps in a little bit more detail.

Uncertainty

Much has been said in EA about cluelessness and crucial considerations, but I’ll highlight a few specific concerns that could make lots of current AI safety work^[5] net negative (with no claim to novelty):

AI governance interventions are obviously high-variance: bad regulation can easily make things worse, many interventions could increase the risk of great power conflict, increased political polarization around AI could be really bad, more centralization of power increases authoritarianism risk, and so on. And technical work can have flow-through effects on these variables that outweigh its direct effects.^[6]
Activist work can polarize people against the cause.^[7]
Human takeover might be worse than AI takeover, and many AI safety interventions effectively attempt to make human takeover more likely relative to AI takeover.^[8]
If powerful AI will be well-described as doing humanlike roleplaying, trying to control it could make it eventually dislike its “oppressors”, or make it less “mentally healthy” in some way. And even without that assumption, AI safety work could lead to an adversarial relationship with AI in other ways.
Future AIs may be moral patients themselves, which would substantially reduce the value of preventing human extinction, and increase the downside risk (including S-risk) of “AI control”-style interventions.
Useless work could contribute to “safety-washing” or a false sense of security.
There’s cultural concerns around scale, professionalization and “mainstreaming”^[9]- decreases in integrity and epistemic virtue could be very bad for achieving good outcomes.
Capabilities externalities could accelerate AI progress, which many think is bad—people have raised this worry about RLHF historically, and raise it about interpretability and evals nowadays. Most infamously, AI safety activity, to varying extents, contributed to the foundings of all three of DeepMind, OpenAI and Anthropic.

These are very difficult to evaluate (and underargued here). My point is not that these are all valid worries—I’m skeptical of several of them. But I don’t think they can be neglected.

Epistemic distortion

In the past, I’ve felt a sense of being overwhelmed at all these considerations, and felt tempted to just avoid thinking about them—but that can’t be the answer. We have to take the uncertainty seriously. Even if I don’t currently have the capacity to go into some kind of deep reflection, I should attempt to make my actions as robust to the uncertainty as I can—for example, by making sure I can course-correct, and by keeping my epistemics in good shape.

Unfortunately, trying very hard conflicts with this—the harder we push toward a goal, the more we bend the evidence to justify it, and the less mental room we have left to step back and question what we’re doing.^[10]

And I think there’s something stronger, too—in an active inference framework, beliefs and desires are both just expectations about the world. Experientially, this rings true to me—the feelings of frustration at not getting what I want and at being taken off guard by something I hadn’t even been paying attention to are very similar. It seems deeply hard to distinguish between what we want and what we believe.

This is a bit more speculative, but sometimes I think people don’t fully absorb this point: It’s not psychological, it’s neurological. There’s a sense in which wanting anything distorts us away from pure self-supervised prediction of the world and compresses us internally into living in a specific hypothesis—a “gut-level” vision of the world, that gets upweighted on the level of our base perceptions. So we may not be able to fully adjust for it by only manipulating psychological factors, e.g. consciously trying to be more objective or less selfish.

Conclusion

To be clear, I have a lot of respect for people who try extremely hard—I wouldn’t be able to do it, I’m often in awe of them.^[11] I’m also not trying to make a statement about how strong the update from this should be (I don’t even have a good enough knowledge about these spaces to have a precise sense of how hard various groups of people are trying). Maybe arguments for trying even harder actually outweigh this on the current margins, I wouldn’t know.

But I have the sense that this simple consideration is underrated, and I hope this post can provide a reference point for it and make people take it into account in their personal deliberations.

^
Also, some apparently seminal academic philosophy stuff that seems interesting: Moral Saints by Susan Wolf, and Bernard Williams’ work.
^
The closest thing to it is probably the strain of thought around What should you change in response to an “emergency”? And AI risk and Slack gives you space to notice/reflect on subtle things—but that still seems centered on the MIRI-style mindset of (strawmanning here) “more AI safety is definitely good and we just need to think hard to find “true” AI safety work” (which isn’t really my mindset). That is, the uncertainty about what to do comes more from their specific inside-view that alignment is very hard, rather than model-agnostic EA/philosophy-style cluelessness. So I think the argument in this post is a more general one that should be convincing to more people (nowadays, there are obviously a lot of non-MIRI-cluster people who are trying extremely hard on AI safety stuff, e.g. this post that I saw recently).
There’s also the “Maximization is perilous” angle, but that’s more about naive optimization in general, and not about facing huge uncertainty specifically (e.g. it applies equally to global health EAs).
Also, shoutout to Slack matters more than any outcome for a personally inspirational framing on related issues.
^
Which is a little counterintuitive, because we usually see the opposite—longtermist EAs being more intense. Although longtermists also have a bigger moral scope and often more urgency (vis-à-vis AI timelines), so may reasonably trade off more sharply against other values and personal well-being.
^
The same argument also works for virtue and general emotional health, but that’s out of scope for this post.
^
Of course, AI safety interventions are extremely heterogenous—but that just increases the extent to which individual decisionmaking is crucial (as opposed to deferring to people).
^
Holden Karnofsky: “Most things that touch policy at all in any way will move us along that spectrum in one direction or another, so therefore have a high chance of being negative [...]
And then most things that you can do in AI at all will have some impact on policy. Even just alignment research: policy will be shaped by what we’re seeing from alignment research, how tractable it looks, what the interventions look like.” (h/t Anthony DiGiovanni)
^
Holden Karnofsky: “there’s also a lot of micro ways in which you could do harm. Just literally working in safety and being annoying, you might do net harm. You might just talk to the wrong person at the wrong time, get on their nerves. I’ve heard lots of stories of this. Just like, this person does great safety work, but they really annoyed this one person, and that might be the reason we all go extinct” (h/t Anthony DiGiovanni)
^
Among other things.
^
I associate these with people like Richard Ngo (and here) and Oliver Habryka.
^
This is well-known in psychology. Also, Opus 4.8 wrote that sentence.
^
I definitely don’t want to imply that if only it weren’t for this argument, I would be an extremely hard worker too, haha.

are we locked into P(doom)? by keivn

keivn — Fri, 12 Jun 2026 21:57:24 +0000

starting with the conclusion

earlier this year I was a part of the latest cohort at AI safety camp (AISC). the project I had joined — Investigating assumptions of the doom debate — had a goal to build a probabilistic assumption tree for AI doom arguments.

several members made versions of the tree, but the one that we collectively worked on, started from a root node “Dangerous AI occurs.” one branch — “model capacity”broke down into a dozen subnodes, from inference and reasoning capability, to model optimization. the other branch —“external environment” — was less developed, with only two nodes: “conducive human policy,” and “sustained capacity for development.” now, both of these could reasonably be argued as already certain, making assigning probabilities to them all but meaningless. nonetheless, this was the branch I took on that sent me down a divergent path from the rest of the team: why was the environment being treated as an afterthought?

it reflected something at the core of the canonical AI doom argument: it is fundamentally grounded around the agent — what it can do, what it wants, whether it could become too powerful to control. but the environment, in which harm actually occurs, barely appears, treated as a passive backdrop, instead of as an active component of the risk.

three conditions

my earliest version of the tree started from first principles: what are the necessary conditions for AI to cause existential harm?

I landed on three:

AI is sufficiently capable
it cannot be stopped
the harm cannot be recovered from

these three conditions would eventually survive dozens of iterations complementing a complete paradigm shift in how I thought about P(doom).

going down the beaten path

I dug into the canonical AI safety concepts: misalignment, instrumental convergence, orthogonality, goal specification, capability thresholds. mapping this to a tree quickly became a convoluted mess. every decomposition led to questions about the definition of the terms we were using—what counts as a goal? does indifference count as misalignment? which capabilities does a system need to be considered dangerous?

our trees were sprawling with over a dozen nodes full of these conceptual claims rather than verifiable ones. it hit me: the mapping wasn’t at an impasse because the questions were hard, but because we hadn’t even agreed on what we were talking about.

taking a step back, I assessed what kind of claim each node was making, and categorized them:

empirical claims are about the current state of the world; true or false; verifiable (e.g. “nuclear weapons exist”)
conditional claims depend on future events; a forecast not an observation; estimable but carry uncertainty (e.g. “AI capability will exceed human-level within a decade.”)
definitional claims are concepts stated as facts; not verifiable so you can’t meaningfully assign a probability to it (e.g. “Sufficiently capable AI will develop instrumental goals.”)

the canonical doom argument relies heavily on definitional claims treated as empirical ones. but probability estimates on definitional claims are just intuitions cosplaying in mathematical structure. we cannot build a meaningful probability tree until we’ve resolved what terms mean. the definitions are a prerequisite.

coming to terms

the root node of the canonical tree says ‘dangerous AI occurs,’ but dangerous how, to what? unpacking it revealed that the terms are hierarchically nested: AI optimizes, optimization affects something, that something is human welfare. ‘dangerous’ becomes ‘harmful’ once you specify what it’s harmful to.

“human welfare” encompasses three conditions that we want to preserve or improve: human life, the environment that sustains human life, and what human life has materially built (infrastructure, civilization). these are the targets of optimization, and the things through which harm, at doom scale, is actualized.

“harmful” is then defined relative to humans’ ability to stop or reverse adverse effects on human welfare.

instead of being a property of the AI, it’s a relational property between the optimization and the corrective capacity of the environment it’s operating in.

so under these definitions, harmful AI already exists — recommendation algorithms erode the social contract. autonomous trading bots create financial fragility. finite resource extraction is scaling to civilization levels for AI infrastructure.

the doom debate is often framed as some future risk from AGI or superintelligence, but the risk from AI is already here, which is why we need to talk about trajectory.

configuration, not agency

I decided to ask a different question.

the canonical approach asks: what properties must AI have to be dangerous? this got us stuck in debates about capability thresholds, goal specification, alignment definitions.

what if instead we start from: what makes an environment vulnerable to catastrophic harm, and how does AI push toward those conditions?

AI dropped from the top of the tree to the bottom. it went from being an assumed dangerous agent to being one mechanism among many that could push environmental conditions toward doom.

danger is not a property of the AI, but rather a property of the configuration between optimization and environment.

the tree became pathway agnostic—it didn’t care whether harm came from a misaligned superintelligence, a misused system, or the compound effects of thousands of deployed bots with complex interactions.

the canonical argument treats the external world as a passive recipient of the agent’s actions. the environment is where harm happens, but it’s altogether missing from the formalization. ignoring it means ignoring most of the actual risk surface, which is in itself dynamic and antifragile.

the formalization

E — environment, internally coupled such that changes propagate to K

K — dangerous conditions, a subset of E

K₀ — current state of K at time t

K+ — K worsening

t — point in time at which K and R₀ are sampled

R₀ — rate of change of K at t (current trajectory)

Rc — rate at which corrective capacity can negate K+

O — optimization, preference ordering of actions to change E

M — K is mutable (accessible, changeable)

C — O sufficient to mutate K (directly or through E)

DAI = AI where K+ | O > R₀ — optimization accelerates worsening beyond current trajectory

Lock in = K+ > Rc — worsening outpaces corrective capacity

Doom = DAI ∧ Lock in

deliberate design decisions:

E is internally coupled. optimization doesn’t need to target K directly. O could change some other part of E and have effects cascade to K.
R₀ already includes existing AI effects. it’s the current rate of change of K, including all existing pressures. DAI asks whether additional optimization makes R₀ worse.
Lock in is agent-agnostic. it’s purely about whether the rate of worsening exceeds corrective capacity, regardless of source.

from threshold to trajectory

the formalization initially included θ, an irrecoverability threshold. doom occurred when K crossed θ, but any threshold is debatable.

to solve this I reframed P(doom) as a trajectory you can’t get off.

at any snapshot in time, dangerous conditions have a state and a rate of change. AI is dangerous if it accelerates that rate. lock-in is when corrective capacity can’t keep up. the question becomes purely about whether or not the worsening is faster than the correction (and if that gap is widening). we’re looking at the probability of a condition instead of an event.

P(Doom) = P(M) × P(C) × P(Lock in)

the tree

Doom — D = DAI ∧ Lock in

├── AI increases P(Doom) — DAI = K+ | O > R₀

│ ├── Dangerous conditions are changeable — M

│ └── Optimization sufficient to change them — C

└── Lock in — K+ > Rc

what this resolves

agnostic of intent — if optimization makes K worse, it’s DAI regardless of why
agnostic of architecture — single agent, swarm, aggregate effects
agnostic of specific capabilities — C asks whether optimization can move K, not which capabilities enable it
no debatable thresholds

what remains open

attribution — we can’t empirically isolate AI’s contribution to the rate of change from other pressures (e.g. climate, geopolitics, etc) that moves K.
this is meant for making a confidence estimate, not a forecast.

the app

this formalization was implemented as an interactive tool with two modes:

single scenario
- select a risk vector (misaligned, misuse, emergent)
- select a substrate (human life, sustaining environment, infrastructure)
- set two snapshots in time with confidence sliders for each node
- see P(doom) at each snapshot, which node moved most, and a trajectory chart
all trajectories
- all nine configurations are displayed simultaneously in a matrix
- click any cell to adjust its sliders
- an envelope chart shows the collective trajectory as a band
  - bottom edge is the most optimistic scenario
  - top edge is the most pessimistic
  - median line through the middle
  - band width shows how much your assumptions matter
  - band height shows how bad things are

the tool can be found here at: https://kvncaldwll.github.io/AISC-P-Doom-Tool/

connecting the dots

my first stab at mapping to the tree was the three conditions: sufficiently capable, cannot be stopped, cannot recover from the harm. dozens of iterations later, those three conditions survived, and became more precisely defined. “sufficiently capable” became M ∧ C — dangerous conditions are changeable and optimization is sufficient to change them. “cannot be stopped” and “cannot recover” merged into lock-in — the rate of worsening outpaces corrective capacity.

everything, from the definitions audit, the environment-first inversion, the dangerous configuration framing, the trajectory reframe — was the process of making those three intuitions rigorous.

developed during AISC, cohort 11, Project 19 - Investigating assumptions of the doom debate. the environment-first framework, formalization, notation, and tool are independent contributions. the project context and team conversations informed the development.

Is Polysemanticity the Way Forward? by Ololade

Ololade — Fri, 12 Jun 2026 20:06:27 +0000

Disclaimer : I used ChatGpt in restructuring ideas, improving flow, clarifying arguments, strengthening transitions, and making the overall piece more persuasive while preserving my core message.

Artificial intelligence researchers often describe polysemanticity as one of the biggest obstacles to understanding how neural networks work. In modern neural networks, a single neuron can represent multiple unrelated concepts simultaneously—a phenomenon known as polysemanticity. This creates a challenge for interpretability because it becomes difficult to determine exactly what role a neuron plays within a model.

Traditionally, interpretability research has viewed polysemanticity as a problem to be solved. But what if we are looking at it the wrong way? What if polysemanticity is not merely an obstacle, but a necessary feature for building more efficient and capable AI systems?

Understanding Polysemanticity and Superposition

To understand polysemanticity, we first need to understand the idea of features.

Features are properties or patterns in data that a neural network learns to recognize. These features can be represented in different ways. Some are dense, while others are sparse. Sparse features can be thought of as vectors containing mostly zeros, meaning that different features rarely overlap or interfere with one another.

When features are sufficiently sparse, neural networks can exploit a phenomenon known as superposition. Rather than assigning one neuron to one feature, a network can represent multiple features within the same neuron. Researchers at Anthropic describe superposition as a strategy that allows linear representations to encode more features than there are available dimensions. In a sense, the network behaves like a much larger network than its actual size.

This efficiency comes at a cost: features no longer correspond neatly to individual neurons. As Chris Olah famously described it, superposition is “the enemy of interpretability.”

Why Interpretability Researchers Worry About Polysemanticity

The concern is straightforward. If a neuron can represent several unrelated features, understanding the function of that neuron becomes significantly harder.

This difficulty has important implications for AI safety and alignment. Many interpretability approaches attempt to understand models by identifying which neurons correspond to specific concepts. If neurons are polysemantic, this mapping becomes unreliable.

From this perspective, polysemanticity appears to be a major problem.

Why Polysemanticity May Not Be the Real Safety Problem

While polysemanticity undoubtedly makes interpretability more difficult, it is worth asking whether eliminating polysemanticity would actually make AI systems safe.

Modern frontier language models contain billions of parameters and neurons. Even if every neuron were perfectly monosemantic—representing only a single feature—it would still be practically impossible to inspect every neuron individually.

One alternative is to work backwards: provide inputs and observe which neurons activate. However, this approach assumes that we know all possible harmful or undesirable inputs in advance. In reality, we do not.

Furthermore, understanding the function of individual neurons does not necessarily imply understanding the behavior of the system as a whole. Neural networks are complex systems whose capabilities emerge from interactions among many components. Safety failures may arise from these interactions rather than from any single neuron.

Therefore, while polysemanticity complicates interpretability, the absence of polysemanticity does not guarantee safety. The challenge of understanding large-scale AI systems remains.

The Scaling Problem

One reason modern language models are so large is that they must learn an enormous number of features from data.

Language itself contains vast amounts of structure, context, syntax, semantics, and world knowledge. Capturing these patterns requires massive networks with billions of parameters. The same architectures are now being applied beyond language to reasoning, vision, scientific discovery, and other domains.

This approach has been remarkably successful, but it is also expensive. Training and deploying frontier models requires substantial computational resources, energy, and time.

As models encounter more features than they have neurons available to represent, polysemanticity naturally emerges. Rather than treating this as a flaw, we might consider it an efficient compression strategy.

What If We Intentionally Designed for Polysemanticity?

Suppose we embraced polysemanticity rather than fighting against it.

If a single neuron can reliably represent multiple features, then fewer neurons may be required overall. Smaller networks could potentially achieve comparable performance while consuming less memory and computational power.

This raises an interesting question: what is the representational capacity of a neuron, and can it be increased?

An analogy can be found in embedding spaces. Consider an embedding with only a single dimension. Such an embedding would struggle to capture meaningful relationships between concepts. As the dimensionality increases, the amount of information that can be represented also increases.

Similarly, if we can develop architectures that encourage neurons to store and process richer combinations of features, we may be able to build more compact models without sacrificing capability.

Features as Directions

Modern representation learning suggests that features are often better understood as directions in a vector space rather than as individual neurons.

A classic example comes from word embeddings, where relationships such as:

V(“king”) − V(“man”) + V(“woman”) ≈ V(“queen”)

can emerge naturally.

Concepts like gender and royalty correspond to directions within the embedding space. Even many supposedly interpretable neurons can be viewed this way: the activation of a neuron corresponds to movement along a particular direction in the network’s representation space.

If features are fundamentally directional rather than neuron-specific, then the expectation that every neuron should represent exactly one concept may be misguided.

Conclusion

Polysemanticity is often framed as a problem because it makes neural networks harder to interpret. However, interpretability and efficiency are not always aligned goals.

The reality is that modern AI systems already operate at scales where neuron-level understanding is insufficient for guaranteeing safety. At the same time, the demand for increasingly capable models continues to push computational requirements upward.

Rather than viewing polysemanticity solely as an obstacle, we might also view it as a powerful mechanism for compression and efficient representation. If future research can harness polysemanticity intentionally and reliably, it may offer a path toward smaller, more efficient models that retain much of the capability of today’s massive systems.

The question is not whether polysemanticity makes interpretability harder—it clearly does. The more interesting question is whether the benefits of polysemanticity outweigh those costs, and whether it represents a fundamental principle of efficient intelligence rather than an inconvenient flaw.

References

Olah, C. Interpretability Dreams.
Roger, F. The Translucent Thoughts Hypothesis.
Anthropic. Toy Models of Superposition.
Alexander, S. God Help Us, Let’s Try To Understand AI Monosemanticity.
Distill. Zoom In: An Introduction to Circuits.

Join Robin Hanson Book Q&A: The Age of Em by Keishi Kimura

Keishi Kimura — Fri, 12 Jun 2026 20:05:10 +0000

We’re excited to announce our June book discussion featuring The Age of Em: Work, Love, and Life When Robots Rule the Earth by Robin Hanson.

Here’s our schedule:

Discussion Meeting: We’re offering two sessions to discuss The Age of Em on Tuesday, June 23rd (1:00–2:00 PM ET or 8:00–9:00 PM ET).
Q&A Session with Robin Hanson: Monday, June 29th at 4:00-5:00 PM ET. This event will not be recorded.

Register here by Friday, June 19 at 11:59 PM ET to participate in both events.

About Pathways to Progress: We’re a community committed to understanding and contributing to human prosperity—examining scientific and technological innovation, economic development, and their roles in advancing progress. Each month, we read a selected book followed by a Q&A with the author. Previous reads include Pieces of the Actionby Vannevar Bush, Where’s My Flying Car? by J. Storrs Hall, and Stubborn Attachments by Tyler Cowen. We also host speaker events with guests like Jason Crawford, Matt Clancy, and Saloni Dattani. Most speaker events are recorded and available on YouTube and Spotify.

Surplus: for massive public good by Austin

Austin — Fri, 12 Jun 2026 17:20:46 +0000

Surplus is an incubator for software startups, organized by Manifund and Mox — to create massive public good in the age of transformative AI. It’s a 3 month program, starting late July in SF. We provide seed funding, advice, peers, intros, and space to focus.

“Surplus” is the value created through positive-sum trades; what markets produce in abundance.

Apply here by June 24!

Projects we’re excited for

We’re open to many proposals, but here are three categories of projects we’re particularly well-suited to incubate. If your idea is adjacent — apply anyway!

1. AI for epistemics and coordination

LLM-powered tools that help people think better, work together, and build common knowledge.

Examples include:

Fact-checking like Community Notes or Pangram
Guardian angels/”Digital Twins”, as described by Gwern
Knowledge bases like Grokipedia or Longterm Wiki
Research reports like Elicit or Deep Research
AI forecasting like Mantic or Futuresearch
AI simulation of human preferences, like Simile or Aaru
AI for dispute resolution — community investigations; prediction-market criteria
AI for democratic resilience, resisting authoritarianism

2. Public-facing websites

Many important concepts could be translated for a wider audience, with thoughtful design and an eye for virality.

Examples include:

Microsites like AI 2027, showcasing concepts through narrative & interactive design
Visualizations like Epoch and Our World in Data
Transparency for what’s happening inside labs, and across the AI supply chain
Courses like Bluedot, helping people upskill into relevant domains
Demos like Nicky Case’s, or of topics from recent alignment research papers
Games like Manifold or Universal Paperclips, teaching concepts through play

3. Community infrastructure

Marketplaces or platforms, addressing common problems in EA, AI safety, and others working towards a beautiful future.

Examples include:

Job platforms like the 80k Job Board or an “AI safety LinkedIn”
- Or other opportunities — fellowships, grassroots political coordination
Funding platforms like Manifund or an open-source S-process
- Or whitelabeled microgrant platforms like ACX Grants or Bluedot Rapid Grants
- Or AI-powered grantmaking and review
Writing platforms like LessWrong, EA Forum, or Alignment Forum
- Or research platforms like Distill or Unjournal

What we offer

$100k in investment, as a SAFE at a $2m post-money cap
- (Maybe a grant if you’re a committed nonprofit, but we’ll try to argue you out of this)
Work alongside a cohort of ~10 founders who care about xrisk and flourishing futures
Weekly office hours and mentorship
Dinners with speakers — shared meals with founders we admire
Office space at Mox, in San Francisco
Demo Day with aligned VCs and philanthropic funders

Timeline

June 12: Applications open
- Apply via this form
June 24: Applications due
- Rolling video interviews; decisions by July 1
July 27: Program kickoff
- 2 weeks of ideating & cofounder matching
- 10 weeks of mentorship, dinners with speakers
Oct 16: Demo Day
- Pitch to an audience of angels, VCs and philanthropic funders

Virtues we cherish

We’re seeking founders who are:

Loving, wholesome, earnest
Humble, called to serve, takes out the trash
Caring, obsessive, dedicated to craft
Fast, iterative, gets things done
Idealistic, optimistic, dreamers
Scrappy, resourceful, practical
Open, honest, works in public
Naughty, humorous, spirited
Honorable, trustworthy
Bountiful, always project-ing
Scope-sensitive, econ-brained

FAQ

Why does Surplus encourage for-profit corps?

First, there are many standard reasons to use a for-profit corporation when trying to do good. For-profits operate with tight feedback loops. They can be more certain that they produce value (see: gains from trade, Paul Graham on wealth, “surplus”). They can tap into a much larger pool of available financing. They compensate founders and employees with high upside upon a successful exit, and thereby draw in better talent.

For-profit models are surprisingly flexible: Elicit, Apollo, Goodfire, Wave, Dwarkesh, Lighthaven and Manifest all demonstrate different approaches to making money while also serving the public interest.

Now is an excellent time to start a for-profit, given vast torrents of funding available from Anthropic employees and OpenAI Foundation. These funds are distributed out of 501c3 entities — but 501c3s can pay for for-profit services, and invest in for-profit corps. There’s a $100B market waiting to be constructed; shovels waiting to be sold.

And ideologically, we think that equity is a beautiful mechanism for value alignment and credit allocation. Manifund has previously experimented with impact certificates to bring this concept to the charity world; now, we think that plain ol’ corporate equity will work fine, maybe with a light sprinkling of retroactive funding or prize rounds or advance market commitments to finance public goods.

Why start a startup, rather than join a lab or an AI safety org?

It is absolutely the case that Anthropic or METR are great places to work. But maybe:

You’re well suited towards starting projects: you enjoy independence, ship fast, update quickly and are willing to fail
You think that establishing new orgs is good for accountability and avoiding groupthink, as a counterweight to frontier labs accumulating talent and money
You have an idea that you can’t stop thinking about, something you think will be great for the world, something that nobody else is doing (or worse: somebody is doing, but badly)

Why should I join an incubator, when vibecoding is so easy?

Building great software takes more than coding. Product taste, visual design, distribution, sales and marketing are all things that 2026 LLMs still fail at. We’ve developed these supplementary skills needed to ship successful products, and would love to foster them in a new generation of founders.

Beyond advice & mentorship, Surplus also provides a cohort of folks working on similar problems, some of whom may be great cofounders. And finally, an incubator is a container for focus, a commitment device, a way to hold yourself accountable and get your idea out into the world.

Do you accept non-software startups?

Maybe? We have the most experience on startups with a major software component, but have also built things like Manifest and Mox. Apply if you wish!

Is Surplus open to students?

Yes! If accepted to Surplus, we do ask that students plan to take a leave of absence in the fall, or otherwise prepare so you can work on your startup without distractions.

Can Surplus provide visas for international founders?

Yes! We can support accepted founders on a J-1 visa, through Mox. See Mox’s J-1 Global Expert Fellowship.

Will the biosecurity interventions succeed in a resource-limited setting like Nigeria in the event of an engineered pandemic? by Nnaemeka Emmanuel Nnadi

Nnaemeka Emmanuel Nnadi — Fri, 12 Jun 2026 16:59:25 +0000

TL;DR

Many biosecurity interventions assume that if we create effective defences to manmade pandemics, society will be able to employ them when necessary. However, this assumption is mostly predicated on factors prevalent in high-income countries, including strong institutions, public trust, dependable healthcare systems, robust surveillance networks, and appropriate finance. In this paper, I suggested that military models designed and assumed to function in Western countries may not be applicable everywhere, particularly in resource-constrained environments. The environment in which the intervention will be deployed is equally important and should be considered during development. I suggest that every intervention pass the Biosecurity Environmental Fit (BEF) test to establish the degree of compatibility between a biosecurity intervention and the institutional, infrastructural, economic, and sociocultural context in which it is deployed.

Background

It is no longer news that as AI system capabilities grow, the risk of misalignment increases. According to Andrew Snyder-Beattie’s article, manufactured diseases will provide more than 99% of existential dangers to humans, whereas natural sources may pose fewer than 1%. According to the paper, the space for possible human attacks is endless, yet the possible portals of entrance into people are finite, highlighting the need for more research to produce low-cost interventions that may be deployed more widely to construct powerful defences. I agree with this principle. However, reading the article from Nigeria with an understanding of the socio-cultural dimensions raises a question that I rarely see discussed in biosecurity circles: Why do we think that defence models created and anticipated to succeed in Western countries will be applicable everywhere?

The Engineered Pandemic Playbook May Fail in Nigeria

In management theory, Michael Porter’s Diamond Model explains why some countries become extremely competitive in specific industries. His primary notion is that success does not result solely from a strategic approach. It results from the interaction of a strategy and the context in which it is implemented. A model that works in one country may not function in another due to differences in the underlying conditions. I believe biosecurity may face a similar difficulty. Many proposed remedies against manufactured pandemics presume the existence of circumstances common to high-income countries:

High levels of trust in public institutions.
Effective disease surveillance systems.
Access to healthcare is nearly universal.
Reliable diagnostics infrastructure.
Effective risk communication.
High adherence to public health guidelines.
Regulatory agencies that have adequate resources.
Complete digital and lab reporting systems.
Funding for implementing policies and interventions

These assumptions are often hidden because they are inherent in the milieu in which many biosecurity researchers live and work. Nigeria has a varied operating environment.

An engineered epidemic will not only affect Nigeria’s formal health system. It would deal with Nigeria’s transportation networks, marketplaces, religious institutions, informal healthcare providers, social conventions, government systems, and economic realities. Healthcare is divided among public, private, informal, and traditional providers.

In Nigeria, disease surveillance coverage is patchy. Approximately 80% of Nigeria’s healthcare facilities are located in cities, leaving rural communities with very limited and uneven access. While many rural regions may have a Primary Health Care (PHC) centre, only around 20% of these facilities are considered functional, equipped, and staffed to offer acceptable care. As a result, any solution should consider how to accommodate these rural residents. Cross-border mobility is extensive.

Much biosecurity thought is based on a plausible assumption: if we create the correct interventions, society will be able to use them when a catastrophic biological catastrophe occurs. What if the difficulty is the context in which the intervention is supposed to be effective?

Some factors that may prevent these Interventions from working

The Infrastructure Issue

Diagnostic capability, genomic sequencing, disease surveillance, laboratory networks, emergency response systems, logistics, and healthcare infrastructure are among the skills, infrastructure, institutions, and resources required for success in biosecurity. Nigeria, however, is home to top-notch scientists and public health experts. However, expertise is insufficient on its own. There are some questions that must be addressed:

Would an engineered pandemic be detected rapidly and early?
Would genomic surveillance track transmission in real time? Considering the fact that most of the part of the city lacks centralised wastewater system for metagenomic sequencing.
Would diagnostics reach remote communities fast enough?
Would medical countermeasures be distributed at scale?
Do we need to build a private system independent of the government to distribute these interventions to shield them from politics?

Quietly, many biosecurity models presume that these capabilities already exist.

2. The Trust Problem

Trust can decline when scientific advice conflicts with cultural, political, or economic interests. Interventions developed in a pandemic response depend on something fragile: trust.

Will citizens believe official guidance?
Will communities accept restrictions?
Will they accept PPEs once they are developed?
Will they invest in buying air filters, or can we develop a cheap air freshener that can be used in the rural communities?
Will misinformation spread faster than public health messaging?

COVID-19 showed that even wealthier countries battled with these issues. Nigeria faces an additional challenge: public trust is unequal and varies greatly among areas, communities, and institutions. An intervention that works on paper can fail if the population does not believe the messenger.

3. The Missing Ecosystem

No industry succeeds solely. The importance of helping countries prepare for a pandemic cannot be overstated. The development of allied services is critical, such as a robust pharmaceutical production system , cold-chain logistics, diagnostics companies, data systems, transportation networks, telecommunications infrastructure, and research institutes. During a pandemic, reaction capacity is dependent on all of these sectors operating concurrently. However, many biosecurity talks focus on the virus rather than the ecosystem required to adapt to it.

A sequencing laboratory is valuable. A sequencing facility linked to dependable logistics, timely reporting systems, diagnostic networks, and public health agencies is transformational.

4. The Hidden Assumption of Global Biosecurity

Reading Andrew Snyder-Beattie’s work on catastrophic biological risk made me question if biosecurity has a hidden assumption. The assumption does not involve pathogens. It’s about countries. Many proposed defences presume a state with robust infrastructure, high institutional trust, integrated supporting systems, and the ability to organize a quick reaction. Nigeria is not unusual in this respect. Much of the Global South experiences comparable constraints. The question isn’t whether biosecurity initiatives work.

The question is, where do they work? An created epidemic would not face an abstract defence system. It would come across the reality of roads, clinics, markets, governance, trust, electricity, telecommunications, informal healthcare networks, and human behaviour. These realities vary greatly from one country to another. And unless biosecurity begins to account for these disparities, we risk designing defence models that are optimum for Boston, London, and Geneva while ignoring Lagos, Kano, Kinshasa, and Dhaka. The next synthetic epidemic won’t care where our models were created. It will only determine whether they are effective where they are most needed.

The Biosecurity Environmental Fit (BEF) test

Therefore, I propose that interventions be developed to pass the biosecurity environmental fitness test. In this situation, the intervention is assessed to determine its compatibility with the institutional, infrastructural, economic, and sociocultural environment in which it is implemented.

Take wastewater metagenomics as an example. While this technique works effectively in resource-rich countries due to the centralized wastewater infrastructure, it fails in countries such as Nigeria due to the lack of centralization. We require more deployment realism in biosecurity planning. As a result, the next generation of pandemic defence must incorporate contextual biosecurity.

I therefore propose this hypothesis that the expected impact of a biosecurity intervention is a function not only of its technical efficacy but also of its environmental fit.

A perfect intervention with poor environmental fit may produce less benefit than a less sophisticated intervention with strong environmental fit.

Rational Animations is a 501(c)(3) nonprofit and is looking for board members by Writer

Writer — Fri, 12 Jun 2026 16:47:40 +0000

TL;DR: We’re a US 501(c)(3) nonprofit (EIN: 99-1838037) making animated videos about AI safety and other topics. The IRS made its tax-exemption determination in October 2024. Our current three-person board is all staff and two of us are romantic partners, which is not ideal. We want at least two independent board members (not current staff and not expecting to be). Expect 10-30 hours per year of effort. The work consists mainly of quarterly meetings, high-level approvals of strategy and budget, review of company performance, and decisions where staff have a conflict of interest, such as compensation. If interested, email business@rationalanimations.com with the subject “Board - <Your Name>”.

Who we are

Rational Animations is a fully remote nonprofit studio producing animated videos about various topics, with a strong focus on AI safety. At the time of writing, the YouTube channel has 450k subscribers and 32 million views. RA was co-founded by Emanuele Ascani (me) and Michela Biancini, who has also been my romantic partner since 2016.

Current board:

Emanuele Ascani—President & Treasurer
Michela Biancini—Board Member
Kristy Steffens—Secretary

All three of us are employed by Rational Animations.

Why we’re expanding the board

We’d benefit from additional board members in two main ways:

All current board members are also staff and two of us are partners. We need unconflicted directors for compensation and other COI-sensitive decisions.
Additional strong judgment about how best to pursue our mission and conduct our operations.

What you’ll do

Here are some things you’ll likely do as a board member. These aren’t set in stone, and we’ll figure out the best ways to coordinate as we go:

Quarterly 30-90 min remote meetings plus occasional short async votes.
Review and approve high-level strategy, key policies, and our performance as a company.
Handle COI-sensitive decisions (e.g., compensation of employed board members).
Occasionally coordinate informally about organizational strategy via Discord or email.

We expect the total effort to be about 10-30 hours in a typical year, more if something unusual demands the board’s attention. This is an unpaid position with a renewable 1-year term.

Note on responsibilities: directors have the standard fiduciary duties of care and loyalty (informed decision-making, acting in RA’s interest, and recusing when conflicted). We carry Directors & Officers (D&O) insurance covering board members for claims arising from their board service.

Who might be a good fit

Mainly, we’re looking for two things:

You are not currently employed by Rational Animations, and do not expect to be in the future.
Strong signals include involvement in AI safety, Effective Altruism, or media production, as well as experience with non-profits (e.g., board membership, governance, finance, or ops roles).

How to apply

Just email business@rationalanimations.com with the subject “Board - <Your Name>”. We’d like to know about your background and why you’re interested in being a board member. Links to your LinkedIn profile, a personal site, or CV are welcome.

Add Free $16 To Donations by Tyler Kolota 🔹

Tyler Kolota 🔹 — Fri, 12 Jun 2026 16:45:06 +0000

Noticed this week that https://Every.org has a few small promotions for different activities on their platform. If one signs up for a new account or has an account they haven’t used in a while, I suggest logging in, liking 3 donations/posts from others on effective orgs, & setting up a $10 monthly donation to an effective org directly from a bank account. After like 2-3 months of recurring donations you’ll have directed $20-$30 in donations, but only spent like $4-$14 from your bank account.

How bad would it be if GPS satellites were shot down? by Jackson Wagner

Jackson Wagner — Fri, 12 Jun 2026 16:38:35 +0000

Losing GPS isn’t an X-risk, but would create a huge disaster on the scale of Covid-19 or bigger.

Hi! From 2020 − 2023 I was one of the early employees at Xona Space Systems, a company working on essentially a next-generation version of GPS. I ended up learning a lot about how GPS works and what it’s used for, and (due to my personal interest in effective altruism) ended up doing some research into what would happen if today’s GPS systems suddenly failed. This post is the product of that research. I discuss:

What could kill GPS: it would be a tempting early target in a war between superpowers, or it could possibly be taken down by superhuman AI’s cyber-hacking capabilities. On the bright side, it’s probably safe from even very large solar storms.
If GPS was destroyed, how bad would this be? I describe all the major areas in civilian life that would be disrupted (mostly summarizing some government reports).
Then I try to crunch some very rough, vague numbers. It looks like losing GPS would (by itself) be an economic hit to the USA perhaps equivalent to a few billion dollars a day. On a very broad, vibes-based level, this might feel approximately as disruptive and frustrating as the Covid-19 pandemic, albeit it would be a disaster with a very different character. Of course, GPS probably wouldn’t be destroyed in isolation, but in the larger context of a terrible and destructive great-power war, so everything would in fact be much worse.

This post got too long, so I split it in half. If people like this one, then I hope to finish up a second post wherein I’ll discuss:

GPS isn’t just for civilian life! The whole reason it would be such a tempting target in warfare is that it also has extreme military usefulness. I describe the various military functions of GPS (JDAMs, yes, but also they were originally quite important for atomic warfare), and speculate a little bit about the military strategy of who wins versus loses if all the GPS systems get blown up. Trying to quantify this side of things is too hard, so I don’t.
There are assorted things we could do as a civilization to better mitigate the risks of losing GPS. I list a few of them. The US military is mostly but not completely on the ball here, IMO. Boosting the resilience of civilization’s access to navigation signals definitely isn’t, like, a new EA cause area or anything. But nevertheless I think the dynamics around GPS might be useful for to know about, for people who are thinking about issues related to great-power competition, geopolitics, nuclear war, AI-2027 style takeoff scenarios, etc.

“Whoa, GPS comes from space?? I thought it was just a thing in my car…”

a GPS III satellite under construction

The Global Positioning System is a constellation of about thirty large space satellites created by the United States Air Force in the 1980s. These satellites provide very accurate position, navigation, and timing (PNT) information to anyone with a GPS receiver. It does this by broadcasting super-accurate timing signals—each satellite has no less than four atomic clocks on board—which your GPS radio receiver can use to triangulate your position. For much more on the technical details, check out this unbelievably well-made and interactive explainer website (which is now part of the onboarding experience for all new hires at Xona!)

The constellation looks like this

GPS was originally created for its military utility, which we’ll talk more about later. But it’s most notable today for its usefulness throughout many areas of civilian life. Beyond enabling countless everyday smartphone apps like google maps, uber, etc, it’s also crucial for all sorts of transportation and logistics tasks, construction, shipping, precision agriculture, etc. Basically any industry where lots of physical stuff moves around, is an intensive user of GPS. Plus, unexpectedly, the cellphone network and power grid are also significantly dependent on the super-accurate timing signals from GPS!

Overall, GPS is a pretty big deal—the detailed NIST study described here assessed that, in 2017, GPS was directly contributing about 1.5% of US GDP compared to an alternative scenario where GPS was never created and everybody had to get their positioning/timing information some other way. The world economy is getting more GPS-intensive all the time; Claude thinks that the most reasonable way of extrapolating the data in that report would give a figure of about 2% − 3% of GDP in 2026.

And that’s “counterfactual value contributed by GPS versus a world where we never built it”—by contrast, the value that would be destroyed by suddenly losing GPS in real life would be significantly larger. This UK study tries to look at how bad a GPS disruption would be, and (once you translate all the results to account for the fact that the UK’s economy is much smaller than the USA’s) it foresees an economic impact from suddenly losing GPS that’s perhaps 5x − 10x bigger than NIST’s reckoning of the counterfactual benefits of the system. So that all adds up to maybe, like, an impact equivalent to 15% of the GDP of all rich-world countries? Maybe more?? Seems bad!

To be clear, there are multiple satellite-navigation constellations

So far I’ve just been saying “GPS”. But, because of the importance of GPS for both military and civilian life, any superpower worth its salt wants to have its own satellite-navigation constellation (“GNSS constellation”) under its sovereign control. The original US system is called GPS; Russia’s system is called Glonass; Europe’s is Galileo, China’s is Beidou. Japan and India also have partial satellite-navigation systems, each consisting of around seven satellites that enhance the accuracy of satellite navigation signals over Japan and India respectively. But in this post I’ll just talk about GPS, because:

The US and Chinese systems are the most militarily relevant, both because those satellite constellations themselves have the highest performance / the most advanced military features, and also because those countries have the most advanced militaries generally.
The US system is the most economically significant by a wide margin—a lot of receivers (especially older ones that might be built into critical infrastructure) only listen to GPS, even though it’s perfectly possible to build a receiver that listens to GPS, Galileo, and Glonass at the same time (eg, your phone can probably pick up all three, using Galileo and Glonass signals to slightly refine the accuracy of its position data).
Many of the scenarios that could cause GPS to fail (hacked by advanced AI, shot down in a war between superpowers, unprecedented solar storm) could easily lead to several of these systems failing at once (eg, a Chinese or Russian attack on GPS might also target Galileo, and the US might then retaliate by taking down Beidou and Glonass). The USA’s system is also probably the toughest to kill—it’s likely more radiation-hardened than Galileo and others, likely the most cybersecure of all the constellations, plus the US military would probably be better able to defend it than China or Russia could defend their systems, etc.
Since original GPS is both the toughest to kill, and the most essential for the world economy, and most crucial for maintaining the current global balance of power, I am going to mostly keep talking as if there is only one GPS system.

My former employer, Xona Space Systems, has an IMO innovative and credible plan for a kind of next-gen GPS system that would be different in many ways from traditional GPS—stronger signals, higher precision, more satellites, lower orbits, with each individual satellite much smaller and cheaper than traditional GPS sats (somewhat like a Starlink or OneWeb constellation, but for navigation). Such a system, and the fact that it’s being developed by a private company trying to turn a profit rather than a superpower military seeking battlefield advantage, means that Xona’s system will have some unique properties compared to traditional GPS constellations. But Xona’s constellation doesn’t actually exist yet, so we’ll mostly ignore it for now—I’ll talk more about it in the section on potential risk-mitigation steps in Part 2 of this essay.

What could kill GPS?

There are lots of *local* threats to GPS—for example, Russia and Ukraine do extensive electronic warfare that jams GPS signals throughout much of eastern Ukraine. More mundanely, occasionally stuff will happen like the time some trucker installed an overpowered GPS jammer in his truck to block the GPS tracking devices put in the truck by his employer, and the trucker accidentally shut down the entire port of Newark when the jammer started interfering with harbor operations. GPS signals are easily jammable because they are very weak. (They’re actually *quieter* than the ambient background radio noise on earth’s surface, which seems like it ought to make the signals literally impossible to detect! But it’s barely doable using some fancy math / radio magic, explained in the later parts of that aforementioned interactive website). It’s also possible to “spoof” GPS signals, broadcasting a fake signal that can be used to steer unsuspecting drones, planes, and ships off course (or to cheat at Pokemon Go). ^[1] But these are all local issues, certainly not civilization-threatening.

Also, sometimes individual GPS satellites fail (for normal “aerospace engineering is hard” reasons) and need to be replaced. But the GPS system is designed with redundancy in mind, and could even survive the failure of several satellites at once.

But how could the entire GPS system could be brought down? There seem to be three main possibilities:

Great-power war

As I describe later on, GPS was originally created by the Air Force because it has a variety of incredibly valuable military applications. If you’re a superpower and you want to launch a big surprise attack on another superpower, disabling their GPS satellites would be an aggressive but potentially appealing move to include in your opening strikes.

The most obvious way to destroy a GPS constellation would be to launch precision anti-satellite missiles capable of reaching all the way up to Medium-Earth Orbit (MEO, about 20,000 kilometers above the earth’s surface) where such satellites live. This requires bigger missiles than are needed for taking out Low-Earth-Orbit spy satellites (which orbit below 500 kilometers), but is perfectly within the capabilities of both the USA and China. Today’s GPS satellites do not really have defenses against any kind of physical attack, nor could they likely maneuver quickly enough to dodge such a missile.

As an alternative to missiles, you could launch your own satellites that could maneuver alongside GPS satellites and sabotage them in various ways. This has various pros and cons versus missiles:

It’s not exactly stealthy (basically, everything in space can be tracked by radar all the time) but it’s more ambiguous / deniable / hard-to-attribute than a missile launch.
On the downside, a co-orbiting satellite is more expensive and finicky than missiles.
An attack by co-orbiting satellites might take years to set up, but you could then hope to disable all GPS satellites simultaneously at the push of a button, whereas a missile-attack campaign would have to unfold over many hours, probably days.
An attack by co-orbiting satellites could avoid creating a destructive shrapnel cloud (which might even harm your own navigation satellites!) and could potentially be reversible.

China, the USA, and Russia do all sorts of cloak-and-dagger shenanigans with co-orbiting satellites all the time, so this kind of thing is definitely an existing technology.

Perhaps the least-destructive and least-aggro option would be to launch your own satellites capable of simply blaring out radio noise on GPS frequencies, jamming GPS indefinitely on a continental or potentially worldwide scale. Russia apparently already has this capability; perhaps China and the US do as well. Of course, if you started doing this, other superpowers might then try to shoot down your jamming-satellites using missiles, but that would be a big escalation.

Fortunately, all these methods of attack are limited to countries with advanced space & missile programs, so it is not like any random tiny rogue nation can threaten to destroy GPS.

Hacking by superhuman AI???

People sometimes worry about whether it’s possible to hack GPS and disable the satellites. The usual response to this is something like “Cyberattack is always possible in principle, but GPS is a hardened military system resistant to intrusion, thus would require great resources that perhaps only the strongest cyber powers possess, and maybe not even then.” But nowadays with Claude Mythos discovering troves of zero-day exploits in countless pieces of critical software, this scenario has probably become more relevant! This method has some unique aspects compared to the more kinetic approaches described above:

It’s more deniable and harder-to-attribute than anything described above.
It’s a less aggro move than any of the options above, so if you had this capability, you might feel there was a lower threshold for pulling the trigger on using it.
Depending on how access to AI hacking capability shakes out, it could potentially be pulled off by smaller powers who lack advanced space and missile programs.
It’s the kind of thing a rogue AI could possibly do by itself even if it didn’t have much in the way of real-world resources.

Although satellite navigation systems pride themselves on their extremely high uptime and reliability, there have been a variety of mundane bugs and screwups over the years resulting in things like the wrong timing data being sent from GPS for a few hours, or even a weeklong outage in Europe’s Galileo constellation in 2019. So, it is certainly not inconceivable that GPS could be at least temporarily disabled by sophisticated hacking. One hopes that Project-Glasswing-style AI cyberdefense initiatives will be able to stay ahead of any such hacking attempts.

An unprecedented solar storm, maybe

Solar storms produce a lot of radiation that can fry satellites. Fortunately, navigation satellites are already extremely radiation-hardened, far beyond what’s required for most satellites, for two reasons:

They are already designed to spend decades hanging out in MEO, the most toxically radioactive part of earth orbit.
GPS satellites were originally designed to operate through an atomic war (more on this later), including nuclear weapons being detonated in space and generating EMP effects that would fry most normal satellites.

Probably the only satellites more rad-hardened than GPS are deep-space NASA missions to Jupiter, like Juno and Europa Clipper. Consequently, per this RAND report, although radio interference from a Carrington-scale solar-storm would likely render GNSS temporarily inoperable for several days, they lean against the idea that it could permanently disable GPS satellites? The report says that ‘an unpublished but publicly disclosed FEMA report from 2010—Mitigation Strategies for FEMA Command, Control, and Communications During and After a Solar Superstorm—found that, in such an event, there was a “possible” loss of enough GPS satellites to reduce the constellation below the 24 usually required, a less-dramatic and less-consequential failure (Emerson, 2017).’ That doesn’t sound too bad, IMO—sounds more like degraded, less-accurate-than-usual, occasionally-patchy capacity rather than the whole system being destroyed.

This EU study seems to concur; they say that the direct effects on ground-based electrical infrastructure from a Carrington Event would be far worse than the effects on satellite infrastructure, such that it’s basically not worth bothering to put effort into solar-storm mitigation for satellites while there’s still so much important work to do on the ground. (And, to reiterate, MEO GPS constellations are more resilient to solar storms compared to all other satellite infrastructure.)

But everyone just anchors on the Carrington event! I wondered: is this a mistake? Are people neglecting the possibility of outlier mega-storms that might be, say, 10x stronger than Carrington, even if they’re 10x less likely? Turns out, not really! Solar storms can get somewhat worse than Carrington, although more powerful storms become exponentially rarer. Per this paper, a storm as bad as the Carrington event or worse has about a once-in-100-years probability, while an event 2x as powerful as Carrington or worse has about once-in-500 years probability, so that’s 2x more powerful but 5x less likely. More importantly, solar storms from stars like our Sun are thought to top out at about 4x as powerful as Carrington—so there isn’t some long tail of increasingly-powerful but decreasingly-likely storms. 4x as bad as Carrington is simply as bad as it gets (and these are incredibly rare).

Other stuff

Another way that a solar storm can harm satellites is that all the extra solar energy puffs up the highest layers of the atmosphere, increasing drag on low-earth-orbit satellites such that they can reenter the atmosphere and burn up. This would be killer for many earth-observation telescopes, among others. But GPS is 20,000km high instead of 400km, so this problem isn’t relevant.
You might be thinking “what about kessler syndrome, where orbital debris from a few satellites snowballs and takes out everything in orbit?” But kessler syndrome is vastly more likely in LEO (where there are thousands of satellites zooming around in a relatively small volume of space a few hundred kilometers above earth’s service) than in MEO (where there are only a few dozen satellites spread over a much vaster area of space). Kessler syndrome in LEO wouldn’t affect MEO—the only way you end up with significant debris danger in MEO is if someone is already deliberately blowing up GPS satellites, in which case your main problem is that you are at war, not that you also have to deal with some space debris.
In the lore of the movie “The Matrix”, humanity at one point does some kind of geoengineering to darken the skies, hoping to starve their robot enemy of solar power. In some kind of Terminator-style robot war where humanity is fighting against rogue AI, I could imagine nations deliberately shutting off GPS to try and disproportionately hurt the robot / drone armies that might be even more reliant on it than humans are. But this would be intentional, thus not exactly a “threat to GPS” per se.

What would break in the aftermath of losing GPS?

This section is based on summary of two helpful and extensive resources:

First, a 2019 study by the NIST, which attempts to tally up the counterfactual benefits of GPS to the economy, sector by sector, from 1984-2017.
Second, a 2021 UK study specifically focused on the effects of a short-term GPS outage of seven days.

The UK study paints a good picture of all the short-term chaos, but not which things would recover versus which would stay broken (or get worse) as an outage dragged on past the seven-day window. The NIST study repeatedly uses the idea of a 30-day outage as a thought experiment, but only in service of their overall goal of estimating counterfactual financial benefits to the private sector. This results in some big discrepancies – the UK report states that the worst impacts would be from gnarled traffic gridlocking major cities and degradation of emergency services, while the NIST report only briefly considers those same issues and doesn’t incorporate them into their overall estimates, instead acknowledging that their numbers are a conservative estimate. Still, by putting the two reports together, a general vision of the potential disaster emerges.

In summary: people’s phones would basically turn to bricks—not just in that Google Maps would stop working, but the actual entire cell network would collapse over a few days. Logistics/deliveries of all kinds (amazon, the postal service, rideshare services, et cetera) would get insanely backed up and stop working, shipping in major ports would similarly grind to a halt, and there would be mild but widespread economic pain in other industries. Traffic chaos would snarl the roads of most cities, at least initially. In the future, the performance of self-driving cars and other autonomous systems would be much degraded, or perhaps stop working altogether.

On the bright side, the power grid would become more fragile but probably not actually collapse. Agriculture would be somewhat impaired but would basically still be able to get the job of growing food done.

Impact on the cell network & smartphones

4G LTE and 5G cellphone networks would go down as cell towers lost precision timing over the first few days. Would this just mean that much of smartphone functionality is crippled, but leave basic texting and calling intact? Or would the intense network congestion (squeezing all our modern cellphone use onto 3G and 2G networks) cause even the most basic services to fail? Unfortunately, nobody knows, since a catastrophic national failure of the cellphone system has never happened before. According to the NIST report, “one expert thought that wireless networks would fail completely after about 2 weeks, while several others thought that some service (most likely voice and text service only) would still remain at the end of 30 days.”
- Internet services are a different story and much less GPS-dependent than the cell network. So, even if the cell network collapsed completely, we’d still be able to connect via wifi—it wouldn’t be a total communications blackout of the sort that towns sometimes experience after hurricanes or earthquakes.
- Obviously, all location-based apps that directly use GPS information would lose their functionality: Google Maps, Uber, searching for nearby attractions/restaurants, etc.

Impact on the power grid

Power networks would be at higher risk of problems, but wouldn’t be instantly thrown into cascading failures:
- Per NIST: “Most published studies conclude that widespread grid failures are not to be expected from a major disruption of the GPS signal (NERC, 2012). The electrical system is highly distributed, and the existing SCADA system could be engaged quickly to serve as an adequate backup system for any GPS-supported functions. This capability reduces the likelihood that a large-magnitude event such as widespread cascading outages would occur. However, the loss of GPS would affect system monitoring operations and effectiveness, leading to a slightly increased probability of adverse events. The impact would be similar to the retrospective scenario (but more short-lived) and would lead to the following impacts over the 30-day GPS outage period: 1. increased time to identify, trace, and mitigate/correct faults; 2. increased probability and duration of small-scale outages; and 3. increased probability (albeit low) of large-scale blackouts. Long-term impacts such as infrastructure damage are unlikely.”
- “As described in detail in the previous sections, the primary way in which utilities are currently employing time stamps is to perform post-event and forensic analyses to understand the behavior of the grid and prevent future equipment and power failures. Beyond time stamps, utilities are also leveraging PMU data to tune up models and develop more accurate generation estimates. With these applications as a backdrop, the main consequences of losing GPS would almost solely be reflected in an increase in the difficulty of managing the system and responding to outages efficiently and in a timely manner. While problematic, the consequences derived from a scenario in which time stamp and modeling applications are not present pose no existential risk to the electric grid. In other words, generators will not trip off, and T&D operations will not stop functioning if GPS satellite communications fail unexpectedly.”

Impact on maritime industries

Maritime navigation would become much harder, such that operations at all large ports would come to a standstill:
- “Increased transit time and increased caution close to shore will result in unexpected delays that have economic impacts associated with late delivery of commodities. However, our analysis indicated that the greatest bottleneck for the import and export of commodities would be the interruption of port logistics. Very quickly cargo ships would be queuing up for days and even weeks as ports are not able to process their containers. From this perspective, the 1 to 2 days of navigation delay due to the loss of GPS becomes insignificant because it does not matter if a ship is a day late arriving at a port if it will need to queue for a week before being unloaded.”
“Commercial fishing would have greatly decreased yields, and many would not attempt to fish.”
- Claude says that wild-caught marine fish supply about 3% of humanity’s protein intake, and 1% of all food calories. ALLFED seems to think it’s not a real food-supply disaster until we face a shock that takes away 5% of calories globally, so even if yields went literally to zero, this would be bad but not a total catastrophe. We’d still have aquaculture, which actually accounts for more than half of all seafood consumed globally.

Impact on travel and logistics

UPS, Amazon, Uber, and essentially all other delivery and taxi companies use GPS directions to direct their employees along optimized package delivery routes each day. In the absence of GPS, drivers and dispatchers would have to rely on local knowledge and static maps. Given the daily volume of packages that they deliver, such a disruption would basically create an instant disaster, particularly if an outage occurs during a busy season.
- An exception to this would be mail services like USPS, plus utilities like trash collection, which run standardized routes (the same routes every day or week) that drivers learn well.
Roads would become significantly more dangerous and much more congested as drivers were forced to consult maps, signs, memorized waypoints, etc, rather than relying on navigation. This would result in snarled traffic in all major cities, major time losses, excess fuel consumption from all the idling cars, etc.
- Not mentioned in either report, but this would naturally get worse in future scenarios where autonomous Waymo-style cars are more common, as precise GPS locations are a key sensor input to most self-driving cars, and losing it would probably render self-driving cars totally unable to navigate anywhere.
911 services would become congested because calls no longer pass GPS data to them. The UK report models that even after surging additional capacity into emergency response, response times would be generally much slower and 3% of 911 calls might simply get dropped.
- The UK study thinks that this is a very big deal: “applications in emergency services, maritime, and road together account for 87.6% of the total economic loss” for their 7-day outage scenario. This is partially because, unlike NIST who are just tallying up economic benefits to the private sector, the UK is making a more utilitarian calculation taking into account the value of life, which surfaces the immense value of the emergency medical services that would possibly collapse in an extended GPS outage. But on the other hand, 7 days isn’t enough time for the cellphone network to fully collapse, so the UK study misses that and other slow-rolling problems.

Impacts on other industries

There would be widespread economic pain in industries like oil & gas, mining, and construction, etc. These wouldn’t be catastrophic, in part because it would mostly impair the construction of new projects, not the ongoing process of pumping oil and extracting ore from existing wells / mines.
Precision agriculture and robotic tractors would be impaired, but farmers would still be able to plant. Nevertheless, agriculture is such an important industry that even a small impairment adds up to a big loss in absolute terms.
There would be myriad smaller covid-esque disruptions to supply chains generally, in addition to the large specific impact on the industries mentioned earlier.
Would the stock market collapse due to lack of precision timing?? Fortunately, they are actually completely on the ball and 100% prepared: “The financial services sector uses GPS to time stamp financial transactions. GPS’s timing capability allows exchanges and trading houses to cost-effectively time stamp every transaction request received in keeping with the precision required by financial regulations. In the event of a 30-day GPS outage, financial markets would need to adjust, but operations would be minimally affected. Most exchanges and sizable trading houses have rubidium or cesium clocks that can provide sufficient holdover to continue operations. Although the financial services sector views the falsification of GPS signals, or spoofing, as a significant concern because it can affect data reliability, sector representatives do not view an observable loss of GPS for 30 days as having a substantial economic impact.”

Reckoning an overall cost per day

Losing GPS sounds pretty bad, but how bad would it be exactly? The best way to compare the risk of GNSS failure to other kinds of disruption is to make an overall estimate of the total economic costs. Fortunately, our two studies do just that. Unfortunately, they disagree significantly.

The 2017 NIST report says that a 30-day GPS outage would potentially cost the US economy around 1 billion per day.
- That estimate applies to the world of 2017, but our economy’s dependence on GPS has been growing very rapidly every year since 2010. We would like to update these estimates for the more GPS-intensive world of 2026.
Meanwhile, the 2021 UK study’s 7-day outage scenario cites a cost to the UK economy of 1 billion pounds per day. This is actually a WAY higher estimate of the damages, since Britain’s economy and population are both way smaller than the USA. If you multiply the 2021 UK study’s estimate by the ratio between US and UK GDP (7.5x), or by the population ratio (5x), you get a predicted impact on the USA’s economy of between 7.5 billion dollars and 11 billion dollars per day.

Why the discrepancy between the NIST report’s 1 billion versus the UK report’s implied ~10 billion?

Well, for starters, the NIST study is from 2017 while the UK report is from four years later. The economy is very rapidly becoming more GPS-intensive. Helpfully, the 2021 UK report is actually a follow-up to an earlier UK report conducted in 2017! They say that “overall, compared to the 2017 iteration of this report, the total economic benefits have increased by 102%, more than doubling in magnitude. A majority of this change is due to increases in the Emergency Services and Road sectors. In each sector an increase in device penetration (smartphones, satnavs, and insurance telematics devices) explain much of the growth.” However, although the benefits have doubled, the costs of GPS outage in their report have only increased by about 6% -- increased outage costs from the fact that society is now much more GPS-intensive are apparently being offset by the fact that some sectors have greater resilience to an outage than they did in 2017 (although looking at the numbers I am not totally convinced—eg they think drivers would be less impaired by a GPS outage today versus in 2017??).
- So maybe we should try to bring the NIST estimate into 2021 by raising it by somewhere between 6% and 102%. (Plus 11% for inflation between 2017 and 2021, I guess.) But this still doesn’t solve most of the gap.
The biggest difference is coming from the fact that the UK report makes something closer to a utilitarian calculation that includes various measurements of human welfare (eg the various costs of dropped 911 calls, lost hours spent in traffic), whereas NIST is just tallying up more direct economic benefits to private businesses. On the other hand, the NIST report’s month-long outage scenario catches potentially serious impacts on the cell network and on agriculture that the UK report’s brief weeklong outage. Here is the NIST report’s table summarizing their outage scenario, followed by a screenshot of some claude analysis comparing the two reports:

Overall, I’m tempted to think that a true picture of the cost of permanently losing GPS would look worse than either report suggests—it’s certainly reasonable to expect emergency services to drop 3% of their calls like the UK report models, but in a longer-than-seven-days scenario it seems crazy to ignore the fact that agriculture and the cell network would be seriously impaired.

On the other hand, a longer outage would tend to see a lower average cost to the crisis in terms of billions of dollars per day. One-time costs can only hit one time, and the longer a crisis dragged on, the longer people would have to find ways of routing around problems and bottlenecks.
So maybe overall, an extended GPS outage in 2021 might cost the US economy something like 2 billion to 5 billion per day?

To bring that estimate five years forward into 2026, we have to wonder if the economy’s ’GPS-intensiveness” has doubled yet again, just like it did in the years 2017 − 2021 (per the UK report), or in the years from 2014 − 2017, or the years 2010 − 2013 (per NIST). Seems plausible that it might have! So that would maybe be like $4B - $10B per day for the US economy in 2026? And the further into the future you imagine losing GPS, probably the worse it gets, since the world economy will continue to get more GPS-intensive via more use of technologies like drones, autonomous vehicles, precision farming, cellphone networks, et cetera.

Hard to say, but this feels approximately Covid-19-scale-ish

As a rough guess, it looks like Covid-19 cost the United States at least 5 billion per day in the year from March 2020 to March 2021:

$1.2 trillion in lost economic activity, comparing 2020’s GDP to the counterfactual case where the economy kept growing at its 2019 pace.
$0.6 trillion in welfare losses from the 750,000 premature deaths the US suffered in the first year of the pandemic. On average, each death robbed its victim of about 8 “quality-adjusted life years” that they might have otherwise lived to enjoy, valuing each QALY at the common value of $100,000.
Other factors are harder to account for: the emotional toll of social-distancing isolation, business and school closures, the drawbacks of falling ill even if you don’t die (from annoying flu-like symptoms to “long covid”), various silver linings like advancements in technology spurred by the urgency of the crisis.
But $1.8 trillion (about $5B/day for a year) seems like a good rough estimate.

Of course, the details of losing GPS would look nothing like the details of the covid-19 pandemic. Instead of roads in major cities lying eerily empty, they’d be paralyzed by gridlock. Instead of relying on computer technology (zoom calls, etc) to adapt to changing circumstances, it would be the unexpected failure of a ton of computer technology that causes constant problems. And while Covid-19 was a fairly even mix of economic and health damage, the loss of GPS services would be a more purely economic hit. (Although the frustration and anger of millions of people lost at confusing intersections and stuck in traffic on snarled roads and unable to get their packages delivered, certainly might rival the difficulties of social isolation and business/school closures of the covid era.)

Like Covid, the loss of GPS would be a worldwide disaster, not something specific to the United States (although perhaps especially intense in rich countries generally, which I expect are the most GPS-intensive economies).

How long would an outage last?

Naturally, outage length would depend on the nature of what killed GPS:

A solar storm or mild cyberattack might result in a mere days-long outage.
Whereas if all the satellites were destroyed by missiles then we’d have to live without GPS for the several years it would take to manufacture and launch replacement satellites.
An intermediate case might occur if one nation’s satellite system (eg, Russian Glonass or European Galileo) had been spared in the fighting, in which case civilian life would gradually return to normal as we upgrade all our old infrastructure with receivers capable of picking up the signals of the surviving constellation.
Conversely, if one of the other non-GPS satellite systems was disabled, this might have world-shaking geopolitical implications in the context of a military conflict, but the economic damage would be pretty limited, since to my knowledge there are only a few non-military systems that rely exclusively on Glonass, Galileo, or Beidou—almost everything can receive GPS plus something else.

Remember: you’re still at war with China, or Russia, or AI, or possibly the Sun!

Keep in mind that GPS likely wouldn’t be going down in isolation — it would probably collapse as part of a larger crisis, either a great-power war, an exceptionally powerful solar flare (which would destroy many other satellites as well, and damage electrical grids on the ground), or perhaps a wave of AI-assisted cyberattacks. So the negative impacts of suddenly losing GPS — a shock to supply chains and the routines of daily life roughly comparable in magnitude to the covid-19 pandemic — would be overlaid on top of the unrelated effects of the larger crisis. For example, the fact that the electrical grid has now become harder to monitor / debug / repair thanks to the loss of precision GPS timing information, might not play well with the fact that your adversary (whether China, AI, or The Sun) is at that very moment probably doing everything they can to sabotage and destroy your electrical grid.

* * *

Okay, thanks for reading! If you liked this post, let me know, and stay tuned for part 2, covering GPS’s military utility and some of the ways that civilization could try to obtain more robust, resilient access to positioning and timing services. You can sign up for my substack, Nuka Zaria, if you want to be notified when I publish it!

^
Xona’s proposed navigation system, with stronger signals and various encryption/verification mechanisms, would help address these local issues somewhat; there are also other solutions that generally involve using backup, non-satellite-based navigation systems.

What new AI safety orgs get wrong about operations in year one? by AartiMahajan

AartiMahajan — Fri, 12 Jun 2026 15:23:34 +0000

Note: I used an LLM as a thinking and drafting aid for this post. The views are my own.

**TL;DR:** The field increasingly agrees that AI safety undervalues founders and operators. I’ve spent 12 years building the operational scaffolding for early-stage non-profits in other sectors, and I’m now learning the AI safety context firsthand. Here are four operational traps that catch new orgs in year one — governance, the “boring” foundations, coordination cadence, and hiring order — and why getting them right frees researchers to do research.

*Writing this to think out loud and invite correction — I’m newer to the AI safety context than to operations, so if I’ve misread how these dynamics play out in this field specifically, I’d genuinely value the pushback.*

I think this matters more than it looks, because of a quiet pattern: research orgs tend to be led by researchers. But founder skills and researcher skills aren’t the same thing. So, in year one, the domain expert often ends up doing operations — struggling, and at the expense of the research only they can do.

I’ve spent 12 years building operational infrastructure for early-stage, resource-constrained organisations in other sectors — from a white canvas, where little or nothing existed. I’m now learning the AI safety context firsthand, working with a new AI safety community as it sets up. I don’t have experience of AI-safety-org war stories. But the year-one operational traps are remarkably consistent across sectors, and they’re worth naming:

1. No governance backbone. No board, no decision rights, no basic policies. It feels premature — until a funder, a hire, or a conflict forces the question and there’s no structure to absorb it.

2. The boring “foundation” gets deferred. As fractional COO at a non-profit fund, I built the compliance and governance scaffolding from scratch — GDPR, AML/KYC policy with external counsel, board packs and decision-tracking. None of it was visible work. All of it was the difference between an org that could legally take money and hire people, and one that couldn’t.

3. No cross-functional cadence. As soon as there’s more than two people, work needs rhythm — who’s doing what, by when, and how it connects. Without it, the founder becomes the single point of coordination for everything.

4. Hiring in the wrong order. Hiring for visible gaps instead of the highest-leverage one — which, for a researcher-founder, is often someone to own operations so they can return to the research.

None of this is exotic. It’s exactly what incubators like Catalyze exist to address — they explicitly help founders with strategy and with finding co-founders who complement their skills. The structural fix is simple to state: let the researcher do research, and bring in someone whose job is to build the organisation around them.

That’s the work I care about. If you’re starting an AI safety org and the operational layer feels like a fog, I’m always happy to compare notes.

Launch a biosecurity group on your campus by sharmaayushmaan

sharmaayushmaan — Fri, 12 Jun 2026 14:08:17 +0000

Disclaimer: Although I work on the Groups Team at CEA, I’m offering this in a personal capacity, and this post does not constitute an endorsement or service offered by CEA.

TL;DR: I’m offering personal mentorship to anyone serious about starting a biosecurity group at a top university. I work on CEA’s Groups team (whilst also being an Epidemiology student at Oxford) and spend most of my time supporting people launching new groups. I think biosec-specific university groups are especially high-leverage right now, and I want to make it easier for more of them to exist.

Why university biosecurity groups

Biosecurity is talent-constrained, especially for more experienced grad students/seniors who might have high entrepreneurial fit to start new orgs or own a particular part of the ecosystem. Funders I’ve spoken to consistently say they could deploy more money if they had the people, especially generalists who can start new orgs. It’s relatively clear that the bottleneck for the community is in talented, agentic people who can start their own initiatives.

AI safety had a version of this problem five or six years ago, and university groups turned out to be one of the most effective interventions for fixing it. A large fraction of new technical researchers and policy hires in AIS over the last few years passed through a university group or an adjacent fellowship at some point. Groups like MIT AIS, Harvard AISST, CAISH and OASI have produced a steady stream of people now working at places like Anthropic, GovAI, and AISI.

Biosec doesn’t have a strong presence on university campuses. Biology and public health undergrads, who are the obvious audience (but not the only one), mostly don’t encounter GCBR or longtermist-flavoured biosec through default coursework. I’m keen to see groups start to shift discourse towards biosecurity being one of the top problems university students state they’d consider working on.

One thing to flag: AIS field-building benefited from the topic becoming culturally mainstream, and biosec might not get the same benefits. But meeting curious students (particularly graduates) where they are and giving them a structured path feels worth a shot. I’m especially excited about students at T20 US universities and students at G5 schools in the UK.

What a group actually does

A successful biosecurity group likely runs a Bluedot biosec course, facilitates 1:1 career conversations, and plugs members into the broader ecosystem, towards orgs like BlueDot Impact, ERA, and CG. There’s been promising evidence from schools (like Oxford, Cambridge, and Harvard) running mentored research programs. I’ve seen especially strong evidence from EA-adjacent groups betting strongly on EAGs and retreats and would be keen to see this too.

Why I’m offering this

I work on the Groups team at CEA, where most of my time goes towards managing and incubating EA groups at top universities in the US and UK. This is the hardest part of running a group, and where most failures happen. The first few months can heavily decide the counterfactual impact of the group.

I’m doing this in a personal capacity. CEA doesn’t have dedicated support for biosec-specific groups right now, and from talking to would-be organisers I think there’s enough demand that one person taking it seriously can have a high tailed impact.

Apply

If you’re interested in starting a group, please send an email to ayushmaan.sharma@dph.ox.ac.uk with the following:

A brief bio & context on your background (especially any EA/GCBR work you’ve engaged with)
Any prior organising experience
Your vision for the group (1-2 lines)

Free human coaching for high-impact folks by Joel Patrizio

Joel Patrizio — Fri, 12 Jun 2026 13:46:08 +0000

TL;DR: Offering you free coaching in my app. We have a pro bono program w/ free spots for folks doing impact work. A real human coach doing daily check-ins plus a monthly video call. You can apply here: https://www.goalswon.com/giving-back/ . AMA in the comments.

Hey I’m Joel. My co-founder Simon Newstead (active in the impact community) and I started as accountability buddies, and realised how much it helped having someone actually trained in holding us accountable so we ended up building GoalsWon around that idea.

We have a pro bono program (for every paying client we take on, we open a free spot for someone in the high-impact community, ie the paid side exists to fund the giving-back side) and over the years we’ve coached hundreds of people in AI safety, animal welfare, and global health and we’re opening more spots now that’s why I’m sharing it here.

How it works: you get a real human coach (not a bot!) doing daily check-ins on your goals (work or personal), plus a monthly video call.

We know a lot of high-impact work fails at follow-through and not strategy, so we’re here to help with the consistent accountability part so it improves your output and keep burnout away as you do all the important work you do.

Our only ask: if you sign up, actually use it. Spots are limited and we want them going to people who’ll show up and make the most of it.

Apply and/or read more about it in: https://www.goalswon.com/giving-back (scroll to the “Apply for a free slot” white box at the bottom.

If you have any questions, happy to answer anything in the comments!

Safe for What World? Why the AI safety field may be asking the wrong question by Benoît L.

Benoît L. — Fri, 12 Jun 2026 13:42:16 +0000

I used an LLM to help redraft some arguments for an EA specific audience and it likely contains ca.10% AI-generated text, but I’ve edited/rewritten it extensively and endorse it.

Epistemic status: The empirical claims rest on institutional sources cited at the end. The central argument, that AI safety research as a field does not ask whether its deployment trajectory is compatible with the physical systems it draws on, is visible in what the field’s research agenda includes and excludes; I treat this as observable rather than inferred. By “AI safety research” I mean specifically the alignment, x-risk, and long-term safety research agenda as the field currently defines it (most recently: Singapore Consensus on Global AI Safety Research Priorities, June 2025), not compute efficiency or energy research that exists in adjacent technical communities but only barely connect to the safety frame. The structural and sociological reasons for this absence are, I will argue, understandable. Understandable does not make them acceptable. How much the framing would change safety research priorities if adopted is genuinely uncertain. I intend to develop a concrete research agenda in a follow-up post, and am actively seeking engagement with researchers and funders willing to work through what that programme would look like in practice.

I spoke recently with an economist who specialises in AI. Thoughtful, well-read, deeply engaged with the question of how artificial intelligence will reshape economic activity. At some point the conversation turned to growth. I mentioned physical constraints. His answer: “Like physical limits? But they at least allow for multifold increases in GDP, no?”

Well, no. Or only if you look at GDP as it is defined: a function of capital and workforce. You accumulate one, you deploy the other, and the economy expands.

I asked where energy came from in that model. A puzzled pause. Where minerals came from. Another pause. The conversation moved on.

This exchange is not a criticism of one economist. It describes a structural absence in how most educated people, economists, technologists, policymakers, safety researchers, were trained to think about economic systems. The dominant framework treats the economy as a financial system that happens to use some physical inputs. Thermodynamics does not appear in the model. The depletion of ore bodies does not appear. The absorption capacity of forests and oceans does not appear. These are externalities in both the technical and the colloquial sense: things that happen outside the system being studied. This is not a recent oversight. It is baked into the intellectual tradition that most working scientists and researchers inherited, largely without being aware of inheriting it.

The AI safety field has inherited this blind spot. It asks serious questions about model behaviour, misuse, and long-term alignment between machine intelligence and human values. What it does not ask is what world this technology is being built for, whether that world has the physical capacity to support it, and whether the physical systems it depends on are stable enough for current safety assumptions to remain meaningful. This piece tries to open that question.

The economy is a physical system

Nicholas Georgescu-Roegen, working in the 1970s, made an observation whose continued absence from mainstream economics is itself a kind of data point: economic processes transform low-entropy resources into high-entropy waste, and the laws of thermodynamics do not negotiate with financial instruments. The specific entropy framing remains contested in parts of ecological economics. The empirical record makes the same point without requiring any theoretical position on entropy. We are extracting renewable resources faster than they renew themselves, degrading ecosystems faster than they regenerate, and emitting pollution faster than natural systems can absorb it. These are measurable rates, tracked by multiple independent institutional sources, pointing consistently in the same direction across several decades. None of this appears in the models that guide investment decisions, technology deployment, or safety research.

The standard production function (GDP as a function of capital and labour) omits the variable that does the actual physical work. Robert Ayres and Benjamin Warr spent years tracking the relationship between useful work, meaning energy successfully converted into economic output, and GDP growth across the twentieth century. The correlation came out at close to 0.9. Not price, not capital investment: the availability of useful energy. The economy grew because energy was cheap, abundant, and increasingly available. The model that describes that growth does not include the variable that drove it.

Global material extraction makes the same point from a different direction. The UN Environment Programme’s International Resource Panel puts annual extraction at 104 billion tonnes in 2023, up from 22 billion in 1970. The economy grew because we found and consumed vastly more physical stuff. Capital and labour organised that consumption. They did not replace it. Experts agree: Eah additional % of GDP requires more mineral extraction than the previous one.

There is a financial dimension that sharpens this. Global debt stood at $348 trillion at the end of 2025, roughly three times global GDP of around $115 trillion according to the Institute of International Finance. Here again, each dollar of new debt generates progressively less GDP than the last. US Federal Reserve data shows this decline running consistently from the early post-war decades, when a dollar of new debt produced more than 70 cents of GDP growth, to well below 10 cents by the time of the 2008 financial crisis. The analysis that popularised these precise figures comes from a non-neutral financial source and the numbers are contested; the direction of the trend in the underlying Federal Reserve data is not. The money being created flows into financial assets and real estate, inflating valuations rather than building productive capacity. The financial system is pricing the future as if the physical constraints described above do not exist.

Three things happening at once

Start with energy. Conventional crude oil production peaked somewhere between 2005 and 2008, according to successive IEA World Energy Outlook editions. What kept total supply figures rising afterward was not new conventional discovery but unconventional sources (fracked tight oil, oil sands, deepwater), each more expensive to extract, each with steeper decline rates than the fields they supplemented. Eighty percent of global primary energy still comes from fossil fuels. Oil, while about a third of the total is still the blood powering all of our economy. The era of cheap, abundant, high-quality energy has been ending for two decades, covered by increasingly costly substitutes.

The ore grades being mined are declining in parallel. In Chile’s copper mines, which hold between 40 and 55 percent of known global reserves, the average grade dropped 28.8 percent in a single decade. At today’s average of around 0.5 percent copper content, roughly 99.5 percent of everything extracted is waste rock requiring permanent management, generating acid drainage, and consuming water and energy in quantities that scale inversely with ore quality. Energy consumption in those operations rose 46 percent between 2003 and 2013 while copper output rose 30 percent. S&P Global puts gold head grades down 13.4 percent since 2012, copper down 7.5 percent across the industry. The direction has been consistent across decades, metals, and continents.

The scale of what a genuine energy transition would require makes this more acute. Simon Michaux of the Geological Survey of Finland calculated in 2021 what replacing the existing fossil fuel system with electrified alternatives would demand in copper, lithium, cobalt, nickel, and rare earths. His specific figures attracted serious criticism: Carbon Tracker argued, with some justification, that he overstated battery requirements by assuming chemical compositions the market was already moving away from. The rebuttal conceded the underlying point, that resource bottlenecks are real, geopolitically concentrated, and not resolved by efficiency improvements on any timeline relevant to a 2050 target. The critics’ own reassuring comparison, that transition minerals weigh 300 times less than the fossil fuels they replace, measures refined metal against fuel stream without accounting for the 200 tonnes of rock that must be mined, crushed, processed, and permanently managed as tailings to produce one tonne of copper at current ore grades. It is a unit comparison, not a physical one. Both Michaux and his critics agree that the mineral extraction implied by the transition is incompatible with a 2050 deadline without fundamental changes to the system being designed. The argument concerned the specific figure, not whether the number is compatible with the deadline.

The third pressure operated quietly for long enough that it barely registered until recently. Forests, soils, and oceans have been absorbing roughly half of human CO₂ emissions for decades, a subsidy provided free by systems we did not build and cannot replace. In 2023, the land carbon sink collapsed to its lowest recorded level since systematic measurement began in 1958, absorbing between 1.5 and 2.6 billion tonnes of CO₂ against a recent average of 9.5 billion. Atmospheric CO₂ grew at 86 percent above the previous year’s rate despite fossil fuel emissions rising by less than one percent. A single year’s data does not establish a permanent trend; the 10 New Insights in Climate Science 2025, produced by more than 70 scientists across 21 countries, concluded that land and ocean systems are approaching the limits of what they can absorb, and that the 2023 signal is consistent with multiple converging indicators rather than a single anomalous year.

These three pressures are not independent. Lower ore grades require more energy per tonne of useful metal. Higher energy use under a still-fossil-dominated grid adds to the load on absorption systems that are already weakening. A destabilised climate damages the water systems and agricultural soils that mining and food production depend on. The problems compound, they do not queue.

Into this world, we are building AI infrastructure at speed

The International Energy Agency’s 2025 report on energy and AI puts data centre electricity consumption growing at 15 percent per year, more than four times faster than all other sectors combined, with AI as the primary driver. Electricity consumed by AI-focused infrastructure alone surged 50 percent in 2025. By 2030, US data centres are projected to consume more electricity than the production of aluminium, steel, cement, and chemicals combined. Natural gas and coal are expected to cover over 40 percent of the additional load through the end of the decade. In material terms, the AI buildout is partly a fossil fuel buildout, landing on a grid that is already struggling to decarbonise.

The hardware side draws on the same mineral base already under pressure from transition demand. Copper, lithium, cobalt, and rare earth elements appear in both the energy transition bill of materials and the AI infrastructure bill of materials. Neither the Michaux calculation nor its rebuttal included AI infrastructure demand. The 2020 Nature Communications study by Sonter et al., which found that without strategic planning, mining threats to biodiversity from renewable energy production alone may surpass those averted by climate change mitigation, was published before the current AI buildout began. Two large-scale industrial demands are now competing for the same geographically concentrated, grade-declining mineral base simultaneously, and no major assessment has modelled them together. This is not a minor gap in the literature. It is a missing calculation that would change the shape of every supply chain and resource governance discussion currently underway.

Computational efficiency has improved substantially and the electricity consumed per AI query has fallen sharply. This does not resolve the system-level picture. The number of tasks, the size of models, and the scale of deployment are growing faster than efficiency gains can offset. The rebound effect, whereby lower cost per unit enables more units, is a well-documented pattern in energy economics and is operating here at scale and speed.

A scenario analysis published in June 2026 by a team of European AI researchers (europe2031.ai) maps the geopolitical and competitive consequences of the AI buildout in considerable detail, projecting global AI compute to 370 gigawatts by 2031. It does not once ask what physical systems that buildout draws on or what it costs the resource base it enters. This is not a criticism of the authors, who are doing exactly what they set out to do. It is an illustration of the inherited frame: rigorous analysis of AI’s trajectory conducted without the physical world appearing anywhere in the model. The absence is structural, not deliberate.

Why this question is not being asked

The AI safety field has produced work that matters. Alignment research, interpretability, governance frameworks, threat modelling: these are real contributions to real problems pursued by serious people. The question of whether AI’s deployment trajectory is compatible with the physical systems it draws on is nevertheless absent from the research agenda. The Singapore Consensus on Global AI Safety Research Priorities, the field’s most authoritative current self-definition, explicitly scopes itself to making AI more trustworthy rather than more powerful, covering general-purpose AI systems and their behaviour. Physical resource constraints, thermodynamic limits, and material supply chains do not appear. This is a data point about where the field currently draws its boundaries, not a criticism of the document itself.

Nobody would design a building without accounting for gravity, not because gravity is exotic specialist knowledge but because it is the physical system the building operates within. The analogous omission in economic modelling, treating the physical world as external to the system being studied, is simply not taught at all. The relationship between economic activity and physical resource systems is a specialist literature sitting at the intersection of ecological economics, industrial ecology, and earth system science. A researcher trained in mathematics, computer science, or analytic philosophy (which describes the majority of the AI safety field) would not encounter it unless they went looking, and there is no obvious reason to go looking for something you have not been told is missing. The blind spot is structural to the education that produced the field.

The field is also young and moving very fast. In any intellectual community, raising an entirely new category of question, one that crosses disciplinary boundaries, resists formalisation in the terms the field currently uses, and does not connect obviously to the model-behaviour problems already on the research agenda, is genuinely difficult. In a community where the core questions are still being established and where capability development creates constant pressure to address immediate problems, it is more difficult still.

Any young, tight-knit intellectual community develops shared assumptions before it has the diversity of background needed to challenge them from within. The EA Forum’s own norms explicitly name the scout mindset, the drive to see what is actually there rather than to defend a position, as a core value, precisely because such communities recognise this risk. Physical resource constraints are exactly the kind of question where the scout mindset, applied consistently and across disciplinary lines, would likely produce different conclusions than the current research agenda reflects.

There is a structural question about research funding worth naming carefully. Open Philanthropy is the dominant funder of AI safety research and simultaneously holds a significant equity stake in Anthropic. This is public information, and I raise it not to impute bad faith, as the people involved are thoughtful and their commitment to addressing AI risk is genuine, but to name a structural condition that exists independent of anyone’s intentions. Questions that would require interrogating the pace and scale of deployment are structurally harder to fund from inside that arrangement than questions about how to make the technology safer once deployed. This is a risk worth naming as a risk, and I would suggest it is also a reason for funders to want this conversation rather than to avoid it. I would welcome a direct engagement with Open Philanthropy and other major funders about whether a research programme in this space is something they would consider supporting.

Three bodies of EA-adjacent work a reader might cite as closing this gap do not quite reach it. The “Great Energy Descent” series (EA Forum, 2022) covers broadly the same physical-limits terrain but predates the current AI buildout and asks whether civilisation survives resource constraints, not whether safety guarantees engineered for a stable world hold in an unstable one. ALLFED and civilisational resilience research asks whether re-industrialisation is possible after a discrete catastrophe; the concern here is gradual pre-catastrophe degradation, not recovery after a triggering event. The accumulative x-risk literature, discussed below, models AI-induced societal instability, which runs in the opposite causal direction. The gap between the 2022 series and the Singapore Consensus of 2025, which still does not include these questions, is itself evidence of what this piece describes.

The questions that are not being asked

There is a literature on accumulative AI x-risk, the idea that AI risks can manifest gradually through compounding societal erosion rather than a single catastrophic event. That work is valuable and closer to this paper’s terrain than most safety research. The critical difference is the direction of causality. The accumulative literature models AI as the cause of instability: incremental AI-induced disruptions erode societal resilience until a triggering event produces irreversible collapse. The question here runs in the opposite direction: what does pre-existing physical instability — instability that precedes and is independent of AI — do to AI safety guarantees that were designed for a stable world? An AI system operating in a world where multiple physical tipping points have already been crossed faces a different risk landscape than one operating in a world broadly continuous with the recent past. That is not a question the accumulative x-risk literature asks, because it treats societal stability as the outcome to be explained rather than the input to be modelled. These are complementary concerns with different research implications. The second has not been addressed.

The AI safety field’s own self-definition, as reflected in the Singapore Consensus and the broader alignment and x-risk research agenda, proceeds from a shared background assumption: that the world into which AI is being deployed is broadly continuous with the world of the recent past. Institutions remain functional. Energy systems remain available. Supply chains remain intact. The physical systems underpinning social and economic stability, while stressed, do not break in ways that fundamentally alter the operating environment for AI systems or the humans overseeing them. This assumption is not stated because it does not need to be stated. It is the water the field swims in, and it is becoming an empirical claim rather than a neutral methodological baseline.

The evidence assembled in this paper, covering ore grade trajectories, energy transition constraints, carbon sink weakening, and the compounding of AI infrastructure demand on top of transition demand, points consistently in the same direction. The question is not whether these trends exist. It is whether safety research that does not account for them is modelling the actual risk landscape or a simplified version of it that may not survive contact with the next decade.

The research this gap calls for has two faces that cannot usefully be separated, because they are the same question approached from opposite ends. First: as the physical systems underpinning economic and social stability weaken, what happens to the risk profile of AI systems operating in that environment? Safety research models AI risk against a stable background. A field serious about long-term safety would also model it against an unstable one. What does misalignment look like when institutions are under severe resource stress? What does concentration of AI capability mean when the physical infrastructure it depends on is geopolitically fragile and supply-constrained? What are the specific failure modes of AI systems deployed into a world where multiple ecological or societal tipping points have already been crossed, not as an exotic scenario, but as a plausible trajectory given current data?

Second: what are the physical requirements of full-scale AI deployment, and are those requirements compatible with the resource base available to meet them alongside the energy transition already underway? This is not a question about marginal efficiency improvements. It is a question about aggregate demand. Nobody has calculated the combined mineral and energy requirements of simultaneous full-scale energy transition and AI infrastructure buildout drawing on the same depleting, geographically concentrated resource base. That calculation does not exist because the field boundaries that would generate it do not exist: energy transition planning and AI deployment planning proceed in separate institutional silos with no shared physical accounting framework. The absence of the calculation is not a weakness in this argument. It is an illustration of it. Producing that calculation is one of the research questions this paper is calling for, not a precondition for calling for it.

I am conscious that a full research programme in this space is beyond the scope of a single post and requires collaboration with researchers who hold technical depth in physical systems modelling, resource economics, and safety research simultaneously. I intend to develop the agenda in a follow-up post. If you are working on adjacent questions, or think the agenda I am pointing toward is misguided, I would welcome the conversation, and am particularly interested in engaging with teams and funders willing to work through what this looks like in practice.

Where the conflict between AI deployment trajectories and physical system capacity actually sits is a precondition for the kind of governance a finite world will eventually require. That question is developed elsewhere. The point here is narrower: the conflict needs to be mapped before it can be governed.

Safe for what world?

An AI system that does not say harmful things, does not pursue misaligned goals, and resists weaponisation is not safe in any meaningful sense if its deployment is accelerating the depletion of the systems that economies and societies depend on, or if the safety guarantees engineered for one world do not hold in a significantly different one.

The world AI is being built for is not stable in the ways the safety research assumes. Its energy supply is transitioning under physical constraint. The mineral base that transition depends on is simultaneously being asked to support an AI infrastructure buildout that was not in any transition model. The carbon absorbers that cushioned decades of emissions are weakening faster than the forecasts predicted. None of this is speculative. It is in the published data of the IEA, the UNEP, the FAO, and the scientific bodies tracking atmospheric carbon.

None of it makes AI development wrong. It makes the absence of these questions from the safety agenda a choice, even if that choice was arrived at through structural inheritance rather than deliberate exclusion. A technology deployed without reference to the physical world it runs on and draws from is not a solution to the problems of that world. It is one more claim on systems that are already overdrawn.

The safety field is serious and the people in it are asking real questions. They have been handed a frame that excludes the physical world, and they are working inside it without knowing it is a frame. At the pace and scale AI is now moving, that gap has stopped being academic.

Safe for what world? The field has not yet asked. The answer will shape everything that follows.

Sources and data

Energy-GDP correlation: Ayres, R. & Warr, B. (2003, 2009), INSEAD / Chalmers University; IEA World Energy Balances Database (1960–2023). Global material extraction: UNEP International Resource Panel, Global Material Flows Database (1970–2023). Global debt: Institute of International Finance, Global Debt Monitor (February 2026). Marginal productivity of debt: Federal Reserve Bank of St. Louis, FRED Database; precise figures from Weiner, K., Monetary Metals (2017–2024), a non-neutral source, treated as illustrative of a trend documented in the underlying data. Conventional oil peak: IEA World Energy Outlook 2008; Aleklett et al. (2010), Energy Policy vol. 38. Ore grade decline: S&P Global Market Intelligence (2024); Calvo, Mudd et al. (2016); Escondida operational data, Earth Resource Investments (2025). Energy transition mineral demand: Michaux, S. (2021), Geological Survey of Finland; rebuttal: Hoekstra, A. & Carbon Tracker (2022). Carbon sink: Ke, Ciais et al. (2024), National Science Review vol. 11; 10 New Insights in Climate Science 2025, Future Earth / The Earth League / WCRP. AI energy demand: IEA, Energy and AI (2025); IEA, Key Questions on Energy and AI (April 2026). Mining and biodiversity: Sonter, L.J. et al. (2020), Nature Communications 11, 4174. AI safety research scope: Singapore Consensus on Global AI Safety Research Priorities (June 2025). Accumulative x-risk: Olejniczak, M. (2024), ‘Two Types of AI Existential Risk: Decisive and Accumulative’, arXiv:2401.07836. European AI compute: Juijn et al., Europe 2031 (June 2026), europe2031.ai.

Where do data people fit in the animal advocacy movement? by Richie

Richie — Fri, 12 Jun 2026 12:16:35 +0000

AI use disclosure: 0 AI used in this particular post.

Perhaps about once a month someone messages me on LinkedIn or the Hive Slack community, asking how a data scientist/analyst/engineer can use their career to help animals. This is something I’ve spent nearly 5 years thinking about! I originally trained as a data science after my PhD because I wanted to build skills to make the world a better place. These are my reflections on what I’ve learned, and conversations I’ve had with others.

Definitions

Data professional: umbrella term for data scientists, data analysts, data engineers, statisticians, MLOps people, analytics engineers blah blah….
I’m going to use the term “animal protection” to describe both animal protection non-profits and alternative protein companies. Most alternative protein companies don’t consider themselves to be part of the animal movement (or wouldn’t admit it publicly), but my assumption is that the reason you the reader would consider going into alternative proteins is because you want to help animals. So I’m just going to lump them all in together.

Paradoxically, while there is demand for data science in the animal protection movement, there is little demand for data professionals in animal protection organisations.

The bad news

Simply put, most orgs cannot afford a data professional. Not only that, they don’t have enough data, analytics or machine learning (ML) use cases to need even part-time data person. This includes both non-profits as well as alternative protein companies (who are usually small startups). I typically see 1-3 data roles a year advertised across the whole movement. There are probably less than 15 data scientists in paid employment in the entire movement.

How can organisations need data science but not data scientists?

Across all organisations, animal or no, for profit or non-profit, there is growing need for data skills. However, this doesn’t necessarily mean there is a growing need for dedicated experts like data scientists. Generally, it’s a growing need for everyone to have data fluency. Here are some ideas off the top of my head about how “non data people” might leverage data, machine learning and AI:

Data science skills and use cases for animal protection organisations

More generally:

Any role that has to use spreadsheets at some point would benefit from data skills. If you have spreadsheets, you have data. Some data skills make you better at using those spreadsheets, and if you know R or python you can take that data out of spreadsheets and do much more with it (or more likely, direct AI to do it).
Literally everyone needs to learn to use generative AI, because it is already impacting basically all jobs.
AI models can do data analysis, but often make questionable analytical decisions that only a data person would catch.

More specifically:

Marketers might upskill in AB testing
Researchers or Analysts might learn to scrape websites.
Researchers would also likely benefit from statistical training (which data professionals usually have) to do better “guesstimating”: estimating the size of a market, or the potential impact a program could have.
If you monitor your organisation’s socials or comms, then social media analytics is important to understand what’s doing well and what isn’t.
Product/project managers, as well as executives at medium to larger organisations would often benefit from dashboards and automated reports, but might only need a few.
Fundraisers might pull together enough data on donors to do some cluster analysis to understand what types of people donate to the organisation.
Website/app developers might add simple AI features into apps and websites
Any organisation that wants to get serious about evaluating the impact of its programs and campaigns would benefit from “causal inference”, a discipline from economics that is increasingly being brought into data science.

I’m a data professional that wants to help animals. What are my options?

There are broadly 3 key ways a data professional can directly participate in the animal protection movement and one indirect way:

Doing a non-data job where you can use data skills
Fractional roles
Consulting
Earn to Give (spoiler: this is the main one I recommend)

Use your data skills inside a non-data job

As outlined above, plenty of roles within animal protection/alt proteins would be enhanced with some data skills. You could be one of those people! Whereas outside the movement you might be a data analyst working in the marketing department, in the movement, be a marketer that can do data analytics.

It may feel strange or even sad to train as a data scientist, only to not “be” a data scientist. It may even make you feel like the training was pointless. But I would advise placing your identity as an animal advocate first, and focus on your commitment to helping animals. Ask yourself not “how can I help animals as a data professional?” but “How can I help animals with my data skills?”. Or perhaps you could create a new identity of yourself as a “hybrid data scientist/marketer” or “data-driven fundraiser”.

Fractional roles

Some people work part-time at one organisation and part-time at a different one, splitting their work week between 2 organisations. For example, in early 2024 I worked 3 days a week at Peak AI as a data science consultant (non-animal) and 2 days a week at Bryant Research as a data driven researcher (an animal protection research org). Then, I split my week 50:50 between Bryant and being the Director of Data and AI at Greener by Default (I’ve since handed off the role but remain a part time data consultant for them).

Fractional roles split it down even further: You might work at 3,4 even 5 organisations. Each one only needs you (and can probably only afford you) for 1 day a week, or 1 week a month, or any other arrangement. This is likely to be more possible in the animal protection space than many other areas of the economy because organisations are all allies, rather than direct competitors. You could never work for Adidas and Nike at the same time, because they’re direct competitors. But you can simultaneously work for two organisations that both want to end factory farming. It would even have the indirect benefit of increasing potential for collaborations. At Bryant Research, some of our Bryads also work in academic labs or campaigning orgs, and we all benefit from being closer to each other (they’re not data scientists but the logic is the same).

It seems to be common in the animal protection movement to have multiple part time roles, so orgs are likely to be open to a fractional role arrangement if it benefits them. If you have your eye on a few organisations, I advise speaking with a few of them about this option and making them aware of it if they are not.

Consulting

Another option is for the animal protection movement to have consultants and consulting firms that concentrate expertise and then do contract work for various organisations.

Currently, when it comes to specifically data, there are a few consultants, but no consulting firms. I think the movement could always use strong data consultants, but don’t think there’s enough demand for a consulting firm specifically for data. However, a consultancy network where data consultants share work and help each other might be useful (I give this a mild but not trivial chance of success).

I’d probably not call yourself a data consultant (or similar) though. I would aim to become a Monitoring and Evaluation (M&E, sometimes called MEL) consultant or a research consultant that specialises in data-heavy projects. The main analysis needs of orgs are data-heavy research projects, and using data to measure the impact of their campaign for funders. The hard truth for data people here is that there are already research consultants and MEL consultants that have domain knowledge you won’t have, and probably enough data skills to do a reasonable job. You’ll have to put the time in to build your skills in MEL or research.

I won’t repeat the standard generic pros and cons of being a consultant. The disadvantage compared to fractional roles is that consultants are external to the organisation and so do not know all the organisational context and culture. This can make them slightly less effective at delivering projects if those projects are not well planned and the consultant is not good at discovery and scoping (i.e. figuring out the real problem to solve, not naively solving the problem the client puts in front of them). You also don’t always feel like part of a team which can be a bit lonely.

It is worth noting that while consultants typically earn good money, this is not the case in animal protection. The animal protection movement is terribly underfunded. Expect to earn significantly less than if you were a consultant for for-profit companies (as in, like 20-40% less!). That said, if you want to earn lots of money, you probably wouldn’t be in this space in the first place.

Earning to give

This is an indirect way to help animals, but honestly, it’s the main one that I would recommend if you’re a data professional wanting to help animals. If you work in data, you can earn very good money in the for-profit sector.

So I advise that you do that, and then donate 10-30% of your income to animal orgs. If you are a high-paid data engineer or machine learning engineer, your donations might be enough to pay someone in the movement’s entire annual salary. The animal movement is one of the least well-funded social justice movements, so donations are often needed far more than skills.

This is probably not be a satisfying answer for many people: donating doesn’t feel the same as contributing directly. But similar to what I said above, always keep in mind that the animals must come before our own egos, and if this is the best way you can have an impact for animals, then you should feel very good about yourself for doing it. I personally have huge respect for people who donate significantly to the movement.

Find more about Earning to Give here.

What about AI consulting and generative-AI roles?

I intentionally left this out of earlier versions of this post but people keep asking me about it, so I’ll address it here. Some people feel like a data scientist joining the movement as a generative AI consultant is a natural fit. I don’t think it is.

I definitely think the animal movement needs more people with AI skills, and if you’re a data person thinking about retraining as a generative AI person, I think that’s a great idea.

However, I want to emphasise that I strongly believe that data people have little to no advantage when it comes to retraining in generative AI. AI before ChatGPT and AI after ChatGPT are technically similar tech, but in practice they are fundamentally different. If you’ve built AI solutions for a decade, I don’t think you’re any better placed to be a generative AI consultant than someone who’s been a marketer for 10 years or someone who’s worked in HR for 10 years. In fact, I personally think the people best placed to move into generative AI consulting are operations people and management consultants (I plan to write about this).

So while I would recommend people take this path, I wouldn’t say this is a particularly promising path specifically for data people. Be aware you’ll be starting from the same point as everyone else.

Conclusion

Animal protection could benefit a lot from data science skills, but lacks the funding and scale to make hiring dedicated data professional feasible. If data professionals want to make an impact for animals whilst also making a living, they might consider non-data science roles that use data science skills, fractional roles if they can get them, or staying in the for profit sector and donating.

Acknowledgements

Thanks to Antonieta Pais Nunes Lopes for inspiring this post. Also thanks to Kyle Behrend and Sam Tucker for valuable insights and comments.

Faith In Humanity by AlejoAcelas🔸

AlejoAcelas🔸 — Fri, 12 Jun 2026 07:28:18 +0000

[AI Use Disclosure: Asked Claude to draft this, then entirely rejected its suggestions]

I’ve read a lot of good moral philosophy linked from the EA Forum, but not much of it (Joe Carlsmith excepted) has focused on some of the not-evidently-utilitarian traits and dispositions one could cultivate to become the kind of person that does the most good.

I’m discovering there’s some meticulous writing on that topic as well, and I wanted to share here a piece that is currently resonating with me:

History and literature provide striking examplesof people who are morally admirable, in part, because of theirprofound faith in people’s decency. But moral philosophers have largely ignored this trait, and I suspect that many philos-ophers would view such faith with suspicion, dismissing it as aform of naïvete or as some other objectionable form of irrationality. I argue that such suspicion is misplaced, and that having a certain kind of faith in people’s decency, which I call faith inhumanity, is a centrally important moral virtue. In order tomake this view intuitively more plausible, I discuss two moralexemplars – one historical and the other literary – whose lives vividly exhibit such faith. Then I provide a rationale for theview that having such faith is morally admirable. Finally, I discuss cases in which someone’s faith in humanity can lead her tomake judgments that are, to some degree, epistemically irrational. I argue that the existence of such cases does not pose a serious objection to the view that having faith in humanity is amoral virtue. Rather, it makes salient important limits on therole that epistemic, as opposed to practical, rationality shouldoccupy in our ideals of how to live.

Paper: Faith in Humanity, Ryan Preston-Roedder, Philosophy and Phenomenological Research (2013). https://philpapers.org/archive/PREFIH-2.pdf

[New Paper] Prioritizing Risks from AI: A Delphi Study of 272 Experts by Peter Slattery 🔸

Peter Slattery 🔸 — Thu, 11 Jun 2026 20:56:58 +0000

TL;DR: We ran a Delphi study with 272 international AI experts to prioritize 24 AI risk domains from the MIT AI Risk Domain Taxonomy. In a business-as-usual scenario, experts judged a more than 10% chance of catastrophic outcomes (i.e., ‘more than 1 million human deaths or more than a USD 100B in financial loss or civilizational-scale intangible impacts’) from 18 of our 24 AI risk domains over the next five years.

They also identified a responsibility gap: AI users and affected stakeholders are most vulnerable, while general-purpose AI developers and governance actors are seen as most responsible for reducing the risks.

Below are three of the key findings and related visualizations.

Key finding 1: Experts judge that many risks could cause catastrophic outcomes under current trajectories

18 of 24 risks were judged to have a more than a 10% chance of causing catastrophic outcomes (which could include more than one million deaths, more than $100 billion in financial losses, or other harms) by 2030 under a business as usual scenario.

Figure: Experts’ mean catastrophic risk probability under business as usual and with pragmatic mitigations. Note: “Business as usual” assumes organizations and governments continue their existing practices but do not implement additional AI-specific risk mitigations; “Pragmatic Mitigations” assumes organizations and governments make pragmatic, cost-effective efforts to address AI risks.

Key finding 2: Those most vulnerable to AI risks are not those most responsible for addressing them

According to experts, general-purpose AI developers and governance actors such as governments, regulators, and standards bodies hold primary responsibility for addressing AI risks. In contrast, AI system users and affected stakeholders such as members of the public are most vulnerable to AI risks.

This mismatch means that those who are most responsible for addressing AI risks are not those who are most vulnerable, leading to misaligned incentives in addressing the most important AI risks.

Figure: Experts assessed who is vulnerable to AI risks and who is responsible for addressing them.

Key finding 3: Information, finance & insurance, and national security are the most vulnerable sectors

Across most risks, experts identify information, finance & insurance, and national security as the most vulnerable sectors. The results also show how vulnerability differs across sectors and risk categories.

Figure: Expert consensus on sector vulnerability for AI risks.

A 7-minute overview of the study and findings ↓

Read our paper and explore the interactive results here.

Disclosure: I used an LLM to help generate the first draft of content in this post. I then rewrote and reviewed the text, and I endorse the final version. We also used LLMs as part of the research and communication, for instance, in generating the images and interactives.

The Quest To Find The Next Big Communicators In AI Safety by Akshyae Singh

Akshyae Singh — Thu, 11 Jun 2026 19:18:38 +0000

In September 2025, I’d become increasingly convinced that a fieldbuilding program for content creators could solve a long-standing bottleneck of expanding reach and trust beyond the AI safety and EA bubble.

I had graduated from UCLA a few months earlier when I came across the AI-2027 report which had a significant impact on me. I rejected my six-figure tech consulting offer, left my place in LA, and moved to the first place I could find in the Bay Area to contribute to AI Safety. Contrary to my expectations, only a small subset of people were working towards what seemed to be far more important than building yet another AI B2B SaaS.

I wanted to change this. The power of a good external-facing comms effort is underrated. AI in Context, If Anyone Builds It Everyone Dies, Rob Miles is the first interaction with AI Safety for many, including mine. It has the power to empower mainstream audiences to act by creating trust and relatability, and meeting them where they’re at- social media. Moreover, I felt that I had the right background to do this- my experience as an activist, ex-founder building products for clients Google & BCG, and operator running one of the largest AI Safety conferences in LA.

In my initial few months of moving to SF, I conducted creator events and campaigns with orgs like CAIS, one of which ended up reaching over 2.5M+ views through 5 creators. While this was impactful, it was clear that we were heavily talent bottlenecked on this new kind of audience builder and educator. These were- The next Rob Miles. The blogger who speaks to Gen Z. The next AI-2027-shaped artifact. And, just as importantly, communicators like Petr Lebedev embedded at Palisade Research who can power dissemination from inside the orgs doing the work.

That’s when Austin Chen (who was also interested in this) and I started Frame Fellowship, to find and accelerate the next big communicators in AI safety.

Note: Early bird Applications for Cohort 2.0 are now OPEN! Apply

Video: Intro video for Frame 1.0

The Fellowship

We ran our first cohort from January-March 2026- a San Francisco based residential accelerator for content creators focused on AI safety with 11 fellows selected out of 150+ applications. Our applicant pool included youtubers, viral tiktokers, established creators interested in AI safety, creators with unique backgrounds in law, journalism, and defense, all focused on different sub-niche audiences.

Frame Fellows at the final showcase at Lighthaven, Berkeley

The program offered various benefits such as mentorship from leading creators in AI Safety such as Rob Miles, AI in Context, Species, Doom Debates and others, office and studio space, generous funding, and access to the bay area AI Safety, founder, and creator communities.

Frame’s first iteration was a success- producing 260+ pieces of content reaching 7M+ estimated views over just 8 weeks. Fellows significantly grew their reach and audience (many 2-12x’ed), and 60% received post-program funding or contracts/partnerships from FLI, Control AI, BlueDot, Seismic, and others. Moreover, fellows leveraged their audiences to build policy outreach tools, communities that gained 100+ signups in the first week, protests, and research datasets.

Some examples of our fellows work-

The Philosophy Quietly Turning AI Companies Evil | Mateus De Sousa
AI Just Retaliated Against a Developer | Veronica Hylak
5 things we can do to push back against OpenAI’s Pentagon deal | Caitlin Yardley
AI companies can’t race ahead without asking human questions | Janet Oganah
Explaining the METR graph | Gaetan Selle

some popular videos by our creators

Some Success Stories-

Fellow Mateus De Sousa grew from 1,500 to 5,000+ followers and surged from 20,000 to 200,000 average views per video. He launched a YouTube channel focused on AI politics and industry accountability, got hired by Seismic for content strategy, and secured Future of Life Institute funding to continue full-time for a year after the fellowship. He also received interest from BlueDot and 80,000 Hours.
Fellow Veronica Hylak produced 10+ YouTube videos, nearly doubled her channel to 400k views and 35k subs, and secured a contract with CAIS. Her videos on AI psychosis led three universities to reach out directly. A student built their thesis on preventing AI psychosis specifically because of her content. Veronica is also building a research dataset from hundreds of DMs she’s received from people with AI psychosis.
Fellow Michael Trazzi Michael organised a 200+ person protest outside Anthropic, OpenAI, and xAI, attended by Scott Alexander, Nate Soares, and Daniel Kokotajlo, and covered by The New York Times and The Atlantic. His SB-1047 video is now a required resource in a UC Berkeley policy course. He is building “Stop the AI Race” into a sustained movement targeting a 1,000-person protest.

Check out other fellow stories on our website!

Frame itself also made headlines in The Washington Post and The Transformer, and a mention in SF Standard.

Other Features of the program (with pics!)

The fellowship also created structured sessions for mentorship, feedback, and collaboration. Here’s a few features of the program:

Biweekly standups to track progress and surface blockers

left: fellow Caitlin pitching her theory of change; top right pic: fellows Michael & Lauren discussing content strategies; bottom right: Fellow Janet receiving feedback on her video

Group content sessions to enable cross-collaboration. Every week fellows came together to make fun content with each other!

Front-loaded mentorship. We packed most mentor sessions into the first week, so the rest of the program could be spent executing rather than scheduling.

left: Chana Messinger (AI in Context, 80000 hours, 350k+ ); right: Drew Spartz (Species, 350k+)

Social nights, games, casual chats to build community.

left: fellows Gaetan and Vin drinking Hot Chocolate (that I made); right: creator dinner with other creators in SF

The group house. This was unofficial, but we decided to live together throughout the program as well! This enabled bonding in a way that wouldn’t have happened across separate apartments.

left: right after a snack run; right: We love Sterling!!!

The Conversation event: We also ended strong with a flagship event at Lighthaven, bringing together 100 leading creators, journalists, media voices, and AI safety folks to discuss the most effective strategies for educating the public.

The Conversation at Lighthaven to celebrate and discuss AI Safety media

Testimonials:

“We went from EAG to seeing Bernie Sanders to SXSW to Lighthaven to meeting researchers I’d followed on Twitter for years” ~Caitlin Yardley (Creator 6k) - Frame Fellow

“I’ve probably looked at a thousand grant proposals over the years, and the Frame Fellowship seems like the highest-leverage funding opportunity I’ve come across.” ~Drew Spartz (Species − 300k) - Leading AI Safety Youtuber

“I would’ve benefited greatly from the support that Frame offers when I was starting out on YouTube. It can be a very lonely career, esp in the beginning. This is likely to reduce burnout and greatly increase their odds of success.” ~Siliconversations (150k) - AI Safety animations channel

“This program opened my eyes. Without it, I am 100% positive I would’ve advanced maybe 5% of what I did in 2 months. A critical part of the journey for me.” ~Veronica Hylak (Youtuber 25k) - Frame Fellow

What we learnt

Now the most important part- what did we learn, and what’s next!

1. Optimize for inputs, not outcomes. The inputs that matter: more videos, more hypothesis testing, shorter feedback loops, useful connections, sharper theories of change, and calls-to-action that map to them. Outcomes follow inputs.

2. Clarity in schedule, logistics. Work expands to fill the time you give it. Cohort 2 will have more forcing functions and clearer weekly deliverables. Admin work- Reimbursements, visa-related compliance, took significantly longer than I budgeted for.

3. Longer cohort. Fellows consistently flagged that 8 weeks wasn’t enough. Cohort 2 might run 10 weeks in-person plus a 2-week remote pre-program.

4. Expand the team. For Cohort 2 we’re looking to hire a team of 2-3. (If you want to get involved, details at the end of the post)

5. Deeper mentor and partner org involvement. In Cohort 1, mentors mostly gave one-off sessions. Partner orgs were similar. For Cohort 2 we’re going to match mentors and orgs to specific creators for maximum impact!

Looking Ahead

The next few years will likely see AI hit a tipping point that drives massive societal and economic change. As AI climbs the list of things people care about, that attention can be leveraged: to empower action, build movements, and demand AI that enables democracy and agency rather than stripping them away.

Doing that requires communicators who can reach people with trust and scale. They are the bridge.

We are guided by three foundational pillars:

Find & Accelerate new voices & projects through the fellowship and training resources
Support & brief existing voices & projects- through grants, partnerships and other creator tools.
Connect AI safety orgs to our vetted network of creators, comms leads, and amplifiers

What we’re aiming for in the next year:

120+ vetted creators trained, with trusted communities, increasing AI salience across millions more people
Become the first point of contact for AIS orgs and companies that want to communicate with audiences at scale.
Incubate projects that make AI safety a more salient issue and build a greater public movement.

How to Get Involved

Cohort 2 is scheduled from August to November 2026. There are a few ways to plug in:

Early bird Applications are OPEN! APPLY

We’re looking for ambitious communicators who can own a niche, and come up with creative ways to increase distribution.

Join the Team

We’re hiring for Cohort 2. I’m looking for high-agency operators. You’re a good fit if you think you can run the program end-to-end just by yourself. Email me at akshyae@framefellowship.com your email will be the application. We are a non-profit that works at startup speed.

Partner with us

If you run an AI safety org doing impactful work, we are opening a new “Amplifier” track where we embed promising fellows to amplify your messages. We can help you 10x your impact by reaching more people in your target audience. get in touch

Consider Donating

We’re raising funds for Cohort 2. Reachout if you’re interested.

Building Ideas-First EA by Mjreard

Mjreard — Thu, 11 Jun 2026 17:40:49 +0000

TLDR: Sarah Bluhm and I are funding and mentoring ideas-first (as opposed to, e.g., careers-first) EA community builders. If you’re at one of these universities or know someone who is, we want to talk to you, especially this subset:

University of Toronto
University of Michigan
UCLA
USC
NYU
Columbia
Claremont Colleges (Claremont McKenna, Scripps, Harvey Mudd, Pomona, Pitzer)
Stanford
Harvard
MIT
Cornell
Georgetown
Northwestern

**Contact us here or at contact@nest-ea.org**

The Network for EA Support & Training (NEST)

We (Matt Reardon and Sarah Bluhm) founded NEST, whose mission is to build vibrant in-person EA communities by teaching organizers the craft of communicating EA ideas with courage and authenticity.

I (Matt) am a former lawyer and 80k advisor who blogs, tweets, and podcasts about EA. Sarah worked on CEA’s university groups team and has a background in bioengineering.

NEST’s philosophy is that EA communities should be platforms for individuals to form their own views on the central questions of Effective Altruism: what are the most important issues in the world, why those issues, and what should I do about them?

This contrasts with approaches to community building that act as recruiting funnels for specific organizations or fields. We think thoughtful consideration of the ideas and arguments around Effective Altruism will independently motivate some individuals to plug into high impact roles, but we are not optimizing for this.

We are optimizing for the quality of the conversations and arguments that come out of the groups we support. Do people come away understanding these 200 concepts well? Do they have robust ethical outlooks that account for all the important intuitions and considerations? Perhaps most importantly, do they have people who will help them act on these beliefs?

University groups

We’ve decided to focus on university groups at a subset of these schools. Society has conveniently created the densest clusters of intellectual vibrancy among people who are at an age where they are still deciding what they believe and what to commit themselves to. And these clusters have a lot of built-in infrastructure for social connection.

I wouldn’t invest community building efforts in different settings until I was confident this ecosystem was thoroughly supported and I still see major gaps here. While a few EA groups at these schools are thriving,^[1] many have only 1-3 committed members who don’t find their groups especially rewarding. Others have no group at all.

I think it’s easily worth funding at least one full time person working to seed and invigorate an EA group at each one of these universities. The bottleneck is finding the people with the context, judgment, extraversion, motivation, and work ethic to make that happen.

I think the team at CEA—who have been far and away leading efforts in this area for years—would agree that this is the bottleneck and that they feel spread too thin. So Sarah and I are adding ourselves to the mix,^[2] and also taking a different approach with regard to scale: we will focus intensively on just 7 of these universities and potentially hire community builders who will focus on 1-2 universities each.

What we provide

Our mission is to train organizers in communicating ideas and support them in building groups. Here’s what we expect that will entail:

Helping organizers master the full scope of EA-relevant ideas and perspectives so they’re comfortable leading any discussion through:
1. 1-1 coaching
2. Multi-day workshops with the best thinkers in EA
3. A curated library of the best ~200 foundational EA articles
Hiring full time organizers
Grants of up to $20,000 to cover group expenses
Crafting events, fellowship curricula, and sourcing speakers
Advising group members 1-1
Coming to campuses ourselves to speak on EA topics and themes

Connecting points 3 and 6 here, I’m excited to give a talk railing against the Copenhagen Interpretation of Ethics. Doing good is (mostly) about achieving good outcomes in the world, not the purity of your intentions or how much you sacrificed to reach those outcomes. The groups we support will be fun and well-funded. NEST will not be trying to share EA ideas with one hand behind its back while Bain and McKinsey throw cocktail parties.

For us, fun entails both generic things like good food and nice spaces, but also the affordance for organizers to try things they’re excited about without extensive justification. We want to be similarly permissive with respect to topics and subject matter. As long as you’re taking the question of doing Big Good seriously, we want you to host discussions of everything from RCTs, to fundamental science, to YIMBY, to post-AGI politics, and more.

We also want groups to engage with non-EA ideas aimed at doing good. Giving proponents of these ideas space and a full hearing is often a powerful way to understand both the rigor and the flaws of common EA views.

The next step

As above, we’re looking for interested people at universities on our long list, especially these ones listed above. This is for both potential organizers and potential (non-organizer) group members. It is also for both you (the reader) and people you know at these universities (e.g. a high school classmate who’s interested in EA).

I want to build people up into a great ideas-first EA communicators. Or if they’re already there, I want to throw resources at them and raise their ambitions. So if we haven’t spoken, get in touch!^[3]

^
Looking at you, UChicago, UC Berkeley, and Yale
^
Well with Sarah it’s complicated, but I’m definitely adding myself
^
You can also email us directly at matt@nest-ea.org or sarah@nest-ea.org, especially if you’re recommending a friend

First EA Italy Fundraising Campaign: A Brief Overview by Fabio Sandrecchi

Fabio Sandrecchi — Thu, 11 Jun 2026 16:44:55 +0000

I used an LLM to help draft this post and it likely contains >10% AI-generated text (translation and correction), but I’ve edited and rewritten it extensively and endorse it.

First EA Italy Fundraising Campaign

Hello everyone!

My name is Fabio, and I am a member of the board of directors of EA Italy APS. I have been involved with Effective Altruism for around ten years, but I believe this is my first post on the Forum.

The Italian EA community and I would like to share the progress and goals of our current fundraising campaign (individual donors). We hope that our experience may be useful to other national or local groups facing similar challenges, or simply encourage local groups to start a fundraising campaign from scratch.

Previous Situation

EA Italy APS was founded several years ago to support the growth of the Effective Altruism movement in our sun-kissed country. One of the main challenges we face is the different culture around charitable giving and philanthropy, although I suspect that other Southern European countries encounter similar obstacles.

Initially, the community had a full-time paid community builder, to whom we owe much of the creation and development of EA Italy. She also served as our first president. Later, the project was supported by two part-time paid community builders (approximately 20 hours per week each, often more). During this period, our activities were funded through grants from the international EA community.

What Happened in 2025

In 2025, our president decided to pursue a different career path, and a new board of directors was elected.

During the same year, largely due to changes in the broader funding landscape, two grant applications we submitted were unsuccessful. As a result, we had to rethink how to ensure the long-term sustainability of our association.

Our First Fundraising Campaign

After reviewing our priorities and goals, we launched our first fundraising campaign in mid-May.

We are aiming to raise a modest annual budget of €2,300, allocated across three categories:

Operational costs: €650
Funded book clubs (two book clubs, two groups per book club) and resources such as domain hosting, email marketing tools, and other essential services: €1,050
Outreach and advertising: €600

As you may have noticed, because our association is entirely volunteer-run, there are no staff costs.

So far, we have raised approximately €1,400 in about one month, reaching 64% of our target.

The largest contributions have come from members of the board and supporters from the international EA community. However, we have also received donations from members of the Italian community, which is particularly encouraging.

Our Goals

Although our fundraising target is relatively small, reaching it is not straightforward because:

The Italian EA community is still relatively small.
Italy has a different culture around charitable giving compared to some other countries.
While some of us have fundraising experience, this is our first campaign of this kind, and we still have much to learn.

Our primary goal is simply to ensure that EA Italy can continue operating and organizing EA-related events and programs, including career workshops, one-on-one conversations, introductory programs, and book clubs.

More broadly, we want to learn how to build a sustainable and effective national community with limited financial resources and without relying entirely on international grants.

We aimed to be as transparent as possible about our budget, and we shared a detailed breakdown with potential donors (more information here, in Italian).

Having already reached 64% of our goal, we are reasonably satisfied with the results so far, but we hope to reach 100% before the end of June.

How We Collected Donations

The campaign is entirely online.

Our fundraising activities include:

Announcements through our national Slack workspace.
A series of newsletters.
Social media posts on Instagram and Facebook (now planned for mid-June).
A dedicated fundraising landing page on our website.
Digital donation methods, i.e. bank transfer, PayPal, and Satispay.

You can find our fundraising page here: https://altruismoefficace.it/raccolta-fondi-2026 (currently available only in Italian).

Final Thoughts

This is our campaign in a nutshell. We hope that other local and national EA groups may find some value in our experience. I think the main issues were related to communication: we missed our planned timing on social media, and there was a gap before the launch of the fundraising landing page on our website. We still have a lot to learn, so we would be grateful for any questions, feedback, or suggestions.

Please feel free to also contact us at info@altruismoefficace.it or reach out to me directly.

We promise to share the results of our campaign in mid July, editing this post.

Thank you for reading!

Effective Altruist Writing Is Impactful: Here’s What To Do About That by Bentham’s Bulldog

Bentham's Bulldog — Thu, 11 Jun 2026 15:18:28 +0000

Crosspost.

There’s a common pessimistic view according to which persuasion on important topics is almost impossible. This is backwards. Persuading people on high-stakes topics is amazingly impactful and underdone.

Now, it’s certainly true that any individual article won’t change the minds of most people. But how could it be otherwise? If most articles convinced most people, then readers would constantly be buffeted around by whatever they last read—one day a Catholic, the next a radical vegan, the next an atheist.

Most of the biggest ideas I believe today came to me through some singular article or video. I first encountered effective altruism through a TED Talk, wild animal suffering through an article, and animal welfare because of Alex O’Connor’s video. I wasn’t immediately convinced. But these pieces introduced me to the central ideas that I came to see were correct, after I reflected more.

When Scott Alexander wrote a post arguing that people should pledge to give away 10% of their lifetime earnings, he was able to get 60 people to take the pledge. That probably saved about 120 lives. About 30 people who have taken the pledge mentioned my posts, which probably saved about 60 lives.

I am not Scott Alexander. I don’t have his amazing ability to write six hundred pages filled with subtle Kabbalistic analysis. My blogging approach has generally been “write things that I think are true, in ways that are clear, and give solid arguments for them.” Scott Alexander is some kind of weird blogging witch—I am not a witch. I’m you.

(Funniest campaign ad in history!)

But by writing a bunch of articles about EA topics, I’ve been able to raise enough money to save hundreds of lives and help billions of shrimp. When some bloggers got together and did a shrimp fundraiser, we were able to raise $137,000 for the Shrimp Welfare Project—enough to spare about 2 billion shrimp.

A recent survey of my readers revealed that:

29% said my articles had convinced them to donate somewhere they wouldn’t have donated to otherwise. In many cases, this was just a few dollars given once—but in just as many cases, people set up monthly donations to highly effective charities. Some people said it was counterfactually responsible for them giving thousands of dollars to FarmKind.
7.8% said the blog was responsible for them taking a different career or doing some other high-impact project that they wouldn’t have otherwise done.
66.1% said I’d changed their mind on some topic.
7.5% said the blog was responsible for them taking the Giving What We Can pledge. If the real number is even 1%, then this would make the blog counterfactually responsible for 100 pledges and about 200 lives saved.
35% said the blog contributed to them going vegan, vegetarian, or being some kind of reducetarian.

I don’t think I’m particularly special in this regard. I’m not an especially naturally skilled writer (just look at my earliest blog posts). If you write true things reasonably clearly, you can probably have a pretty big impact. If you have a blog with 100 subscribers, and convince one of them to take the pledge, that’s about as much counterfactual impact as giving away $10,000.

A bunch of young interesting EA writers have started substacks: Amos Wollen, Silas Abrahamsen, Glenn, Value Locked In, and Irrational Community. My guess is that on average each person who has done this has gotten lots of money donated to effective charities. This is an insane fact about our world. By starting a blog on the internet and posting regularly, you can save a bunch of people’s lives, in expectation.

This has some big implications.

First, if you are thinking about starting a blog, do it. Maybe, probably it will go nowhere. But even if it remains pretty small, you might just save someone’s life, or spare a bunch of sentient beings from extreme suffering. Blasting important ideas into the ether is really impactful. The only situation in which I’d recommend against this is if there’s some other super impactful thing that it would compete with. But if you’re a student or an academic, start writing on the internet about important topics!

Probably you should do this even if you haven’t given much thought to blogging before. Before I started blogging, I didn’t think of myself as a writer. I didn’t even like writing much. I just started it on a whim because Michael Huemer had some objections to utilitarianism that I thought were wrong!

Second, if you have a blog, make sure to write about important topics. It’s easy to get lost in trying to be novel and only focus on cool intellectual topics. But you should try to regularly say things that are important, even if they’re not that novel. It isn’t a brilliant insight that you should give away your income to effective charities, but if you write that down in an article, and send it out to your readers, you might save a bunch of lives. When writing a post, have in the back of your mind the question: if I convinced every one of my readers that what I was saying was true, would this make the world better?

A number of pretty influential writers read this blog. If any of you people are reading this article, I am speaking to you: please write a post arguing that people should give money to effective charities. Suggest explicitly that they take the Giving What We Can Pledge, and explain what it is. Mention some specific highly-effective charities. Remember: when Scott Alexander did this, he prevented around 120 deaths! You can prevent large numbers of deaths too, just by writing an article.

Even if you already said something vaguely similar in 2018 or whatever, say it again. You should try to write reasonably frequently about impactful subjects. Maybe a few people will find it somewhat repetitive, but other people will hear it for the first time, and take the pledge, and extra children will not die! My sense is that basically every big writer writes way too infrequently about impactful subjects.

Return to the classic drowning child experiment. If a bunch of children were drowning in far-away ponds, and the only way to save them was to publish a blog post suggesting that people nearby pull them out of ponds, then obviously you should publish that blog post! This would be true even if you published a vaguely similar blogpost a bunch of years ago that some of them have read. On top of this, write about high impact careers!

Third, if you start writing, and your articles are good, don’t stop! A common failure mode is people start blogs, they go pretty far, and then they sort of run out of ideas and peter out (cough Connor Jennings cough). Now, in some cases this is because they start working on other important things (e.g. I hear Glenn is out saving the world, so he has an excuse). But The Droll Scroll and Amos Wollen—you are without excuse, sorry. You were both gifted with supernatural writing powers and should use them for good!¹

Fourth, even if you are not a blogger, but instead a member of the hoi polloi, try to share articles on important topics with other people. Restack them, share them with friends and family, etc. It is very easy to hit the restack button when you come across important articles on EA topics—doing so enables these articles to get a wider reach.

Fifth, I think it’s plausible that becoming a paid subscriber for high impact EA blogs is pretty impactful. Now, obviously this is very self-serving, so you have some reason to be suspicious. But having more paid subscribers helps a blog do better in the subtack rankings, thus potentially moving large amounts of money to effective charities. It wouldn’t surprise me if one of the better things you can do with a few dollars a month is becoming a paid subscriber to Silas Abrahamsen’s blog, for example, or Amos Wollen’s (especially on the off chance it would rouse him from his slumber).

Sixth, if you start a blog, try to ensure it isn’t just read by EAs! While you should sometimes write about the most important topics, you shouldn’t only write about the most important topics. If I only talked about shrimp, that wouldn’t even be good for the shrimp. Ideally you want to make EA ideas more salient among non-EAs.

Do other things to make sure your blog gets more exposure. For example, I just got Twitter premium and started cross-posting my articles on Twitter. My guess is that more people should do things like that. If doing that for every single article gets one extra person to take the pledge, it will have much more than paid off.

A common blog failure mode is getting nerd-sniped by interesting topics that no one else cares about. Now, I’m not above this, as my fifty billion articles about subtleties in anthropic reasoning attest. But writing about topics that are neither important nor interesting to most people should be thought of as akin to drinking: fine in moderation, objectionable in excess. That is, unless you are Scott Alexander, and you can write an engaging piece about literally anything.

There are real and significant problems in the world. Much can be done about them. It is easy to get lost investigating cool puzzles that no one cares about. But you should know that doing so trades off against convincing people of what matters most.

Frontpage posts - Effective Altruism forum viewer

A simple argument for trying less hard by Elias Schmied

Uncertainty

Epistemic distortion

Conclusion

are we locked into P(doom)? by keivn

start­ing with the conclusion

three conditions

go­ing down the beaten path

com­ing to terms

con­figu­ra­tion, not agency

the formalization

from thresh­old to trajectory

the tree

what this resolves

what re­mains open

the app

con­nect­ing the dots

Is Polysemanticity the Way Forward? by Ololade

Un­der­stand­ing Poly­se­man­tic­ity and Superposition

Why In­ter­pretabil­ity Re­searchers Worry About Polysemanticity

Why Poly­se­man­tic­ity May Not Be the Real Safety Problem

The Scal­ing Problem

What If We In­ten­tion­ally De­signed for Poly­se­man­tic­ity?

Fea­tures as Directions

Conclusion

References

Join Robin Hanson Book Q&A: The Age of Em by Keishi Kimura

Surplus: for massive public good by Austin

Pro­jects we’re ex­cited for

1. AI for epistemics and coordination

2. Public-fac­ing websites

3. Com­mu­nity infrastructure

What we offer

Timeline

Virtues we cherish

FAQ

Will the biosecurity interventions succeed in a resource-limited setting like Nigeria in the event of an engineered pandemic? by Nnaemeka Emmanuel Nnadi

TL;DR

Background

The Eng­ineered Pan­demic Play­book May Fail in Nigeria

Some fac­tors that may pre­vent these In­ter­ven­tions from working

The In­fras­truc­ture Issue

2. The Trust Problem

3. The Miss­ing Ecosystem

4. The Hid­den As­sump­tion of Global Biosecurity

The Biose­cu­rity En­vi­ron­men­tal Fit (BEF) test

Rational Animations is a 501(c)(3) nonprofit and is looking for board members by Writer

Who we are

Why we’re ex­pand­ing the board

What you’ll do

Who might be a good fit

How to apply

Add Free $16 To Donations by Tyler Kolota 🔹

How bad would it be if GPS satellites were shot down? by Jackson Wagner

“Whoa, GPS comes from space?? I thought it was just a thing in my car…”

To be clear, there are mul­ti­ple satel­lite-nav­i­ga­tion constellations

What could kill GPS?

Great-power war

Hack­ing by su­per­hu­man AI???

An un­prece­dented so­lar storm, maybe

Other stuff

What would break in the af­ter­math of los­ing GPS?

Im­pact on the cell net­work & smartphones

Im­pact on the power grid

Im­pact on mar­i­time industries

Im­pact on travel and logistics

Im­pacts on other industries

Reck­on­ing an over­all cost per day

Hard to say, but this feels ap­prox­i­mately Covid-19-scale-ish

How long would an out­age last?

Re­mem­ber: you’re still at war with China, or Rus­sia, or AI, or pos­si­bly the Sun!

What new AI safety orgs get wrong about operations in year one? by AartiMahajan

*Note: I used an LLM as a think­ing and draft­ing aid for this post. The views are my own.*

Launch a biosecurity group on your campus by sharmaayushmaan

Why uni­ver­sity biose­cu­rity groups

Why I’m offer­ing this

Apply

Free human coaching for high-impact folks by Joel Patrizio

Safe for What World? Why the AI safety field may be asking the wrong question by Benoît L.

starting with the conclusion

going down the beaten path

coming to terms

configuration, not agency

from threshold to trajectory

what remains open

connecting the dots

Understanding Polysemanticity and Superposition

Why Interpretability Researchers Worry About Polysemanticity

Why Polysemanticity May Not Be the Real Safety Problem

The Scaling Problem

What If We Intentionally Designed for Polysemanticity?

Features as Directions

Projects we’re excited for

2. Public-facing websites

3. Community infrastructure

The Engineered Pandemic Playbook May Fail in Nigeria

Some factors that may prevent these Interventions from working

The Infrastructure Issue

3. The Missing Ecosystem

4. The Hidden Assumption of Global Biosecurity

The Biosecurity Environmental Fit (BEF) test

Why we’re expanding the board

To be clear, there are multiple satellite-navigation constellations

Hacking by superhuman AI???

An unprecedented solar storm, maybe

What would break in the aftermath of losing GPS?

Impact on the cell network & smartphones

Impact on the power grid

Impact on maritime industries

Impact on travel and logistics

Impacts on other industries

Reckoning an overall cost per day

Hard to say, but this feels approximately Covid-19-scale-ish

How long would an outage last?

Remember: you’re still at war with China, or Russia, or AI, or possibly the Sun!

Note: I used an LLM as a thinking and drafting aid for this post. The views are my own.

Why university biosecurity groups

Why I’m offering this

The economy is a physical system

Three things happening at once

Into this world, we are building AI infrastructure at speed

Why this question is not being asked

The questions that are not being asked

Paradoxically, while there is demand for data science in the animal protection movement, there is little demand for data professionals in animal protection organisations.

How can organisations need data science but not data scientists?

Data science skills and use cases for animal protection organisations

I’m a data professional that wants to help animals. What are my options?

Use your data skills inside a non-data job

Fractional roles

Earning to give

What about AI consulting and generative-AI roles?

Key finding 1: Experts judge that many risks could cause catastrophic outcomes under current trajectories

Key finding 2: Those most vulnerable to AI risks are not those most responsible for addressing them

Key finding 3: Information, finance & insurance, and national security are the most vulnerable sectors

A 7-minute overview of the study and findings ↓

Other Features of the program (with pics!)

Looking Ahead

Early bird Applications are OPEN! APPLY

Partner with us

Consider Donating

The Network for EA Support & Training (NEST)

University groups

First EA Italy Fundraising Campaign

Previous Situation

What Happened in 2025

Our First Fundraising Campaign

How We Collected Donations

Final Thoughts