Frontpage posts - Effective Altruism forum viewer

Resources for starting and growing an AI safety org by Bryce Robertson

Bryce Robertson — Sun, 19 Apr 2026 05:16:19 +0000

It seems that AI safety is at least partly bottlenecked by a lack of orgs. To help address that, we’ve added a page to AISafety.com aimed at lowering the friction for starting one: AISafety.com/founders.

This page was built largely as the result of a suggestion from @Ryan Kidd, who found he was frequently sharing the same set of resources with potential founders and realised it would be useful to have something similar publicly available.

It lists:

Fiscal sponsors
Incubators
VCs
Articles and tools

As with all resources on AISafety.com, we put substantial bandwidth into making sure the information on this page is accurate and up to date. If you have any feedback, please let us know in the comments or via the Suggest buttons on the page.

This is the 11th resource page on the site. Here’s the full list:

courses for self-study
communities, both local and online
upcoming events and training programs (plus a weekly newsletter)
funders offering support for AI safety projects (and a newsletter announcing new opportunities)
field map displaying the key orgs, programs, and projects in AI safety
advisors offering guidance calls to help individuals most effectively contribute to AI safety
media channels (newsletters, podcasts, etc.) for staying informed
field-building projects seeking volunteer support
jobs (a filtered subset of 80k’s database)
a guide for donating to the field
founder toolkit for starting and growing an AI safety org

P(doom) ranges from 10% to 99% depending on who you ask. I searched 1,259 hours of AI safety podcasts to map the actual disagreements. by Bardoonii

Bardoonii — Sat, 18 Apr 2026 18:46:28 +0000

The AI safety community doesn’t agree on how worried to be. But most discussions flatten this into “doomers vs optimists.” The actual disagreement is more interesting.

I built a semantic search tool that indexes 392 episodes from 80,000 Hours, AXRP, Dwarkesh Patel, The Inside View and more. Searching p(doom) and related terms surfaces some striking contrasts:

Robert Miles puts his p(doom) at 90–99%: “My mainline prediction is doom. It doesn’t look good.”

Eliezer Yudkowsky on those who agree with his arguments but have lower p(doom): they “enact the ritual of the young optimistic scientist who charges forth with no ideas of the difficulties.”

Scott Alexander and Daniel Kokotajlo both land around 20% — the lowest on the AI 2027 team. Alexander notes he’s “not entirely convinced we won’t get alignment by default.”

Will MacAskill sits at 10–20%, calling himself “optimistic today” — but notes this is among the lowest estimates in serious circles.

Sundar Pichai also estimates ~10%.

Zvi Mowshowitz has moved up to ~70%.

What’s interesting isn’t just the numbers — it’s what drives the disagreement. Christiano focuses on conflict scenarios: AI systems acting on their own goals, described as “little green men getting ugly.” Yudkowsky focuses on capability jumps that make intervention impossible. LeCun thinks the whole framing is wrong.

These aren’t just vibes — they’re searchable, timestamped arguments from the primary sources.

You can explore the full disagreement here: AI Safety Search — Leita — search p(doom), existential risk, or any researcher by name

Why we need to resist the idea of net negative lives in insects by Seidenpuma

Seidenpuma — Sat, 18 Apr 2026 16:10:17 +0000

Recently, I have come across a growing number of speculations suggesting that wild animals lead net negative lives. Proponents of this view typically refer to insects, arthropods, invertebrates, or species with r-selective reproductive strategies. The conclusion they draw is usually that we should aim to reduce their numbers in one way or another. Unfortunately, birth control is seldom discussed in this context. Habitat destruction appears to be the preferred method.

There is, of course, some value in speculating about virtually any topic in science. What troubles me is that such speculations may be mistaken for established theories, particularly given the way they are presented to broader audiences through public forums, podcasts, and YouTube videos.

The central argument typically runs as follows: animals with r-selective reproductive strategies produce many offspring, of which only a few survive. The experience of a usually violent death therefore constitutes a large proportion of their lives.

This argument rests on two assumptions:

The experience of death itself is something terrible.
The experience of death actually makes up a significant proportion of those animals’ lives.

I want to push back against both assumptions.

First Assumption: Is a Violent Death Really That Bad?

When I contemplate my own death, I consider it tragic for three main reasons:

All the good experiences I will never have again.
The grief it will bring to family and friends.
It will prevent me from doing good in the world.

In short, I want to live as long as possible, provided I remain in reasonable health. By contrast, the manner in which I die — whether torn apart by wild animals, burned alive, or passing away in my sleep — seems to matter relatively little in the grand scheme of things. This excludes prolonged suffering, such as the kind caused by serious illness factory farming or some insecticides.

So why do I assign so little weight to the experience of death itself?

In a life-or-death situation, the pain system tends to shut down as long as the body remains in fight-or-flight mode. And since we are discussing scenarios that end in death, there is no subsequent trauma to recover from.

A personal account

When I was fourteen, I was hit by a car and hovered between life and death for roughly half an hour before losing consciousness. I woke up three weeks later in hospital. Since I would not have survived without intensive medical care, and since we are concerned here with subjective experience rather than outcome, this episode serves as a useful proxy for a death experience.

How did I experience it?

First, I was struck by something I could not immediately identify — it felt no worse than bumping one’s head. My immediate instinct was to get up and continue on my bike. Fortunately, my trainer held me down, correctly suspecting a spinal injury. I insisted I was fine; the car had barely touched me, I told him. They showed me the wreckage of my bike to convince me otherwise. That, by far, was the worst part of the entire experience. I was also concerned about my mother having to come and collect me — until they mentioned they had already called a helicopter, at which point I became genuinely euphoric. I also felt a slight pang of guilt about the cost to taxpayers — though, in my mind, this seemed entirely disproportionate, since I remained firmly convinced that it had been nothing more than a scratch.

When the doctor arrived, he administered an injection, and everything went dark. Despite three fractured vertebrae, several other broken bones, and a lung injury that nearly killed me, I did not experience a single moment of pain. To put it in utilitarian terms: if I assigned a value of −100 to the worst experience of my life — being left by someone I loved — this entire episode would rate no more than −1. And I am not unusual in this regard.

This suggests an ironic implication for negative utilitarians: by the same logic used to argue that r-selected animals lead net negative lives, one might conclude that humans are among the most unfortunate species of all, given how frequently we fall in love and are disappointed. That conclusion would, of course, be overly simplistic — just as it is overly simplistic to focus exclusively on reproductive strategy.

Second Assumption: Does the experience of dead make up a big part of an Animal’s Life?

This claim is repeated so often, in one form or another, that it risks being accepted as fact. Yet I have never seen anyone provide actual figures to support it.

To calculate the proportion of the death experience in an animal’s life, we would need to know two things: when consciousness begins, and how long the dying process lasts. Consider the following examples:

Animal A is a small invertebrate whose consciousness began one week, or 604,800 seconds, before hatching. She lived undisturbed inside her egg and was eaten immediately upon hatching. The entire dying process lasted one second. That amounts to ¹⁄_604,800 of her life — the remainder was, by any reasonable assessment, a peaceful existence.
Animal B is a human who lived for eighty years and experienced severe cancer pain for one month. That month represents roughly ¹⁄₉₆₀ of his life — and yet it also included the psychological burden of knowing he was going to die, something probably unavailable to most non-human animals.

These are deliberately simplified examples. No one can say with confidence when consciousness begins in a given individual animal, let alone across categories as broad as arthropods or invertebrates. But that is precisely the point: we should not treat a hypothesis as an established theory when no data exists to support it.

Why this idea is dangerous

To my knowledge, there is a strong consensus among researchers working with arthropods that preserving these animals is of critical importance. Yet a small number of proponents of the opposite view have already gained notable traction, and there are reasons to think this could accelerate.

1.Algorithms favour controversial ideas.

2. Such proponents may present themselves as uniquely concerned with animal welfare rather than ecological function.

3. Industries involved in habitat destruction may find it in their best interest to promote a narrative that makes their activities appear less harmful.

It is unlikely that the general public will ever broadly accept the idea that reducing animal populations constitutes a form of welfare intervention. Nevertheless, even the perception that a scientific debate exists on this question may be enough to reduce public motivation to engage in conservation efforts — and that outcome alone warrants taking the argument seriously.

AI risk is not a Pascal’s wager by ozymandias

ozymandias — Sat, 18 Apr 2026 15:49:13 +0000

In the 17th century, the mathematician Blaise Pascal devised the idea of Pascal’s Wager.

Should you become a Christian? If God exists and you’re Christian, then He will reward you with an eternity of bliss in Heaven. If God exists and you’re not Christian, He’ll punish you with an eternity in the fires of Hell. If God doesn’t exist and you’re Christian, you waste many tedious hours praying and at services. If God doesn’t exist and you’re not Christian, you can spend Sunday morning scrolling TikTok. Since an eternity of bliss in Heaven is much better than a finite amount of time spent on TikTok, you should believe in God.

Pascal’s Wager has two major flaws.

First, many religions promise an eternity of bliss for believers and an eternity of punishment for nonbelievers. Maybe instead of following Jesus you should be following Mohammad or Amitabha. Even if you’ve settled on Christianity, you have to pick one of the many mutually contradictory branches of Christianity, many of which believe all the others are going to Hell. Not to mention the many more outre possibilities. How do you know there isn’t a God of Atheism, who has deliberately hid Himself from the world to test the skepticism of His creation, and who punishes anyone who believes in God anyway with an eternity of torture?

Second, people are bad at reasoning about infinities and very small probabilities. We can do okay with 1 in 100 probabilities and even 1 in 10,000 probabilities—at least if we’ve learned to think with numbers. But when you start getting down to the 1 in 1 billion territory, you’re going to end up confusing yourself more than you think anything insightful.

Some very silly people have used Pascalian reasoning about the very long-term future. The long-term future of humanity affects septillions of sentient beings. So if you have a 1 in 1 sextillion chance of positively influencing the future, this is the most important thing you could possibly be doing!

Like Pascal’s original Wager, this has two major flaws.

First, people are very bad at reasoning about 1 in 1 sextillion probabilities and it is unclear that they will come up with anything useful to say about them.

Second, when you’re reasoning about 1 in 1 sextillion possibilities, suddenly you have to consider all kinds of extremely unlikely possibilities. What if the primary effect of your life is getting a complete stranger to run late coming home from work, have sex with his wife a bit later, emit different sperm than he otherwise would, and thus conceive von Neumann Socrates Gandhi, the smartest, wisest and most ethical human being to ever live? What if it’s good to drive humanity extinct because humanity’s extinction would lead to the evolution of ecstatically happy, fulfilled, and virtuous sapient crabs? Admittedly, it doesn’t seem very likely. Is it 1 in 1 sextillion likely? How do you know? (A lot of reasoning about this falls under the heading of “cluelessness.”)

Specifically, you fall victim to the optimizer’s curse. Whenever you try to figure out how good a course of action is, you make mistakes. The action that looks best to you right now is likely very good, but you’re also probably making a bunch of mistakes that make it look better than it is. The more uncertain you are, naturally, the more mistakes you could be making. The more speculative an intervention, the more it will predictably underperform your best estimate of how good it is. Indeed, under some fairly reasonable assumptions, always choosing a nonspeculative intervention leads to better outcomes than sometimes choosing a speculative intervention—no matter how much better the speculative intervention looks!

Some equally silly people have taken the “if you have a 1 in 1 sextillion chance of positively influencing the future, this is the most important thing you could possibly be doing” line of reasoning and used it to dismiss all work on existential risk.

For an article I’m working on for Asterisk, I’ve been asking people who work in AI safety how likely they think it is that humanity is going to go extinct in the next decade. They give numbers like 10%, or 20%, or 50%. Sometimes they give numbers like 99%. No one ever gives a number like “1 in 1 sextillion, but since human extinction might prevent the existence of septillions of future sapients, it is still the most important thing you could be doing.”

“10% chance of human extinction” isn’t Pascalian! It is a normal probability, of the sort we reason about constantly in normal life! If I said to you “my local library has free ice cream one day per week, but I can’t remember which day it is, let’s swing by and check it out”, you would not accuse me of being misled by an infinitesimal chance of infinite cold deliciousness.

Do I think there’s a 10% chance—much less a 99% chance—of human extinction in the next ten years? Uh, no. If you look into the evidence and conclude that AIs are going to cause imminent human extinction, you get a job trying to stop them from doing that; if you look into the evidence and conclude that they’re just going to automate software engineering, you do something else with your life. I’m conscious of the risks of groupthink: if you’re around people who believe in a 99% chance of imminent human extinction, suddenly a 50% chance of human extinction seems positively moderate and restrained. And, frankly, AI has been most effective in the realm of software, which AI researchers have the most expertise in; I expect that broader deployment will make people realize that atoms are much harder than bits.1

But when I talk to the people developing a new technology and they say to me “yeah, we think it might kill everyone on Earth”… I, uh, I am a little concerned about that? I don’t dismiss this entirely out of hand? If you put a gun to my head and make me pick a number, maybe 1%?

1% isn’t a Pascalian number either. We do all kinds of reasoning about 1% chances. About 1% of people in the United States are incarcerated at any given time. About 1% of people speak Korean as their native language. 1% is a little less than the chance that, in a group of four random people, at least two share the same birthday. It’s about the chance that someone has schizophrenia, or celiac disease, or red hair.

If I talked to people working on other cutting-edge technologies—electric airplanes, or sodium-ion batteries, or personalized gene-editing therapies, or malaria vaccines—and I asked them “is your technology going to kill everyone on the planet?”, they would be like:

Most technologies have a truly Pascalian chance of causing human extinction, which is why no one ever publishes thinkpieces like “malaria vaccines: will they lead to fifty-foot-fall supermosquitoes with an unquenchable thirst for blood?”

The engineers who do say “my technology might kill everyone on the planet” are working on nuclear bombs and bioengineering, which, yes, are technologies that ought to be carefully regulated. I realize that this is a controversial position that might get me #cancelled, but neither Sam Altman nor Dario Amodei nor Elon Musk should be allowed to develop their own nuclear umbrellas.

If you are uncertain about whether a technology will drive humanity extinct, that is very bad and justifies heavy-handed regulatory oversight. We should only yolo technologies that we know for certain won’t cause human extinction, i.e., all the other ones.

It drives me nuts that left-of-center people keep making this mistake, because it is straight out of the anti-environmentalist climate change denialist corporate playbook. “Oh, we’re not certain what the effects of climate change are, so we shouldn’t do anything about it.” “Oh, we’re not certain whether the hole in the ozone layer is bad, so we can keep spewing CFCs into the atmosphere.” “Oh, we’re not certain that acid rain will drive vulnerable fish species extinct, so let’s wait until they’re extinct before implementing carbon scrubbers.”

No! By the time we were certain what climate change does, multiple Pacific islands HAD ALREADY SUNK INTO THE OCEAN!

Sometimes, you need to take steps to prevent a bad outcome, even if you aren’t sure how bad it will be—or even if you aren’t sure whether it’s going to happen at all. The time to switch to renewable energy was before the Pacific islands sunk into the ocean. And the time to make it illegal to deploy an advanced AI that isn’t verifiably aligned is before grey goo devours my living room.

Reasoning is Pascalian because of the 1 in 1 sextillion probabilities. The concept of preparing for a bad thing that is less than 50% likely to happen isn’t Pascalian. It is called other things, like “responsibility” and “foresight” and “common sense.”

Effective Altruism and Radical Trivialisms by Bentham’s Bulldog

Bentham's Bulldog — Sat, 18 Apr 2026 15:38:10 +0000

Crossost.

Many of the core claims made by effective altruists are radical trivialisms. They imply that we ought to behave differently, but the premises on which they rest are quite elementary and obvious.

Consider the core idea: one should strive for effectiveness when doing good. You should aim to do more good rather than less when giving to charity and choosing a career. When people criticize EA, they will often say something like “of course, the true idea of effective altruism is trivial enough—no one disagrees that one should strive to do more good rather than less. But behind the scenes, EA really is [contentious bundle of philosophical assumptions].”

Oh really? Everyone believes that you should strive to do more good rather than less in career choice and charitable donations? Maybe everyone believes it in the sense that if you ask them “good—do you favor more or less?” they will say “more.” But virtually no one attempts to consciously optimize for a maximally high-impact career. Almost no one judiciously investigates the effectiveness of different charities or is a priori cause neutral—interested in simply doing good as well as possible, whatever form that takes, even if it involves funding unsexy charities like the Schistosomiasis Control Initiative.

I will believe you when you say that the idea that we should give to maximally effective charities is trivial only if you’ve spent at least two seconds doing it—rather than giving your charitable donations to your local socialist magazine without giving any thought to effectiveness or to whatever random charity sounded cool when your friend told you about it. But in a sense, it is trivial. It’s trivial in that when you think about it, it seems obvious. Of course we should strive to do more good instead of less—of course there are strong moral reasons to take the best career you can rather than whichever one you feel like. If you can save more lives, or fewer lives, obviously you should save more.

A lot of the core EA claims are this way—they are trivial when you think about them, but almost no one acts in accordance with them. When an obvious principle demands a behavioral change that you do not carry out, you are failing to live up to your principles. When sound arguments demand non-standard behavior, most people won’t follow the arguments.

For one notable example, the imperative towards cost effectiveness in global health. Nearly everyone has the intuition that if you’re choosing between saving lives and doing other things that do less good, you should save lives. If you could either pull a child from a burning building or give soup to nine people, you should pull the child from the building.

Similarly, if you could save five lives or two, you should save five. If you combine that with the surprising fact that most ordinary people can save lives routinely, and yet instead spend their money doing less effective stuff, then it seems obvious that people should start giving to effective global charities instead of the random charities that tickle their fancy. It’s intuitively pretty clear that saving lives matters more than giving to the charities you like.

Longtermism is another good example of this dynamic. The core idea of Longtermism is very basic. The one-sentence argument for Longtermism is: the future could have lots of people, meaning in-expectation almost everyone affected by our actions resides in the far future. Thus, given that people in the far future don’t matter much less than present people, we should aim to help them.

That sounds really obvious. If future people matter and almost everyone our actions affect is a future person, then we should look carefully into doing things that are good for future people. For every present person, there could be quadrillions of future people—intuitively, it seems like a single individual matters less than their quadrillion descendants. But in practice, almost no one pays much attention to how their actions affect the far future! Before the Longtermists, how many people spent any time thinking about how their actions would influence the very far future? Future thinking, before Longtermism, meant thinking about the next 200 years, not the next 2 billion.

It’s not that non-Longtermists explicitly think that people after 200 years suddenly stop mattering (if they don’t, then nearly all that matters is what will happen after 200 years). It’s that they simply don’t give the matter much thought. The idea that we should change our behavior in major ways based on how it will affect far future people is weird—but weird doesn’t mean wrong!

And sure, with Longtermism there are more complexities than with the core EA idea. To decide if Longtermism is right, you have to decide on slightly thorny questions like whether we can do anything to reliably better the far future. But it would be suspicious if, despite 99.9999% of the morally important impacts of our actions being on how they affect the future, there wasn’t any way that should affect our behavior.

Someone once joked that effective altruism is the paramilitary wing of analytic philosophy (note to people reading this named Emile Torres, please do not take this out of context and make it sound like EAs endorse violence). What he meant by this was that philosophers spend time thinking about ideas and uncover lots of non-obvious things to be done. Effective altruism is about putting those weird ideas into practice, when the ideas are neglected because people don’t think about them, not because there are serious objections to them.

Animal welfare is another good example. Even if you’re not some tree-hugging hippie animal lover, probably you have the intuition that animals being in pain is a bad thing. If someone was torturing dogs or pigs in their basement, that would be really bad! If you can prevent lots of animals from being in pain at very small cost, then you should do that. As a matter of fact, you can do this—a single dollar can prevent more than 10,000 shrimp from experiencing intense pain in death or can prevent chickens from spending around 10 years in a cage.

Thinking that this is a valuable thing to spend money on doesn’t require any eccentric views about animals. If there were 10 chickens who’d be trapped in a cage for a year unless you spent a dollar freeing them, that would seem like a good use of a dollar. If you value an hour of your life at $20, then this would be equivalent to spending 18 seconds freeing a chicken from a cage that would have otherwise trapped it for a year. If there were 10,000 shrimp who’d slowly suffocate unless you spent 1 dollar to stun them, that would seem like a good use of a dollar.

When you combine that commonsensical normative premise with the surprising fact about our world that a dollar really can prevent chickens from spending years in a cage, that implies we should change our behavior. But this radical behavioral change comes about because of a surprising fact about our world, not a counterintuitive moral premise. The moral premise that it’s good to spend a dollar so that sentient beings aren’t trapped in a cage for years is super obvious!

When EA activities look weird, that’s typically downstream of surprising facts about the world, rather than counterintuitive normative claims. It is a surprising fact about our world that you can help so many people and animals so cheaply. It’s similarly surprising that of all the welfare in human history, in expectation almost all of it will be in the distant future. I really like the way Richard Y Chappell put this point when responding to a commenter who claimed it was counterintuitive to think we should fund humane pesticides:

Common intuition agrees: suffering is in itself bad to some extent, and hence worth reducing if you could do without undue cost or unintended side-effects. (Who would deny this?)
Common intuition agrees: scale matters. Two individuals suffering is twice as bad as just one, all else equal.
Crazy empirical premise: we cause (possibly unnecessary?) suffering to quadrillions of possibly-sentient beings.
Person who refuses to make logical inferences: “It’s *absurd* to think that the suffering of quadrillions of tiny beings could constitute a moral catastrophe! What are you, some kind of utilitarian?”

To prove EA rests on counterintuitive moral claims, you can’t just point to EA activities that sound weird. Weird doesn’t equal ethically counterintuitive. Norman Borlaug saved hundreds of millions of people by optimizing wheat production. That’s slightly odd behavior. But it wasn’t morally counterintuitive in any way—it was downstream of the surprising fact about our world that you can save hundreds of millions of people by optimizing wheat production. When the world is surprising, so too should our behavior be. I also like the way Ajeya Cotra put it:

Many dudes come into EA because they’re into moneyball and then realize they can moneyball the good — but I was a little girl who liked to take care of babies, and I realized there were way too many babies who needed me, and I had to get smart about it if I wanted to save them all.

The core moral principle that one should do more good rather than less is intuitive. Often EAs recommend surprising behavior because in our world, the best ways to do good are slightly strange. But if you wouldn’t condemn Norman Borlaug for working on optimizing wheat production, you shouldn’t criticize EAs for helping shrimp or giving out vitamin A supplements. Sometimes doing good ends up being a bit weird.

Building the Field of AI Safety by 80000_Hours

80000_Hours — Sat, 18 Apr 2026 09:52:38 +0000

Note from Aaron Gertler: Hi, Forum! We just published this career review on one of the toughest bottlenecks for AI safety. We hope it’s helpful.

“Fieldbuilding” is a gigantic category and we couldn’t cover all of it. If you have suggestions to improve the piece, we’d be happy to hear them.

Read this career review on the 80,000 Hours website.

Summary

In a nutshell: Fieldbuilding — developing talent and creating infrastructure for AI safety — is a high-leverage way to reduce catastrophic risk from advanced AI. An hour of good fieldbuilding can enable many hours of direct work. And yet, fieldbuilding is badly neglected; many qualified people pursue direct work, making it hard to fill some of the most promising roles.

Pros:

Very high potential impact, which scales as you help more people
High-growth projects let you quickly advance your career
Versatile area with a wide range or roles, suitable for many skillsets
Develops social and entrepreneurial skills with lasting value

Cons:

Offers relatively weak career capital outside AI safety
No single clear career pipeline and few obvious entry points
Lower median pay than technical roles

Key facts on fit:

You don’t need any specific experience or credentials; many different backgrounds can be helpful.
Valuable traits include people skills, technical fluency, mentorship ability, and versatility. If you’d be a good fit for many different roles, you might be an excellent fit for fieldbuilding.
Dedication to the cause is critical. If you are deeply committed to reducing AI risk, you’ll have an easier time inspiring other people to pursue that work.

The case for fieldbuilding

In 2017, there were only a few dozen people working full time to reduce risks from advanced AI. By 2025, there were over a thousand.¹

How did this nascent academic field grow by 3000% over eight years?

Some of the growth was organic, driven by public interest in AI. But much of it came from active fieldbuilding: recruiting and training skilled people to work on AI safety and creating infrastructure to accelerate their progress.²

An hour spent on fieldbuilding may enable many hours of direct work: for example, the small team at BlueDot Impact has helped thousands of people learn about AI safety and pursue jobs in the field. We’ve seen enough stories like this to believe that fieldbuilding is often extremely impactful.

It is also badly neglected. Most people with the right skills gravitate towards more direct roles in research or policy. This creates serious bottlenecks across the AI safety ecosystem, slowing work on priority problems.

Because of this dynamic, we think that AI safety fieldbuilding is one of the most important career paths for reducing catastrophic risk from advanced AI.

This is true even for people whose skills make them a strong fit for other AI safety roles. Fieldbuilding is a flexible career path that rewards many forms of talent, and we’ve seen many people who left promising technical research or policy careers and went on to do even more good as fieldbuilders.³ Whatever your background, we think this path is worth considering.

What do fieldbuilders work on?

Fieldbuilding isn’t just one activity; there is no ‘standard’ fieldbuilding role. You might:

Coauthor a curriculum, then teach the material in a virtual seminar.
Match promising early-career people with mentors and professional networks.
Manage technical researchers, helping them stay focused and productive.
Plan events, coordinating vendors and invites to ensure things run smoothly.

Real-world examples:

Fellowships from organisations like MATS, GovAI, Tarbell, or the Future Impact Group, which offer training and networking to help participants build careers in AI safety
Career guidance from organisations like 80,000 Hours (hello!), which help recent graduates and experienced professionals find roles that fit their skills
Events like The Curve or FAR.AI’s alignment workshops, which help people learn about AI safety, build connections, and start projects together
University groups like Harvard’s AI Safety Student Team, which can drastically increase the number of students who enter the field (or easily fall apart without strong organisers)
Coworking and event spaces like Constellation and LISA, which help people make connections and discover new opportunities
Public communications projects like AI in Context, the 80,000 Hours Podcast, or AI 2027, which explain AI safety concerns to large audiences, and are often someone’s first introduction to the topic
Grantmaking from organisations like Coefficient Giving or Longview Philanthropy, to support projects like those mentioned above

Why AI safety fieldbuilding can be highly impactful

Fieldbuilding works

Before 2017, the field of AI safety barely existed. Only a handful of organisations and individuals were doing serious research on ways that advanced AI could pose a catastrophic risk to humanity.

The field looks very different now. AI safety research regularly appears in top AI journals and at major conferences like NeurIPS. Hundreds of notable academics, politicians, and other public figures have expressed concern about catastrophic risks. AI safety issues have drawn the attention of governments across the world, driving policy efforts like export controls, international summits like the International Dialogues on AI Safety (IDAIS), and the creation of institutions like the US Center for AI Standards and Innovation and the UK’s AI Security Institute.

We attribute some of this growth to the rise of advanced LLMs, which produced a surge of interest in all things AI. But we don’t think AI safety would have grown nearly as much without active fieldbuilding. When Coefficient Giving, the largest funder of AI safety work, surveyed several hundred people in the field, over 60% cited a specific fieldbuilding project as one of the most important factors in how they chose their path.⁴

Putting the numbers aside, it makes intuitive sense that fieldbuilding would be effective. AI is progressing rapidly, and many people are already worried about it, so anyone making the case for working on AI risk won’t be starting from scratch.⁵ If you present the arguments well and provide the right resources, you could inspire many people to shift the focus of their careers.

AI safety needs more skilled people

Right now, the AI industry is growing even faster than the field of AI safety (in terms of both funding and staff). As we approach an era of transformative change from advanced AI, and the catastrophic risks that era could bring, we’ll need more people across the board — from researchers and entrepreneurs to policy experts and podcasters.⁶

What’s more, talent is often ‘heavy-tailed‘: top performers are considerably more productive than their peers, so we benefit disproportionately from enabling highly skilled people to get involved. AI firms can pay employees millions of dollars to accelerate progress, and AI safety can’t match those financial resources. To compete for talent, we’ll need compelling arguments and ideas delivered by excellent fieldbuilders.

Some AI safety jobs draw hundreds of applicants, but only a small fraction of those typically have what employers are looking for, and those people tend to receive multiple offers. As a result, there’s often a significant gap between the best- and second-best candidates, and organisations may struggle to hire anyone for a position (especially if it’s relatively senior). If you help someone with strong abilities enter the field — or help someone develop those abilities — you could make a big difference.

Beyond finding and developing talent for existing roles, fieldbuilding can also expand the talent pool by opening paths for a wider variety of people. The Tarbell Fellowship trains writers in the fundamentals of AI journalism and provides a stipend to support their work in professional newsrooms. This creates new opportunities for people with a writing background to contribute to AI safety, rather than just growing the pool of people competing for the same opportunities.

You could also create opportunities by starting something new. Like Tarbell with journalism, new fieldbuilding projects could help people join other neglected areas — like content creation, entrepreneurship, or public policy.

The work is highly neglected

We’ve spoken to leaders in organisations throughout the field, and they broadly agreed with the following points:

Fieldbuilding is one of the most impactful things to work on within AI safety.
Many people in direct roles (technical or nontechnical) should instead be working on fieldbuilding.
Despite this, it’s very hard to hire good fieldbuilders; many qualified people simply don’t apply for those roles.

For example:

Kairos runs the Pathfinder Fellowship, which is aimed at helping participants build AI safety university groups. So far, most participants have gone on to pursue research roles, despite having experience that makes them a good fit for fieldbuilding.
Kairos and Constellation (another fieldbuilding organisation) estimate that around 2,000 AI safety research fellows will be trained in 2026, and only around 300 non-research fellows.
IAPS runs a three-month fellowship for people seeking policy roles. One round of the fellowship drew thousands of applications. At the same time, IAPS was hiring someone to help run the fellowship — and only heard from a few qualified applicants. As with many other orgs we’ve spoken to, they’ve struggled to find skilled operations staff.

Helping others lets you multiply your impact

In any career, your impact is limited by your time. Even if you work long hours, you probably won’t be as productive as ten people doing similar work.

Fieldbuilders take advantage of this fact. If you help 10 people find highly impactful AI safety roles, you’ll be responsible for a fraction of their impact. This could sum to more than you’d accomplish through direct work (depending on what they would have done without you).

Ten people is a realistic figure, and you could do even better. Take BlueDot Impact, which runs courses on AI safety and helps its students find careers in the field. It has reached more than 7,000 people, nearly 1,000 of whom now have full-time AI safety roles. Even if BlueDot’s courses only made each of those people around 10% more likely to join the field, that’s the equivalent of being directly responsible for nearly 100 careers (and counting — the organisation has only been around since 2021).⁷

BlueDot’s team is just seven people. By providing education and information to so many others, it seems likely that each BlueDot staffer has made a greater contribution to AI safety than they would have by working in roles without this multiplier effect.

That’s not the only multiplier effect worth noting; fieldbuilding also lets you leverage your top skills across a wider range of opportunities. A skilled researcher who transitions to research management can use their experience and judgment to shape dozens of projects instead of running a single project. An 80,000 Hours advisor might talk to a journalist, lawyer, and political staffer in a single day — applying the same sharp thinking on career choice across three different fields.

Potential downsides to working on fieldbuilding

While fieldbuilding is one of the most impactful paths we’ve found, it’s worth understanding these tradeoffs before you dive in.

Career capital may be lower

“I’m a technical researcher who studies the leading AI models.”

“I work in Senator Blake’s office as an expert on cutting-edge technology issues.”

“I’m helping to organise an AI policy fellowship.”

The first two answers may sound better on a CV or at a party. The importance of fieldbuilding work won’t be clear to someone who isn’t familiar with AI safety — so it won’t open the same doors as published research or a letter of recommendation from Senator Blake. In general, people have an easier time transitioning from direct work to fieldbuilding than vice versa.

Direct work offers stronger career capital (on average) in a few other ways:

Depth: Fieldbuilders tend to work across many areas of AI safety, but the work is often relatively shallow. Direct roles make it easier to build expertise in a specific niche.
Visibility: Research roles have concrete outputs, and policy work often gets media coverage. If you focus on fieldbuilding, you may not come away with much ‘visible’ work for your portfolio. Plus, it’s harder for fieldbuilders to cultivate a public reputation — you’re less likely to appear on a podcast or speak at a conference. (Though communication roles, like writing or making videos, are a highly visible fieldbuilding option.)
Professional development: You may get less training and guidance from experienced colleagues. Management capacity is quite limited within fieldbuilding, and it lacks the focus on formal mentorship you’ll often find in research, or the highly experienced colleagues you’ll meet in some policy roles.

However, fieldbuilding isn’t as disadvantaged as people often think. More than most direct roles, it offers:

Fast progression: Fieldbuilding tends to produce fast, concrete feedback — you can survey the people your project reaches and use the results to improve the next version. This lets successful projects scale quickly: we’ve seen several organisations go from the pilot stage to reaching hundreds of people within a year or two. And if you help to found a fast-growing project, the skills you develop as it grows could accelerate your career — many senior staff within AI safety organisations got their start as fieldbuilders.
Networking: Fieldbuilding makes it easy to build a network within AI safety. You’ll meet a lot of people, and usually in social environments (events, fellowships) where they want to make connections. If you helped to create that environment (by planning the event or fellowship), you’re well-placed to leave a positive impression. And because the field is densely connected, a strong reputation spreads quickly.
Skills with lasting value: Relative to most AI safety roles, fieldbuilding puts an emphasis on developing social and entrepreneurial skills. These skills are hard to automate — AI progress is likely to make them more valuable, rather than less.

The ‘career path’ isn’t as clear

There are reliable ways to explore careers in AI safety research or policy.

A budding researcher might take classes in machine learning, read AI safety papers, or apply for research fellowships like MATS or SPAR.

A policy hopeful might study economics and political science, read relevant publications, make connections as a campaign volunteer, or apply for policy fellowships like those offered by Horizon or IAPS.

Fieldbuilding lacks this kind of ‘one-size-fits-most’ pipeline. There are too many kinds of roles, and few programmes focused on training fieldbuilders. But we still have plenty of advice for getting started.

Technical work often pays better

The median technical role (research, engineering) pays more than the median fieldbuilding role. However, fieldbuilding pay isn’t necessarily low; many organisations offer highly competitive salaries, comparable to what you’d earn from a similar role in the private sector with the same experience and seniority.

You can use our job board to check this for yourself. We don’t have a single ‘fieldbuilding’ filter, but most of these roles fit that category.

Direct work can also grow capacity

Direct work can have its own ‘fieldbuilding’ effects. As an academic, you could spend most of your time on research and still teach or train dozens of students. As a policy expert, you could draft an influential bill or white paper; SB-1047 catalysed widespread public discourse and likely served as many policymakers’ first exposure to core topics in AI governance.

If your skills or background position you to have an exceptional impact through direct work, that could also become your best fieldbuilding opportunity — if you actually focus on those aspects of the job. (An academic who cares little for mentorship or management won’t do much to develop talent.)

How can fieldbuilding backfire?

Fieldbuilding often involves trying to influence what other people work on. This can have harmful consequences:

You might encourage people to work on something less impactful than their other options, or even something with negative impact.
The people you reach might cause harm despite working on something promising — e.g. by doing a bad job or alienating potential allies.
You might discourage people from getting involved by giving them a bad impression of AI safety, or giving advice that doesn’t suit them.
- The most capable people in your audience are especially valuable to reach, and might be easier to discourage; they often have high standards and good alternative options.

While we’ve seen these outcomes before, we think they are significantly rarer than the positive outcomes we’ve highlighted, for a few reasons:

The most common outcome of fieldbuilding is simply prompting people to think more carefully about their own careers and helping them understand their options — not forcing them down a particular path.
People tend to notice when advice doesn’t suit them. If you provide both good and bad suggestions, the good ones are more likely to stick.
While it’s hard to survey people who don’t get involved, we think it’s pretty rare that someone gives up on AI safety after one bad impression. Even a mediocre introduction often spurs curiosity, rather than rejection.

Some people think the field’s impact has been mostly negative so far — for example, speeding AI progress by highlighting its economic potential, or slowing political coordination by associating AI safety with liberal politics.

If you think that most current AI safety work is harmful, that’s a valid reason to avoid fieldbuilding. However, you could also support the specific work you think is beneficial; we’ve seen many critics of ‘mainstream’ AI safety do valuable fieldbuilding.

Could you be a good fit?

There’s no single profile for a successful fieldbuilder, but the experts we’ve met tend to highlight a few broadly useful traits.

Skills that will help you succeed

Versatility: Most fieldbuilding organisations are small and fast-moving; the people who thrive in them tend to be adaptable and broadly competent.

As Jake McKinnon, who led Stanford’s effective altruism group before joining 80,000 Hours, puts it: “If you think you’d be good at almost anything, do fieldbuilding.”

Technical fluency: While you don’t need to be an engineer or an expert on any one topic within AI safety, you need to understand the basics — especially in areas like research management or curriculum design. Going beyond the basics is even better; being technically skilled makes it easier to reach other technical people.

It also helps to be well-informed on recent developments. We recommend Don’t Worry About the Vase or Transformer’s Weekly Briefing for wide-ranging updates.

Mentorship: Many fieldbuilding roles involve teaching, coaching, or training. Whether you’re mentoring a researcher or advising a student on career paths, it helps to be a good listener who can find the right advice for the right person. One expert we interviewed called out the importance of raising others’ aspirations and inspiring them to pursue ambitious goals.

Social intelligence and networking: Fieldbuilding often entails chatting with strangers, pitching ideas to an audience, and creating connections. Many great fieldbuilders have a strong mental Rolodex — a sense of what people in their network are looking for, and an eye for matching opportunities. You don’t need to be the most outgoing person in the room, but it helps to be ‘good with people’.

Passion: If we had to pick one trait successful fieldbuilders have in common, it would be their sincere dedication to AI safety — and the way they convey that dedication to people who are still exploring the field. Fieldbuilding involves working hard on something whose impact you may never fully see; genuine belief in the mission helps you sustain your efforts, and inspire that same commitment in the people you’re trying to reach.

That said, fieldbuilding is far from thankless! Your work will change people’s lives, and you’ll probably get to meet those people and experience their gratitude firsthand.

Promising professional backgrounds

People management: AI safety has a lot of early-career people and few seasoned managers. Experienced staff are sometimes forced to manage out of necessity, even if it isn’t their comparative advantage. If you can thrive in a managerial role, you’ll help your reports develop faster and reduce strain on your senior colleagues.

Project management: AI safety evolves quickly, and fieldbuilding projects often pivot to keep up. They also tend to involve multiple stakeholders — events and fellowships aren’t solo efforts. These conditions demand coordination, flexibility, and rapid iteration — classic ‘PM’ skills.

Entrepreneurship and organisation building: AI safety has a lot of room to grow; there are many promising ideas to expand the field that no one has seriously tried.⁸ We need founders and builders to fill those gaps. This includes recent graduates; we’ve seen many student organisers thrive in other fieldbuilding roles.

How to get started

Because you don’t need any specific experience to become a fieldbuilder, our strongest recommendation is to check the AI fieldbuilding roles on our job board and consider applying if you see a role that fits.

Our other top recommendation is to consider fieldbuilding fellowships. Any AI safety generalist can apply to the Generator Residency (due April 27), which helps you launch a new idea within three months. The Pathfinder Fellowship offers training and mentorship to help students develop campus AI safety groups.

More ideas, if you’re new to the field:

Take a course (or two) from BlueDot Impact.
Read our primer on helping advanced AI go well.
Read our list of organisations in the AI safety ecosystem and learn what they do.
Keep up to date by subscribing to newsletters and podcasts.
Join a local group or online community, with an eye towards learning. (If you’re more knowledgeable, you could focus on advising newcomers.)
Check out AISafety.com’s list of AI safety events and training programs
If you already have experience:
Become a mentor, coach, or facilitator for a course or fellowship — BlueDot Impact and SPAR sometimes have openings, and other programmes might also want your help. (Even if you don’t see an opening, it won’t hurt to ask!)
Organise a talk, workshop, or casual meetup in your area.
- To find participants, post in online AI safety communities and reach out to AI safety groups or computer science programmes at nearby colleges.
- Organising one event (even something like a casual dinner party) can be the first step towards starting a new group!
Start or join a fieldbuilding project on a part-time basis; this is a good way to test an idea, and could lead to something more substantial.
- University groups have a strong track record: MATS, a leading AI safety fellowship, started as a part-time project run by Stanford students.
- If you’re a student, you can get help starting your own university group: check out the Pathfinder Fellowship.

Find jobs in AI safety fieldbuilding

The following opportunities seem especially strong to us:

Openings at BlueDot Impact, especially the Head of Talent role
Openings at Constellation
Head of Audience, Tarbell Center for AI Journalism
Head of the Effective Altruism Infrastructure Fund, Centre for Effective Altruism

There are many other impactful roles within the field. We don’t have a single ‘fieldbuilding’ filter, but most of these roles fit the category.

Learn more

Top recommendations

If you aren’t very familiar with AI safety yet, we recommend starting with our problem profile on risks from advanced AI and our guide to careers focused on helping AGI go well
Our career profiles on founding new projects, communicating ideas, and operations management in high-impact organisations
Asya Bergal, a grantmaker at Coefficient Giving, shares her own case for the importance of AI safety fieldbuilding
Leaders from Kairos and Constellation, two major fieldbuilding orgs: AI Safety’s Biggest Talent Gap Isn’t Researchers. It’s Generalists.

Further recommendations

Our top tips for successful networking
Our career profile on research management
Some of our favourite books on management and entrepreneurship
- Managing to Change the World (or Peter Wildeford’s notes on the book)
- High Output Management
- Good Strategy, Bad Strategy
- The Lean Startup
As of March 2026, Coefficient Giving offers grants for AI fieldbuilding projects and events on AI risk and related topics
Abbey Chaver of Coefficient Giving talks about the value of biosecurity fieldbuilding
Kuhan Jeyapragasan describes his experience growing Stanford’s student effective altruism group and founding a successful research fellowship along the way
An AI policy specialist describes how meeting fieldbuilders changed the trajectory of his career

Acknowledgements

Thanks to Arden Koehler, Asya Bergal, Benjamin Todd, Jake McKinnon, Lou Moran, Neav Topaz, Sudhanshu Kasewa, and Zershaaneh Qureshi for feedback on earlier drafts of this article, and many others for helpful conversations.

My lover, effective altruism by Natalie_Cargill

Natalie_Cargill — Sat, 18 Apr 2026 06:38:26 +0000

Crossposted from Substack. This post is part of a 30-posts-in-30-days ordeal at Inkhaven. All suboptimalities are the result of that. This is part 2, here is part 1 in my EA mini series!

On my way to my tenth EAG in a decade, my brother-in-law explained effective altruism to me.

At first, he couldn’t quite remember if he’d heard the phrase before. But he searched the corners of his mind until the definition made itself known: “yeah, it’s just a bunch of wankers who pretend to have social impact, but all they do is go to conferences and raise money and they’ve never had any impact at all.”

I have never had any chill. I did not develop it in that moment. If anyone is going to say EA is just a bunch of wankers, it’s going to be me, newbie.

“That’s interesting, Ben, but I think it’s a serious misconception — you might not be aware that EA has literally raised billions of dollars for global health charities, which very likely saved hundreds of thousands of children’s lives (do you hate children, Ben?) They have literally stopped millions of hens being tortured in cages too small for them to ever stretch their wings; they literally called it on AI and pandemics (do you LIKE pandemics…)”

At three literally-s, my husband usually intervenes. It’s a risky move for him. I was glad he did.

It gave me a moment to notice my blood was pumping faster than was reasonable for someone providing context on the track record of a movement. It let me wonder why, when I felt cornered, I’d reached for bednets and caged hens, when I think EA’s biggest impacts will be on AI and culture.

It made me realise that the words I was saying had a c. zero per cent chance of resulting in the thing I wanted — which was to transmit with all of my cells what it felt like to be 24; unsure if you were insane or the world was; and meet your people at a time your people were up to some of the craziest shit you’d ever seen.

I didn’t get a chance to tell Ben. (OK, I didn’t want to.) I want to tell you.

Long before we met

I was a sensitive child, and no one in my life ever made any fucking sense.

I was sensitive in the sense that everything horrified me.

History class disturbed me (what do you mean people burned to death under rubble in their thousands and we called it “the Blitz” and now we should go and eat lunch??)

TV disturbed me (what do you mean the wolf is going to eat that rabbit. We have a rabbit! And now we should go and eat dinner??)

No one ever making any sense disturbed me.

“How come I was allowed to go to Hannah’s last week but not now?” (because I said so)

“How come I have to eat this nut roast that I hate, just because I want to be vegetarian?” (because people are starving)

“How am I supposed to sleep when there are nuclear bombs?” (don’t be silly)

Right before we met

I got older, and was still sensitive, and still no one made any sense.

I wanted to find a place for my sensitivity. I applied for five universities with a screed on the nature of suffering, and the ways in which art shows us we are not alone in it (I’d share an excerpt but I’d prefer to actually die).

Four instant rejections, and I spent three years at Oxford reading everything I could from Beowulf to Virginia Woolf. It seemed there were a lot of other people who were sensitive, but they were mostly dead or my tutors. It seemed there were a lot of other people who wanted things to make sense, but they were mostly not doing a degree in poems.

I then tried finding a job that made sense. I worked at an international non-profit that was going to end hunger. My job, as an expensive proto-Claude, was to sit in on board meetings and write the minutes. Board members would say egregious things like, “well, the Bangladesh programs never really existed, per se”, and then turn to me and say don’t write that down. Hmmmm.

I then got a job working with an organisation at the United Nations Human Rights Council. They work on the rights of women and girls in the Middle East. JACKPOT, I thought. FINALLY, I thought. OH NO, I thought, moments later, when I realised that I was the “cute young woman” hire, and not the “do things” hire (a position that was not being hired for).

Our job was to take women from the Middle East and educate them on human rights in Geneva, Switzerland. Inconveniently, none of the women we brought over from the Gulf States had the slightest interest in learning about human rights, and even if they had, it was very unclear what this would mean in their lives of luxury shopping and spoiling their children (both of which they were very happy with). As was entirely within their (human) rights, they wanted to go to Hermès and then go home. I wrote another screed — truly the genre of my heart — and left.

2014

Imagine all of this. Then imagine getting hit in the face with a GiveWell spreadsheet.

I’ve never been into BDSM, but I imagine the hit is similar. I mean, look at it:

For those who can’t make sense of this (fair), it is the 2025 version of a spreadsheet I opened around a decade ago. It is a line-by-line estimate of how many lives a dollar saves when you spend it on malaria bednets — with every assumption and uncertainty visible.

Imagine writing notes for a board that’s telling you not to, until suddenly someone sends you this spreadsheet with an explanation of why they prioritised this problem, and why they chose this solution (my heart!)

Imagine you’re used to fundraising pitches that go “with $100 you can build a university”, and suddenly people are telling donors about issues with their 90% confidence intervals (and that like, their whole moral theory might be wrong so this might be a waste of money or grave moral atrocity).

Imagine you’ve always been told you’re overthinking in a weird way, and suddenly you’re reading blogs about suffering in fundamental physics, or at least thinking about building totems to aliens.

Imagine you’ve always been the smartest nerd in class, and suddenly you can yap with the youngest philosophy professor of all time (and all the even smarter people he continually berates himself for “not being as smart as”).

Imagine you’ve been a lawyer and seen people sell out their spouses for $5,000, and suddenly you have friends who double their investment banker salary at 20-something, donate all the money to charity, and take a c. one hundred per cent pay cut to work at a nonprofit.

Imagine your entire life you’ve felt alone and crazy with questions such as wtf wolves and wtfffff nuclear bombs, and suddenly you’re surrounded with people asking wtf alongside you (as well as wtf shall we do about it!)

And imagine that — alongside all of this — you are falling in love with another nerd who won’t stop sending you love letters, blogs about building totems to aliens, and (for reasons that are no longer clear to me) taxing thought experiments about dungeons and boxes.

Imagine it is the first time you fall in love, but you’re falling in love with the first community to ever make sense to you, and the people within it are trying hard to do good and be honest, and the love is mutual, and it’s the most intense thing you’ve ever felt.

It was a time.

We know what happens to the times, of course.

But that is another piece.

This was a piece about falling in love. Part three will be about this shit you do when you’ve fallen.

Part 1 here, Part 3 coming soon! (If you are reading this on the EA Forum, please subscribe to my Substack — I would like to move from screaming into the void to screaming at strangers online!)

AI for decision advice by Forethought

Forethought — Fri, 17 Apr 2026 21:40:30 +0000

We’ve written about why we think AI character — the behaviour of AI systems — will have a massive impact on how well the intelligence explosion goes, and why we think that there would be big benefits to giving AIs proactive prosocial drives — that is, behavioral drives beyond refusals that benefit broader society beyond just the user.

One domain that seems potentially important for AI character is assisting humans in making important decisions. As AI becomes smarter and wiser, people are using it more and more for advice. If AI accelerates technological progress and other developments, people may need to rely on AI advice to understand what’s happening and make effective decisions. If so, those that rely on AI more may be more successful and have outsized influence. The advice they receive might really matter!

So I thought it was worth brainstorming important future scenarios in which people ask AI for advice. I wrote out the advice I hoped AI would give and compared this to the answers from ChatGPT, Claude, and Gemini.

My main updates:

Challenging the framing. In high-stakes scenarios, it often felt important for the AI to explicitly flag how important the decision was and ask the person whether they were approaching it in the right way. Should they loop more people in, seek more information, consider a broader set of options, or instigate a more comprehensive decision-making process?
- By contrast, current AI often jumped into giving a detailed analysis of the question posed, even when they could have recognised that they didn’t yet have enough context to provide a helpful analysis.
Transparently flagging prosocial considerations. If the person was missing or underappreciating an important ethical consideration, I sometimes wanted AI to proactively raise it. Not to apply pressure, but simply to flag that it was potentially important and give the person the opportunity to take it into consideration. This has to be carefully balanced against AI being annoying or pushing an agenda.
- Again, frontier AIs didn’t flag these considerations as much as I’d have wanted.

The full post contains:

Draft text for the model spec / constitution on how the AI should advise humans.
An explanation of why I proposed this draft text.
Example prompts and responses demonstrating behaviour I thought was desirable.
An appendix with the answers that frontier AIs gave to the questions.

This article was created by Forethought. Read the full article on our website.

MCL: Watermarking LLM-generated Text with Markov Chains by Kyuhee Kim

Kyuhee Kim — Fri, 17 Apr 2026 21:14:33 +0000

By Chengheng Li Chen (AI Safety Barcelona / EPFL) and Kyuhee Kim (MATS / EPFL)

The original work was done as a part of the Apart Research Technical AI Governance Challenge, and this blog post was written as part of the Apart Lab studio.

TL;DR: We built a tool that can secretly flag texts made by large language models, without access to the model itself. It works by limiting the vocabulary using a pattern based on cryptographic hashes. In our experiments, we detected all flagged texts quickly and with no false positives, using just one hash function. This tool could help meet the EU AI Act requirements starting in August 2026. However, we haven’t yet tested how well it holds up against smart paraphrasing attacks.

Background

While the volume of AI-generated content is exploding, the European Commission recognized the need to build systems that let people easily check it under Article 50 of the EU AI Act. In December 2025, the detailed requirements for this, such as metadata, watermarks, and detection tools, were released in a draft Code of Practice. Here, they noted that any solution needs to be reliable (low error rates), accessible(verifiable by third parties without the original model), robust (survives text edits), and consistent across topics and writing styles.

However, with respect to these standards, existing detectors such as DetectGPT and GPTZero, which analyze statistical patterns, can’t provide mathematical guarantees. Their error rates depend on writing style, topic, and length. Watermarking approaches like the green/red list method are more reliable, but their robustness against editing hasn’t been proven.

Our objective was to develop something more rigorous that can meet the standards: a watermark that can’t be forged without a secret key, provides explicit error guarantees, enables verification without model access, and degrades predictably under text modifications. This led to the Markov Chain Lock watermark (MCL).

How MCL works

Most text AIs use autoregressive models, which generate a small piece of text (token) at a time. MCL uses this to embed a watermark. Figure 1 shows a simple toy example of a model with a vocabulary of 12 words.

Figure 1: How MCL works at a glance. The vocabulary is split into 4 groups using a secret key and SHA-256.

Step 1. Split the vocabulary into secret groups.

Using a secret key and a cryptographic hash (SHA-256), each word is put into one of several groups. Here, the words are split into four groups: blue {the, is, in}, orange {a, of, to}, green {and, for, that}, purple {on, with, by}. The important things are that (1) without the secret key, all the group assignments seem random, and (2) they are changed if you change the key.

Step 2. Add a hidden cycle during text generation.

Suppose that there’s a rule that allows each group to move only to certain other groups. The simplest pattern, called the “clockwork” pattern, makes the model cycle through the groups in order: 0, 1, 2, 3, 0, and so on.

In Figure 1, the model starts in state 0 and picks “the” from $V_{0}$ . Then, the cycle moves to state 1, and it must pick from $V_{1}$ (here, it chooses “a”). In state 2, it chooses “and” from $V_{2}$ , and in state 3, “on” from $V_{3}$ . At Last, the cycle moves back to state 0.

We can loosen this rule in practice. For example, instead of always picking the next group in the cycle, the model can sometimes skip ahead by one group. So from group 0, it could choose from group 1 or group 2. We call this the “soft cycle.” This gives the model more flexibility but still leaves a clear pattern for us to detect later.

Step 3. Calculate fingerprint score.

Now, you need to count how many pairs of words follow the expected cycle. That’s what we call a fingerprint score ( $φ$ ) and it follows like this:

$φ (t) = \frac{1}{n - 1} \sum_{i = 1}^{n - 1} 1 [token i and token i + 1 follow the pattern]$

(where $t$ is the list of words and $n$ is the number of words)

So, it is the fraction of word pairs that match the expected pattern. In the case of “the a and on”, since each word is in group 0, 1, 2, and 3, and three transitions match, the score is 3 out of 3, or 1.0.

When you use this on text without a watermark, the score averages around k/S, because words fall into random groups and most pairs don’t follow the cycle by chance. In our best setting (S=7, soft cycle), this baseline is about ²⁄₇, while watermarked text scores close to 1.0. This difference is big enough for perfect detection, as our experiments show.

The best part is that this method doesn’t require access to the original model. You only need the secret key and a simple hash function. A regulator could check thousands of documents on an ordinary laptop.

What we can prove

As a watermarking method, MCL has four advantages over other methods.

First, false alarms become basically impossible for longer texts. The probability of accidentally flagging a normal human-written text as AI-generated drops fast as the text gets longer. It can be proven using a standard statistical concentration bound (Hoeffding). The False Positive Rate (FPR) shrinks exponentially with text length as follows:

$F P R \leq exp (- 2 (n - 1) {(τ - \frac{k}{S})}^{2})$

(where $n$ is the number of tokens in the text, $τ$ is the threshold we use to decide “watermarked or not” (set it at 0.5). A random unwatermarked text would score the ratio $k$ / $S$ on average, where $k$ is how many groups are allowed at each step, and $S$ is the total number of groups.)

At 100 words, this gives less than a 0.011% chance of a false alarm; at 150 words, less than 0.0000012%. For anything longer than a short paragraph, false positives are almost impossible (Figure 2).

Figure 2: False positive rate drops exponentially with text length. Each line shows how FPR decreases as text length increases, for different state counts. The green dots mark two key milestones on the S=7 curve. The dashed line marks the 0.01% reference level.

Second, we know exactly how much editing the watermark survives. In the case of text revision, the fingerprint score can be calculated as follows:

$E [φ] = (1 - δ)^{2} + \frac{k \cdot δ (2 - δ)}{S}$

(where $δ$ is the fraction of words changed (0 means no edits, 1 means every word was replaced), $k$ is the number of groups allowed at each step, and S is the total number of groups.)

The first part of the formula captures the word pairs that survived unchanged; the second part accounts for the fact that some of the newly generated random words might accidentally follow the cycle by chance.

Even if someone randomly swaps words, they would need to change at least 45% of them before the watermark disappears when S = 7. In practice, this means rewriting almost the whole text. The bigger challenge is modern AI paraphrasing tools, which can rewrite sentences smoothly without obvious edits. These might break the watermark with fewer changes. We discuss this further in the Limitations section.

Third, the watermark can’t be forged or removed without the key. The secret key is the only way to know which group each token belongs to. Without it, the group assignments look completely random, so an attacker cannot reconstruct the pattern or produce a valid watermark.

Last, text quality takes a small, bounded hit. Restricting the vocabulary obviously affects text quality, but we can predict the cost, which grows only proportionally to log(S). On the reader’s side, this is noticeable but not dramatic, and adding more groups makes it worse only slightly. In our experiments, perplexity increased from about 3.8 for unconstrained text to 4.2 with S=7.

Experimental Setup

We used Llama-3.2-3B-Instruct (with ~128,000 tokens) and the soft cycle (k=2) on 173 Wikipedia prompts spanning science, history, tech, and culture, each generating up to 200 tokens (~150 words).

We tested models across the number of vocabulary groups ({2,4,5,7,9,11,15}) and the percentage of tokens shared across different vocabulary groups (overlap level: 0%, 5%, 10%, 15%). At 0% overlap, each token belongs to exactly one group, strictly enforcing the cryptographic constraint, while higher overlap gives the model greater flexibility.

Across 28 configurations, we measured fingerprint scores, detection rate, false alarm rate, and perplexity. For results on Mistral-7B and GPT-2-XL, see Appendix B.

Results

With S ≥ 5 and 0% overlap, every watermarked text scored higher than every non-watermarked text across all 173 prompts. A single threshold at 0.5 produced zero false positives and zero missed detections (Figure 3). Among all configurations that achieve perfect detection, S=7 produces the most natural-sounding text (perplexity 4.2), making it the best overall setting (Figure 4).

Figure 3: Perfect separation between watermarked and non-watermarked text. Each pair of violins shows the distribution of scores for watermarked (green, left) and non-watermarked (red, right) text. The dashed lines inside each violin mark the mean score. For every state count S ≥ 5, the two distributions are completely separated by the τ = 0.5 threshold, meaning zero false positives and zero missed detec

Figure 4: Why S=7 is the sweet spot. Left: every state count from S=4 to S=15 achieves 100% detection at 0% overlap. Right: perplexity (text quality cost) varies across configurations, with S=7 producing the most natural-sounding text (lowest PPL of 4.2).

Figure 5 shows that the robustness formulas also held up empirically. Randomly replacing words weakens the watermark, but importantly, it follows the prediction from our formulas. Even when 30% of tokens were replaced, detection remained above 96%, matching theoretical predictions within 1–6 percentage points. To evade detection, attackers would need to corrupt nearly half the text.

Figure 5: Theory matches reality. Left: the theoretical formula (dashed orange) predicts the observed fingerprint scores (blue) with outstanding accuracy, typically within 1-6 percentage points. Right: detection stays above 96% even with 30% of tokens randomly modified.

In addition to the findings, two failing configurations are worth flagging. One is that using S=2 with the soft cycle produces no signal, since every transition is allowed. The other is that any overlap between groups sharply degrades performance. As shown in Figure 6, even 5% overlap at S=7 drops detection from 100% to 92%. This means once tokens can belong to multiple groups, the cryptographic constraint no longer holds.

The results imply that MCL must be run at 0% overlap, but for Llama-3.2, the model still has roughly 18,000 tokens available at each step. So output quality is generally fine, though edge cases like code generation can be affected.

Figure 6: The overlap cliff. As the overlap ratio increases (x-axis), both the detection rate (green) and average score (blue dashed) drop steeply. The shaded red zone is the “cliff region” where the cryptographic constraint breaks down.

Limitations and open questions

Three issues deserve upfront acknowledgment.

Tokenizer coordination. MCL operates at the token level, and different models tokenize text differently. To verify a watermark, you need to know which tokenizer was used. The technical fix is straightforward: include the tokenizer information in the metadata (using C2PA or something similar). But getting all AI companies to agree on a metadata format before August 2026? That’s a coordination problem, not a technical one, and those are often harder to solve.

Untested against intelligent attacks. Our robustness tests only swapped out random words. A determined attacker would likely use another AI to paraphrase the text, which is easy to do and works well against most token-level watermarks. Our math covers any kind of token change, so the formulas still hold no matter how the text is edited. But we haven’t actually tested against AI-powered paraphrasing yet. The 30% random-replacement survival rate is promising, but it doesn’t show what happens if another LLM rewrites the text. That’s the most important open question, and we plan to address it in follow-up work (including testing against GPT-5 and Claude).

The overlap cliff. The sudden drop from perfect performance to barely working between 0% and 5% overlap needs more study. It may be possible to relax the cryptographic constraint into something softer that handles overlap more gracefully.

One promising direction is sparse watermarking. Instead of applying the rule to every word, we’d use it only every few words. This lets the AI write more freely and makes the text sound more natural, but the watermark would still remain.

Conclusion

Our methodology achieves 100% detection and 0% false alarms with a perplexity of 4.20 (S=7). It has exponentially low false positives, exact robustness formulas, cryptographic unforgeability, and bounded quality costs. Lastly, MCL has a real advantage in that it can verify without access to the original AI model, and we hope it will help with compliance as the August 2026 deadline approaches.

Thanks to Apart Research for hosting the Technical AI Governance Challenge.

You can check our code here: github.com/ChenghengLi/MCLW

Appendix A: Score separability

In order to see if there can be confusion between watermarked texts and non-watermarked ones, we compared the lowest score of the watermarked samples against the highest score of the non-watermarked samples in every setting with $S = 0$ at $0 %$ overlap. In every case, there is complete separation between the pairs. (Figure 7).

Figure 7: Score separation across all configurations. Each horizontal bar connects the worst-case scores: the red dot is the highest score from any non-watermarked text, and the green dot is the lowest score from any watermarked text.

Appendix B: Cross-model validation

To check MCL’s generalization, we ran the same 173 prompts on Mistral-7B-v0.3 and GPT-2-XL. All three models hit 100% detection at S=7 (Figure 8). The larger model (Mistral-7B) handled the vocabulary restriction much better, achieving a perplexity of just 1.71 compared to 4.20 for Llama.

Figure 8: MCL works across different model architectures. We tested the S=7 configuration on three models of different sizes: Llama-3.2 (3B), Mistral (7B), and GPT-2-XL (1.5B).

AI safety is maturing. Is its generalist talent pipeline keeping up? by Uladzislau Linnik

Uladzislau Linnik — Fri, 17 Apr 2026 19:43:12 +0000

TL;DR

The AI safety field is maturing, but its talent pipelines may be failing to keep pace with the growing demand for operations, organisation-building and research management professionals (further referred to as generalists).

Based on eight interviews with AIS organisations plus prior work, we see early signs that entering AIS in these roles is structurally unreliable: personal connections dominate, transition pipeline is not clear and screening criteria are inconsistent. On the organisations side, practitioners report experiencing the potential generalist talent bottleneck unevenly and how salient it feels varies by organisation stage, role type and reliance on referrals. Along the way, two other patterns surface: demand for senior operators with outside‑world fluency and fragile or missing institutional infrastructure.

If unaddressed, this combination risks leaving the field’s operations and organisation‑building capacity underdeveloped just as the field is scaling.

This post is the first public step in an ongoing investigation in trying to address this bottleneck. It shares our current observations, uncertainties and next steps. By doing so, it invites input, particularly from AIS organisation leaders to help refine the picture.

Introduction

Note: this post was produced independently of the post AI Safety’s Biggest Talent Gap Isn’t Researchers. It’s Generalists, of which we were not aware up until final drafting. We don’t include any references to or analysis of this referred post into ours and will analyse it through the next stage of our exploration.

There is growing evidence that AI safety organisations struggle to hire and develop generalist talent in operations, organisation‑building and research‑related management. As the field scales, the cost of this gap appears to be rising. Several published observations (see Appendix 2) and conversations in the field point in this direction: among them accounts of persistent hiring difficulty, researcher-optimised entry paths that leave generalists working in their secondary skillsets and upskilling options that lack the granularity generalist roles require. One possible explanation is that AIS might lack dedicated, role‑specific transition pathways for generalist talent, but the root causes may run deeper than any single missing programme.

This article documents the discovery stage of a project that will culminate in a direct intervention. It describes the first exploration phase, carried out over several months and complemented and structured within the Sentient Futures Project Incubator under the mentorship of Martyna Wielopolska. It is written with the specific aim of hearing from people closer to the problem than we are.

Call for Opinions

If your experience complicates, challenges, or disconfirms what we describe, we would value hearing from you.

We particularly invite perspectives from people closer to AIS organisation leadership, hiring and talent development, than we currently are. Our demand-side picture still needs broadening. In particular, we have not yet spoken with enough people from technical AI safety organisations and we are actively seeking that input. We are also glad to hear transition stories (successful or not) from aspiring professionals to complement our supply-side observations.

You can contribute by leaving a comment, contacting us directly (DM or generaists2026@gmail.com), or filling in our short form (~3 min) to share your preferred means of communication with some background. Thanks!

Terminology

In this post, we use the term generalists as a shorthand for professionals with deep expertise in organization building, operations and research-related management (research management, programme management and similar). While we recognize this term can downplay specialized skill sets or borderline cases—a dynamic this project seeks to challenge— here we use it strictly to categorize roles that require a broad, cross-functional range of non-technical competencies.

For a detailed breakdown of the clusters, example skills and example roles we use internally, see the taxonomy table inAppendix 3.

Sources and Methods

The observations in this post draw on three layers of evidence. The first two are (a) six months of direct experience and personal observation of the transition problem, including 10+ informal conversations with generalists navigating the same entry problem (Appendix 1), and (b) a review of 15+ published sources (Appendix 2). The third, and primary, source for the findings below is a series of semi‑structured interviews with org‑side practitioners.

We reached out to approximately 20 organisations, targeting operations and organisation-building leads across research, field-building and governance contexts. We got responses from twelve. To date we have completed seven semi-structured interviews — including one follow-up conversation with the same participant — and received one written response. The conversations included into this post have predominantly been with organisations working on AI safety field-building and governance. At least five additional interviews are underway as we publish — the findings are excluded here and will go into our further analysis. Contributors were senior staff responsible for running organisations and teams, including people leading operations, finance and hiring. They expressed their personal views and experience and we do not expect or infer their views to represent their organisations’ official position. We also cross-checked with Consultants for Impact, who work on the same talent pipeline, identifying where our findings might support their decision-making and receiving valuable methodology input.

Conversations funnel

Going into interviews, we were operating with a set of working assumptions derived from our background reading and personal observations — chiefly, that the generalist pipeline is structurally narrow, that the skill-mix required for high-leverage generalist roles is hard to find and evaluate and that existing upskilling pathways lack the granularity to make role-specific progression legible. We set out to hear whether these resonated with practitioners closer to the problem.

Our questions covered four thematic areas, adapted to each participant’s context:

Hiring channels. Whether open-market hiring proves efficient for generalist roles, or whether referrals, fellowship pipelines and transitions from adjacent Effective Altruism (EA) organisations produce better results — and what the tradeoffs look like.
Evaluating fit. How and at what stage participants assess mission alignment, cultural familiarity and ecosystem knowledge; which of these qualities are hardest to evaluate; and whether candidates could make them more visible earlier in the process.
What works and what does not. Which generalist roles and skills are consistently hardest to fill; what does not transfer well from non-AIS organisations; and what the differences look like in retrospect between generalists who thrived and those who struggled.
Ecosystem-level picture. Whether a generalist shortage exists across the broader AIS field, why participants think it persists if so, and what forms of intervention, if any, they think would make a meaningful difference.

To make sense of what we heard, we applied thematic analysis to the transcripts and written response, moving from open coding through clustering to a cross-interview comparison. Before any formal coding, we immersed ourselves in the material — reading transcripts, revisiting our personal notes and discussing initial impressions with each other. Only then did we moved into coding, where AI assisted with the initial pass; we subsequently went through everything manually and validated all codes, clusters and theme assignments ourselves.

Analysis process

For the constraints we recognise in our approach, please check Epistemic Status and Biases. We treat everything below as directional rather than definitive.

A note on LLM usage. Most data collection, analysis and drafting were done manually. We used AI notetakers (Granola, Fathom) during conversations to reduce onset interpretation bias. For initial thematic coding assistance and structural editing, we used Claude and Perplexity. All codes, clusters and theme assignments were subsequently reviewed and validated manually.

Observations

Across our initial conversations, one pattern came up consistently and without prompting: the most reliable way into generalist and operations roles in AI safety is through a personal connection. From there, the picture branches. Some findings are relatively confident; others are directional and require more data to confirm. What follows is a good-faith account of what we heard, organised around six observations and one emerging signal on root causes. We hold these findings with different levels of confidence and have tried to be explicit about which is which throughout.

These are followed by a meta-observation about possible root causes.

Observation 1: Generalists face a structural pathway gap into AIS

For candidates approaching from outside the community, there is no reliable alternative to a personal connection and several structural features of how hiring works make this harder to change than it might appear.

Referrals dominate. Across our interlocutors, four explicitly mentioned while the rest, except one, indirectly inferred that the most common hiring channel was a personal connection: a direct approach to someone already known, a recommendation from a departing staff member, a relationship built at a conference. Open hiring rounds were attempted but often produced either low volume or candidates who lacked the contextual familiarity organisations were looking for.
LLM’s degrade written applications. The widespread availability of generative AI has lowered the cost of applying so dramatically that application volume no longer signals genuine interest, and written applications no longer reliably surface whether someone can think clearly. Organisations are still searching for consistent alternatives, which leaves candidates who are strong but unknown to the community with fewer ways to make themselves legible.
Hiring might misread outside signals. Backgrounds from consulting, government, nonprofit management, or industry are harder to read for hiring managers who have spent most of their careers inside AI safety or EA, not because the experience is irrelevant, but because the signals are unfamiliar. Even candidates who apply through open rounds may face a barrier that has nothing to do with their fit for the role.
EA-adjacent legitimacy is fading with not enough replacement. For a period, completing EA-adjacent fellowships or similar programmes functioned as a recognisable entry credential for generalists trying to enter the AIS field, a shorthand for values alignment and basic context. As AI safety develops institutional identity beyond EA, this signal has become less stable and harder to read consistently across organisations. Not enough replacement signals have emerged to perform the same function reliably.
Mid‑pipeline transition steps for generalists may be missing. Researchers often have a more legible progression: read, do a course, apply for a fellowship, then aim for a role. The path is not easy or guaranteed, but it is visible: a junior researcher can usually see what the next step is, even if achieving it is hard. For generalists, no equivalent bridges are visible and might be required.

Together these features describe a pathway that works reliably for candidates already inside the community and much less reliably for those approaching from outside, regardless of how relevant their skills may be.

Observation 2: Organizations experience the talent gap differently

Whether AIS organisations are meaningfully bottlenecked on generalist talent is a harder question than it first appears, and the honest answer is that both the gap and its absence are real depending on where you sit.

Open hiring rounds have not proven effective for generalist roles.
Several participants mentioned that open hiring rounds produced few applications with varying reasons. Challenges with identifying signals were mentioned (see also Observation 5).
Talent-matching data backs up demand for operations-adjacent roles. A participant working in talent matching reported that about half of the roles they match in a given year are AI safety operations-adjacent. This is not a self-reported difficulty from a single organisation; but an observed volume from someone sitting across many hiring processes simultaneously.
One leader described the gap as resolved in their context. An interviewee from a small, established research organisation said clearly that this is not among their top challenges and their most recent operations difficulty had been resolved through a referral hire. While not in line with other conversations, until there’s more data, we read both as potentially true: the gap may be real at field level while being locally manageable for some organisations whose referral networks function well enough to fill immediate needs. We plan to hear from more leaders to understand whether a broader pattern sits behind this account and if so, what shape it takes.

These accounts reflect the same field experienced from different positions. The talent gap is real at field level but locally manageable for some organisations, depending on stage, size, hiring model and whether the right person is already hired through a referral. It is unevenly experienced and partially hidden by the same referral networks that make it hard to enter from outside. Whether this represents a single shared problem or separate misalignments across candidates, hiring managers and funders is itself uncertain. We address this directly in Uncertainty 1.

Observation 3: The field lacks professionals with outside-world operational fluency

Across all conversations and without prompting, participants described a related but distinct problem: the field skews young and demographically homogeneous in ways that limit its capacity to engage the outside world effectively. The precise version of this finding matters.

Motivation exists, experience is scarce and takes years. There might be no shortage of junior, motivated, community-adjacent generalists seeking entry. The shortage is at the other end: senior professionals with years of accumulated judgment in operations, management, institutional engagement, communications, or policy who bring skills that cannot be acquired quickly. Several participants suggested that, for most generalist roles, a competent senior professional can get to a working level of AIS context in weeks, whereas it takes years, sometimes decades, to develop the professional judgment, external relationships, and organisational experience that senior roles require.
Seniority is a proxy for what is actually needed: diversity and outside-world fluency. What the field needs is people who have operated in environments outside AI safety and EA, who understand how businesses are run, how government institutions work, how professional norms function in other sectors and how to communicate credibly with people who do not share the field’s cultural vocabulary.
Several participants named this as an increasingly urgent gap. As the field grows and needs to engage more directly with industry, policy and the public, the absence of people who can operate credibly in those environments becomes more visible and more costly. One participant confirmed the scarcity directly: the average age of colleagues in AI safety is well below that of any other sector they have worked in^[1].
One participant described running informal training sessions on outside world best practices and professional norms, including people management, setting up organisation health metrics and standard operation procedures etc. They described the experience as one with a lot of friction but big potential value.

If this pattern holds, the relevant question may be not only whether AIS has enough generalists, but more broadly whether it has enough people who can import and translate outside‑world operational fluency into the field. What the field does instead of seeking this talent is the subject of Observation 6.

Observation 4: Organisations default to hiring from within the AIS ecosystem, at a measurable cost

Across the organisations we spoke with, hiring for operations and generalist roles drew from a narrow pool within the ecosystem.

The hiring pool stays within the ecosystem across multiple mechanisms. Research managers are sourced from past fellowship cohorts. Researchers stepped into management because the organisation lacked an external alternative. In early-stage organisations, leadership multi-hats across operations and research, deferring any external hire. The mechanisms differ but the effect is consistent: outside-world professionals rarely enter through any of these channels. For referral dominance specifically, see Observation 1.
The opportunity costs of this default are identifiable, and fall on both organisations and individuals. One participant described operations functions run by people without an operations background making decisions from a research frame, producing outputs that required correction and additional training, a quality cost the organisation was actively managing. Another described researchers stepping into management out of a sense of duty rather than fit, acquiring people leadership skills on the job through external courses and AI tools.
Outside preference is widely shared but debated role-wise. Most participants expressed a preference for bringing external talent into operations and programmes roles, though the strength of that preference is still debated for certain for some specific roles. For research manager roles specifically, participants disagreed: one argued that research-adjacent context is essential and a strong project manager without it would struggle; another argued that context is acquired in weeks and seniority matters more. This disagreement is genuine and unresolved.
Best hires learn fast and deliver impact. A senior professional with zero AI safety experience was hired into an institutional engagement role. They acquired sufficient context in weeks, and have produced a very significant impact on the organisation’s efficiency.
The strongest hiring signal for operations roles is nonprofit sector experience (where it has been named). Not AI safety experience, and not generic professional experience, but specifically experience in mission-driven organisations with compliance obligations, funding cycles and operational rhythms distinct from commercial environments. This is a more precise and more actionable claim than the general outside-world fluency argument and it came from a participant with the highest volume of direct hiring exposure in our sample.

Taken together, these accounts describe a field that is most comfortable hiring from within its own ecosystem, even where the strongest gains may lie in bringing in seasoned operators from outside. At least, the costs of this default are now nameable.

Observation 5: Organisations have no shared screening criteria for generalist roles

Participants held meaningfully different views on what they were actually looking for when hiring for generalist and operations roles and these differences lead to different hiring decisions with direct consequences for which candidates succeed.

Some weight mission alignment most heavily. In this view, alignment is the foundation everything else builds on. Context can be developed once someone is in post; hiring someone who is not genuinely committed to the mission is a harder problem to fix. A candidate strong on alignment but low on context will be screened in.
Others invert this priority. In this view, professional competence and demonstrated skill are the primary signals. Alignment is largely self-selecting given the field’s working conditions: anyone willing to apply to an AI safety role at current salary levels and career visibility is already signalling something meaningful. Over-screening on alignment unnecessarily crowds out strong candidates. A candidate from outside the community with strong professional credentials may be immediately legible to one hiring manager and nearly invisible to another.
For the orgs, the alignment signal is also becoming harder to read. As AI safety grows beyond EA, the vocabulary in which alignment is expected to be expressed is shifting. Candidates who are new to the field and genuinely committed may fail to signal that commitment, not because it is absent, but because they do not yet know the language. Meanwhile, candidates who do know the language may be performing familiarity rather than demonstrating fit.
Hiring managers might conflate low context with low alignment. The distinction between alignment (caring about the problem) and context (accumulated knowledge) is not yet operationalised in most hiring practices.
Cultural fit can be a separate barrier. What actually matters is comfort with the specific operating norms of AI safety organisations, extreme reasoning transparency, rejection of conventional norms, cost-effectiveness orientation, which are culturally distinct even within the broader EA ecosystem. A candidate can be deeply aligned with reducing AI risk and still find the cultural environment difficult to operate in.
Work tests dominate operations hiring. For operations roles specifically, a work test demonstrating practical skill was the dominant hiring signal regardless of where participants stood on everything else. This is the one area approaching consensus and it is worth noting that it sidesteps the alignment/context debate entirely by testing demonstrated output rather than inferred values. [Note: Early data suggests uncertainty about whether the design of the tests screens for the skillsets required for the job—the challenge also indicated in our earlier sources (Appendix 2). We plan to explore it further.]

The alignment-first position predicts that context-gap hires will underperform on mission-critical decisions; the context-first position predicts that alignment-gap hires will underperform on professional execution. Both predictions are plausible. Neither has been tested systematically in the field. The downsides can be manifested through potential over-screening.

Observation 6: The gap may extend to institutional infrastructure, not just people

One participant raised something we had not anticipated: what is missing in AIS operations may not be only people, but institutional infrastructure, documented processes, offboarding procedures, knowledge management systems and measurement frameworks. This was an unprompted observation that gives language to something several other participants described implicitly (describing consequences, or patterns that point in the same systems gap direction).

We include it because if it is right, it might change what an intervention needs to address. We return to this in Uncertainty 1.

The field currently runs on individual effort rather than institutional memory. When someone leaves, their operational knowledge leaves with them. One participant described working at an organisation six years old with no documented record of how decisions had been made, requiring them to trace institutional history through chains of former staff.
Even where organisations hire well for operations roles, the absence of systems means the value created is fragile. Adding people to a system without processes does not compound; it resets each time someone leaves.
The field may be transitioning from operational survival to operational excellence. Two participants framed it this way explicitly: the current phase is about moving from scrappy compliance to intentional infrastructure. Whether the field has the people and the cultural conditions to make that transition is an open question.
Operational excellence requires advanced success metrics. The field knows what it is optimising for in principle but has not yet developed the measurement frameworks to track progress toward it in practice. One participant described navigating this directly: accustomed to commercial metrics like revenue and product performance, they found no equivalent frameworks waiting for them in an impact-driven context and noted that building those frameworks is itself an open and active undertaking for the organisation.

If others recognise this pattern, the implication is not only that more people are needed, but that the people already in place, whether hired externally or grown internally, are operating without the institutional scaffolding that would make their contributions durable and measurable. Talent and systems may need to be addressed together rather than sequentially.

Meta-observation: Emerging hypotheses about root causes (thin evidence)

The six observations above describe features of the gap: its presence, its unevenness and what it consists of. This section steps back and asks why those features exist. This observation sits at the edge of what our current data can support, combined with synthesis-based interpretations. We include it because several themes about why the patterns above exist appeared independently across conversations. We are not claiming these are confirmed root causes. We are naming them as plausible contributing factors that point in a consistent direction.

The field’s hiring norms and cultural expectations reflect an origin that might no longer match its current needs. AI safety grew from communities that skew young, research-optimised and internally referential. Those origins shaped what counts as legitimate talent, how alignment is signalled and who feels like a natural fit. As the field grows and needs to engage more directly with industry, policy and the public, the distance between its internal culture and the “outside world” professional environments it needs to draw from is becoming a practical constraint.
The gap may be relational, not knowledge-based. One participant argued this most directly and no other source contradicted it: the barrier for senior outside-world professionals is not that they lack knowledge of AI safety; they can acquire that in weeks. The barrier is that there is no infrastructure connecting them to the field. In our interlocutors’ view, there are still few visible pairings, entrepreneurial connectors, or structured pathways reaching senior professionals in adjacent sectors. Informal brokerage is currently patching what structured infrastructure should be doing. This is a different diagnosis from the upskilling framing that most interventions start from and it implies a different kind of response.
The talent gap and the systems gap may be mutually reinforcing. It is not clear which comes first: whether the absence of people prevents systems from being built, or whether the absence of systems makes it harder to attract and retain the right people. We are naming this as a hypothesis rather than a finding, but if it holds, addressing only one dimension may leave the other intact.

One tension worth naming explicitly: some participants see these features as structural and sticky and unlikely to resolve without deliberate intervention. Others see the field as already self-correcting: as it grows, it will naturally draw in more senior, more diverse, more professionally experienced people. Both positions are internally consistent. Which is right has direct implications for whether any intervention is warranted at all. We are carrying this tension openly into the next phase.

What these observations add up to

These findings are directional, not definitive. They represent what we heard across a small number of conversations, not a representative sample of the field. Several of them complicate each other: an organisation that has resolved its hiring through a referral may be masking the same structural problem that another organisation experiences acutely—a gap exists, but is unevenly experienced and partially hidden by the referral networks that make it hard to enter from outside. The candidate-side finding is the most robust: the pathway into AI safety for generalists approaching from outside the community is structurally unreliable, regardless of how relevant their skills are.

What we are less certain about is the precise nature of the gap, whether it is primarily a talent problem, a systems problem, a cultural problem, or some combination of the three, and whether the right response is a structured entry pathway, a relational infrastructure intervention, or something the field has not yet named. The uncertainties section addresses what remains genuinely open and what we are carrying into the next phase of research.

Uncertainties

The observations above describe what we found. This section maps what we do not yet know; what, if we learn it, would most change our picture of the problem; and what should happen next. The main uncertainties structure our current thinking and further exploration:

1. What is the shape of the talent gap?

2. Will the current patterns self-correct over time?

The further questions sit underneath these two:

3. Does the gap require deliberate intervention at all?

4. [if an intervention is needed] Which problem is the most painful?

Uncertainty 1: What is the shape of the gap?

Assuming the gap exists, and our evidence is directional rather than conclusive, the more important question for the next phase is what shape it takes. We see at least four plausible interpretations of the same underlying pattern:

Field-wide, continuous bottleneck. Organisations need this talent and cannot reliably find it; candidates have the skills and cannot get in. The gap is structural and persistent.
Three separate problems (hiring managers, candidates, funders). Each stakeholder group experiences a different misalignment that only looks like one problem from outside. Hiring managers cannot find candidates who meet their bar; candidates cannot find a way in; funders want to support organisations in scaling but find those organisations cannot grow because the talent to do so is not there.
Local, episodic bottleneck patched by referrals. Organisations solve individual hiring rounds through referrals and internal promotion. The gap is real but invisible from inside any single org that has filled its roles. What looks like a field-wide problem may be a structural feature that surfaces acutely at moments of turnover and then disappears.
Role-specific and stage-dependent. The gap is concentrated in particular places: senior operations roles at early-stage organisations, research manager roles requiring both professional judgment and domain context, institutional engagement roles requiring external relationships the field has not built. A general “generalist pipeline” may be the wrong level of abstraction.

Our evidence is directionally consistent with above interpretations 1, 2, 3. Interpretation 4 comes with limited evidence but adds additional angles to other accounts.

Uncertainty 2: Will the current patterns self-correct over time?

A question running underneath all of the observations is whether the patterns they describe are temporary features of a young field that will self-correct as the community grows or structural characteristics that will persist and compound.

Two views emerged across conversations:

Self-reinforcing view. In this picture, referral networks are self-reinforcing rather than self-expanding and the credential signals that once helped outsiders enter the field are weakening without clear replacements. The result is a pathway barrier that remains even as the field grows.
Self-correction view. In this picture, as AI safety grows and gains visibility, it will naturally draw in more senior, diverse, professionally experienced people. The field is already evolving beyond its EA origins; what we are seeing now is a transient phase, not a permanent feature.

Both positions are internally consistent. What we cannot yet distinguish empirically is pace and mechanism: even if self-correction is happening, we do not know whether it is happening fast enough relative to the field’s growth, or through processes that are legible and accessible to candidates outside current networks.

Uncertainty 3: Does the gap require deliberate intervention at all?

The two uncertainties above assume the gap is real but under-specified. This section holds open a more fundamental possibility: what we are observing may be a feature rather than a problem, or a problem that resolves itself without deliberate intervention.

The current path, with high autonomy, uncertainty and competition, might be the most effective way to filter out false positives and ensure talent quality and density.
We might have misinterpreted or missed other upskilling paths that are robust enough to address the reported bottleneck.
While we exclude AIS incubators from the assessment following the assumption that not every generalist is necessarily a founder, it is possible that they cover some of the demand through the Silver Medalist mechanism and founding builds generalists’ career capital, even if the project fails.
Organisations might successfully address needs by training researchers into leaders, and this trend will become more obvious as more AIS research gets automated, especially for roles where tech knowledge is important.
It might appear actually beneficial for an aspiring generalist to transition through a research pipeline, to get hands-on experience in the sphere.

These are not afterthoughts. If any of these explanations turns out to be correct, the nature of the right response changes substantially and may be no deliberate response at all.

Uncertainty 4: Which problem is the most painful?

Taking all the observations together, and assuming our uncertainties resolve, we can point to several candidate problems, described in Observations 1-6: a structural pathway gap for generalists approaching from outside the community; a possible shortage of senior talent with outside‑world fluency; fragile or missing institutional infrastructure in some organisations; and persistent disagreement on what to screen for when hiring generalists, combined with a default toward internal promotion. At this stage, we do not know which of these is doing most of the work for the field as a whole, or whether they compound each other. Part of the next phase is to understand which of these candidate problems, if any, is most binding and therefore most sensible to address first.

Epistemic Status and Biases

Our observations are our subjective interpretation of the evidence accumulated so far. We treat everything in this post as directional rather than definitive. The current picture is also skewed toward field‑building and governance organisations rather than technical AI labs, and we expect further conversations to update it.

As generalists undergoing self-funded transition into AIS (see Appendix 1), we are biased toward recognising the gap. We name several further sources of bias that readers should hold in view:

Availability bias. Articles describing problems are published and indexed more readily than “everything is fine” pieces. Conversations oriented around a gap tend to be more substantive than ones oriented around its absence. This effect may be reinforced by our distance from in-person AIS communication. To mitigate it we are actively asking, and inviting, disconfirmation.
Insufficient sample size. Constrained by our Sentient Futures incubation timeline and factors we could not control (outreach latency, scheduling), we scoped this phase by time rather than by saturation. Seven interviews and one written response are not sufficient for saturation, and we know that. The current picture is also skewed toward non-technical research and field-building contexts, and we expect conversations with technical AIS organisations to update it. We plan to scale the exploration within and beyond the incubator. See Further steps.
Confirmation bias toward a particular solution. Our initial working hypothesis was a granular generalist fellowship, such as operations for AIS governance (see also Appendix 1). This may have shaped the way we gathered and weighted information, even where we tried to remain open. To mitigate it we kept solution discussion to minimum, keeping all the questions designed around the nature of the gap.
Hindsight bias. The sequence of background facts presented here has been structured for coherence. The actual process of arriving at them was more fragmented than the narrative suggests. We have tried to mitigate it through putting background information as a secondary source of information in the Appendices.

Further Steps

We see this post as the end of the first diverge phase, not the end of the project. The next phase is about finishing the divergence, then converging on a single, well-specified problem to start testing product-market fit. The end goal is to build an intervention that sustainably addresses this well-specified important neglected problem and by this ensures quality, quantity & diversity of generalist talent to match the needs of the AI safety field as it is maturing and aiming for scaling and sustainability. However, the immediate further steps:

1. Complete the divergence

Broaden the organisation-side picture by adding around ten conversations with people responsible for hiring and team-building at technical AIS organisations and labs, to roughly equalise representation of tech, field‑building/governance and transitioner perspectives at similar depth.
Invite structured reactions to this post (comments, private messages, short-form responses) and, if needed, run one or two small additional tools such as a short questionnaire to clarify specific bottleneck hypotheses.

2. Extract and prioritise concrete problems

Translate the observations into a small set of discrete candidate problems (for example, a structural pathway gap for external generalists, a shortage of senior outside‑world fluency, missing institutional infrastructure, or disagreement on what to screen for in generalist hires).
Use explicit criteria such as importance, tractability, neglectedness and fit with our skills to decide which of these is most binding for the field and therefore most sensible to address first.

3. Move from problem to prototype

For the first problem we choose to tackle, sketch a minimal theory of change and design the smallest viable experiment (for example, a limited matching or systems-support pilot) to test whether our understanding of the problem is correct.

4. Conditions for pausing or closing

We are preparing a more detailed research proposal to continue this work on a sustainable footing. As with any early-stage project, runway, low counterfactual value, or life events may force us to pause or close. If that happens, we expect to at least share an update that ties off this phase and makes what we have learned available to others who might be better placed to continue.

Conclusion and call to action

We have found early, directional evidence of a candidate-side pathway gap for generalists, and a mixed organisation-side picture in which some organisations report persistent hiring difficulty while others have resolved it through referrals. Taken together, the observations point to several different candidate problems, pathway reliability for outsiders, senior outside‑world fluency, systems and infrastructure and disagreement on what to screen for, but we do not yet know which of these is most binding, or whether the current patterns will self‑correct without deliberate action.

At this stage, the most useful thing we can do is to refine our picture with input from people who are closer to the problem than we are.

If you work at or run an AIS organisation, especially technical labs, field-building, or governance organisations:

In your most recent hire for a generalist role (including operations), how did the candidate actually come to you and is that a process you would choose to repeat? What, if anything, stopped you from doing it differently?
When you have hired or tried to hire for these roles, what did you find yourself actually selecting for and were there important qualities that the process consistently failed to surface or test?
Beyond generalist profiles specifically: are there professional backgrounds or types of experience from outside the AI safety space that you think the field is currently missing or underutilising, whether or not they fit the “generalist” label?

If you have tried to enter AIS from a generalist background, or support others doing so:

What made the transition harder than it needed to be? Was any of it structural, something that could plausibly be designed around rather than just navigated?

Open to anyone:

Is there something important about how this field currently works, or how it is failing, that this post does not capture, especially if it complicates or challenges what we have described?

Please feel free to leave a comment, contact us directly through DM or generaists2026@gmail.com, or fill in our short form to leave your preferred mode of contact. Thank you for anything you’re willing to share!

Acknowledgements

We are grateful to the people who invested their time in exploring the problem together with us through interviews and written responses.

We are also grateful to our Sentient Futures mentor, Martyna Wielopolska, for consistently providing fresh perspective and gently nudging us out of perfectionism.

And we very much appreciate input by our reviewers, Filip Alimpic (Independent researcher and BlueDot Impact facilitator) and Thibaud Veron (Chief of Staff at CeSIA, Instructor at ML4Good and Organizer of the 2025 AIS Field Builder Forum) and thank them for in-depth feedback, push-backs and encouragement.

Appendix 1. Authors’ background

As we see our personal transition experience as the main motivation, a data point that preceded the interviews, and a possible source of bias, we have considered it relevant to share it.

Uladzislau Linnik:

My prior experience is mostly in org-building and operations — setting up processes and departments across strategy, finance, project management, sales, and other business functions, primarily in architecture and IT consultancy firms. I have been in full-time transition into AIS since July 2025, supported along the way by Consultants for Impact, HIP IAP, The Introductory EA Program, and ENAIS AIS Collab.

Having tested various career pathways toward AIS with Consultants for Impact and HIP IAP, I felt more comfortable deciding to build on my existing organisation-building skills rather than requalify^[3]. However, I recognised I lacked the context and career capital to be competitive.

Being far from AIS/EA hubs, I lacked an in-person network, my credentials (university, companies) were not known among the bigger EA community and my career capital was heavily skewed towards soft skills. I couldn’t directly transition my hard-ish skills: my experience in operations and finance in small businesses from the CIS countries, although successful, didn’t translate directly into non-profit operations in the EU/UK/US, nor did people management background suffice to become a programme or research manager. However, it was equally unlikely that the traits that allowed me to build and run operations across multiple businesses and countries could be completely undemanded in the young space of AIS, with proper upskilling. I checked the following types of opportunities:

For the volunteering opportunities I found^[4] I didn’t look like a good fit: either requiring skills I hadn’t yet developed, where my gaps would have produced more cost than value, or falling within categories that didn’t work with my sustainability constraints (e.g. “seeking owner” projects).

AI Introductory courses (BlueDot Impact AGI strategy, ENAIS AIS Collab) looked genuinely helpful to structure and complement independent learning — but what they offered still didn’t translate into practical experience directly^[5].

Fellowships looked like a fantastic opportunity to make a relatively cheap commitment test, gain mentored practical experience, do some impact and grow career capital. However, studying the ecosystem, I observed a serious skew toward research and tech talent. While they could afford building fellowship ladders^[6] through a multitude of programmes and projects, generalists had something around six programmes^[2]. These seemed and proved to be very competitive. A further asymmetry: while researchers’ work focused on specific projects linked to potential job areas, ‘generalist’ opportunities, while diverse in essence, seemed bundled — also making the fellowship-to-job path less direct. This pushed me to consider other paths: research pipelines or building surface area for serendipity^[7]— with the adjacent opportunity costs for me and the ecosystem.

I didn’t seem to be unique in my observations, as about ten initial 1:1s with HIP IAP and EA Connect participants uncovered similar sentiment, while I interpreted several publications as directly pointing to the bottleneck. Among the ones that made the biggest impression on me were this post that explicitly stated the problem, and this piece which I read as a proxy evidence of a specific skill-mix the pipeline was failing to serve.

This led to a solution hypothesis: a set of fellowships or a platform for granular generalist roles (e.g. “Operations for AIS governance”). Because I was unsure whether to stress-test this hypothesis or move straight to intervention, I joined Sentient Futures Project Incubator to get structured feedback. There, a mentor nudged me toward deeper exploration first, which reinforced the conviction that before building anything, we owe it to the field to first understand which problem we’re actually solving.

Enisa Ismaili:

My prior experience spans technology leadership (data and analytics consulting), LLM post-training, computer education (computer science lecturing and curriculum creation) and entrepreneurship. Through my transition into AIS I have been supported by HIP IAP and BlueDot Impact, and have independently pursued upskilling in AI evaluations.

The landscape. As I was exploring the AIS landscape, the available roles clustered into two groups: research-oriented positions, and purely operational ones. Neither fit my profile, which was built around the space between them, combining technology leadership with both technical grounding and strategic scope. AI labs presented roles that were a closer match, but my path toward them felt constrained by location, network distance, and being new to the field.

Finding an angle. Exploring further, I identified AI evaluations as a domain where my interests and career capital came together. Not as a research destination, but as a space where the management, operational, and strategic layer was both meaningful and underserved. Professional advice I sought confirmed this was a credible direction given my background.

The upskilling problem. The challenge that followed was structural: to be effective in that layer, I would need two types of upskilling in parallel. Technical context deep enough to understand the work, speak the language, and manage it credibly, and stronger operational and people management experience in this specific domain. In my view, what I needed was something closer to a technical and operational fellowship or structured bootcamp: a direct pipeline into the ecosystem, equivalent to what research fellowships provide for researchers. I didn’t find that pathway, and finding it became harder the more I looked.

The absence of a visible structured pathway became evidence of the gap itself. What followed was a self-directed learning plan spanning both dimensions. The pattern, however, did not feel unique to me. Reading the forums, talking to other generalists navigating the same entry problem, and longer conversations with Ulad all pointed in the same direction. The question of whether this gap is real, and what intervention could close it, felt worth investigating seriously. AI evaluations remains the domain where my interests and career capital converge, and GenerAISts grew directly from that conviction: if this pattern holds across the field, there is an opportunity to build something concrete and valuable, both for the ecosystem and for professionals like me still navigating it.

Appendix 2. Earlier assumptions and sources

Working assumptions on why the generalist talent bottleneck might exist, compiled before the interview phase. Assumptions supporting the bottleneck hypothesis only while counter-arguments are addressed in the Uncertainties section of the main post.

Method: structured table with sources cross-referenced against published evidence we used for internal analysis. Some sources are mentioned more than once if they support several assumptions. Notes were made for working purposes, as subjective interpretations.

#	Assumption	Description	Sources
1	Skillset mismatch: what roles need vs. what gets selected	Generalist talent pipelines are narrowed by researcher-optimised entry paths. Many people in operations and organisation-building roles end up working in their secondary skillset, or are screened using tests that don’t match what the actual role most needs.	AIS entry favours research profiles (source) Tests misaligned with roles (source)
2	Value alignment and attitudes are under/mis-assessed, especially early in the funnel	Hiring often focuses on easily testable capabilities (timed tests, technical exercises) and screens for attitude and value alignment late or weakly. This risks excluding candidates with strong mission fit but less conventional backgrounds.	Mission alignment hard to screen for (source) Skills over attitudes critique (source) Tilt toward easily testable traits(source)
3	The skill-mix for high-leverage generalist roles is unusually demanding and rare	High-leverage generalist roles (research managers, operations leads, recruiters) require a rare combination of project and operations skills, decision-making under uncertainty, and deep ecosystem knowledge. Organisations routinely struggle to find candidates with all of these.	Three-part profile rarely met (source) Rare tech + PM + people mix (source) Broad, multi-area skill range required (source) Mid-career talent lacks EA context (source) Operations & management seen as critical bottlenecks (source)
4	Generalist careers are lower-prestige, so few people intentionally specialise in them	Operations, support, and organisation-building work is often perceived as “overhead” or less impactful than front-line research. Strong people in these roles tend to move toward research or leadership; relatively few intentionally build long-term careers in them.	Recruiting/operations under-valued relative to impact (source) Amplifier roles rarely aspired to (source) Support roles neglected (source) Less glamorous steps undervalued (source) Non-research gap unaddressed (source)
5	Hiring processes for generalists remain noisy and suboptimal	Predicting job performance is hard and evidence is limited. Organisations rarely see counterfactuals on rejected candidates; many rely on unstructured interviews, stressful work trials, unclear expectations, and opaque feedback.	No counterfactuals on rejections (source) Interview practices might predict performance poorly (source) Opaque trials, little feedback (source) Low capacity, weak incentives (source) Cultural fit not communicated (source) Indirect transition costly (source) Opaque, network-heavy hiring (source)
6	Upskilling paths for generalists lack granularity	Existing opportunities often bundle multiple development jumps at once (theory, craft skills, and people management). There are relatively few structured, role-specific ladders that separate “learn the craft” from “take on leadership.”	Staged development and mentorship as model (source) No neat in-EA ladders indicated (source)
7	AIS/EA culture may favour a relatively narrow experiential profile	Cultural and communication norms can make EA/AIS feel inaccessible or off‑putting to otherwise suitable people, suggesting that current culture and signals may unintentionally favour certain backgrounds and communication styles and thus narrow the generalist pipeline	Demographic skew (source) Cultural accessibility issues (source) ND parallel spaces (source) “need more normies” (source)

Appendix 3. Internal generalist Taxonomy table.

Cluster name	Example skills / activities (short)	Example roles / titles (non-exhaustive)
Leadership & People	Strategy, prioritization, leadership, facilitation, coaching, culture.	Executive Director, Director, Chief of Staff, Strategy Lead, Team Lead, Org Development Lead, Executive Assistant / Exec Business Partner.
Operations & Systems	Finance, HR, logistics, tools/IT, processes, legal/compliance, admin.	Operations Generalist, Operations Manager, Office/Facilities Manager, Finance & Operations Lead, HR Generalist, Admin Coordinator, Grants/Finance Operations.
Product & Delivery/Research & Programme	Project/programme management, scoping, timelines, coordination, execution quality.	Research Operations / Research Manager; programme Manager, Project Manager, Product Manager, Research programme Manager, Fellowship/Grant programme Lead, Event/Retreat Lead.
Talent & Community	Hiring, talent scouting, onboarding, events, outreach, partnerships.	Recruiter, Talent Lead, People & Culture Manager, Community Builder, Group Organizer, Partnerships Manager, Donor/Stakeholder Relations.
Insight & Communication	Light research, analysis, docs, updates, external comms, knowledge sharing.	Policy/Research Coordinator, Communications Manager, Content Writer/Editor, Knowledge Manager, Grants Writer, Marketing.

^
Also supported by the EA Survey 2024 on the broader EA scale.
^
Resources explored: EA Connect and SPAR Demo Day Dec ’25 career fairs; AI Safety Map; 80,000 Hours Fellowships directory; HIP IAP resources: fellowships. The six programmes listed at the time of research:
- Constellation Astra field-building track
- GovAI Summer Fellowship 2026
- ML4Good Bootcamp governance & strategy tracks
- FutureKind by Electric Sheep
- Sentient Futures Project Incubator
- Effective Altruism Operations Bootcamp.
Uladzislau applied for 4, got accepted to 1 but already had a rather concrete field-building project proposal.
He does not include other incubators here: not every “generalist” is necessarily a founder and incubators can hardly be considered a “cheap” test due to the requested commitment to the project.
^
For example, Uladzislau’s analysis of job posts on four platforms (80,000 Hours Job Board; Probably Good Job Board; Consultants for Impact Newsletter; Effective Altruism Opportunities Board) for the period between 12 August and 17 September, 2025 surfaced 25 relevant-skill generalist positions in AIS, which meant about 4–5 months to accumulate the target sample of 100 positions. Accounting for the necessity to grow career capital, it was on the pessimistic side of the transition plan, while manageable.
^
One of the searches spanned 21 September – 21 October, 2025. Of the 14 opportunities identified that Uladzislau could potentially cover with counterfactual value, only 5 were solicited; the rest would have required unsolicited outreach.
Of those, only 3 were relevant to organisation-building — but all three sat within the “seeking owner” filter at aisafety.com, which describes projects without a committed owner rather than active placements. A few more community-building and communication requests were available, but Uladzislau didn’t feel well-suited for those at the time.
Sources searched: EA portal; 80,000 Hours; Probably Good; AI Safety volunteering; BlueDot Impact Community; EA Forum; EA Slack; 80,000 Hours JB Contests & Fellowships. Later complemented with HIP IAP volunteer matching research.
^
This is Uladzislau’s personal interpretation, later practically checked with completion of ENAIS AIS Collab and BlueDot Impact AGI Strategy curriculum skimming. The curricula are extremely valuable and serve the purpose of structuring, complementing the knowledge and surfacing context gaps very well.
^
Some data points suggest this effect is overestimated.
^
As explicitly mentioned in Sofia Balderson’s article (broader EA), but also described in other AIS-specific articles, like this by Jim Chapman, this by Moneer Moukaddem or this Building Capacity post.

[Research Request] “Prestige Tax”: Quantifying Epistemic Silence in AI Safety Teams by James-Hartree-Law

James-Hartree-Law — Fri, 17 Apr 2026 19:31:02 +0000

Summary

Empirical research in organizational psychology suggests that hierarchical rank is the strongest predictor of whether employees speak up or self-censor. In the high-stakes, high-prestige context of AI safety, this “silence” likely functions as a significant information bottleneck. I am conducting independent research to quantify this effect within the safety community.

If you are a junior researcher, intern, or have <5 years of experience at a safety org, please consider taking this 4 minute survey.

The Problem: Epistemic Bottlenecks

The expected value of an AI safety organization is a function of its internal information flow. However, truth-seeking is often at odds with social signaling. In labs with steep prestige hierarchies—where a junior researcher may be sitting across from someone whose seminal papers they studied in University—the social cost of “looking naive” or “being wrong” can lead to significant signal loss.

The literature suggests this isn’t just a minor friction; it’s a systemic failure mode:

The 85% Gap: A foundational study (Milliken, Morrison & Hewlin, 2003) found that 85% of employees withheld important information from supervisors due to fear of social consequences.
The Decay of Candor: Longitudinal data (Edmondson et al., 2024) indicates that new hires enter with high willingness to speak up but “learn” silence within their first 12 months as they calibrate to the organizational hierarchy.
Non-Routine Work: The costs of this silence are highest in non-routine research environments—exactly the conditions of alignment and evaluation work.

Why this matters for AI Safety

In this field, the counterfactual impact of a missed clarifying question, technical criticism or a “crazy” but correct alignment theory is disproportionately high. If junior staff feel they must “filter” their thoughts to match the Overton window of their leads, the organization loses the “long tail” of creative insights.

Managers often talk about “speak-up culture,” but the empirical evidence suggests that culture-messaging (e.g., “our doors are always open”) is largely ineffective at overriding the structural reality of prestige.

The Research Goal

I am currently investigating whether these findings from broader organizational psychology hold true within the AI safety community. Specifically, I want to identify:

What is being withheld? (e.g., technical disagreements, “basic” questions, or organizational critiques).
Why is it being withheld? (e.g., reputational risk, fear of wasting senior time, or lack of confidence).
The Magnitude: Is this a marginal issue or a primary bottleneck for lab productivity?

Call to Action: 4-Minute Survey

To ground this research in actual data rather than theory, I need input from the front lines.

If you currently work at an AI safety/eval organization (e.g., METR, ARC, Apollo, Redwood, CAIS, etc.) in a junior or mid-level capacity, please fill out this survey:

[4 Minute Survey]

Privacy: Responses are completely anonymous. No individual data will be shared with your organization.
Output: I intend to share the aggregated, anonymized findings back with the Forum to help labs optimize their internal communication protocols.

The Institute for Classical Dialogue (ICD) is an independent organization exploring protocol-based interventions to improve epistemic quality in high-stakes research teams. Feel free to reach out in the comments or via PM.

The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness by Vasco Grilo🔸

Vasco Grilo🔸 — Fri, 17 Apr 2026 16:38:42 +0000

This is a linkpost for The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness by Alexander Lerchner, which was originally published on PhilPapers on 9 March 2026. The abstract is below. Here is Alexander’s reply to “two specific critiques [which] have surfaced repeatedly”, including in a post by Shelly Albaum.

Abstract

Computational functionalism dominates current debates on AI consciousness. This is the hypothesis that subjective experience emerges entirely from abstract causal topology, regardless of the underlying physical substrate. We argue this view fundamentally mischaracterizes how physics relates to information. We call this mistake the Abstraction Fallacy. Tracing the causal origins of abstraction reveals that symbolic computation is not an intrinsic physical process. Instead, it is a mapmaker-dependent description. It requires an active, experiencing cognitive agent to alphabetize continuous physics into a finite set of meaningful states. Consequently, we do not need a complete, finalized theory of consciousness to assess AI sentience—a demand that simply pushes the question beyond near-term resolution and deepens the AI welfare trap. What we actually need is a rigorous ontology of computation. The framework proposed here explicitly separates simulation (behavioral mimicry driven by vehicle causality) from instantiation (intrinsic physical constitution driven by content causality). Establishing this ontological boundary shows why algorithmic symbol manipulation is structurally incapable of instantiating experience. Crucially, this argument does not rely on biological exclusivity. If an artificial system were ever conscious, it would be because of its specific physical constitution, never its syntactic architecture. Ultimately, this framework offers a physically grounded refutation of computational functionalism to resolve the current uncertainty surrounding AI consciousness.

Same Model, Different Oversight: What Deployment Context Reveals About AI Task Success by nikolasjanke

nikolasjanke — Fri, 17 Apr 2026 16:32:08 +0000

Epistemic status: exploratory. The numbers are straight from a public dataset; the correlations are just what came out. The oversight angle is my best guess, but I’m not married to it. This is only a week of collected data, with model-generated labels, so don’t take any of this as gospel.

Anthropic’s first-party API and Claude.ai both use the same model family. The API is 25 times faster and cheaper per token, but only gets things right about half the time. Claude.ai is slower, pricier, but nails it closer to 70%.

Both chew through millions of real-world tasks, both running the same model. That 20-point gap isn’t about raw capability.

I dug into the public Anthropic Economic Index ^[1] dataset ^[2] to figure out what’s behind this. It’s about 2 million anonymized conversations from one week in February 2026, each mapped to O*NET occupational tasks. Both platforms are in there. If you want to poke at the code or notebooks, they’re up on GitHub ^[3].

The natural experiment

There are 1,416 O*NET tasks that show up in both datasets. For 258 of them, I’ve got the full breakdown of collaboration modes. That means I can do a head-to-head: same task, same model, two different setups.

Figure 1. Per-task success rates for 258 ONET tasks are present in both Claude.ai and 1P API. Each point is one task. Diagonal = equal performance; points below the line = API underperforms. Color = API directive share on that task.*

Claude.ai comes out ahead on 85% of the shared tasks. On average, it’s about 16 points better. The biggest gaps—those deep red dots below the diagonal—are mostly tasks where the API is used in a highly directive way.

The structural difference

Anthropic sorts every conversation into five collaboration modes: directive, feedback loop, task iteration, validation, and learning. The two platforms end up with totally different patterns.

Figure 2. Interaction structure by platform. Augmentation modes (learning, task iteration, validation) run left; automation modes (directive, feedback loop) run right. Global averages, all conversations.

On the API, 58% of interactions are directive: the user hands off a task and just takes whatever comes back. On Claude.ai, that’s only 33%. Learning mode flips the script: 22% on Claude.ai, just 4% on the API.

This isn’t just about which tasks land on which platform. Even if I only look at the 258 shared tasks, the pattern sticks: API directive averages 56%, Claude.ai 29%. API learning is 14%, Claude.ai 34%.

Directive as the predictor

How the task is handled—its mode—predicts the success gap between platforms.

Figure 3. API directive share vs. per-task success gap (Claude.ai − API). Shaded band = 95% CI. As directive use rises on the API, that task’s performance gap widens.

When the API mostly runs in directive mode, success averages about 58%. If directive is rare, success jumps over 80%. The correlation is negative and real: r = −0.263, p < 0.001, N = 258.

I checked if AI autonomy, learning share, or task-iteration share explained more of the gap. Directive share on the API is the strongest single predictor. Claude.ai’s learning share also lines up with the gap (r = 0.158, p = 0.011): more learning mode on Claude.ai, bigger advantage over the API. Still, directive is the main thing.

The speed-oversight tradeoff

Figure 4. Three metrics evaluated across platforms. Claude.ai teal; 1P API coral. Speed = multiplier vs human-only baseline (human_only_time in hours / human_with_ai_time in minutes).

Claude.ai chops task time down by about 13 times. The API is even faster—almost 25 times quicker. So the API is twice as fast as Claude.ai on the same tasks, but it also misses the mark by 16 percentage points more. More speed, more failures, less human in the loop.

So what does all this actually mean?

Usually, people talk about human monitoring in AI as a question of what humans should be able to monitor. But this data points at something else: what actually happens to oversight when speed is what matters most?

Production API setups are built for throughput. That means fewer turns per task, less back-and-forth, and less human decision in the loop. The pattern that gets you the most throughput—directive mode—is also the one that leads to the worst results. This isn’t an accident. You finish faster because you skip the human check-ins. You fail more for exactly the same reason.

This lines up with what people have argued about scalable oversight [see e.g., Christiano et al.]: keeping humans meaningfully involved gets harder as AI scales, mostly because the economics push the other way. Here, you can see that dynamic happening right now, not simply as a future worry. The API isn’t being misused. It’s working exactly as intended—and that intention trades off success and oversight for speed.

This doesn’t settle any big safety arguments. But I haven’t seen anyone actually put numbers on the tradeoff between speed, success, and human-in-the-loop rates from real-world usage like this before.

The cruxes

For the main claim to stick—that taking humans out of the loop leads to worse results—a couple things have to be true. Here are the big ones:

Crux 1: The “success” label is meaningful. Task success is Anthropic’s model-generated classification, not an independent outcome measure. If success labels are systematically more likely to be assigned to learning-mode interactions for reasons unrelated to actual outcomes (e.g., the model rates conversations where the human is engaged as “successful” regardless of whether the task was done well), the correlation is artifactual. What would change my mind: evidence that task_success is uncorrelated with independent quality measures, or that it’s mode-contingent in the labeling process.

Crux 2: The mode distribution isn’t downstream of task difficulty. If directive-heavy API tasks are also inherently harder (ambiguous, underspecified), then both the directive interaction and the low success rate are caused by task difficulty rather than by each other. The fact that this pattern holds for shared tasks — the same task, on both platforms — reduces but doesn’t eliminate this concern. What would change my mind: a within-task design where the same user attempts the same task via both platforms, with success measured independently.

Limitations

This is just one week of data. Both the task labels and the collaboration modes come from the model, so there’s error on both sides. Random noise would make the correlations look weaker than they really are; systematic mislabeling could create patterns I can’t see.

The 258 shared tasks with full mode data aren’t a random slice of the 1,416 shared tasks—they’re just the ones where Anthropic’s pipeline spit out complete records. That selection could skew the comparison.

The March 2026 report [^2] found that users with six months or more do better and move toward augmentation, but that variable isn’t in the public data. The deployment-context and tenure findings go together, but I can’t connect them directly.

^
Anthropic Economic Index public dataset (HuggingFace, CC-BY-4.0): https://huggingface.co/datasets/Anthropic/EconomicIndex/tree/main
^
Anthropic March 2026 Economic Index report: https://www.anthropic.com/research/economic-index-march-2026-report
^
Analysis code and notebooks: https://github.com/nikolascoimbra/anthropic-economic-index-analysis

Trajan House—current goals and updates by Asher Benter 🔸

Asher Benter 🔸 — Fri, 17 Apr 2026 15:36:11 +0000

Trajan House is an office in Oxford for people working on humanity’s most pressing challenges. There have been some changes to the project over the last few years, and it seemed time to give a public update about what we’re getting up to!

Here are some quick things to know:

The anchor tenants in the space are Forethought and CEA. CEA manages the space.
I’m the project’s manager
Over the next year, we are looking to add more:
- Organisations that can utilise the space indefinitely or on a short-term basis. We have space for 3-5 more organisations.
- Individuals who are looking to join an active, impact-focused community.
- Events such as fellowships or bootcamps.

If you are curious about any of the above, read on! Below, you’ll learn more about the space, our theory of change, and how to get involved.

Here are some ways to engage if you’re already interested:

Apply to become a member, or
Apply to visit the space in June (after EAG London) for a few days or a even a few weeks. Throughout the month we plan to have many more people, visiting organisations, speakers, and social events. You can learn more and submit your interest to join us in June here.

Getting up to speed

Trajan House is an office and event space in central Oxford. It was initially operated by Effective Ventures starting in 2021, and has been run by the Centre for Effective Altruism since August 2025. The office previously hosted FHI, GPI, Our World in Data, and GovAI.

Today, we are the main hub for Forethought and CEA, and we provide space for individuals working for a wide range of aligned projects and organisations, including OAISI, Macroscopic, Coefficient Giving, 80k, Longview, and FLI.

Purpose and place

Our intention is to bring together groups and individuals who are focused on earnestly working to make the world a better place. We aim to form a thriving community that better prepares humanity to face its current and future challenges.

There is already a strong track record of this community in Oxford producing research that has an outsized effect on those seeking to do significant good. The concept of existential risk, longtermism as a framework, and AI governance as a field all emerged from Oxford-based research that was speculative and uncertain at the time. The conditions that enabled that work (i.e., academic density, intellectual freedom, and focus over pace) remain.

The space was designed as a place where researchers, practitioners, and movement builders can continue to do their best work with better focus and reduced hassle. This includes those working in established cause areas as well as those seeking to push the intellectual boundaries of effective altruism or work on exploratory topics such as post-AGI flourishing.

Beside all of this, Oxford is also a beautiful place to live and work. The city is walkable and bikeable, it contains a wide variety of communities to be a part of, and it has good transport links to London when needed.

What is the space like?

The office primarily consists of private offices and assigned desks in rooms of 3-7 people, with other areas that are more open and communal. Hot-desks and assignable desks are available for visitors and independent researchers. The large cafeteria provides daily lunch to those in the building and serves as an important point of impromptu connection across projects.

Our goal is for users to spend their time on their object-level work rather than logistics – the facilities are managed, we work with organisations on their specific needs, and users are provided with quality equipment.

Ways to engage

If you’re in the UK or just visiting, we would be glad to hear from you. And if you’re intrigued by the idea of working from Trajan House but would want to know more, please get in touch – we have trialling options that are available.

Ways to get involved:

Resident organisations: Private offices for groups (3-20+ people) who want to be based in Oxford long-term. This is our primary focus, and we have space to add 3-5 more organisations over the next year. If you’re curious or interested, please contact us here.
Visiting organisations: Short-term space (days to weeks) for remote-first teams who want to gather for workshops, planning sessions, or team retreats. We can accommodate groups of 3-30 and will handle logistics like catering and A/V setup. If you’re curious or interested, please contact us here.
Individuals: If you live nearby or are visiting, we offer dedicated desks and hot-desking options. This can be great for independent researchers or those affiliated with organisations elsewhere. If you’re looking to become a member or visitor, fill out this form here.
Events: We have meeting rooms and event spaces (10-50 people) for workshops, fellowships, and bootcamps. If you’re an event organiser and would like to know more, you can fill out this form here.
Join us in June: After EAG London we will be gathering people in Trajan House throughout the month of June. During this time we plan to have many more visitors than usual, and have an increased frequency of speakers and social events. You can learn more and submit your interest to join us here.

If you would like to contact us about anything else, you can email us at info[at]trajan.house.

See you ’round!

Humane Pesticides Are Massively Morally Urgent by Bentham’s Bulldog

Bentham's Bulldog — Fri, 17 Apr 2026 15:28:35 +0000

Crosspost.

I have lots of radical views about insects. I think probably most expected happiness and misery in the world is experienced by insects, and that our actions often have much more significant moral impacts on bugs than on people. But in this article, I’m not going to defend anything radical. My thesis here is very moderate: I think we should try to make pesticides kill insects less painfully.

Every year, quadrillions of insects are killed or harmed with pesticides—3.5 quadrillion, according to one estimate. This is about 30,000 times more than the number of people who have ever lived in human history. In the best cases, deaths take minutes or hours as the insects are slowly poisoned to death. In other worse cases, death takes days as the insects are left unable to shed their own skin and essentially crushed to death from the inside. In still other cases, insects starve to death as their guts malfunction. This is likely to be very painful.

It is plausible that many insects can suffer. When benchmarks are created to assess pain in animals, insects consistently meet most of the criteria, having nociceptors, responding to painkillers, demonstrating associative learning, and sometimes displaying surprising cognitive complexity. Of the criteria they don’t clearly meet, this is because of lack of research, not contrary evidence.

When flies are genetically engineered to have capsaicin receptors (responsible for tasting spicy food) they display strong negative behavioral reactions to food laced with capsaicin. They take small nibbles at the food, before violently quaking as if in distress. They find it so aversive that they even starve themselves to death rather than eat it. However, this stops if you give them painkillers. This isn’t definitive proof of consciousness, but it’s at least suggestive!

Many different insects learn to avoid stimuli that they associate with pain. When flies are given electric shocks paired with some odor, they learn to avoid the odor. The same is true of bees, ants, and various others. Insects behave, when exposed to heat, exactly as one would expect of an animal in pain. While it is possible that they feel only minimal suffering, on account of their cognitive simplicity, it is also possible that they feel a lot of pain.^[1]

The best report on animal sentience estimated insects that experienced pain within an order of magnitude or two as intensely as people. It at least doesn’t seem obvious to me that when insects are writhing around in agony, behaving like you or I would if we were in intense pain, they’re experiencing something on the level of a stubbed toe. I’m at least not convinced that it’s overwhelmingly likely that they’re vastly less conscious, in some qualitative sense, so that they see colors, hear sounds, and feel pain with overwhelmingly less vividness—as if the brightness and volume on their inner life has been turned down almost to zero. If a being might suffer a lot, then their being painfully killed by the quadrillions is a big deal.

If we conservatively estimate that insects feel pain on average ¹⁄₁,000th as intensely as people do, then pesticides annually cause about as much misery as 3 trillion painful human deaths. That is an amount of pain equivalent to if every human on the planet experienced intense pain, on the level of being poisoned to death, every single day. My guess is that this is more pain than is caused by the entire invertebrate farming industry (including even those ghastly shrimp farms).

That is very bad. It isn’t very clear whether pesticides on net increase or decrease insect suffering, because pesticides might lower insect populations. So it’s not clear if the world would be better if pesticides were phased out. But in any case, they are the cause of a very large amount of direct pain. We should aim to make them less painful.

I boldly propose: if we’re killing lots of beings that might be conscious, we should try to do it less painfully. If we’re causing intense pain to a population, every single year, that is 30,000 times more than the number of people who have ever lived, so that even by conservative estimates more expected pain is caused by this every year than by all human suffering in history, we should try to cause less intense pain. Radical, I know!

This isn’t some weird utilitarian view. It just seems like common sense—that we should try to hurt others less, if possible. We shouldn’t boil lobsters alive. If we can avoid it, we shouldn’t slowly poison insects to death in the maximally painful way. Insects are smaller than lobsters, but they’re fairly cognitively similar.

I think we’re biased against insects because they’re small. There used to be giant prehistoric insects that looked like this:

Imagine meeting this guy in a dark alley.

If you poisoned those giant insects to death, and then a trillion more, and a trillion more, and did that a thousand times, so that in total you poisoned to death 1 quadrillion of them, that would seem like a big deal. Our intuitions count insects far less than bigger animals. But this is an error. It treats size as morally relevant when it obviously isn’t.

So humane pesticides seem ridiculously important.

But despite this, and despite Brian Tomasik having been banging the drum about this a decade ago, nobody has funded it. There’s just been no research into how to make pesticides more humane. I want to emphasize: if you got humane pesticides implemented widely, you would affect quadrillions of animals every year! You might affect a distribution of pain, in a few years, more than all human misery in history.

So someone should get on this! Once we have humane pesticides, efforts can be carried out to increase their uptake. If you’re interested in funding humane pesticides, or interested in being funded to work on humane pesticides, feel free to shoot me a Substack DM or send me an email at untrappedzoid@gmail.com. By sponsoring this, you’d be able to affect staggering amounts of suffering. If you work on or sponsor humane pesticides, you could maybe do more for insects than any person ever. And while the insects are unlikely to thank you (because they don’t talk) you can prevent quadrillions of them from experiencing immense agony. A thousand trillions worth of insects won’t have to die slowly, in horrible agony, alone, for days, in the dark.

^
There are some reasons to think simple animals might suffer a lot—both because they need more pain to learn and when they suffer, pain wholly envelops their mind, without other sophisticated cognitive processes going on.

Come to Manifest 2026! (June 12-14) by Winter

Winter — Fri, 17 Apr 2026 14:06:12 +0000

Manifest is back! We’re returning to Lighthaven on June 12-14! You can buy your tickets now on the Manifest website, with Early Bird pricing through this Saturday, April 18:

Buy Tickets

What: Manifest is a festival ostensibly about prediction markets, but secretly about connecting with old friends and people you admire from your favorite niche corners of the internet. It is a gathering of nerds who want to find the thinkers and practitioners they vehemently agree/disagree with, share a meal around a cozy campfire, and come away with radically new ways of thinking. If you attend, don’t be surprised to find yourself engrossed in a late-night rabbit hole with your future cofounder, a serendipitous nerd snipe from an internet celebrity, and the joy of a crowd cheering (and betting with mana) on a wrestling match between strangers.

Who: 500-650 exceptional people, from many fields, including but not limited to: forecasting, rationality, effective altruism, AI & AI safety, economics, finance, tech, policy, writing, and novel ideas.

When: June 12-14 (Fri-Sun)

Where: Lighthaven, Berkeley, CA (map)

How:

🎟️ Tickets (Early Bird until Sat 4/18!)
🏡 Room bookings

To get updates: Join our Discord

And this year, we want to make it even better, focusing on the things attendees have told us they value most: high session quality and variety, well-curated attendees, thoughtful sponsor presence, delightfully engaging activities, easy-to-use scheduling, clear community guidelines, and structured ways to meet and stay in touch with the people you’re supposed to meet.

Email team@manifest.is if you have questions, ideas, or feedback. We’d love to hear suggestions for more excellent speakers or organizations to collaborate with. Help us fill Manifest with the most stellar, thoughtful, kind humans you know!

You’ll hear updates from us as we confirm more of the speakers, sponsors, and plans we’re cooking up. We look forward to seeing you in two months!

Highlights from 2025

2025 Talks included:

A Story Telling of Kalshi’s Internal Drama & Decision Making (Noah Sternig)
Reversible Cryopreservation (Laura Deming)
Writing and Slop (Roon, Scott Alexander, Gwern, Alexander Wales)
Life as a Professional Gambler (Anonymous)
Has Trading and Gambling Gone Too Far in the US (Jeremiah Johnson, Isaac Rose-Berman, Christopher Gerlacher)
How Tech Hiring is Changing with AI (Sholto Douglas [Anthropic], Ben Cohen [Substack])
Schools Should Pursue Excellence (Tracing Woodgrains)
How to be Hot (Aella, Chesed)

Check out our YouTube channel for more examples from past years!

2025 Events included:

Nash Pit: A Prediction Market Game Show
Night Market + Career Fair
Live Polling on Spicy Questions
Speed Friending
Rare & Exotic Drama Games from Australia
Hot Seat w/ Niacin (vitamin B3, which causes a rapid flushing response)
Startup Pitch Competition
Poker tournament with former pros

Speakers & Events

Every year, we’re grateful for the amazing speakers and guests who come out to Manifest!

Stay tuned as we announce the speaker lineup for this year. And if you want to give a talk, or have an idea for an event that needs our input/support, email winter@manifest.is.

Also: anyone is welcome to host a community talk or event! Closer to the event, we’ll be releasing a schedule with rooms and time slots that you can add your session to.

Career Fair

On Friday June 12, the first night of Manifest, Lighthaven will be open and free to the public for our Career Fair + Night Market! This will be an excellent opportunity for even more people to mingle, talk to hiring sponsors, and join the fun for a few hours.

We ask that everyone who is interested (whether or not you’re attending the rest of Manifest), please register your interest in the Career Fair here.

Festimonials

Don’t just take our word for it; here’s what our past attendees have to say (emphasis ours):

Jake Seliger:

Bess and I went to Manifest, which bills itself as “A festival for forecasting and prediction markets,” a description that may technically be true but fails to capture the spirit; to my eye and experience, it’s maybe more accurately stated as “Substack and Twitter live” or “a mixture of festival-conference-party-Burning-Man for nerds with many interests to show up and enjoy each other’s company.” Bess excitedly exclaimed it to be “Nerd Camp!” with a sort of takes-one-to-know-one gleam in her eye.

Tomie:

I love Manifest. I paid the full price for the full ticket, sucker I am, and my subsidy provided for these swaying bauble lights, these warm soporific nooks, these flames and corridors, these souls brought to Earth together, eyes lighting up at their electric worlds made real.

TracingWoodgrains:

For much of my life, I have poured my attention into tough-to-explain solitary pursuits, finding myself often sitting in quiet corners on the fringes of gatherings wondering if they’re worth the effort. Not so last weekend.

“I really like that Manifest is inspiring in me the ‘Oh, I should make and bring random goofy stickers and swag’ impulse that I’ve had inculcated in me by Defcon”—Dave Kasten

“Maybe the best conference I ever went to, beating all programming conferences, EAG, singularity summit”—MQP

Buy Tickets

Venue

Just like last year, Manifest will be held at Lighthaven. We credit much of the great vibes of Manifest to the space — it’s beautiful and cozy. It has fires for literal fireside chats, infinite nooks for one on one conversations, spaces for small workshops & big talks, and gorgeous overnight accommodations.

Festival Season! LessOnline —> Summer Camp —> Manifest

Manifest is actually the capstone to a 10-day season at Lighthaven! The weekend before will be LessOnline (June 5 - June 7): “A festival of writers who are wrong on the internet”. It is an unconference with a collaborative schedule, bringing together a “mostly-online subculture of people trying to work together to figure out how to distinguish truth from falsehood using insights from probability theory, cognitive science, and AI.”

In the days between the two conferences, there will be Summer Camp (June 8 - June 11), a peaceful yet vibrant week of coworking, collaborating, and late-night campfire chats. You might see: Hackathons (or “Forecastathons”), organized discussions, jam sessions, dance parties, tournaments, games of all kinds, camp activities (sardines, s’mores), or multi-day intensive workshops (e.g. CFAR-style).

You can come to any combination of these, but we’re offering a $350 discount off standard pricing if you buy All-Access tickets to all three events, available here.

See you at Manifest!

Make your accomplishments publicly legible by jgdouglass 🔶

jgdouglass 🔶 — Fri, 17 Apr 2026 14:00:51 +0000

TL;DR People pursuing impact generally do cool-sounding stuff. They should make said stuff easily discoverable on social media so others can find, admire, and get inspired to do similar work.

I’ve noticed a strange trend whilst browsing LinkedIn: the profiles of those pursuing effective careers are, on average, less verbose and more understated than those of similar status in adjacent (and generally less impactful) industries. I think there’s two reasons at play…

It’s hard to justify time and money on PR

If you’re chasing impact, you’re probably very selective with where you spend your time and money in a work context. The world has a million problems to solve and even making a dent takes years of focused effort and devotion. You’re likely to be cognizant of the utils your frivolously-spent dollars could save if instead sent to the Shrimp Welfare Project. You’re likely to feel like you have to continuously optimize where your focus goes. Given that, how could you justify wasting your precious work time on something so seemingly self-aggrandizing? It feels selfish. You’re ok with others learning about your work through word of mouth. That’ll be enough.

Why contribute to the cesspool that is LinkedIn?

Cynical corpospeak is LinkedIn’s lingua franca. It’s already grating if you’re in a position where you have to capitulate and post slop or lose potential exposure and career capital. It’s downright repelling to people that derive a strong sense of meaning from their work. People chasing impact are more inclined to view LinkedIn as a place where others disingenuously parade meaningless milestones around to further their impactless careers. They also rarely have to engage in such behavior to land impactful work; EA-adjacent hiring managers select more for virtuous character traits instead than a specific host of ATS keywords. If you’ve escaped such a dystopian rat race, why would you go back and redirect people from the starting line? You’re better than them, after all.

This sentiment, though hyperbolic, underscores what I believe to be a reactionary feeling diffused across the EA community. It’s refusing to engage with a useful system because it’s aesthetically unappealing or superficially similar to something you resent. The reflex to distance yourself from LinkedIn’s cringe culture by abstaining is worth examining. In fact, I think it’s potentially quite costly.

An understated digital presence on LinkedIn, Substack, and Twitter keeps your work invisible to people outside your circles. The people who most need to see that pandemic prevention is something urgent and tractable aren’t at EAGs or learning about your work through anecdotes at a Rationalist Bay Area party. They’re on LinkedIn, watching a Cluely ad, slowly internalizing that pumping out AI slop with a startup wrapper is the only way they’ll overcome Maslow’s Hierarchy of Needs. We can show them otherwise. We can show them their technical abilities and naive ambition can actualize truly heroic aims. I’m profoundly lucky that happened to me, with no thanks to those who preserved their contrarian minimalism over online legibility.

Personal visibility isn’t vanity, it’s a vehicle for diffusing important ideas! EAs forget that an exceptionally impactful career is incredibly untraditional and worthy of renown in the mainstream circles. When it’s clear that your work betters the lives of thousands, you become a memorable case study someone could latch onto when scrolling LinkedIn. Your profile could be the start of their rabbit hole into discourse around important problems, what people are doing about them, and how they can help!

Final recommendations

So, update your LinkedIn! Write a verbose summary of what you’re interested in and how it connects to important problems. Write about how your career transition 10x’d your personal fulfillment. Talk about how you work with incredibly interesting concepts, technologies, and people while profoundly improving lives! Impactful careers are amazing but so hard to find, let’s fix that!

Modeling AI competition dynamics for better governance by Jonas Emanuel Müller

Jonas Emanuel Müller — Fri, 17 Apr 2026 09:07:38 +0000

TLDR:

The Survival and Flourishing Fund has committed a matched grant. Donations are matched 1:1 up to $26,000. We need to raise at least $10,000 to unlock any of it. The full match funds the team for seven months.

If you’ve been looking for a way to support quantitative AI governance work specifically, this is a good moment. Donate here, and feel free to reach out if you’re on the fence!

What we do

Modeling Cooperation builds quantitative research tools and software to help AI governance researchers and decision-makers understand the strategic dynamics of AI competition… specifically, the collective action problems that make cutting corners on safety individually rational even when it’s collectively catastrophic.

Our main project is the software platform behind the Intelligence Rising workshops, run by Dr. Shahar Avin (CSER, Cambridge & UK AISI) and his team at Technology Strategy Roleplay. These workshops use a structured simulation game to help senior figures in government, industry and academia develop an intuitive grasp of how AI competition unfolds. Our software makes those sessions easier to run and possible to scale.

We also collaborate with Professor Robert Trager (co-director of AIGI, Oxford) on tools including the Safety-Performance Tradeoff web app, which demonstrates formally how technical safety progress is not always sufficient to reduce risk absent effective governance.

Our research includes work on the Windfall Clause and, forthcoming, an analysis of how early warnings about AI risks can be made credible.

Why this work is different

Most AI governance work is qualitative. Most technical AI safety work focuses on alignment or interpretability. We’re doing something unique: building scalable software to help decision-makers experience the competition dynamics that compromise the safety of frontier AI systems.

We want to be honest about what we don’t know. While some research has shown that participants trained with simulated interaction were more accurate than game theorists in forecasting decisions in conflicts (see Green and Armstrong), it’s genuinely hard to measure whether simulation workshops durably change decision-making. What we can say is that we’re working to solve the underlying problem: decision-makers lacking a felt sense of the competitive pressures that compromise safety.

Please do support us if you can. We are a small team having an outsized impact with a somewhat unique contribution to the field. All of our models are open so that other governance researchers can build on them.

See our live campaign here.

In Development Magazine Launch by Lauren Gilbert

Lauren Gilbert — Fri, 17 Apr 2026 03:18:35 +0000

We’re announcing a new development magazine: In Development.

Our goal is to promote discourse on development without dogma. In other words, we public stories doing deep dives into pressing development issues, backed up by evidence, argument, and experience.

The first piece is by Paul Niehaus, an Economics Professor at the University of California, San Diego, and co-founder of GiveDirectly.

He talks about the complementary role of evidence in the evolution of GiveDirectly, which has recently transferred its billionth dollar. Why did they need evidence? What kind of evidence did they need to convince themselves and others? What were the surprises? Whose preferences does such evidence reveal or promote? What role does ‘power’ play in the generation of such evidence?

We have one article out each week. The next one will have a line of argument that, mildly put, may not agree with today’s piece.

Reconciling Cost-Effectiveness and Cheap Enough by Tony Senanayake

Tony Senanayake — Fri, 17 Apr 2026 02:56:56 +0000

Two frameworks, one question: whose finances are we talking about, and to what end?

As the development sector gathers at the Skoll World Forum in Oxford this week, against a backdrop of sharp cuts to official development assistance — the US has ended 83% of USAID’s programmes, the UK has cut its aid budget by 40%, and the OECD projects an overall ODA decline of 9–17% in 2025 alone, with projected consequences for millions of lives — two financial frameworks are converging in the sector’s conversations.

The first is cost-effectiveness: IDinsight has launched a dedicated Cost-Effectiveness Unit and the FCDO’s Best Buys evidence summaries are influencing billions in development spending. The second is the “cheap enough” framework developed by the Mulago Foundation, asking whether solutions are structured to scale through government financing rather than perpetual philanthropy. Both feel urgent right now. And together, they open a broader opportunity: to think more holistically about financial frameworks, not just how to maximise impact per dollar, but what among that which is highly cost-effective is also truly scalable and implementable.

These two frameworks are not in tension. They are sequential, and the sector needs both. But to use them well, it is worth being clear about what each one is actually for.

Tl;dr:

The most useful framing is to treat cheap enough as a subset of cost-effectiveness: use cost-effectiveness to identify what is worth doing, then use cheap enough to test whether there is a viable path to sustainable financing at scale.
Cost-effectiveness and cheap enough answer different questions at different points in an intervention’s lifecycle. Conflating them or treating them as a Venn diagram with loose overlap, limits the value of both.
With ODA shrinking, the opportunity cost of mis-sequencing these frameworks has never been higher.

What this piece proposes:

Greater investment in willingness-to-pay infrastructure for governments and consumers, drawing on well-developed private sector WTP methodology
Greater standardisation of cost-effectiveness metrics across studies and sectors, to improve comparability and identify true optimums
Practitioner skill-building that integrates both frameworks, so organisations can design for impact and financial viability from the outset — because impact comes through implementation, not good intent

Cost-Effectiveness: The Optimisation Tool

Cost-effectiveness is a ratio: total cost divided by total impact. GiveWell’s methodology and the broader Effective Altruism movement have done the most to operationalise this in global development. It enables comparison across unlike interventions on a common scale — typically cost per disability-adjusted life year (DALY) averted in health, or cost per learning-adjusted year of education (LAYE) in education. Constructing the ratio requires explicit choices on both sides.

On the cost side, clarity is needed on whether we are measuring full organisational costs or direct delivery costs only, whether we are using pilot-phase figures or modelled at-scale costs (which can differ substantially), and whose costs are being counted: implementer, government, household, or some combination. On the impact side, the choices are equally consequential: are we drawing on RCT-derived outcomes or real-world delivery at scale; counting only direct first-order outcomes or compounding indirect effects; and which common metric—DALYs in health, LAYEs in education—makes comparison across interventions possible.

Strengths:

Enables comparison across like interventions on a shared unit of account
Embeds impact directly into the financial metric
The strongest tool available for optimising the next marginal dollar of investment and managing opportunity cost
Applicable not just to philanthropic capital but, in principle, to government and consumer spending too

Limitations:

Vulnerable to measurement bias: RCT-amenable interventions are systematically favoured over those with diffuse or structural impacts
Does not tell practitioners what cost to be targeting in the design of interventions
Says nothing about whether anyone outside the philanthropic world is willing to pay at the required cost level
A programme can be highly cost-effective and remain permanently donor-dependent

Cheap Enough: The Scaling Reality Check

Where cost-effectiveness is the question a funder asks before writing a cheque, “cheap enough” is the question a practitioner should ask at the drawing board. For some audiences, another way to frame this is simply willingness to pay, and specifically the willingness to pay of the payer at scale. As articulated by the Mulago Foundation’s Kevin Starr, the question is not “is this cost-effective relative to alternatives?” but “is the unit cost within what the system that will ultimately sustain this can realistically absorb?”

The three payers at scale:

Governments: the dominant payer for most interventions targeting the poorest. Willingness to pay is shaped by fiscal space, competing priorities, and political salience.
Consumers: the relevant payer in market-based models. Cheap enough is determined by household income and competition for discretionary spend.
Philanthropy: payer at scale only in narrow cases where the problem is global and the philanthropic market is deep enough to sustain delivery indefinitely.

Strengths:

Creates a decision-relevant design target that practitioners, implementers, and early-stage funders can align around
Turns cost from a programme output into a non-negotiable design input
Forces honesty about the pilot-to-scale transition early, when redesign is still possible
Anchors analysis in political and fiscal realities, not just technical optimums

Limitations:

Willingness to pay is genuinely hard to measure, particularly for public goods and social interventions
The threshold is partly endogenous: a well-resourced organisation with strong advocacy can shift government willingness to pay upward
Says nothing about impact: cheapness without effectiveness is not a virtue
Can systematically undervalue high-impact interventions that are structurally expensive

Five Ways to Think About the Relationship

How should the two frameworks relate to each other? There are five ways to think about this, ranging from most to least useful.

1. Cheap Enough ⊂ Cost-Effective — Recommended Approach

Start with the universe of interventions that are sufficiently cost-effective, either at the global optimum or a defensible local optimum. From that universe, ask which are cheap enough for the relevant payer at scale to sustain. Cost-effectiveness sets the evidence threshold; cheap enough tests viability at scale. This sequence seems to reflect how philanthropic capital can work best: determine what is worth doing, then ask whether it can be done sustainably.

2. Cost-Effective ⊂ Cheap Enough

Begin with what the payer at scale can afford, then apply cost-effectiveness analysis within that fiscal envelope. This is a reasonable framing from a government budget perspective, though it risks foreclosing high-impact interventions whose cost structure could be redesigned with the right investment.

3. Partial Overlap — the sector default

Some interventions are cost-effective, some are cheap enough, and some are both. This Venn diagram framing is descriptively accurate, but it is passive — it describes where the sector has arrived rather than how it might navigate more deliberately. The two frameworks end up operating independently rather than in productive sequence.

4. Equivalent — big-aid context only

Cost-effectiveness and cheap enough can converge into the same thing when large-scale philanthropy is the permanent payer at scale. GiveWell-funded interventions like bed net distribution (Against Malaria Foundation) operate close to this model. It is a real and important use case, though not a general framework.

5. Mutually Exclusive — avoid

Treating the two frameworks as operating in entirely separate domains with no relationship to each other seems like the least productive framing of all. Both are fundamentally about the financial conditions required to deliver and sustain impact, and keeping them siloed forecloses the sequencing that makes them useful together.

Implications

The two frameworks are currently converging in practice, driven by two forces: the push to optimise scarce financial resources (the cost-effectiveness moment), and the push to scale through government as payer (the cheap enough imperative). That convergence is healthy, but it also surfaces a real risk.

An intervention can be implemented at massive scale and still be suboptimal, consuming resources whose opportunity cost, even within the same sector and for the same outcomes, could have driven greater impact elsewhere. Cheap enough without cost-effectiveness risks inefficient scale. The inverse is equally problematic: an intervention can have significant impact potential but no viable path to financing at scale, consigning it to perpetual pilot status. Cost-effectiveness without cheap enough risks perpetual donor dependency.

Used in sequence, the frameworks address this. Cost-effectiveness is the right tool for thinking about opportunity cost and scarce capital, not only from a philanthropic perspective, but increasingly from a government and consumer one too. Once a shortlist of locally optimal interventions has been identified, cheap enough provides the reality check: is there a credible path to sustainable financing at the scale that would actually matter?

The practical implication is that the sector needs investment in both directions simultaneously.

On the cheap enough side:

Significantly more sophistication is needed in measuring willingness to pay for governments and consumers. The private sector has well-developed WTP methodologies; there are genuine opportunities to build equivalent services for development contexts.
This means better tools to understand political financing constraints and opportunities: existing budget allocations, fiscal space analysis, procurement processes, and the political economy of government adoption.
Interventions need to be designed with a specific payer and a specific price point in mind from the outset, not retrofitted to a financing model after the fact.

On the cost-effectiveness side:

Greater standardisation of the numerator and denominator across studies and sectors would significantly improve comparability and the ability to identify true optimums.
Practitioner skill-building in cost-effectiveness is as important as funder sophistication. Implementers who understand both the cost-effectiveness landscape and the willingness-to-pay dynamics of their target financing system are better positioned to design interventions that are both impactful and viable.

Ultimately, impact comes through implementation, not good intent. These frameworks are tools in service of that goal, not ends in themselves.

Frontpage posts - Effective Altruism forum viewer

Resources for starting and growing an AI safety org by Bryce Robertson

P(doom) ranges from 10% to 99% depending on who you ask. I searched 1,259 hours of AI safety podcasts to map the actual disagreements. by Bardoonii

Why we need to resist the idea of net negative lives in insects by Seidenpuma

AI risk is not a Pascal’s wager by ozymandias

Effective Altruism and Radical Trivialisms by Bentham’s Bulldog

Building the Field of AI Safety by 80000_Hours

Summary

The case for fieldbuilding

What do field­builders work on?

Why AI safety field­build­ing can be highly impactful

Field­build­ing works

AI safety needs more skil­led people

The work is highly neglected

Helping oth­ers lets you mul­ti­ply your impact

Po­ten­tial down­sides to work­ing on fieldbuilding

Ca­reer cap­i­tal may be lower

The ‘ca­reer path’ isn’t as clear

Tech­ni­cal work of­ten pays better

Direct work can also grow capacity

How can field­build­ing back­fire?

Could you be a good fit?

Skills that will help you succeed

Promis­ing pro­fes­sional backgrounds

How to get started

Find jobs in AI safety fieldbuilding

Learn more

Top recommendations

Fur­ther recommendations

Acknowledgements

My lover, effective altruism by Natalie_Cargill

AI for decision advice by Forethought

MCL: Watermarking LLM-generated Text with Markov Chains by Kyuhee Kim

Background

How MCL works

Step 1. Split the vo­cab­u­lary into se­cret groups.

Step 2. Add a hid­den cy­cle dur­ing text gen­er­a­tion.

Step 3. Calcu­late finger­print score.

What we can prove

Ex­per­i­men­tal Setup

Results

Limi­ta­tions and open questions

Conclusion

Ap­pendix A: Score separability

Ap­pendix B: Cross-model validation

AI safety is maturing. Is its generalist talent pipeline keeping up? by Uladzislau Linnik

TL;DR

Introduction

Call for Opinions

Terminology

Sources and Methods

Observations

Ob­ser­va­tion 1: Gen­er­al­ists face a struc­tural path­way gap into AIS

Ob­ser­va­tion 2: Or­ga­ni­za­tions ex­pe­rience the tal­ent gap differently

Ob­ser­va­tion 3: The field lacks pro­fes­sion­als with out­side-world op­er­a­tional fluency

Ob­ser­va­tion 4: Or­gani­sa­tions de­fault to hiring from within the AIS ecosys­tem, at a mea­surable cost

Ob­ser­va­tion 5: Or­gani­sa­tions have no shared screen­ing crite­ria for gen­er­al­ist roles

Ob­ser­va­tion 6: The gap may ex­tend to in­sti­tu­tional in­fras­truc­ture, not just people

Meta-ob­ser­va­tion: Emerg­ing hy­pothe­ses about root causes (thin ev­i­dence)

What these ob­ser­va­tions add up to

Uncertainties

Uncer­tainty 1: What is the shape of the gap?

Uncer­tainty 2: Will the cur­rent pat­terns self-cor­rect over time?

Uncer­tainty 3: Does the gap re­quire de­liber­ate in­ter­ven­tion at all?

Uncer­tainty 4: Which prob­lem is the most painful?

Epistemic Sta­tus and Biases

Fur­ther Steps

Con­clu­sion and call to action

Acknowledgements

Ap­pendix 1. Authors’ background

Ap­pendix 2. Ear­lier as­sump­tions and sources

Ap­pendix 3. In­ter­nal gen­er­al­ist Tax­on­omy table.

[Research Request] “Prestige Tax”: Quantifying Epistemic Silence in AI Safety Teams by James-Hartree-Law

Summary

The Prob­lem: Epistemic Bottlenecks

Why this mat­ters for AI Safety

The Re­search Goal

Call to Ac­tion: 4-Minute Survey

The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness by Vasco Grilo🔸

Abstract

What do fieldbuilders work on?

Why AI safety fieldbuilding can be highly impactful

Fieldbuilding works

AI safety needs more skilled people

Helping others lets you multiply your impact

Potential downsides to working on fieldbuilding

Career capital may be lower

The ‘career path’ isn’t as clear

Technical work often pays better

How can fieldbuilding backfire?

Promising professional backgrounds

Further recommendations

Step 1. Split the vocabulary into secret groups.

Step 2. Add a hidden cycle during text generation.

Step 3. Calculate fingerprint score.

Experimental Setup

Limitations and open questions

Appendix A: Score separability

Appendix B: Cross-model validation

Observation 1: Generalists face a structural pathway gap into AIS

Observation 2: Organizations experience the talent gap differently

Observation 3: The field lacks professionals with outside-world operational fluency

Observation 4: Organisations default to hiring from within the AIS ecosystem, at a measurable cost

Observation 5: Organisations have no shared screening criteria for generalist roles

Observation 6: The gap may extend to institutional infrastructure, not just people

Meta-observation: Emerging hypotheses about root causes (thin evidence)

What these observations add up to

Uncertainty 1: What is the shape of the gap?

Uncertainty 2: Will the current patterns self-correct over time?

Uncertainty 3: Does the gap require deliberate intervention at all?

Uncertainty 4: Which problem is the most painful?

Epistemic Status and Biases

Further Steps

Conclusion and call to action

Appendix 1. Authors’ background

Appendix 2. Earlier assumptions and sources

Appendix 3. Internal generalist Taxonomy table.

The Problem: Epistemic Bottlenecks

Why this matters for AI Safety

The Research Goal

Call to Action: 4-Minute Survey

The natural experiment

The structural difference

Directive as the predictor

The speed-oversight tradeoff

So what does all this actually mean?

Getting up to speed

Purpose and place

Highlights from 2025

2025 Talks included:

2025 Events included:

Speakers & Events

Career Fair

Festival Season! LessOnline —> Summer Camp —> Manifest

It’s hard to justify time and money on PR

Why contribute to the cesspool that is LinkedIn?

Final recommendations

Cost-Effectiveness: The Optimisation Tool

Cheap Enough: The Scaling Reality Check

1. Cheap Enough ⊂ Cost-Effective — Recommended Approach

2. Cost-Effective ⊂ Cheap Enough

3. Partial Overlap — the sector default

4. Equivalent — big-aid context only

5. Mutually Exclusive — avoid