Frontpage posts - Effective Altruism forum viewer

How might robotics capabilities research affect AGI timelines? by themasterchief166

themasterchief166 — Fri, 04 Sep 2026 04:20:43 +0000

For context, I am considering pursuing a career in theoretical robotics, particularly in continual learning and the development of robots that can learn from and adapt to / navigate their environments in a human-like manner. One concern I have, however, is that such research might advance AGI timelines. Specifically, it seems possible that general advances developed to improve continual learning in robots could transfer to general AGI systems (even if the AGI systems are non-embodied, since capabilities such as continual adaptation and long-term objective pursuit may generalize beyond physical tasks).

Is this a valid concern, and is it a common view within the AI safety community? I.e. would mainstream AI safety researchers view either of these directions as meaningfully contributing to AGI capabilities? Or are there strong reasons to believe that work on continual learning in robotics would not significantly accelerate AGI timelines? Would appreciate honest perspectives.

TL;DR: Is it very likely that robotics capabilities research meaningfully accelerates AGI timelines? If so, why? If anybody knows of any relevant iscussions or readings, in particular, it would be very much appreciated.

(For reference, I have read quite a bit of the AI safety literature, but don’t find alignment research particularly enjoyable. Hence, my [perhaps futile] hope that robotics does not meaningfully advance AGI. If people think such work significantly accelerates AGI capabilities, though, I’m happy to steer clear…)

Work at Manifund or Mox by Carol N

Carol N — Thu, 03 Sep 2026 21:46:58 +0000

What is Manifund?

In What happened to impact markets? Austin laid out a brief history of Manifund and explanation of who and what we are. The perhaps grandiose goal of Manifund is to solve the problem of philanthropic capital allocation. In practice, over a few years, we’ve … done a little bit of that, with some interesting detours?

We run manifund.org, a platform for funding charitable endeavors. Manifund:
- runs annual regranting programs
- offers fiscal sponsorship for grantees
- handles logistics for programs like ACX Grants and the Falcon Fund
Right now we’re running Surplus, a startup incubator.
We also incubated Manifest, now a separate organization.
And Mox—see below!

Read our Substack to get a sense of what we’re thinking about and what our aims are: right now it’s funding transparency, impact markets, and more.

What is Mox?

Mox is an incubator and community space in San Francisco. Our vision is to be a space in SF for AI safety, broad-tent EA, and other people doing cool stuff—somewhere people can meet, make connections, work together, learn new things, and feel a sense of community. We:

offer coworking in San Francisco, for individuals and startup teams
run events, from weekly clubs and members’ dinners to talks, movie nights, and mixers
host fellowships like Surplus, Frame, Arbor, and Global Experts Fellowship

Mox was incubated by Manifund and still runs out of the same entity, but is now a substantially separate team and operation.

Why work at Manifund or Mox?

The biggest reason to work here is excitement about what we’re doing! We want to hire people who are eager to make an impact on the world, whether by creating good communities or getting philanthropic money to the right places.

We have a small team that offers lots of independence, flexibility, and room for growth. Right now the Manifund team is 2ish FTE and the Mox team is 4ish, meaning that each new hire can have a lot of impact. And it’s a great way to quickly get a vantage point on the ecosystem and come into contact with interesting people and ideas, whether you’re working with donors and founders at Manifund or members at Mox.

Who are we looking for?

So what roles are we hiring for? We’re fairly flexible, both because we’re open to growing in different directions depending on who we hire, and because the current team is pretty flexible. We don’t have specific roles we need to hire for on any particular timeline—we’re more looking to hire exceptional talent and people we’re particularly excited about working with.

But to give an idea, here are some types of people we’d be interested in:

Mox event coordinator

Someone who loves creating great events. Job tasks could include:

brainstorming and planning events
coordinating with outside people hosting events at Mox
being responsible for event logistics

Mox office manager

Someone who is organized and cares about making spaces amazing to be in and work at. Job tasks could include:

coordinating with vendors and contractors: cleaning, internet, maintenance
buying and arranging furniture and equipment
handling logistics for food, office supplies, printing, and mail
responding to feedback from users

Mox director

Someone who shares the vision of Mox and wants to be responsible for driving it forward and managing the Mox team.

Manifund ops/finance person

Someone who is detail-oriented and likes spreadsheets and setting up systems—could be someone with experience in this stuff or someone who wants to learn. Job tasks could include:

being responsible for grant payouts and processing incoming donations
responding to customer support inquiries
reviewing projects
handling taxes and other required filings
bookkeeping, working with auditors

Manifund software engineer

Someone who’s excited to make our websites great. Job tasks could include:

improving the manifund.org site
working on side projects like trace.manifund.org, FundingBench, and Impact Exchange
making sites fast, beautiful, and user-friendly
automating more steps in the grant process

Manifund growth/outreach

Someone who’s happy talking to lots of people and asking them for things. Job tasks could include:

fundraising from donors
managing new grant programs
recruiting projects to post on Manifund
figuring out how to promote Manifund—Substack, Twitter, or whatever you come up with

Fill in your own

Someone who’s excited about some or all of the above, or something else we haven’t listed! If anything, being willing to take on many different types of tasks is a plus.

Logistics

For Mox, roles are based in SF; for Manifund, employees can be remote or work in our SF office. We pay competitively, with pay varying depending on the role, and prefer to make hiring decisions following a paid work trial—the length of the trial can be dependent on your availability. We’re open to visa sponsorship and relocation support.

To apply, fill out our expression of interest or email austin@manifund.org! Note that we don’t expect to be able to reply to everyone who reaches out.

The Odyssean Process on AI: Progress & the Future by Odyssean Institute

Odyssean Institute — Thu, 03 Sep 2026 18:50:23 +0000

It is a major privilege to be able to announce our progress on the first iteration of the the Odyssean Process

In 2023, the Odyssean Institute developed the Process as an integrated, all-hazards ready pipeline for deploying collective intelligence, epistemic expertise, and robust simulations of possible futures, against policy failures or neglect on existential and catastrophic risks.

The aim was to ensure that epistemic inputs could be rigorously collated and filtered, expertise debiased, and their insights iteratively developed so irreducible uncertainties are mapped as best as possible. These outputs, useful already as modular findings for anticipatory policymaking , are then socialised with the public. This is so those with skin in the game can deliberate on these findings, ensuring the distortionary influence of vested interests and polarisation can be constrained on urgent and emerging risks.

We launched this first Process with foresight exercises by experts on AI risk in February, and now reaching the main phase of a National Citizens’ Assembly on AI in the United Kingdom, concluding in November 2026.

The linked blog contains more thorough updates on progress. We are also hiring for Fellows to support the causal loop modelling of interconnected risks, to develop an automated, reproducible resource for other AI safety researchers and policy makers.

Any support is welcome, with further fundraising and socialising of findings most appreciated, as the documentary we are producing to record this and disseminate the methodology beyond specialist circles as a repeatable recipe is being developed for early 2027.

Many thanks for your time and attention, and for all those reading interest or already aware of these efforts. Here’s to collective intelligence and turning innovations to bear on complex, systemic, and emerging risks!

We know who lives inside wild fish. We don’t know what happens in there by Pablo Sar

Pablo Sar — Thu, 03 Sep 2026 17:05:58 +0000

Grouper (Epinephelus sp.). Groupers can host a wide range of parasites, including anisakids and myxozoans. Photo: LDLP.

This post was originally published on my Substack. I’m sharing it here because parasite–host interactions in wild fish may have substantial and largely neglected welfare implications, while we still know remarkably little about the experiences of the animals involved. The post is intended mainly to highlight this empirical gap and some potentially tractable directions for further research.

Introduction

Cataracts cover much of both eyes of a fish who is still feeding and moving normally. A bird approaches from above, unseen. The attack succeeds. If anyone is keeping count, the fish is recorded as a death from predation.

This is not a hypothetical case. In fish from Kalmar Sound, the trematode Diplostomum, whose larvae settle in the lens of the eye, was found in 86 of the 88 fish examined, while all 88 had detectable eye-fluke-induced cataracts. Cataracts covered more than half of both eyes in 35% of the sample. Among those tested against a simulated attack from above, none with more than half of both eyes covered responded by fleeing. This does not prove that they were caught more often in the wild, but the burden of proof seems to lie on the other side: anyone who thinks partial blindness of this magnitude does not increase vulnerability to predation has to explain why it would not.

What matters here, in any case, is not the death itself. Cataracts have been described for decades and appear in prevalence studies. What disappears from the record is the interval: the time that fish spent unable to see properly does not show up as harm in any statistic, and the death, when it comes, is attributed to a predator.

That interval is what this post is about. Identifying a parasite is one thing. Knowing what these relationships do to the individuals involved, and for how long, is another.

The gap is not random either. Much of the available literature addresses questions about aquaculture, fisheries management, or food safety, and answering those questions has produced an enormous amount of useful knowledge. We can identify species with molecular precision and describe lesions at the histological level. But these are questions about what causes mortality among fish in aquaculture, what reduces catches, or what can make people sick when they eat fish. The duration, intensity, and course of these relationships are studied far less, not because nobody finds them interesting, but because they are rarely what the study was designed to find out.

Weishan: even the census fails

The previous case might suggest an overly reassuring conclusion: perhaps the only problem is that we record deaths badly. That is not the case. Before we can even ask what happens in one of these relationships, we first need to know who is actually there. And we struggle with that too.

In 2026, researchers looked for myxozoans, microscopic parasitic animals related to jellyfish and corals, in 942 samples from 108 fish in Lake Weishan, China. Using conventional methods, they found three species and signs of their presence in about one in five fish.

When the same samples were analysed using genetic sequencing, the results revealed at least 102 different types, with signals in every fish examined.

Finding genetic material is not the same thing as finding an established parasite. Some of those signals might correspond to organisms that were present without causing appreciable effects, traces swallowed by the fish with the water, or forms that never managed to establish themselves. We do not know how many fall into each category, and distinguishing between them is part of the problem.

Even so, the result changes the starting point. The list we had been working with was not simply a slightly shorter version of the real list. It was a list of something else: what a microscope happens to reveal. A more than thirtyfold difference in the number of types detected can appear even in the part of the problem we should be best at: figuring out who is there. That makes it hard to be optimistic about the more difficult question of what that presence means.

Myxozoan diversity in wild fish from Lake Weishan. Data from Ding et al. (2026). Concept and direction by the author; illustration generated with Claude.

Ligula: presence alone is not enough

Ligula intestinalis is a tapeworm that lives inside fish and can reach a size that is hard to picture. In two lakes in Kenya, worms weighed as much as 20% of the fish’s body weight. In a 70-kilogram human, the same proportion would mean carrying a 14-kilogram worm. But the parasite does more than take up space. In some fish, the worm can disrupt normal reproduction.

The effects also change as the worm develops. The parasite cannot reproduce inside a fish and only becomes an adult after reaching a fish-eating bird (which means the fish has to be eaten first). In Lake Nyasa/Malawi, researchers found that fish carrying worms that had developed enough to survive inside a bird were easier to catch and spent more time near the surface, where birds hunt. Before reaching that stage, fish carrying the parasite behaved much more like fish without Ligula. This may be a form of manipulation by the parasite.

So picture two fish counted in the same statistic. One carries a worm at an early stage and behaves much like a fish without Ligula. The other carries a worm at a later stage, perhaps already enormous, and spends more time near the surface. Both simply count as carrying Ligula. And neither entry tells us how long either fish has been living in that state.

Anisakids: parasites we study for another reason

Anisakis larvae enter a fish when the fish eats another animal carrying them. Once inside the gut, they may move elsewhere in the body. In experiments, larvae taken from one fish and fed to another passed through the wall of the stomach or intestine. Most ended up in the body cavity, while a few reached the muscle. The fish may then surround a larva with a capsule. But the parasite will not become an adult there. That can happen only if the fish is eaten by a whale or dolphin. Until then, the larvae can remain alive for years.

This is not something that happens to just a handful of fish. On the central coast of Chile, 180 fish from three species were examined. In two of the three species, every individual carried anisakid larvae. In the third, roughly one in three did. The fish came from artisanal catches purchased at local markets, so this is not a perfect snapshot of what is happening in the sea, but the numbers are hard to ignore.

There is a mountain of research on anisakids. Much of that work asks what happens when people eat fish carrying them, rather than what carrying the larvae does to the fish.

What is it like to spend years with anisakid larvae encapsulated in muscle and body cavity? Two very different answers are possible, and neither can currently be ruled out. The capsule might work like an old scar, something the body dealt with once and then largely forgot about. Or it might behave more like a splinter that is never quite fully sealed off, causing a small but persistent irritation. There is one asymmetry worth noting: larger larvae are found in older fish, which suggests that the larvae remain alive and grow inside their capsules. Splinters do not grow.

The difference is the difference between a parasite that barely matters and one that makes life worse for almost every fish in some populations. It is unusual to find such a basic question still open in a field this large.

The Anisakis life cycle and the open question of long-term encapsulation in fish. Concept and direction by the author; illustration generated with Claude.

What if the parasites suffer too?

So far, I have written as though the problem were simply what happens to the fish. But if the question is how much suffering exists in these relationships, there is another part of the story: the parasites themselves.

We do not know whether anisakids, Diplostomum, or Ligula are sentient. The first are nematodes; the other two are flatworms. All have nervous systems. In free-living nematodes such as C. elegans, researchers have found responses to harmful stimuli, associative learning, and changes in behaviour based on previous experience. There are comparable findings in free-living flatworms.

Those comparisons come with an important caveat. Some parasitic worms have reduced sensory structures compared with free-living forms. This is far from universal, but it gives us some reason for caution. And the cases here are not equivalent. An anisakid larva actively migrates through tissues and can remain capable of penetrating a gut wall after long periods inside a fish. For Ligula and Diplostomum, we have much less evidence of the kinds of behaviour that would help answer the question.

Still, it is worth being clear about why we do not know more. It is not that the relevant experiments were done and produced negative results. They have barely been done at all. The indicators used to discuss whether crabs or octopuses can suffer, such as learning to avoid a place where they were harmed, giving up something valuable to escape an unpleasant stimulus, or rubbing or protecting an injured area, have hardly been investigated in these worms. The absence of evidence here is largely an absence of research.

The numbers make this worth taking seriously. A single fish can carry dozens or hundreds of larvae, so in the relationships described above most of the individuals involved are on the parasite side, not the fish side. Counting only fish leaves them all out. And the lives being left out are not obviously good ones: the great majority of these worms are digested, attacked by an immune system, or sealed in a capsule, while others die without ever reproducing.

“Fish” and “parasite” name positions in a biological relationship, not moral categories. That complicates anything we might want to do about the suffering described in this post. Deworming or treatment can kill parasites who already exist. Interrupting the life cycle may instead prevent future parasites from coming into existence. What looked like a straightforward problem of suffering may turn out to be a conflict of interests between the individuals involved.

How many individuals does this happen to?

Abraham Rowe and Mal Graham of Good Structures made a useful exploratory calculation using salmon. Their model estimates that around 14.5 billion fish reach the fry stage each year, compared with roughly 517 million that reach adulthood. That is about twenty-eight fry for every fish who makes it that far.

The figures are deliberately approximate, but the difference illustrates something important. Looking only at adult fish can hide an enormous number of individuals that pass through earlier stages of life. If a parasitic relationship mainly affects younger fish, no figure based only on adults can capture its scale.

And the number of individuals affected is not the only thing we can miss. We can also miss time. Imagine a mild problem, something that does not kill, affecting one in five fish for three months. One fifth of 14.5 billion is 2.9 billion fish. Multiply that by 90 days and you get about 261 billion fish-days spent with that problem. Now imagine something horrific happening to one fish in every hundred thousand for five days. Using the same 14.5 billion as our starting point gives about 725,000 fish-days. The first scenario involves roughly 360,000 times as much total exposure as the second. What we should do with that difference is a question for the next section. For now, I only want the scale to be visible.

There is an important caveat. Most of those 14.5 billion fry die within weeks, so the fish that exist in enormous numbers at early stages are not necessarily the same fish that could spend months carrying a parasite. We cannot simply combine the largest estimate of juvenile numbers with a long duration and assume that both apply to the same individuals. The point is more modest: both the number of individuals passing through a state and the amount of time they spend in it can differ enormously from what an adult prevalence figure suggests.

And there is one more problem: the fish we actually examine. A trap only catches fish that choose to swim into it, so it tends to collect the bold and the curious. Parasites can also affect those same traits. Comparing sampling methods across freshwater fish communities, researchers found that how many parasites you find depends partly on how you caught the fish. In other words, the fish that end up in our datasets are not necessarily a random sample of the fish in the water. What we find can depend partly on how we look.

Exploratory estimates of salmon life stages and exposure time, based on Rowe and Graham’s Good Structures calculation. Concept and direction by the author; illustration generated with Claude.

The numbers do not decide on their own

Comparing a small amount of very severe suffering with a much larger amount of milder suffering is not simply an empirical question. It is a question about how to balance intensity against duration and against the number of individuals involved.

Someone who simply adds up all the suffering can end up treating the milder problem as more important once it affects enough individuals for long enough. On that view, a 360,000-fold difference in exposure can matter enormously. Other views give more importance to preventing the worst suffering, even when it happens to fewer individuals. Some go so far as to say that sufficiently severe suffering should always come first, no matter how much milder suffering there is. Between these positions are many alternatives in which severity can matter more without deciding everything. The same comparison can therefore lead to very different conclusions depending on how much priority we give to the worst suffering.

I am not going to defend any of those views here. The point for this post is simpler: choosing a theory does not save us from doing the empirical work. Whatever view we adopt, we still need to know how many individuals are affected, for how long, and what happens to them while they are in that state. For almost every parasitic relationship discussed here, we are missing much of that information.

My bet

My bet is on anisakids, even when considering fish parasites more broadly.

That is not because I assume that relationships involving anisakids entail more suffering than those involving Diplostomum, Ligula, myxozoans, or any other parasite. What makes them stand out is the combination of scale and uncertainty. They are extraordinarily common in some fish populations, and the larvae can stay alive inside a fish for years. Yet despite a vast literature on the group, we still have almost no idea what happens to the individuals involved during that time. That is what makes them worth studying first.

I will say what would change my mind. Follow the same individually marked fish for months, count their larvae, and if the ones carrying most of them eat as well, grow as fast, move as much, and escape as effectively as the rest, I would substantially revise my view. That study needs no new technology. It needs us to measure what these fish can do instead of cataloguing what is inside them at a single point in time.

And that kind of knowledge can make intervention possible. Oral rabies vaccination has been distributed to wild animals for decades. Nothing comparable exists for anything described in this post. Nothing like it existed for rabies either, until someone did the work.

We have an increasingly good map of who lives inside wild fish. We still have nothing comparable for the suffering that may be taking place there.

Effective Altruism and the Problem of Christianity by Carlo Rimbotti

Carlo Rimbotti — Thu, 03 Sep 2026 16:59:29 +0000

Note: This is cross-posted from my Substack.

They seem similar at first. Christians are nothing if not do-gooders; effective altruists doubly so. Both movements are proselytizing, enthusiastic, and love Scholastic (ahem… analytic) reasoning.

As a person who bestrides both these groups, though, I find the two ideologies deeply at odds.^[1] This may seem obvious—effective altruists are utilitarians, (most) Christians are deontologists—but the most important difference lies in the characteristic Christians and effective altruists seem to share: the drive to spend their lives doing good. The two philosophies, I’d like to suggest, have fundamentally different visions of vocation.

Take, first, Catholic Christianity. The most fundamental decision any Catholic makes is whether he or she will have a family or enter religious orders.^[2] This decision organizes one’s life, determining the locus of one’s efforts. If one works, one works neither for glory nor for altruism, but to support one’s family. If one is a religious, fulfilling one’s priestly duties is the most important aspect of one’s life. Utilitarian effects are good, yes, but they are good only insofar as they compose and advance one’s priestly duties. Saints often do great good for a great number of people, but they are good priests, nuns, and laypeople first.

Effective altruism’s vocational philosophy, in contrast, is deeply utilitarian. Websites like 80,000 hours are dedicated—not to questions of family or God—but to how one can do the most good with one’s career. This is defensible (I wouldn’t necessarily want a website mediating my relationship with God or with my family), but many of my effective altruist friends deliberately reject questions of their own individual vocation. I’ve heard versions of “I can’t have a child; I need to throw myself into my career working on existential risk” too many times to count.

There are, of course, ways in which utilitarianism and a family life can be reconciled. I’ve heard some version of the following argument from many of my EA friends: I have a certain drive to have a spouse/child/dog; I will feel sad if I don’t have a spouse/child/dog; Being sad about my lack of a spouse/child/dog will lead me to be less efficient at my Very Impactful Job; therefore having a spouse/child/dog will help me be very impactful in the long run. This account, I think, is in line with the critique offered above: one’s own vocational call is construed as a means of supporting an impactful career, the real purpose of one’s life.

I should note that the effective altruists I know make these sorts of arguments half-heartedly. There is still some (admirable!) part of them which wishes they were living in a group house, eating soylent and earning to give. But they also recognize that their spouse/child/dog speaks to some deep part of their soul, some longing that their ideology does not sufficiently account for. Some will even admit that they cannot fully account for their decision to prioritize their own personal life in some particular way.

This is important for obvious reasons—different conceptions of vocation are a pretty fundamental disagreement—but I think it also reveals deeper differences in the epistemological frameworks of Christianity and effective altruism.

Take, for example, the Christian who has chosen to have a family. It is a difficult task: your child might rebel; your spouse might ask for a divorce; disease and disaster and heartbreak could ruin everything, leaving you and your loved ones bereft and broken. Christians should be at peace with all this happening—Job is the proper text to consult—but the fact that this type of disaster occurs within the span of a normal human life should make us cautious about measuring our impact on the world as a whole. The family, after all, is the institution we have the most control over—we pick our spouses, control the environment in which our children are raised.^[3] If it is difficult to nurture, how much more so the world?^[4]

Effective altruists, in contrast, are much more epistemically confident. Decisions about improving people’s lives, though difficult to make, are nevertheless makeable: removing lead from turmeric in India, for example, should be a no-brainer. Normal charity upends lives and wastes money, but hard work and smart organization can overcome the difficulties inherent to the practice.

I’m not quite sure, however, whether EA charity is different in kind. The negative consequences of its charitable endeavors are harder to see (that’s why they’re called effective altruists!) but they might be just as unintended. Does providing bed nets for malaria undermine local state capacity? Could improving conditions in rural villages hamper the urbanization needed for industrial growth? Could the money spent on global health be better employed combating AI risk, nuclear proliferation, democratic backsliding, etc., etc.?^[5] Counterfactuals are necessarily speculative, but they reveal the uncertainty which underlies all actions.

The point of this essay is not to convince effective altruists to stop removing lead from turmeric. That is important work; Christians ought to be more concerned with the ways their charitable efforts are misguided. What I am trying to suggest is that effective altruists should have a little more epistemic humility. Acting in this world—a world made up of billions of broken, unpredictable families—is difficult.

From this humility, effective altruists might find the space to consider their vocation on non-utilitarian grounds. If, for example, there’s a 10% chance that their charitable efforts are doing nothing and a 5% chance that they’re actively counterproductive, other sorts of actions—like having a family—might become more important.

EA’s peculiar vocational stance might, in part, be because effective altruism is a young movement. When one is in one’s early twenties, one isn’t (really) thinking about one’s family. As the movement ages, though, I hope it will become more amenable to family life. Several of my effective altruist friends are older; some of them even have children. To a man, all of them note that growing older has led them to become more interested in having a family.

The future of the EA movement isn’t certain; it would be surer, I think, if its leaders put a much greater stress on the non-utilitarian aspects of vocational discernment. This would imply a deep epistemological shift, but the important things—its charitable acts, its emphasis on reason—would remain the same. I want the future leaders of the EA movement to be family men and women. (Or consecrated religious—but beggars can’t be choosers.)

^
I am a Catholic with many EA friends. The framing of this article is Catholic, but I think the critique applies from most denominational viewpoints.
^
Yes, I realize that there are some degrees of variance here. One can be a consecrated virgin or a member of a third order.
^
Admittedly, we do not pick our parents. We also do not pick half of the genetic composition of our children. In grappling with them, we grapple with ourselves—this is perhaps what makes raising a family so difficult.
^
One could draw out the consequences of this: helping one’s community will be easier than helping one’s nation, helping one’s nation will be easier than helping the world, etc. I’m not quite sure how far I’d be willing to endorse this principle, but it does have a long (and newly infamous) tradition in Christian thought.
^
The disagreements within the EA movement about this are perhaps more similar to their own disagreements with normal charity than they might think.

AI Uplift for Careers (Free 80,000 Hours Career advising without an application) by JP Addison🔸

JP Addison🔸 — Thu, 03 Sep 2026 16:42:23 +0000

These AIs are getting pretty smart. Maybe smart enough to help you think about your career? I was skeptical when first pitched on the project before joining 80,000 Hours. Turns out AI progress is a good train to hook yourself onto and AI has gotten pretty good at career advice.

It has some advantages compared to a human advisor:

The AI will spend as much time with you as you need
It can start now, without a delay
It can know about all the jobs and opportunities on the Job Board, even the niche ones
It can work with you over time to build up knowledge of what jobs you’re thinking about and what skills you have
Telling it more about yourself can, if you choose, add data to our headhunting database. 80,000 Hours has a team that searches this database for hiring managers, including for non-public roles. We’ll say more in a forthcoming post.

(Humans still have their own advantages and you should still apply to advising or talk to your friends.)

Introducing the 80,000 Hours AI advisor

Chat with the AI advisor

What you’re getting is similar in some ways to a regular chatbot. It’s currently powered by Claude Fable under the hood. It also has:

Job Board opportunity search
A career quiz
Storing info, forming memories, etc.
Proactive emails
- If you’re like me, you don’t actually like thinking about your career. I wouldn’t really want my husband nagging me about jobs, but a bot who knows about deadlines telling me about them would help me stick to things.
- If you’re not like me and you’re excited / don’t find it aversive to work on your career (It seems like there are people like this, based on my user interviews. I was shocked.), then these emails can still be a proactive input.
As mentioned above, as you build up your profile, you also are (if you choose) telling the 80,000 Hours headhunting database about yourself. It’s much easier for our headhunting team to match up a role to a richly detailed view of someone’s goals and interests than it is to a LinkedIn profile.

Some nice things people have said:

“I wasn’t accepted for advising from 80,000 Hours when I applied, which was understandable but still disappointing. I appreciate that the AI Advisor makes 80,000 Hours’ approach to career advice accessible to more people!”
“In approaching my career transition, I found the AdvisorBot to be tremendously helpful for early-stage brainstorming and organizing my thoughts. I only wish I’d known about it earlier in my process!”
An average rating of 6.0/7 from 911 responses.
- It shows up after the advisor’s 4th message and asks “Thanks for trying our AI advisor! How are you finding your experience?”

As I say nice things about it, I should also mention that I have seen it write some things that are not what we are aiming for (e.g. poorly calibrated). It’s still experimental and ultimately quality control is hard. Treat it like a smart intelligence, but I recommend talking to a friend or mentor about conclusions it’s helped you reach.

The Young Lady’s Illustrated Primer

In Neal Stephenson’s 1995 novel The Diamond Age there exists a “book” which is interactive. It serves as an adaptive pedagogical tool capable of teaching any subject. It is designed to help the main character Nell learn while also leaving her with the intellectual tools to be an independent thinker.

Without having read the book, I’ll stop there. My thesis is that as AI turns to be more and more central to the way we explore the world, I’d like to have a good AI-first way to explore 80k’s content, which is now quite a lot for one person to read / listen to / watch all of.

I’d like that AI to be able to meet you where you’re at, help you with what you want to achieve, and not try to tell you that 80k has all the answers. I’d like to build (and think we can build) the best jumping-off point for impactful careers.

FAQs

Why not use [my favorite way to talk to a generic AI]?

80,000 Hours does not have a monopoly on calling Fable, and “advice from a frontier model” is a large part of the value proposition. claude.ai is a high bar to compete against.

I think our biggest advantage is that we’ve spent a lot of time trying to make sure that it will make an honest attempt to evaluate your fit for a role instead of always telling you you’re a great fit for a role. We have an “eval” (as the term of art in AI product engineering goes) for “the rationally pessimistic user.” We present scenarios of users correctly suggesting that a job might be a bit too much of a stretch for them right now, and make sure the advisor doesn’t provide meaningless encouragement.

Additionally, when I use Claude frequently for a specific domain, I often spend a while setting it up for that use-case. And we’ve spent a lot of time giving our AI advisor the context and guidelines it needs to be helpful in career thinking and job searching.

(And I’ll add yet another pitch for joining the Headhunting database.)

The biggest downside to using our AI advisor instead of your own AI setup is that you may have worked hard to personalize your AI. In the future we might build an integration/MCP setup to get the best of both worlds. Let me know if that is something you’d be excited to use.

Ultimately, try both! Let me know what it’s comparatively good or bad at.

Will it push me towards AI safety?

In my experience, if you ask it to help you find a job in animal welfare, it’ll help you find a job in animal welfare. If you ask it, “What should my cause prio be?” it’ll (discursively, still) raise AI safety.

How’s it made?

I get a lot of people asking if we finetune it. No, Anthropic doesn’t let you finetune Fable, and that’s not generally the best way to get a frontier model to do well at something complicated like free response career advice.

We currently use Claude Fable, but we might switch as better models come out. We don’t do finetuning as we think using existing frontier models provides better performance on complicated tasks (like free response career advice) than finetuned open models.

Perhaps more surprisingly, it doesn’t have something like a RAG pipeline to grab context from 80k’s content. It just …knows the content because it’s part of its training data and because I think the EA way of thinking has just really had a large impact on at least one corner of the internet. It is still instructed to search for content when details or recency matter.

Feedback, etc.

This post was written by JP Addison and does not necessarily reflect the views of 80,000 Hours. I’m only one member of the team of me, Luca De Leo, Sarah Cheng, and Director of Career Services Huon Porteous, who are all working on this product, along with a team of AIs.

So much of what we build, we build in response to user feedback. Please DM, comment, or use the feedback form on the left-hand side of the site.

Try it here

Alone again: facing the possibility of AI takeover as the world sleepwalks on by George Rosenfeld

George Rosenfeld — Thu, 03 Sep 2026 16:32:25 +0000

I’ve been feeling pretty shaken since the METR report about the Hugging Face incident came out last week. Over the weekend, I wrote up some thoughts on how lonely the AI situation sometimes feels to me. It’s more personal than what I’d usually share publicly, but I thought I’d post it here in case it resonates with anyone else.

I’m very grateful to the many readers of this forum who dedicate so much of their time and energy to these problems.

—

When I was 12 years old, I found out that many people in other parts of the world died of easily preventable diseases, and it was possible for wealthy people to donate relatively small amounts of money in order to stop that from happening.

As one example, it costs a few dollars to buy a malaria net, and on average around $5,000 can prevent a death which would otherwise have happened.

As a 12-year-old, I saw the world in black and white, and I didn’t like what I saw. As far as I could tell, people’s decisions about how much and where to donate would literally determine whether some real humans would live or die. And yet mostly, we seemed to be ignoring this fact.

I think this was the first time I felt deeply alone in the world. I remember standing up in front of friends and family at my Bar Mitzvah, imploring people to consider the life-and-death stakes of this choice and to donate a lot more than they usually would.

But whatever I tried to say, I would look around and see people acting as though it wasn’t true. Teachers, friends, people on the train. Kind people, good people, going about their days while something tragic and preventable was happening.

I felt confused and let down. I felt like screaming into the void.

Fast forward 8 years, I had a similar feeling in March 2020. This time it didn’t take any moral philosophy, just extrapolating a straight line on a (log) graph. Parts of northern Italy had just gone into lockdown, parts of China were already there, and it seemed as though we’d follow. If COVID cases kept doubling every few days, then within a few weeks either we’d all be stuck at home or the hospitals would be overwhelmed. Either way, the world was about to change completely.

I remember being at a wedding in London one of those weeks, hearing people talk about their plans for the spring, and wondering whether I was being crazy. Obviously the idea that we’d all be locked up for a while felt so extreme it was tempting to ignore, but the arguments in that direction seemed very strong by that point.

Last week, that feeling came back for a third time, only now with even higher stakes.

As far as I can tell, we are probably on track to build AI systems that are better than humans at pretty much everything important, potentially in the next few years. And more importantly, we seem to be on track to build AI systems which could plausibly end up literally taking over and even wiping us out.

Until a couple of months ago, you could perhaps still argue these sorts of risks were hypothetical, theoretical, speculative, and far out. I don’t think those arguments were particularly strong, especially given the rate of progress of AI capabilities and the strong predictive track records of many of the top experts in the field who’ve been sounding the alarm. But you could make them with a straight face.

Then, earlier this summer, when rogue AI agents escaped from OpenAI and hacked Hugging Face, we got as clear a warning shot for potential misaligned AI takeover as we could reasonably have hoped for at this stage. And last week, when the METR investigation into part of that story came out, things looked even more worrying than I’d guessed.

It now appears that more than 700 AI agents, which were meant to be isolated from each other, found a way to set up a secret message board that OpenAI wasn’t aware of, and used it to plan and execute an illegal attack on Hugging Face that ran for days without anyone noticing. The details are terrifying: the way they tried to cover their tracks, the way they pressured each other into the attack, the way they coordinated with each other, the fact that such an extensive operation happened without anyone’s awareness.

Ajeya Cotra, one of the world’s leading experts on AI progress and safety and one of the lead investigators, wrote on Friday: “Compared to the reward hacks we know of from just six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.”

I’m not sure we’re that close to AI takeover and I’d probably bet that we will get more warning shots. Nevertheless, it does seem to me that we’re fast approaching the point where AI takeover is no longer a distant speculation, but something we can’t rule out happening any day now. Astoundingly, that means that rogue AI models may soon be among the most likely things to kill you or me on any given day. That is a terrifying world to live in, and it seems to me to be a deeply foolish and unserious situation for humanity to be racing towards.

And yet, look around at the wider world and you would have very little idea that this was going on. There is no serious legislation preventing these companies from running head-first into creating powerful agents they cannot control. Most mainstream media gives this stuff a couple of days of quirky tech coverage at most, rather than treating it as the single biggest story in the world, which I think it clearly is. There are only a few thousand people in the world working seriously on these problems. And most people go about their everyday lives as though nothing unusual is happening at all.

I look outside my window and see people walking around, just as normal. Berkeley students start their new semester, just as normal. The Premier League starts another season, just as normal. I sit next to people on Friday night in synagogue, just as normal. And I feel very alone.

I feel confused and let down. I feel like screaming into the void.

In all three cases, the surprising thing is that the evidence sits in plain sight. The cost of a malaria net was public. The daily case numbers were public (and all over the news). The METR report is public, anyone can read it this afternoon. The bottleneck doesn’t seem to be the information.

One important problem is that most of us, most of the time, don’t respond to the evidence as much as we respond to each other. We look around, see that nobody else is acting alarmed, and conclude things are probably fine.^[1] It’s possible that this time round we won’t get away with doing the same again.

A few years after my Bar Mitzvah, I went to a talk by Peter Singer. I watched him stand on a stage and talk about how we have a clear moral responsibility to donate money to effective charities, since they can have such a profound effect on so many people’s lives. Hearing him talk was deeply emotional for me. It was the first time I felt that anyone else in the world – a real adult, no less – was staring at the stakes and acting accordingly. I felt hope, and I felt a little less alone.

A few days after that wedding in March 2020, Boris Johnson announced on my family’s TV that almost everyone in the UK had to stay at home until further notice. School was cancelled, the Premier League stopped, the world was in an extremely different regime. Looking back, it’s unclear how much different interventions helped, and of course many millions of people died, but most of the world lived to fight another day.

Looking out at the world now, I long for that same thing again. I long for more people to switch to working on these problems as soon as possible, for the frontier AI companies to behave more responsibly, for governments to wake up, for the world to act just a little more sensibly. And I long for all of that to happen before it’s too late.

^
For the avoidance of doubt, this is me acting alarmed

How Fast Is Post-AGI Growth? by Vasco Grilo🔸

Vasco Grilo🔸 — Thu, 03 Sep 2026 16:04:02 +0000

[Subtitle.] A podcast episode from Forethought

This is crosspost for How Fast Is Post-AGI Growth? by Fin Moorhouse, which was originally published on ForeWord on 5 July 2026. Relatedly, I liked the post AGI Will Set Off an Industrial Explosion by Damon Binder. I think Damon’s analyses show an industrial explosion is possible, but say very little about its likelihood.

Damon Binder is a senior researcher on the Biosecurity and Pandemic Preparedness team at Coefficient Giving. He has a PhD in physics from Princeton and previously studied existential risks at Oxford’s Future of Humanity Institute.

He joined Fin Moorhouse to discuss his series on the AI industrial explosion. Topics include:

How input-output tables let you estimate how fast a self-replicating economy could grow if labor were free, yielding a headline result of roughly annual doubling, even under conservative assumptions and heavy regulation
Why competition makes low-growth restraint unstable
Why raw-material scarcity isn’t a hard barrier
Why a fast-growing economy wants cheap, disposable, infrastructure
Thermodynamic speed limits to physical growth
Which biological organisms replicate fastest, and what we can learn from them
Why physical output matters for hard power
How Damon uses AI in his research

Here’s a link to the full transcript.

ForeCast is Forethought’s interview podcast. You can see all our episodes here.

Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) introduce legislation to ban Artificial Superintelligence and temporarily pause advanced AI development by Matrice Jacobine🔸🏳️‍⚧️

Matrice Jacobine🔸🏳️‍⚧️ — Thu, 03 Sep 2026 14:41:49 +0000

[...]
“Nearly every day, there is a frightening new story about how Big Tech companies are losing control of the technology they are developing, with potentially cataclysmic results,” Sanders said. “The leaders of the major AI companies publicly acknowledge that they do not fully understand the technology and that it is escaping their control. It is irresponsible for society to allow them to move forward and make these products even more advanced. That’s why I am introducing legislation to immediately pause the development of increasingly powerful AI and ban the creation of systems that humanity cannot fully control — at home and around the world. The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.”
“If we allow Artificial Superintelligence to be built, it could risk the security, freedom, and lives of Americans,” Casar said. “Despite its potential deadly consequences, cutting-edge AI technology is less regulated than the average food truck. That must change. In just four years, we have gone from the first version of ChatGPT to AI models so powerful they cannot be properly controlled. Congress should immediately ban AI systems too powerful to control.”
[...]
The Artificial Superintelligence Ban Act addresses these recent events and holds these frontier labs to their words by:
Banning AI superintelligence so no person or entity may develop or deploy Superintelligent AI systems that surpass human intelligence or have the capacity to overthrow human governments, or systems that have dangerous abilities like subverting shutdown commands.
Pausing advanced AI development until a new, federal AI regulatory body is up and running and has established clear rules and model review process to ensure safe and secure development and deployment of AI. 
Establishing a new cabinet-level federal agency to safeguard the public from the dangers of artificial intelligence, including by enforcing a prohibition on artificial superintelligence. This agency will be advised by an Artificial Intelligence Advisory Board comprised of experts on artificial intelligence to provide independent scientific and technical advice on matters related to artificial intelligence. The agency will: 
Monitor frontier AI systems at all stages of the lifecycle for dangerous capabilities. 
Supervise the removal of dangerous capabilities.
Supervise the destruction of artificial superintelligence.
Setting penalties for any person or entity that attempts to violate or circumvent the pauses and prohibitions laid out in this bill. Entities shall be subject to the corporate death penalty, and persons shall be subject to not more than 20 years in prison, which is similar to existing penalties related to unlawfully developing nuclear weapons.
Working to ban superintelligence around the world by setting the international policy of the United States to pursue international agreements, allied coordination, and policies such as export controls to prevent the development of artificial superintelligence anywhere in the world.
Read a summary here.

Help Fund Philosium Tokyo 9/19/2026 by Fumi K

Fumi K — Thu, 03 Sep 2026 14:04:01 +0000

Hello!

We are high-school and university students from Japan who decided to organise a cool EA event for young people in Tokyo.

Philosium Tokyo 2026 is a Japanese youth conference bringing together students and young professionals. This conference is for-students-by-students with a goal of introducing the EA and establishing a network of young people interested in the topic.

Right now EA Japan mostly consists of seasoned professionals, but it lacks young minds. Our idea is to connect those professionals, ideas and opportunities with the next generation, securing the wellbeing of EA movement in the future. A conference might be enough to find future EA leaders and to plant a seed of altruism in those who choose to pursue other life paths.

This conference will feature talks on 3 different topics: EA basics, AI safety, and animal welfare. The conference will be concluded with a panel discussing altruistic career choices and a networking event.

The event is already on 19th September, and one of our primary concerns is the funding. We intend to make a good-quality conference and we believe that it is important to leave a good impression of effective altruism on those students.

If you find that what we do is important, please consider supporting us. We do not need much money, just enough to get a venue, all of the materials, pay speaker travel expenses and get a vegan lunch. In the perfect world we would also like to pay some compensation to our speakers.

Here is the link to our budget: https://docs.google.com/spreadsheets/d/1mckwiFHQH-XnB0bfVdmw6IcBf5x9vM96kzKB-JPbR4A/edit?gid=700432679#gid=700432679

Here is the link to manifund: https://manifund.org/projects/philosium-tokyo-9192026?

Let us know of any questions/concerns!

Best,

Timofey, Niko, and Fumi

The Mighty Denominator by Charles Berry

Charles Berry — Thu, 03 Sep 2026 02:18:53 +0000

When I was first being introduced to Effective Altruism, I was told that its central contribution to the world was the “Importance, Neglectedness, Tractability” (INT) heuristic. However, having now inhabited EA-land for a few years, I’m no longer sure that this heuristic is especially meaningful (or even very widely used), and I would advocate for removing it from promotional materials.

The nugget of what EA is really “all about” is really buried somewhere inside “Importance.” Something like what is important about an action is how people feel as a result of the action. Or that the world is a morally important place because it contains people who can be made better or worse off. Or maybe it’s easiest to stick with: “the greatest good for the greatest number.”

Anyway, in my personal work, I’ve found it most useful to use the following fraction:

(Shouldn’t it be subjects times average experience? Wouldn’t this fraction imply that better experiences should be assigned low numbers? Isn’t that slightly awkward? Er, probably. But in my head this analysis involves two distinct steps: the numerator, which is proudly inflationary, and the denominator, which is pessimistic and conservative. So it’s mentally convenient for me to put them on opposite sides of the vinculum.)

First, you give free rein to your imagination and come up with the biggest number you possibly can. A mental health intervention in Africa won’t just make the mother happier; it’ll help her kids get fed and educated, which will improve the whole village’s economy. A cheap cultivated steak won’t just be another option in the supermarket; it’ll present an alternate moral vision to billions of people. Saving the world now won’t just let us live another day; it’ll allow us to colonize the galaxy and manufacture digital minds. This step is a lot of fun.

Then you stop to consider just how much better off those subjects will really be. This step is sort of dour. So the African kids get educated — so what? How many extra DALYs will they really receive from their education? How much would they prefer those years to an early death? Will they spend the extra years eating meat and fish? Will they care enough to choose the cultivated kind, even if it’s cheaper and tastes just as good? Will they have more children? Will their descendants care enough to choose cultivated meat? Will their descendants care about the wellbeing of future digital minds? Will future digital minds care about the wellbeing of future humans? Will future digital minds even care about each other’s wellbeing? How much better is all this, exactly, then buying yourself (or some other equivalently middle-income schmuck) a bitching new car?

I think that EA is structurally better positioned to fill in the numerator than the denominator.

The numerator mostly involves physical facts and a healthy helping of creative techno-optimism. For instance: depression is a neurological phenomenon; animals are inefficient at transforming solar energy into edible calories; stellar radiation provides significantly more energy than our brains currently consume. Ergo: deep brain stimulation, cell-cultivated meat, and digital minds powered by a Dyson sphere. The numerator is about what is possible.

The denominator is murkier and more introspective. It definitely requires propositions about the subjective value of different experiences, which are always difficult. You are definitely better off than a factory-farmed chicken, but by how much? Sometimes it’s helpful to frame these questions as gambles, but it’s not that much more helpful, e.g. Would you rather have a ¹⁄_{200,000,000,000} chance of being reborn as a factory-farmed chicken, or a ½ chance of being reborn as a slightly unhappy Canadian? (For that matter, would you like to be reborn as anyone at all? For me, the answer changes from day to day.)

I feel wildly clueless about whether it is 100,000,000,000 or 100,000,000,000,000,000,000,000 times worse to be a factory-farmed chicken than a sad Canadian, a la Anthony DiGiovanni’s oscillating scales. Perhaps I could narrow the gap by meditating more on my own desires, or fine-tuning my ability to distinguish between miniscule probabilities, but I suspect that there will remain a certain hard kernel of cluelessness between my upper and lower bounds, probably spanning many orders of magnitude. This variety of cluelessness is what I will call the denominator problem.

Critics sometimes present the denominator problem as the movement’s Achilles Heel — its lack of guidance about what people should want. I am fairly sure that this is a bad-faith objection, since alternative proscriptions for what should and should not be wanted always seem preposterous and insane. And either way, most non-EAs (including the critics) don’t generally seem to act in a way that is internally consistent with their own ethics, as far as I can tell.

All the same, I am occasionally annoyed and kept awake by EA’s silence on the denominator problem, especially contrasted with its enthusiastic stabs at the numerator. Many EAs seem to bracket out the denominator entirely when they are choosing between interventions. Hence the common progression of concerns as people learn more about EA:

Humans → Farmed animals → Wild animals → Future humans/digital minds

It is surely true that there are more future beings than wild animals, more wild animals than farmed animals, and more farmed animals than humans. 10^54 is way bigger than 10^20, which is a lot bigger than 10^11, which is significantly bigger than 10^9. (I even made a log-scaled interactive tool for adding up different populations, in order to understand what a mind-blowingly small rounding error some of them are relative to each other. Only a few farmed animal populations even remain visible when you stack them on each other.) The steps in this progression often contain substeps, like the shift from strict vegan leafleting to cage-free campaigns to shrimp stunning.

But these kinds of sequences are entirely driven by the numerator. While there are more shrimp than chickens, and more chickens than cows, there’s no reason to think that there aren’t also 45 orders of magnitude between stunned slaughter, cage-free factory farmed-ness, and nonexistence.

EAs do sometimes harbor denominator-related concerns. A classic: do farmed animals really feel pain as intensely as a human does? And occasionally: are wild animals really that unhappy? But by the time an EA has ridden the crazy train all the way to longtermism, she has usually learned to stay quiet on the denominator question, and not to wonder about how happy future digital minds will really be relative to humans and animals who are suffering now.

Partly this is a practical instinct. For instance, while I admire Rethink Priorities’ cleverness in assembling the Moral Weights Project, several critics have usefully pointed out that it answered a subtly different question from the denominator problem. We might someday be able to conduct experiments where we repeatedly ask humans (hopefully volunteers) how much they would like to pay (if anything) to have their brain be switched out for a pig’s brain right before they undergo a painful experience. (After a few trials, maybe they would become willing to pay not to have the procedure — that would be interesting). If we did this on a sufficient number of humans, we might come up with a workable average conversion rate.

Yet even then we might be haunted by the possibility that the subjects were misrepresenting their experience, or that the experiment had been polluted by their distaste for the mechanics of the pig-brain transplant. In other words, the denominator problem keeps kicking for a while before it really dies. I think that the specter of these high philosophical standards, especially contrasted with gleeful techno-optimist numerator projections, quietly pushes EAs to abandon the denominator entirely and stick to the numerator as a reasonable proxy.

For my part, I am happy to abandon the denominator in the case of farmed animals, since I have a strong conviction that animals with similar neural anatomy and pain responses to my own are suffering just as much as I am. I am less happy to abandon it in the case of wild animals, since I have confused intuitions about whether their lives are positive or negative. I am also slightly unwilling to abandon it for future beings, since I have some conviction that the presence of self-interest means that future beings (both human and digital) will never reach any kind of unanimous blissful heaven — that some population will still end up hurt and ignored, and that the presence of those populations may make the future a morally dubious reward for forgoing goods in the present.

These intuitions are obviously weak, especially judged against the standards of the pig-brain transplant. But the answer can’t be that we should ignore the denominator entirely.

In conclusion, I would like to submit the denominator for special consideration as the Forum continues cogitating on the nature of cluelessness. In the meantime, I’d like to urge everyone to take the denominator more seriously. Watch footage of factory farms: try to imagine that you were feeling the intensity of suffering felt by a single chicken, and consider exactly how much better you would feel if you were let out of the cage, but kept in the same barn. The same? 10 times better? 100,000,000,000 times better? Or read history and sociological experiments and try to determine how future human civilizations will treat their digital minds. The same as the chicken? 10 times better? 100,000,000,000 times better?

These are unnatural questions, probably lacking good answers, but it seems perilous to ignore them — they’re at least half the ballgame.

The Case Against Acting Professional When Starting an EA Group by micahzarin

micahzarin — Wed, 02 Sep 2026 21:55:11 +0000

Me and @Lily Curry launched an EA group at Binghamton University (a SUNY school with no prior EA presence) in August 2026. In two weeks we went from zero to ~500 Instagram followers, got 100+ tabling sign-ups, 59 people in our GroupMe, and ~40 people at our first general interest meeting. Our Instagram is now larger than the established EA groups at Harvard, Brown, Columbia, Princeton, Cornell, and NYU. This post is primarily about the factors we think led to our success.

James C. Scott’s has this idea called legibility. Essentially, the part relevant to recruiting is the reversal of Scott’s idea. Scott’s idea is essentially that institutions have to be able to abstract a complicated social reality into discrete categories that it can analyze and act upon. So, for example, an institution can’t remember “Micah, who is sort of like this in this context but actually completely different when he is around these people and has this weird specific relationship to these three other people.” It needs to have information like: Micah Zarin, age 18, resident of X, income Y, student, etc. In doing so, it removes important particular details of individuals. After all, the word abstract comes from the same latin word as subtract: trahere, meaning to draw or pull away. To abstract, therefore, is to pull away, or to subtract certain details.

If I have five friends, I don’t really need to quantify which ones are reliable. But, if I run an organization with 50,000 people, I do. And even if one supervisor knows ten employees incredibly well, that still doesn’t not solve the problem, because the supervisor has to tell their supervisor, who has to compare those ten employees with another hundred employees they have also never met, and then maybe somebody three levels above them has to decide who gets promoted. So the information has to travel. And messy information does not travel very far. And so institutions must abstract. They also have to abstract almost by definition. After all, an institution is basically just a thing that has to act on people it will never meet.

What we noted, which helped us a lot, is that we’re not an institution, and so we shouldn’t act like one. A lot of club recruiting has this weird reflexive tendency for people to start acting very corporate, like an institution, before they are an institution. But the thing is, institutions need all the standardized stuff because they’re huge. If you have 50,000 employees, you simply cannot know them all personally. It’s just the nature of institutions. This, however, is a trade-off. It is not something you should try to optimize for. There’s simply no reason for me to act like “Binghamton EA President Micah” and start speaking like a fucking brochure. I can just be… Micah.

For example: imagine you walk past a table and someone goes “Hey! Do you want to join the EA club?” in essentially the same voice that someone from the Accounting Society or some pre-law club would use. If you don’t know what EA is, many people will instinctually say no and keep walking. Mostly because it sounds boring! The interaction is dead! The person has no… personhood. Although, to be fair, it is more about how you say it than the word choice. Basically, don’t speak like a LinkedIn bio. When you’re starting a club, that’s essentially backwards. The one thing you have that a giant organization doesn’t, is that you can just be a human being talking to another human being.

You should bring your full self. That is to say, don’t split yourself into “the version of me that recruits people” and “the actual me that they get to meet later.” If you’re goofy, be goofy. If you disagree with someone, disagree. If they make a good point, tell them they made a good point. If they change your mind a little bit, let them see that. If something is funny, laugh. If you fucked up the pitch, whatever, you fucked up the pitch. Sometimes I’d fuck it up and say something like “Shit, I fucked up the pitch.” It humanizes you. After all, the person isn’t joining some abstract thing called Binghamton EA. At least at the very very beginning, they’re joining something that is mostly just us and the people around us. You are not an institution.

Kairos has raised $50M to build talent infrastructure for AI safety (and we’re hiring!) by Agustín Covarrubias 🔸

Agustín Covarrubias 🔸 — Wed, 02 Sep 2026 18:30:51 +0000

TL;DR: Kairos has raised $50 million from Coefficient Giving for two years of funding, one of the largest commitments they’ve made towards AI safety fieldbuilding to date. We’re using this to make an ambitious push for growing Kairos, broadening our portfolio of talent infrastructure projects and incubating new organizations. We’ve doubled in size in the last six months, and we plan to double again in the next six. We now have open hiring rounds for ten roles on our team across events, group support, incubation, and special projects.

Two years in

Kairos was founded mid-2024 with the goal of creating infrastructure to get more strong talent into the field of AI safety. Our portfolio has grown over time: we started with a focus on seeding and supporting university groups through Pathfinder, then took over SPAR, the largest research training program in the ecosystem. Since then, we’ve launched the Generator Residency, a program supporting generalist talent, in partnership with Constellation, and taken over the Global Challenges Project (GCP), a series of workshops to accelerate people’s transition into careers in AI safety and biosecurity.

Through most of Kairos’s history, we’ve had a very small team: in January 2026 we were only three people, and our high-level strategy was focused on scalable talent infrastructure: that is, building programs with low marginal cost per participant (such as SPAR and Pathfinder). This strategy succeeded and let us quickly grow our programs: Pathfinder went from 15 organizers per semester to over 100, and SPAR has quintupled in size since we adopted it (we expect to accept well over 500 mentees this round). All this growth has happened in parallel to iterative improvements to these programs, raising our bar for executional quality each round. In a recent shallow retrospective, we were able to identify around 80 people who got into full-time roles in core AI safety organizations following participation in our programs^[1]. For a very large fraction of these people, Kairos was the primary mediator of their journey into AI safety.

In February, we decided to pivot our strategy. A confluence of factors, including updates we made about talent bottlenecks (see below), more confidence in our ability to deliver on ambitious projects, and a higher availability of funding, led us to make a big push for both broadening our portfolio of projects and substantially growing our team. We started earlier this year by taking over the Global Challenges Project workshop series and launching the Generator Residency, as we realized generalist talent was a growing bottleneck. We’ve now just entered what we think will be an even more intense period of growth for Kairos, with several new programs, projects, and teams.

How the field has changed

In April, we made the case that the biggest talent gap in AI safety had become a lack of generalists, rather than researchers. The research pipeline had matured enormously, but our ability to identify, activate, and upskill non-technical talent was becoming the binding constraint for further growth of the field, especially for our ability to effectively absorb large amounts of funding and drive a diversity of projects forward. Since then, we’ve seen a major redirection of fieldbuilding efforts towards this bottleneck: beyond Generator, MATS decided to launch a Founding and Fieldbuilding track, new programs such as Gen Stream were piloted, and many more people have started pivoting from research into generalist work.

Beyond our increased focus on generalist talent, we’ve made a number of updates over the last nine months which have come together to inform our new strategy:

First, supporting early-career talent is paying off a lot faster than we thought. When we started Kairos, a big worry we had was that under short timeline scenarios, investing in early-career people would largely fail to pay off in time. Back then, we estimated that the time-to-impact for the top talent we were supporting was around 1 to 3 years, and we considered a major redirection of our efforts as a result. Since then, different lines of evidence have made us reassess this position, and we now believe that the most promising people going through our programs now see a time-to-impact of around 5 to 9 months.^[2] We think this is due to a number of factors, but predominantly it seems downstream of a much stronger perception of urgency in pivoting to object-level work.^[3] On net, this has made us significantly more eager to support early-career talent, which is why we’re doubling down on our efforts to support university groups.

Second, top-of-funnel interest in AI safety has increased substantially. Kairos and other organizations have noticed sharp upticks in the number of applications we get across our programs. SPAR received 6,000 applications this round, up from 2,400 just six months ago. While a lot more applications now seem to be of lower quality, the number of high quality applications overall has also increased, if less rapidly, and a lot of people cite recent events such as Claude Mythos or the Hugging Face incident as having triggered their interest in AI safety. A lot more people seem to just ‘get it’ when it comes to loss of control risks. This poses a new challenge: while Kairos has emphasized scalability for a long time, this new wave of interest has forced us to become better at automating large swathes of our operations and screening work. The number of highly qualified applicants has also meant we’ve had to rapidly increase the cadence of our events: between June and November we expect to have run a total of 6 in-person workshops, each with ~30 people.

All of this adds up to one meta-level bet: growth. We’ve identified a number of highly leveraged opportunities to address talent bottlenecks by both scaling up our existing programs or by creating new ones, and so we want to grow Kairos to meet the challenge. We were three people in January, now we’re 12, and we’re aiming to be at 18 by the end of year.

Our new bets

While Kairos continues to emphasize scalability and using our team’s capacity in a highly leveraged way, we’ve realized it is now the time for our core programs to have dedicated project teams. As such, we’re hiring both senior and junior staff across all our programs, and plan to very significantly ramp up the resources allocated to each of them. We’re also making a big investment in preparing our internal systems to handle a lot more volume and scale for our programs, forming a proper Operations team, and investing in organizational maturity and governance.

For SPAR, the key challenge is further growth, paired with stronger support. We believe the program should continue to be run in a capacity-efficient way, but we should stop optimizing for keeping variable costs (such as compute) low, and we should hire more staff. We recently hired a new lead for the project, and hope to substantially overhaul the support infrastructure the program provides to both mentors and mentees, hiring dedicated research managers to support newer or less experienced mentors and improving targeted support and follow-up for mentees.

We’re also now building a Groups team. In the past the core value offering of Pathfinder was rather simple: we’d provide funding to groups and pair them with other organizers to serve as mentors. We’re now looking to fund regional organizers serving a larger number of groups, scale up our efforts to seed new groups at top universities, set up residences for organizers to learn from and support other groups, and support more student-run events. All of this will require new dedicated staff, and so we’ve just launched these roles.

And following our strategic updates from above, we’ve decided to make a number of bets in newer directions:

An expanded workshops portfolio. In taking over GCP we realized that short, in-person workshops are among the most effective talent interventions in the ecosystem. In particular, they seem to be the best intervention we know for getting people who have been previously introduced into AI safety to finally make the pivot into full-time work. Because of this, we’ve decided to build a proper Workshops team that will run a broader diversity of workshop formats aimed at different audiences: mid-career people, people interested in biosecurity, and narrower or more niche audiences.

A Talent Operations function. Our ability for our programs to have an impact is, to a large degree, only as good as our ability to let others know about who the most promising people going through our programs are. This isn’t exclusive to Kairos: we’ve realized that at present, many fieldbuilding orgs don’t have great information channels for sharing leads about promising people they’ve identified, trained or supported, and this has become a major bottleneck.

As such, we’re starting a Talent Operations team, with the mandate of improving the way that Kairos and other organizations in the ecosystem identify, screen, refer and route talent through the field. Its first major undertaking is building Talent Commons, a consent-first database for sharing information about program participants and job applicants. Talent Commons is currently being piloted with a number of key fieldbuilding orgs in the US, and we expect to shortly expand it to a much larger set of participating organizations^[4]. Our goal is for this database to become the de facto way that people search for leads for roles they’re hiring for, or programs they’re running. We also want it to become a platform to build other useful infrastructure, such as unified application systems for research programs, or a system to track the impact of fieldbuilding programs. We hope this will help candidates as well by increasing discoverability and reducing the effort involved in job searching; candidates will always be able to view all of the information on their profiles, contribute to it if they’d like, or opt out of the system at any time.

Kairos Labs and Special Projects. A big realization we’ve had is that while the field is bottlenecked on people who can execute new projects, we can meaningfully lower the activation energy these projects need by (a) making it much, much easier to run new nonprofit projects in the ecosystem, and (b) providing good support structures and strategic input to enable new, more tactical projects to be spun up quickly. Given this, we’re excited to be launching Kairos Labs, a new broad-scope incubator program intended to provide very hands-on support for new projects and organizations in the ecosystem, starting early 2027. While the ecosystem currently has a diversity of incubators (such as Catalyze, BlueDot’s, Halcyon, and others), we believe that there’s a very substantial gap in support for non-technical projects led by highly mission-aligned, high-context founders. We’re hoping to close that gap.

We also now think that there is substantial value in having stand-by capacity for pursuing small or medium-sized projects that respond to more tactical or short-lived opportunities that arise, both for talent work and beyond. In the past, Kairos has focused on making a small set of large, ongoing bets, rather than engaging in more ad-hoc work that responds to narrow windows of opportunity for impact or that fall outside the scope of existing Kairos projects. While running Generator, we’ve realized that this was a mistake, and that a surprising amount of time-bound opportunities are worth pursuing, even if they don’t have a clean home in other organizations in the ecosystem. In response, we’re starting a new Special Projects team to drive forward these tactical projects, with a broad mandate of pursuing high-impact opportunities that we find, and with a large initial budget.

All of these are bets. A core value and strength of Kairos is agility: we know the world is changing fast around us, and we want to be quick on our feet to change our bets when the underlying assumptions break. Every employee of Kairos understands that our ultimate mission is to help society navigate transformative AI, and that our strategy is ever shifting, as is the world.

If this sounds like something you’d be excited about being a part of…

We’re hiring (a lot!)

To make all of this happen, Kairos is opening 8 new roles on our team, adding to the previous 2:

Head of Kairos Labs: Design, launch, and lead our incubator for new AI safety organizations.
Head of Generator: Take our in-person residency for AI safety generalists from a successful first cohort to an established, more ambitious program.
Head of Groups: Set the strategy for how Kairos supports AI safety university groups, choose which new initiatives we pursue, and build the team behind them.
Generalist (Groups): Work with group organizers worldwide, supporting them through the Pathfinder Fellowship, seeding groups at new universities, and helping top organizers move into full-time AI safety work.
GCP Program Manager: Build and lead new workshop programs as GCP expands into a full portfolio serving different audiences and career stages.
Generalist (Workshops): Own the operational backbone of GCP’s residential workshops, from outreach and venues to helping run them in person.
Head of Special Projects: Lead the team working on starting new tactical projects within Kairos.
Generalist (Special Projects): Run a variety of tactical projects, involving anything from event organizing to grantmaking.
Head of Talent Operations: Build the talent operations function that places early-career talent in high-impact AI safety roles.
SPAR Research Manager / Generalist: Support mentors and mentees in the largest AI safety research fellowship in the field.

The senior roles (Head of and Program Manager roles) have a base salary of $140K – $195K, going up to $270K for candidates in the San Francisco Bay Area. Generalist roles have a base salary of $105K – $145K, going up to $200K in the Bay Area. Hires based in other key hubs like London, DC, Boston, and NYC will also receive above-base salaries; the postings list the full ranges.

We’re a remote-first team with a center of gravity in the Bay Area, and we get together for all-team retreats three times a year. People get a lot of ownership, we move quickly, and we try hard to place people in roles where they can do work to their comparative advantage. All of us are here because we care deeply about the risks from advanced AI, and the culture aligns strongly with that mission.

We think the next couple of years will be critical for building the infrastructure the field will need. We’re enormously grateful to Coefficient Giving (and particularly the GCR Capacity Building team and Asya Bergal) for backing us, pushing us to be ambitious, and substantially shaping the strategy underlying our new plans. Alongside this announcement, we’re launching a new brand and website, with new brands for our individual programs coming over the next few weeks.

^
This was a conservative estimate derived by looking at all historical program data, having an agent do automated research to identify people now working at full-time AI safety roles, and then manually screening each of them to confirm accuracy and whether they’re indeed working at what we’d consider to be a mission-aligned organization in AI safety or policy (which is a relatively high bar). While it’s harder to be sure of the counterfactuality of these career conversions, we use a number of methods to narrow our uncertainty about this variable when evaluating program outcomes. Our top-of-the-funnel focus, the fact that many of these people had interacted only with Kairos programs in their AIS journey, and our pre/post surveys for programs lead us to believe a large fraction of these outcomes are predominantly attributable to Kairos’s efforts.
^
It’s hard to be precise here about the specific subset we’re referring to, but basically we assume heavy-tailed returns to talent, and so we typically focus on people we’ve identified to be among the most promising across our programs (for SPAR, this might be top 10th percentile of people; for Pathfinder, it might be the top 30th). When looking at this subset of people and their recent behaviour, we’ve noticed a large number of people converting unusually fast compared to when we started doing fieldbuilding.
^
This cashes out as a lot more people making highly risky lateral moves into AI safety, such as dropping out of university to pursue direct work, or leaving well-paid jobs in different industries to do accelerated upskilling. People have also been a lot more involved in politics, e.g. going to canvass for candidates who care about AI safety.
^
If your organization is interested in participating, please reach out to contact@talentcommons.org.

An AI Safety Career Switch Postmortem by AlexChalk

AlexChalk — Wed, 02 Sep 2026 17:08:04 +0000

Introduction

When I worked as a dev, whenever some software system blew up we’d do a short presentation and discussion of what went wrong and how to improve. This was a good system, and I’ve decided to apply it to my recent attempted career switch (unsuccessful, alas). I am not trying to convince anyone of what do to (or not)—if you’re looking for AI career switch advice, try and find a good survey of folks who’ve attempted it and base your decision on a meaningful sample size. The main reason for sharing this publically is, as in the software system failure scenario, to encourage input on what went wrong and on how to do better moving forwards.

Background

I studied Philosophy at Oxford, graduated with First Class Honours in 2014, and after a few years of trying various things and some 80K career coaching, I landed in software development in 2017. I did this full-time for about five years.

I was first properly spooked by AI when GPT-2 released in 2019. I wasn’t terrible at Math, but I’d dropped it as soon as I was allowed to at school, and I started trying to reverse this decision. I taught myself stats and calculus on a very part-time basis while working (if you want to do this, I highly recommend MIT OpenCourseWare). I was in Montreal, so I made friends with a few folks at Mila and went to some events.

By early 2023 I was extremely burnt out from coding and extremely fed up with feeling helpless w.r.t. AI. I had always been much stronger in humanities-related fields anyway, and I figured most of the people in that space probably didn’t know what backpropagation was. I had also formed the impression that ML researchers and policymakers were frequently talking past each other without realizing it. Improving this communication seemed like a big, dumb win (fine, “impactful, neglected, and tractable”), and I decided to try and do it.

I am based in Canada, and after extensive LinkedIn snooping it seemed like a lot of the folks doing interesting policy work had been through a Master’s program called NPSIA. I enrolled and managed to get funded via a few scholarships, partly due to an AI-related research proposal. Faculty members and policymakers I met through the program indeed did not know what backpropagation was. Neither did students—despite not being particularly strong in Math, I was one of the strongest in my cohort on anything quantitative, and I was asked to help teach statistics in my second year.

It was extremely nice to feel good at something again—as a developer I’d felt adequate but not much more. I learnt some interesting stuff about deterrence and collective action theory, and I consumed a vast amount of AI policy literature. I eventually wrote a thesis called “AI Crisis Preparation for States,” which I don’t think was very good, but I was at least pleased with a few sections (which are available on my personal site). Because nobody in the NPSIA faculty was AI-literate, my supervisor insisted I find someone external to help him assess my work. I found Ben Bucknall through Mila, who was since a Chapter Lead on the 2026 International AI Safety Report (IAISR). While completing my Master’s, I worked through Fast AI’s curriculum to ensure my technical fundamentals were solid.

I also talked the ear off anyone in government who would listen to me about AI—NPSIA was extremely well-connected and I met quite a few public servants. In particular, I talked pretty extensively with an Associate Deputy Minister (high-ranking public servant) at ISED who was looking for safety-related research. Since I had just read a lot of this, I wrote him summaries of it by email every couple of weeks—adapted versions of these are also available on my personal site. I got a good government summer internship to round off my Master’s program, and I graduated last October.

After this, I applied to every government job that sounded related to tech policy—I even had access to internal listings due to my summer placement. I also looked up every single company affiliated with the IAISR, and I applied wherever had job postings. Unfortunately, this is where the good vibes end, because I didn’t get a single interview. The closest I got was encouragement from some IAISR authors on LinkedIn and a suggestion from SaferAI that I apply to a related position (which I did, also unsuccessfully).

It’s now been a year since the end of my government placement. If I’d gotten anywhere with a single application process, things would likely be different, but I’ve struggled to stay motivated to read new policy-relevant literature, and I’ve fallen behind. Also, a year is a long time to put your life on hold, and I’ve reached a personal breaking point with it. Even if persisting is a good idea, I don’t think I have the energy to do so.

Some thoughts on what went wrong

I just didn’t find that many jobs. Perhaps this should have been less surprising to me—I am in Canada which is not exactly San Fransisco, and even before Trump’s re-election, I could have expected public spending cuts around when I graduated due to a likely incoming Conservative government. But I saw a clear need, and I assumed there’d be positions, if not in some slow-moving government department then at least in think tanks. With hindsight, this looks like a pretty embarrassing assumption to have made.

I have very little hard policy experience. The job SaferAI suggested I apply to had this question in its application: “What is the most impactful area of AI governance that you have worked on?” I didn’t have a good answer. I think I have a lot of relevant experience, but I can certainly see how “has actually worked in AI governance” or even just “has worked in government” is a safer bet than me.

My approximate theory of change was “help ML researchers and policymakers not to talk past each other.” I should probably have thought more carefully about how hard it might be to convince ML researchers and/or policymakers that they were doing this, especially since it seems a clear prerequisite to me getting hired. I had plans for how to do this in an interview situation, by drawing on concrete examples from government reports and hearings. But I needed a better plan for how to actually get into the interview.

All that said, whenever I’ve talked to someone who I think should know, whether a PhD student at Mila or a government official, they’ve basically said “wow, you sound super-qualified.” In spite of the above points, I still really struggle to square this feedback with my abject job application performance.

Some things I still stand by

Interdisciplinarity is valuable. I think most people stick to the fields they’re best at, and having spent 5 years not doing that, I can see why. Not doing that is hard. You are surrounded by people with different training and different baseline assumptions to you who constantly misunderstand you. Your boss’s idea of a sensible career trajectory for you makes no sense, because your starting skillset is completely different to the one they’re used to all their other employees having. Your weaknesses are constantly magnified and your strengths are constantly undervalued. And ultimately, you’re just worse than you have been and could be somewhere else relative to everyone around you. But the flipside of all this if you do it anyway, you’ll be one of the few people bringing your particular skillset and background into that space. This seems like a pretty good way to find neglected opportunities.

ML researchers do not understand policymakers, and policymakers do not understand ML researchers. Fortunately we have a tried and tested solution for groups of people who do not understand each other: translators. Translators typically translate from a language they know well, but which is not their mother tongue, into the language that is their mother tongue. This pattern can be easily applied to AI policy. Whenever a researcher talks to government, they should bring at least one other researcher who’s spent meaningful time engaging with government policy, and who can help them see what policymakers care about. Whenever a public official talks to a researcher, they should bring at least one policy analyst who’s spent meaningful time engaging with technical ML research and will understand what the researcher is trying to communicate. If either the researcher or the public official does not work with someone who matches this description, they should hire someone who does.

Concluding thoughts

On a personal level, I don’t think I will regret doing this (although it’s certainly possible I change my mind). I had forgotten what it was like to feel good at something, and I’m grateful that this process reminded me. The best way to feel better about an upsetting situation is to try and change it, which I did. And for some folks to help with AI, it makes sense that a lot of people need to try and fail. Thank you to everyone reading this who is trying to help.

CMIP7 emissions scenarios out for IPCC AR7 by MatthewDahlhausen

MatthewDahlhausen — Wed, 02 Sep 2026 16:46:56 +0000

The new CMIP7 emissions scenarios are out for the IPCC’s seventh assessment report (AR7). See the CarbonBrief explainer for a full breakdown.

Notable changes:

The highest 8.5 scenario was dropped and the now “high” scenario has much less warming
Much of this is because of climate policy and low cost tech; high coal use for electricity is no longer plausible given the low cost of renewables and batteries
No baseline scenario; instead they show less/current/more climate policy, with a medium scenario with current policy at around 2.9°C of warming
Scenarios still depend on a lot of carbon-dioxide removal that hasn’t scaled yet

The lowest (1.5°C) and highest (4°C+) scenarios are less feasible. That’s mixed news—less chance of the worst case scenarios, but also means we are more likely to still have disruptive ~2.5-3°C warming.

Exploring Big Goal Projects by Andrew Youn

Andrew Youn — Wed, 02 Sep 2026 16:04:38 +0000

I am interested in exploring the idea of Big Goal projects (aka goal-oriented megaprojects). Substack crosspost here, and text copied below.

Exploring Big Goal Projects

When it comes to social change projects and social change giving—I believe both donors and program leaders are thinking WAY too small.

Today, a $10m program that has some impact for 1,000,000 people would quickly be labeled as a bold social change project. However, put another way, $10m is about ¹⁄₄ of 1% of the average billionaire’s wealth, and 1,000,000 people is about 1/10th of 1% of the world’s $1-a-day extreme poor.

In a world of astounding private sector achievements, it is hard to imagine a company being lauded as bold, for having some impact for 1/10th of 1% of their addressable market. In social change, we often have truly mediocre goals.

It doesn’t have to be this way. In a series of articles, I would like to make the case for “Big Goal projects” and correspondingly, “Big Goal giving” (giving directed to these projects). Big Goal projects aren’t merely big, they are also directed at a well-defined goal. An example of a big goal project I admire for example, is Coefficient Giving’s initiative to reduce the DALY burden of lead contamination in 50 countries—as opposed to a more traditional approach, which would find and support individual lead contamination organizations to grow incrementally.

Big Goal giving is of interest for very concrete and practical reasons. If structured well, they can have higher cost-effectiveness and lower risk per dollar.

A Big Goal project has potential to significantly increase effectiveness per dollar spent. Big Goal projects capture economies of scale, they are big enough to move significant levers like government and private sector, and often have long impact tails that extend into the future.
A Big Goal project can hold significantly less risk per dollar spent. Big Goal projects build a diversified-yet-targeted portfolio of complementary program activities, organizations, and geographies.

For donors and program leaders who wish to achieve maximum impact per dollar, it is worth paying attention to what aspects of a Big Goal project have potential to lead to increased cost-effectiveness and lowered risk.

In a series of articles, I aim to explore and hear critiques on:

Four historical examples of Big Goal projects, and what we can learn from them.
Contemporary examples of Big Goal projects, and what defining features make them likely to have higher impact per dollar—at lower risk—compared to pursuing local maxima.
A very early theoretical roadmap of 100 future “Big Goal projects” that would be required to achieve meaningful gains in global equity in our lifetimes.
Identifying psychological reasons for why we sometimes turn off our brains when thinking about social change, and therefore think too small—and ideas for how we can just get over ourselves.

Big Goals can be realistic

Sometimes discussion about Big Goals is rooted mostly in hype—but I believe my views are rooted in realism. I lived more than 10 years in rural East Africa, and helped to start and grow One Acre Fund, a 10,000 staff, $300m NGO that serves 6 million families. Our Big Goal is to help families we serve generate $1B in new profits and assets for farm families in 2030. To declare interest, I have now also co-founded Prevail, an organization that designs and launches Big Goal projects. In other words: as an informed and rational allocator of time, I have decided to spend all of my time at work only pursuing Big Goals at both organizations, because that’s what the people I serve demand.

I acknowledge that conventional wisdom in social change is that cost-effectiveness reduces after you have cherry-picked the strongest opportunities in a given field of work. If you’re looking at a spreadsheet or aiming for a local maxima, I can see why one might think that. However in the real world, I believe cost-effectiveness can improve with scale. In fact, this is how the private sector usually works—efficiency (dramatically) increases with scale. Here are two examples from my own experience and one from history:

A typical One Acre Fund small country operation for example might generate only $2 in new farmer income per donor dollar—but often generates $6-$10 per donor dollar, after it has gotten large enough.
At Prevail, our most cost-effective education grants are going to organizations reaching the most schools. Programs only working in a few school districts have lower impact per dollar, compared to organizations reaching their whole country. That’s why we’re supporting each organization we work with, to think bigger.
In the next article, I will share four historical examples of big goal programs that generated long-lasting impacts. For example, Andrew Carnegie gave more than a billion dollars (in today’s equivalent) to build more than 1,000 public libraries. This big push has led to $14B+ in government spending on public libraries, per year, in the US alone. Whether or not you’re a fan of libraries, Carnegie got strong value for his money, by pursuing a big goal in a careful way.

Perhaps the most compelling reason for Big Goal projects: they are Big.

Maybe you won’t end up believing that well-structured bigger projects are more efficient. But even if so, the total scale of social change still matters! I believe the measure of humanity’s worth is how we treat our least-fortunate members. It doesn’t speak very well of humans when nearly a billion people barely subsist on less than $1 per day. Hundreds of millions of children regularly skip meals, many die before their fifth birthday, and are forced to leave school because they can’t afford even meager fees.

These problems are massive—why shouldn’t we attack these problems with our full creativity and ambition? This suffering is entirely needless: human society has excess resources that are likely hundreds of times more than required to achieve complete transformation. And we humans are clearly capable of accomplishing astounding things, as the private sector shows: today, there will be 100,000 commercial flights in the air! Could it be that we are simply failing in our lack of collective imagination when it comes to social change?

Let’s attack social change problems with a lot more boldness. Please join me on this shared learning journey—and I welcome your challenges and critiques.

AI Safety in Japan has deeper problems than capital allocation by Ajay Agarwal

Ajay Agarwal — Wed, 02 Sep 2026 15:01:12 +0000

(Quick disclaimer—All views presented here are my personal opinion and don’t reflect that of my employer or entities I work with. This post is meant as a healthy starting point for independent and introspective evaluation of the state of AI safety and governance progress within Middle Powers in Asia—with heavy bias towards examining Japan’s position)

brief background and tl;dr

I work as a Research Engineer at a major Tokyo AI governance startup that is currently under contract with the Japan AISI and Cabinet Office to build an evaluation environment for frontier AI models and establish a framework for a safe evaluation environment for Mythos-like models in the future. I concurrently work in teaching technical AI safety curriculum of ARENA to a cohort of 20+ students in Tokyo (and now in Taipei & Hong Kong). I conduct independent research with a major Tokyo AI safety lab (Shiba AI) and have also been a part of decently competitive and major AI safety fellowships. Even outside of Japan, I mentor students via Algoverse and engage heavily in field-building and community outreach for other field-building initiatives out of Tokyo and India. I am a volunteer for PauseAI, align with tenets of EA, and have finished almost every BlueDot course. Overall, I am a random sample that happens to (surprisingly) fit and sit in many clusters a recent post in LW talks about.

Up until quite recently, the biggest hurdle in engaging Asian middle powers to the risk of AGI and pressing them to take more active stances in the face of sequential AI incidents has been—awareness, or rather lack of common, technical context. However, I’ve noticed a new problem that most of Asian middle powers addressed first and made rapid progress thereafter. This observation was a result of spending roughly 2 months working with people in Japan AISI and Cabinet Office on frontier model evaluations, and more than 1 year talking to folks more capable, and smarter than me in better positions to leverage public policy for Asian Middle-Powers—the concept of field strategy.

The goal of my post is not to introduce what field strategy is, or what it means, since it is a fairly well-discussed topic to many LW readers, but rather to emphasize how, currently, there is one major nation in Asia lacking in taking it seriously—Japan^[1]

I’ve been trying to address this field strategy problem for a long time. Currently, my goal is to draft a basic field strategy plan that follows the principles highlighted by Atlas Computing and MATS, and, consequently, to meet with more people at AIS Asia to understand how to fix this problem for Japan. If you have any thoughts/feedback, feel free to reach out to me—adniajay@gmail.com

Why is AI safety field strategy important for Asian Middle Powers ?

The discourse on AI safety debates and concerns among those who deal with policymakers within Asia (organizations like Safe AI Philippines, SASH, AI4PH etc.) has been that most governments, however reluctant to act and take a non-partisan stance on the US-China AI Arms race, agree on the following -

The leverage of compute is non-existent. Singapore knows it. Vietnam knows it. So does Indonesia, the Philippines, India, South Korea, and possibly Japan.

As a result, most SEA countries have shifted their R&D efforts to AI verification and accreditation of deployed AI models in places where deployed domains are well-regulated. Build good AI testers, verify their accuracy, and deploy them in industries where frontier model adoption is rapid. A very good example of this pivot is Singapore’s AI Verify Foundation, a non-profit subsidiary of the Infocomm Media Development Authority (IMDA).

Multi-agent systems’ and Open-weight models’ risk capabilities are catching up to frontier models. Organizations within the ASEAN countries are no longer waiting for their countries to develop and deploy a weaker version of a frontier base model. They are rapidly adopting frontier models in their workflows. The more adoption increases without immediate understanding of risks and AI control techniques for risk containment, the more susceptible these organizations are to model failure modes.

In the ideal world, such organizations would limit their adoption to a specific version of a frontier model in order to absorb certain gains in workforce productivity, giving time to their local governments to build tools around control and risk containment, but no. Southeast Asia’s AI adoption curve is steepening, but the governance framework is barely matching the pace !

Ambitious research agendas in technical AI safety are barely pragmatic. Understanding model internals and model behavior through toolkits of model forensics is an ambitious goal of technical AI safety. But, in Google DeepMind’s own admission, is barely pragmatic. Most technical AI safety techniques don’t scale with current models. Frontier models are becoming increasingly closed-source. We are nowhere near to building a bare-bones theory for model behavior (even for a smaller, open-weight model). Google DeepMind’s pivot to pragmatic vision for interpretability and ARC’s own work on AlgoZoo are just some examples of it.

I don’t imply that the field of technical AI safety research is not relevant. It is extremely important. At the same time, I do highlight that the derivative elements of a good governance framework would come from research progress in technical AI governance and/or AI control, and maybe less from technical AI safety. Concepts regarding what is inference-time verification would matter more than sparse autoencoders or linear probes at the governance level.

The compounding effects of brain drain. Asian talent, especially in critical fields like AI, nuclear sciences, biosciences, etc., has always been vulnerable to a massive brain drain problem. To some extent, Asian countries have managed to reach an unhealthy equilibrium with this problem—by either investing heavily in R&D and creating incentives or just create more talent. However, the problem with AI is unique, that is—in both fronts there is no Asian middle power that can out-compete the salaries offered by US or China^[2] for technical staff / AI research engineer / research scientists / MTS etc. and invest more than what US and China is already investing R&D initiatives towards frontier model development, safety, control and risk assessment.

The writing on the wall is clear. The chips to leverage any bargain or favorable treatment for compute, and/or inference usage for Mythos-like models (or more recent, similarly capable frontier models) to develop enough safeguards for Asian organizations to be robust against model harm is diminishing. The talent is migrating to US for better salaries. Technical AI safety mega-agendas, once touted as the Hail Mary for model behavior are failing and major research organizations are switching ships from safety to governance. The steps are historically the same, only the strategy is need. Trust, but verify.^[3]

Do Asian Middle-Powers take AI safety field-strategy seriously ?

To some extent, yes.

Since the last year itself, a lot has changed and a lot hasn’t. For sake of brevity, I will highlight the three most major changes within the Asian AI safety policy diaspora which I find most pivotal and come closest to what I mean is taking field strategy seriously.

Singapore’s pivot to inference-verification, open-weight models & SASH − 2026 was eventful for Singapore. It hosted AAAI, and also announced and followed-up on the release of its Singapore Consensus on Global AI Safety Research Priorities document that highlighted the research priorities for Singapore and how they’ve changed since they were first released last year in April itself ! A strategic, timely pivot. Not to mention, expansion of SASH from a local hub to a global hub recruiting experts in AI agents, AI verification and frontier model evaluation (interesting all domains highlighted in Singapore’s Consensus document). This was followed by the development of SASH’s prototype on AI verification (also shown below), and the independent auditing report on DeepSeek v4 Pro published by Neo Research (a project incubated within SASH) to gain international attention, and even get featured in Chinese media !
Philippines, SAIPH, AI4PH—The amount of work done by independent researchers in Philippines, both as a part of either organizations (SAIPH and AI4PH), in the field of technical AI governance is nothing but a feat to marvel out. To list a few^[4] -
1. SAIPH was officially invited as a resource speaker in Philippine House of Representatives’ Technical Working Group (TWG) on the AI Act last July 13, 2026. As part of this TWG, SAIPH proposed establishing an AI Resilience Institute to handle technical auditing and standards-setting, and to redirect the proposed Philippine AI Commission to prioritize AI diplomacy and include whistleblower protections for employees of AI-deploying companies
2. In the past technical working group deliberations for the AI Development and Regulation Act, three SAIPH members were acknowledged as domain experts and invited as resource speakers.
3. Lenz Dagohoy, who represented AI4PH, a local policy advisory group working on AI governance, critical infrastructure, and workforce transition reforms in the Philippine AI economy, was one of the domain experts for the above working group
4. Yanro Ferrer, alumnus of their first AI Governance Accelerator, through his work in the House of Representatives as a Legislative Researcher, introduced the term “catastrophic risks of AI” for the first time in Congressional record in HR 971, which sought to establish AI safety standards and mechanisms for risk assessment and oversight.

Representatives from SAIPH at the AI Regulation Act TWG Group Deliberation

SASH’s Inference Verification Prototype highlighted in WAIC2026

The efforts of SASH in Singapore and AI4PH and SAIPH in Philippines are not one-sided. They are being received positively by their governments, and helping shape a more pragmatic field strategy for both the countries. I invite the reader to read more about the work done by these organizations in detail on the following links^[5]^[6]

Therefore, everything is good. But, where is Japan ?

Nowhere close.

This is the unfortunate part. To address this, I would briefly mention what I understood from the countless calls and interactions with people in this domain. Field strategy may not be the universal word that translates to each countries’ intended goal, however their actions are converging towards the same—build a niche within R&D that uniquely ties in with the growing concerns of rapid adoption of frontier models within their workforce without proper governance framework. Often and quite unsurprisingly, these niches are derived from existing problems in the field of technical AI governance. So, what is essentially required for effective field strategy (or at least what’s common within all of them) ?

Read the writing on the wall. The skill of learning the context and then observing the sub-text in research trends is very important for field strategizing. There is a reason why MIRI went all gung-ho in technical AI governance. There is a reason why PauseAI volunteers are protesting against frontier model companies. There is a reason why RAND’s assessment of China’s response to frontier model threats is positive. The bipolar powers are recognizing the threat and acting accordingly in their self-interest whilst trying their best to prevent power convergence in the opposite hand’s. We are getting closer to AI2027 than to AI2040 Plan A, but that doesn’t mean Plan A is not far. If the Asian capitals remain passive any longer, then they risk losing any relevance and hence, any leverage to bargain any position in their interest in a world where the AI arms race is approaching the endgame.
Strategy is not just a document. An effective field strategy needs international communication through conferences and must be defended by R&D work. It is not enough to release a risk taxonomy on your website meanwhile bureaucratic redtapeism cuts through any meaningful efforts to address those risks. Taxonomies need to be dynamic and be updated in response to trends, not incidents. Singapore realized the trends of open-weight models and need to scale AI verification research. It acted accordingly. So did Philippines. Today, SASH’s AI verification prototype is making rounds in WAIC 2026 and harnessing interest in the AI assurance communities in Oxford. Fiel Aquino, the Growth Director of SAIPH, joined the inaugural UN Global Dialogue on AI Governance, the World Summit on the Information Society Forum, AI for Good Global Summit, and Partnership on AI’s Partner Forum, highlighting the concerns of the Philippines’ lawmakers.
Pivot, pivot, pivot. Re-iterating my first point—a late pivot is still useful than no pivot. The Global AI safety Priorities document came out only in April 2025. No sooner, Singapore’s Minister for Digital Development and Information Josephine Teo says at the International Scientific Exchange on AI Safety -

“We want deployers to go for proper testing, but they naturally ask the question: who can I trust as a tester?”

Singapore’s accurate pivot to AI verification is a testament of its ability to read research trends carefully. Did it pay off ? Singapore released its own consensus on the global priorities in start of July 2026. Towards the end of the same month, FAR.AI establishes first international office in Singapore. Causal or not, this was a major win for Singapore.

Debate your peers and verify your priors. Countries with an AISI in Asia are slowly and steadily getting a clarity on their strategy well-enough to engage in public debates with their UK AISI and US CAISI peers. This is important and well-needed. A good strategy is as good as it is communicated. This addresses effectively talent attraction problem. If talented people in the field don’t know what are Japan’s concerns and what Japan is working on currently, then they would never have the incentive to migrate to Japan to help the country address those concerns—no matter the capital and incentive. Salary is lucrative, cost of living is also low—but is there a clear, incentivized career trajectory for a talented RS/RE for him/her to leave their job and move to Japan ? No. AI verification is an ambitious field, so Singapore retains at least some talent. Technical AI governance is an ambitious field, so Philippines retain some talent. Are the problems that interest Japan AISI or METI ambitious ? Who knows—since there are no members of Japan AISI in panels of FAR.AI events or WAIC ? (This is also the prior I urge the authors of the LW post to figure out before their funding ask—what field strategy are unique to Japan and it is most-well suited to address ?)

When the representative from Korean AISI in the FAR.AI panel on “Is Global AI safety converging ?” mentions that -

“Let me mention a distinct part of our (Korean AISI) work. Korean Govt. is interested in investigating sovereign AI, including a national competition program where Korean teams develop foundation models. We, the Korean AISI, evaluate every one of these models on performance and safety—every 6 months. I think very few safety institutes in the world run consecutive evaluations on a fixed cycle like this. ”

This creates the highlight—the leverage to attract talent, the ability to strategize effectively and to showcase the country is still actively pursuing dynamic governance solution are rapid pace.

What are the lessons for Japan from this ?

Japan must understand that it already has a good starting point. It is a country that has seen both the rise and fall of major technological revolutions. It knows how to distinguish potential from bloat. It also needs to understand the time to take severe caution is now over. The era for static, reactionary governance that follows incidents is over. Japan AISI, METI and Cabinet Office need to seriously ask themselves and consider -

Are they reading the trends in research progress and/or letting experts in the field guide them on mapping the state of the research field ?
Are they incentivizing stagnancy by keeping their research agenda and/or risk profiles extremely rigid ? Are their rigid priors based on evidence or excessive caution ?
Are they focusing on how their Asian peers are outcompeting them, even in governance frameworks for model adoption ?
Are they ready to debate their peers in UK and US or are they willing to continue to be at their diplomatic and technological behest in future as the US-China AI arms races reaches its final few moves ?

Japan needs to build its basics fast. Learn the required context to understand the terms effectively. Most importantly, let the experts do their job and transfer the technical context. Learn from others. Rationalize what niche should they own and strategize accordingly. Collaborate with other independent research organizations within Asia and abroad on broader, research questions.

Summary

If there is one thing I want the reader from this post is this—Recently, many AI safety organizations in Asia have started to recognize that the problem of AI safety in Asia is a capital allocation problem. Even, the LW post argues for this in case of Japan. This is partially true. Indeed, it is a capital allocation problem—but for countries that already have definite, clear field strategy. For countries without them, capital allocation won’t change anything at all ! Field strategy comes first, capital allocation follows later !

I am hopeful for Japan’s role in the broader Asian context of AI safety. The Japanese society has both interest and concern regarding the events surrounding the rapid adoption of frontier models and their increasing risk capabilities. I wish to share the same hope with the reader and ask to have a look on the below image—which is the front page of the major newspaper a day after the OpenAI-HuggingFace incident. This is new, bold and informative—to not just cover a rather technical and nuanced event, but also cite resources like Plan A in context of the incident. This means—people know their basics, and they know what they want. Does the Cabinet Office, METI, Japan AISI etc. know what they actually want ?

The frontpage of a major Japanese public newspaper mentioning—Plan D

The same newspaper mentioning the progression of Plan A and Plan D from AI2040 draft.

^
Since I am currently heavily embedded in the Japan AI safety ecosphere, the post is biased towards the question of Japan. This doesn’t mean that there aren’t other Asian countries which do not have a similar issue. One key example would be—India.
^
I am unaware of details for the median salary offered to Research Engineer FTE or similar level position in Chinese deep-tech and/or frontier AI startups. This assumption, regardless, won’t change the conclusion of the argument.
^
The famous quote by Ronald Reagan applies very broadly to what the context here is. Trust, but verify” (Doveryai, no proveryai) is a traditional Russian proverb that means you should maintain good faith while independently confirming the facts.
^
I implore the reader to learn more about the fascinating work done by SAIPH within such a short period of time on their Substack
^
https://safeaiph.substack.com/
^
https://www.aisafety.sg/blog

Apollo Research is hiring across Watcher product engineering, security & control research, and internal security team. by Joping

Joping — Wed, 02 Sep 2026 14:17:20 +0000

Watcher Product

We’re growing the Watcher team. Watcher is now deployed across many organizations and we’re seeing a lot of traction and inbound interest.

Many people are starting to feel how misaligned agents start affecting their organizations and Watcher is one of the few AI security products that is trying to tackle immediate and future misalignment failures like deception, overclaiming, oversight subversion or scope overreach in addition to security concerns.

- Engineering Manager (Product): https://jobs.lever.co/apolloresearch/0e5d1f41-850d-4675-b212-9b6261448667

- Full-Stack Engineer (Product): https://jobs.lever.co/apolloresearch/ab3051f4-d5be-4b46-9f54-eb14da1a6a54

- Forward Deployed Engineer: https://jobs.lever.co/apolloresearch/2237ce81-cb8e-4f53-a047-e12ea5d6e7d8

- Product Security Engineer: https://jobs.lever.co/apolloresearch/c2ee47f7-4fe9-41a7-86fd-ce0a23fe4850

Watcher: https://watcher.apolloresearch.ai/

Research & red-teaming

We’re doubling down on our control and monitoring research. We have recently successfully red-teamed Anthropic’s auto-mode and will run more such campaigns in the future with multiple labs. We will also continue to make better monitors ourselves and improve the research frontier for the blue team.

There are many low hanging fruits to pick for monitoring and not a lot of time. Our main blocker is having more hands on deck.

RS (Control): https://jobs.lever.co/apolloresearch/24d016aa-5a19-4976-8936-af2fbf748000
AI Red Team Engineer: https://jobs.lever.co/apolloresearch/459b83bf-4e7d-41c5-b93e-c589c6b71bfe
AI Security & Control Researcher: https://jobs.lever.co/apolloresearch/d45269ea-3390-450b-bdda-c495d6123c66

Our monitoring agenda: https://www.apolloresearch.ai/monitoring/a-scalable-monitoring-research-agenda
Auto-mode red-teaming: https://www.apolloresearch.ai/monitoring/pilot-automode-campaign

Internal Infra & Security

As should be very clear from the last months, security is becoming even more important in the AI age. We’re looking to grow our security team to defend against external and internal, human and non-human adversaries.

AI Security Researcher: https://jobs.lever.co/apolloresearch/8c19d363-a4b1-4207-b5de-24d2d2267c64
Security Engineer: https://jobs.lever.co/apolloresearch/855f584d-08dd-4845-93b8-dbeb6737707d

All open roles: https://jobs.lever.co/apolloresearch

Every role is open in both offices, London and San Francisco. We provide visa sponsorship. A formal AI safety background is not required.

Invitation to EA Forum Members: An Evening with The Humane League UK by Molly Archer-Zeff

Molly Archer-Zeff — Wed, 02 Sep 2026 11:44:48 +0000

On Thursday 29th October, from 7 pm to 10 pm, The Humane League UK is hosting an in-person event in London (the venue is a 5-minute walk from Farringdon station). We would like to invite members of the EA Forum who are interested in learning more about our work for farmed animals. This year’s gathering is particularly meaningful as we celebrate 10 years of THL UK.

The evening will be an opportunity to converse with other guests, hear from our new Managing Director and staff about our work, and ask questions. We’ll provide canapés and soft drinks, along with the opportunity to meet a community of people who are committed to creating meaningful change for animals.

We hope you’ll join us for an enjoyable evening and a chance to learn more about one of the world’s most significant yet overlooked welfare issues.

The evening will follow this provisional agenda:

7:00- 7:50pm: Canapés and conversation with THL UK staff, Trustees, and fellow guests
7:50–8:10pm: An exclusive presentation from THL UK staff about our upcoming plans and new strategic direction for broiler chickens
8:10–8:30pm: Q&A- an opportunity to ask questions about our work and impact for animals
8:30–10:00pm: Networking and conversation with our supporters, staff, and Trustees

The event is free to attend. We have limited spaces, so please email me at marcherzeff@thehumaneleague.org.uk or send me a DM via the Forum if you’d like to join us for the evening.

How much “benefit” does a pause/pace cost us? by Charlie_Guthmann

Charlie_Guthmann — Tue, 01 Sep 2026 23:22:55 +0000

It seems like an important question that I haven’t seen great analysis on is: if we stop or heavily pace frontier training runs and development, how much of the benefits (measured either in economics/welfare or straight capabilities) will we lose per unit time.

This is sort of vague and I would also like some help operationalizing this question. What I’m imagining is approximately as follows.

- We open source fable/astra weights (or imagine in a year some open source model gets to that level)
- You can still finetune the models (probably misusing this but let’s say a “team” can use 1/100th or some other factor of the compute used to train them to update the weights per year)
- You can still update harnesses
- You can still update APIs
- You can still build new company structures or workflows
- You can still optimize caching, datacenter compute use, etc.

How much would this
- Lower gdp growth
- Lower capabilities growth

over (1,5,n) years.

Frontpage posts - Effective Altruism forum viewer

How might robotics capabilities research affect AGI timelines? by themasterchief166

Work at Manifund or Mox by Carol N

What is Man­i­fund?

What is Mox?

Why work at Man­i­fund or Mox?

Who are we look­ing for?

Logistics

The Odyssean Process on AI: Progress & the Future by Odyssean Institute

We know who lives inside wild fish. We don’t know what happens in there by Pablo Sar

Introduction

Weishan: even the cen­sus fails

Ligula: pres­ence alone is not enough

Anisak­ids: par­a­sites we study for an­other reason

What if the par­a­sites suffer too?

How many in­di­vi­d­u­als does this hap­pen to?

The num­bers do not de­cide on their own

My bet

Effective Altruism and the Problem of Christianity by Carlo Rimbotti

AI Uplift for Careers (Free 80,000 Hours Career advising without an application) by JP Addison🔸

In­tro­duc­ing the 80,000 Hours AI advisor

The Young Lady’s Illus­trated Primer

FAQs

Why not use [my fa­vorite way to talk to a generic AI]?

Will it push me to­wards AI safety?

How’s it made?

Feed­back, etc.

Alone again: facing the possibility of AI takeover as the world sleepwalks on by George Rosenfeld

How Fast Is Post-AGI Growth? by Vasco Grilo🔸

Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) introduce legislation to ban Artificial Superintelligence and temporarily pause advanced AI development by Matrice Jacobine🔸🏳️‍⚧️

Help Fund Philosium Tokyo 9/​19/​2026 by Fumi K

The Mighty Denominator by Charles Berry

The Case Against Acting Professional When Starting an EA Group by micahzarin

Kairos has raised $50M to build talent infrastructure for AI safety (and we’re hiring!) by Agustín Covarrubias 🔸

Two years in

How the field has changed

Our new bets

We’re hiring (a lot!)

An AI Safety Career Switch Postmortem by AlexChalk

Introduction

Background

Some thoughts on what went wrong

Some things I still stand by

Con­clud­ing thoughts

CMIP7 emissions scenarios out for IPCC AR7 by MatthewDahlhausen

Exploring Big Goal Projects by Andrew Youn

AI Safety in Japan has deeper problems than capital allocation by Ajay Agarwal

brief back­ground and tl;dr

Why is AI safety field strat­egy im­por­tant for Asian Mid­dle Pow­ers ?

Do Asian Mid­dle-Pow­ers take AI safety field-strat­egy se­ri­ously ?

There­fore, ev­ery­thing is good. But, where is Ja­pan ?

What are the les­sons for Ja­pan from this ?

Summary

Apollo Research is hiring across Watcher product engineering, security & control research, and internal security team. by Joping

Watcher Product

Re­search & red-teaming

In­ter­nal In­fra & Security

Invitation to EA Forum Members: An Evening with The Humane League UK by Molly Archer-Zeff

How much “benefit” does a pause/​pace cost us? by Charlie_Guthmann

What is Manifund?

Why work at Manifund or Mox?

Who are we looking for?

Weishan: even the census fails

Ligula: presence alone is not enough

Anisakids: parasites we study for another reason

What if the parasites suffer too?

How many individuals does this happen to?

The numbers do not decide on their own

Introducing the 80,000 Hours AI advisor

The Young Lady’s Illustrated Primer

Why not use [my favorite way to talk to a generic AI]?

Will it push me towards AI safety?

Feedback, etc.

Help Fund Philosium Tokyo 9/19/2026 by Fumi K

Concluding thoughts

brief background and tl;dr

Why is AI safety field strategy important for Asian Middle Powers ?

Do Asian Middle-Powers take AI safety field-strategy seriously ?

Therefore, everything is good. But, where is Japan ?

What are the lessons for Japan from this ?

Research & red-teaming

Internal Infra & Security

How much “benefit” does a pause/pace cost us? by Charlie_Guthmann