A personal take on why you should work at Forethought (maybe)

By Lizka @ 2025-10-14T08:59 (+61)

Basic facts:

In the rest of this post, I sketch out more of my personal take on this area, how Forethought fits in, and why you might or might not want to do this kind of work at Forethought.

Others at Forethought might disagree with various parts of what I say (and might fight me or add other info in the comments). Max reviewed a first draft and agreed that, at least at that point, I wasn’t misrepresenting the org, but that’s it for input from Forethought folks — I wrote this because I dislike posts written in an ~institutional voice and thought it could be helpful to show a more personal version of “what it’s like to work here.”[1]

A background worldview

Here’s roughly how I see our current situation:

Forethought’s mission is to help us “navigate the transition to a world with superintelligent AI systems.” The idea is to focus in particular on the questions/issues that get less attention from other people who think that advanced AI could be a big deal. So far this has involved publishing on topics like:

(If you want to see more, here’s the full “research” page, although a lot of my favorite public stuff from my colleagues is scattered in other places,[2] e.g. the podcast Fin runs, posts like this one from Rose, Tom’s responses on LW & Twitter, or Will’s recent post on EA. And the same goes for my own content; there's more on the EA Forum/LW & Twitter.)

So in my view Forethought is helping to fill a very important gap in the space.[3] But Forethought is pretty tiny,[4] and collectively my sense is we’re nowhere near on track for understanding things to a degree that would make me happy. I think we don’t even have a bunch of the relevant questions/issues/dynamics on our radar at this point.

Quick sketch illustrating this perspective

(Things that give me this sense include: often running into pretty fundamental but not well articulated disagreements with "state-of-the-art" research in this space or with others at Forethought, often feeling like everyone seems to be rolling with some assumption or perspective that doesn’t seem justified or feeling like some proposal rests on a mishmash of conceptual models that don’t actually fit together.)

So I would love to see more people get involved in this space, one way or another.[5] And for some of those people, I think Forethought could be one of the best places to make progress on these questions.[6]

Why do this kind of work at Forethought? (Why not?)

I joined (proto-)Forethought about a year ago. This section outlines my takes on why someone interested in this area might or might not want to join Forethought; if you want the “official” version, you should go to the main job listing, and there’s also some relevant stuff on the more generic “Careers” page.

Doing this kind of research alone — or without being surrounded by other people thinking seriously about related topics — seems really hard for most people. Being able to develop (butterfly) ideas in discussion with others, quickly get high-context feedback on your drafts or input on possible research directions,[7] and spend time with people you respect (from whom you can learn various skills[8]) helps a lot. It’s also valuable to have a space in which your thinking is protected from various distortion/distraction forces, like the pressure to signal allegiance to your in-group (or to distance yourself from perspectives that are “too weird”), the pull of timely, bikeshed-y topics[9] (or the urge to focus on topics that will get you lots of karma), the need to satisfy your stakeholders or to get funding or to immediately demonstrate measurable results, and so on. It’s also easier to stay motivated when surrounded by people who have context on your work.

And joining a team can create a critical mass around your area. Other people start visiting your org (and engaging with the ideas, sharing their own views or expertise, etc.). It’s easier for people to remember at all that this area exists. Etc.

I think the above is basically the point of Forethought-the-institution. 

Many of the same properties have, I think, helped me develop as a researcher. For instance, I’ve learned a lot by collaborating with Owen, talking to people in Forethought’s orbit, and getting feedback in seminars. The standards of seriousness[10] set by the people around me have helped train me out of shallow or overly timid engagement with these ideas. And the mix of opportunity-to-upskill, having a large surface area / many opportunities to encounter a bunch of different people, and protection from various gravitational forces (and in particular the pull to build on top of AI strategy worldviews that I don’t fully buy or understand) has helped me form better and more independent models.[11]

(I also think Forethought could improve on various fronts here. See more on that a few paragraphs down.)

Working at Forethought has some other benefits:

Sample of crazy/messy diagrams I made at various points during my time at Forethought.

(The job listing shares more info on stuff like salary, the office[14], the support you’d get on things like making sure what you write actually reaches relevant people or turns into “action”, etc.)

Still, Forethought is not the right place for everyone, and there are things I’d tell e.g. a friend to make sure they’re ok with before they decide to join. These might include:

I'll also list some things that I personally wish were different:

Who do I think would/wouldn’t be a good fit for this?

I’ve pasted in the official "fit" descriptions below (mostly because I don’t really trust people to just go read them on the job listing). 

The main things I want to emphasize or add are:

And as promised here’s the official description of the roles:

Senior research fellows will lead their own research projects and set their direction, typically in areas that are poorly understood and pre-paradigmatic. They might also lead a small team of researchers working on these topics.

 

You could be a good fit for this role if:

  • You care about working on important questions related to navigating rapid AI progress
  • You have your own views on what is most important and ideas for research directions to pursue.
  • You have a strong track record of producing original research, particularly in poorly understood, interdisciplinary domains
  • You can work autonomously and are able to make consistent progress on gnarly problems without much guidance from more established researchers (since there often aren’t people who have thought more deeply about the questions you’d be tackling)
  • You can communicate clearly in writing (and verbally)

Research fellows are in the process of developing their own independent views and research directions, since they might be earlier-career or switching domains.

Initially, [r]esearch fellows will generally collaborate with senior research fellows, to produce research on important topics of mutual interest. We expect research fellows to form their own view on topics that they work on, and to spend 20-50% of their time thinking and exploring their own research directions.

We expect some research fellows to (potentially rapidly) develop their own research agenda and begin to set their own direction, while others may continue to play a more collaborative role within the team. Both are very valuable.

 

You could be a good fit for this role if:

  • You care about working on important questions related to navigating rapid AI progress.
  • You are developing your own views on what is most important and ideas for research directions to pursue.
  • You have a track record of getting obsessed with a project, committing to it, and performing well. This will most likely be in research or academia, but could also be in other domains.
  • You are productive, autonomous, and strong at quantitative and conceptual reasoning.
  • You can communicate clearly in writing (and verbally).

More basic info

The location, salary, benefits, etc, are included in the job listing (see also the careers page). If you have any questions, you could comment below and I’ll try to pull in relevant people (or they might just respond), or you might want to reach out directly.

There’s also the referral bonus; Forethought is offering a £10,000 referral bonus for “counterfactual recommendations for successful Senior Research Fellow hires” (or £5,000 for Research Fellows. Here’s the form.

In any case: 

The application looks pretty short. Consider following the classic advice and just applying.

A final note

One last thing I want to say here — which you can take as a source of bias and/or as evidence about Forethought being a nice place (at least for people like me) — is that I just really enjoy spending time with the people I work with. I love the random social chats about stuff like early modern money-lending schemes[21] or cat-naming or the jaggedness of child language learning. People have been very supportive when personal life stuff got difficult for me, or when I’ve felt especially impostory. Some of my meetings happen during walks along a river, and at other times there’s home-made cake. And we have fun with whiteboards:

 

  1. ^

     More specifics on the context here: Max asked me to help him draft a note about Forethought’s open roles for the EA Forum/LessWrong; I said it’d be idiosyncratic & would include critical stuff if I did that; he encouraged me to go for it and left some comments on the first partial draft (and confirmed what I was saying made sense); I wrote the next draft without input from Max or others at Forethought; made some edits after comments from Owen (hijacked some of our coworking time); and here we are.

  2. ^

     I’ve mentioned that I think more of this (often more informal) content should go on our website (or at least the Substack); I think others at Forethought disagree with me, although we haven’t really invested in resolving this question.

  3. ^

     (which is one of the main reasons I work here!)

  4. ^

     Forethought has 9 people, of whom 6 are researchers. If I had to come up with some estimate of how many people overall (not just at FOrethought) are devoting a significant amount of attention to this kind of work, I might go with a number like 50. Of course, because so much of this work is interdisciplinary and preparadigmatic, there’s no shared language/context and I’m quite likely to be missing people (and per my own post, it’s pretty easy to get a skewed sense of how neglected some area is). (OTOH I also think the disjointedness of the field hurts our ability to collectively understand this space.)

    Overall, I don’t feel reassured that the rest of the world “has it covered”. And at least in the broader community around EA / existential risk, I’m pretty confident that few people are devoting any real attention to this area.

  5. ^

     While I’m at it: you might also be interested in applying for one of the open roles at ACS Research (topics listed are Gradual Disempowerment, AI/LLM psychology/sociology, agent foundations)

  6. ^

     I wanted to quickly list some examples somewhere, and this seemed like a fine place to do that. So here are sketches of some threads that come to mind:

    (I’m not trying to be exhaustive — basically just trying to quickly share a sample — and this is obviously filtered through my interests & taste [a])

    -- How can we get our civilization to be in a good position by the time various critical choices are being made? Should we be working towards some kind of collective deliberation process, and what would that look like? Are there stable waypoint-worlds that we'd be excited to work towards? [Assorted references: paretotopia, long reflection stuff, various things on AI for epistemics & coordination, e.g. here]

    -- What might ~distributed (or otherwise not-classical-agent-shaped) powerful AI systems look like? How does this map on to / interact with the rest of the threat/strategic landscape? (Or: which kinds of systems should we expect to see by default?) [See e.g. writing on hierarchical/scale-free agency]

    -- Is it in fact the case that states will stop protecting their citizens’ interests as (or if) automation means they’re no longer “incentivized” to invest in a happy etc. labor force? (And what can/should we do if so?) [Related]

    -- How should we plan for / think about worlds with digital minds that might deserve moral consideration? When might we need to make certain key decisions? Will we be able to find sources of signal about what is good for the relevant systems that we trust are connected to the thing that matters? (And also stuff like: what might this issue do to our political landscape?) [See e.g. this, this, and this]

    -- How would AI-enabled coups actually play out? (What about things that look less like coups?) [ref]

    -- Which specific “pre-ASI” technologies might be a big deal, in which worlds, how, ...?

    -- More on how coordination tech (e.g. structured transparency stuff, credible commitments) could go really wrong, which technologies might be especially risky here, etc.

    -- What might it look like for institutions/systems/entities that are vastly more powerful than us to actually interact with us in healthy (virtuous?) ways?

    -- If AI systems get deeply integrated into the market etc., what would that actually look like, how would that play out? [E.g. more stuff like frictionless bargaining, or how things could go unevenly, or cascading risk stuff, perhaps.]

    [a] If I try to channel others at Forethought, other things probably become more salient. E.g. how acausal trade stuff might matter, more on modeling dynamics related to automation or the intelligence explosion, exploring how we might in fact try to speed up good automation of moral philosophy, more on governance of space resources, more on safeguarding some core of liberal democracy, etc.

     

  7. ^

    Sometimes there might be too many comments, I suppose:

  8. ^

     One important dynamic here is picking up “tacit” skills, like underlying thinking/reasoning patterns. As a quick example, I’ve occasionally found myself copying a mental/collaborative move (often subconsciously) that I’d appreciated before.

  9. ^

     Part of my model here is that my brain is often following the “easiest” salient paths, and that reasoning about stuff like radically new kinds of technology, what the situation might be like for animals in totally transformed worlds, what-the-hell-is-agency-and-do-we-care, etc. is hard. So if I don’t immerse myself in an environment in which those kinds of questions are default, my focus will slip away towards simpler or familiar topics.

  10. ^

    Or maybe “realness”? (Tbh I’ll take any opportunity to link this post)

  11. ^

    As an aside: I’d really like to see more people try to form their own ~worldviews, particularly trying to make them more coherent/holistic. Because the space is extremely raw and developing pretty quickly (so basically no one has the time or conceptual tools to fit all the parts together on their own), I think large chunks of the work here rest on the same shaky foundations, which I want to see tested / corrected / supported with others. I also think this is good for actually noticing the gaps.

  12. ^

     I’m already on the record as an appreciator of good management, so it might not be too surprising that I’m really grateful for the help I get from my Max. But I really think this is pretty crucial, and often overlooked (maybe especially in research contexts), so I’m still emphasizing it.

  13. ^

     a call to adventure ? (another post I love linking to)

  14. ^

     which tbh might be underemphasized there; the opportunity to work in person from a nice office is a game-changer for some

  15. ^

     In the past I’ve been overly concerned about what might not be in scope. Because hesitation around this felt distracting to me, I’ve got an agreement with Max right now that I’ll just focus on whatever seems important and he can flag things to me if he notices that I’m going too far out / account for this retrospectively. (So far I haven't hit that limit.)

  16. ^

    (This was called “Explorethought”. There’s an abundance of puns at Forethought.; do with that information what you will.)

  17. ^

     I like this aspect a lot, fwiw. (Especially since it doesn’t end up feeling that people are out to “win” arguments or put down others’ views.)

    (Once in a while I get reminders of how weird Forethought’s culture can seem from the outside, e.g. when I remember that many (most?) people would hesitate to say they strongly disagree with a more senior researcher's doc. Meanwhile I’ve been leaving comments like “This breakdown feels pretty fake to me...” or “I think this whole proposal could only really work in some extreme/paradigmatic scenarios, and in those worlds it feels like other parts of the setup wouldn’t hold...”)

  18. ^

    As one example, near the start of my tenure at Forethought, I ended up spending a while on a project that I now think was pretty misguided (an ITN BOTEC comparing this area of work with technical AI safety), I think partly because I hadn’t properly synced up with someone I was working with.

    (Although the experience itself may have been useful for me, and it’s one of the things that fed into this post on pitfalls in ITN BOTECs.)

  19. ^

    If interested see some notes on this here

  20. ^

     Related themes here, IIRC: Fixation and denial (Meaningness) 

  21. ^

     Although I failed to explain it to some other people the other day, so I need some rescuing here

MaxDalton @ 2025-10-15T09:32 (+18)

Thanks for writing this up Lizka! I agree with most of it, including most of the ways I'd like us to be different.

Flagging some disagreements:

 - I feel like "smooth working together on projects" is pretty good on average: I agree with the example you give as a mostly-failed collab, though I think it was ultimately OK in that it resulted in your ITN post. But I think that there have been a lot of productive co-authored papers (I think most of our pieces are co-authored, and most of those collabs gave gone well.)

 - I also feel like "give each other lots of feedback" is generally pretty good: weekly pretty detailed feedback between managers, weekly commenting on work-in-progress drafts, 360 feedback every retreat. I think we should add in a monthly 360 though, I've been meaning to do that for a while. In fact, I'd maybe emphasize that if people don't want a fair amount of feedback, it might not be a great fit.

+1 to your other criticisms though, and I hope that this hiring round can help us bring in even more perspectives!

Other things I'd emphasize:

Impact focus: We have a culture of discussing research prioritization, and we all care about our work having impact.  I think this is contestable and could be bad for some people:  if you think you do your best work in a purely curiosity-driven way, Forethought might not be ideal.

Also, Stefan Torges recently joined to help make sure that research papers lead to action, e.g. by engaging with AI companies to make sure they introduce mitigations, and developing policy ideas. I hope that this makes it easier for researchers to focus on research, while also making sure that our research leads to some real difference. (To be clear, we're early-days figuring this out, and we might not be able to find a way to get this to work.)

+1 to just really enjoying working with this set of people, too!

Also, Fin and Mia just put out a podcast about Mia's take on what it's like to work at Forethought, in case people want another personal perspective!

Mia Taylor @ 2025-10-15T19:00 (+14)

[Cross-posted from LW]

I joined Forethought about six weeks ago as a research fellow. Here's a list of some stuff I like about Forethought and some stuff that I don't like about Forethought. Note that I wrote this before reading OP closely.

 

Things I like:

 

Things I don't like:

cb @ 2025-10-14T14:28 (+11)

(Meta: I really liked this more personal, idiosyncratic kind of "apply here" post!)

Could you say a bit more about the "set of worldviews & perspectives" represented on Forethought, and in which ways you'd like it to be broader?

Lizka @ 2025-10-14T23:07 (+9)

ore Quick sketch of what I mean (and again I think others at Forethought may disagree with me):

  • I think most of the work that gets done at Forethought builds primarily on top of conceptual models that are at least in significant part ~deferring to a fairly narrow cluster of AI worldviews/paradigms (maybe roughly in the direction of what Joe Carlsmith/Buck/Ryan have written about)
    • (To be clear, I think this probably doesn't cover everyone, and even when it does, there's also work that does this more/less, and some explicit poking at these worldviews, etc.)
    • So in general I think I'd prefer the AI worldviews/deferral to be less ~correlated. On the deferral point — I'm a bit worried that the resulting aggregate models don't hold together well sometimes, e.g. because it's hard to realize you're not actually on board with some further-in-the-background assumptions being made for the conceptual models you want to pull in.
    • And I'd also just like to see more work based on other worldviews, even just ones that complicate the paradigmatic scenarios / try to unpack the abstractions to see if some of the stuff that's been simplified away is blinding us to important complications (or possibilities).[1]
  • I think people at Forethought often have some underlying ~welfarist frames & intuitions — and maybe more generally a tendency to model a bunch of things via something like utility functions — as opposed to  thinking via frames more towards the virtue/rights/ethical-relationships/patterns/... direction (or in terms of e.g. complex systems)
    • Quick links illustrating some stuff on what I'm trying to gesture at might include this & this
  • We're not doing e.g. formal empirical experiments, I think (although that could also be fine given this area of work, just listing as it pops into my head)
  • There's some fuzzy pattern in the core ways in which most(?) people at Forethought seem to naturally think, or in how they prefer to have research discussions, IMO? I notice that the types of conversations (and collaborations) I have with some other people go fairly differently, and this leads me to different places.
    • To roughly gesture at this, in my experience Forethought tends broadly more towards a mode like "read/think -> write a reasonably coherent doc -> get comments from various people -> write new docs / sometimes discuss in Slack or at whiteboards (but often with a particular topic in mind...)", I think? (Vs things like trying to map more stuff out/think through really fuzzy thoughts in active collaboration, having some long topical-but-not-too-structured conversations that end up somewhere unexpected, "let's try to make a bunch of predictions in a row to rapidly iterate on our models," etc.)
    • (But this probably varies a decent amount between people, and might be context-dependent, etc.)
  • IIRC many people have more expertise on stuff like analytic philosophy, maybe something like econ-style modeling, and the EA flavor of generalist research than e.g. ML/physics/cognitive science/whatever, or maybe hands-on policy/industry/tech work,  or e.g. history/culture... And there are various similarities in people's cultural/social backgrounds. (I don't actually remember everyone's backgrounds, though, and think it's easy to overindex on this / weigh it too heavily. But I'd be surprised if that doesn't affect things somewhat.)

I also want to caveat that:

  • (i) I'm not trying to be exhaustive here, just listing what's salient, and
  • (ii) in general it'll be harder for me to name things that also strongly describe me (although I'm trying, to some degree), especially as I'm just quickly listing stuff and not thinking too hard.

 

(And thanks for the nice meta note!)

  1. ^

    I've been struggling to articulate this well, but I've recently been feeling like, for instance, proposals on making deals with "early [potential] schemers" implicitly(?) rely on a bunch of assumptions about the anatomy of AI entities we'd get at relevant stages. 

    More generally I've been feeling pretty iffy about using game-theoretic reasoning about "AIs" (as in "they'll be incentivized to..." or similar) because I sort of expect it to fail in ways that are somewhat similar to what one gets if one tries to do this with states or large bureaucracies or something -- iirc the fourth paper here discussed this kind of thing, although in general there's a lot of content on this. Similar stuff on e.g. reasoning about the "goals" etc. of AI entities at different points in time without clarifying a bunch of background assumptions (related, iirc).

MaxDalton @ 2025-10-15T09:17 (+8)

Thanks for writing the post and this comment, Lizka!
 

~deferring to a fairly narrow cluster of AI worldviews/paradigms (maybe roughly in the direction of what Joe Carlsmith/Buck/Ryan have written about)

I agree that most of Forethought (apart from you!) have views that are somewhat similar to Joe/Buck/Ryan's, but I think that's mostly not via deferral?

+1 to wanting people who can explore other perspectives, like Gradual Disempowerment, coalitional agency, AI personas, etc. And the stuff that you've been exploring!

I also agree that there's some default more welfarist / consequentialist frame, though I think often we don't actually endorse this on reflection. Also agree that there's some shared thinking styles, though I think there's a bit more diversity in training (we have people who majored in history, CS, have done empirical ML work, etc).

Also maybe general note, that on many of the axes you're describing you are adding some of the diversity that you want, so Forethought-as-a-whole is a bit more diverse on these axes than Forethought-minus-Lizka.

OscarD🔸 @ 2025-10-15T00:31 (+6)

I think I agree maybe ~80%. My main reservation (although quite possibly we agree here) is that if Forethought hired e.g. the 'AI as a normal technology' people, or anyone with equivalently different baseline assumptions and ways of thinking to most of Forethought, I think that would be pretty frustrating and unproductive. (That said, I think brining people like that in for a week or so might be great, to drill down into cruxes and download each others' world models more.) I think there is something great about having lots of foundational things in common with the people you work closely with.

But I agree that having more people who share some basic prerequisites of thinking ASI is possible, likely to come this century, being somewhat longtermist and cosmopolitan and altruistic, etc, but disagree a lot on particular topics like AI timelines and threat models and research approaches and so forth can be pretty useful.

Lizka @ 2025-10-15T03:49 (+6)

Yeah, I guess I don't want to say that it'd be better if the team had people who are (already) strongly attached to various specific perspectives (like the "AI as a normal technology" worldview --- maybe especially that one?[1]). And I agree that having shared foundations is useful / constantly relitigating foundational issues would be frustrating. I also really do think the points I listed under "who I think would be a good fit" — willingness to try on and ditch conceptual models, high openness without losing track of taste, & flexibility — matter, and probably clash somewhat with central examples of "person attached to a specific perspective."

= rambly comment, written quickly, sorry! =

But in my opinion we should not just all (always) be going off of some central AI-safety-style worldviews. And I think that some of the divergence I would like to see more of could go pretty deep - e.g. possibly somewhere in the grey area between what you listed as "basic prerequisites" and "particular topics like AI timelines...". (As one example, I think accepting terminology or the way people in this space normally talk about stuff like "alignment" or "an AI" might basically bake in a bunch of assumptions that I would like Forethought's work to not always rely on.) 

One way to get closer to that might be to just defer less or more carefully, maybe. And another is to have a team that includes people who better understand rarer-in-this-space perspectives, which diverge earlier on (or people who are by default inclined to thinking about this stuff in ways that are different from others' defaults), as this could help us start noticing assumptions we didn't even realize we were making, translate between frames, etc.

So maybe my view is that (1) there were more ~independent worldview formation/ exploration going on, and that (2) the (soft) deferral that is happening (because some deferral feels basically inevitable) were less overlapping. 

(I expect we don't really disagree, but still hope this helps to clarify things. And also, people at Forethought might still disagree with me.)

  1. ^

    In particular:

    If this perspective involves a strong belief that AI will not change the world much, then IMO that's just one of the (few?) things that are ~fully out of scope for Forethought. I.e. my guess is that projects with that as a foundational assumption wouldn't really make much sense to do here. (Although IMO even if, say, I believed that this conclusion was likely right, I might nevertheless be a good fit for Forethought if I were willing to view my work as a bet on the worlds in which AI is transformative.)

    But I don't really remember what the "AI as normal.." position is, and could imagine that it's somewhat different — e.g. more in the direction of "automation is the wrong frame for understanding the most likely scenarios" / something like this. In that case my take would be that someone exploring this at Forethought could make sense (haven't thought about this one much), and generally being willing to consider this perspective at least seems good, but I'd still be less excited about people who'd come with the explicit goal of pursuing that worldview & no intention of updating or whatever.

    --

    (Obviously if the "AI will not be a big deal" view is correct, I'd want us to be able to come to that conclusion -- and change Forethught's mission or something. So I wouldn't e.g. avoid interacting with this view or its proponents, and agree that e.g. inviting people with this POV as visitors could be great.)

William_MacAskill @ 2025-10-16T18:26 (+4)

If this perspective involves a strong belief that AI will not change the world much, then IMO that's just one of the (few?) things that are ~fully out of scope for Forethought

 

I disagree with this. There would need to be some other reason for why they should work at Forethought rather than elsewhere, but there are plausible answers to that — e.g. they work on space governance, or they want to write up why they think AI won't change the world much and engage with the counterarguments. 

Tax Geek @ 2025-10-16T10:40 (+2)

On the "AI as normal technology" perspective - I don't think it involves a strong belief that AI won't change the world much. The authors restate their thesis in a later post:

There is a long causal chain between AI capability increases and societal impact. Benefits and risks are realized when AI is deployed, not when it is developed. This gives us (individuals, organizations, institutions, policymakers) many points of leverage for shaping those impacts. So we don’t have to fret as much about the speed of capability development; our efforts should focus more on the deployment stage both from the perspective of realizing AI’s benefits and responding to risks. All this is not just true of today’s AI, but even in the face of hypothetical developments such as self-improvement in AI capabilities. Many of the limits to the power of AI systems are (and should be) external to those systems, so that they cannot be overcome simply by having AI go off and improve its own technical design.

The idea of focusing more on the deployment stage seems pretty consistent with Will MacAskill's latest forum post about making the transition to a post-AGI society go well. There are other aspects of the "AI as normal technology" worldview that I expect will conflict more with Forethought's, but I'm not sure that conflict would necessarily be frustrating and unproductive - as you say, it might depend on the person's characteristics like openness and willingness to update, etc.

OscarD🔸 @ 2025-10-15T18:34 (+2)

Nice, yes I think we roughly agree! (Though maybe you are nobler than me in terms of finding a broader range of views provocatively plausible and productive to engage with.)

William_MacAskill @ 2025-10-16T18:23 (+5)

I can't speak to the "AI as a normal technology" people in particular, but a shortlist I created of people I'd be very excited about includes someone who just doesn't buy at all that AI will drive an intelligence explosion or explosive growth.

I think there are lots of types of people where it wouldn't be a great fit, though. E.g. continental philosophers; at least some of the "sociotechnical" AI folks; more mainstream academics who are focused on academic publishing. And if you're just focused on AI alignment, probably you'll get more at a different org than you would at Forethought. 

More generally, I'm particularly keen on situations where V(X, Forethought team) is much greater than than V(X) + V(Forethought team), either because there are synergies between X and the team, or because X is currently unable to do the most valuable work they could in any of the other jobs they could be in. 

rosehadshar @ 2025-10-15T12:38 (+10)

Thanks Lizka!

Some misc personal reflections:

One minor addition from me on why/not to work at Forethought: I think the people working at Forethought care pretty seriously about things going well, and are really trying to make a contribution.

I think this is both a really special strength, and something that has pitfalls:

And then a few notes on the sorts of people I'd be really excited to have apply:

William_MacAskill @ 2025-10-16T18:13 (+8)

Thanks for writing this, Lizka! 

Some misc comments from me:

finm @ 2025-10-15T17:46 (+8)

Thanks for writing this! Some personal thoughts:

Tom_Davidson @ 2025-10-16T17:45 (+4)

I also work at Forethought!

I agree with a lot of this, but wanted to flag that I would be very excited for ppl doing blue skies research to apply and want Forethought to be a place that's good for that. We want to work on high impact research and understand that sometimes mean doing things where it's unclear up front if it will bear fruit.

SummaryBot @ 2025-10-14T15:54 (+2)

Executive summary: A candid, personal case for (maybe) applying to Forethought’s research roles by 1 November: the org’s mission is to navigate the transition to superintelligent AI by tackling underexplored, pre-paradigmatic questions; it offers a supportive, high-disagreement, philosophy-friendly environment with solid ops and management, but it’s small, not a fit for fully blue-sky independence, and the author notes cultural/worldview gaps they want to broaden (tone: invitational and self-aware rather than hard-sell; personal reflection).

Key points:

  1. Roles and incentives: Forethought is hiring Senior Research Fellows (lead your own agenda) and Research Fellows (develop views while collaborating), plus possible 3–12 month visiting fellows; referral bonus up to £10k (Senior) / £5k (Research), and the application is short with a Nov 1 deadline.
  2. Why Forethought: Its niche is neglected, concept-forming work on AI macrostrategy (e.g., AI-enabled coups, pathways to flourishing, intelligence explosion dynamics, existential security tools), aiming to surface questions others miss and build clearer conceptual models.
  3. Research environment: Benefits include close collaboration, fast high-context feedback, protection from distortive incentives (status games, quick-karma topics), strong operations, and hands-on management that helps convert messy ideas into publishable work and sustain motivation.
  4. Tradeoffs and culture: Expect some institutional asks (feedback on drafts, org decisions), not total freedom in topic choice, and a culture of open disagreement and heavy philosophy; author thinks the team’s worldview range is too narrow and collaboration/feedback timing could improve.
  5. Who might fit: No specific credentials required; prized traits are curiosity about AI, comfort with pre-paradigmatic confusion, willingness to try/ditch frames, skepticism plus openness, flexibility across abstraction levels, and clear written communication.
  6. Epistemic stance: This is a personal take, not an institutional pitch; the author enjoys the team and day-to-day atmosphere, acknowledges biases, and frames Forethought as a strong option for some—but not all—researchers interested in shaping society’s path through rapid AI progress.

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.