I have many disagreements with the funding decisions of Open Philanthropy, so some divergence here is to be expected. 

Separately, my sense is Open Phil really isn't set up to deal with the grant volume that the LTFF is dealing with, in addition to its existing grantmaking. My current guess is that the Open Phil longtermist community building team makes like 350-450 grants a year, in total, with 7-8 full-time staff [edit: previously said 50-100 grants on 3-4 staff, because I forgot about half of the team, I am sorry. I also clarified that I was referring to the Open Phil longtermist community building team, not the whole longtermist part]. The LTFF makes ~250 grants per year, on around 1.5 full-time equivalents, which, if Open Phil were to try to take them on additionally, would require more staff capacity than they have available. 

Also, Open Phil already has been having a good amount of trouble getting back to their current grantees in a timely manner, at least based on conversations I've had with various OP grantees, so I don't think there is a way Open Phil could fill the relevant grant opportunities, without just directly making a large grant to the LTFF (and also, honestly, the LTFF itself isn't that well set-up to take advantage of the opportunities that are presenting themselves to us, given our very limited staff capacity and even longer response times, hence my endorsement for a somewhat modest but not huge amount of funding).

I currently expect the default thing that would happen if most of our grantees were to apply to Open Phil is that Open Phil either wouldn't get back to them for many months, or they would get a response quickly saying that Open Phil doesn't have time to evaluate their grant request, but that they are encouraged to apply to other grantmakers like the LTFF [edit: or they might fund a small fraction of these new applications and reject the rest, though my guess is Open Phil would prefer to refer them to us on the margin]. 

[Speaking for myself, not Open Philanthropy]

Empirically, I've observed some but not huge amounts of overlap between higher-rated applicants to the LTFF and applicants to Open Philanthropy's programs; I'd estimate around 10%. And my guess is the "best historical grant opportunities" that Habryka is referring to^[1] are largely in object-level AI safety work, which Open Philanthropy doesn’t have any open applications for right now (though it’s still funding individuals and research groups sourced through other means, and I think it may fund some of the MATS scholars in particular).

More broadly, many grantmakers at Open Philanthropy (including myself, and Ajeya, who is currently the only person full-time on technical AI safety grantmaking), are currently extremely capacity-constrained, so I wouldn't make strong inferences that a given project isn't cost-effective purely on the basis that Open Philanthropy hasn't already funded it.

1. ^{^}

    I don’t know exactly which grants this refers to and haven’t looked at our current highest-rated grants in-depth; I’m not intending to imply that I necessarily agree (or disagree) with Habryka’s statement.

Responding specifically to

OpenPhilantropy writes that LTFF "supported projects we often thought seemed valuable but didn’t encounter ourselves." But since the chair of the LTFF is now a Senior Program Associate at OpenPhilantropy, I assume that this does not apply to existing funding opportunities.

Having a chair who works at Open Phil has helped less than one might naively think. My impression is that Open Phil doesn't want to commit to evaluating LTFF applications that the LTFF thinks are good but doesn't have the ability to fund. We are working out how to more systematically share applications going forward in a way that doesn't create an obligation for Open Phil to evaluate them (or the impression that Open Phil has this obligation to the public), but I think that this will look more like Open Phil having the option to look at some grant applications we think are good, as opposed to Open Phil actually checking every application that we share with them.

Here are my guesses for the most valuable qualities:

1. Deep technical background and knowledge in longtermist topics, particularly in alignment. 
   1. Though I haven't studied this area myself, my understanding of the history of good funding for new scientific fields (and other forms of research "leadership"/setting strategic direction in highly innovative domains) is that usually you want people who are quite good at the field you want to advance or fund, even if they aren't the very top scientists. 
      1. Basically you might not want the best scientists at the top, but for roles that require complex/nuanced calls in a deeply technical field, you want second-raters who are capable of understanding what's going on quickly and broadly. You don't want your research agenda implicitly set by mediocre scientists, or worse, non-technical people.
   2. Because we give more grants in alignment than other technical fields, I think a deep understanding of alignment and other aspects of technical AI safety should be prioritized over (eg) technical biosecurity or nuclear security or forecasting or longtermist philosophy.
      1. The other skillsets are still valuable ofc, and would be a plus in a fund manager.
2. Consistency and reliability. 
   1. Because the LTFF chair can easily be a bottleneck, we want someone who can be quite reliable and is good at unblocking problems rather than have things slow down while waiting for their approval.
   2. This both means fairly high consistency during "normal" weeks but also a decent degree of emotional resiliency etc during more stressful times, either a lack of frequent other professional/life commitments or a process to work around such commitments, etc.
   3. I think some of LTFF's past dysfunctions can be attributed to Asya (our current fund chair) genuinely being unsure whether LTFF work is higher EV than her day job at Open Phil.^[1]
   4. This is also the main reason I don't think I should be chair. Otherwise I'm an okay fit (though not an amazing fit because I think my technical understanding isn't sufficiently deep). But for health etc reasons (and also just looking at my own empirical track record), I don't think I have enough reliability that the LTFF deserves.
3. Good generalist judgment.
   1. Good grantmaking relies on a lot of judgment calls, some obvious, some more subtle. The LTFF chair will need to set policies, hire people, etc, in ways that result in consistently good judgment calls. Most likely, this means the chair ought to have rather good judgment themselves. 

Here are some other qualities that I think are also valuable, but lower priority:

1. High professional integrity.
   1. It is very very important that we hire someone who, in their role as LTFF chair, consistently prioritizes the common good over personal gain.
      1. More than most other roles, there are a number of times where someone in the LTFF chair role can sacrifice the LTFF goals for short-term personal gain. We need someone who consistently tries their best to choose the higher-integrity option, whether for altruistic- or integrity- based reasons.
      2. This can be both obvious stuff (COIs, incentives or personal biases compromising professional judgment) and more subtle effects (eg overlooking mistakes or potential negatives of LTFF when talking to funders, chasing shiny and more prestigious proxies rather than being grounded in serving the good).
   2. I mention this below the top 3 not because I think this quality is less important than the other qualities, but because my guess is that it's less rare, especially conditional upon good judgment. My guess is that most people who fulfill the above criteria would have sufficiently high professional integrity, though it'd be very bad if we hire a chair who is below that bar.
2. Vision 
   1. Ideally they can come in with a vision of how to make LTFF great, instead of being in reactive mode and just trying to do locally good and reasonable things.
3. Good project and people management 
   1. Being a good manager is probably good for running a good fund. Though actually I think running LTFF requires this less than for leading most multi-person longtermist projects; LTFF grantmakers and grantees are both fairly autonomous. Still, the bare minimum of good management you need is higher for being an LTFF fund chair than for (eg) being an independent longtermist researcher.
4. Good stakeholder management.
   1. You should be the type of person who can understand and try to work with the priorities of funders, grantees, fund managers, advisors, etc. You don't need to be stellar at it, and you don't need to be loved, but at least you need most stakeholders to (correctly) believe that you care about their perspective and you're willing to work with them.
5. Good communications ability
   1. As with the above, a "nice-to-have" is for people to easily be able to understand your perspective. That said, this isn't a critical ability as others on the fund (eg myself) can cover for a chair who is otherwise great but is not very good at written communication.
6. Solid professional network in areas of interest for the LTFF. You can probably come in without much of a network, but I imagine things will go more smoothly if the new chair has connections they can use to promote new active grantmaking projects and new jobs, ask for advice from experts on tricky grants, etc. But I think this is something someone can build up over time as well, and shouldn't be a pre-requisite.

1. ^{^}

   I'm pretty confused about how the numbers add up, naively the 5th hour on LTFF has to be more important than the 40th hour at OP; given the relative scales of the two organizations. But I don't really know which projects Asya is responsible for, how much I'm underestimating OP giving due to anonymous donations, etc.

I don't think there is a team-wide answer, and there certainly isn't an institutional answer that I'm aware of. My own position is a pretty-standard-within-EA form of cosmopolitanism, where a) we should have a strong prior in favor of moral value being substrate-independent, and b) we should naively expect people to (wrongly) underestimate the moral value of beings that look different from ourselves. Also as an empirical belief, I do expect the majority of moral value in the future to be held in minds that are very different from my own. The human brain is just such a narrow target in the space of possible designs, it'd be quite surprising to me if a million years from now the most effective way to achieve value is via minds-designed-just-like-2023-humans, even by the lights of typical 2023-humans.

There are some second-order concerns like cooperativeness (I have a stronger presumption in favor of believing it's correct to cooperate with other humans than with ants, or with aliens), but I think cosmopolitanism is mostly correct.

However, I want to be careful in distinguishing the moral value or moral patiency of other beings from their interests. It is at least theoretically possible to imagine agents (eg designed digital beings) with strong preferences and optimization ability but not morally relevant experiences. In those cases, I think there are cooperative reasons to care about their preferences, but not altruistic reasons. In particular, I think the case for optimizing for the preferences of non-existent beings is fairly weak, but the case for optimizing for their experiences (eg making sure future beings aren't tortured) is very strong.

That said, in practice I don't think we often (ever?) get competitive grant applications that specialize in helping non-humans in the LT future; most of our applications are about reducing risks of extinction or other catastrophic outcomes, with a smattering of applications that are about helping individuals and organizations think better (eg via forecasting or rationality training or improving mechanism/institutional design), with flow-through effects that I expect to be both positive for reducing global catastrophic risks and for other long-term outcomes.

I think last time we checked, it was ~a month in the median, and ~2 months on average, with moderately high variance. This is obviously very bad. Unfortunately, our current funding constraints probably makes things worse^[1], but I'm tentatively optimistic that with the a) new guest fund managers, b) more time to come up with better processes (now that I'm onboard ~full-time, at least temporarily) and c) hopefully incoming money (or at least greater certainty about funding levels), we can do somewhat better going forwards.

(Will try to answer other parts of your question/other unanswered questions on Friday).

1. ^{^}

   Because we are currently doing a mix of a) holding on to grants that are above our old bar but below our current bar while waiting for further funding, and b) trying to refer them to other grantmakers, both of which takes up calendar time. Also, the lower levels of funding means we are, or at least I am, prioritizing other aspects of the job (eg fundraising, public communications) over getting back to applicants quickly.

This is a hard question to answer, in part because it depends a lot on the researcher. My wild guess for a 90%-interval is $500k-$10m

Annoy away – it's a good question! Of course, standard caveats to my answer apply, but there's a few caveats in particular that I want to flag:

• It's possible that by 2028 there will be one (or more) further longtermist billionaires who really open up the spigot, significantly decreasing the value of marginal longtermist money at that time
• It's possible that by 2028, AI would have gotten "weird" in ways that affect the value of money at that time, even if we haven't reached AGI (e.g., certain tech stocks might have skyrocketed by then, or it might be possible to turn money into valuable research labor via AI)
• You might be considering donation opportunities that significantly differ in value from other large funders in the field
• This is all pretty opinionated and I'm writing it on the fly, so others on the LTFF may disagree with me (or I might disagree with myself if I thought about it at another time).

In principle, we could try to assign probability distributions to all the important cruxes and Monte Carlo this out. Instead, I'm just going to give my answer based on simplifying assumptions that we still have one major longtermist donor who prioritizes AI safety to a similar amount as today, things haven't gotten particularly weird, your donation opportunities don't look that different from others' and roughly match donation opportunities now,^[1] etc.

One marginal funding opportunity to benchmark the value of donations against would be funding the marginal AI alignment researcher, which probably costs ~$100k/yr. Assuming a 10% yearly discount rate (in line with the long-term, inflation-adjusted returns to equities within the US), funding this in perpetuity is equivalent to a lump-sum donation now of $1M, or a donation in 2028 of ($1M)*(1.1^5) = $1.6M.^[2]

Then the question becomes, how valuable is the marginal researcher (and how would you expect to compare against them)? Borrowing from Linch's piece on the value of marginal grants to the LTFF, the marginal alignment grant often is a bit better than something like the following:

a late undergraduate or recent graduate from an Ivy League university or a comparable institution requests a grant to conduct independent research or comparable work in a high-impact field, but we don’t find the specific proposal particularly compelling. For example, the mentee of a fairly prominent AI safety... researcher may request 6-12 months’ stipend to explore a particular research project that their mentor(s) are excited about, but LTFF fund managers and some of our advisors are unexcited about. Alternatively, they may want to take an AGISF course, or to read and think enough to form a detailed world model about which global catastrophic risks are the most pressing, in the hopes of then transitioning their career towards combating existential risk. 

In these cases, the applicant often shows some evidence of interest and focus (e.g., participation in EA local groups/EA Global or existential risk reading groups) and some indications of above-average competence or related experience, but nothing exceptional.

[Note: I think this sort of grant is well below the current funding threshold for the LTFF, given that we're currently in a funding crunch. But I would generally expect, for the longtermist community as a whole over longer timespans, the marginal grant would be only a bit higher than that.]

Note that for many of the people funded on that kind of grant, the main expected benefit would come not from the direct work of the initial grant, but instead on the chance that the researcher winds up being surprisingly good at alignment research; so, in considering the value of the "marginally-funded researcher," just note that it would be someone with stronger signals for alignment research than described above.

So from this, if you think you'd be around equivalent to the marginally-funded alignment researcher (where it's worth it to keep funding them based on their research output), I'd think your labor would be worth about $1-2M in donations in 2028.^[3] It's harder to estimate the value for people doing substantially better work than that. I think values 10x that would probably apply to a decent number of people, and numbers 100x higher would be rare but not totally unimaginable.

1. ^{^}

   Or that current donors are appropriately weighing how much they should spend now vs invest so even if the nature of donation opportunities differs, the (investment-adjusted) value of the donations is comparable. Note that I'm not trying to claim they are actually doing this, but the assumption makes the analysis easier.

2. ^{^}

   There's a few simplifying assumptions here: I'm neglecting to consider how the cost of living/wages may raise this cost, and I'm also neglecting to consider how the labor wouldn't last in perpetuity but only for the remainder of the person's career (presumably either until they reach retirement, or until AI forces them into retirement or takes over).

3. ^{^}

   In principle, a very-marginally worth it to fund person may be (on net) worth much less than this, since funding them would also cost the money to fund them. In practice, I think this rough calculation still gives us a good general ballpark for people to index on, as very few people are presumably almost exactly at the point of indifference.

Is it important to see identifiable information (so that a donor could fund that grant specifically), or are you more interested in the types of projects/grantees we'd like to fund? Here is a fictional example of the thing I have in mind.

Funding Request for AI Safety Community Events

Location: New York

Project Goals:

Strengthen communication and teamwork among AI safety researchers via workshops, social gatherings, and research retreats. Promote academic understanding and reach in the realm of AI cross-safety through seminars and workshops. Enhance the skills of budding researchers and students via reading groups, workshops, and tutorials.

Ongoing 3rd-year PhD with multiple publications in AI safety/alignment. Part of the Quantum Strategies Group and has connections with leading innovators in Quantum Computing. Mentored several students on AI safety projects. Key roles in organizing several AI safety events and workshops. Conducted lectures and tutorials on ethical considerations and technical facets of AI. Budget: Between £5000 - £20000. Costs encompass event necessities like food, venue, travel, and recognition awards for achievements.

Current Funding: A grant of $4000 from FLI designated for the STAI workshop with an additional £400 from other sources. No intention to use the funds for personal expenses.

Alternative Funding Sources: Considering application to the nonlinear network.

I'd personally like to see more well-thought out 1) AI governance projects and 2) longtermist community building projects that are more about strengthening the existing community as opposed to mass recruitment. 

The LTFF chair at least hasn't stepped down yet! Asya is leaving in October, IIRC, and by then we hope to have found a new chair.

I can't comment much on the EAIF. It does seem kind of bad that they didn't find a replacement chair before the current one resigned, but I don't know the details.

Somewhat embarrassingly we've been overwhelmed enough with grant requests in the past few months that we haven't had much time to discuss grants, so there hasn't been much opportunity for things to be controversial among the fund managers.

But guessing about what kinds of things I disagree most with other people on, my sense is that grants that are very PR-risky, and grants that are more oriented around a theory of change that involves people getting better at thinking and reasoning (e.g. "rationality development"), instead of directly being helpful with solving technical problems or acquiring resources that could be used by the broader longtermist community, tend to be the two most controversial categories. But again, I want to emphasize that I don't have a ton of data here, since the vast majority of grants are currently just evaluated by one fund manager and then sanity-checked by the fund chair, so there aren't a lot of contexts in which disagreements like this could surface.

I am not sure these are the most controversial, but I have had several conversations when evaluating AIS grants where I disagreed substantively with other fund managers. I think there are some object-level disagreements (what kinds of research do we expect to be productive) as well as meta-level disagreements (like "what should the epistemic process look like that decides what types of research get funded" or "how do our actions change the incentives landscape within EA/rationality/AIS").

I've answered both you and Quadratic Reciprocity here.

I don't know how many points I can really cleanly communicate to such a heterogeneous group, and I'm really worried about anything I say in this context being misunderstood or reified in unhelpful ways. But here goes nothing:

• First of all, I don't know man, should you really listen to my opinion? I'm just one guy, who happened to have some resources/power/attention vested in me; I worry that people (especially the younger EAs) vastly overestimate how much my judgment is worth, relative to their own opinions and local context.
• Thank you for applying, and for wanting to do the right thing. I genuinely appreciate everybody who applies, whether for a small project or large, in the hopes that their work can make the world a better place. It's emotionally hard and risky, and I have a lot of appreciation for the very small number people who tried to take a step in making the world better.
• These decisions are really hard, and we're likely to screw up. Morality is hard and longtermism by its very nature means worse feedback loops than normal. I'm sure you're familiar with how selection/rejections can often be extremely noisy in other domains (colleges, jobs, etc). There aren't many reasons to think we'll do better, and some key reasons to think we'd do worse. We tried our best to make the best funding decisions we could, given limited resources, limited grantmaker time, and limited attention and cognitive capabilities. It's very likely that we have and will continue to consistently fuck up. 
  • This probably means that if you continue to be excited about your project in the absence of LTFF funding, it makes sense to continue to pursue it either under your own time or while seeking other funding.
• Funding is a constraint again, at least for now. So earning-to-give might make sense. The wonderful thing about earning-to-give is that money is fungible; anybody can contribute, and probabilistically our grantees and would-be grantees are likely to be people with among the highest earning potentials in the world. So if you haven't found a good match for direct work (whether due to personal preferences or external factors like not receiving funding), earning-to-give can be a great option for both impact and other desiderata. 
• Please don't work on capabilities in a scaling lab, or otherwise contributing to ending humanity. I don't know how much you care about my opinion, or even if you should. But while some people find it surprisingly comforting to work on projects that are destructive for the world when they suffer a temporary setback in attempting to save it, I suspect this will end up being the type of thing they'd regret, in addition to being straightforwardly^[1] altruistically bad.

1. ^{^}

   assuming that you agree with the object-level assessment that working in scaling labs hastens the world ending. Obviously there are reasonable object-level disagreements to be had here! (And I'm far from certain about that claim myself).

Nah, at least in my own evaluation I don't think Open Phil evaluations take a large role in my evaluation qua evaluation. That said, LTFF has historically^[1] been pretty constrained on grantmaker time so if we think OP evaluation can save us time, obviously that's good.

A few exceptions I can think of:

• I think OP is reasonably good at avoiding types-of-downside-risks-that-I-model-OP-as-caring-about (eg reputational harm), so I tend to spend less time vetting grants for that downside risk vector when OP has already funded them.
• For grants into technical areas I think OP has experience in (eg biosecurity), if a project has already been funded by OP (or sometimes rejected) I might ask OP for a quick explanation of their evaluation. Often they know key object-level facts that I don't.
• In the past, OP has given grants to us. I think OP didn't want to both fund orgs and to fund us to then fund those orgs, so we reduced evaluation of orgs (not individuals) that OP has already funded. I think switching over from a "OP gives grants to LTFF" model to a "OP matches external donations to us" model hopefully means this is no longer an issue.

Another factor going forwards is that we'll trying to increase epistemic independence and decrease our reliance on OP even further, so I expect to try to actively reduce how much OP judgments influence my thinking.

1. ^{^}

   And probably currently as well, though at this very moment funding is a larger concern/constraint. We did make some guest fund manager hires recently so hopefully we're less time-bottlenecked now. But I won't be too surprised if grantmaker time becomes a constraint again after this current round of fundraising is over.

Presumably this will differ a fair bit for different members of the LTFF, but speaking personally, my p(doom) is around 30%,^[1] and my median timelines are ~15 years (though with high uncertainty). I haven't thought as much about 10% timelines, but it would be some single-digit number of years.

1. ^{^}

   Though a large chunk of the remainder includes outcomes that are much "better" than today but which are also very suboptimal – e.g., due to "good-enough" alignment + ~shard theory + etc, AI turns most of the reachable universe into paperclips but leaves humans + our descendants to do what we want with the Milky Way. This is arguably an existential catastrophe in terms of opportunity cost, but wouldn't represent human extinction or disempowerment of humanity in the same way as "doom."

I'm answering both this question and Neel Nanda's question 

What kinds of grants tend to be most controversial among fund managers?

in the same comment. As usual, other fund managers are welcome to disagree. :P

A few cases that comes to mind:

• When a grant appears to have both high upside and high downside risks (eg red/yellow flags in the applicant, wants to work in a naturally sensitive space, etc).
  • Fund managers often have disagreements with each other on how to weigh upside and downside risks.
• Sometimes research projects that are exciting according to one (or a few) fund managers are object-level useless for saving the world according to other fund managers.
  • Sometimes a particular fund manager champions a project and inside-view believes it has world-saving potential when other fund managers disagrees, sometimes nobody inside-view believes that it has world-saving potential but the project has outside-view indications of being valuable (eg the grantee has done useful work in the past, or has endorsements by people who had), and different fund managers treat the outside-view evidence more or less strongly.
• Grants with unusually high stipend asks.
  • Sometimes a better-than-average grant application will ask for a stipend (or other expense) that's unusually high by our usual standards.
  • We have internal disagreements both on the object-level of whether approving such grants is a good idea and also under which set of policies or values we ought to use to set salaries ("naive EV" vs "fairness among grantees" vs "having a 'equal sacrifice' perspective between us and the grantee" vs "not wanting to worsen power dynamics" vs "wanting to respect local norms in other fields" etc).
    • For example, academic stipends for graduate students in technical fields are often much lower than salaries in the corporate world. A deliberation process might look like:
      • A. Our normal policy of "paying 70% of counterfactual" would suggest a very high stipend
      • B. But this will be very "out of line" with prevailing academic norms; it's also a lot more expensive than following normal academic norms, where grad students are paid in non-monetary value often
      • C. But our grantees often want the non-monetary gains from a graduate degree much less than the typical grad student. Most of our grantees in academia don't really want to be professors, and care less about academic prestige than typical. They often sought grad school primarily because it's one of the few places with good technical mentorship in AI safety-adjacent fields.
      • D. But if we pay them our normal level, then this is sort of "defecting" on prevailing academic norms. It'd look weird, draw attention, etc
      • E. But if the stipends are as low as normal in CS academia (especially in some locations/universities), this is also creating a disparity between our academic grantees and other grantees, even if the former are actually higher impact...
      • F. etc...

One thing I'd be interested in seeing is more applications from people outside of the Anglosphere and Western Europe. Both because of intellectual diversity reasons and fairly naive arguments like lower cost-of-living means we can fund more projects, technical talent in those countries might be less tapped, etc. Sometimes people ask me why we haven't funded many projects by people from developing countries, and (at least in my view) the short answer is that we haven't received that many relevant applications.

Personally, I'd like to see more work being done to make it easier for people to get into AI alignment without becoming involved in EA or the rationality community. I think there are lots of researchers, particularly in academia, who would potentially work on alignment but who for one reason or another either get rubbed the wrong way by EA/rationality or just don't vibe with it. And I think we're missing out on a lot of these people's contributions.

To be clear, I personally think EA and rationality are great, and I hope EA/rationality continue to be on-ramps to alignment; I just don't want them to be the ~only on-ramps to alignment.

[I realize I didn't answer your question literally, since there are some people working on this, but I figured you'd appreciate an answer to an adjacent question.]

Yep, grantees are definitely allowed to do so and it sometimes happens! 

I'll let someone who knows the numbers better answer with stats. 

So I wish the EA funding ecosystem was a lot more competent than we currently are. Like if we were good consequentialists, we ought to have detailed internal estimates of the value of various grants and grantmakers, models for under which assumptions one group or another is better, detailed estimates for marginal utility, careful retroactive evaluations, etc. 

But we aren't very competent. So here's some lower-rigor takes:

• My current guess is that of the reasonably large longtermist grantmakers, solely valued at expected longtermist impact/$, our marginal grants are at or above the quality of all other grantmakers, for any given time period.
• Compared to Open Phil longtermism, before ~2021 LTFF was just pretty clearly more funding constrained. I expect this means more triaging for good grants (though iiuc the pool of applications was also worse back then; however I expect OP longtermism to face similar constraints). 
• In ~2021 and 2022 (when I joined) LTFF was to some degree trying to adopt something like a "shared longtermist bar" across funders, so in practice we were trying to peg our bar to be like Open Phil's.
  • So during that time I'm not sure there's much difference, naively I'd guess LTFF to do better than OP per $ by the lights of LTFF fund managers, and OP to do better by the lights of the median OP longtermist grant associate.
• In 2023 (especially after June), the bars have gotten quite out of skew because of our liquidity issues. So I expect LTFF marginal grants to be noticeably better than OP's at the current margin (and I moderately expect the median longtermist grantmaker at OP to agree with this assessment). 
  • However, if LTFF fundraising will go as well as I currently expect it to, I think by ~October or so I expect us to roughly recalibrate to having a bar similar to OP's. We (or at least I) am not trying very hard to substantially exceed OP's bar in an ideal case.
• Note that I'm trying my best to compare LTFF marginal grants to actual OP marginal grants. Unlike us OP also has a very large warchest, and I happen to be very confused about the value of OP's last dollar, which might be a more salient comparison for the in-practice counterfactual.
  • My understanding is that OP will give more grants if they have more grantmaker capacity
• I know less about SFF, but my guess is that we're noticeably better than them. My reasoning is that I think their grants are somewhat high variance, and some of their grants are rather bad by my lights; while I haven't seen much evidence for SFF having a higher proportion of positive "hits" to justify the high variance. So my guess is the heavier left tail without a correspondingly higher right tail means that SFF grants have a lower mean, and maybe a lower median as well.
• I think we did better than the now-defunct Future Fund (both the main team and the regranting program) per $. I think Future Fund was trying to move a lot of money on fast timescales, and their deal flow was somewhat limited by applications that OP didn't pick up (which is much less of a problem at LTFF's scale). Though to be fair they were founded in 2022 when cost-effectiveness with money was much less of a concern^[1].
  • I also have the same "fatter left tail, not much evidence in favor of a fatter right tail" objection as I did with SFF. 
• Some potential relevant thoughts here are in my adversarial selection in longtermist grantmaking post.
  • I would draw attention to this self-quote "Most, although not all, of the examples are personal experiences from working on the LTFF. Many of these examples are grants that have later been funded by other grantmakers or private donors."
• I think a number of other medium-sized grantmakers (Longview, Effective Giving, GWWC's Longtermist Fund, etc) are trying to explicitly or implicitly funge with OP's last dollar while adding a few more constraints, so naively I'd guess that LTFF's better than them to the extent we're better than OP (plus a little bit more to account for the additional cost of those constraints). 
• A weak piece of outside-view evidence for LTFF being better than other longtermist funders for the marginal dollar is that other funders (OP, SFF) have given money to us to regrant, whereas we have not afaict given money to other funders to regrant.
  • Though the same evidence can adequately be explained by us being more power-hungry and/or insufficiently humble, of course.
• A weak piece of outside-view evidence for LTFF being worse than other LTFF funders for the marginal dollar is that other institutional funders haven't directly offered to cover our funding gap (yet), though to some degree we're also actively seeking to have more independence from them.
• I'm less sure about Habryka's new thing (Lightspeed Grants), Manifund, and other new grantmakers. I think "too soon to tell" is my current stance.
  • In Lightspeed's case, my impression is that they share a number of both applicants and evaluators with LTFF, so if they otherwise have a better process, I wouldn't be surprised if their grants are competitive with or better than ours.
  • Otoh my impression is that they were more swamped with work than anticipated, so presumably this means some sacrifice in decision quality.
• Of course, "quality of grants evaluations/$" isn't the only thing that matters in a grantmaking organization. I think we do worse on other desiderata: 
  • I think we are solidly middle-of-the-pack or above average in terms of $s moved/grantmaker time or impact/grantmaker time.
  • I think we do a pretty shitty job in terms of grantee experience:
    • We are rather slow in getting back to applicants.
      • I think Manifund is better? Lightspeed was trying to be better as well, not sure if they succeeded.
    • As many people on the forum complained about, we rarely give feedback to rejected applicants, and limited feedback to approved applicants as well.
  • Our "brand" is less solid than Open Phil's. I suspect this limits our applicant pool some.
  • I suspect we do very poorly on donor experience as well.
    • To some degree our donor experience is non-existent, eg until recently we didn't even offer to talk to our largest donors.
    • That said, one plus of the current model is that we aren't really Goodharting on or otherwise optimizing for non-consequentialist donor preferences, simply because to a large extent we aren't even aware of them!
  • I suspect we're leaving a ton of value on the table by not trying to engage with new donors, who might counterfactually not have given anything to longtermist orgs.
  • I think we do reasonably well on transparency, making informative posts, etc, especially recently. 
    • But this is compared to a relatively mediocre baseline.
  • Though far from the most important priority, I wouldn't be surprised if LTFF is less fun to volunteer or work at than at other grantmaking organizations, especially as a grant evaluator. 
    • Asya: "I also suspect that the lack of active discussion about grants has made the fund a worse experience for fund managers— I might describe the overall shift in the culture of the fund to have gone from "lively epistemic forum" to "solitary grantmaking machine".)"
    • In comparison, I expect being a Future Fund regrantor to be more fun, SFF's S-process and Manifund to have more productive discussions, etc.
  • SFF changes their grant evaluators regularly to minimize goodharting by grantees; this is not something LTFF directly optimizes for nearly as much, so I suspect we're worse at it.
    • If anything, I'd expect our level of public transparency (eg this Q&A, my posts) to make Goodharting even easier than baseline; this is a conscious tradeoff we're making in favor of greater transparency.

1. ^{^}

   My day job at the time was trying to do research to identify good "longtermist megaprojects" lmao.

Yes, everyone apart from Caleb is part-time. My understanding is LTFF is looking make more full-time hires (most importantly a fund chair to replace Asya).

(Speaking for myself) It seems pretty interesting. If I understand your position correctly, I'm also worried about developing and using AGI before we're a philosophically competent civilization, though my own framing is more like "man it'd be kind of sad if we lost most of the value of the cosmos because we sent von Neumann probes before knowing what to load the probes with." 

I'm confused about how it's possible to know whether someone is making substantive progress on metaphilosophy; I'd be curious if you have pointers. 

As a practical matter, I don't recall any applications related to metaphilosophy coming across my desk, or voting on metaphilosophy grants that other people investigated. The closest I can think of are applicants for a few different esoteric applications of decision theory. I'll let others at the fund speak about their experiences.

From my perspective, they seem great! If there is an existing project in a niche, this usually means that the niche is worth working on. And of course it seems unlikely that any of the existing ways of doing things are close to optimal, so more experimentation is often worthwhile! That said, 3 caveats I can think of:

• If you are working in a space that's already well-trodden, I expect that you're already familiar with the space and can explain why your project is different (if it is different). For example, if you're working in adversarial robustness for AI safety, then you should be very aware that this is a subject that's well-studied both in and outside of EA (eg in academia). So from my perspective, applicants not being aware of prior work is concerning, as is people being aware of prior work but not having a case for why their project is different/better. 
  • If your project isn't aiming to be different/better, that's also okay! For example, your theory of change might be "a total of 2 FTE-years have been spent on this research area, I think humanity should spend at least 10+ years on it to mine for more insights; I'm personally unusually excited about this area."
    • But if that's the case, you should say so explicitly.
• I'm more hesitant to fund projects entering a space with natural monopolies. For example, if your theory of change is "persuade the Californian government to set standards for mandatory reporting of a certain class of AI catastrophic failures by talking to policymakers^[1]", this is likely not something where several different groups can realistically pursue in parallel without stepping on each other's toes. 
• I'm wary of new projects that tries to carve out a large space for itself in its branding and communications, especially when there isn't a good reason to do so. I'm worried about it both in cases when there are already other players in a similar niche, and when there isn't. For example I think "80,000 Hours" is a better name than "Longtermist Career Consulting." The former can spin down naturally and can allow other orgs (like Probably Good) to enter the space, the latter name is somewhat uninviting.
  • Also see earlier comment by me here; I still stand by it.

1. ^{^}

   IIUC note also that as a fiscally sponsored project of EV, we may not legally be allowed to regrant to legislative political projects.

Right now, ~2-3 

In my personal opinion, the LTFF has historically funded too many bio-related grants and hasn't sufficiently triaged in favor of AI-related work.

Not a specific place yet! In the past we've asked specific large donors to cover our costs (both paid grantmaker time and operational expenses), going forwards we'd like to move towards a model where all donors pay a small percentage, but this is not yet enacted.

In the meantime, you can make a donation to EA Funds, email us and say you want the donation to be earmarked for operational expenses. :)

Speaking personally, I think there is a possibility of money becoming obsolete, but I also think there's a possibility of money mattering more, as (for instance) AI might allow for an easier ability to turn money into valuable labor. In my mind, it's hard to know how this all shakes out on net.

I think there are reasons for expecting the value of spending to be approximately logarithmic with total spending for many domains, and spending on research seems to fit this general pattern pretty well, so I suspect that it's prudent to generally plan to spread spending around a fair bit over the years.

I also just want to note that I wouldn't expect this whole question to affect behavior of the LTFF much, as we decide what grants to fund, but we don't make plans to save/invest money for future years anyways (though, of course, it could affect behavior of potential donors to the LTFF).

On LessWrong, Lauro said:

IMO a good candidate is anything that is object-level useful for X-risk mitigation. E.g. technical alignment work, AI governance / policy work, biosecurity, etc.

To add to that, I'd expect practice with communication and reasoning transparency and having a broad (not just deep) understanding of other work in your cause area to be quite helpful. Also, to the extent that this is trainable, it's probably good to model yourself as training to become a high-integrity and reasonably uncompromising person now, because of course integrity failures "on the job" are very costly. My thoughts on who could make a good LTFF fund chair might also be relevant.

Newbie fund manager here, but:

I strongly agree that governance work along these lines is very important; in fact, I'm currently working on governance full time instead of technical alignment research. 

Needless to say, I would be interested in funding work that aims to buy time for alignment research. For example, I did indeed fund this kind of AI governance work in the Lightspeed Grants S-process. But since LTFF doesn't currently do much if any active solicitation of grants, we're ultimately bottlenecked by the applications we receive.

From a brief look at our How applicant heard about us question data, I think the breakdown over the top 200 or so applications we have received is something like:

• EA community (EA Forum, local groups, events, personal connections): 50-60%
• AI safety/EA programs (SERI-MATS, GovAI, etc.): 10-15%
• Direct LTFF outreach: 5-10%
• Recommendations from experienced members: 5-10%
• LessWrong, 80K Hours: 5-10%
• Career advising services: <5%
• Previous applicants/grantees: <5%
• Online searches: <5%

The EA community seems to be the dominant source, accounting for around half or more of referrals. Focused AI safety/EA programs and direct LTFF outreach collectively account for 15-25%. The remaining sources are more minor, each representing less than 10% likely. But this is an approximate estimate given the limitations of the data. The overall picture is that most people hear about LTFF through being part of the broader community.

We do have some plans to fundraise from high net-worth individuals, including doing very basic nonprofit things like offering to chat with some of our biggest past donors, as well as more ambitious targets like actively sourcing and reaching out to HNWs who have (eg) expressed concerns about AGI x-risk/GCRs but have never gotten around to actually donating to any AI x-safety projects. I don't know if we have the expertise for this, to some degree this is an empirical question.

We have no current plans to raise money from companies, governments, or (non-OP) large foundations. 

I haven't thought about this much at all, but my current weakly-held stance is that I think a longtermist grantmaking organization is just a pretty odd project for governments and foundations to regrant to. I'd be more optimistic about fundraising and grantwriting efforts from organizations which are larger and have an easier-to-explain direct impact case: ARC, Redwood, FAR, CHAI, MIRI(?), etc.

I think raising money from companies is relatively much more tractable. But before we were to go down that route, I'd need to think a bit more about effects on moral licensing, safetywashing, etc. I don't want to (eg) receive money from Microsoft or OpenAI now on the grounds that it's better for us to have money to spend on safety than for them to spend such $s on capabilities, and then in a few years regret the decision because the nebulous costs of being tied with AI companies ^[1]ended up being much higher than I initially modeled.

1. ^{^}

   One advantage of our current ignorance re:donors is that fund managers basically can't be explicitly or subtly pressured to Goodhart on donor preferences, simply because we don't actually know what donor preferences are (and in some cases don't even know who the donors are).

We use paper form for the application form, Airtable and Google docs for evaluation infrastructure (making decisions on what we want to fund) along with many Airtable and zapier automations. EV then uses some combination of salesforce, xero etc. to conduct due diligence, make the payment to the grant recipients etc.

My impression is that we are pretty Salesforce reliant on the grant admin side, and moving away from this platform would be very costly. We are not salesforce reliant at all on the evaluation side.

We don’t have any internal tooling for making botecs, people tend to use whatever system the like for this. I have been using squiggle recently and quite like it though I do think it’s still fairly high friction and slow for me for some reason.

Past experience with fairly independent research and access to high-quality mentors (so they are less likely to become directionless and/or depressed) are positives for me.

Speaking for myself: it depends a lot on whether the proposal or the person seems promising. I'd be excited about funding promising-seeming projects, but I also don't see a ton of low-hanging fruit when it comes to AI gov research.

There's no built-in feature but you can email us or link to google docs; as a practical matter I think it's much more common that applications are updated by having different funding needs or the applicants decided to pursue different projects (whether their new project needs or doesn't need funding) than because the applicant now looks significantly stronger.

You can also reapply after being rejected if your application is now substantially more competitive.

I'm hoping that LTFF will work towards being much more efficient going forwards, so it becomes less practically useful for applicants to feel a need to update their applications mid-evaluation. But this is aspirational; in the meantime I can totally see value in doing this.

Thanks for the offer! I think we won't have the capacity (or tbh, money) to really work on soliciting new grants in the next few weeks but feel free to ping Caleb or I again in say a month from now!

How small and short can a grant be? Is it possible for a grant to start out small, and then gradually gets bigger and sources more people if the research area turns out to be significantly more valuable than it initially appeared?

In general, yes it can be arbitrarily short and small. In practice I think EV, who does our operations, has said that they prefer we don't make grants <$2,000 (? can't remember exact number), because the operational overhead from them might be too high per grant to justify the benefits. 

I don't have strong opinions here on form; naively I'd prefer some combination of longform work (so the entire message gets across without sacrificing nuance), popularity, and experimentation value.

In terms of content, I suspect there's still value left in detailed and nuanced explanations of various aspects of the alignment problem, as well as distillation for the best current work on partial progress on solutions (including by some of our grantees!)

In general I expect this type of communication to be rather tail-heavy, so the specific person and their fit with the specific project to matter heavily. Ideally I think I'd want someone who 

• a) has experience (and preferably success) with their target type of communications, 
• b) who has or can easily acquire a fairly deep understanding of the relevant technical subjects (at all levels of abstraction),
• c) who actually likes the work, 
• d) and d has some form of higher-than-usual-for-the-field integrity (so they won't eg Goodhart on getting more people to sign up for 80k by giving unnuanced but emotionally gripping pitches).

Note that I haven't been following the latest state-of-the-art in either pitches or distillations, so it's possible that ideas that I think are good are already quite saturated.

I know it sounds weird, but I'm pretty confident that this mitigates risks of a specific class.

I'd be interested in hearing what specific class of risks this mitigates. I'd be open to doing something like this, but my guess is that the plausible upside won't be high enough to justify the operational overhead.

If a project is very sensitive fund managers have been happy to discuss things in person with applicants, (e.g. at EA/rationality events) but we don't have a systematic way to make this happen rn and it's necessary infrequently enough that I don't plan to set one up.