Reflections on my time on the Long-Term Future Fund

By abergal @ 2023-08-02T01:32 (+179)

I'm stepping down as chair of the Long-Term Future Fund. I'm writing this post partially as a loose set of reflections on my time there, and partially as an overall update on what's going on with the fund,[1] as I think we should generally be transparent with donors and grantees, and my sense is the broader community has fairly little insight into the fund's current operations. I'll start with a brief history of what's happened since I joined the fund, and its impact, and move to a few reflections on ways the fund is working now.

(Also- you can donate to the Long-Term Future Fund here, and let us know here if you might be interested in becoming a fund manager. (The Long-Term Future Fund is part of EA Funds, which is a fiscally sponsored project of Effective Ventures Foundation (UK) (EV UK) and Effective Ventures Foundation USA Inc. (EV US). Donations to the Long-Term Future Fund are donations to EV US or EV UK.)[2])

A brief history of my time on the Long-Term Future Fund

I joined the Long-Term Future Fund as a fund manager in June 2020. At the time, it was chaired by Matt Wage, had five fund managers (including me), and ran three rounds per year, with order ~50 applications per round,[3] giving away an average of ~$450K per round in both 2019 and 2020, for a total of ~$1.35M per year.

Matt Wage left the fund, and I was appointed the new chair in February 2021. We also hired a number of new fund managers, and decided to try out a guest manager system where we had more temporary fund managers work with the fund. (We're still doing that today.)

The biggest change that I pushed for during my time as chair (which I believe was originally suggested by Ozzie Gooen, thanks Ozzie) was switching from three fixed rounds a year to a rolling application system, where anyone could apply to the fund at any time. My current guess is that this was a pretty big boost to the fund's impact, via giving us access to a bunch of counterfactual grant opportunities that other funders (who either didn't have rolling applications, or were funding a more constrained set of things) didn't have access to. I also pushed for a switch away from mandatory public reporting for our grant applicants (which I discuss somewhat below), which I also think gave us access to better grant opportunities, and which I also overall still endorse.

At a high-level, the fund consists of a number of fund managers who work a relatively low number of hours per week (generally 3 - 10, though I think it might be trending even lower recently). EA Funds itself only has one full-time employee— Caleb Parikh. There's been significant fund manager turnover since I joined the fund, generally because relevant fund managers have wanted to prioritize their other work— since I joined the fund, Matt Wage and Helen Toner have left, Adam Gleave, Evan Hubinger, and Becca Kagan have joined as permanent fund managers and then left, Linchuan Zhang has joined as a permanent fund manager, and we've had a number of guest managers join and leave the fund.

Also during my time on the fund, the volume of our grantmaking work has scaled significantly— whereas in 2020, we received 211 applications and funded 34 as grants, worth ~$1.3M dollars total, from March 2022 to March 2023, we received 878 applications and funded 263 as grants, worth ~$9.1M dollars total.

The fund’s impact

My reflections below focus on ways I think the Long-Term Future Fund has been suboptimal over my time as chair, and I got feedback on my original draft of this post that this made it seem like my view of the fund was negative overall, which wasn’t my intention. For clarity, my best guess is that overall, the Long-Term Future Fund has been, and continues to be, pretty (positively) impactful: 

To the extent that this section is shorter than others, it's because I think most of our impact, which has come through our grantmaking, is well-documented in other places, e.g. our payout reports and our grants database, and I think it's generally more interesting (both for others, and for my reflection) to think about ways in which things could have gone better. (I also wanted to make clear that most of my reflections on suboptimality are about our grantee experience, rather than our grantmaking quality (which I have no particular reason to think has declined)).

Reflections

Problems with scale

I think the Long-Term Future Fund hasn’t done a good job staying fast and reliable in the face of a massive increase in application and grant numbers. (As I say above, we went from 34 grants in 2020 to 263 grants from March 2022 - March 2023). In particular, I don’t think we’re able to get back in time reliably on time-sensitive applications, and have taken a very long time to get back on a small number of applications– our median response time from January 2022 to April 2023 was 29 days, but our current mean (across all time) is 54 days (although the mean is very unstable). As the single person arguably most responsible for the fund functioning well, and definitely most responsible for its increase in applications, I think I should take a fair share of the blame for this.

In my view, a fundamental problem at play here is that the fund isn't structurally well set-up to handle its current application load. In general, I think having a bunch of people working part-time on something outside of their main job is not an amazing set-up— even for extremely conscientious people, it's hard to juggle multiple responsibilities, especially when there's no formal managerial relationship or accountability, meaning it's difficult to count on people (chiefly and including me) working reliable hours, or keeping track of deadlines. I think switching the fund to rolling applications made these dynamics even worse, as it meant that fund managers had a lot more applications to evaluate, had to keep track of a lot more individual deadlines, and had to work more consistent hours to keep on top of those deadlines (as opposed to everyone being able to plan around a constrained busy stretch during each funding round). The collapse of FTX also substantively worsened this dynamic, as it meant that we got more applications, each application decision was harder and had higher stakes (as a result of the overall decrease in funding in the space), and at least one fund manager was overwhelmingly busy dealing with the aftermath.

As our applicant pool has increased and our fund managers have turned over rapidly, I personally have been more of a bottleneck on our funding decisions, as I've had to do more work assigning applications, tracking deadlines, and providing a second eye on applications that only one fund manager looks at (which is currently the default for most applications, given the ratio of applications to fund managers).[4] I've recently spent some time writing up some policies for how the LTFF should operate that I can share with new fund managers, in the hopes that it will save me time in the long run, as the amount of time I end up spending per application we get is approximately inversely proportional to how long the primary evaluator investigating has been on the fund. Overall, I think the chair role has expanded significantly since I joined the fund— originally, the chair was originally just a slightly special fund manager who had the job of assigning applications; now, it feels more like I'm half-heartedly managing a small team who don't interact with each other much by default.

(I also suspect that the lack of active discussion about grants has made the fund a worse experience for fund managers— I might describe the overall shift in the culture of the fund to have gone from "lively epistemic forum" to "solitary grantmaking machine".)

The main consequence of all of the above is that I think the Long-Term Future Fund has been significantly less reliable and consistent in the last year, and especially in the last eight months (since the collapse of FTX). I'm sorry to any applicants that this affected, and I'm especially sorry to the extent that the LTFF has set false expectations around our speed and reliability— we previously changed our application form to say "if your application is time-sensitive, we may or may not have the capacity to get back to you in time", but I now think that language is too weak, and have changed it to say "the LTFF is low on capacity and may not be able to get back to you by your stated deadline— we encourage you to apply to other funders as well if you have a time-sensitive ask." (That said, I think applicants who want time-sensitive decisions should still apply to the LTFF— we do get back to people in time in a lot of cases, and I'm pushing us to increasingly prioritize applications that look particularly promising at a quick glance.)

Some obvious paths to improving the fund on this dimension are a) hiring more fund managers; b) hiring a full-time chair, or otherwise restructuring the fund so as not to have one person bottlenecking its work as much (e.g. by giving multiple more senior fund managers the responsibility to be a second eye on grant evaluations by more junior ones); or c) changing the way the fund operates in a more major way, e.g. going back to a round-based system, or changing the fund to a collection of independent regrantors. In my remaining time as chair, I plan to work on hiring, but currently think it's unlikely that I'll push for any major restructures. (But it seems like something to consider for whoever replaces me).[5]

Historically, we've had trouble hiring fund managers, especially in technical AI alignment, largely for the reasons mentioned above (people generally want to focus on their work). I think there's an extent to which I've contributed to our difficulty in hiring, in that I'm not sold that people doing good direct work should be taking on additional responsibilities as fund managers (so haven’t been great at convincing people to join), though others on the fund feel differently, and I'll let them comment below.

I also think it's possible that some combination of existing and new funders will increasingly fill the historic niche that the LTFF has had (as an always-open funder accepting applications for anything), which would be a good outcome in my eyes, and could reduce the LTFF's workload to a more sustainable level.

Transparency

Several people have told me that they feel disappointed that the Long-Term Future Fund hasn't been putting out detailed payout reports the way we have historically (at least not since this report covering our giving through 2021). I stopped mandating those reports, and we switched to a grantmaking database, as I felt like the fund was overwhelmed with evaluating applications, though my hope was that individual fund managers would still be interested in publishing grant reports independently. (In hindsight, it seems obvious that this wasn’t going to happen without active encouragement on my part, though Linch did write this great post about lessons he learned as a grantmaker.) I also pushed to switch the fund away from mandatory reporting for all of its grants, which means that some fraction of the grants we make are no longer publicly reported.[6]

I think both of these actions arguably trade off one theory of change for the fund, "contributing to the epistemic commons", for another, "making more (good) grants". I personally have always been more sold on the latter theory of change (for the LTFF in particular), so think that these were good trades, but I think there's room for disagreement on that, and all things equal, I think it's too bad that we haven't put out more payout reports over the last year. We recently did release a new payout report here, and I'd guess the fund is likely to continue putting them out at least somewhat more frequently, as it does more independent fundraising.

  1. ^

    We've also recently posted updates on our grantmaking over the past year and an update on actions we're taking to increase the separation between Open Phil and EA Funds.

  2. ^

    Effective Ventures Foundation (UK) (EV UK) is a charity in England and Wales (with registered charity number 1149828, registered company number 07962181, and is also a Netherlands registered tax-deductible entity ANBI 825776867). Effective Ventures Foundation USA Inc. (EV US) is a section 501(c)(3) organization in the USA (EIN 47-1988398). Please see important state disclosures here.

  3. ^

     For the rounds I had access to.

  4. ^

     I haven't substantially increased my amount of time spent on the fund in response to this increased workload, both for personal prioritization reasons, and as a result of an agreement that I made with my employer, Open Philanthropy, about how much of my time I would spend on the LTFF.

  5. ^

     One thing I feel less bullish on than I did in the past is solving the problem with the set-up above by decreasing the bar for fund manager hires-- at least in my experience, I felt like bringing on people who I thought were less strong resulted in more overhead for me, which just made the part of the process that I was bottlenecking worse.

  6. ^

    From January 2022 to April 2023, 11% of our total funding recommended has gone to grants that weren’t publicly reported.

Rebecca Kagan @ 2023-08-03T19:02 (+76)

(I was previously a fund manager on the LTFF)

Agree with a lot of what Asya said here, and very appreciative of her taking the time to write it up. 

One complimentary point I want to emphasize: I think hiring a full-time chair is great, and that LTFF / EA Funds should in general be more willing to hire fund managers who have more time and less expertise. In my experience fund managers have very little time for the work (they’re both in part-time roles, and often extremely busy people), and little support (there’s relatively little training / infrastructure / guidance), but a fair amount of power. This has a few downsides:

  1. Insular funding: Fund managers lean heavily on personal networks and defer to other people’s impressions of applicants, which means known applicants are much more likely to be funded. This meant LTFF had an easy time funding EAs/rationalists, but was much less likely to catch promising, non-EA candidates who weren't already known to us. (This is already a common dynamic in EA orgs, but felt particularly severe because of our time constraints.) 
  2. Less ambitious funding: Similarly, it's particularly time-intensive to evaluate new organizations with substantial budgets, or novel theories of change. This meant that we were more likely to fund individuals, and had a harder time making large grants to new ambitious orgs. (We were also more likely to lean on OP’s evaluation of the person or org.)

I left the LTFF because of capacity issues related to being on the EV board post FTX collapse, but I didn't return because I ended up feeling pretty skeptical of a funding dynamic that moved money primarily within existing social circles with relatively little supervision, especially post FTX. I think full-time fund managers & more structure and support could help with this. In the past, I think we’ve been recruiting from too narrow of a pool — I think there are a fair number of people who could do this job well (and that the quality of work they can do improves if they have more time and more support), and moving in that direction could be good for the fund. (Though I haven't actually done the hiring for these roles, so I could be wrong.)

Really excited about EAF/LTFF potentially being more independent from OP & having more full-time staff. I think both of those are really great directions to move in, and I'm very appreciative to Asya & Caleb for putting in the time to think it through.

Joel Becker @ 2023-08-02T08:27 (+33)

Thank you for sharing these reflections, Asya! And for your service as the LTFF chair!

I feel confused about the difficulty of fund manager hiring. One source of confusion comes from the importance of expertise (/doing-good-direct-work), as you touch on in the post:

Historically, we've had trouble hiring fund managers, especially in technical AI alignment, largely for the reasons mentioned above (people generally want to focus on their work). I think there's an extent to which I've contributed to our difficulty in hiring, in that I'm not sold that people doing good direct work should be taking on additional responsibilities as fund managers (so haven’t been great at convincing people to join)

In addition to the high opportunity cost of time for expert fund managers, I would have guessed that small differences between the EV of marginal grants pushes in the direction of expertise being less important. But then I don’t understand why hiring fund managers would be unusually challenging. Wouldn’t deemphasizing expertise increase the pool of eligible fund managers, thereby making hiring easier?

(Perhaps I‘m confusing relative and absolute difficulty — expertise being less important would make hiring relatively easier, but it’s still absolutely tough?)

The second source of confusion comes reconciling the difficulty of finding fund managers with the fact that FTXFF and Manifund seemed to find part-time grantmakers quite easily. I don’t know how many regrantors and grant-recommenders FTXFF ended up with, but the last rumour I heard was between 100 and 200. Manifund are currently on 16 and seem keen to expand. I would’ve thought that there is some intersection between regrantors with the top, say, 30% of grantmaking records by your light, those satisfying other hiring criteria you might have, and those currently willing to work with LTFF.

Is the difference in the scale of grants LTFF fund managers make vs regrantors? Or expectations around regularity of response (regrantors are more flexible)? Or you’re not excited about the records of regrantors in general? Or something else?

abergal @ 2023-08-05T00:20 (+7)

Re: deemphasizing expertise:

I feel kind of confused about this-- I agree in theory re: EV of marginal grants, but my own experience interacting with grant evaluations from people who I've felt were weaker has been that sometimes they’re in favor of rejecting a grant that I think would be really good, or missing a consideration that I think would make a grant pretty bad, and furthermore it's often hard to quickly tell if this is the case, e.g. they'll give a stylized summary of what's going on with the applicant, but I won't know how much to trust that summary, so feel compelled to read the full grant application (which is bad, because I already bottleneck the process so much).

I basically feel pretty confident that lowering the bar for fund managers would lead to worse grants by my lights, but I don't think I have a great grasp on the full space of trade-offs (i.e. how much worse, exactly? is the decrease in quality worth it to be able to get through more grants in a timely way?); it's totally plausible to me there would be some set-up that would overall be better than the current one.

Re: comparing to FTXFF and Manifund:

I think the pitch for being a regrantor for the FTXFF or Manifund is pretty different than the one for the LTFF, both in terms of structure and raw number of hours. 

As a regrantor, you get to opt-in to making the grants you're most excited about on your own time, whereas on the LTFF, you're responsible for spending a certain number of hours per week (historically, we've asked for a minimum of 5, though in practice people work less than that) evaluating incoming grant applications. (As a concrete instance of this, Adam Gleave is currently a regrantor at Manifund but left the LTFF a while ago -- this isn't to cast aspersions on Adam; just to illustrate that people have differing preferences between the two.)

I do think a possible restructure for the LTFF would be to switch to an opt-in regranting set-up, but that would be a pretty different way of operating. (I'd guess a bunch of good grants coming in through our application form would be missed by default, but it could still be overall preferable from the perspective of being more sustainable for fund managers.)

Joel Becker @ 2023-08-08T11:44 (+9)

Thank you for the helpful replies Asya.

Re: deemphasizing expertise: 

I would imagine that some of the time saved in hiring expert grantmakers could be spent training junior grantmakers. (In my somewhat analogous experience running selection for a highly competitive program, I certainly notice that some considerations that I now think are very important were entirely missing from my early decision-making!) Should I think about your comment as coming from a hypothetical that is net or gross of that time investment?

As for improved set-ups, how about something like:

  1. Junior grantmaker receives disproportionate training on downside considerations.
  2. Junior grantmaker evaluates grants and rates downside risk.
  3. Above some downside risk cut-off, if the junior grantmaker wants to give funding, the senior grantmaker checks in.
  4. Below the cut-off, if the junior grantmaker wants to give funding, the funding is improved without further checks.

(If you think missing great grants is a bigger deal than accepting bad ones, analogously change the above.)

Intuitively, I would guess that this set-up could improve quite a bit on the waste-of-your-time and speed problems, without giving up too much on better-grants-by-your-lights. But I'm sure I'm missing helpful context.

Re: comparing to FTXFF and Manifund: 

Definitely makes sense that the pitches are different. I guess I would have thought of this as part of "other hiring criteria you might have" -- considerations that make it more challenging to select from the pool of people with some grantmaking experience, but for which some people tick the box.

calebp @ 2023-08-03T22:01 (+5)

In addition to the high opportunity cost of time for expert fund managers, I would have guessed that small differences between the EV of marginal grants pushes in the direction of expertise being less important. But then I don’t understand why hiring fund managers would be unusually challenging. Wouldn’t deemphasizing expertise increase the pool of eligible fund managers, thereby making hiring easier?

I think a decent number of grants are pretty high variance and quite a few grants that seem marginal to some seem pretty negative to others (particularly more experienced fund managers).

I would’ve thought there is some intersection between regrantors with the top, say, 30% of grantmaking records by your light, those satisfying other hiring criteria you might have, and those currently willing to work with LTFF.

I didn't quite parse this, but my impression is that we either weren't able to attract excellent fund managers, or we were looking for different things to FTXFF and Manifund such that our bar ended up looking higher. I do think that our experiments with hiring people that we are less excited about have made me less keen to hire more people close to the bar but more keen to create junior positions and increase management capacity.

Joel Becker @ 2023-08-08T11:45 (+1)

Thanks Caleb! Noting that my reply to Asya is relevant here too.

Chris Leong @ 2023-08-02T22:22 (+4)

I talked to someone from Manifund who said that it was easy to find people to put up their hand to be grantmakers, but harder to get them to actually give out grants.

Daniel_Eth @ 2023-08-03T09:51 (+7)

Random idea that just came to me – require regranters to put down a "deposit" (say, $2,000 if their budget is $100,000), which they only get back once they've donated at least 50% of their budget. Not only would this create an incentive for them to actually regrant, but it would also make them think a bit harder before accepting to be a regranter about whether they were actually committed. (OTOH, I can also think of negative aspects of this setup.)

Joel Becker @ 2023-08-03T07:48 (+5)

Seems like:

  1. FTXFF did move a lot of money, alas.
  2. Speed due to indecision is a very solveable problem (e.g. tie funding that a given regrantor can give out to shorter window, or have shared pot that other regrantors could use on their preferred projects first).
  3. The indecision is due to the ~need to actively seek out grants, which wouldn’t be a problem for LTFF.

(In case relevant: I am a Manifund regrantor who just got back from holiday and plans to start working on it today! Thank you for the gentle push! :P)

Esben Kran @ 2023-08-14T10:47 (+22)

Thank you for sharing your reflections and for the work you've done on the EA Funds, Asya! I appreciate the role the Funds have played over the past years.

our median response time from January 2022 to April 2023 was 29 days, but our current mean (across all time) is 54 days (although the mean is very unstable)

A few questions arise from your mention of the Funds' response times. I would love to hear more about the numbers and information here. For instance, how did the median and mean change over time? What does the global distribution look like? The disparity between the mean and median suggests there might be significant outliers; how are these outliers addressed? I assume many applications become desk rejects; do you have the median and mean for the acceptance response times?

Sanjay @ 2023-08-03T13:37 (+16)

It seems that a central bottleneck for the fund is that a few key people are decision-makers, and they are very busy, which makes it hard to operate quickly at scale and be transparent.

When SoGive ran its grants programme last year, we tackled these problems by getting more junior people to help.

I.e. the structure was:

In many ways, this structure seemed to work well, since it allowed for the following:

Here are some potential downsides:

I'm not trying to say that EA Funds should definitely do something like this, more that if it's useful to explore what happened when someone tried something different, I'd be happy to discuss.

calebp @ 2023-08-03T21:56 (+9)

>It seems that a central bottleneck for the fund is that a few key people are decision-makers, and they are very busy, which makes it hard to operate quickly at scale and be transparent.

I think this is at least somewhat true. We have tried out having more junior managers on the fund with mixed success. The EAIF currently has "assistant fund managers" which I think was a good experiment for us to run, and I think it's generally gone well.

My impression is that SoGive gave out something like $300k and had 26 applicants so it doesn't seem super comparable to me to the LTFF (I think last year we had ~1000 applications), and I'd guess that your methods don't scale particularly well to the kind of grantmaking the LTFF does (but I could be wrong).

I also somewhat disagree with Asya re our transparency, I think that we are falling short of where I'd like us to be, but if you compare us to other grantmaking programs that have existed for more than 1 year I think we look pretty good transparency wise (e.g. Longview, Effective Giving, Open Phil) though plausibly they don't need to be as transparent as they are raising less from the public.

Linch @ 2023-08-03T17:13 (+15)

Thank you for your leadership on the LTFF Asya! I really appreciate working with and learning from you. In particular, I learned from your attitude towards candor, humility, and consistent prioritization/triage mindset in internal discussions; by default I tend to get off-track a lot and/or be perfectionist about things that don't matter as much.

Larks @ 2023-08-13T01:27 (+5)

I think both of these actions arguably trade off one theory of change for the fund, "contributing to the epistemic commons", for another, "making more (good) grants".

I think this misses an important other theory of change: "raising money". In fact it seems that the EA Funds are uniquely unsuited to making anonymous grants, because unlike with OP or SFF/Lightspeed, LTFF making an anonymous grant denies the information not only to the general public but also to the donors. 

Jeroen Willems @ 2023-08-09T13:50 (+5)

I think having paid (part time or full time) fund managers with less expertise makes sense. Having such a high turnover of fund managers isn't great for grantees either. I'm not really sure what the cons of paid fund managers are, but I can imagine that there's a good argument against it that would change my mind. Having less expertise could be a great thing, as your mind isn't set on a particular view and you can still gather insights from people who do have expertise. And while they perhaps won't be experts in AI safety or biosecurity, they could be(come) experts at managing funds. Perhaps a pool of experts could be created that fund managers can ask questions too, as long as it won't take up more than 2 hours a week of their time (or a different cut off).

Linch @ 2023-08-09T20:35 (+8)

LTFF (and I think EAIF as well) already offers pay to fund managers. Some fund managers take them up on it; I personally didn't until recently (when I started investing more time into LTFF than RP work, mostly on the communications front). 

Rebecca @ 2023-08-30T02:04 (+4)

I am quite confused about why keeping track of individual deadlines isn’t automated?

weeatquince @ 2023-08-24T17:52 (+4)

Thank you Asya for all the time and effort you have put in here and the way you have manged the fund. I've interacted with the LTFF a number of times and you have always been wonderful: incredibly helpful and sensible. 

quinn @ 2023-08-03T18:42 (+3)

and let us know here if you might be interested in becoming a fund manager

It seems like there should be more of a scouting and nominations system. I don't expect self-nominations to work very well for basic dunning-kreuger reasons. 

calebp @ 2023-08-03T22:03 (+5)

I think the plan is for this to be a pretty lightweight self-nomination process, we'll do a quick skim and approach people that we are really excited about. It is mostly so that people can opt in to the process and show that they are somewhat enthusiastic about becoming a fund manager.