Manifund Q1 Retro: Learnings from impact certs

By Austin, Rachel Weinberg, Saul Munn @ 2024-05-01T16:48 (+28)

Manifund is a philanthropic startup that runs a website and programs to fund awesome projects. From January to now, we wrapped up 3 different programs for impact certificates (aka venture-style funding for charity projects): ACX Grants, Manifold Community Fund, and the Chinatalk essay competition.

Overall, we’ve learned a lot and are happy with the projects we’ve funded, but are less excited by impact certs than before — it’s been hard to get investor interest, and we still haven’t found a use case where certs led to better funding decisions. For the next quarter, we’re trying out a bunch of things (including Manifest, regranting, and prize challenges) while looking to get product-market fit.

What we’ve been working on in Q1

ACX Grants 2024

Manifund hosted the 2024 round of ACX Grants, which included both direct funding to some projects and an impact market for any project that opted-in and did not receive direct funding. ACX directly funded 33 projects for a total of $1.35mm; on the impact market, investors funded 12 projects to their minimum bars, for ~$50k total at a combined valuation of ~$200k.

Background

Astral Codex Ten (ACX) is a blog by Scott Alexander on topics like reasoning, science, psychiatry, medicine, ethics, genetics, AI, economics, and politics. ACX Grants (ACXG) is a program in which Scott helps fund charitable and scientific projects — see the 2022 round here and his retrospective on ACX Grants 2022 here.

In this round (ACX Grants 2024), some of the applications were directly funded by Scott; the rest were given the option to participate in an impact market, an alternative to grants or donations as a way to fund charitable projects. Manifund transferred funds from donors to grantees; hosted project proposals on our website; and ran the impact marketplace.

ACXG Direct Funding

Scott evaluated all of the applications along with his team of judges, then directed a total of $1.35M to 33 projects. You can see the results on ACX or on the “Grants” tab of Manifund’s ACXG 2024 page. Example grants:

Open source predictors for polygenic screening, by Gene Smith

Building anti-mosquito drones, by Alex Toussaint

Solving Far-UVC ozone generation, by Jacob Swett

This was relatively straightforward to run, as making direct grants is similar to what we did in our regranting program from last year. The biggest operational burden was collecting funding from a variety of donor-advised funds for donations to the ACX Grants program.

ACXG Impact Market

Projects that weren’t given a direct grant were eligible to participate in an impact market, and the ones that opted in are listed on the “Impact Market” tab. You can read more our announcement post about the impact market here. Example projects that were funded:

Run a public on line Turing Test, by Cameron Jones

Scaling Legal Impact for Chickens, by Alene Georgia Anello

Platform for migrants to start microbusinesses, by Emily Kerr-Finell

Five retroactive prize funders agreed to participate: next year’s ACX Grants, the Survival and Flourishing Fund, the Long-Term Future Fund, the Animal Welfare Fund, the Effective Altruism Infrastructure Fund. In 2025, they’ll award prize funding to successful projects, and the investors who bet on those projects will receive their share in charitable dollars.

Some key differences between this impact market and previous impact markets we’ve run:

Retro funding: With regard to retro funding, this impact market:

• …was the first one we ran that has multiple retroactive funders. We’ll have a better understanding of how this “went” after the retroactive evaluations happen (next year).
• …had the highest range of retro funding. Retro funders committed to evaluating projects retrospectively as if they were a prospective grant with a 100% success likelihood, which meant that they will be competitive with the $5-$33 million that the retro funders disburse yearly. This also made some of the investing process more difficult, as it also made the likelihood and quantity of retro funding less clear. However, the goals of the retro funders were broadly well-defined (and further collated in our announcement post). Also, attempts to make this more well-defined end up trading off against flexibility of both the retro funders and the project creators — and it will have been good to test the limits of an impact market in this direction.
• …had the least defined criteria. We broadly said “aim for the goals of the retrofunders,” and let everything else figure itself out, rather than “here’s a specific goal, achieve it.” This also made some of the investing process more difficult, as it also made the likelihood and quantity of retro funding less clear.

Project type & scope:

• Since this impact market was for projects that were already rejected from a previous round, we did experience a bit of selection, both advantageous and adverse. Most noticeably, a lot of the projects that would’ve been obviously-good to fund had already received funding from the previous round, which meant that only the projects that were not obviously-good were included (which included some fantastic projects, just not the ones which were obviously fantastic upon first glance).
• There was no central or obvious “theme” (besides being for ACX Grants, which itself is incredibly broad). Previous impact markets had been about AI worldviews, or forecasting & prediction markets, or the Manifold community. The wider theme lead to a more diverse array of projects, making it harder for investors to compare different projects.

The ACXG Impact Market overall went okay, but there were definitely a few areas that the impact market was lacking on:

• Participation: we had less participation from investors — both in terms of external $ invested and comments/interaction — than we had hoped. One of the projects directly funded in ACX Grants (Building anti-mosquito drones, by Alex Toussaint), went viral on Twitter and received a bunch of small donations. We would’ve liked to see this kind of momentum on one of our impact markets proposals, to build up a more sustained environment of donation & interaction.
  • We launched a $500 micro-regranting program to get more investors to participate, see below for more details on how that went (spoiler: pretty well!).
• Complexity/confusion: the mechanisms behind an impact market are confusing, but the auction mechanisms we used were especially confusing & complex. We’re considering ways that we can mimic the style of a SAFE or an MFN clause from VC investing onto an impact market.
• Financial operations: we ran into a few unexpected and uncorrelated issues with our payment processor, Mercury. For the vast majority of grants, there wasn’t any problem, but for a few grants it lead to delays of about 4-5 weeks

Micro-Regranting Program

Halfway through the impact market, we noticed that it was receiving less attention than we would’ve hoped. We decided to run a “micro-regranting program,” an extremely scaled-down version of the regranting programs we’ve run in the past.

We described the program in the application form:

Manifund is assembling a small cohort of charitable-funding enthusiasts to try their hands at impact investing ... If selected, you'll get to allocate $500 of charity budget to invest in impact certificates for the 2024 Astral Codex Ten Grants program; you can donate any profit from the investments to the charity of your choice. We're looking for people who are excited to thoughtfully consider how to allocate their investments, similarly to Manifund's regrantors (see examples here), and to post some feedback on the funding decisions they make.

The description “the world’s tiniest and most wholesome hedge fund” was aptly used. And overall, the micro-regranting program was a huge success! Some parts that went particularly well:

• We posted a few announcements to elicit applications, and we received 26 applications in just a few days (way more than we expected, much faster than we expected), and approved 24 of them. The high acceptance rate wasn’t reflective of an especially low bar for micro-regrantors, we just got a ton of fantastic applications!
• We moved pretty fast on this — it took us just about a week from the first announcement of the micro-regranting program to having built up a solid cohort of micro-regrantors.
• The micro-regrantors left many thoughtful comments, gave us and project creators detailed feedback over calls & in the Discord, and helped allocate about $17k in funding. Project creators seemed to enjoy chatting with the micro-regrantors in the comments, helping everyone better understand these proposals.
• The micro-regrantors overall enjoyed the process. From our feedback form (n=17), the average response to “did you enjoy being a micro-regrantor?” on a scale of 1 (hated it) to 3 (neutral) to 5 (loved it), the mean response was a 3.8, and the median was a 4. Some anonymized quotes:
  • “People talk on the [EA] Forum all the time about grantmaking, some suggesting it is wizardry for the talented few, others suggesting it should be for the democratic-EA masses. Unsurprisingly, it turns out the truth may be somewhere in between. I took on the role mainly for the educational (to me) / experiential value, and I wasn't disappointed with what I got in return for my time investment.”
  • “[I enjoyed] having to seriously think about what projects will get traction and stand a good chance of making impact, [as well as] engaging with other projects from a microgranting perspective - purpose to my engagement.”
  • “I aimed to build a small portfolio of projects and I did build it. It was fun!”
• We received a lot of helpful feedback about the Manifund site that might have otherwise been difficult to tease out of the average user.

We were really happy with how the micro-regranting program went. But some things went less well than we would’ve hoped:

• Most of the projects needed more than $500, which meant that micro-regrantors needed to coordinate with other micro-regrantors — or expect follow-on donations from external folks. Both of those happened, but weren’t done nearly so well as possible. Some ideas we had for this in the future:
• Have both more-concerted coordination points and more coordination points overall. We had a channel in our Discord for micro-regrantors, which was really great for coordination between a few of the micro-regrantors who used it, but didn’t work well for most of them. Some possible ideas: a video call with all of the micro-regrantors to chat about projects they wanted to coordinate on; a dedicated section on Manifund for micro-regrantors to discuss grants; a shared email thread; asking micro-regrantors slightly more strongly to join & use the Discord; etc.
• Reach out to external donors. We had at least one large donor contribute additional funding based on decisions made by micro-regrantors, but there were obvious steps we didn’t take to seek out additional large donors; e.g., email some we know, make an announcement on the EA Forum, etc. Part of the problem here was that we were pretty time-crunched, which constrained the actions we felt comfortable request of external donors.

Some of the micro-regrantors found the structure of the impact market (including the valuations, auction mechanism, equity structure, retroactive funding, etc) extremely confusing. This was mirrored by some of the feedback we received from project creators, and we intend to reduce the confusion & complexity in future rounds.

Some of the micro-regrantors strongly disliked the framing of an impact market as encouraging micro-regrantors to be profit-seeking. We probably could’ve framed this better in our request for micro-regrantors, and selected for those who’d be especially interested in the profit-seeking component of an impact market.

The timing was pretty tight — we would’ve liked to give micro-regrantors a few more weeks to sift through projects and make decisions, which would’ve also let them coordinate more and let us contact some external donors.

Again — overall, we were incredibly happy with the results of the micro-regranting program. We intend to run additional programs like this in the future.

Big thanks to Anton Makiievskyi, who funded all of the micro-regrantor budgets — and allocated an extra $5k to Jason’s budget after noticing Jason’s deep & thoughtful comments on the platform and on the Discord. (And that sort of dynamic is exactly the sort of thing we were hoping would come out of the micro-regranting program!)

Manifold Community Fund

We ran an impact market for Manifold community projects from November 2023 to February 2024. We had some ideas about changes that we wanted to make for the (then upcoming) ACX Grants 2024 impact market, and thought this would be a good way to test those on a smaller scale, in addition to a way to get some cool contributions to Manifold.

In the end, we paid out $16k to 12 projects:

Here are some changes we tested on the structure of impact markets, and how we think they went:

• Ongoing evals and payouts: instead of allocating all of the prize money at once at the end of the program, we awarded prize money (i.e. made buy offers) once per month, for a total of three evaluation rounds. We think this provided useful feedback for creators, as evaluators generally wrote comments about why they valued the project the way that they did, but that the actual offers were pretty irrelevant. At each intermediate evaluation, we were only valuing work done so far, so expected valuation was always higher than the offers and no offers were actually accepted until the final round.
  • We also learned that evaluation takes a long time and isn’t that fun—participation in evals from the Manifold team went down each month, and in particular the final evals were delayed by an entire month.
• Automated Market Maker (AMM): for previous impact markets we had run, there was very little active trading on the certificates after the seed round, perhaps because the only way to trade was via limit orders so you always needed to find a trade partner. To make active trading easier, we implemented an automated market maker. This did facilitate more active trading, but overall didn’t seem worth it. For one, all of the AMMs were overpriced, perhaps because we implemented buying and selling but not shorting, which made betting against a project if you hadn’t previously bet for it impossible. Additionally, some equity and some USD was needed to seed the AMM, which meant that the seed round valuations had to be artificially inflated.
• No auction: in order to simplify the fundraising process, we eliminated the auction[link to about] from the seed funding process. Instead, creators just chose the amount they wanted to raise and made an offer at the implied valuation. This was indeed a simplier mechanism to understand, but put a lot more pressure on the creator to set their parameters wisely, because their funding minimum and their funding goal are the same number, and choosing it requires trading off probability of raising any funding at all with the chance to raise more money. On the other hand, with the auction, they can set a low minimum and let the market give them more if it so pleases. All-in-all, this change seemed to cause people more confusion, not less.

So, most ideas we tested with the Manifold Community Fund we did not end up bringing to the ACX impact market. There were, however, some really awesome projects that came out of this, specifically Case’s contributions to the Manifold codebase and wasabipesto’s updates to Calibration City.

Overall, this updated us against impact markets for this kind of use case. As with the ACXG impact market, we were disappointed with the amount of investor participation upfront, and throughout the Community Fund. The best projects to come out of this didn’t actually get any upfront funding, which suggests a simple prize would have worked just as well in this case. And making an easy-to-understand UX for both the creators and the investors in the impact market remains an unsolved problem.

ChinaTalk Essay Competition

ChinaTalk is a twice-weekly newsletter and podcast on China, technology, and US-China relations to an audience of over 35,000, hosted by Jordan Schneider. They have previously received a $17k regrant on Manifund.

Late last year, we partnered with ChinaTalk to sponsor and host their essay competition:

ChinaTalk has teamed up with Manifold to give away $6,000 in prize money to essays making a bold prediction on the future of China. We’ll choose winners in February, 2024.

The top 3 essays will win a cash prize and will be interviewed as a guest on the ChinaTalk podcast to discuss their essay with Jordan.

We’re inviting you to submit an essay making a bold prediction on a subject that ChinaTalk’s audience would connect with.

We agreed to partner on this because:

• We wanted to try to make “impact certificates for essay contests” work. We’d previously tried this with OpenPhil’s AI Worldviews Contest; however, we didn’t receive many submissions (and none of the winning ones) through our platform, as we didn’t secure an official partnership with OpenPhil in time.
• Chinatalk offered to take on most of the promotional and organizational work, in exchange for Manifold paying for their time to do so. As promotion and organization are two of the most expensive pieces of organizing such a prize competition (with the third being the cost of the prize itself), we were curious to see how outsourcing this would turn out
• We wanted to commission high-quality essays that would make good use Manifold’s prediction markets. We’d seen other competitions (like the ACX Book Reviews Contest) produce extremely good essays that got a lot of organic virality, and thought ChinaTalk was well-positioned on to do this given their reach and expertise in the subject matter.

In the end, the competition was won by Lily Ottinger, with her piece "Sino-Soviet Split 2.0”. While we learned a lot from this partnership, I think that Manifund did not achieve the goals we’d set out to with this competition, for a few reasons:

• The impact certificate framing was confusing to the competitors; our site also presented some technical difficulties for essay submission (which we’ve now fixed). Moreover, we didn’t catch the attention of any investors; the use case of “speculating on the best essays” did not seem to catch on.
• We received fewer submissions than we’d hoped for, and most submissions did not make use of concrete predictions or markets in the way that we’d wanted. Jordan himself did write up a great post with concrete predictions, but this was largely orthogonal to the essay submissions themselves.
• Though the ChinaTalk team (both Jordan and Caithrin Rintoul, who handled much of the competition logistics) were professional and excellent to work with, any kind of partnership across orgs is fraught — there are always many delays in communication and nuances to work out. In particular, this competition was something like our third priority (behind the Manifold Community Fund and ACX Grants), so we were less responsive and focused on making the competition go well.

People Updates

Manifund is growing! For the first time since we started our work, we’ve brought a new full-time hire: Saul Munn, to run strategy and operations for Manifund. You might remember Saul from his work on Manifest last year, which he’s returning to organize this year; he also cofounded the OPTIC intercollegiate forecasting tournament.

We also engaged two folks to help us out as consultants:

Dave Kasten planned and conducted a variety of interviews with charity leaders outside of EA, to see whether they might be interested in working with Manifund on impact certs.

Lily Jordan wrote up explanations of how Manifund operates, kicked off our ACX Grants microregranting program, and improved a bunch of site UX.

Thanks to Dave & Lily for all your help! We’d love to publish the research they’ve done for us at some point; in the meantime, you can view drafts here and here.

Finally, Austin has officially left the Manifold team, to focus on Manifund and related work. On a related note, we’re considering rebranding away from “Manifund” so that the separation between Manifold and Manifund is more clear (the two orgs are separately incorporated with separate finances, and no longer share employees).

Site Updates

Throughout this quarter, we’ve generally been more focused on wrapping up funding rounds which required a lot of operational support than with site changes. Still, here were a few significant features we added recently:

Comment reactions: you can now react to comments with emojis, or even leave tips on comments which go to the commenter’s charity balance.

Better agreement display: now, the grant agreement displays the grantee’s agreement as a signature rather than simply a checkbox, and displays Manifund’s countersignature.

Draft mode: for the ACXG impact market, we added a “draft mode” that projects can be in before they’re published as proposals. In this mode, the creator can save edits to their project over multiple sessions, rather than needing to publish immediately.

Where we’re headed in Q2

With the ACX Grants impact market and other rounds wrapped up, we now want to take a step back to explore, “what do we need to get to product market fit”? We have a bunch of ideas on directions we can take Manifund; over the next quarter, we’d like to get more concrete signal about which of these are promising. Some of the initiatives we already have in flight:

Manifest 2024

Manifest is a festival for forecasting and prediction markets. It’s more of a Manifold event in spirit, but it’s being planned by the Manifund team — co-led by Rachel and Saul! It officially kicks off Jun 7-9, but LessOnline and Summer Camp start the week before; we expect this to be quite busy with this in the next few weeks leading up to the event.

(Get your tickets at https://manifest.is; it’s going to be a blast!)

AI Safety Regranting

We’re renewing our AI Safety regranting program, allocating ~$1.6m to 6 world-class experts in the field: Adam Gleave, Dan Hendrycks, Evan Hubinger, Leopold Aschenbrenner, Ryan Kidd and Neel Nanda. More details about this soon, but we’re excited to be running this again and hope to scale it up, with more funding and regrantors this year!

Mixtral Challenge (with Trenton Bricken & Dwarkesh Patel)

On a recent podcast episode, Dwarkesh, Trenton and Sholto suggested the idea of having a prize challenge to better understand the open-source Mixtral model, based on the success of the Vesuvius challenge. This got memed into existence on Twitter, with Nat Friedman and others offering $45k in prize funding for this challenge. We’re working with Trenton and Dwarkesh to run this competition on Manifund, where we host the contest site and handle funds transfer, while Trenton serves to coordinate and judge.

Fiscal sponsorship

We’ve been talking with orgs like Asterisk, PauseAI, and Sage about providing a fundraising platform and fiscal sponsorship relationship (aka 501c3 tax deductability for donations to these orgs). We’re already informally sponsoring many of the projects on our site, but providing this as a proper service solves a pain point for orgs who need this to receive donations.

EA Funds collaboration

We’ve been in close contact with Linch and Caleb of EA Funds, as their work (especially on the LTFF) are the closest in nature to what Manifund has tackled. There are a bunch of ways Manifund and EAF could work together; we’d be excited to combine their grantmaking expertise with our focus on delivering great software.

Manifold Charity program

We’re still supporting mana donations to charity based out of our original Future Fund grant, but we’re likely to sunset this program once Manifold supports real money prizes via sweepstakes.

Other

People we’d love to chat with

As we scale up, there’s a lot of folks we’d like to talk to and learn from. If this describes you (or someone you can intro), reach out to austin@manifund.org!

Creators who’d like to launch a grants program or prize challenge for their audience (eg Scott Alexander, Dwarkesh Patel)

Grantees and orgs who are curious about working with Manifund, whether as a fiscal sponsee, impact cert founder,

Medium-sized individual donors ($10k+/year) who find Manifund’s approach exciting and want to fund one of our programs, whether that’s regranting, impact certs, individual projects or something else

Links & callouts

Aaron Silverbook’s Lumina Probiotics has been on a tear lately, with a very hyped launch; coverage from Scott Alexander, Yishan Wong, Richard Hanania, and other bloggers and press; and sales going through the roof — we’re glad to have been an early investor!

Alex Toussaint’s mosquito-killing drone project recently blew up on Twitter, with a video demo of the sonar getting 1M views and attracting $2500+ in crowdfunded donations from 10 folks.

Johnny Lin and Joseph Bloom, two separate Manifund grantees, have teamed up. Neuronpedia now provides web tooling for sparse autoencoders, using visualization tools similar to those used by Anthropic, and has since received follow-on funding from LTFF.

Joel Tan of CEARCH (ACXG 2024 grantee) published this excellent report on the landscape of EA meta funding.

Nonlinear Network is running a funding circle for AI safety and related fields, with hundreds of project applications, reviews from experts, and >$1m moved in donations in just a couple weeks.

Alexander Berger of OpenPhil recaps 2023 and writes about their funding plans for 2024.

Ambitious ideas we’d be excited about

We’ve been brainstorming on a bunch of other things we could do; here are some very rough notes. If you’d like to help make these happen, comment below or reach out to austin@manifund.org!

Extending the regranting model

“celeb regranting as a service” — manufacture more “ACX Grants & Emergent Ventures”

See this proposal we’d sent to Dwarkesh Patel: “Dwarkesh Grants” proposal, which led to the Mixtral Challenge collab described above

microregrantors — crowdsourcing EA donation decisions

Building a robust impact marketplace

Invest Anthropic & OpenAI stock — instead of certs on tiny orgs with little data, would there be more excitement for certs on larger orgs? This might just look like allowing

Manifund DAF — medium-sized (~$10-500k) independent donations inside EA are currently a hodgepodge; we could build up an ecosystem of medium-sized donors and projects seeking funding.

SAFIE (Standard Agreement for Future Impact Equity) — The SAFE simplified early startup investing. Can we do this for early charity funding — and make it backwards compatible with regular grants?

Funding more home runs

Hosting amazing projects (”hits”) — in venture capital, a few hits dominate the portfolio; anecdotally, this seems to be the case in early-stage charity funding too. How do we find these projects and encourage them to come to Manifund?

E/Accelerator — or do we try to directly create more of them, especially with a bias towards software (Manifund’s comparative expertise in the EA ecosystem?)

EA Common App — when Manifold was first getting started, applying to a bunch of different funders for mostly the same information was kind of a pain. Like the collegiate Common App, can we make one for “EA organizational funding”?