Quick proposal: Decision market regrantor using manifund (please improve)

Thanks for the writeup, Nathan; I am indeed excited about the possibility of making better grants through forecasting/futarchic mechanisms. So I'll start from the other direction: instead of reaching for futarchy as a hammer, start with, what are current major problems grantmakers face?

The problem that seems most important to solve: "finding projects that turn out to be orders of magnitude more successful/impactful than the rest". Paul Graham describes funding seed-stage startups as "farming black swans", which rings true to me. To look at two example rounds from ACX Grants, which I've been involved in:

• ACX Grants: Many of the projects look good, but a handful seem to have gotten outlier success; I would count Lars and Will's Valuebase, the Oxfendazole group, and our own Manifold as having gone on to raise millions in further funding.
• ACX Forecasting Mini-grants: Still a bit early to tell, but OPTIC and BaseRateTimes (which we missed!) seem to have hit their goals and continue on to work on cool things.

So right now, I'm most interested in mechanisms that help us find such founders/projects. Just daydreaming here, is there any kind of prediction mechanism that can turn out a report as informative as the ACX Grants 1-year project update? The information value in most prediction markets is "% chance given by the market", which misses out on the valuable qualitative sketches given by a retroactive writeup.

Other promising things:

• Asking grantees to set up markets for their own outcomes; eg "If funded, will we successfully publish a paper that receives >10 citations within 1 year?" this might clarify exactly what goals the grantees are trying to hit.
• Doing some kind of impact analysis for alignment work in past years; imagine a kind of "AI Safety Nobel Prizes" which identify what work turned out to be the most important. This would give future forecasting tools something concrete to predict on.

Some notes (written quickly!): 

1. You write "it can either be an advisory system or it can be a deterministic system," but note that an advisory system misses out on a key benefit of what I understand as ~classic futarchy: the fact that it solves the problem of conditionality not being the same as causality. See here (from my earlier post):
   1. Causality might diverge from conditionality in the case of advisory/indirect markets.^[10] Traders are sometimes rewarded for guessing at hidden info about the world—information that is revealed by the fact that a policy decision was made—instead of causal relationships between the policy and outcomes.^[11]
      1. For instance, suppose a company is running a market to decide whether to keep an unpopular CEO, and they ask if, say, stocks conditional on the CEO remaining would be higher than stocks conditional on the CEO not staying. Then traders might think that, if it is the case that the CEO stayed, it is likely that the Board found out something really great about the CEO, which would increase expectations that the CEO would perform very well (and stocks would rise). So the market would seem to imply that the CEO is good for the company even if they were actually terrible.
   2. There's also a post on this (haven't re-read in a while, only vaguely remember): Prediction market does not imply causation
2. I think getting a metric that's good enough would be really difficult, and your post undersells this
   1. E.g. it should probably take into account the cost. I'm a bit suspicious of metrics that depend on a vote 5 years from now. There are still issues with forecasting on decently long periods, and if I'm thinking about how I would bet on a metric like this, I'd go look at the people doing the voting, I'd be trying to forecast how beliefs and biases of the relevant group would change (which could be useful, but it's not exactly what grants are for), etc. If the point is ability to scale this up, the metric should be applicable to a wide variety of grants. Grant quality could depend on information that isn't accessible to a wide audience (like how much a grant supported a specific person's upskilling), etc. 
   2. But maybe my argument is ~cheemsy and there are versions of this that work at least as well as current grantmaking systems, or would have sufficient VoI — I'm not sure, but I want to stress that at least for me to believe that this is not a bad idea, the burden of proof would be on showing that the metric is fine. 
3. I'm also honestly not sure why you seem so skeptical that manipulation wouldn't happen, but maybe there are clever solutions to this. I haven't thought about this seriously (at least, not for around 2 years). 

Relatedly, 2 years ago I wrote two posts about futarchy — I haven't read them in forever, they probably have mistakes, and my opinions have probably shifted since then, but linking them here anyway just in case:

• Summary and Takeaways: Hanson's “Shall We Vote on Values, But Bet on Beliefs?”
• Issues with Futarchy

I don't buy that someone can jump on the markets and mess with them. Manifold (or better real money) markets have great incentives against this. I am unaware of any time this has happened in a way that it would in a live prediction market.

Manifold markets hasn't solved longterm market incentives yet:
https://manifold.markets/Nu%C3%B1oSempere/this-question-will-resolve-positive-4a418ad86de3#jdUj58EdDXBbF8ZNyXaT
There are even accounts dedicated to messing with Manifold markets:
https://manifold.markets/ButtocksCocktoasten?tab=questions
Manifold doubled mana loan amounts this month to try to mitigate the longterm market incentive issues on the platform