Measuring how often an AI research agent fools itself: an open protocol (seeking feedback and funding)

By Igor @ 2026-10-09T14:17 (–4)

Disclosure: drafted with AI assistance, like the project itself. I am responsible for every claim.

AI agents increasingly run research on their own and report discoveries. As far as I know, there is still no routine way for an overseer to know in advance how often a given agent would announce a discovery where none exists.

The October 2025 episode is a clean example. GPT-5 was said to have solved open Erdős problems; it had found existing solutions in the literature. The model did something useful. The claim about what it did was wrong, and nothing in the pipeline was built to catch that.

Since August 2026 I have been building SDCP (Scientific Discovery Control Plane), a working prototype of oversight for AI-driven discovery. It is run by one person and two AI models from different companies.

The rules

1. Pre-register before computing. Hypotheses, thresholds and analysis plans are frozen and hashed before data are seen.
2. Calibrate on null worlds. Every discovery procedure runs on worlds with no hidden structure, so its false-discovery rate is measured with confidence bounds, not assumed.
3. Seal. A frozen model version runs once; its forecasts are committed by hash, unopened.
4. Separate generator and evaluator. A second model reviews the first, can stop it, and is the only role that opens sealed results.
5. Correct in the open. Errors are appended, never rewritten. The system is allowed to say "my language cannot express this" instead of forcing a fit.

What it has produced so far

• Calibrated false-discovery rates for the engine on null worlds. Caveat: these are synthetic worlds I designed myself; the calibration has not yet been tested on worlds built by anyone else.
• Blinded runs opened only by the evaluator. Two caveats: in the blinded comparisons run so far, the new engine did not beat a simple baseline; and the blindness is procedural, not technical (one person, one computer, no hardware isolation), so it rests on discipline and logging rather than on enforcement.
• A record of my own failures: several "novel" ideas withdrawn after prior-art review (most turned out to be known: clustered symbolic regression, model-based recursive partitioning, Automatic-Statistician-style search), and several engine versions rejected for failing calibration.
• A first pre-registered prediction on public NHANES data, confirmed in an independent later cohort. It is preliminary, and the effect is probably already known.

No new discovery yet.

What I am asking for

• Critique. Where is this protocol weaker than it looks? What existing work already does this better?
• Datasets. Holders of data who would let a blinded test run on it. The data would go to the independent evaluator, not to the AI that builds the engine: once the builder has seen a dataset, a test on it is no longer blind.
• Funding. The project is self-funded and the bottleneck is AI compute. The Manifund goal is $5,000: six months of compute ($2,400), a partial stipend and a buffer. The page is open until 8 November.

Links:

• Manifund: https://manifund.org/projects/sdcp-an-open-protocol-for-checking-ai-research-agents-discovery-claims
• Project page: https://claude.ai/artifact/UYFp2CXdF78dLEPTU4H6VU
• Contact: eezzydizzy@gmail.com