Should there be just one western AGI project?

I think an important consideration being overlooked is how comptetntly a centralised project would actually be managed.

In one of your charts, you suggest worlds where there is a single project will make progress faster due to "speedup from compute almagamation". This is not necessarily true. It's very possible that different teams would be able to make progress at very different rates even if both given identical compute resources.

At a boots-on-the-ground level, the speed of progress an AI project makes will be influenced by thosands of tiny decisions about how to:
 

• Manage people
• Collect training data
• Prioritize research direcitons
• Debug training runs
• Decide who to hire
• Assess people's perfomance and decide to should be promoted to more influential positions
• Manage code quality/technical debt
• Design+run evals
• Transfer knowledge between teams
• Retain key personnel
• Document findings
• Decide what internal tools to use/build
• Handle data pipeline bottlenecks
• Coordinate between engineers/researchers/infrastructure teams
• Make sure operations run smoothly
   

The list goes on!

Even seemingly minor decisions like coding standards, meeting structures and reporting processes might compound over time to create massive differences in research velocity. A poorly run organization with 10x the budget might make substantially less progress than a well-run one.

If there was only one major AI project underway it would probably be managed less well than the overall best-run project selected from a diverse set of competing companies.

Unlike the Manhattan project - there's already sufficently strong commercial incentives for private companies to focus on the problem, it's not already clear exactly how the first AGI system will work, and capital markets today are more mature and capable of funding projects at much larger scales. My gut feeling is if AI was fully consolidated tomorrow - this is more likely to slow things down than speed them up.

Executive summary: While centralizing Western AGI development into a single project could have major strategic implications, the authors argue it's unclear whether this would be beneficial overall and tentatively conclude it would be net negative due to risks from power concentration.

Key points:

1. Race dynamics: Centralizing would reduce competition between Western projects but has unclear effects on US-China competition, potentially intensifying rather than reducing that race.
2. Power concentration is a major concern: A single project would concentrate unprecedented power, reducing pluralism and increasing risks of coups/dictatorship, though good governance design could partially mitigate this.
3. Information security implications are ambiguous: While fewer projects means less attack surface, a single project might attract more serious attacks and earlier attempts at theft.
4. Rather than pushing for centralization specifically, efforts should focus on improving outcomes under either scenario through robust safeguards and governance structures.
5. The authors reject arguments that centralization is inevitable, noting multiple projects could remain economically viable and government involvement doesn't necessitate full centralization.

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.