Ground Zero: Frontier Research to Policy Startup

By Sarthak.Agrawal @ 2026-10-14T13:53 (+2)

Status:


The Setup

A lot of people want to get into frontier AI policy, and I have some bad news. Frontier AI policy isn’t a field. It’s not something you can get into. There’s nothing you can read that will tell you what to do, why to do it, and how to do it. Look around you; everyone who has any idea of what might work is currently trying to implement it; the state of the art is locked down in legacy outlook email & tattered field notebooks.

And it gets worse. You might think that you can dive into the work that is public, the bills that have been introduced, the executive orders that have been signed, the concrete policy wins that other people are promoting as the right thing to do. But you can’t take any of that at face value. Technical researchers and engineers are used to extremely high transparency contexts — you can audit an algorithm and study source code.

The one professional field that is well suited to approaching policy strategically is the legal field — law is extremely opaque, there is virtually no transparency into the inner workings of legal conflicts, and thus far harder to assess the competence of lawyers. This is also why we can’t just hire armies of lawyers to work on these problems and expect that to work.

But, there’s good news. (Call to action in the postscript!)

The Bitter Lesson of Policy

The bitter lesson of policy is that it doesn’t matter how much money you throw at lobbying to accomplish your goals; in the long run, the most resilient institutions will win. The US Constitution was created in 1789, and has had 3 major revisions since then that stand out to me personally:

  1. The Marshall Court: the Supreme Court from 1801-1835 under Chief Justice Marshall. He gave SCOTUS judicial review and made it an equal branch to the Congress and the Presidency.
  2. Reconstruction: the period of rebuilding the USA after the civil war, largely remembered as the time 4 million freed slaves constituting 12% of the total US population were integrated into society.
  3. Presidency as Sovereignty: the balance of power in the American federal government has become increasingly & overwhelmingly concentrated in the individual person of the President.

A reasonable assessment of the US Constitution is that despite the best of intentions, the framers ultimately failed to craft a balance of powers and have led to a world they would have hated. Another reasonable assessment is that the US Constitution is the single most brilliant piece of legal architecture in the history of mankind, that has stood the test of time and evolved alongside the most powerful nation in the world.


An unreasonable assessment would be to claim that any of the politicking around the specific concerns people had about their lives in the 1800’s actually matters to us in 2026. Maybe it did — maybe you can prove that advocating for a certain regulatory strategy on printing presses is the pivotal legal implementation that enabled the 1st amendment to have the broad, fundamental role it does now in 2026. That would be an incredibly helpful and useful analysis, and I am extremely excited to read it.

Until then, my corollary to the bitter lesson is the bootstrap hypothesis: the expected value of all policy on the long-term future lies in the effect of your policy engagement on the norms, institutions, and people around you. Yes, this is different because we have a few years, not 200. But I don’t think that matters.

  1. Unless you can solve inner and outer alignment with purely technical means,  any AI control solution will involve a legal architecture. Short timelines and fast takeoff imply you should prioritize the long-term resilience of that legal architecture, since you will quickly become obsolete.
  2. As policymakers are accelerated with AI, the amount of policy research, analysis, and implementation that can be done within the coming several years could eclipse what has been done in decades past.
  3. The closer the world gets to a potential existential risk, the more effort that will be poured into this. Windows to act for policy come rarely, and when they arrive, you want to be equipped to propose the best possible institutional design.

This is why I’m so excited about advancing state capacity as an AI safety strategy; I suspect the real existential risk reduction is the mechanism design we architect along the way. The 3 examples I gave of revisions to the US constitution are forks in the nature of the American state. They are changes in the underlying rules of the game. It’s easy to hate the player, and it’s noble to hate the game.

But it’s best to change the game to the one you can win. Mechanism design is all you need.

 

The Beautiful Irony of Lobbying

The beautiful irony of lobbying is that the more skilled an agent is in achieving extremely complex strategic goals, the more challenging it is to identify them. This is well known in entrepreneurship as “hiring is hard”, in poker as “if you don’t see the fish, you are the fish”, and in AI Control as the scalable oversight problem.

Because the most complex strategic goals, such as in policy making, cross technical and social domains, the individuals capable of enacting them are even harder to identify. Exemplary success in these areas is indicated by the lack of a maker’s mark.

Policy isn’t just an opaque environment. Policy is an extremely competitive social environment filled with complex social signaling, counter-signaling, and intrigue. I believe the best policy professionals behave with exceptional integrity in accomplishing their goals, communicating who they are, and what they want to achieve.

That is extremely compatible with keeping your cards close to your chest. And unluckily for those who want to hire lobbyists, the most sincere lobbyists know they need to fly under the radar to achieve their policy wins. In a world where many are here for the wrong reasons, the people with a commitment to America and their values are the least likely to claim credit. It makes sense for a startup founder to brag about their work — ego & grandstanding are extremely well correlated with raising funding. But it does not make sense for a lobbyist to do the same, because raising funding typically does not outweigh the cost to their work.

 

The Bad News about Strategy

Take the problem you care most about in the world. It is either on a path to being solved without your effort. Or it isn’t. If the problem is going to be solved, then you have no counterfactual impact by working on it; you aren’t actually doing anything useful. But if the problem is not going to be solved, the sum total of the field without you is not enough to solve it.

In some domains and application areas, this isn’t a problem. If you need to manufacture a good, adding material resources to produce more things will help. But Frontier AI policy is not a domain with ideas in need of resources; it is a domain with resources in need of ideas. Every idea that has been done so far has not worked yet; there is no evidence that it will ever work. In this world, adding more capital in the form of money or labor that executes more of what is already being done is not changing the field.

Doing more of something that isn’t proven to be working does not increase or decrease the odds of success. Because there is no concrete evidence that anyone working on Frontier AI policy can solve the problem, you have no reason to believe that the existing experts can do it.

The bad news about strategy is that you need to believe in your own strategy before you can evaluate anyone else’s.

In the defense of those of us who have tried this work, there’s no way to know if it will work before it’s too late. That’s the bad news, and that’s why we need you. We need people who understand the nature of frontier AI, want to work on policy solutions, but aren’t practitioners or researchers. We need policy research engineers to build the bleeding edge of Frontier AI policy.

That’s the start of the good news — if you’re down to treat your policy career like a startup founder, always be out of your depth, earn extremely little money, and have extremely little meaningful feedback on whether you are doing anything useful, Frontier AI policy is extremely high impact.

But you have to remember the bad news. It’s not a job, certainly not a career. It’s a lifestyle.