EU policymakers reach an agreement on the AI Act

Thanks for sharing! Speaking as a European I think this is a pretty good summary of the latest state of events.

I currently expect the full text of the Act as agreed on in the trilogue to be published by the EU some time in January or February.

Another worrisome+unclear reported exemption is for national security.

Thanks for sharing!

Küspert also says “no exemptions,” which I interpret to mean “no exemptions to the systemic-risk rules for open-source systems.” Other reporting suggests there are wide exemptions for open-source models, but the requirements kick back in if the models pose systemic risks. However, Yann LeCun is celebrating based on this part of a Washington Post article: "The legislation ultimately included restrictions for foundation models but gave broad exemptions to “open-source models,” which are developed using code that’s freely available for developers to alter for their own products and tools. The move could benefit open-source AI companies in Europe that lobbied against the law, including France’s Mistral and Germany’s Aleph Alpha, as well as Meta, which released the open-source model LLaMA." So it’s currently unclear to me where the Act lands on this question, and I think a close review by someone with legal or deep EU policy expertise might help illuminate.

It's a shame this is so unclear. To me this is basically the most important part of the act, and intuitively seems like it makes the difference between 'the law is net bad because it gives only the appearance of safety while adding a lots of regulatory overhead' and 'the law is good'.

Thanks for this overview, Trevor. I expect it'll be helpful– I also agree with your recommendations for people to consider working at standard-setting organizations and other relevant EU offices.

One perspective that I see missing from this post is what I'll call the advocacy/comms/politics perspective. Some examples of this with the EU AI Act:

• Foundation models were going to be included in the EU AI Act, until France and Germany (with lobbying pressure from Mistral and Aleph Alpha) changed their position.
• This initiated a political/comms battle between those who wanted to exclude foundation models (led by France and Germany) and those who wanted to keep it in (led by Spain).
• This political fight rallied lots of notable figures, including folks like Gary Marcus and Max Tegmark, to publicly and privately fight to keep foundation models in the act.
• There were open letters, op-eds, and certainly many private attempts at advocacy.
• There were attempts to influence public opinion, pieces that accused key lobbyists of lying, and a lot of discourse on Twitter.

It's difficult to know the impact of any given public comms campaign, but it seems quite plausible to me that many readers would have more marginal impact by focusing on advocacy/comms than focusing on research/policy development.

More broadly, I worry that many segments of the AI governance/policy community might be neglecting to think seriously about what ambitious comms/advocacy could look like in the space.

I'll note that I might be particularly primed to bring this up now that you work for Open Philanthropy. I think many folks (rightfully) critique Open Phil for being too wary of advocacy, campaigns, lobbying, and other policymaker-focused activities. I'm guessing that Open Phil has played an important role in shaping both the financial and cultural incentives that (in my view) leads to an overinvestment into research and an underinvestment into policy/advocacy/comms. 

(I'll acknowledge these critiques are pretty high-level and I don't claim that this comment provides compelling evidence for them. Also, you only recently joined Open Phil, so I'm of course not trying to suggest that you created this culture, though I guess now that you work there you might have some opportunities to change it).

I'll now briefly try to do a Very Hard thing which is like "put myself in Trevor's shoes and ask what I actually want him to do." One concrete recommendation I have is something like "try to spend at least 5 minutes thinking about ways in which you or others around you might be embedded in a culture that has blind spots to some of the comms/advocacy stuff." Another is "make a list of people you read actively or talked to when writing this post. Then ask if there were any other people/orgs you could've reached out, particularly those that might focus more on comms+adovacy". (Also, to be clear, you might do both of these things and conclude "yea, actually I think my approach was very solid and I just had Good Reasons for writing the post the way I did.")

I'll stop here since this comment is getting long, but I'd be happy to chat further about this stuff. Thanks again for writing the post and kudos to OP for any of the work they supported/will support that ends up increasing P(good EU AI Act goes through & gets implemented). 

Great summary. 

I was pleasantly surprised at how good this turned out to be, despite it having to be re-evaluated when Chat-GPT came along, despite the objections of major governments. 

The EU Commission is a fantastic organisation. Yes, massive levels of bureaucracy, but the people there tend to be extremely smart and very committed to doing what's best. Just being accepted to work in the Commission requires finishing in the top 1% or less of a very tough evaluation process and then passing a series of in-person evaluations. 

So normally when they produce a proposal, it has been thought through very carefully. 

Of course it's not perfect, and I especially appreciate that the post ends with tangible ideas for how to help make it more impactful. 

Obviously this is an area where we'll need to keep working all the time as AI evolves, as regulations elsewhere evolve. But good to see someone taking the lead and actually putting something tangible in place which seems to be 80/20 what's needed. Maybe this can be the starting point for an even better US legislation ??

At the very least, in my view, the picture has changed in an EU-favoring direction in the last year (despite lots of progress in US AI policy), and this should prompt a re-evaluation of the conventional wisdom (in my understanding) that the US has enough leverage over AI development such that policy careers in DC are more impactful even for Europeans.

Interesting! I don't quite understand what updated you. To me, it looks like, given the EU AI Act is mostly determined at this stage, there is less leverage in the EU, not more. Meanwhile, the approach the US takes to AI regulation still remains uncertain, indicating many more opportunities for impact.

Executive summary: The EU has reached an agreement on regulations for AI systems, including requirements for general-purpose AI systems that could reduce risks.

Key points:

1. The EU's AI Act will regulate general-purpose AI systems and "very capable" models.
2. It requires threat assessments, model evaluations, transparency, and addressing systemic risks.
3. There are questions around exemptions for open-source models.
4. The Act could influence companies due to the size of the EU market.
5. Effective implementation requires expertise in standards bodies and regulators.
6. More policy research could inform catastrophic risk reduction.

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.