[Link post] Parameter counts in Machine Learning

By Jaime Sevilla @ 2021-07-01T15:44 (+15)

This is a linkpost for https://www.alignmentforum.org/posts/GzoWcYibWYwJva8aL/parameter-counts-in-machine-learning

Pablo Villalobos, Juan Felipe Cerón and Jaime Sevilla (that's me) have released the biggest (AFAIK) public dataset of parameter counts of Machine Learning systems between 1952 and 2020, and discuss some insights in the Alignment Forum.

Model size of popular new Machine Learning systems between 1954 and 2021. Includes n=139 datapoints. See expanded and interactive version of this graph here.

We have received positive feedback so far, so we are linking this here in the Effective Altruism Forum to make it more widely known among a relevant audience.

We intend to work more on this project, and we are keen on having more people join us to cover more ground. We believe this is a good first project for people who are interested in Machine Learning, and specially in Machine Learning scaling laws.

The main compromise is to show up to weekly meetings and read through assigned papers, then note down information in a spreadsheet. Leave a comment or send me a private message if you'd like to collaborate!