Public Weights?

I also like the way you divide up the claims. I think this paper is a really neat demonstration of point 1, and I'm kinda disappointed with the discourse for getting distracted arguing about point 2.

1. Making LLMs public is dangerous because by publishing the weights you allow others to easily remove safeguards.
2. Once you remove the safeguards, current LLMs are already helpful in getting at the key information necessary to cause a pandemic.

I like this way of splitting it up. I think the paper made a good case for point 1, but I think point 2 is greatly overstated. With current tech you would still need an expert to sift through hallucinations and to guide the LLM, and the same expert could do the same thing without the LLM. On this issue current LLM's are timesavers, not gamechangers. 

For this reason I doubt you can convince people to hide their weight now, but possibly you can convince them to do so later, when the tech is improved enough to be dangerous. 

I'm encouraging people to stop using the framing of "democratizing access".

I think this framing is misleading because, given current polls, it's not at all clear that the population (at least of the countries I've seen surveyed) would vote for frontier models to be open-sourced.