There should be an AI safety project board

By mariushobbhahn @ 2022-03-14T16:08 (+24)

The AI safety community has gotten much more accessible over the last few years. The AGI safety fundamentals curriculum provides a great overview of the most important pieces and you can work through Richard Ngo’s Alignment research exercises on your own. The AI safety camp enables you to participate in AI safety projects with good supervision and you can join the AI safety slack.

People often post the results of their work on the Alignment forum and sometimes post about starting a project (see e.g. Ethan Perez’s post). Effective thesis can provide you with supervisors for Bachelor’s and Master’s students but might not be able to find projects for people from industry or Ph.D. students.

However, I think there is still a niche to be filled. Most collaborations in AI safety come through inaccessible personal networks, e.g. personal messages on the forum or conversations at EAG. This means that new people often don’t have an avenue to participate in projects. While the AI safety camp is very good, it is only once a year and is not designed for more intense long-term projects. For example, it is often hard to know which PhDs would be interested in academic collaboration.

Therefore, I propose an AI safety projects board. On this board, anyone could post a short description of an AI safety-related project, its requirements (previous experience, decent coding skills, etc.) and further details (X hours per week, funding available, intended for an academic publication, etc.). Furthermore, it would enable busy senior people in the field to suggest projects for less experienced members of the community, e.g. they post the idea and explicitly state that they don’t have time for extensive supervision.

I currently neither have the time nor the web-building skills to develop and maintain such a tool but maybe someone else is interested in setting it up. Some of the AI safety & longtermist community managers might want to offer support and some longtermist and AI safety funders could provide financial support to anyone who is interested and capable.

If you want to get informed about new posts you can follow me on Twitter.

Jamie Bernardi @ 2022-03-16T18:19 (+6)

Thanks for proposing this, it's a great idea!

I am quite interested in exploring this more. We currently have a prosaic list of potential capstone projects, intended to be done by AGI Safety Fundamentals programme participants after the course. I think forms a prototype of this proposal, however one main difference is it's not really about cutting-edge research, more like skilling-up and fit-testing ideas.

Some questions I am interested in exploring further to see where the needs are for this meta-project:

Do AIS researchers have a list of projects ready-to-go that they don't publicly advertise?
My guess: probably? Most people have too many threads they want to pursue in not enough time - in my experience it doesn't take long to get to this point in a research career.
If researchers do have lists of project ideas, why don't they share the lists already?
My guesses as to why:
1) They haven't (all) written them down in a sharable format
2) There's no logical place to put them (e.g. a board doesn't exist), so no obvious inspiration to publish a list or proof that it's impactful.
3) Working on a project requires a lot of context on previous work done in the same vein. Supporting people who want to pursue a project idea would be a lot of work for the idea-originator, to get the project off the ground.
With a public board, the researcher doesn't get to vet candidates who might try to take the projects on, which might make it difficult to manage deciding who they want to work with or not on top of doing their regular work.
Vetting and management/ops are some of the things AI safety camp and CERI summer research fellowship offers, for that reason.
Why don't other fields have public lists of ideas? E.g. is there a list of research ideas for nuclear fusion? I don't think there is - so other fields also do project coordination behind closed doors, in academic institutions and conferences.
My guess as to why: other fields may just not be well coordinated and maybe we could just do better. They also don't have as much untied funding for individual researchers to spring up and take stuff on, outside of universities/institutions.

Interested if you or others have thoughts!

mariushobbhahn @ 2022-03-16T19:09 (+3)

I think there are multiple reasons:
a) If there is no explicit board people just don't do it because there is no norm and it's work.
b) If you post about your research it might be scooped.
c) People haven't written up the projects in a sharable format.
d) You might not find the right people on such a board?!

I think there are many failure modes for such a board but it seems worth a try at least.

I guess most other fields don't have such a board because sharing culture isn't very strong and you're incentivized to be secret and not share to achieve personal goals.

Esben Kran @ 2022-03-31T04:33 (+2)

Great proposal! I was just sent this post today but we are already working on something along these lines and will publish more during the coming weeks about it (including the web app part).

We are currently working on a spreadsheet-based MVP that I'll share very soon on which we are getting feedback on the structure and content. If anyone wants to either be part of a user interview, write down their ideas or just discuss the idea platform itself, my calendar is wide open and you can send me an email at esben@apartresearch.com at any time.

Additionally, we have worked out a design and started development of a web app (a producthunt-like design with specific choices for AI safety) but want to approve the design thoughts and more before committing 100%, alas the spreadsheet-first method.

I agree with your analyses that it seems much more like a "we are missing a platform to easily incentivize sharing ideas" than a culture problem. A clear example of this was the FTX Future Fund Idea Competition. Loads of ideas present with no problems in sharing them (especially for a reward).