Lessons learned and review of the AI Safety Nudge Competition
By Marc Carauleanu, Chris Leong @ 2023-01-17T17:13 (+5)
TL;DR: We ran a competition to encourage people to stop procrastinating on their AI Safety projects. We had 76 applicants out of which over 40% (31 participants) completed their goal by the end of October and were added to a draw to win a monetary prize. The vast majority of the participants that completed their goals found this competition to be useful. We learned that marketing is paramount for successfully running such a competition, that competitions of this kind would potentially be better run at EAG events and that they could be a good way of getting more people to subscribe to EA newsletters.
Introduction
The AI Safety Nudge Competition aims to encourage people to do things related to AI Safety today instead of procrastinating it into the future by allowing them to enter into a draw if they complete the goal that they set for themselves.
Participants defined a specific goal for themselves, examples include:
• Finish reading Superintelligence
• Finish writing up a relevant blog post
• Organise a local dinner for people interested in AI Safety
If they completed the goal they set out for themselves, they were entered into the draw:
• ten prizes of $100
• two prizes of $100 specifically for Australia and New Zealand
You can see the list of winners here.
Downside risks
We started by asking the applicants if they think their project could potentially have downside risks such as:
• Outreach to famous people, politicians, the media, children, high-net worth individuals, top AI researchers
• Projects that could be controversial, come with significant down-side risks or could produce negative PR
• AI Safety projects with high capabilities externalities
Only 1.3% of the applicants were uncertain about the downside risks of their project and the vast majority self-reported to be certain their project doesn’t fall in this category.
Counterfactual impact
We continued by asking applicants to rate from a scale from 0 to 10 how likely they thought they were able to achieve their goal by the end of October if they did and did not enter this competition to gauge how strong of a nudge they thought the competition would offer. We made it clear that this was only for informational purposes and that it did not affect their application.
Average: 7.73
Average: 4.21
The average response to how likely they think they would be to achieve their goal by the end of October if they entered the competition is almost 2 times bigger (1.83x) than the average response if they wouldn’t enter the competition.
We asked the same questions in the form they had to complete after they finished their project to be added to the prize draw to compare the results.
Average: 7.03
Average: 4.54
The average response to how likely they think they would have been to achieve their goal by the end of October if they entered the competition is 1.54 times bigger than the average response if they wouldn’t have entered the competition.
These results indicate that the participants that finished their project found this competition to be useful for achieving their goal.
Participants who didn’t finish their project in time were encouraged to fill in a form to provide more details.
Most of the people that filled in that form mentioned procrastination (33.3%) as the main reason they didn’t manage to finish their project in time while the second biggest reason (22.2%) is that they were busier than they expected.
The same questions related to the expected counterfactual impact of entering the competition were posed to the participants that didn’t finish their project.
Average: 5.11
Average: 1.44
The average response to how likely they think they would have been to achieve their goal by the end of October if they entered the competition is over 3 and a half times bigger (3.54x) than the average response if they wouldn’t have entered the competition.
We also asked them how we could improve:
- More frequent reminders, maybe pairing people up for goal-buddies & weekly 15m check-ins -- would have made me more accountable
- Maybe more frequent reminders? Even though I ultimately slowly gave up on my goal, every time I received an email from you, I had a slight boost in motivation.
- I think the single reminder email was good!
Newsletter
At the end of the registration form we asked the applicants if they would like to subscribe to our newsletter and 60.5% responded positively.
This indicates that running competitions of this kind could also be a very useful way of getting subscribers to EA newsletters. [1]
Marketing
We underestimated the importance of marketing for a competition of this kind as we announced the competition before we had the marketing materials ready and this resulted in a slow start. As a result of this, we quickly created a poster and a pitch for the competition and shared it in various AI Safety groups on Slack, Facebook, Discord and Twitter, after which more people started applying. Running a competition like this could have a wider impact if it is announced at EAG events or after big EA book launches.
We sent an email in the middle of October and another one a week before the end of October with science-backed productivity tips and a reminder for the participants to complete their project.
Main lessons
- Prepare marketing materials before launch
- Announce the competition earlier (for ex newsletters)
- Look for big EA events which could bring in a lot participants
-What We Owe the Future/other big EA books launches
-EAG conferences
- This competition could be scalable and reproducible (create a template)
-Google folder with files that people can copy[2]
-Documents with advice on how to run it
-Possibly an Asana template
- Make an EA forum post on the competition at the end
- If you are a small student group maybe running a competition over the summer could be a cost-effective path to impact
- This type of competition could keep people engaged over the summer break if you don’t have much organizing capacity
- Some variants could possibly be more scalable and reproducible (as they do not rely on the honour system):
-Blog post nudge competition
-Audible/book reading nudge competition
-AGISF nudge competition (ask for the email of the facilitator for confirmation)
Conclusion
Overall, the AI Safety Nudge Competition showed promise in nudging people to complete their AI Safety projects but competitions like this could have a wider impact with more focus on marketing. We created a folder with all of the forms, documents and email templates we used so that other people could easily run a competition of this kind. Feel free to contact us if you want access.