20+ tips, tricks, lessons and thoughts on hosting hackathons

By gergo @ 2023-11-06T10:59 (+14)

Epistemic status: Based on one relatively new AIS group's experience, so take everything with a grain of salt!

(Crossposted on the LW forum.)

We hosted a couple of hackathons with Alignment Jams so far and found it a great way to engage people who have previously not been exposed to AI Safety. I’m writing down (partly for myself) some of the tips and tricks I learned during these. Please note that I’m using the word hosting a hackathon, as opposed to organising one, as the lion’s share of work is done by the amazing team at Apart Research/AlignmentJam, who get the speakers, and provide funding and mentoring. That said, hosting a hackathon can still be plenty of work, especially if you are doing it for the first time, so hopefully, this post will help with some of that.

Another caveat is that our AIS group is relatively new and we were mostly using these hackathons for outreach towards students unfamiliar with AI Safety. I think some of the tips will still be useful for more established groups as well, just keep in mind that our theory of change might be different than that of other groups.

Here is my list of “tips and tricks,” vaguely organised around different topics:


Relating to the schedule:

Have a specific schedule in your advertisement/signup form

This is more of a hunch, but it’s reasonable to think that especially for people who haven’t interacted with your group before, they will worried along the lines of “What the hell I am going to do there for 40+ hours?”. My hope is that having a clear schedule (you don’t have to closely stick to depending on the vibe etc. once you are there) might be helpful with this. See our past schedule here (including the signup form), which you are free to copy and adapt for future hackathons if you want to.

Set up milestones for your local site

Similarly to the point about schedule, I think it might be useful (for less experienced groups especially) to set some shorter-term milestones. Having +40 hours ahead of you to work can be daunting, so sometimes people don’t know where to start. Some milestones that might be useful: (You don’t have to use all of them and I would be pretty flexible about this, e.g., a group might prefer to not present their work towards the end because they want to work until the last minute before submission.) 

  1. Spend some time looking at the starting resources and come up with 3-5 project ideas
  2. Decide which one to go with 
  3. Formulate a project proposal 
  4.  Present it to the local organisers / other attendees for feedback 
  5. Work on it 
  6. Have a “half-time show” - where groups present their current progress and their plans for the rest of the hackathon 
  7. Present your work locally at the end

Be there ~30 minutes before the start each day

At the very beginning, you want to make sure to set everything up by the official starting time, as some attendees will come exactly on time or slightly earlier. Most people will probably not show up before 10 am though.

Also, make sure you coordinate with the other organisers about what time and how long they are available for a given day, I recommend using Excel or when2meet!


Check-in with attendees about when they plan to arrive/leave in the next day

If you have a hackathon starting on Friday, ask people in the evening when/if they intend to show up the next day. In some cases we had one member of the group show up early, but the others only joined much later or not at all for the given day. Checking in also helps with deciding how much food to get. In the afternoon, it’s also good to check for how long they are planning to stay, as one time we ended up ordering food just before a couple of people left.

On Supporting attendees:


Most people will be shy about asking questions on Discord

One of the wonderful things about Alignment Jams is that you don’t need to have a strong technical/policy background to be able to host the hackathon, after all, that is what the Discord chat’s #questions section is for. Will new people who have questions use it? Nope! I think having semi-regular check-ins (perhaps twice a day + during lunch?) with each group (and maybe with individual attendees as well if you can) about their process, how they are feeling etc. is very useful. If they are new to AIS, you might be able to answer their question or direct them to the right resources even without knowing too much yourself. If their issue is more complicated, you can help them formulate it in a question and post it in the Discord chat for the international organisers/mentors.

Be mindful of group dynamics

You want to make sure that everyone is feeling psychologically safe in their group, which might be harder if this is their first event and/or they are shy/anxious.

Some failure modes to keep in mind are: 

1) Too big of a knowledge gap between group members without the more experienced members realising, so the least knowledgeable member(s) feel(s) left out 


My suggested solution: In the signup form I ask people what kind of group they would like to be in, this way you know if someone wants to be with other newcomers. Alternatively, some might specifically ask to be with more experienced attendees, which is great because they can learn more and even before the hackathon starts the experience is framed more as “I’m going to learn so much from others” as opposed to “Oh wow I don’t know anything compared to my group members)
 

2) One member dominating the group discussions

Usually, a leader emerges from the groups, but you want to make sure that everyone is happy with that person, and that it's not just someone not reading the room well and dominating the group discussion

My suggested solution: Have more regular check-ins with groups that you are not sure are functioning well in this regard, including possibly 1-1s to see how you can support attendees. Ideally, you would also have a person in each group who is at least moderately familiar with AIS but is also good at navigating these dynamics, so they can support newer attendees.

Direct people towards the right resources

Alignment Jam always offers some amazing starting resources for people. I encourage you to remind everyone about these after the intro talk is over, as people tend to forget/not find them. Another great resource is just directing them to the LW/EA forums (perhaps helping them with a few search keywords or directing them to a topic so they can read related posts. 

I think exploring the EA/LW forum is especially useful, as there is a lot of distilled content that is more accessible and shorter than scientific papers.

If someone is very new, I also encourage them to binge-watch a couple of Robert Miles videos first.


Logistics:

It’s kind of hard to know how many people will show up on the day

I mean this mostly for people who are not yet in your network and the hackathon would be the first event by which they interact with your group. For reference, for the first hackathon I ran we had ~13 signups, with ~10 people showing up and ~7 committing for most of the hackathon. For the second one, we had a whopping ~45 applications, but only about half of those people showed up, and even less committed to the whole thing. Understandably this surprised us as the first one had a much higher “sign-up to turn-up” conversion ratio.


One thing you can do to ask people to confirm their attendance (see this past form I used)  which should give you a more up-to-date sense of their availability. If your hackathon starts on Friday, I recommend sending out the email asking for confirmation on Wednesday, with a deadline of Thursday afternoon/evening, as when I did it sooner a lot of people confirmed their attendance and didn’t show up.

Think ahead about how you are going to take care of the food
 

Plan ahead with from which place you are going to order and what, so you can serve a variety of delicious vegan food. I really didn’t expect this to be an issue at first because there are plenty of vegan places in Budapest, however some of them only open late in the morning - eg. if they open at 10, you will be happy if the order gets to you by 11, which is not ideal if you have people arriving between 9-10 am. Subway was a safe choice for us as they open in the morning and have vegan options. Other fast food places should be good too, although in Hungary they have limited vegan options (I imagine they have more in Western countries, but less in second and third-world countries which is something to keep in mind)
 

To save on food costs I would recommend trying to get pastries from a nearby bakery.[1] For clarification, Alignment Jam does not insist (although they probably prefer) that you serve vegan food only, but I think unless it’s very costly for you to do so I think you should. When new people ask why the food is vegan, I usually say that it’s because the hackathon is funded by Open Philanthropy who also works on animal welfare, so I would feel hypocritical to spend their money on animal products. 


For lunch, I think it’s fine to order “proper food”, however, if you are going to go with pizza I recommend buying ones that you or the organisers personally tried before and are great. Vegan pizza can be terrible and you don’t want attendees to remember the event as “that one time I tried vegan pizza and it was terrible”!

I also want to do a better job of saving money on dinner. Again, if you are very constrained on time I think Apart Research is fine with you just ordering food (You have a $75 per person budget after all), but saving money is always nice. Some cheaper options we want to try out in the future are making hummus sandwiches with lots of veggies, wraps (if you can microwave your meat alternative), peanut butter sandwiches, vegan grilled cheese sandwiches, etc.

Have a premade ordering list of the things you will need to buy / order

...So that you don’t have to think on the spot and are less likely to forget 1-2 essential items.

Some things we had/wish we had + GPT’s opinion on what we need for a hackathon:

Fruits[2], chips, coffee, tea, soft drinks[3], fruit juice, water, chocolate, chips, trash bags, headache pills, hand sanitiser, paper towels, wet wipes, nuts and seeds, crackers veggies+hummus, energy bars, covid tests (maybe not anymore?), name tags

I think the fastest way to take care of supplies is to order from a supermarket for the start of the hackathon. Just note that the order might be late and they might not be able to bring a few things, so if something is essential (e.g., if you think name tags are very important) then I would get them separately.

Have a premade Feedback form
 

With all the things going on, this is just so easy to forget, or at least I managed to do it twice! Here is a template that you can use, I recommend making a QR code for the link and filling it out as part of the closing session (similarly to how it is done at EA conferences)

Venue related:

Book rooms in advance

Ideally, you can get a room for free at your university. If not, well, booking places for the weekend can be tricky, so I recommend doing it well in advance if you can. Of course, it is hard to predict how many people you will have (see previous point), which makes it hard to know how big of a room you will need. Eh, so is life.

Book a slightly bigger venue

With the caveats above, I would generally recommend you have a slightly bigger room so you are not jam-packed. This is because our groups differed quite a lot in how much discussion they had during their work, as well as how loud these discussions were. Having a larger room will make sure groups are not bothering each other. What’s even better is if you have an extra room that particularly motivated (or loud) groups can use, so they are less distracted or distracting. Make sure to have the meals together though!

Have more than one organiser on site if you can

To be clear, one person is fine, especially if you don’t have a large group. The reason I say two is preferable is because if something unexpected comes up, then it is nice to have additional “hands on deck”. For example, one time our food order was +1 hour late, so I had to go and pick it up myself - and it was nice that someone else was available to help attendees on site.

Have a spare laptop that you can use for presentations etc.

Another reason it’s nice to have +1 organiser on site! Believe it or not, but one time my laptop decided that it no longer had Bluetooth (?!), so I wasn’t able to connect to our Bluetooth speaker and had to use someone else’s laptop. Stuff like this happens, you never know!

Is your speaker loud enough?

Depending on the size of the room you booked, a small Bluetooth speaker I previously used wasn’t loud enough when we had a bigger group - so just keep this in mind.

Make sure wifi is strong enough at the venue

For obvious reasons. If it’s not good enough, you can rent a portable Wi-Fi hotspot relatively cheaply for a few days.

On Theory of change (ToC):

(Again with the caveat that this applies more if you are organising for people who are newer to AI Safety.)

Your impact will come from getting people excited, and likely not the specific project that they end up doing


I think much of the impact of your event will come from getting a couple of the most engaged attendees exposed to AI Safety and then diving deep afterwards. For this reason, I think it is okay if their first project ends up being somewhat naive and not exactly on point. Of course, you want to give honest feedback and nudge them in the right direction, but I would also recommend keeping some considerations around naive effective altruism in mind.


Have AIS intro books to lend/give to participants at the end

Recently I increased my bar for giving away books (eg. at tabling), but I think if someone takes the hackathon seriously it makes sense to offer them either a free AIS book or the opportunity to borrow it, depending on how freely you want to give books away.

Follow up with promising attendees later

Let’s say people attended your hackathon, it was a blast. This means everyone will sign up for AGISF the next time you are running it, right? Wrong! Okay, hopefully at least some of them will, but the point I’m trying to get across is that you shouldn’t expect this to be the default. People are busy!

I haven’t done an amazing job of this so far, but here are some ways I want to do better in the future:

-Write a follow-up email 1-2 days after the hackathon offering ways they can get involved with the community

-Conduct interviews with first-time attendees to gather feedback on how to improve future hackathons (maybe you can have them opt in for this when they confirm their attendance)

-Nudge people to sign up for your newsletter/mailing list so they are in the loop (perhaps as part of the feedback form, which you could have a separate slot for in the same way EAGx conferences do)

Build relationships with other student groups/youth organisations

Recently I have been working to build relationships with other youth organisations outside of our network by offering them our AGISF course in a cold email.[4] This had some traction, but I think the reason many are hesitant is because they don’t want to commit to an 8-week course without knowing you. 

To be clear I haven’t yet tried cold emailing about the hackathons below, but the next time we run a hackathon I want to do something along the lines of:

Asking one or more youth orgs if they would be open to help co-organise a hackathon about AI Safety. We can take care of food and Apart Research is providing mentoring for the attendees, but booking rooms is pretty expensive so we are looking for partners who would be able to provide a venue or at least help find a cheap space. I think you want to make it very clear that you don’t expect them to pay for taking part in any way. After all, that’s what your funding is for, although it’s always nice if other parties are willing to chip in. (Update: Alignment Jam no longer offers virtual cards and for now they recommend organisers to apply for funding individually).[5] Offer to have a call in the email so you can discuss the details and get to know each other.

Other/Misc:

Charge your (social) battery beforehand


Given that hackathons are during the weekend and will involve socialising planning, stress, etc., if you just have a regular work-week beforehand you might feel drained (I know I have) by the end. If you are going to attend the whole hackathon as an organiser, I recommend taking a day off during the week (eg. Tuesday or Wednesday) so you can be more present during the weekend. The caveat is that I’m an introvert, so others might have different needs!

Log stuff

Have a document open for the whole duration of the hackathon to “log stuff” - lessons, mistakes, and observations that you make on the spot. I think this is very useful as it can be hard to recollect everything afterwards. I wrote down a lot of things that I would have otherwise not remembered, and many of the items ended up in this post as a lesson.

Please share your thoughts!

These are some of the things that came to me, but I would be super excited to hear what others think, especially if:

  1. You disagree with any of my thoughts
  2. You want to share some additional tips based on your experience
  3. You have any questions about hosting hackathons
  1. ^

    However again it is a question of how many vegan options they will have

  2. ^

    I recommend ones that are easy to eat like bananas or grapes

  3. ^

    I recommend getting them in cans so they are less likely to spill, easy to have a wide variety and you can save the leftovers for future hackathons. On the other hand, some people might feel bad about the trash you are producing.

  4. ^

    This essentially entailed me writing them an email offering our course or asking if they were interested in some collaboration.

  5. ^

    I haven't checked in with them about this, but I would assume they would recommend asking for funding as part of your AIS group's general budget from CEA/Openphil.