All Forum events (I’ve been involved in) retrospective

By Toby Tremlett🔹 @ 2025-05-28T13:59 (+17)

I’ve been running events on the Forum for a year and a half now. How have they gone? What have I learned? What has worked? What should I do next? This post aims to start answering these questions. 

Note that this doc is broad to the point of being shallow in some places. It was originally written as a memo for CEA's May team retreat, and it's only been polished up a little in places. 

Feel free to ask questions about anything in the retro. 

Draft Amnesty Weeks

Useful links

Overview

I’ve run two draft amnesty weeks, March 11 – 17 2024, and Feb 24 – March 2 2025.

Draft amnesty week aims to surface good content cheaply, develop relationships with authors, and encourage new authors to post their first pieces.

Before I ran Draft Amnesty Week, Lizka ran a successful Draft Amnesty Day (based on Catherine Low’s suggestion?) in 2022.

Comparing the events

Posts, and what went into them

The first Draft Amnesty Day (DAD) led to as many >50 karma posts as the first Draft Amnesty Week (DAW) (though it was during our biggest year for forum engagement). The second DAW (2025) was more successful in terms of the number of posts, and the amount of posts above 50 karma.

“Prompted posts” refers to the number of posts I remember asking the author to write/ post. I think I’ve missed a few — but it’s definitely interesting that though the majority of work for a DAW event is me talking to people about their posts and encouraging them to post them, the majority of posts didn’t require any particular prompting from me.

Engagement

Engagement on DAW 24 posts was 10.8% of forum engagement that week. On DAW 25, it was 13.2%.

Is this a good thing?

The aim of draft amnesty is to lead to more good posts, both during the week itself, and in the future via encouraging authors. In that sense - engagement is only tangentially related (more posts should equal more engagement- or we aren’t doing sharing right).  

But I think there would also be a failure mode if the % of engagement on draft amnesty posts got too high, and the draft amnesty posts were on average lower quality than published posts (which they should be if we are doing enough to encourage people to post the rest of the time). 
 

The cost of draft amnesty 

The second draft amnesty week took a lot less time to organise than the first. The work includes:

The second draft amnesty week (2025) was especially easy to run.

Takeaways

Draft amnesty seems to be a very cheap way to generate a bunch of posts. This year, I’d like to experiment with a second draft amnesty event. If bi-annual (twice-yearly) draft amnestys lead to the same or similar number of posts, then it is a bit of a no-brainer to run them. 

Debate Weeks

Overview

The idea of polls had been floating around for a while, but the idea of debate weeks became more concrete on an online team retreat in early 2024.

The aim was to find a way to cause particular crucial discussions to happen on the EA Forum. We especially value long comment chains where people change their mind, or posts which respond to other posts — essentially things which push discourse forward. Debate weeks were hypothesised as a way to do that. Overall, they’ve been very successful.

Debate weeks, as I’ve run them, have a slider poll on the forum homepage banner where users can vote and attach a comment. As of the second debate week (Animal Welfare vs Global Health Debate Week) these comments appear in a discussion thread.

There have been three debate weeks to date:

There was also a debate-week-adjacent event before, AI Pause Debate, which followed a different method (no poll, just posts on a topic, like a lecture with responses). It was held in September 2023, and led to 29 posts, with 17 posts (not counting announcement) over 50 karma.

Purpose and goals

Metrics and comparisons

 AIWAWvsGHEC
#posts292514
#posts>50karma7173
#votes559564370
#comments on tagged posts221814457
#engagement hours total (week and two weeks after)3860.33636.6527784156.725
#engagement hours on tag (week and two weeks after)189.4805556646.7416667171.4944444
%engagement on event4.91%17.78%4.13%

The most successful debate week on several metrics was Animal Welfare vs Global Health Debate Week. It had the most posts over 50 karma by far, the most votes on the banner, and the highest engagement hours on the discussion. It also led to a massive number of comments — one of the key positive outputs of all debate week events so far.

Unlike draft amnesty weeks, the percentage of engagement on the event is a positive metric for debate weeks, given that the topic was well-chosen.

Engagement hours

The three engagement hours graphs below don’t necessarily tell you much more than the table above (they are partially included because they took me bloody ages to make). One interesting point I’d highlight is that the AWvsGH engagement did lead to a spike in engagement, and is possibly responsible for overall engagement growing in the month afterwards (perhaps because of all the users, new and old, who started commenting (again) because of the event).

Feedback on debate weeks

A large portion of critical feedback during debate weeks is on the wording of the question. It’s very difficult to produce just the right question for the entire audience, with all their different understandings and assumptions, to engage in. I expect this to remain true.

Some other critical feedback:

The cost of debate weeks

Debate weeks are more costly (in terms of my time) to run than draft amnesty. I reach out to many people beforehand to encourage them to post, and only a few of those do. Deciding on the question and framing it means consulting experts, reading relevant papers, and so on. This can take a while. 

For the Existential Choices Debate Week, I organised a symposium with Will MacAskill — a timeboxed period when a group of experts would engage in a comment debate about the debate question. This was very successful, and led to 145 comments on that post alone. However, it also took a lot of organising and coordination.

Where next with debate weeks?

The core debate week feature is clearly a winner – a slider which encourages you to vote, then prompts you to comment.

The symposium for existential choices week was also a winner – it led to a lot of comments and engagement.

Topics for debate weeks are difficult — I’d like to focus on topics that people could change their mind on, even if it is at the expense of some engagement.

Giving Season & Donation Election

Overview

I’m going to be comparing Giving Season 2023 and 2024 because these are the most direct comparisons (and I was here for both, and project managed 2024).

Giving Season refers to:

Purpose and goals

About Giving Season 2023, I previously wrote that: 
“The main benefits of the event were:

I think this is a pretty good quick description of the goals of the event, though not necessarily in order. For example, we don’t think of fundraising as the primary goal of giving season — causing discussions that lead to better donations, or more respect and centrality for effective giving, are more important than donations which may or may not be attributable to forum activity.

The biggest change I wanted to make in 2024 was to make events, especially the donation election, more discursive. I.e, I wanted the donation election to cause conversation.

Donation Elections

The first donation election was in 2023, and we also ran one in 2024. Both involved voting for charities to win money from a prize pot. However, we made some changes for 2024:

Election engagement

How many voters:

How many comments/ posts/ discussions: 

I haven't done a full analysis here, but it looks as though there were more comments overally during 2023's giving season than 2024's. Specifically, there was an average of 86.5 comments or quick takes per day during 2023 giving season, versus 59.5 comments per day in 2024. 

However, 2024's giving season did achieve the goal of more conversation about the donation election decisions. Using the same vote -> public comment pipeline we use for debate week events, we ended up with 112 comments on a donation election discussion thread in 2024.

Marginal funding week

Marginal funding week started in 2023, and was very successful. Because of its success, we ran it again in 2024.

In 2023, 19 organisations wrote posts, and 23 left shorter comments. In 2024, 46 organisations wrote posts, and 25 left comments. 

The biggest difference which drove more posts in 2024 was that we required a marginal funding comment or post in order for an organisation to be eligible for the donation election.

Other theme weeks, and content

At a first approximation, there was far more effective giving content in the 2024 event than in the 2023. 

2024’s Giving Season tag has 111 posts on it, while 2023’s has 63.

Theme week/eventTotal postsPosts >50Bangers (>100 karma)?
Effective Giving Spotlight (2023)4 (I think, this is why we tag weeks now)30
Donation Debate Week (2023)110
Funding Strategy Week (2024)17104
Pledge Highlight (2024)1481

Raising money directly

Our donation pot for the election raised $30K in ‘23, and only around $15K in ‘24.

Below is the data for 2024 - I don't currently have access to the 2023 data. I know that in 2023 there were a few more large donors, one of which gave at least $5K. 

2024

Not bad, but much less than last year.

This shows up even more in the count by day chart (also in the hex), but people were more willing to donate later in the game, i.e. when they already know where the money is going.

Most donors were relatively small donors.

Raising money indirectly

Sarah estimates that we raised between $74K and $85K for the charities that took part in our election, via platforming them (i.e. without counting the election money). However, we are missing some key data on some of the top charities in the election, including the EA AWF and Rethink Priorities, so we expect this is a significant underestimate. 

It’s hard to compare this to 2023’s, because the model my colleague wrote for last year includes even more uncertainty (between $23,500 and $1.24M raised).

Engagement

Total engagement was much lower at the end of 2024 than at the end of 2023, so it's tricky to do apples to apples here. One key difference is that giving season seemed to cause an overall engagement boost during 2023 in a way it didn’t in 2024, despite the fact that there was far more content in 2024.

The cost of giving season

Last year we estimated costs at around $30-35K (most staff time). I’d expect this year was similar, if a little less (there were some process improvements from last year, and we benefited from experimentation on what works).

Summary –

Appendix/ extras

Glossary of events:

Below are the events that have happened since I started on the Forum team in November 2023.

One day/ not content generating events:

Theme weeks and debate weeks:

SummaryBot @ 2025-05-28T21:33 (+1)

Executive summary: This reflective memo reviews a year and a half of EA Forum event experiments—including Draft Amnesty Weeks, Debate Weeks, and Giving Seasons—finding that lightweight initiatives like Draft Amnesty are cost-effective for surfacing new content, while higher-effort events like Debate Weeks can successfully stimulate discourse but require careful framing and coordination; future iterations will likely double down on these learnings while adjusting based on engagement and quality tradeoffs.

Key points:

  1. Draft Amnesty Weeks are low-cost and effective at generating Forum posts, encouraging new authors, and increasing engagement modestly; the 2025 event outperformed 2024, and the author plans to experiment with running them twice yearly.
  2. Debate Weeks successfully foster deep discussion, especially when well-framed and accompanied by features like homepage slider polls and symposiums; however, selecting and wording debate topics remains challenging and time-intensive.
  3. Animal Welfare vs Global Health Debate Week was the most successful across metrics—posts, karma, comments, and engagement—highlighting the potential of controversial but well-chosen topics, though some feedback flagged the framing as too combative.
  4. Giving Season 2024 increased content volume significantly over 2023 (111 vs. 63 posts), but had lower total engagement and raised less in direct donations ($15K vs. $30K); however, the new ranked-choice Donation Election and public comment thread did increase discourse.
  5. Marginal Funding Week saw strong growth, with 46 participating organizations in 2024 (up from 19), likely driven by requiring posts/comments for donation election eligibility.
  6. The memo underscores tradeoffs between cost, content volume, and quality, advocating for iterating on successful formats (e.g. Draft Amnesty, Symposiums) while refining goals and expectations for each event type.

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.