Perhaps the most plausible way in which this could happen is that the authors of prize-winning posts are incentivized to post more frequently. We therefore examined whether prize-winning authors post more frequently in the six months following their prize than in the six months prior to it, relative to a control group.

I'm surprised this seemed the most plausible mechanism. Surely the incentive should have occurred prior to winning the prize? For my own case, I observed the existence of the prize, which encouraged me to put more work into making my post better, and the winning came later, presumably in part due to this extra effort. Is your idea that winning signals that you are high enough quality to be able to win, and hence its worth trying again?

In fact if winners suspected the Judges would be averse to letting them win 'too often' out of some egalitarian sentiment the effect might go in the opposite direction (though I think this would be very small, and I don't think I used this as a judging criteria).

Sad to see this go

Thanks for this and the related work! Interesting to see the relatively small/insignificant effect on post frequency. I don't want to spend much time commenting here so I'll be very concise with my quick thoughts. They are just for the sake of feedback mainly so don't take them too seriously!

• I still think [55% confidence] that prizes probably do have an effect, mainly on those considering writing posts. The effect might not enough to justify doing them monthly though. However, annual prizes seem more likely to be cost-effective to me.
• Your 'prize posts' also curate good content and that is valuable for forum readers, I think
• I'd like an annual prize for the top posts/comments over the year, potentially with a crowdsourced component (i.e., forum users nominate/vote). 
• I like the creative writing prize and similar. I think it makes me many times more likely to produce creative content (which sadly doesn't overcome my time or ineptitude  :P)
• I like the idea of using prizes to spotlight and incentivise specific types of content.

More on that qualitative feedback: While people generally react quite positively to winning the prize, few people have explicitly told us it made them want to write more (even when we asked directly), and our surveys of the Forum's userbase haven’t found many people saying that the chance of winning a prize leads them to invest more time in writing.

Previously I think I responded that it did not motivate me to write more/better, but in retrospect I think this is just false. At least to me, it feels very arrogant to be hopeful that I could win a prize and therefore encourages dishonesty with myself. I expect this to be similar for other people.

I agree with other commenters who have pointed out that using "more posts by previous prize-winning authors" as a proxy for the stated goal of "the creation of more content of the sort we want to see on the Forum"  seems like a strange way to evaluate the efficacy of the Forum Prize. In addition to the points already mentioned, I would add two more:

• It doesn't consider potential variation in quality among posts by the same author. If prize-winning authors feel that they have set a standard that they feel it's important to continue meeting in the future and that means they post less frequent but more thoughtful articles, that's generally a trade I'd be happy to accept as a reader.
• It ignores the potential impact of the Forum Prize on other people's writing. How many people have been inspired to write something either because of the existence of the prize itself or because of some piece of writing that they learned about because of the prize? I would bet it's not zero.

Indeed, I would argue that the prize adjudication process itself offers a useful infrastructure for evaluating the Forum experience. Since you have a record of the scores that posts received each month as well as the qualitative opinions of longtime judges, you have the tools you need to assess in a semi-rigorous way whether the quality of the top posts has increased or decreased over time.

I also wanted to express that if CEA really is ceasing the Forum Prize as such, that seems like a fairly major decision that should get its own top-level post, as the prize announcements themselves do. As it is, it's buried in an article whose title poses what I think most people would consider to be a pretty esoteric research question, so I expect that a lot of people will miss it.

Going forward, we no longer plan to run the “standard” Forum Prize process.

Some things I liked about the old Forum Prize:

• Highlighting great content
• Recognizing great work

It doesn't need to have money attached, but I hope a regular way to emphasize good work of all sorts (beyond karma, and more selective than inclusion in the forum newsletter) reappears, for the sake of both authors and readers. As a new writer (this is hypothetical; I haven't yet written any polished posts), winning would be very valuable for me (and while it probably wouldn't really affect my writing much, the existence of the forum prize would make me more excited about writing polished posts). As a reader, a regular prize has helped me find excellent content outside of the stuff I normally read.

See also the discussion under How should we run the EA Forum Prize?

Another vote against this being a wise metric, here. Anecdotally, while writing my last post when (I thought) the prize was still running, I felt both a) incentivised to ensure the quality was as high as I could make it and b) less likely to actually post as a consequence (writing higher quality takes longer).

And that matches what I'd like to see on the forum - better signal to noise ratio, which can be achieved both by increasing the average quality of posts and by decreasing the number of marginal posts.