TikTok DGB Giveaway Results

By Ben_West @ 2022-04-15T09:51 (+33)

Summary

• We produced a series of TikTok videos which had ~150,000 views combined, advertising a giveaway of Doing Good Better, which led to 359 people ordering copies
• Two giveaway recipients attended a Virtual Program, and another attended EA Global, although I suspect receiving this book was not counterfactual in these
• I estimate that 4-25 people got mildly more involved in EA as a result, and 6-37 changed their donations
• I think this ROI is not competitive with my normal work at CEA, but it’s relatively easy/fun for me to do TikTok so it’s possible I will do more

Background

Myself and Lacey (@benthamite) and Emma Williamson (@dailydosenews) piloted giving away copies of Doing Good Better on TikTok. Information on the process we followed can be found here.

Funnel

Videos

Summary of Survey Responses

Detailed Survey Responses

Fraction of the book read

Increased Engagement

Four people said something like “I already knew about 80 K but this introduced me to the EA community” or “I am planning to move into EA aligned work in the next year but I was already thinking about this before I got the book; reading it has kept me motivated though.”

None of the responses seemed to indicate that the book was responsible for a large counterfactual change in their lives.

Donations

Six people reported changing where they donate.

Nonstandard applications

Five people reported being inspired to change their lives in ways that didn't seem clearly related to the message of DGB, e.g. starting a company to teach people how to code or initiating a meditation practice.

Unclear responses

Six people reported something like “I apply the principles”, the impact of which is hard to determine.

Potential Harm

One respondent reported being more skeptical of people doing good.

Participation in CEA programs

• I matched the email addresses and names of people who ordered books to the email addresses of people who participate in other CEA programs. I got three matches:
  • Applied to Intro VP two months before ordering a book; not plausibly counterfactual
  • Applied to Intro VP three months after ordering a book; plausibly counterfactual
  • Applied to EAG two days ordering a book; plausibly counterfactual
• Unfortunately I was not able to get any additional information from these people, so it is unclear what impact the book had on them
• My default guess is that they were already interested in EA, and the book might have been a slight increase in their motivation, but was not very counterfactual

Other Benefits

The TikTok with Sam was probably the most widely viewed thing amongst EA’s that I’ve ever created. (Although it’s only my second most popular EA TikTok when measured by number of TikTok views.) Will used it in his opening talk at EA Global, there were a bunch of memes in the EA meme Facebook group, etc.

I think if EA is a fun community then people will be more likely to stay around and increase their involvement, and this video contributed to that slightly, but the impact is pretty hard to calculate. 

Cost-Effectiveness

• My initial video with Sam did not perform well, and it was only after I gave more thought to making it engaging that it became popular. And other videos without Sam were substantially less popular.
  • This seems to indicate that success requires both TikTok expertise and a celebrity to reach a wide audience, which is expensive
  • Although I should note that a successful EA video creator thought my video with Sam was quite bad, so maybe I don’t actually have TikTok expertise. (For what it’s worth, I thought that their suggested improvements would have been worse, but it’s pretty hard to predict the outcome of these things.)
• I gave away more books (even excluding the video from Sam) then Emma did, despite me having ~50 K followers and her having ~500 K followers at the time
  • This is good, in the sense that it means that we can be successful without needing an existing follower base, but bad in the sense that it means we can’t leverage EA’s who do have existing large follower bases.
• I personally thought it was fun to make the videos, and Sam didn’t seem too bothered, so I think the actual costs of this were pretty low.
  • Plus it’s probably good for FTX to have gotten hundreds of thousands of views on TikTok, independent of the EA value
• But if it actually was costly, probably it would not have been worth doing.

Appendix – materials

Below are the materials I used, in case they are helpful to someone else

Survey Email

I A/B tested two different subject lines; they ended up getting a similar number of initial responses (10 versus 12). The survey link was programmatically generated to auto populate the survey with the appropriate email address to be sure that respondents were recorded accurately.

Subjects:

Remember when Benthamite gave you a free copy of Doing Good Better?

Doing Good Better Follow Up – $10 for your response

Body:

Hey Name,

Thank you for requesting a copy of Doing Good Better from our TikTok.  To help us understand the impact of this giveaway, we are hoping that you can fill out a two-question survey.

Link

Whether or not you read the book, we will PayPal you $10 for filling out this two-question survey. You must respond by March 19 to be eligible for compensation.

Please feel free to respond to this email if you have any questions.

Best,

Ben and Lacey <3

Survey

The survey can be found here

Acknowledgments

I would like to thank The Open Philanthropy Project for sponsoring the book giveaway, Jo Anderson for extremely helpful feedback on the survey and email strategy, EA Books Direct for the infrastructure which makes book giveaways so easy, Sam for his dancing abilities, and the EA Infrastructure Fund for funding responses to the survey.

1. ^{^}

   This is assuming there was no response bias, which is obviously a dubious assumption

2. ^{^}

   Due to a glitch in the EABD form, about half of people were unable to opt in to the survey. I expect that future giveaways would have ~twice the response rate