Tentative Summary of the Giving What We Can Pledge Event 2015/2016

By Linch @ 2016-01-19T00:50 (+7)

All views here are my own, as a volunteer for Giving What We Can with a fairly significant influence on the pledge drive. This post does not represent the official position of Giving What We Can but is written with their permission. I suspect that this post is primarily of interest to employees and volunteers of Giving What We Can. However, I decided to release this to the general EA audience because we've had a lot of help from so many different people in EA, and also because this may be of interest for people in other EA organizations who face similar problems, or for people interested in volunteering for Giving What We Can. 

I learned a lot from the pledge drive, and will likely make at least 2 more posts, one suggesting process improvements for the next pledge drive, and the other analyzing pros and cons of paid Facebook advertising. 

Because there were so many different people who contributed, and different people have different privacy settings, I've tried my best to scrub identifying details in this public post whenever possible. Please understand that I have no problem with giving people full credit for their work, and would love to publicly thank you for them. In addition, this summary is a work in progress, and I probably missed a lot of the very important work related to the drive that I'm less familiar with. 

Executive Overview

Giving What We Can ran a pledge drive to get as many people as possible to take the pledge or Try Giving in the same time frame, between December 10th 2015 and January 10th 2016. The drive encouraged somewhere between 60 and 90 additional people (who otherwise would not have taken it during the same time period) to take the Giving What We Can pledge, and also a smaller number of Try Givers. This is the second pledge drive: the first one was held in 2014/5 and organised by Ravi Patel from GWWC Cambridge.

Approaches We Tried
We used a multi-pronged approach to get as many people as possible to be excited and willing to take the pledge. Most of these approaches mobilized volunteer effort of the Giving What We Can and EA community.  

Facebook

Facebook was the primary tool for our social media outreach, and the Facebook event was this pledge drive's focus point. Facebook outreach involved the following: 

 

  1. We created a Facebook event, which we constantly and consistently updated with stories, interesting links, statistical updates, and repeated exhortations to take the pledge. In addition, we heavily encouraged new pledgers to share their stories and pictures on the event page, creating a fairly lively atmosphere (especially near the end). Finally, we messaged everybody who marked themselves as "interested" or "attending" the pledge event asking them if they have follow-up questions, and also suggesting ways they can tell their friends/get more involved. 
  2. During the course of the pledge drive, the volunteer who managed the GWWC Page (which has 7300 followers now, though somewhat less at the beginning of the pledge drive) shared many stories and pictures of people who took the pledge, as well as the pledge event itself (multiple times). 
  3. We asked new pledgers and other people interested to share their pledge stories and pictures on their own Timelines and in groups they're connected with, generally with positive results. 
  4. We actively posted about the pledge drive in EA and EA-affliated Facebook groups, so that anybody in EA should in theory be very familiar with the pledge by the end of the drive. 
  5. We also tried posting in various online philosophy and vegan groups, generally with relatively poor results/engagement (on the other hand we didn't spend much time on it, so too soon to tell for sure whether it is actually a flop).
  6. I spent 137 dollars (of my own money) on Facebook ads, mostly directly promoting the event, with some $$ used to increase the reach of GWWC's links to people on GWWC's blog or HuffPo who talk about why you should take the GWWC pledge. 40 people directly "responded" (that is, marked themselves as "attending" or "going") to the Facebook pledge event from the ads. Unfortunately Facebook ad data is not sufficiently granular to tell who they are, so I wouldn't know if any of them actually took the pledge until GWWC gets around to interviewing people who pledged during this event. As a side effect, we "reached" 13,373 people with the paid ads, with 515 people "taking action", including 412 post likes, 4 post comments, 17 post shares 127 link clicks and 18 new "likes" for the GWWC page. More details in a future post delving into FB ads. 

 

The overall Facebook event was fairly successful, with 13K invited, 311 who marked themselves as "going" (of course, not all of them actually pledged), and 420 "Interested." However, this is worse than our reference class of the 2014/2015 pledge event, with 14K invited, 571 went, and 202 "Maybe."

Other Social Media

 

  1. Twitter: A volunteer for GWWC kindly started @Pledge2016, we also tried to popularize the hashtags #trygiving2016 and #give10percent, and various EAs/GWWC members have spread the message on Twitter. Jacy and Daniel's HuffPo articles are especially popular and seemed to have been retweeted to significantly outside of the general EA blogosphere. 
  2. Tumblr: There was no official GWWC involvement on Tumblr this year, however Tumblr users unitofcaring and sinesalvotorem wrote very powerful posts that each had 100+ notes. I would not be surprised if a couple of people were persuaded by them to pledge or at least to donate more. 
  3. Quora: I spent some time promoting Giving What We Can and effective altruism on Quora. At least one person was directly persuaded by personal messaging to Try Giving, so yay. On the other hand, my answers didn't seem to garner many upvotes, so somebody more fluent with Quora might do even better. 
  4. Thunderclap: About 100 people signed up for the Thunderclap and it helped blast a loud message across various social media accounts. However, there was some confusion over how it worked, and in total we're not sure at this moment if it was worth the social capital.
  5. Google Hangouts: I tried doing daily Google Hangouts for people to ask questions about the pledge, but the turnout was basically nonexistent. So it was a flop. That said, the pledge drive organizers internally communicated well with Google Hangouts.

 

Mobilizing the Giving What We Can community

There were many times when we asked/mobilized the general GWWC and EA community to help. We set up an excel sheet for people to record contacting their personal networks (to keep them on track and minimize overlap). Eleanor Stephens also wrote a nice post on how individuals can help with the pledge drive, which was promoted on Facebook to followers of GWWC's page and shared on the GWWC Community Facebook group. We also set up a "Pledge Event Organizers" Facebook group. 

Contacting EA Leaders
We had an internal list of 20-30 "EA Leaders" (people exceptionally involved in EA Movement-building) who we asked to help share information about the pledge drive. We have received mostly positive responses and have had a lot of help from leaders of other EA organizations, etc.

Reaching out to EA Bloggers
We reached out to many different EA bloggers, asking them to write about the pledge. Many of them agreed, for example unitofcaring and Jaibot. We also had GWWC blog posts by: Ali Ladak, Eleanor Stephens, Zach Leather. Some bloggers have also very kindly written posts without prompting from anybody in GWWC, for example ThingofThings.

Celebrity Outreach
I started a concerted campaign where we tried to get access to "celebrities"-writers, journalists, artists, bloggers and other people whose influence vastly exceeds the monetary value of their donations, on the fringes of EA. While we didn't manage to talk to many of the people I wanted (which is hardly a surprise!), two journalists and an artist have agreed to take the pledge and potentially do an interview with us. Hopefully we'll be able to get those interviews up soon!

Media
Unfortunately, despite our best efforts, we were unable to obtain significant media attention for this pledge drive. However, aside from the two journalists already mentioned, 2 other journalists (from fairly prominent publications) have expressed an interest in covering us in the future. There are also 3 articles out on Huffington Post, by Jacy Reese from Animal Charity Evaluators, Daniel Selwyn, and myself. 

Local Meetups/Chapters
Many of the GWWC chapters and local EA meetups and pages have helped promote the Giving What We Can Pledge Event by sharing our articles, links, and personal messages. GWWC Cambridge was especially impressive, they created a video for the drive (see below) and posted pictures of people who have taken the pledge every day, Humans of New York-style. This seems like an incredibly effective way of showcasing how human GWWC members are, and I will highly encourage other GWWC chapters and EA meetups to try it too.

In addition, both EA London and EA Madison (my local meetup) organized local meetups to talk to people about the GWWC pledge and encourage people to take the pledge/Try Giving, with very positive results.

Video
Giving What We Can: Cambridge created a short, very light-hearted silent movie to encourage people to take the pledge! The video has 1654 views (To be fair, I don't know how many of them were from me) and 62 likes. It's a really cute, fun video.

Progress over Time
Here I include a chart of GWWC's progress during the pledge drive. If you want to learn more about GWWC's progress, you can check out the dashboard yourself.

Screenshot 2016-01-15 at 18.01.59.png

 

I included data from July 2015 onwards so people have the appropriate reference class, and used 5-day moving averages to smooth out spikes. There was a dip right after the pledge drive started, followed by a peak about a week in (December 18th/19th), followed by a large dip again for Christmas, and a peak in the last days of December/Jan.1st. There's a dip in the early days of January but the number of pledgers grew very quickly near the end of the pledge drive, cumulating in 17 people pledging on the last day of the drive (Jan.10th), or about half the average number of new pledgers a month in 2014.

Workload/Timeline
Jon Courtney and Marinella Capriati (GWWC staff) have been working on setting up the event, coordinating with chapters, and EA Outreach since October 2015. The core "pledge drive team" also includes Harri Bescelli, Eleanor Stephens and I (volunteers) who worked primarily during this event. I volunteered essentially full-time from December 10th to January 10th. In total, I estimate about 2.5 person-work-months spent on this pledge drive (one from me, 1.5 from the rest of the team).

We also had help from dozens if not hundreds of EAs/GWWC pledgers with technical help, sharing our articles, retweeting, writing personal stories and posts, about why they took the pledge and entreating friends to do the same, etc, etc.  I can't imagine how to estimate this time.

Impact
We can't really have a good gauge on our counterfactual impact until people who took the pledge during our pledge drive have been interviewed.  That said, there are some ways to estimate it. For example, counting people who marked themselves as "attending" or "interested in" the Facebook event, as of Jan.12th, 66 of them have taken the pledge on or after Dec. 10th 2015, and 13 of them have "Tried Giving" on or after December 10th. 66 may not necessarily be the lower bound, for example some people may have decided to take the pledge in December already and then just signed up for the FB event after hearing about it as an afterthought.  

Another way to gauge the impact of our event is to look at before/after. Giving What We Can has about 1400 members (+-5) as of December 10, 2015, and 1555 by the end of Jan.11th, 2016, or a difference of 155. Eyeballing the dashboard, there usually isn't a spike in the number of pledges in December outside of the two drives, suggesting that the base rate of 50-60 people per month in 2015 is the appropriate counterfactual to consider if we didn't have a drive. So just looking at before/after rates will suggest that the impact of the pledge drive is on about 90 people rather than 66. One plausible reason why people may be inspired by the resources from the pledge drive (blog posts, in-person explanations, etc.) but did not take the pledge is that they don't have a Facebook account, another is that they're very private, a third is that our resources didn't explicitly emphasize the value-added of the FB event.

That said, 90 is not necessarily the upper bound for the long-term impact of this pledge drive. For example, we've had 3 HuffPo articles and many different blog posts come out of this pledge drive, which may marginally increase people's likelihoods of pledging in the ensuing months if not years.

Impact Evaluation

Note that even more so than the rest of the post, this is #justmyopinion, and does not represent the official view of Giving What We Can.

So the most naive way to look at our impact is by just adding up the total pledged donations ($17.5 million, plus Try Giving that hasn't been added) from all the people who marked themselves as "attending" the FB event. Not bad for 2.5 work-months!

But clearly this is not counterfactual or time-adjusted. GWWC's estimations is that each new pledger is worth ~60K in donations (counter-factual and time-adjusted).  However, since GWWC's own "realistic" estimations is that their impact is ~100x, then the correct internal value of a new pledge relative to other things that GWWC could do, is about 600 dollars each. 

By this estimation, the impact of work on this pledge drive, relative to earning to give and donating to GWWC, is somewhere approximately between 66*$600=$39,600 and 90*$600=$54000. 

But this is still somewhat naive. If most of the new pledgers would have ended up taking the pledge anyway, the real counterfactual impact is the delta in how much sooner this pledge drive has resulted in people taking the pledge. For example, if you guesstimate that the average is one year, then the internal estimation of how valuable this pledge drive is could be as low as 66*$600/40=$990! However, I think that this is too pessimistically low for three core reasons: 1)Haste consideration means that hearing about this pledge now is worth more than hearing about the pledge later, 2)The "average" of one year is too low: the median might be around there, but the right end of the counterfactual tail could end up being really high, 3) "just hearing about" the GWWC pledge sometimes happens, but a lot of the time it takes staff or concentrated volunteering time. So many of the people who counterfactually would not have heard of the pledge in the context of the pledge drive would probably have taken up staff and volunteering time longer down the line. 

 

In conclusion, the value of the pledge drive relative to other activities GWWC could undertake is inconclusive, but I’m relatively confident (95%+) that it is more worthwhile than 2.5 person-months spent working at a “typical” job and donating to top GWWC charities.

 

Other Metrics/Outcomes

Get involved

If you're interested in joining the GWWC community, consider making a New Year's resolution to take the Giving What We Can pledge or Try Giving! Alternatively, if you're interested in helping out with the next pledge drive, please email Jonathan Courtney, Marinella Capriati, or myself. 

undefined @ 2016-01-25T06:49 (+5)

Hey, awesome!

I've always been very pleasantly surprised that these things are so much of a success. My prior that you can just ask people to donate 10% for life and they will is very, very, very low -- it just doesn't seem like something that can happen and no other charity I know of does it. Everyone emphasizes making very small asks (though monthly donation asks are common) and ramping up steadily over time. So whenever I see the pledge drive work (twice now), I always feel like I'm looking at a miracle and wondering what the catch is.

My current working hypothesis is that the pledge works, but only on people who were already weakly involved in the EA movement and know other people who have pledged. So this would mean that the pledge event does create genuine impact by spurring people to action, but might be weak in (a) trying to build massive success outside EA and (b) in considering the counterfactual of whether pledgers who pledge during the drive would have just pledged later. (Overall, still probably worth doing.)

I think this makes sense of the evidence that you tried posting in various non-EA online groups that seem relevant but had low engagement and that the Tumblr posts went so well (since they were writing to somewhat EA audiences). This hypothesis also makes sense of the data that the 2014 pledge went better than the 2015 one despite the higher effort put into 2015, since in 2014 there were more EAs on the fence that you could convert over.

I'd use this hypothesis to predict that virtually no one will have joined from a Facebook ad and very few people will join from reading the HuffPo post. Instead, most of your people will report being somewhat familiar with EA before pledging and will come from Tumblr or FB posts. I'm super curious whether your interviews will validate this or not.

-

But clearly this is not counterfactual or time-adjusted. GWWC's estimations is that each new pledger is worth ~60K in donations (counter-factual and time-adjusted).

If most of the new pledgers would have ended up taking the pledge anyway, the real counterfactual impact is the delta in how much sooner this pledge drive has resulted in people taking the pledge. For example, if you guesstimate that the average is one year.

I do think it's worth seriously considering that you're only bringing forward in time the pledge by a year or so.

Also, I think it's worth seriously considering that the people you bring into the pledge through drive events will not be as committed as people who came to the pledge through their own volition (which constitute much of the people from which the $60K estimate was derived).

That being said, even if the 90 people are worth $600 each instead of $60K each, you're still netting ~$22K/mo in counterfactually-adjusted impact, which seems clearly worth doing.

-

However, I think that this is too pessimistically low for three core reasons: 1)Haste consideration means that hearing about this pledge now is worth more than hearing about the pledge later

This doesn't seem that valuable to me, except in so far as you get another year of donations. I would expect some social effect of getting more members sooner, but I expect this effect would not be large (say larger than the value of the additional year of donations) because my prior is that not that many GWWC members are successful at getting their friends to pledge... and even if they do get a friend to pledge, it will likely just bring them in one year sooner than they otherwise would.

-

However, since GWWC's own "realistic" estimations is that their impact is ~100x, then the correct internal value of a new pledge relative to other things that GWWC could do, is about 600 dollars each.

This seems incorrect and needlessly unfair to the pledge event, since the pledge event could just be one of GWWC's activities with the 100x ROI. Obviously GWWC has to do something and you should compare the pledge event to the calculated impact of other GWWC activities, but there's no reason to just drop the impact by 100x.

-

2)The "average" of one year is too low: the median might be around there, but the right end of the counterfactual tail could end up being really high

I'll admit that the "one year" number is quite made up, but I don't see any reason to think that it is distributed with a wide right tail except that it tells a better story for counterfactuals. At minimum, I think chances are high that they'll have just heard about it in the next pledge drive. ;)

Linch @ 2021-09-15T00:05 (+2)

I haven't reread this post (I find it aversive/painful), but for outside view reasons I think you should heavily discount any conclusion or analysis reached by the author of this post, for reasons outlined here and here. 

I would guess that the operational details and factual claims are relatively more trustworthy.

You should assume that the author was pretty junior to this type of analysis and not very good at impact assessments or related points.

undefined @ 2016-01-20T01:04 (+1)

I started a concerted campaign where we tried to get access to "celebrities" [...] two journalists and an artist have agreed to take the pledge

Anecdotally, one person who basically never heard of EA before came to our local group's lunch about Giving What We Can on Jan.9th, and took the pledge in time for the drive--this might set a record for fastest turnaround for hearing about a thing and then committing to it!

That's strikingly successful! Can you say a little more about these people and why they were so unusually receptive? Did they fit into groups of people typically receptive to EA? What was their attitude to charity before? What sort of contact did you have with the journalists and artist?

In addition, both EA London and EA Madison (my local meetup) organized local meetups to talk to people about the GWWC pledge and encourage people to take the pledge/Try Giving, with very positive results.

Can you expand on these results?