Experiments in GiveWell communication

By GiveWell @ 2019-07-29T11:04 (+16)

This is a linkpost to https://blog.givewell.org/2019/07/23/experiments-in-givewell-communication/

One of our top priorities is to increase the amount of money we direct to our recommendations. As part of our effort to do this, we’re planning to try new kinds of communication. We hope to reach people who haven’t heard of or connected with GiveWell in the past, and to increase retention of our current donors by making the experience of donating through GiveWell more compelling.

We are experimenting on our homepage and in emails with using images and making our cost-effectiveness estimates more prominent. Our goal is to improve people’s connection to our work without compromising the accuracy of what we share.

There are potential downsides to this approach. We expect to balance our goal of communicating in a way that is emotionally compelling with our commitment to honesty and not misleading donors or overstating the case for our recommendations.

We’re not planning a major overhaul of GiveWell’s website or other communications in the near term, and we are unsure if we will make major changes in the future. Most of GiveWell’s communications will look as they always have. Our hope in the coming months is to learn whether there are new ways we can communicate about our work to increase our impact. We’re writing this post to share with you the context behind these experiments.

Summary

In this post, we discuss:

• Our communication experiments.
• Challenges and potential downsides of our approach.
• How you can help us improve.

Our communication experiments

We’re initially experimenting with using images and emphasizing cost-effectiveness information in our communications. We selected these experiments based on our intuition, our understanding of best practices in the nonprofit sector, and the feedback we’ve received from GiveWell’s donors and others.

Over the years, we’ve heard from a number of our supporters that they wish GiveWell had more emotionally-oriented content so that they could more easily share GiveWell with their peers or feel more connected to their own gifts. We also understand that most charity fundraisers make emotional appeals tied to specific individuals or projects. Fundraisers may use cost figures to promote their causes, although these figures can be misleading (for example, claiming you can save a child’s life by donating $0.50—we estimate that even a very cost-effective program requires closer to $2,400 to avert the death of a child).

Taking these considerations into account, we want to move in the direction of sharing content that people can easily connect to without sacrificing honesty and accuracy. We plan to make these changes gradually and to see what works before committing to a long-term path. We’re starting by making the following changes to our homepage and certain email content we share:

1. Adding images. We believe we can create a closer connection to our top charities by showing pictures of the work they do—either to illustrate how the program is carried out or to show the people they have helped. We want to do this respectfully.
2. Featuring our cost-effectiveness figures more prominently. Although we don’t advise taking our cost-effectiveness estimates literally, we do think they are one of the best ways we can communicate about the rough magnitude of expected impact of donations to our recommended charities.
   
   A few years ago, we decided not to feature our cost-effectiveness estimates prominently on our website. We had seen people using our estimates to make claims about the precise cost to save a life that lost the nuances of our analysis; it seemed they were understandably misinterpreting concrete numbers as conveying more certainty than we have. After seeing this happen repeatedly, we chose to deemphasize these figures. We continued to publish them but did not feature them prominently.
   Over the past few years, we have incorporated more factors into our cost-effectiveness model and increased the amount of weight we place on its outputs in our reviews (see the contrast between our 2014 cost-effectiveness model versus our latest one). We thus see our cost-effectiveness estimates as important and informative.
   We also think they offer a compelling motivation to donate. We aim to share these estimates in such a way that it’s reasonably easy for anyone who wants to dig into the numbers to understand all of the nuances involved.

We’ve chosen two places to run our initial experiments with using images and emphasizing cost-effectiveness estimates: GiveWell’s homepage and certain email content.

Homepage updates

Our intuition is that someone spending a few minutes on GiveWell’s homepage would not come away with a clear understanding of what GiveWell does or an emotional connection to our work. We hypothesize that adding images, illustrations, and cost-effectiveness estimates will help new visitors better understand and connect to GiveWell’s work. We are also planning to link to a citations page that provides sources for the calculations we use on the homepage and enables readers to easily access the details of our research if they want to vet or understand our claims.

We plan to test two new versions of the homepage this summer; you can see preliminary versions here and here. We’ll be testing these pages against each other and our current homepage. We expect to update our homepage pending the results of this experiment—we’ll likely be looking at visits to our top charities page from the homepage, the bounce rate on the homepage (the percentage of visitors who leave the page without going to other parts of our website), and the duration of time spent on the homepage.

Impact emails

We hypothesize that drawing clearer connections between our donors’ support and what it enables charities to achieve will increase retention of GiveWell’s donors. We think doing so will make the experience of giving through GiveWell more meaningful and memorable. One way we think we can do this is by reporting to donors what we expect the impact of their gifts to be.

Information about impact has always been available on our website via our cost-effectiveness model, but it has neither been linked to individual donation amounts nor sent directly to donors. Up until recently, if a GiveWell donor was interested in the impact of their gift, they’d have to track down the relevant part of our cost-effectiveness model and do their own calculation.

Now, we’re experimenting with more proactively sharing this information. In the fall, we sent an email to a group of our donors who gave to the Against Malaria Foundation (AMF), a GiveWell top charity that distributes insecticide-treated nets to prevent malaria. This email explained how AMF works, using photographs of net distributions, and included our best estimate of the impact each individual’s donation would have, in terms of the nets purchased and deaths averted. You can see an example of this email here. We ended this experiment after a few weeks due to a technical glitch.

This year, we piloted sending an email to donors who supported “Grants to recommended charities at GiveWell’s discretion.” We grant these discretionary funds each quarter to the GiveWell top charity or charities that we believe have the most pressing funding needs. When we made these grants in 2019, we sent an email to donors who contributed to the discretionary funds. The email announced where we chose to grant the discretionary funds and why, along with a description of the charity that received the funds—including images of its work—and a calculation of each donor’s expected impact based on our cost-effectiveness analysis. You can see an example of this email here.

Anecdotally, these emails have been positively received. Over a dozen recipients of the “Grants to recommended charities” emails have contacted us (unprompted) to let us know they appreciated the information. We do not yet feel confident in extrapolating the impact of these emails on donor retention, as most donors give on an annual basis. We plan to continue sending these emails each quarter when we decide where to grant discretionary funds and to assess over the long term whether they impact donor retention.

Challenges and potential downsides

A major challenge we face with this project is striking the right balance between communicating clearly, creating a connection to our work, and honoring our values. We anticipate that:

1. the use of images could fail to treat beneficiary populations with the respect they deserve (for a discussion of some simplistic narratives about the relationship between donors and beneficiaries, see this blog post). We plan to be particularly careful about our selection of images and avoid depictions that do not respect the dignity of our beneficiaries.
2. the use of images might make our marketing harder to distinguish from typical charity outreach.
3. the use of cost-effectiveness figures may make it harder to distinguish GiveWell’s carefulness (the hundreds of hours our researchers collectively spend per year on cost-effectiveness analysis) from charities’ often unjustified claims about cost per impact.
4. the use of cost-effectiveness figures may cause donors to take these estimates literally rather than as a rough sense of the magnitude of expected impact of donations to our recommended charities. To mitigate this and item (3) above, we plan to make links to the detailed analysis behind our cost-effectiveness figures readily available.

The upside of moving more money to top charities and increasing donors’ engagement with our work seems worth tackling this challenge and its potential downsides.

How you can help us

We’re planning to move relatively slowly in this direction and adjust our actions based on the feedback we receive. If you have feedback about how our new communications are changing your view of the GiveWell brand (positively or negatively), please let us know by emailing info@givewell.org.

We’re excited to be on this new path and hopeful it will lead to more funding for our top charities.