A case study for animal-focused local EA movement building: Effective Animal Altruism London

By Jamie_Harris @ 2019-01-23T22:09 (+15)

Disclaimer: I am an employee of Sentience Institute but this post represents entirely my own views, rather than those of my employers or those providing feedback on the post. My actions referred to in this post were either taken before I began my employment at SI or in my spare time since then. Thank you to Hannah Masson-Smyth, David Nash, and Holly Morgan for their comments on an earlier draft of this post, as well as for their work supporting EAAL.

Summary:

Effective Animal Altruism London (EAAL) is one part of is one part of the Effective Altruism (EA) community in London. EAAL works mostly independently from the movement building charity Effective Altruism London.

The goals of this post of this post are:

1. To help those interested in EA movement building to decide how best to support the growth of the effective animal advocacy community, including through the more detailed supplementary material.
2. To provide another perspective into EA movement building more widely, especially regarding metrics other than IASPCs.
3. To elicit feedback on how EAAL could improve, or how we should prioritise our time, including considering the possibility that EAAL isn’t worth spending time on at all.

After modifying for counterfactuals, survey results suggest that EAAL has increased the knowledge of the most engaged members of the EAAL community by over 50%, increased their inclination towards EA, caused at least one significant career plan change, and led to some significant changes in the charities that individuals support. It does not seem to have caused much change in cause prioritisation.

At the end, I include an extremely rough cost-effectiveness estimate that suggests that the resources spent on EAAL are likely to have fallen short of the cost-effectiveness of 80,000 Hours, although possibly by less than an order of magnitude. This difference may be insignificant in the light of cause prioritisation considerations or unmeasured long-term implications.

Objectives of Effective Animal Altruism London

Broadly, the goal of EAAL has been to encourage engagement with effective animal advocacy (EAA), in order to generate impact for animals, i.e. a reduction of animals’ suffering or enhancement of their wellbeing. At times, we have used opportunities to encourage engagement with EA more widely, to generate impact for humanity, such as through broad introductions to EA at the start of an event, although this has never been the explicit main goal of an EAAL event.

Supplementary information

Information on the international, local, and personal context and background to Effective Animal Altruism London can be found here.

Detailed information about how my views have changed on the importance and usefulness of various audience and activity types for Effective Animal Altruism London can be found here. In summary, I’ve shifted away from focusing on a broad audience to supporting those with high inclination towards EAA already and supporting employees of animal charities.

Some considerations for and against forming a specific EAA subgroup of a local EA group can be found here. I conclude that this is only likely to be worth doing in some quite specific conditions.

Although I haven’t necessarily listed all of the sources for the ideas included in these files, the development in my views has been shaped mostly by:

• The experience of running the group and reflection on what we have/haven’t achieved.
• Conversations with others engaged in movement building, especially David Nash.
• Resources put out by CEA, especially the models listed here.
• Conversations and resources specific to EAA, especially from Sentience Institute and Animal Charity Evaluators.
• Various other EA resources, such as those posted on the EA Forum.

Impact

The evidence of impact for several of the metrics used here comes from a survey conducted earlier in January 2019 with some of the more engaged members of the EAAL community. Several questions asked about how respondents’ views, actions, and careers had changed since January 2017, when I started organising EAAL. This survey was sent directly to those who I thought EAAL might have affected significantly and who I thought were unlikely to feel irritated about filling in a 5 to 10 minute survey. It therefore does not represent the average attendee at an EAAL event or the entirety of EAAL’s impact. 28 people responded.

After each measure of potential impact I also asked a variation of this question: “If you answered ‘Yes’ to the above question, what proportion of this change/these changes do you suspect is/are directly or indirectly due to the events, community, and/or support that you have been introduced to via Effective Animal Altruism London? (i.e. what proportion of this change would not have happened if EAAL did not exist at all).” I modified the results for each measure of impact by the given proportion.

Despite the inclusion of these questions, there are several factors that may mean that the 'true' impact of EAAL on reducing animal suffering through these individuals is lower than the survey data implies:

• Without the resources put into EAAL, some of the same impact might have been achieved by Effective Altruism London more widely without additional resources, e.g. if some of the survey respondents had attended wider EA events. In subsequent calculations, I’ll use a discount rate of 15% to account for this. [1]
• The participants may have given unrealistically high answers in order to make the organisers of EAAL feel pleased with our efforts. Note that given the phrasing of the survey, the message that I sent to participants, and the depth of engagement of participants themselves, I wouldn’t expect this effect to be large. [2] In subsequent calculations, I’ll use a discount rate of 20% to account for this.
• Social desirability bias in answering the questions. In subsequent calculations, I’ll use a discount rate of 5% to account for this. [3]
• I’m generally unsure about how to account for the problem of double counting impact between EAAL and the individuals themselves.

For some of the questions, I transformed qualitative responses into numerical scales. These results were then multiplied by the proportion of the change that they attribute to EAAL. In some cases, this number was multiplied again by a figure to represent the likely significance of the change, where those who donate at least 10% of their income or currently do in depth volunteering for animal advocacy organisations were coded as +1. [4]

The original questions can be seen here. The anonymised results and numbers used in the transformations and calculations can be seen here. This sheet contains the precise results of the calculations, whereas the numbers below have been rounded.

Survey data on realisation of actions to generate impact for animals:

These are the most direct measures of EAAL having generated impact so far. I do not believe that they are the only metrics of importance, however.

• EAAL: I asked the survey respondents if their career plans had changed since January 2017 and what proportion of this they would attribute to EAAL. The results suggest that EAAL was responsible for 1.5 impact-adjusted significant plan changes. [5]
• EAAL: I asked the survey respondents whether the charities that they donate to, volunteer for or otherwise substantially support had changed since January 2017 and what proportion of this they would attribute to EAAL. The results suggest that EAAL was responsible for 4.5 “impact-adjusted significant charity changes." [6]
• Personally (although I haven’t checked my employers’ thoughts on this), I suspect that I would not have been able to secure my current role were it not for the opportunities and incentive for deeper engagement in EAA that were provided to me by organising for EAAL. I suspect that my career in EAA would have been delayed by at least a year were it not for Effective Altruism London and the opportunity to organise EAAL events.

Survey data on increased awareness and knowledge:

This metric matters because it may lead to realisation of actions to generate significantly greater impact for animals at later time points. This seems more likely to be the case for deeper and more specific knowledge, especially among those already deeply engaged with animal advocacy (such as employees of charities) or among individuals who seem dedicated to EAA, than it does for wider public awareness.

• EAAL: I asked the survey respondents how far their knowledge and understanding of EA and EAA had increased or deepened since January 2017 and what proportion of this they would attribute to EAAL. The results suggest that EAAL was responsible for an average increase in knowledge and understanding of 70%.
• EAAL: I asked the survey respondents if they had changed their views about the prioritisation of broad cause areas, sub-cause areas within animal advocacy (e.g. farmed or wild animals), or specific interventions within animal advocacy, and what proportion of this they would attribute to EAAL. The results suggest that EAAL was responsible for less than 1 “impact-adjusted significant prioritisation change." [7]

Survey data on increased inclination and support for EAA:

This metric matters because it may lead to realisation of actions to generate significantly greater impact for animals at later time points among the deeply engaged. It is also important among those who are not deeply engaged, as it may have implications for the long-term impact of EA as a wider movement.

• EAAL: I asked the respondents how far their inclination towards and support for EA and EAA had changed since January 2017 and what proportion of this they would attribute to EAAL. The results suggest that EAAL was responsible for an average increase by more than 0.5 on a -2 to 2 scale.
• EAAL: I also asked the survey respondents if they thought that their own inclination hadn’t changed or had decreased but that EAAL had helped to prevent a greater decrease in inclination. Only 8 responded to this question and the results suggested a negligible effect.

Attendance:

Attendance is not inherently valuable or positive. It is therefore only a useful metric insofar as it sometimes serves as the best proxy for the metrics that matter more. Going forwards, I will only use attendance as a measure of success for events intending to promote broad awareness of EAA (or EAAL more specifically). Note that the attendance figures for EAAL include some estimates for a few missing events that are not included in the Effective Altruism London figures; this may give the impression that EAAL attendance has been a slightly larger proportion of wider Effective Altruism London attendance than it actually has. [8]

• EAAL: 320 estimated unique attendees attending once or more in 2018.
• Effective Altruism London more widely: 604 unique attendees attending once or more in 2018.
• EAAL: 506 estimated attendances in 2018 (average 1.5 attendances per tracked attendee).
• Effective Altruism London more widely: 1430 attendances in 2018 (average 2.4).

Repeat attendance:

Repeat attendance seems like a better proxy for engagement with EA / EAA than raw attendance.

• EAAL: 80 estimated separate attendees attending twice or more in 2018. [8]
• Effective Altruism London more widely: 114 separate attendees tracked as attending twice or more in 2018.

Resource costs

The direct financial cost of EAAL over the past two years has been low. I am usually able to use free venues, and suspect that I have spent less than £250 on venue hire in total (where I have paid for venues, I have redirected money from donations I would otherwise have made to EAA charities).

Of course, I have spent money on travel to and from events, food, drink, and snacks for attendees or for myself. At each event, however, this tends to be under £20, and generally comes from personal money that I would otherwise likely spend on myself (e.g. on going to a restaurant or pub with non-EA friends), so the counterfactual cost is low. I would guess that the spending of attendees at the event is used similarly, although this may not always be the case; a friend told me that he used to find Effective Altruism London socials disappointing because he had conceptualised them as a way to have impact, rather than as an enjoyable event, and found it frustrating when they did not obviously appear to lead to impact. In this sense, many attendees may budget their spending (or be subconsciously affected by their spending) differently, and their attendance at EAAL events may detract from time and money that could have been directed towards direct work for animals.

In terms of organisational time, I am fairly confident that organising EAAL was one of the most impactful activities that I could have done for animals, since I previously lacked knowledge of EAAL, or opportunities for direct impact. I am no longer confident that this is the case going forwards. Although it is rare that my involvement in EAAL involves direct trade-offs with research output for my projects for Sentience Institute, if I spent less time on EAAL, I would likely spend more time on more applicable skill-building (such as improving my understanding of statistics) or on small independent contributions to EAA research.

Of course, when events involve external speakers or organisers, there will be time and resource costs for them too.

I estimate I have spent the equivalent of £7800 on EAAL. [9] Hannah Masson-Smyth notes that she has spent the equivalent of £1200 on EAAL. [10] David Nash estimates that he has spent the equivalent of £1000 on EAAL. This brings a combined estimated total of £10,000 spent on EAAL over two years. Note that these estimates are quite uncertain and the counterfactuals are quite unclear. Note that some other individuals have made smaller contributions, but I have not included estimates for their costs here, [11] nor have I included potential costs for attendees or guest speakers.

Extremely rough cost-effectiveness estimates

Note that all of the cost-effectiveness estimates below are very uncertain, and there are likely long-term or indirect implications of our work that have not been factored in that could greatly alter the conclusions. [12] These estimates should be seen as facilitating extremely rough ballpark comparisons for relatively short-term outcomes, rather being precise measures of actual impact.

Given an estimated cost of £10,000 on EAAL, and an estimated 1.56 IASPCs caused through EAAL, the cost per IASPC is £6410.

By comparison, in December 2016, 80,000 Hours estimated that each plan change worked out as having cost £470. This suggests that the cost per IASPC for EAAL is over an order of magnitude larger than it was for 80,000 Hours at that point.

As noted above, I don’t think that IASPCs capture the total impact of EAAL. However, I would be surprised if the total impact of EAAL was more than 3 times as valuable as its impact through IASPCs. The survey is unlikely to have captured all of EAAL’s impact, but I would be surprised if the total impact of EAAL was more than 3 times as valuable as its impact recorded in the survey. It therefore seems that EAAL is likely to fall short of the cost-effectiveness of 80,000 Hours, although possibly by less than an order of magnitude. [13]

Of course, a direct comparison between EAAL and 80,000 Hours assumes that EA movement building generally and EAA movement building are equally valuable. I currently believe that EAA is similarly high in expected value to some of the most promising far future-oriented cause areas, [14] and orders of magnitude higher in expected value than some other cause areas supported more directly by EA movement building, such as work on global health. The calculations here therefore suggest to me that work on EAAL has been similarly cost-effective to work by 80,000 Hours, although readers’ own conclusions may differ by several orders of magnitude, depending on their views on cause prioritisation.

Footnotes

[1] From my knowledge of the participants, my 90% subjective credibility interval is that the impact of EAAL should be discounted by 5% to 45%.

[2] From my knowledge of the participants, my 90% subjective credibility interval is that the impact of EAAL should be discounted by 10% to 70%.

[3] From my knowledge of the participants, my 90% subjective credibility interval is that the impact of EAAL should be discounted by 3% to 15%.

[4] This coding was based on my knowledge of the individuals. Donating 1% was coded as 0.1, donating 10% as 1. Currently working for an animal charity was coded as 2, likely to work for an animal charity in the future or currently doing skilled/in depth volunteering as 1, currently doing some volunteering but where I was unsure about their plans or potential was coded as 0.1. If there was no relevant information about volunteering, donating, or career plans, I coded this as 0.

[5] I accidentally made the survey so that respondents could only tick 1 of these options from a list which included options related to other issues, so the results may slightly undervalue the number of changes of this type.

[6] I made this metric up. Volunteering or donation changes were rated on a scale from +2 (all changed) to 0, then were adjusted for significance and for the proportion that EAAL was responsible for.

[7] I made this metric up. Changed cause prioritisation was rated as 10, changed sub-cause prioritisation as 1, and changed intervention type prioritisation as 0.1. These results were multiplied by the same adjustment for significance as used for donations (see footnote 4) and the proportion attributed to EAAL.

[8] Note that we are missing exact attendances for several events: 1) pre-EAG talks, approximately 50 attendees, of whom I’d guess 40 were unique, 2) Q&A with Kristopher Gasteratos of Cellular Agriculture Society, approximately 20 attendees, of whom I’d guess none were unique, 3) approximately 5 socials, with an estimated average attendance of 12, of whom I’d guess 6 were unique. For each of these, I have halved the number of non-unique attendees to add to the “repeat attendance” figures. The estimated attendance figures include both these estimates for events missing data, and tracked attendances at other events.

[9] Although I have never tracked the time spent on EAAL activities, I would estimate that over the past two years, my mean time input per week excluding the time spent directly at events (i.e. on organisational work, reading specifically for events, and admin) has been approximately 5 hours (90% SCI 1 to 9 hours). If I include the time spent at, or travelling to events, I would expect that this would rise to approximately 6 hours (90% SCI 2 hours to 13 hours). These confidence intervals are so wide partially due to uncertainty about what to count as specifically being intended for EAAL, as opposed to wider engagement in EAA. Across two years, my estimates would mean 520 excluding travel or 624 hours including travel dedicated to EAAL. My current salary works out as about £15 per hour, assuming I work a 40 hour week (I work more than this in practice, or slightly less if I don’t count breaks during the working day). Counting one hour of my time as worth £15, excluding travel time or money spent on travel and snacks, I therefore estimate £7800.

[10] This includes the time spent at the socials (or at least the 'official' time span of the social) as well as reading materials and preparing any materials for the themed socials and admin; sending follow up info follow up conversations with attendees. Hannah notes that the socials are an enjoyable experience for her and she would choose to attend at least 50% of them if she was not an organiser. Excluding the time spent at socials, the estimate would be considerably less, around £450. This does not include the time spent organising The Humane League’s action parties.

[11] Saulius Šimčikas spent some time in setting up EAAL in 2016, before I became involved in January 2017. Andrew Leeke and Danielle organised one event each in 2018.

[12] Hilary Greaves has explained that for repeated actions, “there are some highly-structured, systematic reasons for thinking there might be a general tendency of my action to make things better, but there might also for some other reasons be a general tendency to make things worse… when you take the long-term perspective and you take seriously the thought that what you’re ultimately interested in is all of the effects that your actions will have and not just the ones that are nearer in time or easier to measure, actually the total effect of your intervention will be massively dominated by the unforeseen ones… I think really your properly considered EA donation behaviour should be almost entirely driven by what your best guess is about that stuff that we haven’t measured. If that’s so, then there doesn’t really seem to be any place in the picture for the impact evaluations that we have.”

[13] If we estimated that the total impact of EAAL was twice its impact through IASPCs and that the total impact of EAAL was twice as much as the survey suggests, this would suggest £1603 per IASPC.

[14] For an explanation of why, see this post by Jacy Reese